1. Introduction

Hypothesis testing is an important part of inferential statistics. It uses sample information to determine whether the assumptions about the population parameters or the population distribution are credible. It includes parametric tests and non-parametric tests. The most commonly reported statistical test results are based on the significance test of the null hypothesis [1]. The core principle underlying hypothesis testing lies in the concept of small probability—that is, assessing whether an observed result is likely to have occurred by chance in order to evaluate the validity of an assumption [2].

Hypothesis testing is widely applied across various fields. For instance, in the aviation industry, it serves as a critical step in the development and performance validation of aviation equipment, where product indicators must be rigorously tested. In the medical field, after performing descriptive statistical analysis on variables, formal hypothesis testing is essential to determine whether observed differences in treatment effects or group comparisons are statistically significant or merely due to random variation [3].

This paper focuses on evaluating the limitations and differences in the applicability of various hypothesis testing methods. It aims to compare the strengths and weaknesses of different test types and propose potential improvements. The findings contribute to the theoretical development of statistical hypothesis testing and partially address existing gaps in the current research landscape.

2. Hypothesis testing

2.1. Basic concepts

Hypothesis testing, also known as statistical hypothesis testing, is a statistical inference method used to determine whether differences between samples or between a sample and a population are due to sampling error or inherent discrepancies. The basic steps include proposing the null hypothesis (H0) and alternative hypothesis (H1), selecting a significance level, calculating the test statistic, and making decisions based on the p-value or critical value. Common testing methods include the t-test, z-test, chi-square test, and F-test.

2.2. Theoretical foundation

Hypothesis testing is based on the principle of small probability, which evaluates the validity of a hypothesis by observing whether a low-probability event occurs. The process begins by proposing H0, defining H1 as the opposite of H0, and setting α (the threshold for small probability events). A statistical method is then applied to assess the likelihood of H0 being true. If this likelihood is sufficiently small, H0 is rejected; otherwise, it is retained [2].

2.3. Common methods and applicable conditions

t-test: Compares means between two groups; suitable for small samples with unknown population variance.

z-test: Used for large samples with known population variance.

Chi-square test: Tests independence or goodness-of-fit for categorical data.

F-test: Utilized in analysis of variance (ANOVA) to compare means across multiple groups.

2.4. Two types of errors in hypothesis testing

Hypothesis testing may lead to two errors:

Type I error (False Positive): Rejecting H0 when it is actually true, due to sampling randomness placing the sample in the rejection region. The probability of this error is denoted as α.

Type II error (False Negative): Failing to reject H0 when it is false, due to sampling randomness placing the sample in the acceptance region. The probability of this error is denoted as β.

Under fixed sampling conditions, reducing both errors simultaneously is impossible (decreasing one increases the other). The only way to minimize both is to increase the sample size, which highlights a key limitation of hypothesis testing [2].

2.5. Debates and challenges

Hypothesis testing remains a topic of debate despite deeper understanding of its principles. Key controversies include disputes over null hypothesis significance testing (NHST) and the misuse of p-values. A common misunderstanding lies in conflating the conditional probability p(data | H0) (probability of observing the data assuming H0 is true) with p(H0 | data)(probability of H0 being true given the data). Additionally, large sample sizes often yield extremely small p-values, leading to overinterpretation. To address these issues, researchers continually refine and improve hypothesis testing methodologies [4].

2.6. Optimization and alternative methods

Failure Rate Hypothesis Testing

Failure rate hypothesis testing, grounded in goodness-of-fit tests and parameter estimation, offers a robust alternative to traditional statistical hypothesis testing. It features simple parameters, mathematical tractability, broad applicability, and effective risk cost control.

Case Study: Ice Detector Failure Rate Evaluation

Using operational failure data from a similar aircraft model, we assessed the failure rate of ice detectors. Historical fault data for the aircraft’s ice detectors were cleaned and processed, revealing the device’s failure patterns (as illustrated in Table 1 below):

Table 1. The historical record of failures of the ice detector

Note: "⁃" indicates that the data for that month does not meet the criteria, and the component failure rate for that month is excluded from calculation.

For the hypothesis testing on the failure rate of the ice detector, the null and alternative hypotheses are:

1) Null hypothesis (H0): λ ≤ 38.051 × 10⁻⁶ / FH (i.e., the ice detector provided by the supplier meets the failure rate requirement).

2) Alternative (research) hypothesis (H1): λ > 38.051 × 10⁻⁶ / FH (i.e., the ice detector provided by the supplier fails to meet the failure rate requirement).

A significance level of α = 0.05 is selected, resulting in a rejection region for the test statistic z of z > 1.645.

At the significance level α = 0.05, the calculated test statistic z (−0.56) does not fall within the rejection region (z > 1.645). Therefore, we fail to reject H0 and maintain the status quo until compelling evidence emerges to refute H0, i.e., the ice detector’s failure rate meets its design requirements during the aircraft development phase [5].

However, when reviewing monthly sample values in Table 2, seven observations exceed the design threshold, while 24 fall below it. Despite this, raw data distribution alone is insufficient to support definitive conclusions. This exemplifies the classic statistical inference challenge—drawing population-level conclusions from sample data.

Assessing performance metrics (e.g., failure rates) based on operational, flight test, or experimental fault data represents a classic scenario of inferring population characteristics from samples. However, directly extrapolating sample statistics to the population inevitably introduces errors. In civil aircraft performance evaluation, the null hypothesis is typically set as compliance with requirements. For "smaller-is-better" metrics (e.g., failure rates), an upper-tailed test is preferred, while a lower-tailed test is used for "larger-is-better" metrics.

3. Practical applicability analysis

3.1. Hypothesis testing in sports strategy evaluation

Hypothesis testing can analyze tactical strategies in sports. For example, in a badminton match analysis:

Collect match footage of players A and B.

Define variables such as rally length and diagonal shot frequency.

Propose hypotheses:

H1: Player A uses fewer diagonal shots than the opponent but scores more points with diagonals.

H2: Player B excels in winning points through extended rallies.

Using hypothesis testing metrics, the analysis reveals that while Player A’s diagonal shots increase the opponent’s movement and fatigue, they also demand higher physical stamina from Player A. Tactical elements should therefore be analyzed within integrated models that support broader strategic frameworks rather than as isolated variables [6].

3.2. Hypothesis testing in medical research

Medical statistics, a critical interdisciplinary tool, employs hypothesis testing to:

Evaluate new drug efficacy.

Assess treatment effectiveness.

Analyze disease risk factors.

Validate diagnostic test accuracy.

For instance, in a study testing the hypothesis "Drug A significantly reduces blood pressure," a randomized controlled trial divides patients into two groups: one receiving Drug A and the other a placebo. Results show Drug A’s group has a significantly lower mean blood pressure (p < 0.05), confirming its effectiveness [3].

4. Conclusion

This study focuses on the applicability analysis and optimization of hypothesis testing methods, systematically evaluating and refining various approaches. The conclusions align with traditional hypothesis testing: rejecting H0 if the test statistic falls in the rejection region; otherwise, failing to reject H0. However, several limitations remain. In particular, the validity of conclusions in medical research is highly contingent on the accuracy of the null hypothesis. Mis-specification of H₀may lead to erroneous interpretations.To address these challenges, future statistical analyses should prioritize data accuracy and hypothesis specificity to systematically resolve related issues.