There are two basic errors in hypothesis testing:

  • type I error and
  • type II error.

Type I Error

A type I error occurs when a true null hypothesis is rejected (i.e., concluding there is an effect when none exists). The probability of committing a type I error is equal to the level of significance used in the study.

If the chosen level of significance is α=0.05, then the probability of committing a type I error is 5%. If the chosen level of significance is α=0.01, then there is a 1% chance of committing a type I error.

The probability of committing a type I error is directly controlled by the researcher.

Type II Error

A type II error occurs when we fail to reject a false null hypothesis, i.e. when a true alternative hypothesis is not supported.

The probability of committing a type II error is denoted by β.

The probability of rejecting a false null hypothesis is the power of the statistical test, which is calculated by subtracting β from 1:

Power=1- β

The Relationship Between the Two Types of Error

The probability of committing a type II error is inversely related to the probability of committing a type I error. As the probability of committing one error increases, the probability of committing the other decreases. This means that the probability of committing a type II error is greater at a level of significance of α=0.01 than at a level of significance of α=0.05.

The probability of committing a type II error (β) is mathematically complementary to the power of the test (1-β). Consequently, increasing the significance level (α) reduces β, which in turn increases the power of the test.

The relationship between type I and type II errors can also be summarized in a contingency table.

  H0 is true H0 is false
Fail to reject H0 Correct decision Type II error
Reject H0 Type I error Correct decision (power of the test)