What is the term for the probability of making a Type I error when performing multiple statistical comparisons while the null hypothesis is true in each case?

What is the term for the probability of making a Type I error when performing multiple statistical comparisons while the null hypothesis is true in each case?

• A. Experimentwise error rate
• B. Error SSCP (E)
• C. Eta squared (η2)
• D. F-statistic

Answer: (A)

This question is about the overall risk of falsely declaring a finding significant when you run several tests. When you perform multiple statistical comparisons and all null hypotheses are true, the chance that you’ll make at least one Type I error across all tests is the familywise error rate. In many contexts this is also called the experimentwise error rate. The option named Experimentwise error rate matches this concept exactly, describing the probability of at least one false positive across the set of comparisons. The other terms don’t capture this idea: an error SSCP is related to the error sum of squares in the model, not the likelihood of Type I errors across multiple tests; Eta squared is an effect size measure for the proportion of variance explained; and the F-statistic is a test statistic used in ANOVA, not the error rate across many tests.

This question is about the overall risk of falsely declaring a finding significant when you run several tests. When you perform multiple statistical comparisons and all null hypotheses are true, the chance that you’ll make at least one Type I error across all tests is the familywise error rate. In many contexts this is also called the experimentwise error rate. The option named Experimentwise error rate matches this concept exactly, describing the probability of at least one false positive across the set of comparisons.

The other terms don’t capture this idea: an error SSCP is related to the error sum of squares in the model, not the likelihood of Type I errors across multiple tests; Eta squared is an effect size measure for the proportion of variance explained; and the F-statistic is a test statistic used in ANOVA, not the error rate across many tests.