For example, Altman ([11],Figure 4.7, p. 60) showed that even sample sizes of 50 taken from a normal distribution may look non-normal. Second, some preliminary tests are accompanied by their own underlying assumptions, raising the question of whether these assumptions also need to be examined. In addition, even if the preliminary test indicates that the tested assumption does not hold, the actual test of interest may still be robust to violations of this assumption. Finally, preliminary tests are usually applied to the same data as the subsequent test, which may result in uncontrolled error rates. For the one-sample t test, Schucany and Ng [41] conducted a simulation study of the consequences of the two-stage selection procedure including a preliminary test for normality. Data were sampled from normal, uniform, exponential, and Cauchy populations. The authors estimated the Type I error rate of the onesample t test, given that the sample had passed the Shapiro-Wilk test for normality with a p value greater
than αpre. For exponentially distributed data, the conditional Type I error rate of the main test turned out to be
strikingly above the nominal significance level and even increased with sample size. For two-sample tests, Zimmerman[42-45] addressed the question of how the Type I error and power are modified if a researcher’s choice of test (i.e., t test for equal versus unequal variances) is based on sample statistics of variance homogeneity.
In the present study, we investigated the statistical properties of Student’s t test and Mann-Whitney’s U test
for comparing two independent groups with different selection procedures. Similar to Schucany and Ng [41], the
tests to be applied were chosen depending on the results of the preliminary Shapiro-Wilk tests for normality of
the two samples involved. We thereby obtained an estimate of the conditional Type I error rates for samples
that were classified as normal although the underlying populations were in fact non-normal, and vice-versa.
This probability reflects the error rate researchers may face with respect to the main hypothesis if they mistakenly
believe the normality assumption to be satisfied or violated. If, in addition, the power of the preliminary
Shapiro-Wilk test is taken into account, the potential impact of the entire two-stage procedure on the overall
Type I error rate and power can be directly estimated.