Permutation, Parametric, and Bootstrap Tests of Hypotheses

This text is intended to provide a strong theoretical background in testing hypotheses and decision theory for those who will be practicing in the real worldorwhowillbeparticipatinginthetrainingofreal-worldstatisticiansand biostatisticians. In previous editions of this text, my rhetoric was somewhat tentative. I was saying, in e?ect, “Gee guys, permutation methods provide a practical real-world alternative to asymptotic parametric approximations. Why not give them a try?” But today, the theory, the software, and the hardware have come together. Distribution-free permutation procedures are the primary method for testing hypotheses. Parametric procedures and the bootstrap are to be reserved for the few situations in which they may be applicable. Four factors have forced this change: 1. Desire by workers in applied ?elds to use the most powerful statistic for their applications. Such workers may not be aware of the fundamental lemma of Neyman and Pearson, but they know that the statistic they wanttouse—acomplexscoreoraratioofscores,doesnothaveanalready well-tabulated distribution. 2. Pressure from regulatory agencies for the use of methods that yield exact signi?cance levels, not approximations. 3. A growing recognition that most real-world data are drawn from mixtures of populations. 4. A growing recognition that missing data is inevitable, balanced designs the exception. Thus, it seems natural that the theory of testing hypothesis and the more general decision theory in which it is embedded should be introduced via the permutation tests. On the other hand, certain relatively robust param- ric tests such as Student’s t continue to play an essential role in statistical practice.