Examining the performance of p-value based adjustment procedures in nonparametric multiple comparisons

In statistics, multiple comparisons occurs when one considers a set of statistical inferences simultaneously. Multiple comparisons tests are used to determine which group or groups are different after global null hypothesis is rejected. Type I error is increased by performing simultaneous multiple comprasions. In order to avoid this, researchers developed procedures to control the inflation of Type I error. There are many procedures in the literature. In this study, p-value based multiple comparison adjustment procedures are examined. In this study, 8 p-value based multiple comparison adjustment procedures are compared under 5 different distiributions. In chapter 2, definitions of multiple comparisons, error rates, powers and Dunn's multiple comparison test using rank sums are given. In chapter 3, single step Bonferroni and Šidák, step down Holm and Holland-Copenhaver, step up Hochberg, Hommel, Rom and two step Li procedures are examined. In chapter 4, an example is given in order to understand p-value based procedures. In chapter 5, simulation is performed for 3, 4, 5 and 6 groups and different sample sizes for each pair. Family-wise error rate (FWER) and average power are calculated. As a result, Hommel and Li procedures are much more powerful than other p-value based procedures, but as seen in our results, Li procedure does not always preserve the FWER. Bonferroni and Šidák procedures seem to be too conservative as the number of comparisons increase.