Sample size calculation approaches in reliability studies

A standard scale used in the fields of health must have two basic properties: reliability and validity. There are many methods to determine the reliability of scale and the Cronbach alfa reliability coefficient is one of the most widely used reliability measurement in the health sciences. The important point in the analysis is deciding the sample size needed for correctly identifying Cronbach alfa coefficient of the scale. The classical confidence interval method for the population value of Cronbach's alfa reliability coefficient has a restirictive assumption that the multiple measurements have equal variances and equal covariances (i.e. parallel measurement). In 2015, Bonett proposed the use of a confidence interval approach that does not require equal variances or/and equal covariances (nonparallel measures) for calculating sample size to achieve the desired precision and to reach the desired level of power in hypothesis tests. This study aims to investigate the changes in sample size created by these two sample size approaches in general, depending on the different Cronbach alfa coefficients and the number of items included in the scales. Simulation studies are performed for this purpose. Firstly, for the sample size depending on the desired precision based on the parallel measurements; different item counts, varying planned Cronbach alfa levels and confidence interval widths were used. In the case of non-parallel measures, the variance-covariance matrix is examined in addition to those mentioned. Later, the sample size depending on the desired power was evaluated by taking different power values under the H0 hypothesis. Simulation studies of parallel and non-parallel measurement situations were supported by ODI and SF-36 real datasets. Based on the proposed confidence interval and sample size approaches; findings indicate that sample size decreases as the number of items, Cronbach alpha value range and confidence interval width increase. Increasing the required power causes a increase in the sample size. As a result, in the sample size approach based on the desired precision, regardless of the model type (parallel measure or not), as the number of items in the scale increases, the sample size decreases to a certain level, but is fixed when the item number is 50. This fixation of sample size at 50 items was also observed in the hypothesis test method (desired power) for sample size determination.