The effect of weighting methods with reliability coefficient and sampling error variance on effect size, heterogeneity and publication bias in meta analysis

The aim of this study was to examine how heterogeneity statistics, publication bias determination methods were affected by the different number of studies taken as samples when different weighting of effect size methods were used in examining the effect of the 5E teaching method on academic achievement in science education with meta analysis. Methods of weighting the effect sizes discussed in the research; weighting by reverse of sampling error variance and reliability of measures in primary studies. Under these methods, it was also aimed to examine how the inclusion of gray literature and the analysis of fixed and random effects models affected the meta analysis results. The research was of the basic type of research. The heterogeneity methods examined were Q, T2, I2, H2 and R2. Publication bias methods were funnel diagram, Duval Tweedie's trim and fill method, classical fail-safe N method, Begg and Mazumdar rank correlation and Egger's regression test. The number of studies was created by duplication method and determined as k, 2k, 6k and 10k. The data of the research consisted of 112 studies and 149 effect sizes obtained from these studies. In obtaining these studies, 6712 studies were examined and the study group was formed by selecting from these studies according to the predetermined meta analysis inclusion criteria. Studies with the keywords "5E" + "science" + "success", "5E AND science AND success", "5E AND science AND achievement" and "5E AND science AND achievement AND Turkey", accessed by searching databases and archives such as Google Scholar, Dergipark, YÖK Thesis Center, Proquest, Science Direct, ERIC, Taylor & Francis, EBSCOhost, Web of Science. In addition, apart from these databases, all the studies published between 2005-2020 of 5 journals with high impact factors in the field of science were also examined. The obtained studies were coded into Microsoft Excel in the context of the descriptive variables determined and the statistics used to calculate the effect size. CMA and Microsoft Excel were used in the analysis of the coded data. As a result of the research, different meta analysis results were obtained when the inverse of the sampling error variance and weighting methods with reliability were used. When examined in the context of the number of studies, it was seen that the overall effect sizes were not affected by the number of studies, but the standard error decreased as the number of studies increased. When the heterogeneity determination methods were examined, it was found that while the Q for all weighting methods was affected by the number of studies, as expected theoretically due to its structure, T2, I2, H2 and R2 were not affected by the number of studies. In addition, it was observed that the weighting method with the highest heterogeneity was weighting with sampling error variance. On the other hand, the lowest heterogeneity was observed in the weighting method with the reliability coefficient. When the results of publication bias methods were examined, it was determined that there was publication bias in weighting with reverse of sampling error variance, but there was no publication bias in weighting with reliability methods. Also, while the funnel diagram, Begg and Mazumdar's rank correlation and Egger's regression test were not affected by the number of studies, it was observed that the classical fail-safe N and Duval and Tweedie trim-fill methods were affected by the number of studies. In addition, it was determined that the inclusion of gray literature did not cause a significant difference in all weighting methods. Finally, when the results of the meta analysis with fixed effects and random effects model were examined, it was concluded that the variation in weighting with reliability coefficient was the least.