The effect of item review in computerized adaptive testing on measurement error and bias

While computerized adaptive testing (CAT) is a preferred form of test practice because of providing more accurate and reliable results with fewer items, it also has some limitations. At the beginning of these restrictions, item review is not allowed in CAT. Changing the answer of an item in the CAT may cause previously administered items to no longer be suitable for estimating the test taker's real ability. A possible solution to the item review problem is to define the most likely optimal initial value that not only takes into account the uncertainty of temporal ability estimates, but also enables response change. In this direction, it is aimed to determine how the standard error and bias of the ability estimations obtained when prior knowledge is defined in reviewable CAT varies according to the initial rule of the test, the final ability estimation method and the test length. Three way ANOVAs were performed to examine whether there was a difference between the estimations in terms of standard error and bias. Results suggest that the bias and standard error values increased in all conditions when response changes were allowed in the CAT. However, when prior information defined based on the correlation between the subtests in the test battery is used in both the starting rule and the final ability estimation method, results similar to a traditional CAT are obtained in reviewable CAT even if individuals use the Wainer test strategy. As a result, prior knowledge makes the CAT algorithm more useful and resistant to Wainer test strategy.