A horse race among models of strategic thinking across similar games

Human behavior generally deviates from equilibrium in one-shot games. For this reason, a number of strategic thinking models which relax one or more assumptions of equilibrium have emerged. A natural extension to the emergence of these models is to compare their predictive and explanatory powers. In this study we have made a full-fledged comparison of eight prominent models (QRE, Lk, CH, NI, SLk, SCH, GCH and Lm) through a new game and its variations. We have analysed their performances in two ways. First, we out-of-sample predicted experimental results by these models and compared them by calculating the mean of squared distances between predictions and the observed data. Secondly we estimated the models for all games together and compared their log-likelihood values to determine their performance in explaining subjects' behaviors. We found that models with payoff dependent noise had consistently better predictive performances than those without noisy behavior. Our main contribution is to show that a little modification on game structure might lead to drastically different results in the predictive performances and statistical fits of the models. Even across very similar games, there were significant changes on the performances of the models.