Comparison of success of clinical prediction models based on beta coefficient, odds ratio obtained by logistic regression analysis and weighted scores obtained by machine learning algorithms

Clinical prediction models are widely used to predict the risk of future events or diseases for an individual by utilizing predictive factors such as age, gender, and biomarkers when there are multiple predictive variables. In the literature, the creation of clinical prediction models often involves the use of β coefficients, odds ratios, and their integer-rounded scores derived from logistic regression. Some studies also employ machine learning algorithms to generate weighted scores. However, there is no consensus on which scoring system is more effective in clinical prediction models. This thesis aims to evaluate the performance of prediction models created using logistic regression-derived β coefficients, odds ratios, scores obtained by rounding these values, and scores weighted using machine learning algorithms (random forests, support vector machines, etc.) under different scenarios. The goal is to identify the best prediction model. In this study, logistic regression analysis was conducted using simulated and real datasets to determine the score values of risk factors in clinical prediction models with binary outcome variables. Simulation methods were employed for different scenarios, leading to conclusions about which clinical prediction model was more successful in each scenario. A comprehensive dataset from 522 children in the field of pediatrics was used for the logistic regression analysis, and a clinical prediction model predicting malnutrition status was developed. Various performance metrics, including sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), accuracy, F1 score, MCC and AUC values, were calculated to compare model performances. Our research primarily compared logistic regression models for diagnosing clinical prediction models with binary outcomes under different scenarios rather than directly classifying with machine learning algorithms. Additionally, we obtained weights or scores for risk factors in logistic regression as an alternative to logistic regression, using different machine learning algorithms, and compared all model results. When evaluating AUC, accuracy, F1 score and MCC for clinical prediction models created with the simulated dataset, the most successful models were consistently Bayesian GLM, L1 (lasso) based on scores obtained from logistic regression-derived beta, integer beta, odds, integer odds, and machine learning algorithms. The logistic regression analysis using beta coefficients achieved the highest success in predicting malnutrition status in the clinical prediction model. However, considering the usability feature of clinical prediction models, the second most successful model was the one developed based on integer odds ratios. As a novel approach, we propose in this study that machine learning algorithms can be used to determine scores for risk factors in clinical prediction models, instead of creating the entire model using machine learning algorithms. In conclusion, the effectiveness of clinical prediction models is associated not only with good performance but also with being applicable, understandable, accurate, and generalizable under real-world conditions.