Exploring black-box parameter calibration techniques on generic system dynamics models, using synthetic data

This thesis investigates the application of black-box optimization algorithms for parameter calibration in system dynamics models under various noise conditions and limitations. Two representative models, the SEIR epidemiological model and the stock management model, were selected to evaluate the performance of five optimization methods: Grid Search, Random Search, Tree-structured Parzen Estimator (TPE), Covariance Matrix Adaptation Evolution Strategy (CMA-ES), and Gaussian Process (GP) Sampler. Synthetic datasets incorporating pure and autocorrelated noise were generated to simulate real-world observational uncertainties. The findings indicate that Covariance Matrix Adaptation Evolution Strategy (CMA-ES) outperforms other algorithms in terms of accuracy and robustness in most scenarios. Tree-structured Parzen Estimator (TPE) achieved competitive results, whereas Grid Search and Random Search showed limited accuracy and poor adaptability under changing noise conditions. While Gaussian Process (GP) Sampler produced consistent results when calibrating interdependent parameters, its high computational costs made it less efficient compared to other algorithms. Additionally, the extreme condition tests conducted with the stock management model underscored the importance of search space exploration. Algorithms with strong exploration capabilities were able to identify oscillatory behaviors that would otherwise have been overlooked. This study emphasizes the potential benefits of integrating traditional error-based metrics with dynamic behavior analysis to improve calibration processes. Future research should focus on examining hybrid optimization approaches, evaluating more complex models, and incorporating real-world data.