Resampling based regression approach for what-if analysis

Regression and classification are two types of supervised learning tasks in machine learning. Though their target variable types are different their common objective is to find out the relationship between input and output variables. One of the most frequent problems that prevents regression and classification models from performing as expected is multicollinearity. If multicollinearity is present, it can be addressed by removing one or more of the correlated variables from the model or by combining them into a single composite variable to ensure getting consistent results from machine learning models. In practical applications of machine learning, particularly in funnel-based business models, it's often more crucial to monitor the unique impact of each variable on the target than to prioritize model performance. In this study we are trying to explore a solution to that type of problem in which we have to keep all dependent variables and explain their relationship with the target. While this method isn't a replacement for techniques like PCA or dimension reduction that handle multicollinearity, it mainly aids in understanding the relationship between dependent and independent variables in multicollinearity scenarios. Basically it will try to show what will happen on both target and other input variables while a certain change happens in any input variable. From that aspect it will also touch the field of "interpretable machine learning" of which the main objective is explaining the relationship between input and output variables of a machine learning model in a more transparent and interpretable way.