A hybrid prediction approach using multiple linear regression and decision tree

When you wake up one winter morning, you may wonder whether it will rain or will the weather be fine? In our life we fall into many choices that require prediction and anticipation of the answer before starting work. In this thesis, a hybrid method was used between decision tree (regression tree) and multiple linear regression based on the CART mechanism. It used three different datasets to test the approach. The first is the advertising data set, which was represented by using (TV, radio, and newspapers) (X) to show the relationship between these advertising methods with sales (Y) in terms of their impact on sales and purchasing power. This dataset is called as "Advertising". The second data set contains (Species of fish, length, height, width), which are the independent variables (X) and their impact on the weight of the fish, which represents the dependent variable. This dataset is called as "Fish". The third dataset is the effect of the car's specifications on its price, which was considered the dependent variable. The car specification was (car name, fuel type, aspiration, door number, car body, drivewheel, engine location, wheelbase, car length, car width, car height, curb weight, engine type, cylinder number, engine size, fuel system, bore ratio, stroke, compression ratio, horsepower, peak rpm, city mpg, and highway mpg). This dataset is called "Car". The datasets were divided into train and test 80% - 20%, respectively. Where the research steps that represent the study were implemented, by making accurate predictions with the help of linear regression and CART. First, we split datasets using CART. For each leaf, different sub-datasets are filtered and created. The splitting point in the dataset was found with nodes. Our hypothesis is to divide the dataset using CART to increase the accuracy of the estimates. It applied multiple linear regression to filtered datasets. Then, it is compared multiple linear regression estimations using whole data and splitting dataset. The classification and regression tree (CART) algorithm represents a dataset's connection between the dependent variable and independent factors. It consists of a sequential binary dataset partition based on the variable values. Fitting tree models involves repeatedly splitting the data into homogenous groups. The output is a hierarchical tree of relevant decision rules for classification or prediction. Splitting is a procedure that divides the tree from its nodes into two or more nodes. The root node represents the entire sample or population and is divided into two or more groups as homogeneous groups. The nodes that sub-nodes are separated into are called parent and child nodes. Nodes that cannot be divided and have reached the minimum division are called leaf nodes. Pruning is the opposite of splitting, removing child nodes from the root node. In this study, results were compared to predict the value of the dependent variable (Y) using the regression tree method, multiple linear regression, and the particular research method of splitting the regression tree and constructing multiple linear regression models from it in order to select the best method that gives the best prediction based on the R2, MSE, and MAPE values. It was found in this study that splitting the data using multiple linear regression based on the regression tree gave a good result compared to using the multiple linear regression method alone or using the regression tree only. It was also found that the use of one error measure is not sufficient, but more than one error measure must be added to obtain an optimal model.However, it can add a classification and regression tree to divide the data set and find the best result from the hybrid tree and multiple linear regression model. The depth of the tree in an extensive real-life dataset will be increased to see the effect of height. Furthermore, we will delve into alternative approaches to linear regression in a distinct study. It could be scalable and effective in increasing tree size and powerful machine learning techniques. It was found in this study that splitting the data using multiple linear regression based on the regression tree gave a good result compared to using the multiple linear regression method alone or using the regression tree only. It was also found that the use of one error measure is not sufficient, but more than one error measure must be added to obtain an optimal model.