A comparison of shrinkage methods in linear regression

Regression models based on data set play a very important role in statistical research and analysis. One of these models is multiple linear regression model. However, in multiple linear regression analysis, when multicolinearity problem arises, the least squares estimators are obtained with large variance although they are unbiased. This situation negatively affects the accuracy of statistical analysis. In statistical research, by using one of the shrinkage methods, the variance of the estimators can be reduced and the multicolinearity problem can be eliminated. In this thesis, colinearity problem is investigated and Ridge Regression, LASSO, Elastic Net and Liu shrinkage methods are examined to eliminate multicolinearity problem. As an application study, sum of squares errors, mean squares erros mean and determination coefficients of shrikage methods were compared over the data in which colinearity was determined. In these comparisons, all of the above shrinkage methods yielded better results than the least squares, while the LASSO yielded better results than both the least squares and all the shrinkage methods discussed for this data set. As a result, it is concluded that the shrinkage methods instead of least squares method, especially LASSO, give more accurate statistical results in case of multicolinearity problem.