Parameter estimation with the penalized maximum likelihood method

Among the methods of obtaining estimators, the most preferred method by researchers is the maximum likelihood method. The most important feature of this method is that the obtained estimators are asymptotically unbiased and have small variance. A problem encountered by researchers using this method is that there are difficulties in solving the maximization problem in the stage of obtaining estimators. In this case, the likelihood function can be maximized under that constraint by adding constraints to the parameters so that the parameter estimates obtained have better properties. The constraint on the parameters in the new likelihood function is called a penalty. Various penalization methods have been used in this study. One of the fields of study where maximum likelihood method estimator is widely used is regression analysis. In this thesis, ordinary maximum likelihood method, Bridge, Ridge, LASSO and Elastic Net regression models, which are widely used in the literature, are examined and analyzed and comparisons are made as a result of the analysis. With the results obtained, the methods that give the closest results to the maximum likelihood were identified. These models can help decision makers in areas such as health and machine learning to make fast and effective decisions and put these decisions into action. Different perspectives on the penalized ML method are examined with the examples of Generalized Weibull Distribution and Two-Parameter Exponential Distribution and compared with the maximum likelihood method. These models may be preferred by researchers in order to obtain better predictors.