Theses supervised by Dr. Öğr. Üyesi Mustafa Gökçe Baydoğan
12 theses · Boğaziçi University
Personalized product recommendation on second hand platforms
With the advent of online marketplaces which millions of people worldwide visit and make purchase every second, the shopping experience and competition between these platforms have been significantly changed and recommendation systems have become a more critical part of these platforms and gained popularity in the literature. One of these online marketplaces, in which the recommendation system plays a key role, is second hand platforms. In addition to general recommendation problems, these platforms have several problems which are specific to this domain such as compromising extremely unique item sets that makes the problem difficult with respect to other domains. In this study, we propose two staged model pipelines using state-of-the-art NLP techniques word2vec and paragraph2vec to address these problems with high quality personalized product recommendation in a scalable architecture. The model performance is evaluated on both offline experiments which are conducted on historical user clickstream dataset that is gathered from a popular second hand platform and A/B test on a production system. As a consequence of these experiments, the proposed model outperforms the baseline collaborative filtering-based models with respect to selected metrics, in addition, provides significant uplifts on several business metrics in the product system.
Deep learning approaches for multi-site wind power forecasting in the West Aegean region of Turkey
Wind energy is one of the most economical and promising ways of producing renewable energy among today's technologies. But the uncertainties arising from the chaotic nature and the variability of weather events are major problems to the market stabilization and regular maintenance of the wind power systems. Wind farms are usually built in high wind speed potential areas. West Marmara and Aegean regions have the highest number of wind farms in Turkey. Forecasting the wind power production in a region is usually done separately for each wind farm, but forecasting in multi-site context contains more spatial information, thus enables learning from the neighbor wind plants. In this thesis, incorporation of multi-site spatial information, besides the temporal information, to deep learning models is studied. Six alternative deep learning methods, i.e. multilayer perceptrons, recurrent networks, graph neural networks, convolutional networks and their variants are implemented for this purpose. Each model is enhanced with numerical weather predictions to create more accurate long term forecasts, and model parameters are tuned with a hyperparameter optimization. Finally, these models are compared with tree-based boosting, penalized regression and persistence benchmarks over a one-year period. In order to investigate the positive effect of using multi-site approach, recurrent model is trained both separately for each plant and for all the plants at the same time in a multi-site context. Single plant based recurrent model performed better than the multi-site recurrent model, but methods using convolutional layers, significantly outperforms single recurrent, benchmarks and remaining deep learning models.
Example-dependent cost-sensitive gradient boosting machines for credit scoring
Although most of machine learning algorithms try to minimize cost-insensitive losses, many real world applications require cost-sensitive approaches where misclassification costs among classes differ from each other. In addition to misclassification costs, examples in data sets may have nonidentical costs which is a case of example-dependent cost-sensitive learning. For example in credit scoring, mistakenly rejecting a good borrower and approving a bad client with financial distress result in different costs. Additionally, providing variety of credit amounts to applicants makes the credit scoring example-dependent. In other words, falsely approving 100M$ and 1M$ loans produce unequal costs. To overcome this problem, this thesis proposes an example-dependent cost-sensitive loss function. With the introduced loss function, cost sensitivity is handled during the learning process. This is achieved by changing the traditional loss function of Gradient Boosting Machines with the proposed one to make it Example Dependent Cost-Sensitive Gradient Boosting Machines. The proposed algorithm is tested on two real world data sets that include credit amounts and synthetically generated data sets. The algorithm is compared with cost-insensitive learners, previously proposed example-dependent cost-sensitive classifiers that handles cost-sensitivity during learning, a post-processing method called Thresholding and a pre-processing method Oversampling to make cost-insensitive classifiers cost-sensitive. Results show that our method outperforms those four methods in terms of financial savings.
Pricing and order fulfillment for online marketplaces
The main purpose of this thesis is to provide a wholistic model for both pricing and order fulfillment problems of an online retailer. Real sales and shipment data of a company are used for both forecasting and fulfillment decisions. A tree-based ensemble model is offered for the demand forecasting process by considering pricing and promotion effects. The generated sales forecasts are added to the fulfillment model as future orders. These orders can be fulfilled by any FCs by considering the corresponding fulfillment costs. Therefore, the offered data driven model tries to optimize total profit of the company while minimizing these operational costs. These results are compared across different cases for price and capacity levels. Due to the randomness of the generated demand forecasts, a prescriptiveness coefficient is used to evaluate the reliability of the offered results. As a result of this study, an optimal inventory allocation, fulfillment and pricing strategy are provided to the company.
Distance-based learning approaches for multiple instance learning
Multiple Instance Learning (MIL) is a weakly supervised approach that focuses on the labeling of a set of instances (i.e. bags) where the label information of individual instances is generally unknown. Many of the earlier MIL studies focus on certain assumptions regarding the relationship between the bag and instance labels and devise supervised learning approaches. With the ambiguity in instance labels, these studies fail to generalize to the MIL problems with complex structures. To avoid these problems, researchers focus on embedding instance-level information to learn bag representations. In this context, dissimilarity-based representations are known to generalize well. This thesis proposes a novel framework in which each bag is represented by its dissimilarities to the prototypes. The framework consists learning mechanisms that provide fast and competitive results compared to the existing distance-based approaches on extensive benchmark data sets. The first approach is a simple model that provides a prototype generator from a given MIL data set. We aim to find out prototypes in the feature space to map the collection of instances (i.e. bags) to a distance feature space and simultaneously learn a linear classifier for MIL. The second proposal is a tree-based ensemble learning strategy that avoids complex tuning processes and heavy computational costs without sacrificing accuracy. The framework is enriched with the integration of the methods, parameter selection strategy, and ensemble design. Furthermore, the proposed methods are extended to the regression domain, namely Multiple Instance Regression (MIR). MIR is a less commonly studied area where the bag labels are real valued data instead of classes. The experiments show that the performances of all proposals are better than the state-of-the art approaches in the literature.
Probabilistic discriminative region descriptor for time series classification
Detecting discriminivative regions is a recent promising concept in many different domains for various dataset types such as image, text and time series. In time series domain, time series might be large and high dimensional because of the developing storage capacities. Although computational capacities are improved, storage and computation costs are increased. Therefore recent attempts are focused on the decreasing the computational and run time complexities. To decrease the complexity of the models, instead of using raw data, construction of the new feature representation by using the distinctive sub-sequences of the time series is the most common approach. Discriminative sub-sequences are called as shapelets in time series reflect the characteristics of the class of time series. Shapelets provide interpretable results and shapelet-based classifiers have superior accuracy on many time series datasets. Many researchers have proposed shapelet extraction methodologies for classification purpose. This study proposes a novel local feature extraction framework for time series and shapelet-based time series classification pipeline. Proposed framework provides model selection flexibility to describe the time-observation space to find local discriminative regions. After obtaining the discriminative regions, shapelets are extracted on the time-observation space by thresholding the class probability estimates to construct a new feature representation. New feature representation is calculated by the Euclidean distance between shapelets and time series. Finally, a classifier is trained by the new feature representation. Experimental results show that shapelet-based time series classification by using proposed Probabilistic Discriminative Region Descriptor (PDRD) provides competitive results on benchmark datasets.
A tree based categorical variable encoding strategy in supervised learning tasks
Categorical variables are present in most real-world datasets, often consisting of a high number of levels, referred to as high-cardinality categorical variables. Most machine learning algorithms do not have an innate mechanism to deal with categorical variables, hence, their encoding is necessary. Categorical variable encoding is the general term for the conversion of nominal independent variables to a numerical format. Many encoding strategies exist, and they are discussed in this thesis. This thesis presents a novel encoding strategy, categorical split encoding, and also provides an analysis of existing encoding methods. Categorical split encoding uses primary and surrogate split information as the vector representation for categorical variables, through a tree-based algorithm, this method outputs binary columns for each categorical variable making use of target information. Missing values are imputed by using surrogate information, while clustering similar values together based on the path they take through the decision tree algorithm. Various existing encoding strategies are benchmarked for comparison with the proposed strategy. The performance of categorical split encoding and other encoding methods is compared with three different machine learning algorithms (generalized linear models, random forest and xgboost) using datasets from regression, binary and multiclass classification settings. Datasets used are made publicly available for replication purposes. As a result, categorical split encoding provides competitive results compared to existing encoding strategies in various datasets.
A column generation approach to solve ranking problems
Many traditional classification approaches focus on the minimization of misclassification rate. However, this is not a suitable metric in case of imbalance in class distribution and unknown misclassification costs. In such cases, Area under Receiver Operating Characteristics Curve (AUC) is an effective metric, which also quantifies the ranking quality of a classifier. Although this metric can be optimized directly by employing some mixed integer programming models, it is challenging to solve these models due to large number of binary variables. Some alternative formulations such as margin maximizing approaches optimizing surrogate objectives are proposed to solve this problem approximately. These methods extend classical Support Vector Machine (SVM) formulation and aim at minimizing ranking error while penalizing the model coefficients with a cost parameter in the objective. In these approaches, the cost and kernel-related parameters (i.e., type, degree and etc.) must be determined by parameter tuning operations since the test performance is highly reliant on these parameters. Primary aim of this study is to avoid the repetitive experiments to tune the parameters of margin-maximization approaches. We propose a linear programming model and a column generation approach, namely Ranking-CG, to select relevant features in an iterative way to decrease the number of features in the model. Additionally, kernel selection is avoided using the Euclidean distances between points as features to learn the non-linear relations. Ranking-CG is modified slightly to obtain faster convergence by solving a non-linear subproblem at each iteration to find the vector (i.e. prototype) in the feature space that violates dual feasibility the most. Our experiments show that the modified approach, Ranking-CG Prototype, provides competitive results with significantly less number of features compared to margin-maximization approaches.
Multi-objective approaches for multi-target learning
Multi-target datasets (MTD) require simultaneous prediction of several variables hence they are considered to be more challenging in terms of predictive tasks compared to single-target datasets. Mining of MTD requires handling of several problems. To exemplify, scale inconsistencies are widely encountered in the targets. Most of the existing approaches resolve this issue by transforming the targets to the same scale, yet those operations may change the statistical properties of the dataset. Besides, features' scale inconsistencies cause problems in semi-supervised learning (SSL) applications since distance-based calculations are required therein. Another issue with MTD is, to explore alternative ways of including the target relations in learning applications. In this thesis, I develop supervised learning (SL), SSL and feature ranking (FR) models for MTD to deal with aforementioned problems. Benefiting from multi-objective optimization concepts, I aim to propose learning strategies that are robust to the type of the variables processed and utilize the target relations at the same time. Specifically, I propose a multi-objective extension for standard decision trees and a selective classifier chaining strategy for SL tasks. Experimental studies show that proposed models outperform their benchmark models. Besides, multi-objective trees extended to their semi-supervised version so that proposed form could result a competitive performance when the label information is not adequate. Performed experiments show a significant improvement of the proposed model over its benchmarks. In addition, since high-dimesionality and irrelevance in features reduce the effectiveness of a learning model, an embedded feature ranking (FR) procedure to semi-supervised trees is given to address this problem. Applications on several datasets show that, proposed FR procedure enhances the predictive performance compared to its benchmark approaches.
Spatiotemporal forecasting of solar power generation with deep learning
Solar power is one of the most rapidly growing carbon-free power generation solutions. It is considered as a key element in the fight against global climate crisis; however, rapid expansion in the distributed PV power, i.e. plants with less than 1 MW capacity, brings about some problems to the electricity markets. Spatially dispersed positioning of hundreds of plants cause significant variations in the power supply where trading operations depend on accurate forecasts of the future production. In this study, several deep learning techniques are implemented for the day-ahead solar power forecasting problem to predict the aggregated output of over a thousand solar stations distributed over a large area in the Central Anatolian Region. Four different architectures in the literature are adapted to the spatiotemporal numerical weather prediction (NWP) data, along with the proposed parallel locally-connected long short-term memory (PLC-LSTM) architecture. All models are put through a distributed heuristic hyperparameter tuning process using multiple graphical processing units (GPUs). Best-performing trials of each model are selected according to their validation results and compared with each other, together with persistence and an individual plant naive model as benchmarks. The results show that deep learning models work considerably well in spatiotemporal PV forecasting problem, compared to benchmarks. Also, it is seen that even simple architectures can perform close to models with a higher degree of complexity, when a good combination of parameters is obtained with a thorough search procedure. Although there is not a single dominating architecture prevailing in all kinds of performance metrics, PLC-LSTM shows promising results by finding a sweet spot of complexity between the shared-weight and fully-connected architectures, considering the bias and variance of the corresponding models.
Data-driven operators for a hybrid evolutionary algorithm to solve multi-trip rich vehicle routing problem
Vehicle routing problems have been the subject of intensive research due to their difficulty as a combinatorial optimization problem and their importance in real life operations. Heuristics are widely preferred to solve VRPs especially in large scale real-world applications. With the growth of data storage and development of ML tools, data mining has started to be used to enhance heuristics. However, there is not much work involving a general learning scheme within VRP heuristics because of the complicated nature of multi-attribute variants. Most common features of a VRP instance such as routing sequence, vehicle capacity, time windows, unit costs of travel contain valuable information about the quality of the solution. In this study, we propose to gather the aforementioned characteristics of every past solution during the search and build a dynamic predictive model on it. A novel removal operator based on the predictions of this model is integrated into the heuristic. Population-based heuristics are suitable for this task because of their inherent solution pool hence we chose a hybrid evolutionary algorithm designed for a Multi-Trip Rich VRP. We discuss the marginal effects of adding this simple and fast data-driven removal operator into the base method. Moreover, we define modifications of this new operator focusing on different features of the sample and try to interpret their performance. Our experiments on the benchmark instances have shown that this straightforward and adaptable framework generates promising results.
Forecasting with a large number of predictors: A tree based moving average approach
A large number of predictors brings valuable information to time series forecasting problems, as well as difficulty in capturing this information. Short-term electricity load forecasting (STLF) problems are examples of such problems with many predictors, including several temperature values from different regions, a large number of special days, and multiple time-related variables. In such cases, forecasting with many predictors can be problematic in terms of robustness to redundant predictors, tendency to overfit, and the curse of dimensionality. To resolve these problems, this study proposes a novel tree-based ensemble model, tree-based moving average (TBMA), that provides point and probabilistic forecasts and works as an automated feature extraction method. The proposed model deals with a large number of predictors without sacrificing accuracy and does not require a complicated parameter tuning process as the advantages of being a tree-based ensemble model. The distinctive feature of the proposed model from existing tree-based ensemble models is that the suggested model considers the autocorrelation in time series data with the integrated moving average model and extracts useful features. Our comprehensive experiments show that boosting approaches provide significantly better results when features from TBMA are introduced. The proposed approach also provides competitive results compared to benchmark models in point and probabilistic forecasting of Turkey's electricity load.