Technically, database marketing often relies upon classification techniques originating from

statistics, data mining or machine learning. Classification algorithms formalize the relationship

between a categorical outcome and a set of predictive features into a model that can be used for

predictive purposes. The classification performance of predictive models for database marketing is a

primary determinant of the quality of customer targeting, and hence, the return on investment of the

database marketing project. This observation has led to a broad body of literature devoted to the

evaluation and comparison of alternative classification techniques in the context of database marketing.

In this dissertation, ensemble classifiers, constituting a specific category of classification algorithms is evaluated in the context of database marketing. Ensemble learning has received a lot of interest in recent theoretical and applied literature. Ensemble classifiers, or multiple classifier systems (MCS) are a combination of many constituent member classification models, whereby aggregate ensemble predictions are taken as combinations of the ensemble member outputs. The main factor defining the popularity of ensemble algorithms is the strong classification performance that has been observed throughout multiple comparative studies in various domains and applications. An ensemble improves classification performance over the individual classifier if accuracy and diversity are present among the ensemble members, i.e., the constituent models need to have a certain level of expertise, and a certain level of disagreement has to emerge among the ensemble members. Many different ensemble classifier algorithms have been proposed with varying strategies to maintain member accuracy and diversity. At the data level, the input training data set, containing class labels and explanatory features for a number of instances, is translated into training data sets for the ensemble members. Usually this

involves some form of transformation, such as sampling in Bagging, feature extraction in Rotation

Forests or feature selection in the Random Subspace Method. At the classifier level, a decision is made regarding the type or types of base classifiers to be included in the ensemble. Finally,

at the combination level, a combination rule is chosen to combine the member outputs into an

aggregate ensemble prediction. In this dissertation, all studies adopt and evaluate ensemble classifiers in an aim to increase predictive performance.

Chapter II of this dissertation presents a methodology to predict demographical characteristics of anonymous website visitors based on multi-website clickstream data. This study contributes to the literature of web advertising targeting and is relevant for customer acquisition, as discussed earlier. The growth of the Internet and the interactive nature of the medium have led to a substantial increase in web advertising expenditures. As a result, the effectiveness of online advertising, often measured using click-through rates (CTR), has declined dramatically. This trend has initiated a search for strategies to increase the effectiveness of web advertising, among both practitioners and academia. A broad range of tactics has been aimed at targeting, or increasing the correspondence between the advertisement and the receiver.

This targeting is mostly either behavioral, i.e. when web advertising is adapted to web visiting

behavior as a proxy for the visitor’s interests, or demographical, when the advertisement is offered to

demographically pre-defined segments. Demographical targeting is typically difficult in an online

environment, as demographical characteristics of website visitors are not immediately observable or

retrievable. In chapter II, a practical methodology to infer demographical characteristics from website

visitors that can be used for demographical web advertising targeting is presented and evaluated.

Chapter III is situated in customer retention. In particular, rotation-based ensembles, a recently

emerging class of ensemble classifiers is applied and evaluated for the prediction of customer churn.

Two algorithms, Rotation Forest and RotBoost, are compared and benchmarked to a set of well-known

classifiers that are either ensembles or uncombined techniques. Moreover, both rotation-based

ensemble algorithms are modified by incorporating alternative rotation, or feature extraction

components. All classifiers are compared over a set of four data sets from real-life churn prediction

applications.

Chapter IV presents the search for a new, strongly performing classifier algorithm by investigating

the potential of generalized additive models (GAMs) as base classifiers in ensemble learners. This

study does not focus upon a specific application in database marketing, but instead proposes and

validates classifiers that are applicable for the modeling of any binary outcome. In this sense, it

contributes to the three pillars of database marketing, in which predictive models for binary outcomes

are abundant (e.g. churn prediction, response modeling, up-sell modeling etc.).

Three alternative versions of GAM-based ensemble classifiers are considered: GAMbag based on

Bagging, GAMrsm based on the Random Subspace Method (RSM), and GAMens based upon both.

The study compares the three proposed alternatives against each other and against a set of benchmark

algorithms. It also includes sensitivity analyses for the algorithms’ parameters.

Chapter V is dedicated to the proposition and evaluation of a classifier algorithm that meets the

two most important requirements of classification algorithms in customer churn prediction: strong

model performance and model interpretability. Based upon GAMens, GAMensPlus is proposed which

includes an interpretation phase consisting of two instruments of model interpretability: generalized

feature importance scores and bootstrap confidence intervals for smoothing splines. The algorithm is

evaluated in terms of several metrics of classification performance and its ability to deliver insights

into the importance and contribution of churn drivers is illustrated in a specific case study. More information on Koen De Bock’s published papers can be found at this location: www.crm.UGent.be

Then, it was time for a nice reception...