Abstract

Abstract
Algorithm selection methods can be speeded-up substantially by incorporating multi-objective measures that give preference to algorithms that are both promising and fast to evaluate. In this paper, we introduce such a measure, A3R, and incorporate it into two algorithm selection techniques: average ranking and active testing. Average ranking combines algorithm rankings observed on prior datasets to identify the best algorithms for a new dataset. The aim of the second method is to iteratively select algorithms to be tested on the new dataset, learning from each new evaluation to intelligently select the next best candidate. We show how both methods can be upgraded to incorporate a multi-objective measure A3R that combines accuracy and runtime. It is necessary to establish the correct balance between accuracy and runtime, as otherwise time will be wasted by conducting less informative tests. The correct balance can be set by an appropriate parameter setting within function A3R that trades off accuracy and runtime. Our results demonstrate that the upgraded versions of Average Ranking and Active Testing lead to much better mean interval loss values than their accuracy-based counterparts.

Abstract
This article serves as an introduction to the Special Issue on Metalearning and Algorithm Selection. The introduction is divided into two parts. In the the first section, we give an overview of how the field of metalearning has evolved in the last 1-2 decades and mention how some of the papers in this special issue fit in. In the second section, we discuss the contents of this special issue. We divide the papers into thematic subgroups, provide information about each subgroup, as well as about the individual papers. Our main aim is to highlight how the papers selected for this special issue contribute to the field of metalearning.

Abstract
Selecting appropriate classification algorithms for a given dataset is crucial and useful in practice but is also full of challenges. In order to maximize performance, users of machine learning algorithms need methods that can help them identify the most relevant features in datasets, select algorithms and determine their appropriate hyperparameter settings. In this paper, a method of recommending classification algorithms is proposed. It is oriented towards the average ranking method, combining algorithm rankings observed on prior datasets to identify the best algorithms for a new dataset. Our method uses a special case of data mining workflow that combines algorithm selection preceded by a feature selection method (CFS).

Abstract

Abstract