Abstract
Recommender systems arose in response to the excess of available online information. These systems assign, to a given individual, suggestions of items that may be relevant. These system's monitoring and evaluation are fundamental to the proper functioning of many business related services. It is the goal of this paper to create a tool capable of collecting, aggregating and supervising the results obtained from the recommendation systems' evaluation. To achieve this goal, a multi-granularity approach is developed and implemented in order to organize the different levels of the problem. This tool also aims to tackle the lack of mechanisms to enable visually assessment of the performance of a recommender systems' algorithm. A functional prototype of the application is presented, with the purpose of validating the solution's concept.

Abstract
The huge amount of online information deprives the user to keep up with his/hers interests and preferences, Recommender Systems appeared to solve this problem, by employing social behavioural paradigms in order to recommend potentially interesting items to users, Among the several kinds of Recommender Systems, one of the most mature and most used in real world applications are known as Collaborative Filtering. These methods recommend items based on the preferences of similar-users, using only a user-item rating matrix. In this pa™ per we explain a methodology to use Multi™Agent based simulation to study the evolution of the data rating matrix and its effect on the performance of several Collaborative Filtering algorithms. Our results show that the best performing methods are user-based and item-based Collaborative Filtering and that the average algorithm performance is surprisingly constant for different rating schemes.

Abstract
Recommender Systems have become increasingly popular, propelling the emergence of several algorithms. As the number of algorithms grows, the selection of the most suitable algorithm for a new task becomes more complex. The development of new Recommender Systems would benefit from tools to support the selection of the most suitable algorithm. Metalearning has been used for similar purposes in other tasks, such as classification and regression. It learns predictive models to map characteristics of a dataset with the predictive performance obtained by a set of algorithms. For such, different types of characteristics have been proposed: statistical and/or information-theoretical, model-based and landmarkers. Recent studies argue that landmarkers are successful in selecting algorithms for different tasks. We propose a set of landmarkers for a Metalearning approach to the selection of Collaborative Filtering algorithms. The performance is compared with a state of the art systematic metafeatures approach using statistical and/or information-theoretical metafeatures. The results show that the metalevel accuracy performance using landmarkers is not statistically significantly better than the metafeatures obtained with a more traditional approach. Furthermore, the baselevel results obtained with the algorithms recommended using landmarkers are worse than the ones obtained with the other metafeatures. In summary, our results show that, contrary to the results obtained in other tasks, these landmarkers are not necessarily the best metafeatures for algorithm selection in Collaborative Filtering.

Abstract
The large amount of Recommender System algorithms makes the selection of the most suitable algorithm for a new dataset a difficult task. Metalearning has been successfully used to deal with this problem. It works by mapping dataset characteristics with the predictive performance obtained by a set of algorithms. The models built on this data are capable of predicting the best algorithm for a new dataset. However, typical approaches try only to predict the best algorithm, overlooking the performance of others. This study focus on the use of Metalearning to select the best ranking of CF algorithms for a new recommendation dataset. The contribution lies in the formalization and experimental validation of using Label Ranking to select a ranked list of algorithms. The experimental procedure proves the superior performance of the proposed approach regarding both ranking accuracy and impact on the baselevel performance. Furthermore, it draws and compares the knowledge regarding metafeature importance for both classification and Label Ranking tasks in order to provide guidelines for the design of algorithms in the Recommender System community.

Abstract

Abstract
The algorithm selection problem refers to the ability to predict the best algorithms for a new problem. This task has been often addressed by Metalearning, which looks for a function able to map problem characteristics to the performance of a set of algorithms. In the context of Collaborative Filtering, a few studies have proposed and validated the merits of different types of problem characteristics for this problem (i.e. dataset-based approach): using systematic metafeatures and performance estimations obtained by subsampling landmarkers. More recently, the problem was tackled using Collaborative Filtering models in a novel framework named CF4CF. This framework leverages the performance estimations as ratings in order to select the best algorithms without using any data characteristics (i.e algorithm-based approach). Given the good results obtained independently using each approach, this paper starts with the hypothesis that the integration of both approaches in a unified algorithm selection framework can improve the predictive performance. Hence, this work introduces CF4CF-META, an hybrid framework which leverages both data and algorithm ratings within a modified Label Ranking model. Furthermore, it takes advantage of CF4CF’s internal mechanism to use samples of data at prediction time, which has proven to be effective. This work starts by explaining and formalizing state of the art Collaborative Filtering algorithm selection frameworks (Metalearning, CF4CF and CF4CF-META) and assess their performance via an empirical study. The results show CF4CF-META is able to consistently outperform all other frameworks with statistically significant differences in terms of meta-accuracy and requires fewer landmarkers to do so. © 2018, Springer Nature Switzerland AG.