Deep learning techniques have become the method of choice for researchers
working on algorithmic aspects of recommender systems. With the strongly
increased interest in machine learning in general, it has, as a result, become
difficult to keep track of what represents the state-of-the-art at the moment,
e.g., for top-n recommendation tasks. At the same time, several recent
publications point out problems in today's research practice in applied machine
learning, e.g., in terms of the reproducibility of the results or the choice of
the baselines when proposing new models. In this work, we report the results of
a systematic analysis of algorithmic proposals for top-n recommendation tasks.
Specifically, we considered 18 algorithms that were presented at top-level
research conferences in the last years. Only 7 of them could be reproduced with
reasonable effort. For these methods, it however turned out that 6 of them can
often be outperformed with comparably simple heuristic methods, e.g., based on
nearest-neighbor or graph-based techniques. The remaining one clearly
outperformed the baselines but did not consistently outperform a well-tuned
non-neural linear ranking method. Overall, our work sheds light on a number of
potential problems in today's machine learning scholarship and calls for
improved scientific practices in this area. Source code of our experiments and
full results are available at:
this https URL

通过系统性分析 18 种不同的算法提案，深入研究热门的 “前 n 推荐任务” 中的最佳状态，发现在这些方法中只有一种效果明显优于对照组，而其余 6 种算法经常被可比较的简单启发式方法胜过，摆露了当下机器学习研究中的潜在问题，并呼吁在这一领域改善科学实践。