We present the design and methodology for the large scale hybrid paper
recommender system used by Microsoft Academic. The system provides
recommendations for approximately 160 million English research papers and
patents. Our approach handles incomplete citation information while also
alleviating the cold-start problem that often affects other recommender
systems. We use the Microsoft Academic Graph (MAG), titles, and available
abstracts of research papers to build a recommendation list for all documents,
thereby combining co-citation and content based approaches. Tuning system
parameters also allows for blending and prioritization of each approach which,
in turn, allows us to balance paper novelty versus authority in recommendation
results. We evaluate the generated recommendations via a user study of 40
participants, with over 2400 recommendation pairs graded and discuss the
quality of the results using P@10 and nDCG scores. We see that there is a
strong correlation between participant scores and the similarity rankings
produced by our system but that additional focus needs to be put towards
improving recommender precision, particularly for content based
recommendations. The results of the user survey and associated analysis scripts
are made available via GitHub and the recommendations produced by our system
are available as part of the MAG on Azure to facilitate further research and
light up novel research paper recommendation applications.

介绍了微软学术使用的大规模混合论文推荐系统的设计和方法论，并通过用户研究评估了生成的推荐结果的质量，发现需要进一步改进基于内容的推荐精度。