Consider online learning algorithms that simultaneously make decisions and
learn from feedback. Such algorithms are widely deployed in recommendation
systems for products and digital content. This article exhibits a bias of
online learning algorithms towards less risky alternatives, and how it shapes
demand on recommendation systems. First, we consider $k$-armed bandits. We
prove that $\varepsilon$-Greedy chooses a riskless arm over a risky arm of
equal expected reward with probability arbitrarily close to one. This is a
consequence of undersampling of arms with bad reward estimates. Through
experiments, we show that other online learning algorithms exhibit risk
aversion as well. In a recommendation system environment we show that content
that yields less noisy reward from users is favored by the algorithm. Combined
with equilibrium forces driving strategic content creators towards content of
similar expected quality, the advantage for content that is not necessarily
better, just less volatile, is exaggerated.

研究在线学习算法中存在的风险规避现象，它如何影响推荐系统的需求和内容创作者的创作，以及如何通过平衡内容的稳定性和质量来解决这一问题。