As AI systems enter into a growing number of societal domains, these systems
increasingly shape and are shaped by user preferences, opinions, and behaviors.
However, the design of AI systems rarely accounts for how AI and users shape
one another. In this position paper, we argue for the development of formal
interaction models which mathematically specify how AI and users shape one
another. Formal interaction models can be leveraged to (1) specify interactions
for implementation, (2) monitor interactions through empirical analysis, (3)
anticipate societal impacts via counterfactual analysis, and (4) control
societal impacts via interventions. The design space of formal interaction
models is vast, and model design requires careful consideration of factors such
as style, granularity, mathematical complexity, and measurability. Using
content recommender systems as a case study, we critically examine the nascent
literature of formal interaction models with respect to these use-cases and
design axes. More broadly, we call for the community to leverage formal
interaction models when designing, evaluating, or auditing any AI system which
interacts with users.

当 AI 系统进入越来越多的社会领域时，它们日益塑造并受到用户的偏好、观点和行为的影响。然而，AI 系统的设计很少考虑到 AI 和用户如何相互塑造。本论文提出了发展数学上明确规定 AI 和用户相互塑造的形式交互模型的理念，该模型可以用于制定交互实现、通过实证分析监控交互、通过反事实分析预测社会影响，以及通过干预控制社会影响。以内容推荐系统为案例研究，我们对形式交互模型的初始文献在这些用例和设计轴方面进行了批判性审视。此外，我们呼吁社区在设计、评估或审计任何与用户进行交互的 AI 系统时利用形式交互模型。

人工智能与用户相互塑造的会计学：数学模型的作用

Accounting for AI and Users Shaping One Another: The Role of  Mathematical Models

A common explanation for negative user impacts of content recommender systems
is misalignment between the platform's objective and user welfare. In this
work, we show that misalignment in the platform's objective is not the only
potential cause of unintended impacts on users: even when the platform's
objective is fully aligned with user welfare, the platform's learning algorithm
can induce negative downstream impacts on users. The source of these user
impacts is that different pieces of content may generate observable user
reactions (feedback information) at different rates; these feedback rates may
correlate with content properties, such as controversiality or demographic
similarity of the creator, that affect the user experience. Since differences
in feedback rates can impact how often the learning algorithm engages with
different content, the learning algorithm may inadvertently promote content
with certain such properties. Using the multi-armed bandit framework with
probabilistic feedback, we examine the relationship between feedback rates and
a learning algorithm's engagement with individual arms for different no-regret
algorithms. We prove that no-regret algorithms can exhibit a wide range of
dependencies: if the feedback rate of an arm increases, some no-regret
algorithms engage with the arm more, some no-regret algorithms engage with the
arm less, and other no-regret algorithms engage with the arm approximately the
same number of times. From a platform design perspective, our results highlight
the importance of looking beyond regret when measuring an algorithm's
performance, and assessing the nature of a learning algorithm's engagement with
different types of content as well as their resulting downstream impacts.

内容推荐系统的负面用户影响通常被解释为平台目标与用户福利不一致。然而，本文揭示了即使平台目标与用户福利完全一致，学习算法也可能引起用户的负面影响。用户反馈速率的差异可能影响学习算法与不同内容的互动，进而无意中推广具有特定属性的内容。我们的研究结果强调了除了后悔之外，对算法性能的评估还需要关注学习算法与不同类型内容的互动方式及其带来的下游影响。

在线平台中概率反馈能推动用户影响吗？

Can Probabilistic Feedback Drive User Impacts in Online Platforms?

For content recommender systems such as TikTok and YouTube, the platform's
decision algorithm shapes the incentives of content producers, including how
much effort the content producers invest in the quality of their content. Many
platforms employ online learning, which creates intertemporal incentives, since
content produced today affects recommendations of future content. In this
paper, we study the incentives arising from online learning, analyzing the
quality of content produced at a Nash equilibrium. We show that classical
online learning algorithms, such as Hedge and EXP3, unfortunately incentivize
producers to create low-quality content. In particular, the quality of content
is upper bounded in terms of the learning rate and approaches zero for typical
learning rate schedules. Motivated by this negative result, we design a
different learning algorithm -- based on punishing producers who create
low-quality content -- that correctly incentivizes producers to create
high-quality content. At a conceptual level, our work illustrates the
unintended impact that a platform's learning algorithm can have on content
quality and opens the door towards designing platform learning algorithms that
incentivize the creation of high-quality content.

本文研究了在线学习算法对内容创作者质量和创作动机的影响，发现 Hedge 和 EXP3 等算法会促进内容低质次，提出了一种基于惩罚低质次内容的算法以激励高质次的内容创作。