We consider a ubiquitous scenario in the Internet economy when individual
decision-makers (henceforth, agents) both produce and consume information as
they make strategic choices in an uncertain environment. This creates a
three-way tradeoff between exploration (trying out insufficiently explored
alternatives to help others in the future), exploitation (making optimal
decisions given the information discovered by other agents), and incentives of
the agents (who are myopically interested in exploitation, while preferring the
others to explore). We posit a principal who controls the flow of information
from agents that came before, and strives to coordinate the agents towards a
socially optimal balance between exploration and exploitation, not using any
monetary transfers. The goal is to design a recommendation policy for the
principal which respects agents' incentives and minimizes a suitable notion of
regret.
We extend prior work in this direction to allow the agents to interact with
one another in a shared environment: at each time step, multiple agents arrive
to play a Bayesian game, receive recommendations, choose their actions, receive
their payoffs, and then leave the game forever. The agents now face two sources
of uncertainty: the actions of the other agents and the parameters of the
uncertain game environment.
Our main contribution is to show that the principal can achieve constant
regret when the utilities are deterministic (where the constant depends on the
prior distribution, but not on the time horizon), and logarithmic regret when
the utilities are stochastic. As a key technical tool, we introduce the concept
of explorable actions, the actions which some incentive-compatible policy can
recommend with non-zero probability. We show how the principal can identify
(and explore) all explorable actions, and use the revealed information to
perform optimally.

本文研究了在互联网经济中个体决策者在制定战略选择时的信息生产和消费，重点关注探索、开发和利益激励之间的平衡，并提出了一种不使用任何货币转移的建议政策设计方法以协调决策者。结果表明，经过实验模拟，设计的建议政策可以达到恒定的遗憾值，即使在面对随机收益时也可以达到对数遗憾。

贝叶斯探索：在贝叶斯博弈中激励探索

Bayesian Exploration: Incentivizing Exploration in Bayesian Games

Individual decision-makers consume information revealed by the previous
decision makers, and produce information that may help in future decisions.
This phenomenon is common in a wide range of scenarios in the Internet economy,
as well as in other domains such as medical decisions. Each decision-maker
would individually prefer to "exploit": select an action with the highest
expected reward given her current information. At the same time, each
decision-maker would prefer previous decision-makers to "explore", producing
information about the rewards of various actions. A social planner, by means of
carefully designed information disclosure, can incentivize the agents to
balance the exploration and exploitation so as to maximize social welfare.
We formulate this problem as a multi-armed bandit problem (and various
generalizations thereof) under incentive-compatibility constraints induced by
the agents' Bayesian priors. We design an incentive-compatible bandit algorithm
for the social planner whose regret is asymptotically optimal among all bandit
algorithms (incentive-compatible or not). Further, we provide a black-box
reduction from an arbitrary multi-arm bandit algorithm to an
incentive-compatible one, with only a constant multiplicative increase in
regret. This reduction works for very general bandit setting that incorporate
contexts and arbitrary auxiliary feedback.

研究说明在互联网经济和医疗决策等环境中，每个决策者对信息进行开采利用，并产生可能有助于未来决策的信息，社会规划者通过信息披露来鼓励代理商在探索和开采之间取得平衡，从而最大化社会福利，文中提供了一个多臂赌博问题的激励兼容算法，其遗憾在所有多臂赌博算法中（不管是否具有激励兼容性）是渐进最优的。