In-context learning (ICL) is a powerful paradigm emerged from large language
models (LLMs). Despite its promises, ICL performance is known to be highly
sensitive to input examples. In this work, we use in-context influences to
analyze few-shot ICL performance directly from the in-context examples. Our
proposed influence-based example selection method outperforms most baselines
when evaluated on 10 SuperGlue tasks and stably scales with increasing k-shot.
The analysis finds up to a 22.2% performance gap between the most positively
and negatively influential examples. In a case study, we apply our
influence-based framework to quantify the phenomena of recency bias in example
ordering for few-shot ICL.

本文通过使用 in-context 影响性分析 few-shot in-context learning 性能，提出了基于影响的示例选择方法，该方法在 10 个 SuperGlue 任务上优于大多数基线，并且随着 k-shot 的增加而稳定扩展。分析发现最积极和最消极的示例之间的性能差距高达 22.2％。在案例研究中，我们应用影响力框架来量化 few-shot in-context learning 中的近期偏差现象。

具有影响力的上下文示例选择

In-context Example Selection with Influences

We show that Optimistic Hedge -- a common variant of
multiplicative-weights-updates with recency bias -- attains ${\rm poly}(\log
T)$ regret in multi-player general-sum games. In particular, when every player
of the game uses Optimistic Hedge to iteratively update her strategy in
response to the history of play so far, then after $T$ rounds of interaction,
each player experiences total regret that is ${\rm poly}(\log T)$. Our bound
improves, exponentially, the $O({T}^{1/2})$ regret attainable by standard
no-regret learners in games, the $O(T^{1/4})$ regret attainable by no-regret
learners with recency bias (Syrgkanis et al., 2015), and the ${O}(T^{1/6})$
bound that was recently shown for Optimistic Hedge in the special case of
two-player games (Chen & Pen, 2020). A corollary of our bound is that
Optimistic Hedge converges to coarse correlated equilibrium in general games at
a rate of $\tilde{O}\left(\frac 1T\right)$.

研究证明，在多人博弈中使用乐观博弈算法可以实现对数级别的后悔，并可以在一般的游戏中以 $\tilde {O}(1/T)$ 的速率收敛到粗略的相关均衡状态。

一般博弈中的近似最优无悔学习

Near-Optimal No-Regret Learning in General Games

We show that natural classes of regularized learning algorithms with a form
of recency bias achieve faster convergence rates to approximate efficiency and
to coarse correlated equilibria in multiplayer normal form games. When each
player in a game uses an algorithm from our class, their individual regret
decays at $O(T^{-3/4})$, while the sum of utilities converges to an approximate
optimum at $O(T^{-1})$--an improvement upon the worst case $O(T^{-1/2})$ rates.
We show a black-box reduction for any algorithm in the class to achieve
$\tilde{O}(T^{-1/2})$ rates against an adversary, while maintaining the faster
rates against algorithms in the class. Our results extend those of [Rakhlin and
Shridharan 2013] and [Daskalakis et al. 2014], who only analyzed two-player
zero-sum games for specific algorithms.

通过采用具有一种新颖形式的经验回忆的正则化学习算法，我们表明，在多人博弈的普通形式中，该类自适应算法能够实现更快的收敛速率，并实现对近似效率和粗略相关均衡的收敛，并且，对这种类型算法应用的每个玩家，他们的个体后悔降至 $O (T^{-3/4})$，而其效用之和则以 $O (T^{-1})$ 的速度趋于近似最优，在与该类算法相对应的算法维持更快的速率的同时，我们还表明了该类中的任何算法均可通过黑匣子降至 $	ilde {O}(T^{-1/2})$ 的速率来抵抗对手。