We address the challenge of aggregating the preferences of multiple agents
over LLM-generated replies to user queries, where agents might modify or
exaggerate their preferences. New agents may participate for each new query,
making fine-tuning LLMs on these preferences impractical. To overcome these
challenges, we propose an auction mechanism that operates without fine-tuning
or access to model weights. This mechanism is designed to provably converge to
the ouput of the optimally fine-tuned LLM as computational resources are
increased. The mechanism can also incorporate contextual information about the
agents when avaiable, which significantly accelerates its convergence. A
well-designed payment rule ensures that truthful reporting is the optimal
strategy for all agents, while also promoting an equity property by aligning
each agent's utility with her contribution to social welfare - an essential
feature for the mechanism's long-term viability. While our approach can be
applied whenever monetary transactions are permissible, our flagship
application is in online advertising. In this context, advertisers try to steer
LLM-generated responses towards their brand interests, while the platform aims
to maximize advertiser value and ensure user satisfaction. Experimental results
confirm that our mechanism not only converges efficiently to the optimally
fine-tuned LLM but also significantly boosts advertiser value and platform
revenue, all with minimal computational overhead.

我们提出一种无需微调或访问模型权重的拍卖机制，能有效聚合多个参与者对 LLM 生成的用户查询回复的偏好，并能加入可用的上下文信息，实现快速收敛，从而提高广告主价值和平台收入。

在线广告中基于 LLMs 的真实聚合

Truthful Aggregation of LLMs with an Application to Online Advertising

In this paper we investigate the problem of measuring end-to-end Incentive
Compatibility (IC) regret given black-box access to an auction mechanism. Our
goal is to 1) compute an estimate for IC regret in an auction, 2) provide a
measure of certainty around the estimate of IC regret, and 3) minimize the time
it takes to arrive at an accurate estimate. We consider two main problems, with
different informational assumptions: In the \emph{advertiser problem} the goal
is to measure IC regret for some known valuation $v$, while in the more general
\emph{demand-side platform (DSP) problem} we wish to determine the worst-case
IC regret over all possible valuations. The problems are naturally phrased in
an online learning model and we design $Regret-UCB$ algorithms for both
problems. We give an online learning algorithm where for the advertiser problem
the error of determining IC shrinks as $O\Big(\frac{|B|}{T}\cdot\Big(\frac{\ln
T}{n} + \sqrt{\frac{\ln T}{n}}\Big)\Big)$ (where $B$ is the finite set of bids,
$T$ is the number of time steps, and $n$ is number of auctions per time step),
and for the DSP problem it shrinks as $O\Big(\frac{|B|}{T}\cdot\Big(
\frac{|B|\ln T}{n} + \sqrt{\frac{|B|\ln T}{n}}\Big)\Big)$. For the DSP problem,
we also consider stronger IC regret estimation and extend our $Regret-UCB$
algorithm to achieve better IC regret error. We validate the theoretical
results using simulations with Generalized Second Price (GSP) auctions, which
are known to not be incentive compatible and thus have strictly positive IC
regret.

本文旨在通过黑盒方法探讨计算竞拍机制中的端到端激励兼容性（IC）遗憾，提供一种确定可靠估计 IC 遗憾的度量和最小化到达 - 准确估计所需时间的方法。我们设计了面向广告主问题和需求方平台问题的 $Regret-UCB$ 算法，并验证了模拟实验的理论结果，从而扩展了我们的 $Regret-UCB$ 算法以实现更好的 IC 遗憾误差。

在线学习用于衡量广告拍卖中的激励兼容性

Online Learning for Measuring Incentive Compatibility in Ad Auctions

Sponsored search is an important monetization channel for search engines, in
which an auction mechanism is used to select the ads shown to users and
determine the prices charged from advertisers. There have been several pieces
of work in the literature that investigate how to design an auction mechanism
in order to optimize the revenue of the search engine. However, due to some
unrealistic assumptions used, the practical values of these studies are not
very clear. In this paper, we propose a novel \emph{game-theoretic machine
learning} approach, which naturally combines machine learning and game theory,
and learns the auction mechanism using a bilevel optimization framework. In
particular, we first learn a Markov model from historical data to describe how
advertisers change their bids in response to an auction mechanism, and then for
any given auction mechanism, we use the learnt model to predict its
corresponding future bid sequences. Next we learn the auction mechanism through
empirical revenue maximization on the predicted bid sequences. We show that the
empirical revenue will converge when the prediction period approaches infinity,
and a Genetic Programming algorithm can effectively optimize this empirical
revenue. Our experiments indicate that the proposed approach is able to produce
a much more effective auction mechanism than several baselines.

本文提出一种新颖的博弈论机器学习方法，该方法自然地合并机器学习和博弈论，通过双层优化框架学习拍卖机制以实现经验收入最大化。实验证明，该方法能够比几个基线方法产生更有效的拍卖机制。