In the setting where participants are asked multiple similar possibly
subjective multi-choice questions (e.g. Do you like Panda Express? Y/N; do you
like Chick-fil-A? Y/N), a series of peer prediction mechanisms are designed to
incentivize honest reports and some of them achieve dominantly truthfulness:
truth-telling is a dominant strategy and strictly dominate other
"non-permutation strategy" with some mild conditions. However, a major issue
hinders the practical usage of those mechanisms: they require the participants
to perform an infinite number of tasks. When the participants perform a finite
number of tasks, these mechanisms only achieve approximated dominant
truthfulness. The existence of a dominantly truthful multi-task peer prediction
mechanism that only requires a finite number of tasks remains to be an open
question that may have a negative result, even with full prior knowledge.
This paper answers this open question by proposing a new mechanism,
Determinant based Mutual Information Mechanism (DMI-Mechanism), that is
dominantly truthful when the number of tasks is at least 2C and the number of
participants is at least 2. C is the number of choices for each question (C=2
for binary-choice questions). In addition to incentivizing honest reports,
DMI-Mechanism can also be transferred into an information evaluation rule that
identifies high-quality information without verification when there are at
least 3 participants. To the best of our knowledge, DMI-Mechanism is the first
dominantly truthful mechanism that works for a finite number of tasks, not to
say a small constant number of tasks.

本文提出了一种新的机制，基于行列式互信息机制 (DMI-Mechanism)，可以在只有有限数量 (至少为 2) 的任务情况下实现直接的诚实报告和互信息计算，并且可以作为信息精度评估规则。

具有恒定任务数量的主导诚实多任务同伴预测

Dominantly Truthful Multi-task Peer Prediction with a Constant Number of  Tasks

Strictly proper scoring rules (SPSR) are incentive compatible for eliciting
information about random variables from strategic agents when the principal can
reward agents after the realization of the random variables. They also quantify
the quality of elicited information, with more accurate predictions receiving
higher scores in expectation. In this paper, we extend such scoring rules to
settings where a principal elicits private probabilistic beliefs but only has
access to agents' reports. We name our solution \emph{Surrogate Scoring Rules}
(SSR). SSR build on a bias correction step and an error rate estimation
procedure for a reference answer defined using agents' reports. We show that,
with a single bit of information about the prior distribution of the random
variables, SSR in a multi-task setting recover SPSR in expectation, as if
having access to the ground truth. Therefore, a salient feature of SSR is that
they quantify the quality of information despite the lack of ground truth, just
as SPSR do for the setting \emph{with} ground truth. As a by-product, SSR
induce \emph{dominant truthfulness} in reporting. Our method is verified both
theoretically and empirically using data collected from real human forecasters.

本文提出了一种称为代理评分规则（Surrogate Scoring Rules, SSR）的方法，在私有概率信念的情况下，实现了激励兼容性和支配性诚实性，将之应用于多任务场景中，它们能够评估信息质量，即使没有绝对答案。作者通过真实数据的分析和经验验证，证明了这种方法的有效性。