We explore the self-play training procedure of large language models (LLMs)
in a two-player adversarial language game called Adversarial Taboo. In this
game, an attacker and a defender communicate with respect to a target word only
visible to the attacker. The attacker aims to induce the defender to utter the
target word unconsciously, while the defender tries to infer the target word
from the attacker's utterances. To win the game, both players should have
sufficient knowledge about the target word and high-level reasoning ability to
infer and express in this information-reserved conversation. Hence, we are
curious about whether LLMs' reasoning ability can be further enhanced by
Self-Play in this Adversarial language Game (SPAG). With this goal, we let LLMs
act as the attacker and play with a copy of itself as the defender on an
extensive range of target words. Through reinforcement learning on the game
outcomes, we observe that the LLMs' performance uniformly improves on a broad
range of reasoning benchmarks. Furthermore, iteratively adopting this self-play
process can continuously promote LLM's reasoning ability. The code is at
this https URL

在一个叫做 Adversarial Taboo 的对抗性语言游戏中，我们探索了大型语言模型（LLMs）的自我训练过程。通过强化学习和自我博弈，我们观察到 LLMs 在广泛的推理基准上表现出统一的提升，并且通过迭代采用自我博弈过程可以不断提升 LLM 的推理能力。

自主对抗性语言游戏增强 LLM 推理能力

Self-playing Adversarial Language Game Enhances LLM Reasoning

Humans tend to strongly agree on ratings on a scale for extreme cases (e.g.,
a CAT is judged as very concrete), but judgements on mid-scale words exhibit
more disagreement. Yet, collected rating norms are heavily exploited across
disciplines. Our study focuses on concreteness ratings and (i) implements
correlations and supervised classification to identify salient multi-modal
characteristics of mid-scale words, and (ii) applies a hard clustering to
identify patterns of systematic disagreement across raters. Our results suggest
to either fine-tune or filter mid-scale target words before utilising them.

我们的研究关注中等规模的单词的具体性评分，并通过相关性和监督分类来识别显著的多模式特征，应用硬聚类方法来确定评分者之间系统性分歧的模式。结果表明，在利用中等规模的目标词之前，需要对其进行微调或过滤。

中等规模评级的争议性质研究：抽象性 - 具体性连续性案例研究

Investigating the Nature of Disagreements on Mid-Scale Ratings: A Case  Study on the Abstractness-Concreteness Continuum

Scene text removal (STR) is the image transformation task to remove text
regions in scene images. The conventional STR methods remove all scene text.
This means that the existing methods cannot select text to be removed. In this
paper, we propose a novel task setting named selective scene text removal
(SSTR) that removes only target words specified by the user. Although SSTR is a
more complex task than STR, the proposed multi-module structure enables
efficient training for SSTR. Experimental results show that the proposed method
can remove target words as expected.

该论文提出了一种名为选择性场景文本去除（SSTR）的新任务设置，该任务允许用户只去除指定的目标词，通过提出的多模块结构，能够高效训练 SSTR，并展示实验结果证明该方法能如预期地去除目标词。