We present a baseline for the SemEval 2024 task 2 challenge, whose objective
is to ascertain the inference relationship between pairs of clinical trial
report sections and statements. We apply prompt optimization techniques with
LLM Instruct models provided as a Language Model-as-a-Service (LMaaS). We
observed, in line with recent findings, that synthetic CoT prompts
significantly enhance manually crafted ones.

我们提出了 SemEval 2024 任务 2 挑战的基线，其目标是确定临床试验报告部分和陈述之间的推理关系。我们应用语言模型作为服务（LMaaS）的 LLM Instruct 模型进行提示优化技术，并观察到合成 CoT 提示显著增强了手工制作的提示，与最近的研究结果一致。

CRCL 在 SemEval-2024 任务 2 中的简单提示优化

CRCL at SemEval-2024 Task 2: Simple prompt optimizations

Large Language Models (LLMs) have emerged as powerful tools in various
research domains. This article examines their potential through a literature
review and firsthand experimentation. While LLMs offer benefits like
cost-effectiveness and efficiency, challenges such as prompt tuning, biases,
and subjectivity must be addressed. The study presents insights from
experiments utilizing LLMs for qualitative analysis, highlighting successes and
limitations. Additionally, it discusses strategies for mitigating challenges,
such as prompt optimization techniques and leveraging human expertise. This
study aligns with the 'LLMs as Research Tools' workshop's focus on integrating
LLMs into HCI data work critically and ethically. By addressing both
opportunities and challenges, our work contributes to the ongoing dialogue on
their responsible application in research.

通过文献综述和第一手实验，本文研究了大型语言模型（LLMs）的潜力。尽管 LLMs 具有成本效益和高效性等优点，但也存在着诸如提示调优、偏见和主观性等挑战。该研究通过利用 LLMs 进行定性分析的实验提供了新的见解，强调了成功和限制。此外，本文还讨论了缓解挑战的策略，如优化提示技术和利用人类专业知识。我们的工作旨在将 LLMs 有机地融入人机交互数据工作，并积极促进其负责任的应用，以此回应关于 LLMs 在研究中负责任应用的持续对话。

由学徒到研究助理：大型语言模型推动研究

Apprentices to Research Assistants: Advancing Research with Large  Language Models

We propose a new method, Adversarial In-Context Learning (adv-ICL), to
optimize prompt for in-context learning (ICL) by employing one LLM as a
generator, another as a discriminator, and a third as a prompt modifier. As in
traditional adversarial learning, adv-ICL is implemented as a two-player game
between the generator and discriminator, where the generator tries to generate
realistic enough output to fool the discriminator. In each round, given an
input prefixed by task instructions and several exemplars, the generator
produces an output. The discriminator is then tasked with classifying the
generator input-output pair as model-generated or real data. Based on the
discriminator loss, the prompt modifier proposes possible edits to the
generator and discriminator prompts, and the edits that most improve the
adversarial loss are selected. We show that adv-ICL results in significant
improvements over state-of-the-art prompt optimization techniques for both open
and closed-source models on 11 generation and classification tasks including
summarization, arithmetic reasoning, machine translation, data-to-text
generation, and the MMLU and big-bench hard benchmarks. In addition, because
our method uses pre-trained models and updates only prompts rather than model
parameters, it is computationally efficient, easy to extend to any LLM and
task, and effective in low-resource settings.

我们提出了一种新的方法，Adversarial In-Context Learning (adv-ICL)，通过使用一个 LLM 作为生成器，另一个作为鉴别器，以及第三个作为提示修改器来优化背景学习中的提示。我们表明，adv-ICL 在包括摘要、算术推理、机器翻译、数据生成和 MMLU 和 big-bench 难度基准等 11 个生成和分类任务上，相对于最先进的提示优化技术取得了显著的改进。此外，由于我们的方法使用预训练模型，并且仅更新提示而不是模型参数，它在计算上是高效的，易于扩展到任何 LLM 和任务，并且在资源有限的环境中是有效的。