LLMs have demonstrated impressive zero-shot performance on NLP tasks thanks to the knowledge they acquired in their training. In multiple-choice QA tasks, the LM probabilities are used as an imperfect measure of the plausibility of each answer choice. One of the major limitations of the basic score is that it treats all words as equally important. We propose CASE, a Commonsense-Augmented Score with an Expanded Answer Space. CASE addresses this limitation by assigning importance weights for individual words based on their semantic relations to other words in the input. The dynamic weighting approach outperforms basic LM scores, not only because it reduces noise from unimportant words, but also because it informs the model of implicit commonsense knowledge that may be useful for answering the question. We then also follow prior work in expanding the answer space by generating lexically-divergent answers that are conceptually-similar to the choices. When combined with answer space expansion, our method outperforms strong baselines on 5 commonsense benchmarks. We further show these two approaches are complementary and may be especially beneficial when using smaller LMs.

LLMs在NLP任务中表现出令人印象深刻的零样本性能，但其基本分数方法存在限制，我们提出了一种基于常识增强得分和扩展答案空间的方法，它通过赋予单词的重要性权重来提高LM概率分数，该动态加权方法不仅能减少噪声，还能为模型提供隐含的常识知识，结合答案空间扩展，我们的方法在5个常识基准测试中超过了强基线方法，我们进一步证明了这两种方法的互补性和在使用较小的LM时特别有益。

案例：带有扩展答案空间的常识增强评分