Language models (LMs) are capable of conducting in-context learning for
multiple choice reasoning tasks, but the options in these tasks are treated
equally. As humans often first eliminate wrong options before picking the final
correct answer, we argue a similar two-step strategy can make LMs better at
these tasks. To this end, we present the Process of Elimination (POE), a
two-step scoring method. In the first step, POE scores each option, and
eliminates seemingly wrong options. In the second step, POE masks these wrong
options, and makes the final prediction from the remaining options. Zero-shot
experiments on 8 reasoning tasks illustrate the effectiveness of POE, and a
following analysis finds our method to be especially performant on logical
reasoning tasks. We further analyze the effect of masks, and show that POE
applies to few-shot settings and large language models (LLMs) like ChatGPT.

在多项选择推理任务中，语言模型借鉴了人类首先排除错误选项的两步策略以提高性能，通过引入排除过程（POE）的两步评分方法，实验证明 POE 在逻辑推理任务上表现出色，并适用于 ChatGPT 这样的大型语言模型。