Our paper presents team MasonTigers submission to the SemEval-2024 Task 9 -
which provides a dataset of puzzles for testing natural language understanding.
We employ large language models (LLMs) to solve this task through several
prompting techniques. Zero-shot and few-shot prompting generate reasonably good
results when tested with proprietary LLMs, compared to the open-source models.
We obtain further improved results with chain-of-thought prompting, an
iterative prompting method that breaks down the reasoning process step-by-step.
We obtain our best results by utilizing an ensemble of chain-of-thought
prompts, placing 2nd in the word puzzle subtask and 13th in the sentence puzzle
subtask. The strong performance of prompted LLMs demonstrates their capability
for complex reasoning when provided with a decomposition of the thought
process. Our work sheds light on how step-wise explanatory prompts can unlock
more of the knowledge encoded in the parameters of large models.

我们的研究采用大型语言模型 (LLMs) 和几种提示技术解决自然语言理解测试中提供的谜题数据集。通过零瞄和少瞄提示，相较于开源模型，我们的专有模型表现出合理的结果。通过分步提示的迭代提示方法，我们进一步改进了结果。通过使用一系列分步提示，我们在字谜题子任务中获得第二名，在句子谜题子任务中获得第 13 名。提示型大型语言模型表现出较强的性能，证明了当提供思维过程分解时，它们具备处理复杂推理的能力。我们的工作揭示了分步解释性提示如何释放大型模型参数中更多的知识。