Inspired by human cognition, Jiang et al.(2023c) create a benchmark for
assessing LLMs' lateral thinking-thinking outside the box. Building upon this
benchmark, we investigate how different prompting methods enhance LLMs'
performance on this task to reveal their inherent power for outside-the-box
thinking ability. Through participating in SemEval-2024, task 9, Sentence
Puzzle sub-task, we explore prompt engineering methods: chain of thoughts (CoT)
and direct prompting, enhancing with informative descriptions, and employing
contextualizing prompts using a retrieval augmented generation (RAG) pipeline.
Our experiments involve three LLMs including GPT-3.5, GPT-4, and
Zephyr-7B-beta. We generate a dataset of thinking paths between riddles and
options using GPT-4, validated by humans for quality. Findings indicate that
compressed informative prompts enhance performance. Dynamic in-context learning
enhances model performance significantly. Furthermore, fine-tuning Zephyr on
our dataset enhances performance across other commonsense datasets,
underscoring the value of innovative thinking.

Leveraging a benchmark for lateral thinking, this paper examines the effects of various prompting methods on the performance of LLMs, highlighting the benefits of compressed informative prompts, dynamic in-context learning, and fine-tuning Zephyr.

uTeBC-NLP 在 SemEval-2024 任务 9 中：LLMs 能否成为横向思考者？

uTeBC-NLP at SemEval-2024 Task 9: Can LLMs be Lateral Thinkers?

Product bundling has evolved into a crucial marketing strategy in e-commerce.
However, current studies are limited to generating (1) fixed-size or single
bundles, and most importantly, (2) bundles that do not reflect consistent user
intents, thus being less intelligible or useful to users. This paper explores
two interrelated tasks, i.e., personalized bundle generation and the underlying
intent inference based on users' interactions in a session, leveraging the
logical reasoning capability of large language models. We introduce a dynamic
in-context learning paradigm, which enables ChatGPT to seek tailored and
dynamic lessons from closely related sessions as demonstrations while
performing tasks in the target session. Specifically, it first harnesses
retrieval augmented generation to identify nearest neighbor sessions for each
target session. Then, proper prompts are designed to guide ChatGPT to perform
the two tasks on neighbor sessions. To enhance reliability and mitigate the
hallucination issue, we develop (1) a self-correction strategy to foster mutual
improvement in both tasks without supervision signals; and (2) an auto-feedback
mechanism to recurrently offer dynamic supervision based on the distinct
mistakes made by ChatGPT on various neighbor sessions. Thus, the target session
can receive customized and dynamic lessons for improved performance by
observing the demonstrations of its neighbor sessions. Finally, experimental
results on three real-world datasets verify the effectiveness of our methods on
both tasks. Additionally, the inferred intents can prove beneficial for other
intriguing downstream tasks, such as crafting appealing bundle names.

利用大型语言模型的逻辑推理能力，本文探索个性化捆绑生成和基于用户交互的意图推断这两个相关的任务，通过引入动态上下文学习范式，使 ChatGPT 能从相关会话中的示范中获取量身定制和动态的教训，并在目标会话中执行任务。实验结果在三个真实数据集上证实了我们方法在这两个任务上的有效性。此外，推断的意图对于其他有趣的下游任务，如设计吸引人的捆绑名称，也能发挥作用。