Hypothetical induction is recognized as the main reasoning type when
scientists make observations about the world and try to propose hypotheses to
explain those observations. Past research on hypothetical induction has a
limited setting that (1) the observation annotations of the dataset are not raw
web corpus but are manually selected sentences (resulting in a close-domain
setting); and (2) the ground truth hypotheses annotations are mostly
commonsense knowledge, making the task less challenging. In this work, we
propose the first NLP dataset for social science academic hypotheses discovery,
consisting of 50 recent papers published in top social science journals. Raw
web corpora that are necessary for developing hypotheses in the published
papers are also collected in the dataset, with the final goal of creating a
system that automatically generates valid, novel, and helpful (to human
researchers) hypotheses, given only a pile of raw web corpora. The new dataset
can tackle the previous problems because it requires to (1) use raw web corpora
as observations; and (2) propose hypotheses even new to humanity. A
multi-module framework is developed for the task, as well as three different
feedback mechanisms that empirically show performance gain over the base
framework. Finally, our framework exhibits high performance in terms of both
GPT-4 based evaluation and social science expert evaluation.

通过收集社会科学顶级期刊中最近发表的 50 篇论文构建的第一个自然语言处理数据集，该数据集可以解决假设归纳领域中以往局限于特定领域和常识知识的问题，旨在开发一个系统，仅凭一堆原始网页文集就可以自动生成有效、新颖且对社会科学研究人员有帮助的假设。基于多模块框架和三种不同的反馈机制，该框架在 GPT-4 和社会科学专家评估方面表现出较高的性能。