Over decades, neuroscience has accumulated a wealth of research results in
the text modality that can be used to explore cognitive processes.
Meta-analysis is a typical method that successfully establishes a link from
text queries to brain activation maps using these research results, but it
still relies on an ideal query environment. In practical applications, text
queries used for meta-analyses may encounter issues such as semantic redundancy
and ambiguity, resulting in an inaccurate mapping to brain images. On the other
hand, large language models (LLMs) like ChatGPT have shown great potential in
tasks such as context understanding and reasoning, displaying a high degree of
consistency with human natural language. Hence, LLMs could improve the
connection between text modality and neuroscience, resolving existing
challenges of meta-analyses. In this study, we propose a method called
Chat2Brain that combines LLMs to basic text-2-image model, known as Text2Brain,
to map open-ended semantic queries to brain activation maps in data-scarce and
complex query environments. By utilizing the understanding and reasoning
capabilities of LLMs, the performance of the mapping model is optimized by
transferring text queries to semantic queries. We demonstrate that Chat2Brain
can synthesize anatomically plausible neural activation patterns for more
complex tasks of text queries.

该研究提出了一种名为 Chat2Brain 的方法，将大型语言模型与基本的文本 - 图像模型 Text2Brain 相结合，以在数据稀缺和复杂查询环境中将开放式语义查询映射到脑激活图，从而改善元分析的挑战。我们证明 Chat2Brain 能够合成更复杂的文本查询任务的解剖学上合理的神经激活模式。