This paper presents the development of a specialized chatbot for materials
science, leveraging the Llama-2 language model, and continuing pre-training on
the expansive research articles in the materials science domain from the S2ORC
dataset. The methodology involves an initial pretraining phase on over one
million domain-specific papers, followed by an instruction-tuning process to
refine the chatbot's capabilities. The chatbot is designed to assist
researchers, educators, and students by providing instant, context-aware
responses to queries in the field of materials science. We make the four
trained checkpoints (7B, 13B, with or without chat ability) freely available to
the research community at this https URL

本文介绍了一种专门用于材料科学的聊天机器人的开发，利用了 Llama-2 语言模型，并在 S2ORC 数据集中的广阔研究论文上继续进行预训练。该方法包括对 100 万多个领域特定论文进行初步预训练，然后通过指令调整过程来改进聊天机器人的能力。该聊天机器人旨在通过为材料科学领域的查询提供即时的、上下文感知的回复，来帮助研究人员、教育工作者和学生。我们提供了四个经过训练的检查点（7B、13B，具有或不具有聊天功能），供研究社区免费使用。

Quokka: 一个面向材料科学的开源大型语言模型聊天机器人

Quokka: An Open-source Large Language Model ChatBot for Material Science

The intelligent dialogue system, aiming at communicating with humans
harmoniously with natural language, is brilliant for promoting the advancement
of human-machine interaction in the era of artificial intelligence. With the
gradually complex human-computer interaction requirements (e.g., multimodal
inputs, time sensitivity), it is difficult for traditional text-based dialogue
system to meet the demands for more vivid and convenient interaction.
Consequently, Visual Context Augmented Dialogue System (VAD), which has the
potential to communicate with humans by perceiving and understanding multimodal
information (i.e., visual context in images or videos, textual dialogue
history), has become a predominant research paradigm. Benefiting from the
consistency and complementarity between visual and textual context, VAD
possesses the potential to generate engaging and context-aware responses. For
depicting the development of VAD, we first characterize the concepts and unique
features of VAD, and then present its generic system architecture to illustrate
the system workflow. Subsequently, several research challenges and
representative works are detailed investigated, followed by the summary of
authoritative benchmarks. We conclude this paper by putting forward some open
issues and promising research trends for VAD, e.g., the cognitive mechanisms of
human-machine dialogue under cross-modal dialogue context, and
knowledge-enhanced cross-modal semantic interaction.

本文介紹了智能對話系統的演進並提出了基於多模態信息的 Visual Context Augmented Dialogue System (VAD) 與其系統架構，藉此提高與人類互動的自然性和方便性，並探討了幾個有挑戰性與有代表性的研究議題。