This paper presents the design and evaluation of a novel multi-level LLM
interface for supermarket robots to assist customers. The proposed interface
allows customers to convey their needs through both generic and specific
queries. While state-of-the-art systems like OpenAI's GPTs are highly adaptable
and easy to build and deploy, they still face challenges such as increased
response times and limitations in strategic control of the underlying model for
tailored use-case and cost optimization. Driven by the goal of developing
faster and more efficient conversational agents, this paper advocates for using
multiple smaller, specialized LLMs fine-tuned to handle different user queries
based on their specificity and user intent. We compare this approach to a
specialized GPT model powered by GPT-4 Turbo, using the Artificial Social Agent
Questionnaire (ASAQ) and qualitative participant feedback in a counterbalanced
within-subjects experiment. Our findings show that our multi-LLM chatbot
architecture outperformed the benchmarked GPT model across all 13 measured
criteria, with statistically significant improvements in four key areas:
performance, user satisfaction, user-agent partnership, and self-image
enhancement. The paper also presents a method for supermarket robot navigation
by mapping the final chatbot response to correct shelf numbers, enabling the
robot to sequentially navigate towards the respective products, after which
lower-level robot perception, control, and planning can be used for automated
object retrieval. We hope this work encourages more efforts into using
multiple, specialized smaller models instead of relying on a single powerful,
but more expensive and slower model.

该论文介绍了一种新型的多层 LLM 接口，用于协助超市机器人为顾客提供服务，并提出了使用多个经过细分调整的小型专用 LLM 来处理不同用户查询的方法，通过与 GPT-4 Turbo 相比较，实验结果表明多层 LLM 机器人在性能、用户满意度、用户 - 代理合作及自我形象等四个关键领域取得了显著改进，同时还提出了一种超市机器人导航的方法，使机器人能够根据聊天机器人的响应将产品引导到正确的货架上，然后通过更低级别的机器人知觉、控制和规划实现物品的自动检索。

增强超市机器人交互：针对多样化顾客意图的多层次 LLM 会话界面

Enhancing Supermarket Robot Interaction: A Multi-Level LLM  Conversational Interface for Handling Diverse Customer Intents

Artificial intelligence (AI) has gained significant attention in healthcare
consultation due to its potential to improve clinical workflow and enhance
medical communication. However, owing to the complex nature of medical
information, large language models (LLM) trained with general world knowledge
might not possess the capability to tackle medical-related tasks at an expert
level. Here, we introduce EyeGPT, a specialized LLM designed specifically for
ophthalmology, using three optimization strategies including role-playing,
finetuning, and retrieval-augmented generation. In particular, we proposed a
comprehensive evaluation framework that encompasses a diverse dataset, covering
various subspecialties of ophthalmology, different users, and diverse inquiry
intents. Moreover, we considered multiple evaluation metrics, including
accuracy, understandability, trustworthiness, empathy, and the proportion of
hallucinations. By assessing the performance of different EyeGPT variants, we
identify the most effective one, which exhibits comparable levels of
understandability, trustworthiness, and empathy to human ophthalmologists (all
Ps>0.05). Overall, ur study provides valuable insights for future research,
facilitating comprehensive comparisons and evaluations of different strategies
for developing specialized LLMs in ophthalmology. The potential benefits
include enhancing the patient experience in eye care and optimizing
ophthalmologists' services.

通过三种优化策略，我们引入了专门设计用于眼科学的 EyeGPT，该模型的综合评估框架包括不同领域的眼科学数据集、不同用户和多样化的查询意图。该模型在可理解性、可信度和同理心方面与人类眼科医生相当，为开发专门的大型语言模型在眼科学中提供了有价值的见解。

EyeGPT: 大型语言模型的眼科助理

EyeGPT: Ophthalmic Assistant with Large Language Models

Large Language Models (LLMs) have the potential to revolutionize the way
users self-diagnose through search engines by offering direct and efficient
suggestions. Recent studies primarily focused on the quality of LLMs evaluated
by GPT-4 or their ability to pass medical exams, no studies have quantified the
extent of health-related atomic knowledge stored in LLMs' memory, which is the
basis of LLMs to provide more factual suggestions. In this paper, we first
constructed a benchmark, including the most common types of atomic knowledge in
user self-diagnosis queries, with 17 atomic types and a total of 14, 048 pieces
of atomic knowledge. Then, we evaluated both generic and specialized LLMs on
the benchmark. The experimental results showcased that generic LLMs perform
better than specialized LLMs in terms of atomic knowledge and
instruction-following ability. Error analysis revealed that both generic and
specialized LLMs are sycophantic, e.g., always catering to users' claims when
it comes to unknown knowledge. Besides, generic LLMs showed stronger safety,
which can be learned by specialized LLMs through distilled data. We further
explored different types of data commonly adopted for fine-tuning specialized
LLMs, i.e., real-world, semi-distilled, and distilled data, and found that
distilled data can benefit LLMs most.

通过构建基准测试并评估通用和专用语言模型，我们发现通用语言模型在原子知识和指令遵循能力方面表现更好，而专用语言模型在提供安全性方面表现较差，在提炼数据方面可以通过通用语言模型学习。此外，我们还发现提炼数据对于专用语言模型的微调效果最佳。