Retrieval augmented models show promise in enhancing traditional language
models by improving their contextual understanding, integrating private data,
and reducing hallucination. However, the processing time required for retrieval
augmented large language models poses a challenge when applying them to tasks
that require real-time responses, such as composition assistance.
To overcome this limitation, we propose the Hybrid Retrieval-Augmented
Generation (HybridRAG) framework that leverages a hybrid setting that combines
both client and cloud models. HybridRAG incorporates retrieval-augmented memory
generated asynchronously by a Large Language Model (LLM) in the cloud. By
integrating this retrieval augmented memory, the client model acquires the
capability to generate highly effective responses, benefiting from the LLM's
capabilities. Furthermore, through asynchronous memory integration, the client
model is capable of delivering real-time responses to user requests without the
need to wait for memory synchronization from the cloud. Our experiments on
Wikitext and Pile subsets show that HybridRAG achieves lower latency than a
cloud-based retrieval-augmented LLM, while outperforming client-only models in
utility.

利用混合检索增强生成（HybridRAG）框架，在需要实时响应的任务（如作文帮助）中，将客户端模型与云端模型相结合，通过异步生成的检索增强记忆使客户端模型具备生成高效响应的能力，并实现实时响应无需等待云端内存同步，从而克服了检索增强大型语言模型处理时间长的局限性。实验证明，HybridRAG 在延迟方面优于基于云的检索增强大型语言模型，并在效用方面优于仅具备客户端的模型。

混合检索增强生成用于实时作曲协助

Hybrid Retrieval-Augmented Generation for Real-time Composition  Assistance

The task of searching certain people in videos has seen increasing potential
in real-world applications, such as video organization and editing. Most
existing approaches are devised to work in an offline manner, where identities
can only be inferred after an entire video is examined. This working manner
precludes such methods from being applied to online services or those
applications that require real-time responses. In this paper, we propose an
online person search framework, which can recognize people in a video on the
fly. This framework maintains a multimodal memory bank at its heart as the
basis for person recognition, and updates it dynamically with a policy obtained
by reinforcement learning. Our experiments on a large movie dataset show that
the proposed method is effective, not only achieving remarkable improvements
over online schemes but also outperforming offline methods.

本文提出了一种在线人员搜索框架，它采用多模态记忆库作为人员识别的基础，并通过强化学习获得策略进行动态更新。实验结果表明，该方法不仅实现了在线模式下的显著改进，而且胜过了离线方法。