Long-term memory plays a critical role in personal interaction, considering
long-term memory can better leverage world knowledge, historical information,
and preferences in dialogues. Our research introduces PerLTQA, an innovative QA
dataset that combines semantic and episodic memories, including world
knowledge, profiles, social relationships, events, and dialogues. This dataset
is collected to investigate the use of personalized memories, focusing on
social interactions and events in the QA task. PerLTQA features two types of
memory and a comprehensive benchmark of 8,593 questions for 30 characters,
facilitating the exploration and application of personalized memories in Large
Language Models (LLMs). Based on PerLTQA, we propose a novel framework for
memory integration and generation, consisting of three main components: Memory
Classification, Memory Retrieval, and Memory Synthesis. We evaluate this
framework using five LLMs and three retrievers. Experimental results
demonstrate that BERT-based classification models significantly outperform LLMs
such as ChatGLM3 and ChatGPT in the memory classification task. Furthermore,
our study highlights the importance of effective memory integration in the QA
task.

该研究通过引入 PerLTQA 数据集，结合语义和经验记忆，包括世界知识、个人资料、社交关系、事件和对话，旨在调查个性化记忆在 QA 任务中的应用，强调了记忆整合在 QA 任务中的重要性。

PerLTQA: 一个个人长期记忆数据集，用于问题回答中的记忆分类、检索和合成

PerLTQA: A Personal Long-Term Memory Dataset for Memory Classification,  Retrieval, and Synthesis in Question Answering

Compute-in-memory (CiM) has emerged as a compelling solution to alleviate
high data movement costs in von Neumann machines. CiM can perform massively
parallel general matrix multiplication (GEMM) operations in memory, the
dominant computation in Machine Learning (ML) inference. However, re-purposing
memory for compute poses key questions on 1) What type of CiM to use: Given a
multitude of analog and digital CiMs, determining their suitability from
systems perspective is needed. 2) When to use CiM: ML inference includes
workloads with a variety of memory and compute requirements, making it
difficult to identify when CiM is more beneficial than standard processing
cores. 3) Where to integrate CiM: Each memory level has different bandwidth and
capacity, that affects the data movement and locality benefits of CiM
integration.
In this paper, we explore answers to these questions regarding CiM
integration for ML inference acceleration. We use Timeloop-Accelergy for early
system-level evaluation of CiM prototypes, including both analog and digital
primitives. We integrate CiM into different cache memory levels in an Nvidia
A100-like baseline architecture and tailor the dataflow for various ML
workloads. Our experiments show CiM architectures improve energy efficiency,
achieving up to 0.12x lower energy than the established baseline with INT-8
precision, and upto 4x performance gains with weight interleaving and
duplication. The proposed work provides insights into what type of CiM to use,
and when and where to optimally integrate it in the cache hierarchy for GEMM
acceleration.

本文研究了在机器学习推断中将计算内存 (Compute-in-memory, CiM) 技术集成的问题，并通过实验证明了使用 CiM 架构可以提高能源效率，达到低至以 INT-8 精度 0.12 倍能源消耗和重复权重交错技术下最高 4 倍的性能增益，同时提供了关于何种类型 CiM 适合使用以及何时何地最佳集成于缓存层级进行 GEMM 加速的见解。

计算 - 内存：什么、何时、何处

WWW: What, When, Where to Compute-in-Memory

Retrieval augmented models show promise in enhancing traditional language
models by improving their contextual understanding, integrating private data,
and reducing hallucination. However, the processing time required for retrieval
augmented large language models poses a challenge when applying them to tasks
that require real-time responses, such as composition assistance.
To overcome this limitation, we propose the Hybrid Retrieval-Augmented
Generation (HybridRAG) framework that leverages a hybrid setting that combines
both client and cloud models. HybridRAG incorporates retrieval-augmented memory
generated asynchronously by a Large Language Model (LLM) in the cloud. By
integrating this retrieval augmented memory, the client model acquires the
capability to generate highly effective responses, benefiting from the LLM's
capabilities. Furthermore, through asynchronous memory integration, the client
model is capable of delivering real-time responses to user requests without the
need to wait for memory synchronization from the cloud. Our experiments on
Wikitext and Pile subsets show that HybridRAG achieves lower latency than a
cloud-based retrieval-augmented LLM, while outperforming client-only models in
utility.

利用混合检索增强生成（HybridRAG）框架，在需要实时响应的任务（如作文帮助）中，将客户端模型与云端模型相结合，通过异步生成的检索增强记忆使客户端模型具备生成高效响应的能力，并实现实时响应无需等待云端内存同步，从而克服了检索增强大型语言模型处理时间长的局限性。实验证明，HybridRAG 在延迟方面优于基于云的检索增强大型语言模型，并在效用方面优于仅具备客户端的模型。