Is Complex Agentic Memory Always Needed? Naive RAG Is Stronger Than We Thought
MemoryAgentRAG西安交通大学上海交通大学浙江大学香港中文大学复杂的智能体记忆总是必要的吗？朴素 RAG 比我们想象的更强
Abstract
LLM memory is widely used in long-term conversation understanding. Agentic memory systems often use mutation, structural organization, learned retrieval strategies, or other complex operations. Is this complexity always necessary? We study Raw-RAG, a naive RAG baseline using dense retrieval and reranking over raw history to supply evidence to a reader LLM. Across LoCoMo, LongMemEval, MEME, and MemoryAgentBench, Raw-RAG ranks first among the compared memory systems in six of seven benchmark–model settings, outperforming the strongest compared agentic memory system by 0.13–5.53 points.
LLM 记忆被广泛用于长期对话理解。 智能体记忆系统常常采用内容变更、结构化组织、学习得到的检索策略或其他复杂操作。 这种复杂性总是必要的吗？ 我们研究 Raw-RAG：一种朴素的 RAG 基线，通过对原始历史进行稠密检索和重排序，为读取证据的 LLM 提供证据。 在 LoCoMo、LongMemEval、MEME 和 MemoryAgentBench 上，Raw-RAG 在七种基准与模型组合中的六种上位居所比较记忆系统的首位，比参与比较的最强智能体记忆系统高出 0.13–5.53 个百分点。
We investigate why raw-history retrieval remains competitive. Evidence analysis shows that its higher coverage of necessary evidence can offset lower utilization. Useful evidence includes both answer values and the relations and dependency paths needed to derive them. Even complete access to transformed memory stores leaves an accuracy gap relative to raw evidence, highlighting the importance of preserving raw information. These findings challenge the necessity of complex agentic memory for the evaluated conversation tasks and establish naive RAG as an essential baseline.
我们探究了原始历史检索为何仍具有竞争力。 证据分析表明，它对必要证据的更高覆盖率可以弥补较低的利用率。 有用的证据既包括答案值，也包括推导这些值所需的关系与依赖路径。 即使能够完整访问经过转换的记忆存储，相较原始证据仍存在准确率差距，这凸显了保留原始信息的重要性。 这些发现对所评估对话任务中复杂智能体记忆的必要性提出了质疑，并确立了朴素 RAG 作为必要基线的地位。
图1：以较低的摄取成本实现较高准确率。左：七种基准与模型组合上的准确率，各扇区采用截断的线性刻度。右：BEAM-1M 上的累计顺序摄取时间。
1. Introduction
Large language models (LLMs) are increasingly used in long-term interactions, where they must draw on information accumulated across many conversations and tasks. The resulting histories can far exceed a model's context window, yet relevant past information must remain accessible for future responses. To address this challenge, recent research has developed a range of agentic memory systems that store, organize, and retrieve information beyond the active context. Memory management includes extraction, summarization, mutation, consolidation, clustering, graph construction, and community discovery, while retrieval may involve iterative search or trainable retrieval strategies. These developments raise a basic question: when is the added complexity of agentic memory systems necessary? To investigate, we start with a simple alternative: retrieval-augmented generation (RAG) over raw history.
大语言模型（LLM）越来越多地用于长期交互，需要利用在多次对话和任务中积累的信息。 由此产生的历史记录可能远远超出模型的上下文窗口，但相关的历史信息仍须能够被访问，以支持后续回答。 为应对这一挑战，近期研究开发了多种智能体记忆系统，在当前活动上下文之外存储、组织和检索信息。 记忆管理包括抽取、摘要、内容变更、整合、聚类、图构建和社区发现，而检索可能涉及迭代搜索或可训练的检索策略。 这些进展引出了一个基本问题：智能体记忆系统增加的复杂性在何时是必要的？ 为探究这一问题，我们从一种简单的替代方案入手：对原始历史进行检索增强生成（RAG）。
RAG is often considered insufficient for long-term memory. Two common objections to raw-history retrieval motivate our investigation. The first is conceptual: RAG is commonly associated with static corpora, whereas long-term interactions involve superseded facts, deletions, and dependent state changes. Can retrieving history suffice without explicitly maintaining current state? The second is empirical: prior evaluations report RAG trailing agentic memory systems.
人们常认为 RAG 不足以胜任长期记忆。 针对原始历史检索的两类常见质疑促使我们开展这项研究。 第一类是概念上的质疑：RAG 通常与静态语料库联系在一起，而长期交互涉及被新事实取代的旧事实、删除操作以及具有依赖关系的状态变化。 在不显式维护当前状态的情况下，仅检索历史记录是否足够？ 第二类是实证上的质疑：已有评估报告称，RAG 的表现落后于智能体记忆系统。
图2：Mem0 提议的变更很少，实际生效的更少。上方：写入操作分布。下方：LoCoMo（左）和 LongMemEval（右）中各对话的提议变更率与有效变更率。虚线表示按决策数加权的均值；所有比率均以全部输出决策为分母。
These concerns motivate a closer examination of memory management. We first ask how often memory tasks involve dynamic changes, how complex those changes are, and whether they exceed what reader LLMs can handle directly from retrieved evidence. Two preliminary experiments examine explicit state maintenance. We first audit Mem0 on all LoCoMo conversations and all 500 LongMemEval examples. UPDATE and DELETE together account for only 4.39% of the 33,902 write decisions on LoCoMo and 2.46% of the 428,858 write decisions on LongMemEval (Figure 2); many proposed updates are duplicates or malformed.
这些疑问促使我们更仔细地考察记忆管理。 我们首先询问：记忆任务涉及动态变化的频率有多高，这些变化有多复杂，以及它们是否超出了读取证据的 LLM 直接利用检索证据进行处理的能力。 两项初步实验考察了显式状态维护。 我们首先审计了 Mem0 在全部 LoCoMo 对话和全部 500 个 LongMemEval 样本上的行为。 UPDATE 与 DELETE 合计仅占 LoCoMo 中 33,902 次写入决策的 4.39%，以及 LongMemEval 中 428,858 次写入决策的 2.46%（图2）；许多提议的更新是重复的或格式有误。
Effective mutations, defined as persisted text-changing updates or deletions, account for only 1.90% and 1.34% of all decisions. Replacing UPDATE with append-only ADD and DELETE with NONE raises LoCoMo accuracy from 67.36% to 69.42%, without reducing any category score (Table 1). Explicit mutation provides no accuracy gain in this configuration. We then test whether raw evidence is sufficient for dynamic conversation on MEME, which includes exact recall, aggregation, tracking, deletion, cascade, and invalidation tasks. Readers receive the evidence available at the queried phase in chronological order, eliminating retrieval errors but still requiring them to resolve state changes.
有效变更被定义为已持久化且改变文本的更新或删除，仅占全部决策的 1.90% 和 1.34%。 将 UPDATE 替换为仅追加的 ADD，并将 DELETE 替换为 NONE，可使 LoCoMo 准确率从 67.36% 提高到 69.42%，且没有降低任何类别的得分（表1）。 在这一配置下，显式变更没有带来准确率收益。 随后，我们在 MEME 上测试原始证据是否足以应对动态对话，该基准包括精确回忆、聚合、追踪、删除、级联和失效任务。 读取模型按时间顺序接收被查询阶段可见的证据，从而排除检索错误，但仍须解析状态变化。
DeepSeek-V4-Flash reaches 82.32% on cascades and 91.54% on invalidation (Table 2). Performance varies substantially across readers, but these results show that difficult transitions can be resolved from sufficient raw evidence. Together, these diagnostics suggest that capable readers can often resolve changing states directly from raw evidence. To examine the empirical concern, we review 14 prior works and identify coarse retrieval units, restricted retrieval depth, or absent reranking in several evaluated baselines (Appendix E). These limitations leave open whether the reported gaps reflect a need for agentic memory systems or weaknesses in the chosen retrieval baselines.
DeepSeek-V4-Flash 在级联任务上达到 82.32%，在失效任务上达到 91.54%（表2）。 不同读取模型之间的性能差异很大，但这些结果表明，只要原始证据充分，就能够解析困难的状态转移。 总体而言，这些诊断结果说明，能力较强的读取模型往往能够直接依据原始证据解析不断变化的状态。 为考察实证层面的质疑，我们回顾了 14 项既有工作，发现若干被评估的基线存在检索单元粗糙、检索深度受限或缺少重排序的问题（附录 E）。 这些局限使一个问题仍悬而未决：所报告的差距究竟反映了对智能体记忆系统的需要，还是所选检索基线本身的不足。
| Category | Full policy | ADD/NONE only | Δ |
|---|---|---|---|
| Multi-hop | 63.24 | 64.66 | +1.42 |
| Temporal | 49.74 | 53.06 | +3.32 |
| Open-domain | 53.47 | 60.42 | +6.94 |
| Single-hop | 77.05 | 78.28 | +1.23 |
| Overall | 67.36 | 69.42 | +2.06 |
表1：Mem0 在完整变更策略和仅使用 ADD/NONE 策略下的 LoCoMo 准确率。排除类别 5 的对抗性问题。Δ 表示仅使用 ADD/NONE 的结果减去完整策略的结果。
| Task | GPT-4o-mini | DeepSeek-V4-Flash | Qwen3.5-35B-A3B |
|---|---|---|---|
| ER | 98.00 | 99.00 | 97.00 |
| Agg | 47.00 | 63.00 | 56.00 |
| Tr | 81.00 | 92.00 | 89.00 |
| Del | 39.00 | 86.00 | 46.00 |
| Cas | 1.83 | 82.32 | 39.63 |
| Abs | 36.15 | 91.54 | 93.85 |
| Overall | 50.50 | 85.48 | 70.25 |
表2：MEME 阶段可见 oracle 的准确率（%）。Overall 为六项任务的宏平均。
We introduce Raw-RAG as a diagnostic baseline: naive RAG over raw history, using only dense retrieval and cross-encoder reranking to select evidence for a reader. Across four benchmarks, this simple baseline ranks first among the compared memory systems in six of seven benchmark–model settings.
我们提出 Raw-RAG 作为诊断基线：对原始历史采用朴素 RAG，仅使用稠密检索和交叉编码器重排序，为读取模型选择证据。 在四个基准上，这一简单基线在七种基准与模型组合中的六种上位居所比较记忆系统的首位。
Our contributions are summarized as follows:
我们的贡献概括如下：
- We challenge the presumed necessity of explicit memory mutation: capable readers can resolve dynamic states from sufficient raw history, and disabling mutation improves accuracy in the tested configuration.
- We establish Raw-RAG as a strong reference baseline, ranking first in six of seven settings across four benchmarks, showing that naive RAG matches or exceeds the compared agentic memory systems in most settings.
- We identify evidence access and relational completeness as key bottlenecks: retrieving answer values does not ensure sufficient supporting paths, and complete-store access does not close the gap to raw evidence.
- 我们质疑显式记忆变更被预设为必要的观点：能力较强的读取模型能够依据充分的原始历史解析动态状态，而且在所测试的配置中，禁用变更反而提高了准确率。
- 我们确立了 Raw-RAG 这一强有力的参考基线，它在四个基准的七种设置中有六种位居第一，表明朴素 RAG 在多数设置下可与所比较的智能体记忆系统持平或超过它们。
- 我们发现，证据访问与关系完整性是关键瓶颈：检索到答案值并不保证获得充分的支撑路径，而完整访问记忆存储也无法消除与原始证据之间的差距。
2. Related Work
2.1. Agentic memory systems
External-memory architectures organize persistent information through write, management, and read operations. Early approaches summarize or reflect on experience streams, or provide tiered stores and read–write interfaces. Recent approaches extract and mutate facts or maintain evolving graphs, consolidate hierarchical or recurrent stores, or construct selective event-level and abstract representations. Hybrid and lightweight approaches retain source episodes alongside derived memories or selectively maintain events and preferences.
外部记忆架构通过写入、管理和读取操作来组织持久化信息。 早期方法对经验流进行总结或反思，或提供分层存储和读写接口。 近期方法会抽取并变更事实、维护不断演化的图、整合层级式或循环式存储，或构建有选择的事件级表示与抽象表示。 混合式和轻量化方法在保留派生记忆的同时保留原始片段，或有选择地维护事件与偏好。
Concurrent work studies ReFind, which preserves raw chat logs and gives an agent iterative lexical search with session, temporal, context-expansion, and deduplication controls. We evaluate a fixed dense-retrieval and reranking baseline, isolating evidence access from query-time search control. We compare these systems with naive RAG for long-term conversation understanding and diagnose their evidence-access limitations.
同期工作研究了 ReFind，它保留原始聊天日志，并为智能体提供迭代式词法搜索，以及会话、时间、上下文扩展和去重控制。 我们评估一种固定的稠密检索与重排序基线，将证据访问与查询时的搜索控制区分开来。 我们将这些系统与用于长期对话理解的朴素 RAG 进行比较，并诊断它们在证据访问方面的局限。
2.2. Retrieval-augmented generation
Retrieval-augmented language models combine external non-parametric memory with a generator. Lexical and dense retrievers select candidates; late-interaction models and cross-encoders refine ranking. Structure-aware variants use summaries, graphs, or hierarchies. Adaptive variants learn when to retrieve and assess evidence or answers. Long-context and memory-QA studies emphasize evidence selection, granularity, index keys, and temporal cues. We use standard dense retrieval and reranking to assess what agentic memory systems add to long-term conversation understanding.
检索增强语言模型将外部非参数记忆与生成器结合起来。 词法检索器和稠密检索器负责选择候选项，后交互模型和交叉编码器则进一步优化排序。 结构感知变体使用摘要、图或层级结构。 自适应变体学习何时进行检索，以及如何评估证据或答案。 长上下文和记忆问答研究强调证据选择、粒度、索引键和时间线索。 我们使用标准的稠密检索和重排序，评估智能体记忆系统能为长期对话理解带来什么额外价值。
3. The RAW-RAG
图3：智能体记忆系统（EverMemOS 和 Memora）与朴素 RAG（Raw-RAG）在构建、存储和召回阶段的流程。
Our motivation is to test whether agentic memory systems improve long-term conversation understanding beyond retrieval over raw history. Raw-RAG (Figure 3(c)) provides a simple reference, leaving evidence interpretation and state reconstruction to the reader.
我们的动机是检验：相比对原始历史进行检索，智能体记忆系统能否进一步改善长期对话理解。 Raw-RAG（图3(c)）提供了一个简单的参照，将证据解释和状态重建交给读取模型。
Raw-RAG indexes source history without LLM-based extraction, summarization, graph construction, or mutation. Each conversation is serialized with message timestamps and image captions, split into 256-token windows with 32-token overlap, and embedded in a flat index. These embeddings support semantic matching while the original window text remains available as reader evidence.
Raw-RAG 直接为原始历史建立索引，不进行基于 LLM 的抽取、摘要、图构建或内容变更。 每段对话连同消息时间戳和图像描述一起序列化，划分为长度为 256 个 token、重叠 32 个 token 的窗口，再进行嵌入并存入扁平索引。 这些嵌入支持语义匹配，同时保留原始窗口文本作为读取模型的证据。
At query time, the question is embedded and cosine similarity retrieves the 128 closest windows. A cross-encoder jointly scores each question–window pair for relevance and selects the top 15. The selected evidence is then supplied with the question to the reader, which resolves temporal order and relevant state changes when generating the answer. Top-15 limits the number of selected items. Appendix B specifies the full pipeline.
查询时，系统对问题进行嵌入，并利用余弦相似度检索最接近的 128 个窗口。 交叉编码器对每个问题与窗口组成的配对联合进行相关性评分，并选择排名前 15 的窗口。 随后，选定的证据与问题一起提供给读取模型，由其在生成答案时解析时间顺序和相关状态变化。 Top-15 限制的是所选条目的数量。 附录 B 给出了完整流程。
4. RAG Is Stronger Than We Thought
This section presents our experiments evaluating Raw-RAG against agentic memory systems, including benchmark comparisons, ablations, and efficiency.
本节介绍将 Raw-RAG 与智能体记忆系统进行比较的实验，包括基准比较、消融实验和效率评估。
4.1. Benchmarks and baselines
We evaluate memory-augmented conversation on four benchmarks, namely LoCoMo (10 conversations; 1,540 non-adversarial questions on single-hop, multi-hop, temporal, and open-domain reasoning), LongMemEval (500 examples on recall, temporal reasoning, preferences, and updates), MEME (100 episodes; 1,188 phase-specific questions on dynamic memory updates and complex memory dependency understanding), and MemoryAgentBench (2,071 questions from 14 sources on accurate retrieval, test-time learning, long-range understanding, and conflict resolution).
我们在四个基准上评估记忆增强对话：LoCoMo（10 段对话；1,540 个涉及单跳、多跳、时间和开放域推理的非对抗性问题）、LongMemEval（500 个涉及回忆、时间推理、偏好和更新的样本）、MEME（100 个片段；1,188 个针对特定阶段、涉及动态记忆更新和复杂记忆依赖理解的问题），以及 MemoryAgentBench（来自 14 个来源的 2,071 个问题，涉及准确检索、测试时学习、长程理解和冲突消解）。
Baselines include EverMemOS, MemOS, EMem, SimpleMem, MemU, MemMachine, Mem0, Memora, Nemori, MemoryOS, A-MEM, and RecMem, alongside Full-context. On LoCoMo and LongMemEval, we evaluate Raw-RAG under settings fully aligned with the corresponding comparison settings, as detailed in Appendix A. On MEME and MemoryAgentBench, all systems follow the same evaluation procedure. Comparisons under aligned evaluation settings and their scope are presented in Appendix A. Appendices D.1 and D.3 detail benchmark tasks and baseline settings. Appendix A.4 reports evidence lengths and 5,000-token controls, where Raw-RAG remains competitive.
基线包括 EverMemOS、MemOS、EMem、SimpleMem、MemU、MemMachine、Mem0、Memora、Nemori、MemoryOS、A-MEM 和 RecMem，以及 Full-context。 在 LoCoMo 和 LongMemEval 上，我们在与相应比较设置完全对齐的条件下评估 Raw-RAG，详见 附录 A。 在 MEME 和 MemoryAgentBench 上，所有系统都遵循相同的评估流程。 附录 A 给出了对齐评估设置下的比较及其适用范围。 附录 D.1 和 D.3 详细介绍了基准任务和基线设置。 附录 A.4 报告了证据长度和 5,000 token 预算下的对照实验，Raw-RAG 在这些实验中仍具有竞争力。
4.2. Main results
| Method | LoCoMo | LongMemEval | |||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| GPT-4o-mini | GPT-4.1-mini | GPT-4.1-mini | |||||||||||||||
| Single | Multi | Temp. | Open | Overall | Single | Multi | Temp. | Open | Overall | S-U. | S-A. | S-P. | M-S. | KU. | TR. | Overall | |
| Full-context | 83.00 | 66.80 | 56.20 | 48.60 | 72.30 | 86.90 | 77.20 | 74.20 | 56.60 | 80.60 | 85.70 | 98.20 | 16.70 | 51.10 | 76.90 | 60.20 | 65.60 |
| MemU | 72.77 | 62.41 | 33.96 | 46.88 | 61.15 | 74.91 | 72.34 | 43.61 | 54.17 | 66.67 | 67.14 | 19.64 | 76.67 | 42.10 | 41.02 | 17.29 | 38.40 |
| Mem0 | 67.13 | 51.15 | 55.51 | 72.93 | 66.88 | 68.97 | 61.70 | 58.26 | 50.00 | 64.20 | 82.86 | 26.78 | 90.00 | 63.15 | 66.67 | 72.18 | 66.40 |
| A-MEM | 66.83 | 52.13 | 61.99 | 30.21 | 60.84 | 73.25 | 59.93 | 72.90 | 42.71 | 68.83 | 95.71 | 100.00 | 63.33 | 61.65 | 82.05 | 52.63 | 71.60 |
| MemoryOS | 74.55 | 58.87 | 44.24 | 46.88 | 63.64 | 77.05 | 66.31 | 47.66 | 55.21 | 67.60 | 94.29 | 89.29 | 100.00 | 67.67 | 80.77 | 54.89 | 74.40 |
| Nemori | 82.10 | 65.30 | 71.00 | 44.80 | 74.40 | 84.90 | 75.10 | 77.60 | 51.00 | 79.40 | 90.00 | 92.90 | 86.70 | 55.60 | 79.50 | 72.20 | 74.60 |
| RecMem | 79.79 | 72.70 | 57.32 | 58.33 | 72.47 | 86.92 | 78.37 | 75.39 | 57.29 | 81.10 | 95.71 | 96.43 | 86.67 | 65.41 | 75.64 | 68.42 | 76.80 |
| MemOS | 81.09 | 67.49 | 75.18 | 55.90 | 75.80 | 85.37 | 79.43 | 75.08 | 64.58 | 80.76 | 95.71 | 67.86 | 96.67 | 70.67 | 74.26 | 77.44 | 77.80 |
| SimpleMem | 86.01 | 74.94 | 63.76 | 68.40 | 78.25 | 88.15 | 79.55 | 78.82 | 76.04 | 83.87 | 85.71 | 75.00 | 76.67 | 60.92 | 79.48 | 83.46 | 76.87 |
| EMem | 83.00 | 70.20 | 77.10 | 60.20 | 78.00 | 89.70 | 79.00 | 80.00 | 65.20 | 84.20 | 93.80 | 82.10 | 46.70 | 82.10 | 95.40 | 80.60 | 83.00 |
| MemMachine | 93.94 | 84.04 | 80.69 | 73.96 | 88.12 | 95.12 | 88.30 | 91.59 | 71.88 | 91.69 | 92.86 | 100.00 | 55.56 | 66.67 | 86.32 | 58.65 | 74.33 |
| Memora | 87.08 | 69.50 | 79.54 | 60.07 | 80.61 | 91.80 | 78.70 | 86.60 | 59.40 | 86.30 | 98.60 | 78.60 | 83.30 | 78.20 | 97.40 | 89.50 | 87.40 |
| EverMemOS | 91.08 | 86.17 | 81.93 | 66.67 | 86.76 | 96.67 | 91.84 | 89.72 | 76.04 | 93.05 | 97.14 | 85.71 | 93.33 | 73.68 | 89.74 | 77.44 | 83.00 |
| Raw-RAG | 94.57 | 85.93 | 82.04 | 73.61 | 89.07 | 96.79 | 92.32 | 90.86 | 80.21 | 93.70 | 100.00 | 98.21 | 100.00 | 77.69 | 91.03 | 81.45 | 87.53 |
表3：LoCoMo 和 LongMemEval 按问题类型划分的准确率（%）。在 LongMemEval 中，S-U.、S-A. 和 S-P. 分别表示单会话用户、助手和偏好；M-S. 表示多会话；KU. 表示知识更新；TR. 表示时间推理。粗体表示排除 Full-context 后各列的最大值。
| Method | MEME | MemoryAgentBench | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ER. | Agg. | Tr. | Del. | Cas. | Abs. | Overall | AR. | TTL. | LRU. | CR. | Overall | |
| GPT-4o-mini | ||||||||||||
| Full-context | 93.00 | 10.00 | 44.00 | 16.00 | 3.66 | 9.23 | 29.31 | – | – | – | – | – |
| MemU | 52.00 | 6.00 | 31.00 | 19.00 | 1.22 | 10.00 | 19.87 | 52.30 | 9.12 | 22.54 | 20.50 | 26.12 |
| EverMemOS | 44.00 | 3.00 | 50.00 | 20.00 | 2.44 | 10.00 | 21.57 | 43.38 | 48.30 | 24.77 | 29.50 | 36.49 |
| MemOS | 78.00 | 19.00 | 49.00 | 21.00 | 3.66 | 10.77 | 30.24 | 52.03 | 45.73 | 33.13 | 12.00 | 35.72 |
| Mem0 | 70.00 | 5.00 | 61.00 | 14.00 | 7.32 | 9.23 | 27.76 | 57.90 | 46.75 | 20.47 | 40.00 | 41.28 |
| EMem | 48.00 | 36.00 | 63.00 | 37.00 | 6.71 | 12.31 | 33.84 | 57.60 | 49.07 | 17.11 | 30.00 | 38.44 |
| MemMachine | 98.00 | 9.00 | 73.00 | 29.00 | 4.88 | 8.46 | 37.06 | 58.12 | 25.12 | 31.38 | 27.50 | 35.53 |
| Memora | 93.00 | 11.00 | 58.00 | 23.00 | 6.10 | 10.00 | 33.52 | 68.78 | 47.77 | 26.82 | 32.00 | 43.84 |
| Raw-RAG | 97.00 | 19.00 | 78.00 | 26.00 | 4.27 | 11.54 | 39.30 | 67.85 | 47.68 | 31.74 | 46.00 | 48.32 |
| DeepSeek-V4-Flash | ||||||||||||
| Full-context | 97.00 | 59.00 | 88.00 | 63.00 | 48.17 | 23.85 | 63.17 | – | – | – | – | – |
| MemU | 50.00 | 9.00 | 23.00 | 17.00 | 3.05 | 10.77 | 18.80 | 62.12 | 7.13 | 22.82 | 21.00 | 28.27 |
| EverMemOS | 44.00 | 4.00 | 71.00 | 22.00 | 5.49 | 16.92 | 27.24 | 52.85 | 51.02 | 32.29 | 34.00 | 42.54 |
| MemOS | 77.00 | 29.00 | 33.00 | 24.00 | 6.71 | 17.69 | 31.23 | 56.68 | 49.27 | 32.44 | 11.50 | 37.47 |
| Mem0 | 85.00 | 13.00 | 57.00 | 22.00 | 4.27 | 17.69 | 33.16 | 62.92 | 48.70 | 21.19 | 35.00 | 41.95 |
| EMem | 48.00 | 36.00 | 87.00 | 38.00 | 6.10 | 15.38 | 38.41 | 63.62 | 49.72 | 16.53 | 28.50 | 39.59 |
| MemMachine | 100.00 | 11.00 | 85.00 | 48.00 | 7.32 | 16.92 | 44.71 | 70.08 | 27.02 | 44.09 | 27.00 | 42.05 |
| Memora | 92.00 | 2.00 | 57.00 | 38.00 | 9.15 | 20.00 | 36.36 | 75.05 | 50.72 | 31.76 | 34.00 | 47.88 |
| Raw-RAG | 100.00 | 21.00 | 87.00 | 29.00 | 3.66 | 18.46 | 43.19 | 75.78 | 50.39 | 33.48 | 54.00 | 53.41 |
表4：MEME 和 MemoryAgentBench 按子集划分的得分（%）。粗体表示针对各读取模型、排除 Full-context 后各列的最大值。Overall 为六项任务的宏平均（MEME），或四种能力宏平均的均值（MemoryAgentBench）。横线表示因上下文长度限制而无法获得 Full-context 的结果。
Tables 3 and 4 summarize the main results; all evaluation details are provided in Appendix A. Raw-RAG achieves the highest score among the compared memory systems in six of seven benchmark–model settings, excluding Full-context. It leads LoCoMo by 0.65–0.95 points, LongMemEval by 0.13 points, GPT-4o-mini MEME by 2.24 points, and MemoryAgentBench by 4.48–5.53 points. MemMachine leads DeepSeek-V4-Flash MEME among the memory systems by 1.52 points, while Full-context leads Raw-RAG by 19.98 points in that setting. Raw-RAG is competitive across these tasks despite its simpler ingestion procedure.
表3 和 表4 汇总了主要结果；全部评估细节见 附录 A。 排除 Full-context 后，Raw-RAG 在七种基准与模型组合中的六种上获得了所比较记忆系统中的最高分。 它在 LoCoMo 上领先 0.65–0.95 个百分点，在 LongMemEval 上领先 0.13 个百分点，在采用 GPT-4o-mini 的 MEME 上领先 2.24 个百分点，在 MemoryAgentBench 上领先 4.48–5.53 个百分点。 在采用 DeepSeek-V4-Flash 的 MEME 设置下，MemMachine 在记忆系统中以 1.52 个百分点的优势领先，而 Full-context 在该设置下比 Raw-RAG 高出 19.98 个百分点。 尽管摄取流程更简单，Raw-RAG 在这些任务上仍具有竞争力。
4.3. Ablation studies and efficiency
We organize the analysis around three research questions: why Raw-RAG uses its retrieval design, whether additional memory management improves accuracy, and how efficiently it ingests, organizes and retrieves long histories.
我们围绕三个研究问题展开分析：Raw-RAG 为何采用当前的检索设计，额外的记忆管理能否提高准确率，以及它摄取、组织和检索长历史的效率如何。
| Setting | AR. | TTL. | LRU. | CR. | Total | Δ | 95% CI |
|---|---|---|---|---|---|---|---|
| Raw-RAG | 67.85 | 47.68 | 31.74 | 46.00 | 48.32 | – | – |
| − reranker | 59.13 | 44.29 | 29.66 | 44.50 | 44.40 | −3.92 | [-5.80, -2.06] |
| 128/16 | 70.50 | 46.89 | 28.93 | 46.00 | 48.08 | −0.24 | [-1.54, +1.05] |
| 512/64 | 65.68 | 49.27 | 32.49 | 46.00 | 48.36 | +0.04 | [-1.15, +1.25] |
| 1024/128 | 66.28 | 48.08 | 31.79 | 46.00 | 48.04 | −0.28 | [-1.76, +1.19] |
| + display-turn dedup | 68.27 | 37.69 | 31.75 | 46.00 | 45.93 | −2.39 | [-3.53, -1.28] |
| + Atomic facts | 57.25 | 15.37 | 21.17 | 21.00 | 28.70 | −19.62 | [-21.99, -17.21] |
| + Raw summary | 67.53 | 45.89 | 29.62 | 27.00 | 42.51 | −5.81 | [-7.70, -3.90] |
| + Graph-expanded | 65.42 | 46.49 | 31.74 | 26.50 | 42.54 | −5.78 | [-7.50, -4.07] |
表5：Raw-RAG 在 MemoryAgentBench 上的消融实验。灰色表示默认配置，蓝色表示检索变体，橙色表示额外的记忆管理操作。a/b 表示以 token 为单位的分块大小与重叠长度。
| System | Add→ready | Search | Total |
|---|---|---|---|
| EverMemOS | 8217.69 | 1083.79 | 9301.48 |
| Mem0 | 4025.23 | 117.72 | 4142.95 |
| MemMachine | 3104.35 | 411.39 | 3515.74 |
| MemOS | 2591.59 | 177.97 | 2769.56 |
| EMem | 1082.37 | 479.59 | 1561.97 |
| Raw-RAG | 50.32 | 255.59 | 305.91 |
表6：在 10 个 MEME 片段和 110 个问题上的摄取与检索延迟（秒）。Total 为从添加到就绪（Add-to-ready）的时间与搜索（Search）时间之和，不包括答案生成和评估。
图4：最终 top-k 的敏感性：MemoryAgentBench 上的 Total 得分及其相对默认 k = 15 的变化，带有配对的 95% 区间。
RQ1: Why is Raw-RAG designed this way? We first test the roles of reranking, retrieval depth, and chunk size. Table 5 and Figure 4 isolate these choices on MemoryAgentBench using 20,000 paired, source-stratified bootstrap resamples and Holm correction. Removing reranking lowers Total by 3.92 points (
RQ1：为什么这样设计 Raw-RAG？ 我们首先测试重排序、检索深度和分块大小的作用。 表5 和 图4 在 MemoryAgentBench 上分别考察这些选择，采用 20,000 次按来源分层的配对自助重采样，并进行 Holm 校正。 去掉重排序后，Total 降低 3.92 个百分点（
Final-
当最终返回数量
RQ2: Can additional memory management improve Raw-RAG? We evaluate three memory-management operations: atomic-fact extraction, summary augmentation, and graph-based retrieval expansion. These variants reuse the chunking, embedding model, and reranker, while changing the indexed representation or retrieval candidate set. None improves Raw-RAG (48.32%) on the same 14-source MemoryAgentBench Total (Table 5, orange rows). The exact construction and retrieval procedures are given in the representation and retrieval ablation appendix (Appendix C.3), with the extraction prompts in the prompt appendix (Appendix G.2).
Atomic-fact extraction reduces test-time learning from 47.68% to 15.37% and Total by 19.62 points, consistent with loss or disruption of usable demonstrations (Appendix F). Summary augmentation largely preserves Accurate Retrieval performance but loses 5.81 points overall, chiefly on conflict resolution. Graph expansion ties Raw-RAG on long-range understanding yet remains 5.78 points lower overall. Thus, these three memory-management additions reduce overall accuracy in this setting.
原子事实抽取使测试时学习得分从 47.68% 降至 15.37%，并使 Total 降低 19.62 个百分点，这与可用示例的丢失或破坏相一致（附录 F）。 摘要增强大体保留了准确检索能力，但总体得分降低 5.81 个百分点，主要下降发生在冲突消解上。 图扩展在长程理解上与 Raw-RAG 持平，但总体得分仍低 5.78 个百分点。 因此，在这一设置下，这三种额外的记忆管理操作都降低了总体准确率。
RQ3: How efficient is Raw-RAG? We finally examine the computational cost of these additions and compare system-level latency and scaling. On the 11 commonly measured non-conflict MemoryAgentBench sources, the three additions require 21.6–109.3M LLM tokens and 2.9–8.8 hours for Add+Search, versus 12 minutes for Raw-RAG (Table 32). We then compare systems on the same 10-episode, 110-question MEME slice, using one external conversation/search worker and each system's native internal parallelism. Raw-RAG finishes Add-to-ready plus Search in 305.91 seconds (Table 6), 5.1× faster than the next system and 30.4× faster than EverMemOS.
RQ3：Raw-RAG 的效率如何？ 最后，我们考察这些额外操作的计算成本，并比较系统层面的延迟与扩展性。 在各方法共同测量的 11 个非冲突类 MemoryAgentBench 来源上，这三种额外操作的 Add+Search 需要 2,160 万至 1.093 亿个 LLM token，以及 2.9–8.8 小时，而 Raw-RAG 仅需 12 分钟（表32）。 随后，我们在同一份包含 10 个片段、110 个问题的 MEME 子集上比较各系统，使用一个外部对话／搜索工作单元，并保留各系统原生的内部并行方式。 Raw-RAG 完成 Add-to-ready 加 Search 共需 305.91 秒（表6），速度为次快系统的 5.1 倍、EverMemOS 的 30.4 倍。
For scaling, we stream the same 1,710-message BEAM-1M conversation approaching one million tokens into each system with one external worker (Figure 1, right panel). At the endpoint, cumulative Add time ranges from 0.33 hours for Raw-RAG to 26.95 hours for Memora; the next-fastest completed run, MemOS, takes 3.25 hours. Both studies exclude answer generation and evaluation; BEAM-1M includes six completed sequential runs. Appendix D details execution conditions. On these workloads, Raw-RAG combines competitive accuracy with lower ingestion and combined ingestion–retrieval costs.
为考察扩展性，我们使用一个外部工作单元，将同一段包含 1,710 条消息、接近一百万个 token 的 BEAM-1M 对话以流式方式输入各系统（图1 右侧面板）。 在终点处，累计 Add 时间从 Raw-RAG 的 0.33 小时到 Memora 的 26.95 小时不等；在已完成的运行中，次快的 MemOS 用时 3.25 小时。 这两项研究都不包含答案生成和评估；BEAM-1M 包含六次已完成的顺序运行。 附录 D 详细说明了执行条件。 在这些工作负载上，Raw-RAG 同时具备有竞争力的准确率、较低的摄取成本，以及较低的摄取与检索总成本。
5. Why Is Raw-RAG Enough in Most Cases?
5.1. Raw-RAG offsets lower utilization with high coverage
| System | Coverage | Utilization |
|---|---|---|
| MemU | 19.07 | 89.37 |
| MemMachine | 21.54 | 84.30 |
| EverMemOS | 25.11 | 80.00 |
| MemOS | 34.48 | 85.85 |
| EMem | 34.72 | 89.71 |
| Mem0 | 40.03 | 82.39 |
| Raw-RAG | 45.29 | 84.43 |
表7：MemoryAgentBench 上的语义证据覆盖率与读取模型在证据充分条件下的利用率（%）。
| System | Coverage | Utilization |
|---|---|---|
| MemU | 35.69 | 84.20 |
| EverMemOS | 47.64 | 76.86 |
| EMem | 50.08 | 86.55 |
| MemOS | 62.12 | 66.67 |
| Mem0 | 63.22 | 65.51 |
| MemMachine | 70.37 | 82.66 |
| Raw-RAG | 76.35 | 73.76 |
表8：通过确定性检查得到的 MEME 证据覆盖率与利用率（%）。
Does Raw-RAG perform well because it supplies better evidence, or because readers use that evidence more effectively? This distinction matters: improving the reader cannot reliably compensate for missing evidence, while broader retrieval may not resolve reasoning errors.
Raw-RAG 表现良好，是因为它提供了更好的证据，还是因为读取模型能更有效地利用这些证据？ 这一区别很关键：改进读取模型无法可靠地弥补证据缺失，而扩大检索范围也未必能消除推理错误。
We measure coverage, the fraction of questions with sufficient retrieved evidence, and utilization, answer accuracy conditional on sufficient evidence. Coverage is assessed by an LLM judge on MemoryAgentBench and task-specific deterministic checks on MEME (Appendix C.1). Conditioning on coverage separates evidence availability from subsequent answer generation, although the system-specific question sets preclude a controlled comparison of readers.
我们衡量两个指标：覆盖率，即检索到充分证据的问题占比；以及 利用率，即以证据充分为条件的答案准确率。 在 MemoryAgentBench 上，覆盖率由 LLM 评判模型评估；在 MEME 上，则通过针对各任务的确定性检查来评估（附录 C.1）。 以证据覆盖为条件，可以将证据可用性与后续答案生成区分开来，但由于各系统对应的问题集合不同，无法据此对读取模型进行受控比较。
Tables 7 and 8 show that Raw-RAG achieves the highest coverage on both benchmarks: 45.29% and 76.35%. On MemoryAgentBench, coverage spans 19.07–45.29%, whereas utilization spans only 80.00–89.71%. MEME shows the same pattern: coverage varies by 40.66 points versus 21.04 for utilization. Thus, systems differ more in supplying sufficient evidence than in answering correctly once it is supplied. Raw-RAG leads in coverage, helping offset its less-than-best conditional utilization and explaining its strong performance.
表7 和 表8 表明，Raw-RAG 在两个基准上都取得了最高覆盖率，分别为 45.29% 和 76.35%。 在 MemoryAgentBench 上，覆盖率范围为 19.07–45.29%，而利用率仅在 80.00–89.71% 之间变化。 MEME 呈现相同模式：覆盖率相差 40.66 个百分点，而利用率相差 21.04 个百分点。 因此，相比在证据已被提供后能否正确作答，各系统在提供充分证据的能力上差异更大。 Raw-RAG 在覆盖率上领先，有助于弥补其并非最优的条件利用率，也解释了其较强的性能。
Evidence access is a major bottleneck: Raw-RAG supplies sufficient evidence more often, helping offset lower conditional reader utilization.
证据访问是一个主要瓶颈：Raw-RAG 更经常能够提供充分证据，从而有助于弥补读取模型在证据充分条件下较低的利用率。
5.2. Raw history preserves the paths needed for reasoning
Raw history can preserve not only answer values but also the information needed to derive them. Useful retrieval therefore includes the order and dependencies that make one value current.
原始历史不仅可以保留答案值，还可以保留推导这些值所需的信息。 因此，有用的检索结果还应包含使某个值成为当前有效值的顺序信息和依赖关系。
On MEME tracking and cascade after-state questions, an LLM jointly assesses result coverage (required answer values) and path coverage (those values plus chronological order or linked dependency rules and upstream triggers; Appendix G.10). Table 9 reports
对于 MEME 追踪和级联任务中询问变化后状态的问题，LLM 联合评估 结果覆盖率（所需的答案值）与 路径覆盖率（这些值以及时间顺序，或相互关联的依赖规则和上游触发条件；附录 G.10）。 表9 报告了
Raw-RAG achieves the highest conditional path coverage (62.34%), followed by MemMachine (56.70%); MemU reaches only 9.62%. Systems that retrieve answer values can still differ substantially in preserving their supporting relations.
Raw-RAG 获得了最高的条件路径覆盖率（62.34%），其次是 MemMachine（56.70%）；MemU 仅达到 9.62%。 能够检索到答案值的系统，在保留这些值的支撑关系方面仍可能存在显著差异。
Raw-RAG is most useful when it preserves both answer values and the temporal or dependency relations needed to derive them.
当 Raw-RAG 同时保留答案值，以及推导这些值所需的时间关系或依赖关系时，它最有帮助。
| System | Result | Path | P(path | result) |
|---|---|---|---|
| MemU | 59.09 | 5.68 | 9.62 |
| MemOS | 88.26 | 21.21 | 24.03 |
| Mem0 | 81.06 | 38.64 | 47.66 |
| EMem | 76.52 | 37.88 | 49.50 |
| EverMemOS | 78.03 | 39.02 | 50.00 |
| MemMachine | 84.85 | 48.11 | 56.70 |
| Raw-RAG | 90.53 | 56.44 | 62.34 |
表9：LLM 对 MEME 追踪与级联任务中变化后状态问题判定的结果覆盖率和可执行路径覆盖率（%）。最后一列给出以结果被覆盖为条件的路径覆盖率。
图5：GPT-4o-mini 在 MEME 上的结果：原生检索与完整存储访问的比较，以及相对原始证据的差距。
5.3. Preserving raw evidence supports downstream recovery
The preceding analyses identify missing evidence and incomplete dependency paths in retrieved contexts, but do not establish where these gaps arise. Relevant information may remain in memory but escape retrieval, or it may have been discarded or distorted by ingestion-time processing, such as replacing source passages with extracted facts or compressed summaries. We therefore ask: can improved access to the stored memories recover performance, or do these ingestion-time operations leave information gaps that later retrieval cannot repair?
前述分析发现检索上下文中存在证据缺失和依赖路径不完整的问题，但尚未确定这些缺口出现在哪个环节。 相关信息可能仍保存在记忆中，却未被检索到；也可能在摄取阶段的处理中被丢弃或扭曲，例如将原始段落替换为抽取的事实或压缩的摘要。 因此，我们提出问题：改善对已存储记忆的访问能否恢复性能，还是这些摄取阶段的操作留下了后续检索无法修复的信息缺口？
We compare three conditions on MEME. Native uses each system's ordinary retrieval and readout. The complete-store oracle uses gold-informed selection over all exported phase-visible memories. The raw-evidence control selects original annotated support sessions using the same gold information. Both controls share a value-free task policy and a 5,000-token packing budget.
我们在 MEME 上比较三种条件。 原生条件（Native） 使用各系统常规的检索和读取方式。 完整存储 oracle（complete-store oracle） 在导出的全部阶段可见记忆上，借助标准答案信息进行选择。 原始证据对照（raw-evidence control） 使用相同的标准答案信息，选择原始标注的支撑会话。 这两种对照采用相同的不含答案值的任务策略，以及 5,000 token 的证据打包预算。
Figure 5 shows that the complete-store oracle improves all seven systems, but the recovered performance varies across stored representations. Raw-RAG and MemMachine, which retain raw conversational content in their evaluated stores, reach 57.54% and 57.46%, respectively, compared with 61.60% for raw evidence. These results are consistent with a role for information preservation during ingestion in downstream recovery. The two source-preserving systems recover performance closest to the raw-evidence control, whereas access to the complete transformed stores leaves substantially larger deficits.
图5 表明，完整存储 oracle 改善了全部七个系统的表现，但不同存储表示所恢复的性能各不相同。 Raw-RAG 和 MemMachine 在所评估的存储中保留了原始对话内容，分别达到 57.54% 和 57.46%，而原始证据达到 61.60%。 这些结果与摄取阶段的信息保留有助于下游性能恢复这一判断相一致。 这两个保留源内容的系统所恢复的性能最接近原始证据对照，而完整访问经过转换的存储仍留下大得多的差距。
Removing the packing ceiling leaves all seven stores 7.58–29.38 points below raw evidence, with all paired 95% intervals for these gaps excluding zero. However, the residual gap is not a pure measure of information loss during ingestion: even Raw-RAG falls short, and evidence assembly and positional effects such as lost in the middle may also contribute. Appendix C.5 specifies the controls; Appendix A.5.2 reports the full results.
Preserving information during ingestion matters for downstream recovery: information discarded or distorted during ingestion is difficult to recover through retrieval alone.
在摄取阶段保留信息对下游性能恢复很重要：摄取时被丢弃或扭曲的信息，很难仅靠检索来恢复。
6. Limitations and Conclusion
6.1. Limitations and future work
Our evidence is limited to offline conversation on public benchmarks; it does not establish performance for interactive, multimodal, or broader operational settings. More discussions are in Appendix F; raw-history approaches also raise storage and privacy concerns. Future work should evaluate a broader range of models on longer histories while accounting for explicit privacy and deletion constraints. It should also identify when structured or compressed memories offer advantages over raw retrieval, considering trade-offs among accuracy, latency and computational cost. State reconstruction also depends on reader capability. Future evaluations should test whether explicit state maintenance benefits weaker readers or reduces repeated reasoning across queries.
我们的证据仅限于公共基准上的离线对话，无法据此确定交互式、多模态或更广泛实际运行场景中的性能。 更多讨论见 附录 F；原始历史方法也会带来存储和隐私方面的问题。 未来工作应在更长的历史上评估更广泛的模型，同时考虑明确的隐私与删除约束。 还应在权衡准确率、延迟和计算成本的基础上，确定结构化或压缩记忆在何种情况下比原始历史检索更有优势。 状态重建也依赖读取模型的能力。 未来评估应检验显式状态维护能否使能力较弱的读取模型受益，或减少跨查询的重复推理。
6.2. Conclusion
Is agentic memory management always needed for long-term conversation understanding? Across four benchmarks, Raw-RAG, a naive baseline using dense retrieval and reranking over raw history, ranks first among the compared memory systems in six of seven benchmark–model settings. Our diagnostic analyses identify evidence access as the larger observed bottleneck: useful evidence includes not only answer values but also supporting relations and dependency paths. Even complete access to transformed stores does not close the gap to raw evidence. These findings challenge the need for complex agentic memory systems in the evaluated conversation tasks. Improved agentic memory systems should demonstrate gains over a well-specified naive RAG baseline and establish what the added complexity contributes.
长期对话理解总是需要智能体记忆管理吗？ 在四个基准上，Raw-RAG 这一基于原始历史进行稠密检索和重排序的朴素基线，在七种基准与模型组合中的六种上位居所比较记忆系统的首位。 我们的诊断分析发现，证据访问是所观察到的更大瓶颈：有用的证据不仅包括答案值，还包括支撑关系与依赖路径。 即使能够完整访问经过转换的存储，也无法消除与原始证据之间的差距。 这些发现对所评估对话任务中复杂智能体记忆系统的必要性提出了质疑。 改进后的智能体记忆系统应展示其相对定义清晰的朴素 RAG 基线的收益，并明确增加的复杂性究竟带来了什么贡献。