---

Agentic Memory: Learning Unified Long-Term and Short-Term Memory Management for Large Language Model Agents

MemoryAgentRLGRPO10+50+武汉大学阿里巴巴

Yu Y, Yao L, Xie Y, et al. Agentic Memory: Learning Unified Long-Term and Short-Term Memory Management for Large Language Model Agents[J]. arXiv preprint arXiv:2601.01885v2, 2026.

https://github.com/y1y5/AgeMem

---

智能体记忆：学习面向大语言模型智能体的统一长期与短期记忆管理

Abstract

Large language model (LLM) agents face fundamental limitations in long-horizon reasoning due to finite context windows, making effective memory management critical. Existing methods typically handle long-term memory (LTM) and short-term memory (STM) as separate components, relying on heuristics or auxiliary controllers, which limits adaptability and end-to-end optimization. In this paper, we propose Agentic Memory (AgeMem), a unified framework that integrates LTM and STM management directly into the agent's policy. AgeMem exposes memory operations as tool-based actions, enabling the LLM agent to autonomously decide what and when to store, retrieve, update, summarize, or discard information. To train such unified behaviors, we propose a three-stage progressive reinforcement learning strategy and design a step-wise GRPO to address sparse and discontinuous rewards induced by memory operations. Experiments on five long-horizon benchmarks demonstrate that AgeMem consistently outperforms strong memory-augmented baselines across multiple LLM backbones, achieving improved task performance, higher-quality long-term memory, and more efficient context usage.

由于上下文窗口有限，大语言模型（LLM）智能体在长程推理中面临根本限制，因此有效的记忆管理至关重要。 现有方法通常把长期记忆（LTM）和短期记忆（STM）作为分离组件处理，并依赖启发式规则或辅助控制器，这限制了适应性和端到端优化。 在本文中，我们提出 Agentic Memory（AgeMem），这是一个把 LTM 和 STM 管理直接整合进智能体策略的统一框架。 AgeMem 将记忆操作暴露为基于工具的动作，使 LLM 智能体能够自主决定存储、检索、更新、总结或丢弃哪些信息以及何时执行这些操作。 为了训练这种统一行为，我们提出三阶段渐进式强化学习策略，并设计逐步 GRPO，以处理记忆操作引起的稀疏且不连续的奖励。 在五个长程基准上的实验表明，AgeMem 在多个 LLM backbone 上持续优于强记忆增强基线，实现了更好的任务性能、更高质量的长期记忆以及更高效的上下文使用。

1. Introduction

In long-horizon agentic tasks involving multi-step reasoning and complex workflows, the effectiveness of large language model (LLM) agents is fundamentally constrained by the information they can attend to at any given time, which we collectively refer to as the agent's memory. Memory typically falls into two categories: long-term memory (LTM), which persistently stores user- or task-specific knowledge, and short-term memory (STM), which comprises the information contained in the current input context. High-quality LTM supports efficient retrieval of accumulated knowledge, while effective STM management reduces redundancy and preserves salient context. Together, they mitigate the limitations of finite context windows, making their joint management crucial for improving agent performance in complex reasoning settings.

在涉及多步推理和复杂工作流的长程智能体任务中，大语言模型（LLM）智能体的有效性从根本上受限于它们在任一时刻能够关注的信息，我们将这些信息统称为智能体的记忆。 记忆通常分为两类：长期记忆（LTM）持久存储用户或任务特定知识，短期记忆（STM）则由当前输入上下文中包含的信息构成。 高质量 LTM 支持对累积知识的高效检索，而有效的 STM 管理会减少冗余并保留关键上下文。 二者共同缓解有限上下文窗口的限制，因此它们的联合管理对于提升复杂推理设置中的智能体性能至关重要。

图1：独立记忆管理框架与统一记忆管理框架的比较。左：带静态 STM 和触发式 LTM 的传统框架。中：带有额外 Memory Manager 的独立框架，该管理器以智能体方式控制 LTM，而 STM 仍保持静态。右：提出的 AgeMem 框架，其中 LTM 和 STM 通过显式的基于工具的操作被联合且智能地管理。

However, existing research has predominantly treated LTM and STM as independent components. STM is commonly enhanced through retrieval-augmented generation (RAG), such as in MainRAG and ReSum, which expand usable context via external retrieval or periodic summarization. Although effective in some tasks, these methods rely heavily on predefined schedules or heuristic rules, potentially causing infrequent but critical details to be overlooked, while introducing unnecessary noise. In contrast, LTM management has progressed along separate lines, typically categorized into trigger-based and agent-based paradigms. The former executes fixed memory operations at predefined moments, whereas the latter incorporates a specialized memory manager to determine what and how to store. Despite offering more flexibility, most approaches still depend on handcrafted rules or auxiliary expert models, limiting adaptability and increasing system complexity.

然而，现有研究主要把 LTM 和 STM 当作独立组件处理。 STM 通常通过检索增强生成（RAG）来增强，例如 MainRAG 和 ReSum，它们通过外部检索或周期性总结来扩展可用上下文。 虽然这些方法在一些任务中有效，但它们高度依赖预定义调度或启发式规则，可能导致低频但关键的细节被忽略，同时引入不必要的噪声。 相比之下，LTM 管理沿着另一条路线发展，通常被归类为触发式和智能体式两种范式。 前者在预定义时刻执行固定记忆操作，而后者引入专门的记忆管理器来决定存什么以及如何存。 尽管后者提供了更高灵活性，大多数方法仍依赖人工规则或辅助专家模型，限制了适应性并增加了系统复杂度。

As a consequence, LTM and STM are typically treated as separate and loosely coupled modules. As illustrated in Figure 1, existing architectures generally follow two patterns: (a) static STM with trigger-based LTM, or (b) static STM with agent-based LTM. In both settings, the two memory systems are optimized independently and later combined in an ad hoc way, leading to fragmented memory construction and suboptimal performance in long-horizon reasoning tasks. Thus, unifying the management of LTM and STM remains a necessary yet largely unexplored challenge.

因此，LTM 和 STM 通常被视为分离且松散耦合的模块。 如图1所示，现有架构通常遵循两种模式：(a) 静态 STM 搭配触发式 LTM，或 (b) 静态 STM 搭配智能体式 LTM。 在这两种设置中，两套记忆系统都被独立优化，随后以临时方式组合起来，导致记忆构建碎片化，并使长程推理任务中的表现次优。 因此，统一管理 LTM 和 STM 仍是必要但很大程度上尚未探索的挑战。

Nevertheless, achieving unified memory management poses three fundamental challenges. (C1) Functional heterogeneity coordination: LTM and STM serve distinct yet complementary purposes: LTM determines what to store, update, or discard, while STM governs what to retrieve, summarize, or remove from the active context. The challenge lies in designing a unified mechanism that orchestrates their interplay synergistically. (C2) Training paradigm mismatch: Existing reinforcement learning (RL) frameworks adopt markedly different training strategies for the two memory types. LTM-focused training often leverages session-level information available prior to interaction, whereas STM training typically injects distractors to simulate long-horizon contexts. Moreover, standard RL assumes continuous trajectories with stable rewards, which conflicts with the inherently fragmented and discontinuous experiences produced by memory operations, making end-to-end optimization particularly challenging. (C3) Practical deployment constraints: Many agent systems rely on an auxiliary expert LLM for memory control, significantly increasing inference cost and training complexity. How to integrate unified memory management directly into an agent without dependence on external expert models remains an open problem.

不过，实现统一记忆管理会带来三个根本挑战。 (C1) 功能异质性协调： LTM 和 STM 承担不同但互补的目的：LTM 决定存储、更新或丢弃什么，而 STM 控制从活动上下文中检索、总结或移除什么。 挑战在于设计一种统一机制，以协同方式编排二者的相互作用。 (C2) 训练范式不匹配： 现有强化学习（RL）框架对这两类记忆采用明显不同的训练策略。 面向 LTM 的训练通常利用交互前可用的会话级信息，而 STM 训练通常注入干扰项来模拟长程上下文。 此外，标准 RL 假设轨迹连续且奖励稳定，这与记忆操作产生的天然碎片化、不连续经验相冲突，使端到端优化尤其具有挑战性。 (C3) 实际部署约束： 许多智能体系统依赖辅助专家 LLM 进行记忆控制，显著增加推理成本和训练复杂度。 如何在不依赖外部专家模型的情况下，把统一记忆管理直接整合进智能体，仍是一个开放问题。

To address these challenges, we propose Agentic Memory (AgeMem), a unified framework that jointly manages LTM and STM, illustrated in Figure 1 (right). Unlike prior designs that treat memory as an external component, AgeMem integrates both memory types directly into the agent's decision-making process. Through a unified tool-based interface, the LLM autonomously invokes and executes memory operations for both LTM and STM. Furthermore, we design a three-stage progressive RL strategy: the model first acquires LTM storage capabilities, then learns STM context management, and finally coordinates both forms of memory under full task settings. To address the fragmented experience issue across training stages, we design a step-wise Group Relative Policy Optimization (GRPO), which propagates output rewards back to prior memory decisions, thereby alleviating the challenges posed by sparse and discontinuous rewards in RL. We evaluate AgeMem on five long-context, reasoning-intensive benchmarks. Comprehensive results show that AgeMem consistently outperforms strong baselines, validating the effectiveness of unified memory management.

为了解决这些挑战，我们提出 Agentic Memory（AgeMem），这是一个联合管理 LTM 和 STM 的统一框架，如图1右侧所示。 不同于把记忆视为外部组件的以往设计，AgeMem 将两类记忆直接整合进智能体的决策过程。 通过统一的基于工具的接口，LLM 可以自主调用并执行针对 LTM 和 STM 的记忆操作。 此外，我们设计三阶段渐进式 RL 策略：模型首先获得 LTM 存储能力，然后学习 STM 上下文管理，最后在完整任务设置下协调两种形式的记忆。 为了解决训练阶段之间的碎片化经验问题，我们设计逐步 Group Relative Policy Optimization（GRPO），它把输出奖励回传到先前的记忆决策，从而缓解 RL 中稀疏且不连续奖励带来的挑战。 我们在五个长上下文、推理密集型基准上评估 AgeMem。 综合结果表明，AgeMem 持续优于强基线，验证了统一记忆管理的有效性。

Our main contributions are as follows:

我们的主要贡献如下：

• We propose Agentic Memory (AgeMem), a unified agentic memory framework that enables LLM-based agents to autonomously decide when, what, and how to manage both long-term and short-term memory.
• We develop a three-stage progressive RL strategy equipped with a step-wise GRPO mechanism, facilitating effective end-to-end learning of unified memory management behaviors.
• We conduct comprehensive evaluations across multiple models and long-horizon benchmarks, demonstrating the robustness and effectiveness of AgeMem in complex agentic tasks.

• 我们提出 Agentic Memory（AgeMem），这是一个统一的智能体记忆框架，使基于 LLM 的智能体能够自主决定何时、管理什么以及如何同时管理长期记忆和短期记忆。
• 我们开发了一种配备逐步 GRPO 机制的三阶段渐进式 RL 策略，促进统一记忆管理行为的有效端到端学习。
• 我们在多个模型和长程基准上进行了综合评估，证明 AgeMem 在复杂智能体任务中的鲁棒性和有效性。

2. Background and Related Work

Long-term memory (LTM). Persistent LTM is crucial for LLM-based agents operating over extended horizons. Recent work has explored diverse architectural designs for modeling LTM. LangMem provides a modular framework that supports multiple memory types, while A-Mem adopts a Zettelkasten-inspired design that links structured knowledge units to facilitate consolidation. Mem0 proposes a scalable extract-update pipeline and extends it to a graph-based variant for structured reasoning. Zep represents memory as a temporal knowledge graph to enable cross-session and time-aware reasoning. Although effective in organizing and retrieving information, these approaches largely rely on predefined memory structures or heuristic update rules. As memory grows, such designs commonly suffer from increased system complexity and lack adaptive, learning-based strategies for prioritization and forgetting. In contrast, our work aims to learn an adaptive memory policy that allows agents to dynamically decide what to store, update, or forget, depending on task demands and long-term utility.

长期记忆（LTM）。 持久 LTM 对于在扩展时间跨度上运行的基于 LLM 的智能体至关重要。 近期工作探索了用于建模 LTM 的多种架构设计。 LangMem 提供了支持多种记忆类型的模块化框架，而 A-Mem 采用受 Zettelkasten 启发的设计，将结构化知识单元链接起来以促进整合。 Mem0 提出可扩展的抽取-更新流水线，并把它扩展为用于结构化推理的图变体。 Zep 将记忆表示为时间知识图，以支持跨会话和时间感知推理。 虽然这些方法在组织和检索信息方面有效，但它们很大程度上依赖预定义记忆结构或启发式更新规则。 随着记忆增长，这类设计通常会遭遇系统复杂度增加的问题，并缺少用于优先级排序和遗忘的自适应、基于学习的策略。 相比之下，我们的工作旨在学习一种自适应记忆策略，使智能体能够根据任务需求和长期效用动态决定存储、更新或遗忘什么。

Short-term memory (STM). STM in agentic LLMs primarily concerns context selection and retrieval. Retrieval-Augmented Generation (RAG) is the dominant paradigm, expanding usable context by injecting retrieved content into prompts. While effective, RAG does not fundamentally prevent context explosion in long-horizon settings and may introduce irrelevant or distracting information. To address this issue, ReSum periodically compresses interaction histories into compact reasoning states, allowing agents to operate beyond fixed context-window constraints. Yet its summarization schedule remains largely predefined, and aggressive compression risks discarding rare but crucial details. Our approach instead enables agents to learn when and how to retrieve, summarize, or filter context, achieving a more flexible balance between efficiency and information preservation.

短期记忆（STM）。 智能体式 LLM 中的 STM 主要涉及上下文选择和检索。 检索增强生成（RAG）是主导范式，它通过把检索内容注入提示来扩展可用上下文。 虽然有效，但 RAG 并不能从根本上防止长程设置中的上下文爆炸，并且可能引入无关或干扰信息。 为解决这一问题，ReSum 会周期性地将交互历史压缩为紧凑推理状态，使智能体能够超越固定上下文窗口约束运行。 然而，它的总结调度仍很大程度上是预定义的，而且激进压缩存在丢弃低频但关键细节的风险。 我们的方法则使智能体能够学习何时以及如何检索、总结或过滤上下文，从而在效率和信息保留之间实现更灵活的平衡。

Reinforcement learning for LLMs. Reinforcement learning has become an effective paradigm for improving the decision-making and reasoning capabilities of LLM-based agents. Among recent advances, GRPO enhances stability by optimizing policies based on the relative quality of sampled trajectories, removing the need for an explicit value function. GRPO and its variants have shown strong performance in complex reasoning tasks. However, existing RL-based systems generally treat memory as a static or external component, making them ill-suited for the discontinuous and fragmented trajectories associated with memory operations. In contrast, our work integrates RL directly into the memory management process, enabling unified training of both language generation and memory operations.

面向 LLM 的强化学习。 强化学习已经成为提升基于 LLM 的智能体决策和推理能力的有效范式。 在近期进展中，GRPO 通过基于采样轨迹的相对质量优化策略来增强稳定性，并移除了对显式价值函数的需求。 GRPO 及其变体已经在复杂推理任务中展现出强劲性能。 然而，现有基于 RL 的系统通常把记忆视为静态或外部组件，使其不适合处理与记忆操作相关的不连续、碎片化轨迹。 相比之下，我们的工作把 RL 直接整合进记忆管理过程，从而支持语言生成和记忆操作的统一训练。

Positioning relative to RL-based memory agents. Recent work models memory operations as actions and applies RL to optimize them, which shares surface similarity with our approach. However, these methods typically optimize one aspect of memory at a time while treating retrieval, summarization, or short-term context handling as fixed heuristics or separately tuned modules. As a result, early storage decisions and later reasoning behavior are only loosely coupled, and the learning signal does not explicitly connect them. AgeMem instead formulates memory usage as a single learnable control problem under delayed supervision: a unified policy over heterogeneous memory actions (both persistent LTM operations and contextual STM operations) is trained end-to-end, so that storage, retrieval, filtering, and summarization decisions are all optimized jointly with respect to the same terminal task reward.

与基于 RL 的记忆智能体的相对定位。 近期工作把记忆操作建模为动作并应用 RL 来优化它们，这与我们的方法在表面上相似。 然而，这些方法通常一次只优化记忆的一个方面，同时把检索、总结或短期上下文处理视为固定启发式或单独调优的模块。 因此，早期存储决策与后续推理行为只是松散耦合，学习信号并没有显式连接二者。 AgeMem 则把记忆使用表述为延迟监督下的单一可学习控制问题：对异质记忆动作（包括持久 LTM 操作和上下文 STM 操作）的统一策略进行端到端训练，使存储、检索、过滤和总结决策都相对于同一个终端任务奖励被联合优化。

3. Method

We propose Agentic Memory (AgeMem), a unified memory framework that enables LLM agents to autonomously manage both LTM and STM in an end-to-end manner. As illustrated in Figure 1 (right), AgeMem integrates memory management capabilities directly into the agent via a set of specialized tools, enabling the model to learn optimal strategies for unified memory management through a three-stage progressive strategy.

我们提出 Agentic Memory（AgeMem），这是一个统一记忆框架，使 LLM 智能体能够以端到端方式自主管理 LTM 和 STM。 如图1右侧所示，AgeMem 通过一组专门工具把记忆管理能力直接整合进智能体，使模型能够通过三阶段渐进式策略学习统一记忆管理的最优策略。

3.1 Problem Formulation

Unified RL formulation for AgeMem. At each time step $t$, the agent observes a state $s_t\in \mathcal{S}$ composed of the conversation context (short-term memory) $C_t$, the long-term memory store ${\mathcal{M}}_t$, and the task specification $\mathcal{T}$: $s_t=(C_t,{\mathcal{M}}_t,\mathcal{T})$. The specification $\mathcal{T}$ includes the input query $q$, contextual information $I_q$, and (for training only) the expected answer $A_q$. This formulation enables the agent to ground its decision-making in both transient context and persistent knowledge.

AgeMem 的统一 RL 形式化。 在每个时间步 $t$，智能体观测到状态 $s_t\in \mathcal{S}$，它由对话上下文（短期记忆）$C_t$、长期记忆存储 ${\mathcal{M}}_t$ 和任务规范 $\mathcal{T}$ 组成：$s_t=(C_t,{\mathcal{M}}_t,\mathcal{T})$。 规范 $\mathcal{T}$ 包括输入查询 $q$、上下文信息 $I_q$，以及（仅训练时使用的）期望答案 $A_q$。 这种形式化使智能体能够把其决策建立在瞬时上下文和持久知识二者之上。

Given $s_t$, the agent selects an action $a_t\in \mathcal{A}$ from a hybrid action space that includes language generation as well as memory operations. The decision is governed by a parameterized policy ${\pi }_{\theta }$, defined as ${\pi }_{\theta }(a_t|s_t)=P(a_t|s_t;\theta )$, where $\theta$ denotes the LLM parameters and $a_t\sim {\pi }_{\theta }(\cdot |s_t)$. For a trajectory $\tau =(s_1,a_1,\dots ,s_T,a_T)$, the cumulative reward is defined as:

给定 $s_t$，智能体会从混合动作空间中选择动作 $a_t\in \mathcal{A}$，该动作空间既包含语言生成，也包含记忆操作。 这一决策由参数化策略 ${\pi }_{\theta }$ 控制，定义为 ${\pi }_{\theta }(a_t|s_t)=P(a_t|s_t;\theta )$，其中 $\theta$ 表示 LLM 参数，且 $a_t\sim {\pi }_{\theta }(\cdot |s_t)$。 对于轨迹 $\tau =(s_1,a_1,\dots ,s_T,a_T)$，累计奖励定义为：

$$R(\tau )=\sum _i{w}_i\cdot R_i(\tau )+P_{\text{penalty}}(\tau ),$$

where $R_i$ captures task performance and memory quality, and $P_{\text{penalty}}$ discourages redundant storage, excessive tool usage, and uncontrolled context expansion. The optimization objective is:

其中 $R_i$ 捕捉任务性能和记忆质量，$P_{\text{penalty}}$ 用于抑制冗余存储、过度工具使用以及不受控的上下文扩展。 优化目标为：

$${\theta }^{\ast }=\arg \underset{\theta }{max}{\mathbb{E}}_{\tau \sim {\pi }_{\theta }}[R(\tau )].$$

This formulation treats memory management as an integral component of the agent's policy, replacing handcrafted heuristics with a learnable mechanism.

这种形式化把记忆管理视为智能体策略的组成部分，用可学习机制替代人工设计的启发式规则。

Three-stage trajectory structure. To capture long-horizon interactions and progressively train memory capabilities, each trajectory is divided into three consecutive stages: $\tau =({\tau }^{(1)},{\tau }^{(2)},{\tau }^{(3)})$, with a total length of $T=T_1+T_2+T_3$. In Stage 1, the agent engages in casual interactions and may store useful information into LTM. Stage 2 introduces distracting or irrelevant content, requiring the agent to manage its STM through selective retention and compression. Stage 3 presents a task that depends on coordinated use of both retained context and earlier accumulated LTM. A key aspect of this design is that the long-term memory ${\mathcal{M}}_t$ persists across all stages, allowing early knowledge to influence later decisions. In contrast, the context $C_t$ is reset between Stages 1 and 2 to prevent information leakage across phases. The reset before Stage 2 ensures the agent cannot solve the final task via residual context, thereby forcing proper retrieval from LTM and enabling effective training of memory operations.

三阶段轨迹结构。 为了捕捉长程交互并渐进式训练记忆能力，每条轨迹被划分为三个连续阶段：$\tau =({\tau }^{(1)},{\tau }^{(2)},{\tau }^{(3)})$，总长度为 $T=T_1+T_2+T_3$。 在第 1 阶段，智能体进行随意交互，并可以把有用信息存入 LTM。 第 2 阶段引入干扰性或无关内容，要求智能体通过选择性保留和压缩来管理其 STM。 第 3 阶段提出一个任务，该任务依赖于对保留上下文和此前累积 LTM 的协调使用。 这一设计的关键在于，长期记忆 ${\mathcal{M}}_t$ 会贯穿所有阶段持续存在，使早期知识能够影响后续决策。 相比之下，上下文 $C_t$ 会在第 1 阶段和第 2 阶段之间重置，以防止跨阶段信息泄漏。 第 2 阶段前的重置确保智能体不能通过残留上下文解决最终任务，从而迫使它从 LTM 中正确检索，并支持对记忆操作的有效训练。

At each step, we collect an experience tuple $e_t=(s_t,a_t,r_t,\log {\pi }_{{\theta }_{\text{old}}}(a_t|s_t))$, where $r_t$ is typically zero for intermediate steps and assigned after trajectory completion, and $\log {\pi }_{{\theta }_{\text{old}}}(a_t|s_t)$ denotes the $log$ probability under the old policy ${\pi }_{{\theta }_{\text{old}}}$. This representation enables step-wise credit assignment under GRPO and allows the agent to attribute long-term rewards to specific memory decisions across stages. By structuring trajectories in this staged yet continuous manner, the agent learns temporally coherent and task-adaptive memory policies essential for robust long-horizon reasoning.

在每一步，我们收集经验元组 $e_t=(s_t,a_t,r_t,\log {\pi }_{{\theta }_{\text{old}}}(a_t|s_t))$，其中 $r_t$ 对中间步骤通常为零，并在轨迹完成后赋值；$\log {\pi }_{{\theta }_{\text{old}}}(a_t|s_t)$ 表示旧策略 ${\pi }_{{\theta }_{\text{old}}}$ 下的 $log$ 概率。 这种表示支持 GRPO 下的逐步信用分配，并允许智能体把长期奖励归因到跨阶段的具体记忆决策。 通过以这种分阶段但连续的方式组织轨迹，智能体学习时间上连贯且任务自适应的记忆策略，而这对稳健的长程推理至关重要。

表1：AgeMem 中用于操作长期记忆（LTM）和短期记忆（STM）的记忆管理工具。

Tool	Target	Function
Add	LTM	Add new knowledge to $\mathcal{M}_t$
Update	LTM	Modify entries in $\mathcal{M}_t$
Delete	LTM	Remove entries from $\mathcal{M}_t$
Retrieve	STM	Retrieve entries from $\mathcal{M}_t$ to $C_t$
Summary	STM	Summarize segments in $C_t$
Filter	STM	Filter out irrelevant segments from $C_t$

3.2 Memory Management via Tool Interface

AgeMem exposes memory-related operations to the LLM agent through an explicit tool interface (Table 1). The agent can modify its persistent LTM using Add, Update, and Delete, while exercising fine-grained control over STM through Retrieve, Summary, and Filter. Incorporating these tools into the action space transforms memory control from an external heuristic pipeline into an intrinsic component of decision-making. This design allows the agent to adaptively manage memory according to task structure, history, and context.

AgeMem 通过显式工具接口把记忆相关操作暴露给 LLM 智能体（表1）。 智能体可以使用 Add、Update 和 Delete 修改其持久 LTM，同时通过 Retrieve、Summary 和 Filter 对 STM 进行细粒度控制。 把这些工具纳入动作空间，会把记忆控制从外部启发式流水线转化为决策的内在组成部分。 这种设计使智能体能够根据任务结构、历史和上下文自适应地管理记忆。

Each tool serves a distinct functional role in memory management. LTM operations: Add inserts a new entry into the long-term store ${\mathcal{M}}_t$; Update modifies an existing entry identified by memory_id; Delete removes an entry from ${\mathcal{M}}_t$ to prevent accumulation of stale knowledge. STM operations: Retrieve brings the top-$k$ semantically relevant memories from ${\mathcal{M}}_t$ into the active context $C_t$; Summary compresses a specified span of interaction history into a concise representation, reducing context size while preserving essential information; Filter removes context messages whose semantic similarity to a given criterion exceeds a threshold ${\theta }_f$, suppressing irrelevant or distracting content. Together, these six operations provide the agent with expressive yet interpretable control over its memory lifecycle. Full formal definitions and system prompts are provided in the appendix.

每个工具在记忆管理中承担不同的功能角色。 LTM 操作：Add 将新条目插入长期存储 ${\mathcal{M}}_t$；Update 修改由 memory_id 标识的已有条目；Delete 从 ${\mathcal{M}}_t$ 中移除条目，以防止陈旧知识积累。 STM 操作：Retrieve 将来自 ${\mathcal{M}}_t$ 的 top-$k$ 语义相关记忆带入活动上下文 $C_t$；Summary 将指定跨度的交互历史压缩为简洁表示，在保留关键信息的同时减少上下文大小；Filter 会移除与给定标准的语义相似度超过阈值 ${\theta }_f$ 的上下文消息，从而抑制无关或干扰内容。 这六种操作共同为智能体提供了对其记忆生命周期的表达力强且可解释的控制。 完整形式化定义和系统提示见附录。

3.3 Three-Stage Progressive RL Strategy

To learn unified and stable memory behaviors, we propose a progressive three-stage training strategy. For each task instance $q\in \mathcal{T}$, the agent generates a complete trajectory:

为了学习统一且稳定的记忆行为，我们提出一种渐进式三阶段训练策略。 对于每个任务实例 $q\in \mathcal{T}$，智能体生成一条完整轨迹：

$${\tau }_k^{(q)}=({\tau }_k^{(1)},{\tau }_k^{(2)},{\tau }_k^{(3)}),k=1,\dots ,K,$$

where $K$ denotes the number of independent rollouts, and each sub-trajectory ${\tau }_k^{(i)}$ corresponds to a specific training stage.

其中 $K$ 表示独立 rollout 的数量，每条子轨迹 ${\tau }_k^{(i)}$ 对应一个特定训练阶段。

Stage 1 (LTM construction). The agent is exposed to contextual information $I_q$ in a casual conversational setting. The goal is to identify salient information and store it into LTM ${\mathcal{M}}_t$. During the interaction, the short-term context $C_t$ evolves naturally, and the agent may invoke LTM-related tools when appropriate. Formally, this stage yields a sub-trajectory ${\tau }_k^{(1)}=\{e_t{\}}_{t=1}^{T_1}$, where each experience tuple $e_t$ follows the definition in Section 3.1.

第 1 阶段（LTM 构建）。 智能体在随意对话设置中接触上下文信息 $I_q$。 目标是识别关键信息并将其存入 LTM ${\mathcal{M}}_t$。 在交互过程中，短期上下文 $C_t$ 自然演化，智能体可以在适当时调用与 LTM 相关的工具。 形式上，该阶段产生子轨迹 ${\tau }_k^{(1)}=\{e_t{\}}_{t=1}^{T_1}$，其中每个经验元组 $e_t$ 遵循第 3.1 节中的定义。

Stage 2 (STM control under distractors). The short-term context is reset, while the constructed LTM ${\mathcal{M}}_t$ is retained. The agent is then presented with distractor messages---natural-language utterances (e.g., questions or short statements) that resemble plausible conversational context but are deliberately unrelated to the target query---so that the agent must learn to filter or ignore them rather than rely on them for the final answer. The objective is to learn proactive STM control through tool-based operations, such as filtering or summarizing context, in order to suppress noise and preserve useful information. This process forms the sub-trajectory ${\tau }_k^{(2)}=\{e_t{\}}_{t=T_1+1}^{T_1+T_2}$, which emphasizes context filtering and compression capability.

第 2 阶段（干扰项下的 STM 控制）。 短期上下文被重置，而已构建的 LTM ${\mathcal{M}}_t$ 会被保留。 随后，智能体会接收到干扰消息，即看似合理对话上下文但被刻意设置为与目标查询无关的自然语言话语（例如问题或短陈述），使智能体必须学习过滤或忽略它们，而不是依赖它们生成最终答案。 目标是通过基于工具的操作学习主动 STM 控制，例如过滤或总结上下文，以抑制噪声并保留有用信息。 这一过程形成子轨迹 ${\tau }_k^{(2)}=\{e_t{\}}_{t=T_1+1}^{T_1+T_2}$，强调上下文过滤和压缩能力。

Stage 3 (Integrated reasoning and memory coordination). Finally, the agent receives a formal query $q$ requiring both accurate reasoning and effective memory retrieval. The agent must retrieve relevant knowledge from ${\mathcal{M}}_t$, appropriately manage the context $C_t$, and generate a final answer. This stage produces ${\tau }_k^{(3)}=\{e_t{\}}_{t=T_1+T_2+1}^T$, which evaluates the ability of the agent to coordinate long-term memory, short-term context management, and task solution in an end-to-end manner.

第 3 阶段（综合推理与记忆协调）。 最后，智能体接收正式查询 $q$，该查询既需要准确推理，也需要有效记忆检索。 智能体必须从 ${\mathcal{M}}_t$ 中检索相关知识，适当管理上下文 $C_t$，并生成最终答案。 该阶段产生 ${\tau }_k^{(3)}=\{e_t{\}}_{t=T_1+T_2+1}^T$，用于评估智能体以端到端方式协调长期记忆、短期上下文管理和任务解决的能力。

All three segments form a complete trajectory:

所有三个片段组成一条完整轨迹：

$${\tau }_k^{(q)}=(e_1,e_2,\dots ,e_T),T=T_1+T_2+T_3,$$

which is then used for policy optimization in the subsequent step-wise GRPO procedure. For a batch of $B$ tasks, we further aggregate all experiences from $K$ independent rollouts into a unified set $\mathcal{E}=\bigcup _{q=1}^B\bigcup _{k=1}^K\{e_t\mid e_t\in {\tau }_k^{(q)}\}$, with a total size of $|\mathcal{E}|=B\times K\times \overline{T}$, where $\overline{T}$ denotes the average trajectory length. More detailed rollout processes are provided in the appendix.

随后它会被用于后续逐步 GRPO 过程中的策略优化。 对于一批 $B$ 个任务，我们进一步把来自 $K$ 次独立 rollout 的所有经验聚合为统一集合 $\mathcal{E}=\bigcup _{q=1}^B\bigcup _{k=1}^K\{e_t\mid e_t\in {\tau }_k^{(q)}\}$，其总大小为 $|\mathcal{E}|=B\times K\times \overline{T}$，其中 $\overline{T}$ 表示平均轨迹长度。 更详细的 rollout 过程见附录。

Generalizability of the three-stage curriculum. The three-stage structure is not tied to QA-style supervision; it requires only a temporal separation between information exposure and task execution so that the usefulness of memory decisions can be evaluated under delayed outcomes. Each stage fulfills a functional role that can be instantiated in diverse tool-using settings: Stage 1 (information acquisition) requires any pre-task context from which the agent may selectively store information---this could be environment descriptions, retrieved documents, or prior dialogue history, not exclusively QA supporting facts. Stage 2 (interference and context pressure) is already generated synthetically via DistractorGen and requires no dataset annotations. Stage 3 (task execution) provides the downstream objective whose delayed reward determines whether earlier storage and filtering decisions were useful. In this sense, the curriculum depends on information timing and delayed credit assignment rather than QA-style factual supervision.

三阶段课程的泛化性。 三阶段结构并不绑定于 QA 风格监督；它只要求信息暴露和任务执行之间存在时间分离，使记忆决策的有用性能够在延迟结果下被评估。 每个阶段都承担一个功能角色，可在多种工具使用设置中实例化：第 1 阶段（信息获取）需要任何任务前上下文，智能体可以从中选择性地存储信息，这可以是环境描述、检索文档或此前对话历史，而不只限于 QA 支撑事实。 第 2 阶段（干扰与上下文压力）已经通过 DistractorGen 合成生成，不需要数据集标注。 第 3 阶段（任务执行）提供下游目标，其延迟奖励决定早期存储和过滤决策是否有用。 从这个意义上说，该课程依赖的是信息时序和延迟信用分配，而不是 QA 风格的事实监督。

3.4 Step-wise GRPO for Unified Management

We adopt a step-wise variant of GRPO to connect long-range task rewards with memory decisions across all stages. For task $q$, let $G_q=\{{\tau }_1^{(q)},\dots ,{\tau }_K^{(q)}\}$ denote the group of parallel rollouts. Each trajectory yields a terminal reward $r_T^{(k,q)}=R({\tau }_k^{(q)})$. We compute the group-normalized advantage for the terminal step as:

我们采用 GRPO 的逐步变体，将长程任务奖励与所有阶段中的记忆决策连接起来。 对于任务 $q$，令 $G_q=\{{\tau }_1^{(q)},\dots ,{\tau }_K^{(q)}\}$ 表示并行 rollout 组。 每条轨迹产生终端奖励 $r_T^{(k,q)}=R({\tau }_k^{(q)})$。 我们把终端步骤的组归一化 advantage 计算为：

$$A_T^{(k,q)}=\frac{r_T^{(k,q)}-{\mu }_{G_q}}{{\sigma }_{G_q}+ϵ},$$

where ${\mu }_{G_q}$ and ${\sigma }_{G_q}$ are the mean and standard deviation of rewards within $G_q$, $ϵ$ prevents division by zero. This advantage is then broadcast to all preceding steps of the same trajectory $A_t^{(k,q)}=A_T^{(k,q)}$, which assigns a consistent learning signal to all memory and reasoning actions along the trajectory, including those in Stage 1 and Stage 2. In doing so, the final task outcome supervises every intermediate memory decision, enabling long-range credit assignment across heterogeneous stages. We then augment the experience set with advantages, $\mathcal{E}=\bigcup _{q,k}^{B,K}\{(e_t,A_t)|e_t\in {\tau }_k^{(q)},A_t=A_t^{(k,q)}\}$.

其中 ${\mu }_{G_q}$ 和 ${\sigma }_{G_q}$ 是 $G_q$ 内奖励的均值和标准差，$ϵ$ 防止除零。 随后，该 advantage 会被广播到同一轨迹的所有先前步骤 $A_t^{(k,q)}=A_T^{(k,q)}$，从而为轨迹上的所有记忆和推理动作分配一致的学习信号，包括第 1 阶段和第 2 阶段中的动作。 通过这样做，最终任务结果会监督每个中间记忆决策，从而支持跨异质阶段的长程信用分配。 随后，我们用 advantage 扩展经验集合，$\mathcal{E}=\bigcup _{q,k}^{B,K}\{(e_t,A_t)|e_t\in {\tau }_k^{(q)},A_t=A_t^{(k,q)}\}$。

Following GRPO, we maximize the expected objective over all experiences:

按照 GRPO，我们最大化所有经验上的期望目标：

$$\begin{array}{rl}J(\theta )& ={\mathbb{E}}_{(e_t,A_t)\sim \mathcal{E}}[{\rho }_t{A}_t-\beta D_{\text{KL}}[{\pi }_{\theta }\parallel {\pi }_{\text{ref}}]]\\ & =\frac{1}{|\mathcal{E}|}\sum _{q=1}^B\sum _{k=1}^K\sum _{t=1}^{T_k^{(q)}}[{\rho }_t^{(k,q)}{A}_t^{(k,q)}-\beta D_{\text{KL}}^{(k,q)}],\end{array}$$

where the importance ratio ${\rho }_t^{(k,q)}=\frac{{\pi }_{\theta }(a_t|s_t)}{{\pi }_{{\theta }_{\text{old}}}(a_t|s_t)}$ controls the update magnitude under the new policy, $D_{\text{KL}}^{(k,q)}$ denotes the KL divergence penalty between the current policy ${\pi }_{\theta }$ and a fixed reference ${\pi }_{\text{ref}}$, and $\beta$ is a coefficient that balances exploration and training stability.

其中重要性比率 ${\rho }_t^{(k,q)}=\frac{{\pi }_{\theta }(a_t|s_t)}{{\pi }_{{\theta }_{\text{old}}}(a_t|s_t)}$ 控制新策略下的更新幅度，$D_{\text{KL}}^{(k,q)}$ 表示当前策略 ${\pi }_{\theta }$ 与固定参考策略 ${\pi }_{\text{ref}}$ 之间的 KL 散度惩罚，$\beta$ 是平衡探索和训练稳定性的系数。

3.5 Reward Function Design

We design a composite reward that evaluates both downstream task performance and the quality of memory management. The total trajectory-level reward is defined as

我们设计一种复合奖励，同时评估下游任务性能和记忆管理质量。 总的轨迹级奖励定义为

$$R(\tau )={\mathbf{w}}^{\mathrm{\top }}\mathbf{R}+P_{\text{penalty}},$$

where $\mathbf{w}=[w_{\text{task}},w_{\text{context}},w_{\text{memory}}{]}^{\mathrm{\top }}$ are tunable coefficients, and $\mathbf{R}=[R_{\text{task}},R_{\text{context}},R_{\text{memory}}{]}^{\mathrm{\top }}$ correspond to rewards for task completion, context management, and long-term memory management. The penalty term $P_{\text{penalty}}$ captures violations such as context overflow or exceeding the interaction limit. Below, we summarize each component, and precise formulas are provided in the appendix.

其中 $\mathbf{w}=[w_{\text{task}},w_{\text{context}},w_{\text{memory}}{]}^{\mathrm{\top }}$ 是可调系数，$\mathbf{R}=[R_{\text{task}},R_{\text{context}},R_{\text{memory}}{]}^{\mathrm{\top }}$ 分别对应任务完成、上下文管理和长期记忆管理的奖励。 惩罚项 $P_{\text{penalty}}$ 捕捉上下文溢出或超过交互限制等违规情况。 下面，我们总结各个组件，精确公式见附录。

Task completion reward $R_{\text{task}}$. This term provides the primary learning signal by assessing whether the agent solves the task correctly. We obtain a scalar score using an LLM-based judge $S_{\text{judge}}(A_{\text{pred}},A_q)\in [0,1]$, optionally applying a penalty when no answer is produced. This reward encourages accurate, complete task solutions and remains the dominant component to ensure alignment with task objectives.

任务完成奖励 $R_{\text{task}}$。 该项通过评估智能体是否正确解决任务来提供主要学习信号。 我们使用基于 LLM 的 judge $S_{\text{judge}}(A_{\text{pred}},A_q)\in [0,1]$ 获得标量分数，并可在未生成答案时施加惩罚。 该奖励鼓励准确、完整的任务解决方案，并保持为主导组件，以确保与任务目标对齐。

Context management reward $R_{\text{context}}$. This component evaluates STM behavior, focusing on how effectively the agent controls the active context $C_t$. It combines three factors: (i) compression efficiency, promoting economical token usage; (ii) preventive actions, rewarding early summarization or filtering to avoid overflow; and (iii) information preservation, penalizing the loss of critical query-related content. Each factor is normalized, allowing the reward to balance context efficiency against retention of essential information.

上下文管理奖励 $R_{\text{context}}$。 该组件评估 STM 行为，关注智能体控制活动上下文 $C_t$ 的有效程度。 它结合三个因素：(i) 压缩效率，促进经济的 token 使用；(ii) 预防性动作，奖励早期总结或过滤以避免溢出；以及 (iii) 信息保留，对关键查询相关内容的丢失施加惩罚。 每个因素都会归一化，使奖励能够在上下文效率和保留关键信息之间取得平衡。

Memory management reward $R_{\text{memory}}$. This term evaluates LTM operations. It aggregates signals for: (i) storage quality, measured as the fraction of stored entries labeled as high-quality and reusable; (ii) maintenance, rewarding meaningful update or delete operations to mitigate memory staleness; and (iii) semantic relevance, computed using an LLM-based score between retrieved memories and the query. Together, these signals incentivize selective, high-value memory construction and responsible upkeep over time.

记忆管理奖励 $R_{\text{memory}}$。 该项评估 LTM 操作。 它聚合以下信号：(i) 存储质量，用被标记为高质量且可复用的存储条目比例来衡量；(ii) 维护，对有意义的更新或删除操作给予奖励，以缓解记忆陈旧；以及 (iii) 语义相关性，使用检索记忆与查询之间的基于 LLM 的分数计算。 这些信号共同激励选择性、高价值的记忆构建以及随时间进行负责任的维护。

Penalty terms $P_{\text{penalty}}$. Penalties discourage undesirable behaviors such as exceeding the maximum number of dialogue turns or triggering context overflow. Penalty coefficients are chosen so that such violations lead to a substantial reduction in the final trajectory reward, encouraging the agent to maintain safe and efficient memory practices.

惩罚项 $P_{\text{penalty}}$。 惩罚用于抑制超过最大对话轮数或触发上下文溢出等不理想行为。 惩罚系数的选择会使这类违规显著降低最终轨迹奖励，从而鼓励智能体保持安全且高效的记忆实践。

4. Experiments

4.1 Experimental Setup

Datasets. To comprehensively evaluate AgeMem, we select five widely-used datasets in LLM-based agent research: ALFWorld, SciWorld, PDDL, BabyAI, and HotpotQA. These datasets cover embodied action, game-based reasoning, and knowledge-intensive question answering, providing diverse evaluation scenarios. Since the HotpotQA dataset contains both questions and supporting facts, automatically providing Stage 1 contextual information, AgeMem is fine-tuned with RL only on the HotpotQA training set and then evaluated directly on all datasets. Detailed dataset statistics are provided in the appendix.

数据集。 为了全面评估 AgeMem，我们选择了五个在基于 LLM 的智能体研究中广泛使用的数据集：ALFWorld、SciWorld、PDDL、BabyAI 和 HotpotQA。 这些数据集覆盖具身动作、基于游戏的推理以及知识密集型问答，提供了多样化评估场景。 由于 HotpotQA 数据集同时包含问题和支撑事实，能够自动提供第 1 阶段上下文信息，因此 AgeMem 只在 HotpotQA 训练集上使用 RL 微调，然后直接在所有数据集上评估。 详细数据集统计见附录。

Evaluation metrics. For the primary task completion metrics, we adopt Success Rate (SR) for ALFWorld, SciWorld, and BabyAI, Progress Rate (PR) for PDDL, and LLM-as-a-Judge (J) for HotpotQA. Additionally, we employ an LLM-based evaluator to assess the quality of stored long-term memory during knowledge reasoning, measured by Memory Quality (MQ). The prompts of the LLM-based evaluation are provided in the appendix.

评估指标。 对于主要任务完成指标，我们在 ALFWorld、SciWorld 和 BabyAI 上采用 Success Rate（SR），在 PDDL 上采用 Progress Rate（PR），并在 HotpotQA 上采用 LLM-as-a-Judge（J）。 此外，我们使用基于 LLM 的评估器来评估知识推理期间存储的长期记忆质量，并以 Memory Quality（MQ）衡量。 基于 LLM 的评估提示见附录。

Baselines & LLM backbones. We compare AgeMem against four representative agent LTM systems: LangMem, A-Mem, Mem0, and ${\text{Mem0}}^g$ (a graph-based variant officially provided as part of Mem0). To better demonstrate the effectiveness of RL training, we also include AgeMem-noRL, which is not fine-tuned with RL. In ablation studies on STM, we compare STM tools with the RAG approach. For the base agent models, we use Qwen2.5-7B-Instruct and Qwen3-4B-Instruct. More baseline configurations are in the appendix.

基线与 LLM backbone。 我们将 AgeMem 与四个代表性智能体 LTM 系统比较：LangMem、A-Mem、Mem0 和 ${\text{Mem0}}^g$（Mem0 官方提供的基于图的变体）。 为了更好展示 RL 训练的有效性，我们还加入未用 RL 微调的 AgeMem-noRL。 在 STM 消融研究中，我们将 STM 工具与 RAG 方法进行比较。 对于基础智能体模型，我们使用 Qwen2.5-7B-Instruct 和 Qwen3-4B-Instruct。 更多基线配置见附录。

Implementation details. We build agents using the Agentscope framework and fine-tune AgeMem using the Trinity framework. Further implementation details are provided in the appendix.

实现细节。 我们使用 Agentscope 框架构建智能体，并使用 Trinity 框架微调 AgeMem。 更多实现细节见附录。

表2：五个 benchmark 上的性能比较。最佳结果用粗体标记，次优结果以下划线标记。

Method	ALFWorld	SciWorld	PDDL	BabyAI	HotpotQA	Average
Qwen2.5-7B-Instruct
No-Memory	27.16	13.80	10.15	50.80	38.36	28.05
LangMem	38.27	28.29	15.85	51.34	37.43	34.23
A-Mem	34.68	28.06	18.39	58.82	43.95	36.78
Mem0	37.49	26.99	13.96	60.58	46.66	37.14
Mem0${}^g$	35.34	30.50	14.86	58.78	42.06	36.31
AgeMem-noRL	37.90	28.67	8.87	46.34	45.36	33.43
AgeMem (Ours)	41.07	35.55	17.31	61.42	54.44	41.96
Qwen3-4B-Instruct
No-Memory	38.51	47.89	30.14	55.83	47.48	43.97
LangMem	40.89	50.42	28.42	53.80	42.70	43.25
A-Mem	34.31	50.14	34.41	61.35	48.48	45.74
Mem0	41.17	51.38	31.72	60.05	39.16	44.70
Mem0${}^g$	36.69	47.76	29.61	57.59	38.12	41.95
AgeMem-noRL	38.02	50.42	27.52	57.48	54.49	45.59
AgeMem (Ours)	48.97	59.48	35.07	72.56	55.49	54.31

4.2 Main Results

Comparison with counterparts. Table 2 shows that AgeMem achieves the highest average performance on both Qwen2.5-7B-Instruct (41.96%) and Qwen3-4B-Instruct (54.31%), outperforming all baselines across five datasets with relative gains of 49.59% and 23.52% over no-memory, respectively. Compared to the best baselines (Mem0 and A-Mem), AgeMem improves by 4.82 and 8.57 percentage points on average. RL training contributes 8.53 and 8.72 percentage points of improvement over AgeMem-noRL, validating the three-stage progressive RL strategy.

与同类方法比较。 表2显示，AgeMem 在 Qwen2.5-7B-Instruct（41.96%）和 Qwen3-4B-Instruct（54.31%）上都取得最高平均性能，在五个数据集上优于所有基线，相比无记忆分别获得 49.59% 和 23.52% 的相对提升。 与最佳基线（Mem0 和 A-Mem）相比，AgeMem 平均分别提升 4.82 和 8.57 个百分点。 RL 训练相比 AgeMem-noRL 带来 8.53 和 8.72 个百分点的提升，验证了三阶段渐进式 RL 策略。

图2：HotpotQA 上不同方法的 Memory Quality 分数。分数越高表示存储记忆与 ground-truth facts 之间的相关性越好。

表3：HotpotQA 上的工具使用统计。数字表示每个 episode 的平均调用次数。

Tool Category	Qwen2.5-7B		Qwen3-4B
	noRL	GRPO	noRL	GRPO
LTM Tool Statistics
Add Memory	0.92	1.64	2.49	2.64
Update Memory	0.00	0.13	0.13	0.34
Delete Memory	0.00	0.08	0.00	0.22
STM Tool Statistics
Retrieve Memory	2.31	1.95	4.62	4.35
Summary Context	1.08	0.82	0.11	0.96
Filter Context	0.02	0.31	0.15	0.16
Total Calls	4.33	4.92	7.50	8.67

图3：HotpotQA 上不同 STM 管理配置下的平均 prompt token 数量。后缀 “-RAG” 表示采用 RAG 替代基于 STM 工具的管理。

Quality of stored long-term memories. To evaluate the quality of stored memories, we leverage the ground-truth facts provided in the HotpotQA dataset and assess the relevance between stored memories and these facts using an LLM-based evaluator. Figure 2 presents the Memory Quality (MQ) scores for different baselines. AgeMem achieves the highest memory quality on both model backbones, with MQ scores of 0.533 and 0.605, respectively. This indicates that the unified memory management framework not only improves task performance but also promotes the storage of high-quality, reusable knowledge. The comparison with baseline methods further validates that AgeMem's tool-based memory operations lead to more selective and higher-quality memory construction.

已存长期记忆的质量。 为了评估存储记忆的质量，我们利用 HotpotQA 数据集提供的 ground-truth facts，并使用基于 LLM 的评估器评估存储记忆与这些事实之间的相关性。 图2展示了不同基线的 Memory Quality（MQ）分数。 AgeMem 在两个模型 backbone 上都取得最高记忆质量，MQ 分数分别为 0.533 和 0.605。 这表明统一记忆管理框架不仅提升任务性能，也促进了高质量、可复用知识的存储。 与基线方法的比较进一步验证，AgeMem 的基于工具的记忆操作会带来更有选择性且更高质量的记忆构建。

Effectiveness of STM management. We evaluate the effectiveness of STM management by measuring the prompt token count under different configurations on HotpotQA. Figure 3 shows that AgeMem successfully reduces prompt token usage compared to variants without STM tools (-RAG). On Qwen2.5-7B-Instruct, AgeMem uses 2,117 tokens on average, compared to 2,186 tokens for AgeMem-RAG, representing a reduction of 3.1%. On Qwen3-4B-Instruct, the reduction is even more pronounced: AgeMem uses 2,191 tokens versus 2,310 tokens for AgeMem-RAG, a reduction of 5.1%. These results demonstrate that the learned STM management tools effectively control context expansion, enabling more efficient token usage while maintaining task performance.

STM 管理的有效性。 我们通过测量 HotpotQA 上不同配置下的 prompt token 数量来评估 STM 管理的有效性。 图3显示，与没有 STM 工具的变体（-RAG）相比，AgeMem 成功减少了 prompt token 使用。 在 Qwen2.5-7B-Instruct 上，AgeMem 平均使用 2,117 个 token，而 AgeMem-RAG 使用 2,186 个 token，减少了 3.1%。 在 Qwen3-4B-Instruct 上，减少更加明显：AgeMem 使用 2,191 个 token，而 AgeMem-RAG 使用 2,310 个 token，减少了 5.1%。 这些结果表明，学习到的 STM 管理工具能够有效控制上下文扩展，在保持任务性能的同时实现更高效的 token 使用。

图4：LTM、STM 和 RL 组件的消融研究（Qwen2.5-7B-Instruct）。Base：无记忆基线；+LT：AgeMem-noRL-RAG（仅 LTM 工具）；+LT/RL：AgeMem-RAG（带 LTM 工具的 RL）；+LT/ST/RL：AgeMem（带 RL 的完整 AgeMem 系统）。绿色箭头表示相对于基线的性能提升。

Tool usage analysis. Table 3 reports tool usage statistics before and after RL fine-tuning on HotpotQA. RL training substantially increases the use of long-term memory tools, especially Add and Update. On Qwen2.5-7B-Instruct, Add operations rise from 0.92 to 1.64, and Update operations appear after training (0.13 v.s. nearly zero). Similar trends are observed on Qwen3-4B-Instruct, with higher frequencies of both Add and Update. For short-term memory tools, RL leads to more balanced tool usage. The frequency of Filter increases notably (e.g., from 0.02 to 0.31 on Qwen2.5), indicating proactive context control.

工具使用分析。 表3报告了 HotpotQA 上 RL 微调前后的工具使用统计。 RL 训练显著增加了长期记忆工具的使用，尤其是 Add 和 Update。 在 Qwen2.5-7B-Instruct 上，Add 操作从 0.92 增加到 1.64，Update 操作在训练后出现（0.13，而训练前几乎为零）。 在 Qwen3-4B-Instruct 上也观察到类似趋势，Add 和 Update 的频率都更高。 对于短期记忆工具，RL 带来了更均衡的工具使用。 Filter 的频率显著增加（例如在 Qwen2.5 上从 0.02 增加到 0.31），表明出现了主动上下文控制。

Notably, the decrease in Retrieve frequency after RL training (Qwen2.5: $2.31\to 1.95$; Qwen3: $4.62\to 4.35$) reflects a qualitative shift in retrieval strategy rather than undertraining. Before RL, the agent retrieves reactively and repeatedly to compensate for suboptimal Stage-1 storage. After RL optimization, Add/Update frequencies increase (Table 3), improving LTM quality; retrieval then becomes more selective and query-driven, used primarily when previously stored information is genuinely needed. This reduction in retrieval frequency coincides with improved task performance and MQ, indicating greater efficiency rather than insufficient learning. Overall, these patterns suggest that RL training enables coordinated and adaptive memory management. Detailed case studies are provided in the appendix.

值得注意的是，RL 训练后 Retrieve 频率下降（Qwen2.5：$2.31\to 1.95$；Qwen3：$4.62\to 4.35$），反映的是检索策略的质性转变，而不是训练不足。 在 RL 之前，智能体会以反应式且重复的方式检索，以补偿第 1 阶段存储不佳的问题。 在 RL 优化后，Add/Update 频率增加（表3），提升了 LTM 质量；随后检索变得更有选择性、更受查询驱动，主要在先前存储的信息真正需要时才使用。 这种检索频率的下降与任务性能和 MQ 提升同时出现，表明它代表更高效率，而不是学习不足。 总体而言，这些模式说明 RL 训练支持协调且自适应的记忆管理。 详细案例研究见附录。

图5：Qwen2.5-7B-Instruct 上的训练收敛曲线，比较 All-Returns（实线）与 Answer-Only（虚线）奖励策略。

4.3 Ablation Studies

LTM-STM components. To validate the contributions of individual components, we conduct ablation studies on LTM, STM, and RL training. Figure 4 presents results on three representative datasets using Qwen2.5-7B-Instruct as the backbone (results for Qwen3-4B-Instruct are provided in the appendix). Adding LTM alone (+LT) yields substantial gains of +10.6%, +14.2%, and +7.4% over the baseline. Incorporating RL training (+LT/RL) further improves performance, particularly on HotpotQA (+6.3%), demonstrating the effectiveness of our reward-based optimization. The full AgeMem system (+LT/ST/RL) achieves the best results across all benchmarks, with overall improvements of +13.9%, +21.7%, and +16.1%. Notably, adding STM tools provides the most significant boost on SciWorld (+3.1%) and HotpotQA (+2.4%), validating that learned context management outperforms static RAG approaches. These progressive improvements confirm that unified memory management with end-to-end RL is essential for optimal agent performance.

LTM-STM 组件。 为验证各个组件的贡献，我们对 LTM、STM 和 RL 训练进行了消融研究。 图4展示了以 Qwen2.5-7B-Instruct 为 backbone 在三个代表性数据集上的结果（Qwen3-4B-Instruct 的结果见附录）。 仅加入 LTM（+LT）就相较基线带来 +10.6%、+14.2% 和 +7.4% 的显著提升。 结合 RL 训练（+LT/RL）进一步提升性能，尤其是在 HotpotQA 上（+6.3%），证明了我们基于奖励的优化的有效性。 完整 AgeMem 系统（+LT/ST/RL）在所有基准上取得最佳结果，整体提升为 +13.9%、+21.7% 和 +16.1%。 值得注意的是，加入 STM 工具在 SciWorld（+3.1%）和 HotpotQA（+2.4%）上提供了最显著提升，验证了学习到的上下文管理优于静态 RAG 方法。 这些渐进式提升确认，带有端到端 RL 的统一记忆管理对于最优智能体性能至关重要。

Reward function. To demonstrate the effectiveness of our multi-component reward function design, we compare the full reward function (All-Returns) against a variant using only $R_{\text{task}}$ (Answer-Only). Figure 5 shows the reward convergence curves of Qwen2.5-7B-Instruct during GRPO training on HotpotQA. The full reward function leads to significantly faster convergence and higher final performance compared to the task-only variant. As detailed in Table 4, the All-Returns strategy achieves higher LLM-as-a-Judge scores (0.544 v.s. 0.509) while maintaining substantially better memory quality (0.533 v.s. 0.479). Notably, despite using more tokens (2117 v.s. 2078), the All-Returns strategy achieves better overall performance, indicating that the additional context and memory operations contribute meaningfully to reasoning quality. Similar patterns are observed on Qwen3-4B-Instruct.

奖励函数。 为展示我们的多组件奖励函数设计的有效性，我们将完整奖励函数（All-Returns）与仅使用 $R_{\text{task}}$ 的变体（Answer-Only）进行比较。 图5展示了 Qwen2.5-7B-Instruct 在 HotpotQA 上进行 GRPO 训练时的奖励收敛曲线。 与仅任务奖励变体相比，完整奖励函数带来显著更快的收敛和更高的最终性能。 如表4所示，All-Returns 策略取得更高的 LLM-as-a-Judge 分数（0.544 v.s. 0.509），同时保持显著更好的记忆质量（0.533 v.s. 0.479）。 值得注意的是，尽管使用更多 token（2117 v.s. 2078），All-Returns 策略仍取得更好的整体性能，说明额外上下文和记忆操作对推理质量有实质贡献。 在 Qwen3-4B-Instruct 上也观察到类似模式。

FILTER threshold ${\theta }_f$. Table 5 reports performance on HotpotQA as ${\theta }_f$ varies. Performance is stable within ${\theta }_f\in [0.4,0.8]$, indicating that AgeMem is not highly sensitive to precise threshold tuning. When ${\theta }_f$ is too low, filtering becomes too aggressive, discarding potentially useful context and increasing reliance on later reconstruction. When ${\theta }_f$ is too high, filtering becomes overly permissive, allowing marginal context to pass through STM and slightly degrading memory quality. Average token counts remain similar across settings, confirming that differences stem from selection quality rather than context length alone.

FILTER 阈值 ${\theta }_f$。 表5报告了 ${\theta }_f$ 变化时 HotpotQA 上的性能。 当 ${\theta }_f\in [0.4,0.8]$ 时性能稳定，表明 AgeMem 对精确阈值调优并不高度敏感。 当 ${\theta }_f$ 过低时，过滤会过于激进，丢弃潜在有用上下文并增加对后续重构的依赖。 当 ${\theta }_f$ 过高时，过滤会过于宽松，允许边缘上下文通过 STM，并略微降低记忆质量。 不同设置下平均 token 数量保持相近，确认差异来自选择质量，而不只是上下文长度。

表4：在 Qwen2.5-7B-Instruct 上进行 HotpotQA reward function 消融。All-Returns v.s. Answer-Only reward strategies。“TN” 表示 token 数量，“TC” 表示工具调用次数。

Strategy	J ↑	TN ↓	MQ ↑	TC
Answer-Only	0.509	2078	0.479	3.93
All-Returns	0.544	2117	0.533	4.92

表5：AgeMem 在 HotpotQA 上对 FILTER 阈值的敏感性。

$\theta_f$	J ↑	MQ ↑	Avg. Tokens
0.4	0.524	0.511	2089
0.5	0.551	0.550	2116
0.6	0.544	0.533	2117
0.7	0.530	0.526	2149
0.8	0.531	0.510	2134

5. Conclusion

In this work, we propose Agentic Memory (AgeMem), a unified memory management framework that enables LLM-based agents to jointly control long-term and short-term memory through learnable, tool-based actions. By integrating memory operations directly into the agent's policy and training them with a progressive reinforcement learning strategy, AgeMem replaces heuristic memory pipelines with an end-to-end optimized solution. Extensive experiments across diverse long-horizon benchmarks show that AgeMem improves both task performance and memory quality while maintaining efficient context usage. These results highlight the importance of unified, agent-centric memory policies and suggest a promising direction for building scalable and adaptive LLM agents capable of long-term reasoning.

在本文中，我们提出 Agentic Memory（AgeMem），这是一个统一记忆管理框架，使基于 LLM 的智能体能够通过可学习、基于工具的动作联合控制长期记忆和短期记忆。 通过把记忆操作直接整合进智能体策略，并用渐进式强化学习策略训练它们，AgeMem 用端到端优化方案取代启发式记忆流水线。 在多样化长程基准上的大量实验表明，AgeMem 在保持高效上下文使用的同时提升任务性能和记忆质量。 这些结果突出了统一、以智能体为中心的记忆策略的重要性，并为构建具备长期推理能力、可扩展且自适应的 LLM 智能体指出了有前景的方向。

Limitations

While AgeMem demonstrates strong performance across multiple settings, there remain opportunities for further extension. The current implementation adopts a fixed set of memory management tools, which provides a clear and effective abstraction but could be extended to support more fine-grained control in future work.

虽然 AgeMem 在多个设置中表现强劲，但仍有进一步扩展的机会。 当前实现采用一组固定的记忆管理工具，这提供了清晰且有效的抽象，但未来工作可以扩展它以支持更细粒度的控制。

In addition, although we evaluate our approach on five representative long-horizon benchmarks and demonstrate zero-shot transfer across domains, these settings remain relatively controlled compared to open-ended real-world deployments. Evaluation in persistent, long-term dialogue or real-user interaction scenarios is an important next step, and we present our current study as establishing cross-domain transfer under controlled conditions as a foundation for such future work.

此外，尽管我们在五个代表性长程基准上评估了我们的方法，并展示了跨领域零样本迁移，但与开放式真实世界部署相比，这些设置仍相对受控。 在持久长期对话或真实用户交互场景中进行评估是重要的下一步，我们把当前研究呈现为在受控条件下建立跨领域迁移能力，为这类未来工作奠定基础。

Finally, the training currently relies on HotpotQA as the source of three-stage trajectories; extending the curriculum to other data sources with richer interaction structures would further broaden the framework's applicability.

最后，当前训练依赖 HotpotQA 作为三阶段轨迹来源；将课程扩展到具有更丰富交互结构的其他数据源，将进一步拓宽该框架的适用性。