---

AtomMem: Learnable Dynamic Agentic Memory with Atomic Memory Operation

MemoryAgentRL20+中国人民大学清华大学

Huo Y, Lu Y, Zhang Z, et al. AtomMem: Learnable Dynamic Agentic Memory with Atomic Memory Operation[J]. arXiv preprint arXiv:2601.08323v3, 2026.

https://github.com/RUCBM/AtomMem

---

AtomMem：基于原子记忆操作的可学习动态智能体记忆

Abstract

Equipping agents with memory is essential for solving real-world long-horizon problems. However, most existing agent memory mechanisms rely on static and hand-crafted workflows. This limits the performance and generalization ability of these memory designs, which highlights the need for a more flexible, learning-based memory framework. In this paper, we propose AtomMem, which reframes memory management as a dynamic decision-making problem. We deconstruct high-level memory processes into fundamental atomic CRUD (Create, Read, Update, Delete) operations, transforming the memory workflow into a learnable decision process. Through widely used reinforcement learning (GRPO), AtomMem learns an autonomous, task-aligned policy to orchestrate memory behaviors tailored to specific task demands. Experimental results across 3 long-context QA benchmarks and 2 web benchmarks demonstrate that the trained model consistently outperforms prior static-workflow memory methods. Further analysis of training dynamics shows that our learning-based formulation enables the agent to discover structured, task-aligned memory management strategies, highlighting a key advantage over predefined workflows.

为智能体配备记忆对于解决真实世界的长程问题至关重要。 然而，现有大多数智能体记忆机制依赖静态且手工设计的工作流。 这限制了这些记忆设计的性能与泛化能力，凸显出对更灵活、基于学习的记忆框架的需求。 在本文中，我们提出 AtomMem，将记忆管理重新表述为动态决策问题。 我们将高层记忆过程分解为 基础原子 CRUD（创建、读取、更新、删除）操作，把记忆工作流转化为可学习的决策过程。 通过广泛使用的强化学习方法 GRPO，AtomMem 学习一种自主且与任务对齐的策略，根据具体任务需求编排记忆行为。 在 3 个长上下文问答基准和 2 个 Web 基准上的实验结果表明，训练后的模型持续优于采用静态工作流的既有记忆方法。 对训练动态的进一步分析表明，我们基于学习的形式化使智能体能够发现结构化且与任务对齐的记忆管理策略，凸显了其相较预定义工作流的一项关键优势。

1. Introduction

Enabling LLM-based agents to accomplish long-horizon and more complex tasks has been a shared goal across both industry and academia. A critical bottleneck in this pursuit is the design of memory mechanisms. Currently, most memory mechanisms of LLM-based agents rely on static, expert-crafted workflows. In these systems, memory operations are confined to predefined pipelines rather than decided autonomously by the model.

让基于 LLM 的智能体完成长程且更复杂的任务，一直是产业界与学术界的共同目标。 这一追求中的关键瓶颈是记忆机制的设计。 目前，基于 LLM 的智能体所采用的大多数记忆机制依赖静态且由专家设计的工作流。 在这些系统中，记忆操作被限制在预定义流水线内，而不是由模型自主决定。

图1：静态记忆的一体适用工作流往往无法适应多样化任务；动态记忆系统则需要根据具体情境确定最优记忆策略。

A core limitation of these approaches lies in their implicit 'one-size-fits-all' assumption: they impose fixed memory management rules for information retention, rather than allowing the model to make autonomous decisions. Strategies like continuous memory fusion or predefined forgetting schedules may work well in generic scenarios but fail in complex environments. For example, exponential forgetting schedules may prematurely discard early yet critical cues in long-horizon reasoning. As illustrated in Figure 1, the same static workflow successfully preserves important information in Task A, but fails to do so in Task B. This naturally raises a question: how should we design a more adaptive and effective agent memory system?

这些方法的核心局限在于其隐含的“一体适用”假设：它们为信息保留施加固定的记忆管理规则，而不是允许模型自主决策。 持续记忆融合或预定义遗忘计划等策略可能在一般场景中表现良好，却会在复杂环境中失效。 例如，指数遗忘计划可能会过早丢弃长程推理中早期但关键的线索。 如图1所示，同一个静态工作流在任务 A 中成功保留了重要信息，却未能在任务 B 中做到这一点。 这自然引出了一个问题：我们应当如何设计更具适应性且更有效的智能体记忆系统？

To answer this question, we propose AtomMem, which reframes memory management of LLM-based agents not as a fixed workflow, but as a decision-making problem. Drawing inspiration from agent tool learning—where models learn when to invoke tools based on context, we deconstruct high-level memory processes into their fundamental atoms: the standard CRUD (Create, Read, Update, Delete) operations. This atomization transforms a static memory workflow into a learnable decision process.

为回答这一问题，我们提出 AtomMem，它不再把基于 LLM 的智能体记忆管理视为固定工作流，而是将其重新表述为决策问题。 受智能体工具学习的启发——模型会根据上下文学习何时调用工具——我们将高层记忆过程分解为其基础原子：标准 CRUD（创建、读取、更新、删除）操作。 这种原子化把静态记忆工作流转化为可学习的决策过程。

A key advantage of this framework is that the effectiveness of memory management is no longer fundamentally bounded by expert rules, but instead by the model's own capacity to make appropriate decisions. By training with reinforcement learning (RL), the agent can acquire experience over these atomic operations and gradually learn a policy for managing memory in a task-aware manner—retaining information that is important for the completion of the task. In this way, memory is no longer treated as a fixed mechanism, but as a kind of policy behavior optimized through interaction with the environment.

这一框架的关键优势是，记忆管理的有效性不再从根本上受专家规则限制，而是取决于模型自身作出适当决策的能力。 通过强化学习（RL）训练，智能体能够获得执行这些原子操作的经验，并逐步学习一种任务感知的记忆管理策略，保留对完成任务重要的信息。 通过这种方式，记忆不再被视为固定机制，而是成为一种通过与环境交互来优化的策略行为。

We evaluate our method on multiple memory-intensive tasks, which can be broadly divided into two categories: 1) Three multi-hop long-context QA tasks, including HotpotQA, 2WikiMultihopQA, Musique, and 2) Two multi-turn web search task including GAIA and WebWalkerQA. Across all tasks, our approach consistently outperforms prior methods reliant on static memory workflows by approximately 3-8 percentage points under the same Qwen3-8B backbone. These results demonstrate that treating memory management as an atomic-level capability optimized via RL is more effective than relying on predefined routines.

我们在多个记忆密集型任务上评估该方法，这些任务大体可分为两类：1）三个多跳长上下文问答任务，包括 HotpotQA、2WikiMultihopQA 和 Musique；2）两个多轮 Web 搜索任务，包括 GAIA 和 WebWalkerQA。 在所有任务上，采用相同 Qwen3-8B 主干时，我们的方法始终比依赖静态记忆工作流的既有方法高约 3–8 个百分点。 这些结果表明，将记忆管理视为通过 RL 优化的原子级能力，比依赖预定义例程更为有效。

Beyond overall performance gains, we further uncover an empirical insight into how memory should be managed for these tasks. The learned policy exhibits a systematic shift in memory operation usage: the frequencies of Create, Update, and Delete operations steadily increase, while reliance on Read actions decreases and stabilizes at a lower level. Whereas when the task condition changes, the frequencies follow entirely different trends. This suggests that effective memory control for these tasks benefits from learning task-aligned patterns, rather than maintaining a fixed strategy.

除总体性能增益外，我们还进一步揭示了应当如何为这些任务管理记忆的经验性见解。 学习得到的策略在记忆操作使用上呈现系统性转变：Create、Update 和 Delete 操作的频率稳步提高，而对 Read 操作的依赖下降并稳定在较低水平。 而当任务条件改变时，这些频率会遵循完全不同的变化趋势。 这表明，这些任务中的有效记忆控制得益于学习与任务对齐的模式，而不是维持固定策略。

图2：AtomMem 框架概览。智能体在维护外部记忆的同时与环境交互；高层记忆工作流被分解为原子 CRUD 操作，并通过端到端强化学习形成与任务对齐的动态记忆管理策略。

2. Related Works

Static Memory Workflow. Early memory mechanisms in LLM-based agents typically relied on heuristic-based static workflows. These memory mechanisms can be categorized into two types: 1) Imitation-Based: Imitation-based approaches refer to transferring designs from natural systems or other engineering domains into agent memory architectures. For example, MemoryBank draws an analogy between agent memory and human memory, while MemGPT likens the agent’s context to computer memory. 2) Prior-Based: Prior-based approaches refer to carefully crafted workflows designed by human experts based on prior knowledge. Despite their theoretical appeal, these methods share a common limitation: the memory workflow is hard-coded by experts. This rigidity prevents the agent from adapting its memory strategy to different tasks. For example, architectures designed for QA tasks may be difficult to transfer to agent tasks that interact with the external environment. In contrast, our work moves beyond static rules, aiming to learn a dynamic memory policy directly from different task.

静态记忆工作流。 基于 LLM 的智能体所采用的早期记忆机制通常依赖启发式静态工作流。 这些记忆机制可分为两类：1）模仿型：模仿型方法是指把自然系统或其他工程领域的设计迁移到智能体记忆架构中。 例如，MemoryBank 将智能体记忆类比为人类记忆，而 MemGPT 将智能体上下文类比为计算机内存。 2）先验型：先验型方法是指人类专家依据先验知识精心设计的工作流。 尽管这些方法在理论上颇具吸引力，但它们存在一个共同局限：记忆工作流由专家硬编码。 这种僵化性阻止智能体使其记忆策略适应不同任务。 例如，为问答任务设计的架构可能难以迁移到需要与外部环境交互的智能体任务。 相比之下，我们的工作超越静态规则，旨在直接从不同任务中学习动态记忆策略。

Reinforcement Learning in Agent Memory. As reinforcement learning becomes a common method for fine-tuning LLM behavior, some works have begun using RL to enhance agent memory, which we categorize into two paradigms: Summarization-Based: MemAgent and Mem1 utilize step-wise overwriting summaries. Although overwriting can theoretically emulate any atomic operation, the workflow is restricted to a mandatory "update-at-every-step" routine. This ignores information density, forcing redundant updates even when new data is sparse. Heuristic-Tool-Based: Memory-As-Action and AgentFold introduce memory management tools such as context pruning or folding. While providing some dynamic control over memory, the tool designs themselves rely heavily on manual priors. In contrast, we provids the model with only the most atomic memory operations, better highlighting the characteristics of the memory-as-decision-making paradigm.

智能体记忆中的强化学习。 随着强化学习成为微调 LLM 行为的常用方法，一些工作开始使用 RL 增强智能体记忆，我们将其分为两种范式： 基于摘要：MemAgent 和 Mem1 使用逐步覆写摘要。 尽管覆写在理论上可以模拟任意原子操作，但工作流被限制为强制性的“每步更新”例程。 这忽略了信息密度，即使新数据很稀疏也会迫使系统执行冗余更新。 基于启发式工具：Memory-As-Action 和 AgentFold 引入上下文剪枝或折叠等记忆管理工具。 这些方法虽然提供了一定程度的动态记忆控制，但工具设计本身严重依赖人工先验。 相比之下，我们只向模型提供最原子的记忆操作，从而更充分地突出“记忆即决策”范式的特征。

3. Method

In this section, we formulate the memory management of LLM-based agents as a sequential decision-making problem and introduce a complete action space over memory operations.

在本节中，我们将基于 LLM 的智能体记忆管理形式化为序列决策问题，并引入覆盖记忆操作的完整动作空间。

3.1 Preliminaries: POMDP for Memory

We model memory management in an LLM-based agent as a Partially Observable Markov Decision Process (POMDP) $(\mathcal{S},\mathcal{A},P,\mathrm{\Omega },\mathcal{O},\mathcal{R},\gamma )$, where memory is explicitly treated as a controllable component of the environment.

我们将基于 LLM 的智能体记忆管理建模为部分可观测马尔可夫决策过程（POMDP）$(\mathcal{S},\mathcal{A},P,\mathrm{\Omega },\mathcal{O},\mathcal{R},\gamma )$，其中记忆被显式视为环境中可控制的组成部分。

• Global State $\mathcal{S}$. The global state is $s_t=(s_t^{env},s_t^{mem})$, comprising the external environment and the internal memory state.
• Action Space $\mathcal{A}$. An action $a_t\in \mathcal{A}$ is a joint decision $a_t=(a_t^{env},a_t^{mem})$. While $a_t^{env}$ represents task-specific execution (e.g., search), $a_t^{mem}$ denotes a memory management action chosen from our atomic CRUD space.
• Transition Function $P$. The transition $P(s_{t+1}\mid s_t,a_t)$ defines how the state evolves. Notably, the internal memory state $s_{t+1}^{mem}$ is directly modified by the agent's actions.
• Observation Function $\mathcal{O}$. The agent observes $o_t=(o_t^{env},o_t^{mem})$. Crucially, memory is not fully observable: $o_t^{mem}$ is determined by previous memory actions (e.g., Read), making memory access an explicit decision variable.

• 全局状态 $\mathcal{S}$。 全局状态为 $s_t=(s_t^{env},s_t^{mem})$，由外部环境和内部记忆状态组成。
• 动作空间 $\mathcal{A}$。 动作 $a_t\in \mathcal{A}$ 是联合决策 $a_t=(a_t^{env},a_t^{mem})$。其中 $a_t^{env}$ 表示任务特定执行（例如搜索），$a_t^{mem}$ 表示从我们的原子 CRUD 空间中选择的记忆管理动作。
• 转移函数 $P$。 转移 $P(s_{t+1}\mid s_t,a_t)$ 定义状态如何演化。值得注意的是，内部记忆状态 $s_{t+1}^{mem}$ 会被智能体动作直接修改。
• 观测函数 $\mathcal{O}$。 智能体观测 $o_t=(o_t^{env},o_t^{mem})$。关键在于，记忆并非完全可观测：$o_t^{mem}$ 由先前的记忆动作（例如 Read）决定，使记忆访问成为显式决策变量。

When we formalize memory as a POMDP, it means that we can leverage RL algorithms to optimize the LLM's ability to manage memory. Notably, the agent’s memory should be regarded as part of the environment and reset at the start of each independent task, meaning that

当我们将记忆形式化为 POMDP 时，就意味着可以利用 RL 算法优化 LLM 管理记忆的能力。 值得注意的是，智能体记忆应被视为环境的一部分，并在每个独立任务开始时重置，即

$$s_0^{mem}=\varnothing .$$

This setting should be distinguished from another type of memory that accumulates experience across different tasks.

这一设置应与另一类会在不同任务之间累积经验的记忆区分开来。

3.2 Why Atomic CRUD Operations?

We adopt CRUD (Create, Read, Update, Delete) as the atomic memory action space $\mathcal{A}$ based on three foundational properties.

基于三项基础性质，我们采用 CRUD（创建、读取、更新、删除）作为原子记忆动作空间 $\mathcal{A}$。

• Completeness. CRUD constitutes a universal set of operations capable of synthesizing state-transition in memory. Any valid memory state can be reached from the current state through a series of CRUD operations. This guarantees that, under ideal optimization, the agent's memory can achieve its maximal potential performance.
• Atomic Minimality. Any higher-level memory tool design can be viewed as invoking a structured combination of CRUD operations. In contrast, the invocation of CRUD primitives themselves cannot be further decomposed into other finer-grained tool designs.
• Task-agnosticness. The CRUD action framework is not tied to any specific downstream task. Its ability to complete tasks relies entirely on the LLM’s decision-making capability at a given step, which is itself optimizable. Consequently, this action set constitutes a potential foundation for a general-purpose agent memory.

• 完备性。 CRUD 构成一组能够合成记忆状态转移的通用操作。任意有效记忆状态都可以通过一系列 CRUD 操作从当前状态到达。这保证了在理想优化条件下，智能体记忆能够达到其最大潜在性能。
• 原子最小性。 任意高层记忆工具设计都可视为调用 CRUD 操作的结构化组合。相比之下，CRUD 原语本身的调用无法再分解为其他更细粒度的工具设计。
• 任务无关性。 CRUD 动作框架不与任何特定下游任务绑定。它完成任务的能力完全依赖 LLM 在给定步骤的决策能力，而这种能力本身可以优化。因此，这一动作集合构成了通用智能体记忆的一项潜在基础。

In summary, rather than proposing isolated memory tools, we focus on a complete and general-purpose operator set that governs memory state evolution.

总而言之，我们并非提出孤立的记忆工具，而是关注一套控制记忆状态演化的完备通用操作集合。

3.3 Memory Mechanism Implementation

We model memory at step $t$ as a dynamic set

我们将步骤 $t$ 的记忆建模为动态集合

$${\mathcal{M}}_t=\{m_i{\}}_{i=1}^{N_t},$$

where $m_i$ encodes a stored memory entry. Memory manipulation is exposed as a learnable action space

其中 $m_i$ 编码一个已存储的记忆条目。 记忆操作被公开为可学习动作空间

$${\mathcal{A}}^{mem}=\{\mathrm{Create},\mathrm{Read},\mathrm{Update},\mathrm{Delete}\},$$

where each primitive defines a state transition operator over ${\mathcal{M}}_t$. At each decision step $t$, conditioned on observation $o_t$, the policy generates a sequence of memory actions

其中每个原语都定义了作用于 ${\mathcal{M}}_t$ 的状态转移算子。 在每个决策步骤 $t$，以观测 $o_t$ 为条件，策略生成一系列记忆动作

$${\mathcal{A}}_t=\{a_t^1,\dots ,a_t^{K_t}\},$$

which forms a compositional macro-action within a single environment step. Non-read operations are executed sequentially, yielding a composed transition

这些动作在单个环境步骤内构成组合式宏动作。 非读取操作按顺序执行，产生组合转移

$${\mathcal{M}}_{t+1}=a_t^{K_t}\circ \cdots \circ a_t^1({\mathcal{M}}_t),$$

where $a_t^k\in \{\mathrm{Create},\mathrm{Update},\mathrm{Delete}\}$.

其中 $a_t^k\in \{\mathrm{Create},\mathrm{Update},\mathrm{Delete}\}$。

Hybrid Memory Retrieval. The Read operation does not alter the memory state. Instead, it retrieves information from ${\mathcal{M}}_t$ and produces a memory observation: content requested at step $t-1$ forms the observation at step $t$. We implement a hybrid retrieval mechanism that combines deterministic retrieval with selective query-based retrieval:

混合记忆检索。 Read 操作不会改变记忆状态。 相反，它从 ${\mathcal{M}}_t$ 检索信息并产生记忆观测：在步骤 $t-1$ 请求的内容构成步骤 $t$ 的观测。 我们实现了一种结合确定性检索与选择性查询检索的混合检索机制：

• Deterministic Retrieval (Scratchpad). A special memory entry $m_t^{scr}$ is retrieved at every step. This scratchpad captures the global task state and preserves pivotal information necessary for step-wise decision-making. Functionally, it is identical to other memory entries, differing only in its mandatory retrieval schedule.
• Selective Retrieval. The agent generates a textual query $q_t$ as a tool parameter, and relevant memory entries are retrieved based on semantic similarity. Formally, if ${\mathcal{M}}_t=\{m_1,m_2,\dots ,m_N\}$ is the set of memory entries at step $t$, the retrieved set ${\hat{\mathcal{M}}}_t$ is:

• 确定性检索（草稿区）。 每个步骤都会检索一个特殊记忆条目 $m_t^{scr}$。该草稿区捕获全局任务状态，并保留逐步决策所需的关键信息。在功能上，它与其他记忆条目相同，区别仅在于其强制检索计划。
• 选择性检索。 智能体生成文本查询 $q_t$ 作为工具参数，并依据语义相似度检索相关记忆条目。形式上，如果 ${\mathcal{M}}_t=\{m_1,m_2,\dots ,m_N\}$ 是步骤 $t$ 的记忆条目集合，则检索集合 ${\hat{\mathcal{M}}}_t$ 为：

$${\hat{\mathcal{M}}}_t=\mathrm{TopK}\left(\{m_i\in {\mathcal{M}}_t\mid \mathrm{sim}(q_{t-1},m_i)\}\right).$$

Under this unified formulation, the agent’s observation is given by

在这一统一形式化下，智能体的观测为

$$o_t=\{o_t^{env},m_t^{scr},{\hat{\mathcal{M}}}_t\},$$

where ${\hat{\mathcal{M}}}_t$ denotes selective retrieval, and $m_t^{scr}$ corresponds to deterministic retrieval. The hybrid memory retrieval ranks information by importance: critical information is maintained in the scratchpad, while potentially useful information is stored in the vector database. Experiments show that this multi-path retrieval setup enhances both the effectiveness and robustness of the agent’s memory.

其中 ${\hat{\mathcal{M}}}_t$ 表示选择性检索，$m_t^{scr}$ 对应确定性检索。 混合记忆检索按照重要性对信息分级：关键信息保留在草稿区中，而可能有用的信息存储在向量数据库中。 实验表明，这种多路径检索设置同时增强了智能体记忆的有效性和稳健性。

3.4 Optimization Strategy

Since memory operations are realized as structured tokens in the model's vocabulary, optimizing the output sequence likelihood implicitly optimizes the memory policy. We refine the policy using Group Relative Policy Optimization (GRPO) to master complex memory management in multi-turn scenarios. During RL, each training sample corresponds to a multi-step trajectory $\tau =(o_1,a_1,\dots ,o_T,a_T)$, where $a_t$ includes the task-specific action $a_t^{env}$ and the memory operation $a_t^{mem}$, and the observation $o_t$ is defined above. We use task-level success as the reward signal, i.e., no intermediate rewards are provided and only a terminal reward is assigned at the end of the trajectory. After obtaining the reward, we compute the advantage using the following formulation:

由于记忆操作被实现为模型词表中的结构化 token，优化输出序列似然会隐式优化记忆策略。 我们使用组相对策略优化（GRPO）改进策略，使其掌握多轮场景中的复杂记忆管理。 在 RL 期间，每个训练样本对应一条多步轨迹 $\tau =(o_1,a_1,\dots ,o_T,a_T)$，其中 $a_t$ 包含任务特定动作 $a_t^{env}$ 和记忆操作 $a_t^{mem}$，观测 $o_t$ 如上定义。 我们使用任务级成功作为奖励信号，即不提供中间奖励，只在轨迹结束时分配终局奖励。 获得奖励后，我们使用以下形式计算优势：

$$A_i=r_i-\frac{1}{|G|}\sum _{j\in G}{r}_j.$$

where $G$ denotes the set of trajectories corresponding to repeated executions of the same task, and $r_i$ is the terminal reward of the i-th trajectory. Following Dr.GRPO, we do not apply normalization to the advantages. Finally, the advantage is uniformly distributed across all output tokens in the trajectory and optimized according to the following objective:

其中 $G$ 表示同一任务重复执行所对应的轨迹集合，$r_i$ 是第 $i$ 条轨迹的终局奖励。 按照 Dr.GRPO，我们不对优势进行归一化。 最后，优势被均匀分配到轨迹中的所有输出 token，并根据以下目标进行优化：

$$\begin{array}{rl}\mathcal{J}(\theta )=\mathbb{E}[& \frac{1}{G}\sum _{i=1}^G{\rho }_{\theta }^i{A}_i\\ & -\beta {\mathbb{D}}_{\mathrm{KL}}[{\pi }_{\theta }\parallel {\pi }_{\mathrm{ref}}]].\end{array}$$

Here, ${\rho }_{\theta }^i$ denotes the importance sampling ratio for the i-th sample. Notably, we apply task-level advantages uniformly across all tokens, including memory operations. This enables the agent to jointly optimize memory usage and task performance via RL without external modules.

这里，${\rho }_{\theta }^i$ 表示第 $i$ 个样本的重要性采样比率。 值得注意的是，我们将任务级优势均匀应用于包括记忆操作在内的所有 token。 这使智能体能够在无需外部模块的情况下，通过 RL 联合优化记忆使用和任务性能。

表1：长上下文问答基准和多轮 Web 基准上的结果。

Method	HotpotQA		2WikiMQA		Musique		GAIA	WebWalker	Avg.
	200doc	800doc	200doc	800doc	200doc	800doc
Training-free Methods
Full Context	63.5	62.0	55.7	49.2	42.8	41.9	23.3	29.5	46.0
Vanilla RAG	67.8	63.1	46.5	40.0	38.5	37.1	20.4	24.0	42.2
Generative Agents	38.8	10.0	12.3	2.0	19.8	8.4	22.3	29.5	17.9
Mem0	38.2	33.9	24.2	18.3	14.0	11.2	25.2	28.3	24.2
A-Mem	73.5	70.4	62.7	57.1	47.1	41.6	30.1	29.0	51.4
Trained Methods
MemAgent	76.5	71.1	65.8	57.7	54.7	44.5	33.0	50.0	56.7
AtomMem w/o RL	65.9	60.1	52.8	55.0	47.8	40.0	35.2	45.6	50.3
AtomMem (ours)	77.8	72.9	67.5	62.5	55.1	48.5	37.4	48.7	58.8

4. Experiments

In this section, we first introduce our evaluation task and then present the experimental results.

在本节中，我们首先介绍评估任务，然后给出实验结果。

4.1 Evaluation Tasks

4.1.1 Long Context Benchmarks

We collect 3 QA datasets: HotpotQA, 2WikiMultiHopQA, and MuSiQue as our data sources. All three datasets include training and test splits. We perform RL training on the training split and evaluation on the test split. We feed the document to the model, instructing it to memorize information relevant to the question, and finally require the agent to answer using only the memories. We augment the difficulty of these QA datasets along the following two dimensions.

我们收集 3 个问答数据集作为数据源：HotpotQA、2WikiMultiHopQA 和 MuSiQue。 这三个数据集均包含训练集与测试集。 我们在训练集上执行 RL 训练，并在测试集上评估。 我们把文档输入模型，指示模型记住与问题相关的信息，最后要求智能体仅使用记忆作答。 我们沿以下两个维度提高这些问答数据集的难度。

Long-context Setting. Following the RULER benchmark, we construct arbitrary long-context tasks using the following method: We shuffle the relevant documents and interleave them with a large number of irrelevant documents, constructing a needle-in-a-haystack (NIAH)–style task. This augmentation challenges the agent’s ability to identify and remember important information from massive amounts of input. We train on inputs containing 200 documents (about 28K tokens), and at test time scale the setting to 800 documents (about 112K tokens).

长上下文设置。 按照 RULER 基准，我们使用以下方法构造任意长度的上下文任务： 我们打乱相关文档，并在其中穿插大量无关文档，从而构造大海捞针（NIAH）式任务。 这种增强方式考验智能体从海量输入中识别并记住重要信息的能力。 我们在包含 200 篇文档（约 28K token）的输入上训练，并在测试时将设置扩展到 800 篇文档（约 112K token）。

Multi-question Setting. Following MEM1 and Memory-R1, we provide the model with multiple questions simultaneously. The documents relevant to these questions are shuffled and mixed together before being fed to the agent. After processing all documents, the model is required to answer each question individually. This augmentation strategy challenges the model’s ability to manage and maintain multiple semantically independent memories at the same time. Each task contains a randomly sampled number of questions, ranging from 1 to 10.

多问题设置。 按照 MEM1 和 Memory-R1，我们同时向模型提供多个问题。 与这些问题相关的文档会先被打乱并混合，然后再输入智能体。 处理完所有文档后，模型需要逐一回答每个问题。 这种增强策略考验模型同时管理并维护多个语义相互独立记忆的能力。 每个任务包含随机采样的 1 至 10 个问题。

4.1.2 Web Benchmarks

In multi-turn web search scenarios, we chose the Asearcher open-source dataset for training and evaluated our model on the GAIA and WebWalkerQA datasets. In addition to the memory management API, we provide two external tools—Google search engine and Jina URL Reader—to enable the model to access the internet. The maximum number of web tool calls per task is set to $40$ to fully showcase the contribution of the agent’s memory.

在多轮 Web 搜索场景中，我们选择 Asearcher 开源数据集进行训练，并在 GAIA 和 WebWalkerQA 数据集上评估模型。 除记忆管理 API 外，我们还提供 Google 搜索引擎和 Jina URL Reader 两个外部工具，使模型能够访问互联网。 每个任务最多可调用 $40$ 次 Web 工具，以充分展现智能体记忆的贡献。

4.2 Baselines

We evaluate our method against static baselines with hand-designed strategies mem0, A-Mem, GenerativeAgents. We also compared our approach with MemAgent, another paradigm that employs RL for training Agent Memory. For the implementation of MemAgent, we used the same training hyperparameters and random seed. Comparisons also include standard RAG and a full-context baseline, with implementation details provided in Appendix.

我们将该方法与采用手工策略的静态基线 mem0、A-Mem 和 GenerativeAgents 进行比较。 我们还将该方法与另一种采用 RL 训练智能体记忆的范式 MemAgent 进行比较。 在实现 MemAgent 时，我们使用相同的训练超参数和随机种子。 比较还包括标准 RAG 和完整上下文基线，其实现细节见附录。

4.3 Implementation Details

Models. For all agents, we use Qwen3-8B as the base model. For agents that require retrieval, we use Qwen3-embedding-0.6B as the embedding model.

模型。 对所有智能体，我们均使用 Qwen3-8B 作为基础模型。 对于需要检索的智能体，我们使用 Qwen3-embedding-0.6B 作为嵌入模型。

Agent Implementation. We implement memory using a FAISS vector database as the underlying storage. The query for retrieval is provided by the read action. Each action and its XML format will be detailed in Appendix. Long texts are split into chunks of 4k tokens and fed to the agent step-by-step. At each step, if a read operation is triggered, the memory module retrieves 6 relevant entries from the database.

智能体实现。 我们使用 FAISS 向量数据库作为底层存储来实现记忆。 检索查询由读取动作提供。 每个动作及其 XML 格式详见附录。 长文本被切分为 4k token 的块，并逐步输入智能体。 在每一步中，如果触发读取操作，记忆模块会从数据库检索 6 个相关条目。

RL Implementation. We adopt a fully on-policy RL strategy, where each rollout is used for a single update. For QA tasks, we use exactly match (EM) between the model answer and the ground truth as the reward, and for web tasks, we use an LLM-as-a-judge reward. Additional hyperparameters are provided in the Appendix.

RL 实现。 我们采用完全同策略的 RL 策略，每次 rollout 仅用于一次更新。 对于问答任务，我们使用模型答案与真实答案之间的精确匹配（EM）作为奖励；对于 Web 任务，则使用 LLM-as-a-judge 奖励。 其他超参数见附录。

4.4 Main Results

The main experimental results are shown in Table 1. We highlight the following observations:

主要实验结果如表1所示。 我们强调以下观察：

1. AtomMem achieves superior performance and robust scalability across varying task scales. It outperforms all trained and untrained baselines on average. Notably, in the 800-document setting—a 4$\times$ extension of the training context—our model maintains a significant performance lead. This indicates that the agent has learned a content-aware memory policy capable of mitigating information overload as environmental noise increases. 2. RL training substantially optimizes the agent’s memory policy, resulting in large performance gains. After RL training, AtomMem improves by nearly 9 percentage points on average across different task settings. This improvement indicates that directly optimizing memory decisions with task-level feedback is critical for long-horizon tasks. In particular, RL enables the agent to refine when and how memory operations are applied, leading to markedly stronger end performance.

1. AtomMem 在不同任务规模下取得了更优性能和稳健的可扩展性。 它的平均表现优于所有已训练和未训练基线。尤其是在 800 篇文档设置中——训练上下文的 4 倍扩展——我们的模型仍保持显著的性能领先。这表明，智能体已经学习到内容感知的记忆策略，能够在环境噪声增多时缓解信息过载。 2. RL 训练显著优化了智能体的记忆策略，带来了大幅性能提升。 RL 训练后，AtomMem 在不同任务设置下平均提升近 9 个百分点。这一提升表明，使用任务级反馈直接优化记忆决策对长程任务至关重要。具体而言，RL 使智能体能够改进记忆操作的使用时机和方式，从而显著增强最终性能。

4.5 Training Dynamic Analysis

In this section, we provide a detailed analysis of the RL training dynamics of AtomMem.

在本节中，我们详细分析 AtomMem 的 RL 训练动态。

图3：问答任务 RL 训练期间记忆操作的频率。纵轴表示模型每个步骤平均调用记忆 API 的次数。

As shown in Figure 3, RL training on QA tasks induces systematic changes in the agent’s memory operation usage. Specifically, we have the following findings:

如图3所示，问答任务上的 RL 训练使智能体的记忆操作使用发生系统性变化。 具体而言，我们得到以下发现：

1. The model’s behavior shifts from under-managed to task-aligned memory usage. Early in training, the model over-relies on Read actions and largely neglects memory maintenance, leading to redundant retrievals. As training progresses, Read usage decreases sharply, while Create, Update, and Delete actions increase substantially. This transition indicates that the model learns to maintain a compact, task-relevant memory by preserving useful information, revising outdated entries, and removing redundancy. 2. While the frequency of Update action remains low compared to Create action, they represent the critical few that significantly influence the agent's overall performance. We conduct an ablation study as shown in Table 2, demonstrating that removing Update operations leads to a substantial performance drop across all benchmarks, indicating that selectively revising existing memories is critical for maintaining accurate and compact representations as new evidence arrives. In contrast, disabling Delete has only a marginal impact, suggesting that explicit memory removal is less crucial under the current task, which are largely information-accumulation tasks with non-conflicting facts. To verify this, we added experiments in Appendix, showing that when the maximum number of memory entries is limited, the frequencies of these actions exhibit different trends of change, while the importance of actions such as Update and Delete becomes greater.

1. 模型行为从记忆管理不足转向与任务对齐的记忆使用。 在训练早期，模型过度依赖 Read 动作，并在很大程度上忽视记忆维护，从而造成冗余检索。随着训练推进，Read 的使用量急剧下降，而 Create、Update 和 Delete 动作显著增加。这一转变表明，模型学会了通过保留有用信息、修订过时条目和移除冗余来维护紧凑且与任务相关的记忆。 2. 尽管 Update 动作的频率相较 Create 仍然较低，但它们是显著影响智能体总体性能的关键少数。 我们进行了表2所示的消融研究，结果表明，移除 Update 操作会使所有基准上的性能显著下降，说明随着新证据到来，有选择地修订既有记忆对于维持准确且紧凑的表示至关重要。相比之下，禁用 Delete 只产生轻微影响，这表明在当前任务下显式移除记忆并不那么关键，因为这些任务主要是事实不冲突的信息累积任务。为验证这一点，我们在附录中增加了实验，表明当记忆条目的最大数量受到限制时，这些动作的频率呈现不同的变化趋势，而 Update 和 Delete 等动作的重要性会提高。

表2：记忆操作和记忆组件的消融研究。括号中的数值表示相对性能下降。

Method	HotpotQA	2WikiMQA	Musique
AtomMem	77.8	67.5	55.1
Selective Memory Operations
w/o Update	71.4 (-6.4)	62.6 (-4.9)	47.9 (-7.2)
w/o Delete	76.5 (-1.3)	67.3 (-0.2)	54.2 (-0.9)
Memory Components
w/o scratchpad	71.8 (-6.0)	56.3 (-11.2)	46.0 (-9.1)
w/o storage	69.2 (-8.6)	59.4 (-8.1)	43.9 (-11.2)
w/o Both	25.6 (-52.2)	27.1 (-40.4)	12.1 (-43.0)

图4：在 HotpotQA 上优化 AtomMem 及其多个消融变体的训练曲线。

4.6 Ablations

In this section, we conduct ablation studies on the various memory components of AtomMem and examine the impact of some hyperparameters. Ablation of Memory Component. This experiment investigates the contribution of components to the final performance. The ablation results are reported in Table 2. We can see that:

在本节中，我们对 AtomMem 的多个记忆组件进行消融研究，并考察若干超参数的影响。 记忆组件消融。 该实验研究各组件对最终性能的贡献。 消融结果见表2。 我们可以看到：

1. AtomMem exhibits robustness to the removal of individual memory components. Removing either the scratchpad or the external memory storage leads to a moderate performance drop, whereas removing both results in a catastrophic degradation exceeding 40 points. This suggests that when one component is unavailable, the learned policy can still rely on the remaining component to preserve most task-relevant information, rather than collapsing entirely. This indicates that AtomMem is robust to component-level failures. 2. Both the memory storage and the scratchpad contribute substantially to the final performance of AtomMem. Removing either component leads to a consistent performance drop of 5–10 points across all benchmarks. This indicates that the information preserved by the scratchpad and the external memory storage differ fundamentally in domain and usage, such that neither can be fully substituted by the other.

1. AtomMem 对移除单个记忆组件表现出稳健性。 移除草稿区或外部记忆存储都会造成中等程度的性能下降，而同时移除两者会导致超过 40 个百分点的灾难性退化。这表明，当一个组件不可用时，学习得到的策略仍可依赖剩余组件保留大部分任务相关信息，而不会完全崩溃。这说明 AtomMem 对组件级故障具有稳健性。 2. 记忆存储和草稿区都对 AtomMem 的最终性能作出重要贡献。 移除任一组件都会使所有基准上的性能持续下降 5–10 个百分点。这表明草稿区与外部记忆存储所保留的信息在领域和用途上存在根本差异，因此两者均无法被另一方完全替代。

To verify this, we trained another two variants of AtomMem from scratch: scratchpad-only and storage-only. The results are shown in Figure 4. From the experimental results, we observe that the other two variants do not achieve performance comparable to AtomMem. The scratchpad-only variant remains consistently below AtomMem during training, whereas the storage-only variant benefits marginally from RL. This indicates that our design effectively raises the performance ceiling of the agent. The significant gap between AtomMem and its variants suggests that the synergy between the scratchpad and memory storage is a structural necessity for handling complex tasks.

为验证这一点，我们从头训练了 AtomMem 的另外两个变体：仅草稿区和仅存储。 结果如图4所示。 从实验结果中，我们观察到 另外两个变体都未达到与 AtomMem 相当的性能。 在训练过程中，仅草稿区变体的表现始终低于 AtomMem，而仅存储变体从 RL 中获得的收益很小。 这表明我们的设计有效提高了智能体的性能上限。 AtomMem 与其变体之间的显著差距说明，草稿区与记忆存储之间的协同是处理复杂任务的结构性必要条件。

Effect of Hyper-Parameters. In this experiment, we investigate the effect of several key hyperparameters of AtomMem, including chunk size and retrieve number. The chunk size determines the length of the text segment processed by the agent at each step, while the retrieve number specifies how many entries are retrieved from storage at each step. The experimental results are shown in Table 3. From the results, we find that:

超参数的影响。 在该实验中，我们研究 AtomMem 的若干关键超参数，包括块大小和检索数量。 块大小决定智能体在每个步骤处理的文本片段长度，而检索数量规定每个步骤从存储中检索多少条目。 实验结果如表3所示。 根据结果，我们发现：

1. The retrieval size $K$ must match the task’s memory demand. Reducing $K$ from 6 to 3 causes a clear performance drop, while increasing it to 12 brings little benefit. This is because the evaluated benchmarks require only 2–4 hop reasoning, for which retrieving about six documents is sufficient. 2. AtomMem is robust to chunk size. Performance remains consistent across different chunk sizes, due to the base model’s strong long-context understanding and reinforcement learning that enables effective information extraction at varying granularities.

1. 检索数量 $K$ 必须与任务的记忆需求匹配。 将 $K$ 从 6 降至 3 会导致明显性能下降，而将其提高至 12 几乎没有收益。这是因为所评估的基准只需要 2–4 跳推理，检索约 6 篇文档已经足够。 2. AtomMem 对块大小具有稳健性。 不同块大小下的性能保持稳定，这得益于基础模型强大的长上下文理解能力，以及强化学习使模型能够在不同粒度下有效抽取信息。

表3：块大小 C 和检索数量 K 的超参数分析。

C	K	HotpotQA	2WikiMQA	Musique	Avg.
2048	3	76.2	64.2	49.9	63.4
2048	6	76.8	67.6	52.6	65.7
2048	12	75.0	67.3	52.6	65.0
4096	3	74.2	64.8	54.6	64.5
4096	6	76.9	67.5	55.1	66.5
4096	12	77.4	69.6	54.5	67.2
8192	3	76.4	65.2	53.0	64.9
8192	6	78.7	67.9	54.0	66.9
8192	12	76.5	67.5	52.8	65.6

Sensitivity to Embeddings. We evaluate the impact of embedding models on task performance. The experimental results are shown in the Table 4. Frow which we find that:

对嵌入模型的敏感性。 我们评估嵌入模型对任务性能的影响。 实验结果见表4。 由此我们发现：

1. The learned embeddings consistently outperforms random selection across all three tasks. Random selection suffers an average drop of 7.4 percentage points compared to Qwen3-embedding-0.6B. This significant gap indicates that similarity-based matching plays an important role in task performance. 2. The Qwen3 embeddings show a clear trend of improvement as model size increases. Qwen3-embedding-0.6B achieves an average score of 66.5, while the 4B and 8B variants reach 67.2 and 68.2, respectively. This indicates that higher-capacity embedding models can better capture semantic information relevant to the tasks. Overall, the results highlight the importance of choosing an appropriate embedding model for downstream performance.

1. 在全部三个任务上，学习得到的嵌入都持续优于随机选择。 与 Qwen3-embedding-0.6B 相比，随机选择的平均分下降 7.4 个百分点。这一显著差距表明，基于相似度的匹配在任务性能中发挥重要作用。 2. 随着模型规模增大，Qwen3 嵌入呈现明确的提升趋势。 Qwen3-embedding-0.6B 的平均得分为 66.5，而 4B 和 8B 变体分别达到 67.2 和 68.2。这表明，更高容量的嵌入模型能够更好地捕获与任务相关的语义信息。总体而言，结果突显了为下游性能选择适当嵌入模型的重要性。

表4：嵌入模型敏感性分析。

Embedding Model	HQA	2Wiki	Musi.	Avg.
Random Select	68.5	62.6	46.1	59.1
Qwen3-embedding-0.6B	76.9	67.5	55.1	66.5
Qwen3-embedding-4B	77.5	68.2	56.0	67.2
Qwen3-embedding-8B	78.6	69.6	56.3	68.2

5. Case Study

图5：该案例说明，当面对不同任务上下文时，模型会采用不同的记忆管理策略，体现了 AtomMem 的动态特性。

In this section, we analyze the model’s responses on a case-by-case basis to understand what memory workflow the model has learned. As illustrated in Figure 5, we present three scenarios at step $n$ that demonstrate the agent’s learned ability to adapt its memory workflow based on the observation $o_t^{env}$. The example is from HotpotQA, where the LLM made different decisions to complete the task depending on the timing and order in which the key documents appeared.

在本节中，我们逐例分析模型响应，以理解模型学习到了何种记忆工作流。 如图5所示，我们给出步骤 $n$ 的三个场景，展示智能体依据观测 $o_t^{env}$ 调整记忆工作流的学习能力。 该示例来自 HotpotQA，其中 LLM 会根据关键文档出现的时机和顺序作出不同决策来完成任务。

• Case 1: When $o_t^{env}$ contains unrelated documents, the agent uses the scratchpad to log the absence of relevant info and only stores potentially related background entries.
• Case 2: When $o_t^{env}$ provides partial information (e.g., the release date of a single film), the agent commits the newly found evidence to memory and proactively generates a <read_memory> request to retrieve the missing piece.
• Case 3: In the scenario where all required information is present, the agent synthesizes the retrieved facts within the scratchpad to derive the final answer and uses <update_memory> to overwrite useless entries with the conclusion.

• 案例 1： 当 $o_t^{env}$ 包含无关文档时，智能体使用草稿区记录没有相关信息，并且只存储可能相关的背景条目。
• 案例 2： 当 $o_t^{env}$ 提供部分信息（例如一部电影的上映日期）时，智能体将新发现的证据写入记忆，并主动生成 <read_memory> 请求以检索缺失信息。
• 案例 3： 当所需信息全部存在时，智能体在草稿区中综合检索到的事实以得出最终答案，并使用 <update_memory> 将无用条目覆写为结论。

Together, these cases illustrate that the agent has learned a context-sensitive memory workflow, dynamically deciding when to ignore, retrieve, update, or consolidate memories based on the informational sufficiency of the current observation.

这些案例共同说明，智能体已经学习到 上下文敏感 的记忆工作流，能够根据当前观测中的信息充分程度，动态决定何时忽略、检索、更新或整合记忆。

6. Conclusion

In this paper, we propose AtomMem, which reframes agentic memory management as a dynamic decision-making problem by deconstructing complex workflows into atomic CRUD operations. By optimizing this learnable decision process, AtomMem moves beyond the limitations of static, "one-size-fits-all" memory pipelines. Experimental results and training dynamics demonstrate that this approach enables a task-aligned memory policy.

在本文中，我们提出 AtomMem，通过将复杂工作流分解为原子 CRUD 操作，把智能体记忆管理重新表述为动态决策问题。 通过优化这一可学习决策过程，AtomMem 超越了静态“一体适用”记忆流水线的局限。 实验结果和训练动态表明，该方法能够形成与任务对齐的记忆策略。

Limitation

Despite its effectiveness, RL optimization is computationally intensive. Training an agent model to convergence requires approximately 2 to 3 days on an 8-GPU cluster. This computational overhead may become a bottleneck when scaling our approach to even longer-horizon or noisier tasks.

尽管该方法有效，但 RL 优化的计算成本很高。 在 8 GPU 集群上将智能体模型训练至收敛大约需要 2 至 3 天。 当把我们的方法扩展到时间跨度更长或噪声更大的任务时，这种计算开销可能成为瓶颈。

In reinforcement learning, task-level advantages are typically evenly distributed across all actions. However, in reality, there should exist more precise methods for assigning advantages, aiming to measure the contribution of each memory entry to the successful completion of a task. We do not explore this direction in the current work for two reasons. First, developing a new RL algorithm for a single downstream task (agent memory) would be unnecessary, as we have already demonstrated that conventional RL methods can effectively optimize performance. Second, accurately evaluating the value of each memory entry is not a trivial problem, and a clear methodology for doing so remains elusive. We leave this problem for future work.

在强化学习中，任务级优势通常被均匀分配给所有动作。 然而，现实中应当存在更精确的优势分配方法，用于衡量每个记忆条目对成功完成任务的贡献。 我们在当前工作中没有探索这一方向，原因有二。 第一，没有必要为单个下游任务（智能体记忆）开发新的 RL 算法，因为我们已经证明常规 RL 方法可以有效优化性能。 第二，准确评估每个记忆条目的价值并非易事，目前仍缺少清晰的方法。 我们将这一问题留待未来研究。

Ethical Statement

All data used in this work are sourced from open-source datasets and do not contain personal or private information. The LLM is used solely for writing and sentence refinement.

本工作使用的所有数据均来自开源数据集，不包含个人或隐私信息。 LLM 仅用于写作和句子润色。