---

Verifiable Memory: Learning Unified Memory Management with Local and Global Verifiers for Large Language Model Agents

MemoryAgentRL中山大学南洋理工大学腾讯

Sun X, Wang Q, Li H, et al. Verifiable Memory: Learning Unified Memory Management with Local and Global Verifiers for Large Language Model Agents[EB/OL]. arXiv:2608.03137, 2026.

https://github.com/Sun-SYSU-24/VerMem

---

可验证记忆：通过局部与全局验证器为大语言模型智能体学习统一记忆管理

Abstract

Large language model (LLM) agents must retain reusable information, control a bounded active context, and recover earlier evidence during long-horizon interaction. Existing methods commonly optimize long-term memory (LTM) and short-term memory (STM) separately, while unified policies are often trained primarily with trajectory-level feedback, which provides weak credit for individual memory decisions. We present Verifiable Memory (VerMem), a framework that represents LTM, active context, and episodic history as distinct states and controls them with one memory operation policy. Seven atomic operations let the policy add, revise, or soft-delete LTM entries; retrieve LTM into the active context; filter or summarize the active context; and restore selected episodic fragments. VerMem is initialized by supervised fine-tuning and trained with a three-stage reinforcement-learning curriculum. The local verifier scores executable memory transitions, and a global verifier assesses evidence coherence and terminal-memory consistency after task completion. These scores are combined with programmatically computed task, evidence-recall, efficiency, and constraint signals through hierarchical credit assignment. The verifiers are used only during training. Across five benchmarks and two LLM backbones, VerMem achieves the best result on the vast majority of reported metrics and consistently outperforms strong memory baselines. Under controlled online-token budgets on three interactive benchmarks, it also achieves the strongest efficiency--performance frontier among the compared methods.

大语言模型（LLM）智能体必须在长程交互中保留可复用信息、控制有界的活动上下文，并恢复早期证据。 现有方法通常分别优化长期记忆（LTM）和短期记忆（STM），而统一策略往往主要使用轨迹级反馈训练，这对单个记忆决策只能提供较弱的信用信号。 我们提出可验证记忆（Verifiable Memory，VerMem），该框架将 LTM、活动上下文和情景历史表示为不同状态，并用一个记忆操作策略控制它们。 七种原子操作使该策略能够添加、修订或软删除 LTM 条目；把 LTM 检索到活动上下文中；过滤或总结活动上下文；以及恢复选定的情景片段。 VerMem 先通过监督微调初始化，再使用三阶段强化学习课程训练。 局部验证器为可执行的记忆转移评分，而全局验证器在任务完成后评估证据连贯性和终端记忆一致性。 这些分数通过层级信用分配，与程序化计算的任务、证据召回、效率和约束信号相结合。 验证器仅在训练期间使用。 在五个基准和两个 LLM backbone 上，VerMem 在绝大多数报告指标上取得最佳结果，并持续优于强记忆基线。 在三个交互式基准的受控在线 token 预算下，它还取得了所有比较方法中最强的效率--性能前沿。

1. Introduction

In long-horizon tasks involving multi-step interaction, tool use, and complex reasoning, the performance of large language model (LLM) agents depends not only on single-step generation, but also on retaining and using task-relevant information across decisions. Prior work uses agent memory to describe the information that an agent can attend to and use at a given time. Such information includes stored past executions and state representations formed from interaction history. Agent memory is commonly divided into long-term memory and short-term memory. Long-term memory (LTM) persistently stores user- or task-specific knowledge for reuse in later turns, stages, or tasks. Short-term memory (STM) contains information in the current input context and supports ongoing reasoning, action selection, and context control. LTM preserves information across time, while STM keeps the current reasoning state usable. Their coordination is therefore central to long-horizon agent reasoning.

在涉及多步交互、工具使用和复杂推理的长程任务中，大语言模型（LLM）智能体的性能不仅取决于单步生成，还取决于跨决策保留和使用与任务相关的信息。 先前工作使用智能体记忆来描述智能体在给定时间能够关注和使用的信息。 这些信息包括存储的过去执行过程，以及由交互历史形成的状态表示。 智能体记忆通常分为长期记忆和短期记忆。 长期记忆（LTM）持久存储用户或任务特定知识，以便在之后的轮次、阶段或任务中复用。 短期记忆（STM）包含当前输入上下文中的信息，并支持正在进行的推理、动作选择和上下文控制。 LTM 跨时间保留信息，而 STM 让当前推理状态保持可用。 因此，二者的协调是长程智能体推理的核心。

Existing systems often optimize only one side of this process. STM-oriented methods organize or compress the current reasoning state, whereas LTM-oriented methods build external stores for persistent recall. Stored information, however, affects a decision only after it enters the active context at the right time, and reusing task-misaligned experience can propagate errors. The AgeMem policy introduced by Yu et al. places LTM and STM operations in one policy, but its STM actions focus on retrieval, filtering, and summarization. Long-horizon tasks may also require an earlier observation, tool output, or intermediate conclusion to return from episodic history after it has left the active context. This operation differs from retrieving persistent knowledge and from compressing recent context.

现有系统往往只优化这一过程的一侧。 面向 STM 的方法组织或压缩当前推理状态，而面向 LTM 的方法构建用于持久召回的外部存储。 然而，存储的信息只有在正确时机进入活动上下文后才会影响决策，而且复用与任务不匹配的经验可能传播错误。 Yu 等人提出的 AgeMem 策略将 LTM 与 STM 操作放在一个策略中，但其 STM 动作聚焦于检索、过滤和总结。 长程任务还可能要求某个早期观察、工具输出或中间结论在离开活动上下文后，从情景历史重新返回。 这一操作不同于检索持久知识，也不同于压缩近期上下文。

Learning such behavior introduces a second difficulty. Yan et al., Zhou et al., and Yu et al. optimize memory decisions with downstream outcomes or trajectory-level feedback. These signals compare complete rollouts, but do not identify which atomic transition was useful or harmful. Process supervision and transition-level credit improve intermediate feedback, while MiroMind Team combines local and global verification during inference. Operation-level and trajectory-level verification have not been jointly used to train a unified memory operation policy.

学习这类行为带来了第二项困难。 Yan 等人、Zhou 等人和 Yu 等人使用下游结果或轨迹级反馈优化记忆决策。 这些信号比较完整 rollout，却无法识别哪个原子转移有用或有害。 过程监督和转移级信用改善了中间反馈，而 MiroMind Team 在推理期间结合局部与全局验证。 此前尚未联合使用操作级与轨迹级验证来训练统一记忆操作策略。

These limitations leave three challenges for a unified and trainable memory operation policy. (A) Heterogeneous memory coordination. LTM determines what should be stored, revised, or removed. STM determines what should enter, remain in, or leave the active context. The two memory types operate over different states and time scales. Coordinating them within a shared decision process while preserving their functional boundaries remains difficult. (B) Historical-context recovery. The current prompt is only a budgeted view of the full task history. Early observations, tool outputs, and intermediate conclusions may become relevant again after the subgoal changes. Recovering the relevant fragments without reintroducing large amounts of irrelevant history is therefore nontrivial. (C) Multi-granularity credit assignment. The utility of a memory operation is often delayed and depends on later operations. The correct write may become useful only after a later retrieval, while an incorrect filter may cause failure several steps later. Training must assign operation-specific credit while preserving alignment with the complete task objective.

这些限制为统一且可训练的记忆操作策略留下了三项挑战。 （A）异构记忆协调。 LTM 决定应存储、修订或移除什么。 STM 决定什么应进入、保留在或离开活动上下文。 两种记忆类型作用于不同状态和时间尺度。 在保持其功能边界的同时，在共享决策过程中协调二者仍然困难。 （B）历史上下文恢复。 当前 prompt 只是完整任务历史的一个受预算限制的视图。 早期观察、工具输出和中间结论可能在子目标改变后再次变得相关。 因此，要恢复相关片段而不重新引入大量无关历史并不容易。 （C）多粒度信用分配。 记忆操作的效用往往会延迟显现，并取决于之后的操作。 正确的写入可能只有在后续检索后才变得有用，而错误的过滤可能在数步之后导致失败。 训练必须在保持与完整任务目标对齐的同时，分配操作特定的信用。

To address these challenges, we propose Verifiable Memory (VerMem), shown in Figure 1. VerMem preserves LTM, active context, and episodic history as distinct states, while one memory operation policy coordinates LTM maintenance, STM control, and historical-context recovery through seven atomic tools. Training uses a supervised fine-tuning (SFT) warmup followed by a three-stage reinforcement learning curriculum for LTM, STM, and their joint use. The local verifier evaluates each realized memory transition, and the global verifier evaluates the completed trajectory. Their separately normalized advantages are combined at every memory decision. Both verifiers are removed during inference. Across five long-horizon benchmarks and two backbones, VerMem consistently improves task performance and achieves a stronger efficiency--performance frontier. The main contributions of this work are summarized as follows:

为应对这些挑战，我们提出图1所示的可验证记忆（VerMem）。 VerMem 将 LTM、活动上下文和情景历史保持为不同状态，同时由一个记忆操作策略通过七种原子工具协调 LTM 维护、STM 控制和历史上下文恢复。 训练先进行监督微调（SFT）预热，再针对 LTM、STM 及其联合使用执行三阶段强化学习课程。 局部验证器评估每次实际发生的记忆转移，全局验证器评估完成的轨迹。 二者分别归一化的优势会在每个记忆决策处结合。 两个验证器在推理期间都会被移除。 在五个长程基准和两个 backbone 上，VerMem 持续提升任务性能，并取得更强的效率--性能前沿。 本工作的主要贡献总结如下：

• We propose VerMem, a unified agent memory management framework. VerMem coordinates LTM maintenance and STM control through a single memory operation policy while preserving distinct states for LTM, active context, and episodic history. Its atomic action space further supports episode-level context selection.
• We introduce local--global verifier-guided hierarchical credit assignment. VerMem constructs operation-level local advantages and trajectory-level global advantages from stateful multi-step rollouts. The two signals are combined at each memory decision, allowing operation quality and final task utility to jointly guide policy updates.
• We conduct systematic evaluations across diverse complex tasks. We compare VerMem with strong memory baselines on five benchmarks and evaluate its efficiency--performance trade-off under controlled online-token budgets. Ablations further examine unified LTM/STM management, local and global credit signals, and the multi-component reward design.

• 我们提出 VerMem，一个统一的智能体记忆管理框架。 VerMem 通过单一记忆操作策略协调 LTM 维护与 STM 控制，同时为 LTM、活动上下文和情景历史保留不同状态。其原子动作空间还支持情景级上下文选择。
• 我们引入由局部--全局验证器引导的层级信用分配。 VerMem 从有状态的多步 rollout 构造操作级局部优势和轨迹级全局优势。两个信号会在每个记忆决策处结合，使操作质量和最终任务效用能够共同指导策略更新。
• 我们在多种复杂任务上进行系统评估。 我们在五个基准上将 VerMem 与强记忆基线进行比较，并在受控在线 token 预算下评估其效率--性能权衡。消融还进一步考察统一 LTM/STM 管理、局部与全局信用信号，以及多组件奖励设计。

2. Related Work

2.1 Long-Term Memory (LTM)

Long-term memory research studies how agents preserve, organize, and reuse historical information beyond the active context. LangChain Team describes LangMem as a set of modular primitives for recording, searching, and consolidating persistent knowledge. Zhong et al. introduce MemoryBank for long-term recall and user profiling through continual memory updates. The Mem0 framework of Chhikara et al. extracts, consolidates, and retrieves salient information from extended conversations, while the A-Mem framework of Xu et al. organizes memory through dynamic indexing, linking, and evolution. These systems improve persistent-state construction, but their primary concern is how information is stored and organized. Stored knowledge affects current decisions only after it enters the active context at the appropriate stage. Incorrect or task-misaligned experiences may also propagate errors when reused. This motivates joint control of persistent-memory maintenance and active-context use rather than isolated optimization of LTM.

长期记忆研究探讨智能体如何保存、组织和复用活动上下文之外的历史信息。 LangChain Team 将 LangMem 描述为一组用于记录、搜索和整合持久知识的模块化原语。 Zhong 等人提出 MemoryBank，通过持续记忆更新支持长期召回和用户画像。 Chhikara 等人的 Mem0 框架从扩展对话中提取、整合和检索显著信息，而 Xu 等人的 A-Mem 框架通过动态索引、链接和演化来组织记忆。 这些系统改进了持久状态构建，但其主要关注点是信息如何存储和组织。 存储的知识只有在适当阶段进入活动上下文后才会影响当前决策。 错误或与任务不匹配的经验在复用时也可能传播错误。 这促使我们联合控制持久记忆维护和活动上下文使用，而不是孤立优化 LTM。

2.2 Short-Term Memory (STM)

Short-term memory research focuses on maintaining the active context required by ongoing reasoning under a bounded context budget. Qian et al. organize reasoning traces and transient tool outputs into a compact executive memory. Li et al. develop structured schemata and activate query-relevant information for long-document understanding. Wu et al. periodically compress growing interaction histories into concise reasoning states and further adapt agents to reason over these summaries with ReSum-GRPO. These methods improve context usability, but they remain centered on the current input, document, or ongoing trajectory. In long-horizon tasks, useful evidence may remain in the episodic history after it leaves the active context. Retrieval, filtering, and summarization alone do not fully specify which earlier episode should return to the current reasoning state. VerMem addresses this gap by selecting task-relevant fragments from episodic history and materializing them into the active context as an explicit STM decision.

短期记忆研究聚焦于在有界上下文预算下，维护正在进行的推理所需的活动上下文。 Qian 等人将推理轨迹和临时工具输出组织成紧凑的执行记忆。 Li 等人构建结构化模式，并激活与查询相关的信息以理解长文档。 Wu 等人定期把不断增长的交互历史压缩为简洁推理状态，并进一步用 ReSum-GRPO 使智能体适应基于这些摘要进行推理。 这些方法改善了上下文可用性，但仍以当前输入、文档或正在进行的轨迹为中心。 在长程任务中，有用证据离开活动上下文后仍可能保留在情景历史中。 仅靠检索、过滤和总结，无法完整指定哪个早期情景应返回当前推理状态。 VerMem 通过从情景历史中选择与任务相关的片段，并将其作为显式 STM 决策放入活动上下文来填补这一空缺。

2.3 Learning Memory Policies with Verification

Reinforcement learning turns memory management into a learnable decision process. Yan et al. learn structured updates, Huo et al. expose atomic memory operations, Zhou et al. jointly learn consolidation and reasoning, and Yu et al. train a unified LTM/STM policy. These methods primarily optimize memory behavior with answer-level or trajectory-level objectives. Process supervision evaluates intermediate reasoning progress, while Luo et al. and Peng et al. assign credit to agent transitions or hierarchical decisions. MiroMind Team applies local and global verification during inference. VerMem instead uses operation-level and trajectory-level verification signals to train the memory operation policy.

强化学习将记忆管理转化为可学习的决策过程。 Yan 等人学习结构化更新，Huo 等人公开原子记忆操作，Zhou 等人联合学习整合与推理，Yu 等人则训练统一的 LTM/STM 策略。 这些方法主要使用答案级或轨迹级目标优化记忆行为。 过程监督评估中间推理进展，而 Luo 等人和 Peng 等人将信用分配给智能体转移或层级决策。 MiroMind Team 在推理期间应用局部与全局验证。 相比之下，VerMem 使用操作级和轨迹级验证信号来训练记忆操作策略。

3. Method

We propose Verifiable Memory (VerMem), a unified framework that maintains persistent LTM, a budgeted active context, and episodic history as distinct states. As shown in Figure 1, one memory operation policy coordinates LTM maintenance, STM control, and historical-context recovery. The policy is initialized with SFT and optimized through a three-stage reinforcement learning curriculum. The local verifier supplies operation-level semantic scores, while the composite global branch supplies trajectory-level credit from programmatic and global-verifier components. Both verifiers are removed during inference.

我们提出可验证记忆（VerMem），这是一个统一框架，将持久 LTM、受预算限制的活动上下文和情景历史维护为不同状态。 如图1所示，一个记忆操作策略负责协调 LTM 维护、STM 控制和历史上下文恢复。 该策略通过 SFT 初始化，并通过三阶段强化学习课程优化。 局部验证器提供操作级语义分数，而复合全局分支从程序化组件和全局验证器组件提供轨迹级信用。 两个验证器在推理期间都会被移除。

图1：VerMem 概览。左：记忆操作策略协调长期与短期记忆工具，以维护长期记忆和活动上下文。历史上下文恢复为 LLM 恢复相关情景历史，其动作和回答会被记录供后续步骤使用。右：三阶段课程生成 K 条有状态 rollout。局部验证器评估可执行的记忆转移，全局验证器在任务完成后评估证据连贯性和终端记忆一致性。这些分数分别形成归一化的局部与全局优势；约束成本保持独立。验证器分支仅在训练期间使用。

3.1 Problem Formulation

Unified memory operation policy formulation. The fixed task specification is denoted by $q$. At memory-decision step $t$, the long-term memory $M_t$, active context $C_t$, and same-task episodic history $H_t$ define the conceptual state and its policy-visible serialization:

统一记忆操作策略形式化。 固定任务规范记为 $q$。 在记忆决策步骤 $t$，长期记忆 $M_t$、活动上下文 $C_t$ 和同任务情景历史 $H_t$ 定义概念状态及策略可见的序列化：

$$\begin{array}{}\text{(1)}& \begin{array}{rl}{s}_t& =(q,M_t,C_t,H_t),\\ x_t& =g(s_t).\end{array}\end{array}$$

Here, $M_t$ stores persistent information for reuse in later steps or stages, $C_t$ is the budgeted context visible to the task solver, and $H_t$ preserves the ordered record of observations, task actions, tool outputs, intermediate conclusions, and memory decisions from the same task. Separating $C_t$ from $H_t$ allows information to leave the active context without being removed from that record. The function $g$ retains the task specification and current task state, then serializes budgeted views of $M_t$, $C_t$, and same-task $H_t$ within the 8,192-token policy-state limit used by the protocol. It does not concatenate $H_t$ without bound. Equation (1) therefore distinguishes the conceptual state from the bounded input presented to the policy.

其中，$M_t$ 存储供后续步骤或阶段复用的持久信息，$C_t$ 是任务求解器可见的受预算限制上下文，$H_t$ 则保留来自同一任务的观察、任务动作、工具输出、中间结论和记忆决策的有序记录。 将 $C_t$ 与 $H_t$ 分开，使信息能够离开活动上下文而不从该记录中移除。 函数 $g$ 保留任务规范和当前任务状态，然后在协议采用的 8,192-token 策略状态限制内，对 $M_t$、$C_t$ 和同任务 $H_t$ 的预算化视图进行序列化。 它不会无界拼接 $H_t$。 因此，公式 (1) 区分了概念状态与呈现给策略的有界输入。

Let $v_t$ denote an operation type and ${\xi }_t$ its structured arguments. The complete generated memory command is $u_t=(v_t,{\xi }_t)$, where $v_t\in \mathcal{V}\cup \{\varnothing \}$, and $\mathcal{V}$ denotes the seven operation types formalized in Equation (6). The null command has empty arguments. The complete command and memory transition are

令 $v_t$ 表示操作类型，${\xi }_t$ 表示其结构化参数。 生成的完整记忆命令为 $u_t=(v_t,{\xi }_t)$，其中 $v_t\in \mathcal{V}\cup \{\varnothing \}$，而 $\mathcal{V}$ 表示公式 (6) 中形式化的七种操作类型。 空命令的参数为空。 完整命令和记忆转移为

$$\begin{array}{}\text{(2)}& \begin{array}{rl}{u}_t& =(v_t,{\xi }_t),\\ u_t& \sim {\pi }_{\theta }(\cdot \mid x_t),\\ {\tilde{s}}_t& ={\mathcal{T}}_m(s_t,u_t).\end{array}\end{array}$$

For a valid non-null command, ${\mathcal{T}}_m$ commits the specified memory transition. For $v_t=\varnothing$, $M_t$ and $C_t$ are unchanged, and the memory-state transition is an identity transition, so ${\tilde{s}}_t=s_t$. The identity decision is retained in the rollout record but does not create a memory-state modification. The null command remains a trainable decision in the rollout record. Rejected non-null commands leave $M_t$ and $C_t$ unchanged and record the failed decision and its reason in $H_t$. They also incur a constraint cost. Thus, Equation (2) defines ${\tilde{s}}_t$ on every branch.

对于有效的非空命令，${\mathcal{T}}_m$ 提交指定的记忆转移。 当 $v_t=\varnothing$ 时，$M_t$ 和 $C_t$ 保持不变，记忆状态转移为恒等转移，因此 ${\tilde{s}}_t=s_t$。 恒等决策会保留在 rollout 记录中，但不会修改记忆状态。 空命令仍是 rollout 记录中的可训练决策。 被拒绝的非空命令保持 $M_t$ 和 $C_t$ 不变，并在 $H_t$ 中记录失败的决策及其原因。 它们还会产生约束成本。 因此，公式 (2) 在每个分支上都定义了 ${\tilde{s}}_t$。

After the memory transition, the task solver acts on the resulting active context, the environment returns an observation, and a separate interaction transition constructs the next memory-decision state:

记忆转移之后，任务求解器根据所得活动上下文采取动作，环境返回观察，再由单独的交互转移构造下一个记忆决策状态：

$$\begin{array}{}\text{(3)}& \begin{array}{rl}{b}_t& \sim {\pi }_{\varphi }^{\mathrm{task}}(\cdot \mid q,{\tilde{C}}_t),\\ z_{t+1}& \sim {\mathcal{P}}_{\mathrm{env}}(\cdot \mid b_t),\\ s_{t+1}& ={\mathcal{T}}_e({\tilde{s}}_t,b_t,z_{t+1}).\end{array}\end{array}$$

The transition ${\mathcal{T}}_e$ appends the task action and resulting observation to $H_t$ and constructs the next active context. The memory transition in Equation (2) and the task/environment transition in Equation (3) have distinct roles. Repeating them produces a task trajectory $\tau$. Unified management shares the state and task objective but does not merge LTM, active context, and episodic history into one storage structure.

转移 ${\mathcal{T}}_e$ 将任务动作和所得观察追加到 $H_t$，并构造下一个活动上下文。 公式 (2) 的记忆转移和公式 (3) 的任务/环境转移承担不同作用。 重复执行它们会产生任务轨迹 $\tau$。 统一管理共享状态和任务目标，但不会把 LTM、活动上下文和情景历史合并成一种存储结构。

Progressive training and hierarchical credit. The policy follows

渐进式训练与层级信用。 该策略遵循

$$\begin{array}{}\text{(4)}& {\theta }_{\mathrm{SFT}}\to {\theta }_A\to {\theta }_B\to {\theta }_C={\theta }^{\star }.\end{array}$$

Phase A trains LTM maintenance, Phase B trains STM control under distractors, and Phase C trains their coordination in complete tasks. Equation (4) shows the parameter curriculum. For decision $t$ in trajectory $k$, the local verifier produces a raw operation score for an executable command, while the global branch combines post-trajectory verifier scores with programmatic task, evidence-recall, and efficiency signals. Normalization converts the two branches into $A_{t,k}^{\mathrm{local}}$ and $A_k^{\mathrm{global}}$. The executor separately records hard violations as $c_{t,k}$. Their credit is

阶段 A 训练 LTM 维护，阶段 B 在干扰项下训练 STM 控制，阶段 C 在完整任务中训练二者协调。 公式 (4) 展示了参数课程。 对于轨迹 $k$ 中的决策 $t$，局部验证器为可执行命令产生原始操作分数，而全局分支将轨迹后验证器分数与程序化任务、证据召回和效率信号结合。 归一化将两个分支转化为 $A_{t,k}^{\mathrm{local}}$ 和 $A_k^{\mathrm{global}}$。 执行器另行将硬性违规记录为 $c_{t,k}$。 它们的信用为

$$\begin{array}{}\text{(5)}& \begin{array}{rl}{A}_{t,k}^{\mathrm{hier}}=& {\lambda }_{\mathrm{local}}{A}_{t,k}^{\mathrm{local}}+{\lambda }_{\mathrm{global}}{A}_k^{\mathrm{global}}\\ & -{\lambda }_{\mathrm{constraint}}{c}_{t,k}.\end{array}\end{array}$$

We set ${\lambda }_{\mathrm{local}}={\lambda }_{\mathrm{global}}={\lambda }_{\mathrm{constraint}}=1$. The local term distinguishes executable commands within one trajectory, while the global term preserves alignment with the completed task. Equation (5) keeps the local, global, and constraint channels distinct, including for rejected commands.

我们设定 ${\lambda }_{\mathrm{local}}={\lambda }_{\mathrm{global}}={\lambda }_{\mathrm{constraint}}=1$。 局部项区分同一轨迹内的可执行命令，而全局项保持与已完成任务的对齐。 公式 (5) 保持局部、全局和约束通道彼此独立，对被拒绝的命令也同样如此。

3.2 Atomic Memory Tools

The operation-type sets are

操作类型集合为

$$\begin{array}{}\text{(6)}& \begin{array}{rl}{\mathcal{V}}_{\mathrm{LTM}}& =\{\mathrm{Add},\mathrm{Update},\mathrm{Delete}\},\\ {\mathcal{V}}_{\mathrm{STM}}& =\{\mathrm{Retrieve},\mathrm{Filter},\\ & \mathrm{SelectEpisode},\mathrm{Summarize}\},\\ \mathcal{V}& ={\mathcal{V}}_{\mathrm{LTM}}\cup {\mathcal{V}}_{\mathrm{STM}}.\end{array}\end{array}$$

VerMem exposes these seven atomic tools, summarized in Table 1 and formalized in Equation (6), together with the null action $\varnothing$. The LTM tools add, revise, or soft-delete persistent information. The STM tools retrieve LTM content, filter or summarize the active context, and restore relevant same-task fragments from $H_t$ through SelectEpisode. The unified action space lets the policy choose between persistent-memory maintenance and active-context control from the current state. Detailed schemas, preconditions, and transition checks are given in Supplementary Material, Section A.

VerMem 提供这七种原子工具，它们总结于表1并在公式 (6) 中形式化，同时还包括空动作 $\varnothing$。 LTM 工具添加、修订或软删除持久信息。 STM 工具检索 LTM 内容，过滤或总结活动上下文，并通过 SelectEpisode 从 $H_t$ 恢复相关的同任务片段。 统一动作空间使策略能够根据当前状态，在持久记忆维护与活动上下文控制之间做出选择。 详细的 schema、前置条件和转移检查见补充材料第 A 节。

表1：VerMem 的原子记忆工具。

Tool	Target	Function
Add	LTM	Add to Mt
Update	LTM	Create revised versions in Mt
Delete	LTM	Soft-delete entries in Mt
Retrieve	STM	Retrieve Mt → Ct
Filter	STM	Filter Ct
SelectEpisode	STM	Restore Ht → Ct
Summarize	STM	Summarize Ct

3.3 Training Pipeline

Curriculum and SFT warmup. VerMem uses an SFT warmup followed by three reinforcement learning phases. The supervised set ${\mathcal{D}}_{\mathrm{SFT}}=\{(x_i,v_i^{\star },{\xi }_i^{\star }){\}}_{i=1}^N$ contains validated targets for the seven tools and $\varnothing$, where ${\xi }_i^{\star }$ denotes structured arguments and $u_i^{\star }=(v_i^{\star },{\xi }_i^{\star })$. The objective is

课程与 SFT 预热。 VerMem 使用 SFT 预热，随后进行三个强化学习阶段。 监督集合 ${\mathcal{D}}_{\mathrm{SFT}}=\{(x_i,v_i^{\star },{\xi }_i^{\star }){\}}_{i=1}^N$ 包含七种工具和 $\varnothing$ 的已验证目标，其中 ${\xi }_i^{\star }$ 表示结构化参数，$u_i^{\star }=(v_i^{\star },{\xi }_i^{\star })$。 目标为

$$\begin{array}{}\text{(7)}& \begin{array}{r}{\mathcal{L}}_{\mathrm{SFT}}=-{\mathbb{E}}_{(x_i,v_i^{\star },{\xi }_i^{\star })\sim {\mathcal{D}}_{\mathrm{SFT}}}[\log {\pi }_{\theta }(v_i^{\star },{\xi }_i^{\star }\mid x_i)].\end{array}\end{array}$$

Equation (7) trains both operation selection and structured argument generation. Boundary templates and candidates from a frozen DeepSeek-V3.2 teacher are retained only when they satisfy the tool schema and transition constraints. The warmup teaches tool selection, argument generation, and valid structured outputs, but does not determine the delayed task utility of a memory decision. Data construction, filtering, and coverage are detailed in Supplementary Material, Section D.

公式 (7) 同时训练操作选择和结构化参数生成。 来自冻结 DeepSeek-V3.2 教师模型的边界模板和候选项，只有在满足工具 schema 和转移约束时才会保留。 预热教授工具选择、参数生成和有效的结构化输出，但不决定记忆决策的延迟任务效用。 数据构建、过滤和覆盖范围详见补充材料第 D 节。

Phase-wise reinforcement learning. Phase A optimizes ${\mathcal{V}}_{\mathrm{LTM}}\cup \{\varnothing \}$ on construction, revision, deletion, and no-change decisions. Phase B optimizes ${\mathcal{V}}_{\mathrm{STM}}\cup \{\varnothing \}$ on retrieval, filtering, summarization, and historical-context recovery under distractors. Phase C enables $\mathcal{V}\cup \{\varnothing \}$ in complete multi-step tasks, where LTM and STM operations may alternate. Only memory operation policy parameters are transferred across phases; $M_t$, $C_t$, and $H_t$ are initialized independently for every training instance.

分阶段强化学习。 阶段 A 针对构建、修订、删除和不改变决策优化 ${\mathcal{V}}_{\mathrm{LTM}}\cup \{\varnothing \}$。 阶段 B 针对干扰项下的检索、过滤、总结和历史上下文恢复优化 ${\mathcal{V}}_{\mathrm{STM}}\cup \{\varnothing \}$。 阶段 C 在完整多步任务中启用 $\mathcal{V}\cup \{\varnothing \}$，其中 LTM 和 STM 操作可以交替执行。 只有记忆操作策略参数会跨阶段转移；每个训练实例的 $M_t$、$C_t$ 和 $H_t$ 都独立初始化。

Stateful trajectory collection. At policy update $n$ of phase $p\in \{A,B,C\}$, the current policy samples $K$ trajectories from the same phase-specific initial state:

有状态轨迹收集。 在阶段 $p\in \{A,B,C\}$ 的策略更新 $n$ 处，当前策略从相同的阶段特定初始状态采样 $K$ 条轨迹：

$$\begin{array}{}\text{(8)}& \begin{array}{rl}{\tau }_{k,p}^{(q,n)}& \sim {\mathbb{P}}_{p,n}(\tau \mid s_{0,p}^{(q)}),k=1,\dots ,K,\\ {\mathcal{G}}_{q,p}^{(n)}& =\{{\tau }_{1,p}^{(q,n)},\dots ,{\tau }_{K,p}^{(q,n)}\}.\end{array}\end{array}$$

Each rollout stores both the post-memory state ${\tilde{s}}_t$ and the post-interaction state $s_{t+1}$, so that memory-transition credit is not conflated with the subsequent task action. The candidate group in Equation (8) supports subsequent local and global credit assignment. Phase-specific states, termination rules, and token masks are provided in Supplementary Material, Sections D and F.

每条 rollout 同时存储记忆后状态 ${\tilde{s}}_t$ 和交互后状态 $s_{t+1}$，从而避免把记忆转移信用与后续任务动作混淆。 公式 (8) 中的候选组支持后续局部与全局信用分配。 阶段特定状态、终止规则和 token mask 见补充材料第 D 节和第 F 节。

3.4 Local and Global Verifier-Guided Hierarchical Credit Assignment

We optimize the memory operation policy with local and global credit signals. For each task $q$, the $K$ trajectories sampled from the same initial state form the candidate group ${\mathcal{G}}_q=\{{\tau }_1^{(q)},\dots ,{\tau }_K^{(q)}\}$. Let ${\mathcal{B}}_{\mathrm{op}}$ contain every memory decision collected in the current update, including null and rejected commands, and let ${\mathcal{B}}_{\mathrm{exec}}\subseteq {\mathcal{B}}_{\mathrm{op}}$ contain valid non-null commands and valid null decisions. Structural, safety, and budget violations are recorded separately as $c_{t,k}$. For $(q,k,t)\in {\mathcal{B}}_{\mathrm{exec}}$, the local verifier produces $r_{t,k}^{\mathrm{local}}$, which evaluates task relevance, evidence grounding, local progress, and information fidelity. The null decision is treated as a valid identity command and is scored with its own rubric. Rejected commands are not sent to the local verifier; their local advantage is set to zero. After task termination, the global trajectory branch produces the composite score $r_k^{\mathrm{global}}$. Its verifier and programmatic components are defined below.

我们使用局部与全局信用信号优化记忆操作策略。 对于每个任务 $q$，从相同初始状态采样的 $K$ 条轨迹组成候选组 ${\mathcal{G}}_q=\{{\tau }_1^{(q)},\dots ,{\tau }_K^{(q)}\}$。 令 ${\mathcal{B}}_{\mathrm{op}}$ 包含当前更新中收集的每个记忆决策，包括空命令和被拒绝的命令；令 ${\mathcal{B}}_{\mathrm{exec}}\subseteq {\mathcal{B}}_{\mathrm{op}}$ 包含有效的非空命令和有效的空决策。 结构、安全和预算违规被另行记录为 $c_{t,k}$。 对于 $(q,k,t)\in {\mathcal{B}}_{\mathrm{exec}}$，局部验证器产生 $r_{t,k}^{\mathrm{local}}$，评估任务相关性、证据依据、局部进展和信息忠实度。 空决策被视为有效的恒等命令，并使用自己的评分准则。 被拒绝的命令不会发送给局部验证器；其局部优势设为零。 任务终止后，全局轨迹分支产生复合分数 $r_k^{\mathrm{global}}$。 其验证器组件和程序化组件定义如下。

The composite global score is normalized within the candidate group for the same task. Local scores are normalized by operation type. The decision subset and two advantages are

复合全局分数在同一任务的候选组内归一化。 局部分数按操作类型归一化。 决策子集和两个优势为

$$\begin{array}{}\text{(9)}& \begin{array}{rl}{\mathcal{B}}_v& =\{(q,k,t)\in {\mathcal{B}}_{\mathrm{exec}}:v_{t,k}=v\},\\ A_k^{\mathrm{global}}& =\frac{r_k^{\mathrm{global}}-{\mu }_q^{\mathrm{global}}}{{\sigma }_q^{\mathrm{global}}+ϵ},\\ A_{t,k}^{\mathrm{local}}& =\frac{r_{t,k}^{\mathrm{local}}-{\mu }_{v_{t,k}}^{\mathrm{local}}}{{\sigma }_{v_{t,k}}^{\mathrm{local}}+ϵ}.\end{array}\end{array}$$

The global statistics are computed from the $K$ trajectories in ${\mathcal{G}}_q$, while local statistics are computed from operations of the same type in the current update batch. This operation-wise normalization reduces scale differences across tool-specific verifier rubrics. When an update contains too few instances of an operation type, phase-specific running statistics are used. The corresponding thresholds and update rules are provided in Supplementary Material, Section F. The local expression in Equation (9) is defined only for decisions in ${\mathcal{B}}_{\mathrm{exec}}$. For rejected commands, it is not evaluated and $A_{t,k}^{\mathrm{local}}=0$.

全局统计量由 ${\mathcal{G}}_q$ 中的 $K$ 条轨迹计算，局部统计量则由当前更新批次中同类型的操作计算。 这种逐操作归一化减小了工具特定验证器评分准则之间的尺度差异。 当一次更新中某种操作类型的实例过少时，会使用阶段特定的运行统计量。 相应阈值和更新规则见补充材料第 F 节。 公式 (9) 中的局部表达式只对 ${\mathcal{B}}_{\mathrm{exec}}$ 中的决策定义。 对于被拒绝的命令，不计算该表达式，并令 $A_{t,k}^{\mathrm{local}}=0$。

At each memory decision, Equation (5) combines the two advantages with the constraint cost. The local advantage is assigned only to the current atomic operation, while the global advantage provides task-level supervision to all operations in the trajectory. Different memory decisions within one trajectory can therefore receive different learning signals. Poorly rated operations in a successful trajectory therefore need not receive the trajectory's full positive credit, while useful operations in an unsuccessful trajectory may retain a positive local contribution. The final sign still depends on the weighted sum of the local, global, and constraint terms. Rejected commands are not evaluated by the local semantic verifier, so $A_{t,k}^{\mathrm{local}}=0$. They still receive the trajectory-level $A_k^{\mathrm{global}}$, while $c_{t,k}$ provides their direct operation-level penalty.

在每个记忆决策处，公式 (5) 将两个优势与约束成本结合。 局部优势只分配给当前原子操作，而全局优势为轨迹中的所有操作提供任务级监督。 因此，同一轨迹内的不同记忆决策可以收到不同的学习信号。 成功轨迹中评分较差的操作无需获得该轨迹的全部正信用，而失败轨迹中的有用操作可以保留正的局部贡献。 最终符号仍取决于局部、全局和约束项的加权和。 被拒绝的命令不会由局部语义验证器评估，因此 $A_{t,k}^{\mathrm{local}}=0$。 它们仍会收到轨迹级 $A_k^{\mathrm{global}}$，而 $c_{t,k}$ 提供直接的操作级惩罚。

Let $y_{t,k,1:L_{t,k}}$ be the generated tokens that serialize the complete command $u_{t,k}$, including its structured arguments, and let $h_{t,k,j}=(x_{t,k},y_{t,k,<j})$ be the token prefix. The token-level policy ratio and KL term are

令 $y_{t,k,1:L_{t,k}}$ 为序列化完整命令 $u_{t,k}$ 及其结构化参数的生成 token，并令 $h_{t,k,j}=(x_{t,k},y_{t,k,<j})$ 为 token 前缀。 token 级策略比率和 KL 项为

$$\begin{array}{}\text{(10)}& \begin{array}{rl}{\rho }_{t,k,j}(\theta )& =\frac{{\pi }_{\theta }(y_{t,k,j}\mid h_{t,k,j})}{{\pi }_{{\theta }_{\mathrm{old}}}(y_{t,k,j}\mid h_{t,k,j})},\\ d_{t,k,j}^{\mathrm{KL}}& =D_{\mathrm{KL}}({\pi }_{\theta }(\cdot \mid h_{t,k,j})\parallel {\pi }_{\mathrm{ref}}(\cdot \mid h_{t,k,j})).\end{array}\end{array}$$

Here, ${\theta }_{\mathrm{old}}$ denotes the policy frozen before the current update, and ${\pi }_{\mathrm{ref}}$ is the phase-specific reference policy. The coefficient ${ϵ}_{\mathrm{clip}}$ controls PPO clipping, while ${\beta }_{\mathrm{KL}}$ weights the KL regularization term. All command tokens share $A_{t,k}^{\mathrm{hier}}$, but the clipped ratio and KL term in Equation (10) are evaluated token by token. The operation-normalized objective is

其中，${\theta }_{\mathrm{old}}$ 表示当前更新前冻结的策略，${\pi }_{\mathrm{ref}}$ 是阶段特定的参考策略。 系数 ${ϵ}_{\mathrm{clip}}$ 控制 PPO 裁剪，而 ${\beta }_{\mathrm{KL}}$ 对 KL 正则项加权。 所有命令 token 共享 $A_{t,k}^{\mathrm{hier}}$，但公式 (10) 中的裁剪比率和 KL 项会逐 token 计算。 逐操作归一化目标为

$$\begin{array}{}\text{(11)}& \begin{array}{rl}{\ell }_{t,k,j}(\theta )=& min({\rho }_{t,k,j}(\theta )A_{t,k}^{\mathrm{hier}},\\ & \mathrm{clip}({\rho }_{t,k,j}(\theta ),1-{ϵ}_{\mathrm{clip}},1+{ϵ}_{\mathrm{clip}})A_{t,k}^{\mathrm{hier}})\\ & -{\beta }_{\mathrm{KL}}{d}_{t,k,j}^{\mathrm{KL}},\\ {\mathcal{J}}_{\mathrm{GRPO}}(\theta )=& \frac{1}{|{\mathcal{B}}_{\mathrm{op}}|}\sum _{(q,k,t)\in {\mathcal{B}}_{\mathrm{op}}}\frac{1}{L_{t,k}}\sum _{j=1}^{L_{t,k}}{\ell }_{t,k,j}(\theta ).\end{array}\end{array}$$

Equation (11) uses the clipped surrogate from PPO and candidate-group normalization from GRPO. The memory operation policy loss is applied only to tokens that serialize the atomic memory command. Task-solver outputs and environment observations are excluded from this loss. Task specifications, state-serialization tokens, and padding tokens are also excluded. Operations at different time steps retain different advantages, and the factor $1/L_{t,k}$ prevents commands with longer arguments from dominating the update. Loss masks and stabilization settings are provided in Supplementary Material, Sections B and F. The local verifier, global verifier, and reward computation are used only during training.

公式 (11) 使用 PPO 的裁剪代理目标和 GRPO 的候选组归一化。 记忆操作策略损失只应用于序列化原子记忆命令的 token。 任务求解器输出和环境观察不包含在该损失中。 任务规范、状态序列化 token 和 padding token 也被排除。 不同时间步的操作保留不同优势，而因子 $1/L_{t,k}$ 防止参数更长的命令支配更新。 损失 mask 和稳定化设置见补充材料第 B 节和第 F 节。 局部验证器、全局验证器和奖励计算仅在训练期间使用。

3.5 Reward Function Design

The training signal has three explicitly separated channels: semantic quality for executable memory commands, global utility for the completed trajectory, and hard execution constraints. Let $T_k$ be the number of memory-decision opportunities in ${\tau }_k$, including null and rejected commands, and let ${\mathcal{E}}_k$ index its executable commands, including valid null decisions. Null decisions count toward the decision horizon but not as memory-tool calls. Keeping the three channels separate prevents a hard rejection from also receiving an undefined or duplicated semantic penalty.

训练信号包含三个显式分离的通道：可执行记忆命令的语义质量、已完成轨迹的全局效用，以及硬性执行约束。 令 $T_k$ 为 ${\tau }_k$ 中记忆决策机会的数量，包括空命令和被拒绝的命令；令 ${\mathcal{E}}_k$ 索引其中的可执行命令，包括有效的空决策。 空决策计入决策时域，但不计作记忆工具调用。 保持三个通道分离，可以避免一次硬性拒绝同时收到未定义或重复的语义惩罚。

Local operation reward. The local component is

局部操作奖励。 局部组件为

$$\begin{array}{}\text{(12)}& R^{\mathrm{local}}({\tau }_k)=\frac{1}{T_k}\sum _{t\in {\mathcal{E}}_k}{r}_{t,k}^{\mathrm{local}}.\end{array}$$

Equation (12) normalizes by all memory-decision opportunities while summing only scores produced for executable decisions. It is therefore zero when ${\mathcal{E}}_k$ is empty and is used for analysis and monitoring, not as a substitute for the operation-wise advantages. The score evaluates only the current atomic memory operation and its realized state transition. It measures task relevance, evidence grounding, local progress, and information fidelity. For LTM operations, it evaluates whether information has persistent value, whether an update agrees with new evidence, and whether a deletion is justified. For STM operations, it evaluates whether retrieval or episode-level context selection supplies missing information and whether filtering or summarization preserves useful evidence while reducing context load. Structural validity is excluded from this semantic score and is handled by the auxiliary channel.

公式 (12) 以所有记忆决策机会归一化，同时只求和可执行决策产生的分数。 因此，当 ${\mathcal{E}}_k$ 为空时它为零；它用于分析和监控，而不是替代逐操作优势。 该分数只评估当前原子记忆操作及其实际状态转移。 它衡量任务相关性、证据依据、局部进展和信息忠实度。 对于 LTM 操作，它评估信息是否具有持久价值、更新是否与新证据一致，以及删除是否合理。 对于 STM 操作，它评估检索或情景级上下文选择是否提供缺失信息，以及过滤或总结是否在减轻上下文负载的同时保留有用证据。 结构有效性不计入该语义分数，而由辅助通道处理。

Composite global trajectory reward. The composite global branch is computed after task termination:

复合全局轨迹奖励。 复合全局分支在任务终止后计算：

$$\begin{array}{}\text{(13)}& \begin{array}{r}{r}_k^{\mathrm{global}}=\frac{1}{4}(r_k^{\mathrm{task}}+r_k^{\mathrm{evid}}+r_k^{\mathrm{state}}+r_k^{\mathrm{eff}}).\end{array}\end{array}$$

All four terms lie in $[0,1]$. The task score $r_k^{\mathrm{task}}$ and annotated supporting-fact recall $r_k^{\sup }$ are computed programmatically. The global verifier returns only an evidence-coherence score $v_k^{\mathrm{coh}}$ and a terminal-memory-consistency score $v_k^{\mathrm{state}}$. The evidence and state components are

四个项都位于 $[0,1]$。 任务分数 $r_k^{\mathrm{task}}$ 和带标注的支持事实召回率 $r_k^{\sup }$ 通过程序计算。 全局验证器只返回证据连贯性分数 $v_k^{\mathrm{coh}}$ 和终端记忆一致性分数 $v_k^{\mathrm{state}}$。 证据与状态组件为

$$\begin{array}{}\text{(14)}& \begin{array}{rl}{r}_k^{\mathrm{evid}}& =\frac{1}{2}(r_k^{\sup }+v_k^{\mathrm{coh}}),\\ r_k^{\mathrm{state}}& =v_k^{\mathrm{state}}.\end{array}\end{array}$$

Equations (13) and (14) make the boundary explicit: the global verifier supplies only $v_k^{\mathrm{coh}}$ and $v_k^{\mathrm{state}}$, while the composite global branch also includes programmatic task correctness, supporting-fact recall, and efficiency. Terminal-memory consistency requires current active LTM and context to agree with observed evidence and treats $H_t$ as an ordered provenance record: superseded or failed events may remain in history if their status is explicit. Since policy training uses only HotpotQA, $r_k^{\mathrm{task}}$ is derived from answer correctness. Environment success, planning progress, and goal-state satisfaction are used as evaluation metrics on the downstream agent benchmarks rather than as policy-training rewards.

公式 (13) 和 (14) 明确划分了边界：全局验证器只提供 $v_k^{\mathrm{coh}}$ 和 $v_k^{\mathrm{state}}$，而复合全局分支还包括程序化任务正确性、支持事实召回率和效率。 终端记忆一致性要求当前活动 LTM 和上下文与观察到的证据一致，并将 $H_t$ 视为有序来源记录：只要状态明确，被取代或失败的事件可以保留在历史中。 由于策略训练只使用 HotpotQA，$r_k^{\mathrm{task}}$ 根据答案正确性得出。 环境成功、规划进展和目标状态满足度被用作下游智能体基准的评估指标，而不是策略训练奖励。

Let ${\bar{C}}_k^{\mathrm{online}}\in [0,1]$ denote the programmatically computed online cost from tokens, task steps, and executed non-null memory-tool calls. These components are separately min--max normalized within the $K$ candidates for the same task and then averaged. The efficiency term is

令 ${\bar{C}}_k^{\mathrm{online}}\in [0,1]$ 表示根据 token、任务步骤和已执行的非空记忆工具调用程序化计算的在线成本。 这些组件分别在同一任务的 $K$ 个候选项内进行最小--最大归一化，然后取平均值。 效率项为

$$\begin{array}{}\text{(15)}& r_k^{\mathrm{eff}}=\mathbb{I}[r_k^{\mathrm{task}}\ge \delta ](1-{\bar{C}}_k^{\mathrm{online}}).\end{array}$$

We use $\delta =0.5$. This gate prevents low-cost failures from receiving an efficiency reward in Equation (15).

我们使用 $\delta =0.5$。 该门控防止低成本失败在公式 (15) 中获得效率奖励。

Auxiliary constraints. The auxiliary cost is

辅助约束。 辅助成本为

$$\begin{array}{}\text{(16)}& P_{\mathrm{aux}}({\tau }_k)=\frac{1}{T_k}\sum _{t=1}^{T_k}{c}_{t,k}.\end{array}$$

Equation (16) averages direct constraint costs over the memory-decision horizon. The step cost records discrete or hard violations, including invalid tools or arguments, failed state transitions, unsupported writes or updates, unjustified deletions, making protected evidence operationally inaccessible within the remaining decision and token budget, context overflow, exceeding the tool or interaction budget, and information leakage in no-reference settings. Routine token, step, and tool costs do not enter this channel; they are evaluated continuously by the global efficiency term. Terminal budget violations are attached to the final memory decision that precedes termination.

公式 (16) 在记忆决策时域上对直接约束成本取平均。 步骤成本记录离散或硬性违规，包括无效工具或参数、失败的状态转移、无依据的写入或更新、不合理删除、在剩余决策与 token 预算内使受保护证据在操作上无法访问、上下文溢出、超出工具或交互预算，以及无参考设置下的信息泄漏。 常规 token、步骤和工具成本不进入该通道；它们由全局效率项持续评估。 终端预算违规会附加到终止前的最后一个记忆决策。

The scalar displayed in the All-Returns training curve is used only for monitoring:

All-Returns 训练曲线中显示的标量仅用于监控：

$$\begin{array}{}\text{(17)}& m_k^{\mathrm{all}}=\mathrm{clip}(\frac{R^{\mathrm{local}}({\tau }_k)+r_k^{\mathrm{global}}-P_{\mathrm{aux}}({\tau }_k)}{2},0,1).\end{array}$$

The Answer-Only curve analogously displays

类似地，Answer-Only 曲线显示

$$\begin{array}{}\text{(18)}& m_k^{\mathrm{ans}}=\mathrm{clip}(r_k^{\mathrm{task}}-P_{\mathrm{aux}}({\tau }_k),0,1).\end{array}$$

Equations (17) and (18) are strategy-specific monitoring values. They are not policy advantages and are not directly comparable in absolute level. During policy optimization, the local and global signals are normalized separately, while $c_{t,k}$ is assigned directly to the corresponding decision. Tool-specific rubrics, normalization details, completion thresholds, and constraint triggers are provided in Supplementary Material, Section F.

公式 (17) 和 (18) 是策略特定的监控值。 它们不是策略优势，在绝对水平上也不能直接比较。 策略优化期间，局部信号与全局信号分别归一化，而 $c_{t,k}$ 直接分配给相应决策。 工具特定评分准则、归一化细节、完成阈值和约束触发条件见补充材料第 F 节。

4. Experiments

4.1 Experimental Setup

Datasets. We evaluate VerMem on ALFWorld, SciWorld, PDDL, BabyAI, and HotpotQA. VerMem is fine-tuned only on the HotpotQA training split, whose supporting facts and distractors support LTM maintenance, STM control, and historical-context recovery, and is then evaluated directly on all five benchmarks. Memory states are isolated by task instance. Each episode starts with empty $M_0$ and $H_0$, while $C_0$ contains only the task instruction and initial input.

数据集。 我们在 ALFWorld、SciWorld、PDDL、BabyAI 和 HotpotQA 上评估 VerMem。 VerMem 只在 HotpotQA 训练集上微调，其中的支持事实和干扰项支持 LTM 维护、STM 控制和历史上下文恢复，之后直接在全部五个基准上评估。 记忆状态按任务实例隔离。 每个情景从空的 $M_0$ 和 $H_0$ 开始，而 $C_0$ 只包含任务指令和初始输入。

Evaluation Metrics. We report Success Rate (successful episodes divided by evaluated episodes) on ALFWorld, SciWorld, and BabyAI, Progress Rate on PDDL, and the Qwen-Max LLM-as-a-Judge score $J_{\mathrm{judge}}$ on HotpotQA. The main table reports $100J_{\mathrm{judge}}$. Qwen-Max is used only as a held-out evaluator; it is not used for policy training, verifier scoring, or checkpoint selection. Let $S^{\ast }$ be a target macro-average Success Rate and $B^{\ast }$ an online-token budget. $\mathrm{TN}@S^{\ast }$ is the linearly interpolated online-token budget at which the mean success curve first reaches the target $S^{\ast }$, using the two adjacent evaluated budget points that bracket the target. $\mathrm{SR}@B^{\ast }$ is the macro-average Success Rate at $B^{\ast }$. The reward ablation additionally reports Memory Quality (MQ), average online token number (TN), and average memory-tool calls (TC).

评估指标。 我们在 ALFWorld、SciWorld 和 BabyAI 上报告成功率（成功情景数除以评估情景数），在 PDDL 上报告进度率，并在 HotpotQA 上报告 Qwen-Max 的 LLM-as-a-Judge 分数 $J_{\mathrm{judge}}$。 主表报告 $100J_{\mathrm{judge}}$。 Qwen-Max 只用作留出评估器；它不用于策略训练、验证器评分或 checkpoint 选择。 令 $S^{\ast }$ 为目标宏平均成功率，$B^{\ast }$ 为在线 token 预算。 $\mathrm{TN}@S^{\ast }$ 是平均成功曲线首次达到目标 $S^{\ast }$ 时，通过包围该目标的两个相邻评估预算点线性插值得到的在线 token 预算。 $\mathrm{SR}@B^{\ast }$ 是预算 $B^{\ast }$ 下的宏平均成功率。 奖励消融还报告记忆质量（MQ）、平均在线 token 数量（TN）和平均记忆工具调用次数（TC）。

Baselines and Backbones. Base uses the common task solver without an explicit memory operation policy or external persistent-memory store. We compare it with LangMem, A-Mem, Mem0 and its graph-based variant Mem0${}^g$, and AgeMem under Qwen2.5-7B-Instruct and Qwen3-4B-Instruct. VerMem-noVerify is a matched internal control that removes semantic feedback from the local and global verifiers while retaining the same action space, SFT initialization, curriculum, task-outcome reward, transition constraints, and GRPO configuration. Complete implementation and baseline settings are provided in Supplementary Material, Sections C and F.

基线与 backbone。 Base 使用通用任务求解器，不含显式记忆操作策略或外部持久记忆存储。 我们在 Qwen2.5-7B-Instruct 和 Qwen3-4B-Instruct 下，将其与 LangMem、A-Mem、Mem0 及其图变体 Mem0${}^g$，以及 AgeMem 比较。 VerMem-noVerify 是匹配的内部对照：它移除局部和全局验证器的语义反馈，同时保留相同的动作空间、SFT 初始化、课程、任务结果奖励、转移约束和 GRPO 配置。 完整实现和基线设置见补充材料第 C 节和第 F 节。

4.2 Main Results

表2：两个 LLM backbone 下的主结果。ALFWorld、SciWorld 和 BabyAI 报告成功率，PDDL 报告进度率，HotpotQA 报告 100J_judge。每项为种子 42、43 和 44 三次独立运行的均值。Average 是五个显示分数的未加权算术平均。每个 backbone 设置中的最佳结果以粗体表示，次优结果以下划线表示。

Method	ALFWorld	SciWorld	PDDL	BabyAI	HotpotQA 100Jjudge	Average
Qwen2.5-7B-Instruct
Base	27.16	13.80	10.15	50.80	38.36	28.05
LangMem	38.27	28.29	15.85	51.34	37.43	34.24
A-Mem	34.68	28.06	18.39	58.82	43.95	36.78
Mem0	37.49	26.99	13.96	60.58	46.66	37.14
Mem0g	35.34	30.50	14.86	58.78	42.06	36.31
AgeMem	41.07	35.55	17.31	61.42	54.44	41.96
VerMem-noVerify	40.18	33.94	22.41	59.21	53.08	41.76
VerMem (Ours)	46.30	43.39	22.06	65.55	62.75	48.01
Qwen3-4B-Instruct
Base	38.51	47.89	30.14	55.83	47.48	43.97
LangMem	40.89	50.42	28.42	53.80	42.70	43.25
A-Mem	34.31	50.14	34.41	61.35	48.48	45.74
Mem0	41.17	51.38	31.72	60.05	39.16	44.70
Mem0g	36.69	47.76	29.61	57.59	38.12	41.95
AgeMem	48.97	59.48	35.07	72.56	55.49	54.31
VerMem-noVerify	48.63	58.92	34.76	72.11	53.79	53.64
VerMem (Ours)	53.73	66.37	38.75	76.77	63.61	59.85

Overall comparison. Table 2 reports the main results under both backbones. With Qwen2.5-7B-Instruct, VerMem obtains an average score of $48.01$. This is $19.96$ points above Base, corresponding to a relative gain of $71.16\mathrm{\%}$, and $6.05$ points above the strongest external method, AgeMem. VerMem ranks first on ALFWorld, SciWorld, BabyAI, HotpotQA, and Average. PDDL is the only exception, where VerMem-noVerify is $0.35$ points higher. VerMem still exceeds the strongest external PDDL baseline by $3.67$ points. With Qwen3-4B-Instruct, VerMem ranks first in all six reported columns and reaches an average score of $59.85$, outperforming AgeMem by $5.54$ points. The consistent improvement is observed with both evaluated backbones rather than only one of them.

总体比较。 表2报告了两个 backbone 下的主结果。 使用 Qwen2.5-7B-Instruct 时，VerMem 取得 $48.01$ 的平均分。 这比 Base 高 $19.96$ 分，对应 $71.16\mathrm{\%}$ 的相对增益，并比最强外部方法 AgeMem 高 $6.05$ 分。 VerMem 在 ALFWorld、SciWorld、BabyAI、HotpotQA 和 Average 上排名第一。 PDDL 是唯一例外，其中 VerMem-noVerify 高出 $0.35$ 分。 VerMem 仍比最强外部 PDDL 基线高 $3.67$ 分。 使用 Qwen3-4B-Instruct 时，VerMem 在报告的全部六列中排名第一，平均分达到 $59.85$，比 AgeMem 高 $5.54$ 分。 这种持续提升同时出现在两个评估 backbone 上，而不是只出现在其中一个上。

Performance across task types. The improvements vary with the memory requirements of each task. With Qwen2.5-7B-Instruct, VerMem exceeds the strongest external baseline by $5.23$, $7.84$, $3.67$, $4.13$, and $8.31$ points on ALFWorld, SciWorld, PDDL, BabyAI, and HotpotQA. The corresponding gains with Qwen3-4B-Instruct are $4.76$, $6.89$, $3.68$, $4.21$, and $8.12$ points. The largest gains occur on HotpotQA and SciWorld. This pattern is consistent with the greater need for multi-hop evidence aggregation and long interaction-state tracking in these tasks. The smaller gain on PDDL is consistent with its more structured state representation and feedback, although the present experiments do not isolate this factor.

不同任务类型上的性能。 提升幅度随各任务的记忆需求而变化。 使用 Qwen2.5-7B-Instruct 时，VerMem 在 ALFWorld、SciWorld、PDDL、BabyAI 和 HotpotQA 上分别比最强外部基线高 $5.23$、$7.84$、$3.67$、$4.13$ 和 $8.31$ 分。 使用 Qwen3-4B-Instruct 时，相应增益为 $4.76$、$6.89$、$3.68$、$4.21$ 和 $8.12$ 分。 最大的增益出现在 HotpotQA 和 SciWorld 上。 这一模式与这些任务对多跳证据聚合和长交互状态跟踪的更高需求一致。 PDDL 上较小的增益与其更结构化的状态表示和反馈一致，不过当前实验并未单独隔离这一因素。

Cross-task transfer. VerMem is fine-tuned only on HotpotQA, yet it consistently improves over the strongest baselines on ALFWorld, SciWorld, PDDL, and BabyAI. The gains range from $3.67$ to $7.84$ points with Qwen2.5-7B-Instruct and from $3.68$ to $6.89$ points with Qwen3-4B-Instruct. These environments require environment tracking, scientific-progress maintenance, planning-state control, and instruction execution. The transfer results indicate that the learned memory operation policy captures reusable memory behavior rather than task-specific document patterns.

跨任务迁移。 VerMem 只在 HotpotQA 上微调，却持续优于 ALFWorld、SciWorld、PDDL 和 BabyAI 上的最强基线。 使用 Qwen2.5-7B-Instruct 时，增益范围为 $3.67$ 至 $7.84$ 分；使用 Qwen3-4B-Instruct 时为 $3.68$ 至 $6.89$ 分。 这些环境要求进行环境跟踪、科学进展维护、规划状态控制和指令执行。 迁移结果表明，学习到的记忆操作策略捕获了可复用的记忆行为，而不是任务特定的文档模式。

图2：使用 Qwen2.5-7B-Instruct，在 ALFWorld、SciWorld 和 BabyAI 的不同在线 token 预算下得到的效率--性能前沿。曲线比较 A-Mem、Mem0、AgeMem 和 VerMem，并报告三个随机种子上的宏平均成功率。阴影区域表示跨种子的标准差。水平和垂直虚线分别标记 S*=40.0 和 B*=2,500。

表3：使用 Qwen2.5-7B-Instruct，在受控在线 token 预算下的效率--性能比较，其中 S*=40.0，B*=2,500。TN@S* 由包围 S* 的两个相邻评估预算点线性插值得到。TN@S* 越低越好，SR@B* 越高越好。

Method	TN@S* ↓	SR@B* ↑
A-Mem	4,250	33.80
Mem0	3,820	35.60
AgeMem	2,600	39.00
VerMem	2,080	44.30

Efficiency--performance frontier. Figure 2 and Table 3 show that VerMem maintains the highest macro-average SR across the evaluated budgets. Its mean curve first reaches $S^{\ast }=40.0$ at the linearly interpolated estimate of $\mathrm{2,080}$ online tokens, reducing $\mathrm{TN}@S^{\ast }$ by $20.0\mathrm{\%}$ relative to AgeMem, and achieves an $\mathrm{SR}@B^{\ast }$ of $44.30$ at $B^{\ast }=\mathrm{2,500}$, exceeding AgeMem by $5.30$ points. Both verifiers are disabled during evaluation, so these measurements do not include inference-time verification calls.

效率--性能前沿。 图2和表3表明，VerMem 在各个评估预算下都保持最高的宏平均成功率。 其平均曲线在线性插值估计为 $\mathrm{2,080}$ 个在线 token 时首次达到 $S^{\ast }=40.0$，相较 AgeMem 将 $\mathrm{TN}@S^{\ast }$ 降低 $20.0\mathrm{\%}$；在 $B^{\ast }=\mathrm{2,500}$ 时，它取得 $44.30$ 的 $\mathrm{SR}@B^{\ast }$，比 AgeMem 高 $5.30$ 分。 两个验证器在评估期间都被禁用，因此这些测量不包括推理时验证调用。

4.3 Ablation Studies

图3：使用 Qwen2.5-7B-Instruct 的累积组件消融。ALFWorld 和 SciWorld 报告成功率，HotpotQA 报告 100J_judge。Base 不含显式记忆操作策略；+LT 加入 LTM 操作；+LT/ST 加入完整 LTM/STM 工具空间；+noV 加入不含语义验证器反馈的有状态 RL；+V 加入局部与全局信用分支。

Unified memory components. Figure 3 shows cumulative gains from LTM operations, the complete LTM/STM action space, stateful RL, and verifier-guided credit. LTM provides the largest initial improvement, STM control adds further gains on SciWorld and HotpotQA, and full VerMem improves over +noV by $6.12$, $9.45$, and $9.67$ points. The progression is consistent with contributions from both the unified action space and verifier-guided optimization.

统一记忆组件。 图3展示了 LTM 操作、完整 LTM/STM 动作空间、有状态 RL 和验证器引导信用带来的累积增益。 LTM 提供最大的初始提升，STM 控制在 SciWorld 和 HotpotQA 上带来进一步增益，而完整 VerMem 相比 +noV 分别提升 $6.12$、$9.45$ 和 $9.67$ 分。 这一进展与统一动作空间和验证器引导优化两者的贡献一致。

表4：局部与全局信用分支的消融。VerMem-noVerify 使用归一化任务结果优势和约束。+Local 加入局部操作优势。+Global 用复合全局轨迹优势替换仅任务的轨迹优势。+Local+Global 同时使用局部与复合全局优势，表示完整 VerMem。其他架构、SFT 初始化、训练数据、约束和 GRPO 设置保持一致。

Variant	ALFWorld SR	SciWorld SR	HotpotQA 100Jjudge
VerMem-noVerify	40.18	33.94	53.08
+Local	44.72	39.81	58.26
+Global	43.31	37.96	60.18
+Local+Global	46.30	43.39	62.75

Local and global credit branches. Table 4 shows complementary effects. Local feedback yields larger standalone gains on ALFWorld and SciWorld, whereas the global branch yields a larger standalone gain on HotpotQA. This pattern is consistent with, but does not by itself prove, the intended distinction between operation-level and trajectory-level credit. Their combination performs best on all three evaluated tasks.

局部与全局信用分支。 表4展示了互补效果。 局部反馈在 ALFWorld 和 SciWorld 上带来更大的独立增益，而全局分支在 HotpotQA 上带来更大的独立增益。 这一模式与操作级和轨迹级信用之间的预期区别一致，但其本身并不能证明这种区别。 二者组合在全部三个评估任务上表现最佳。

图4：使用 Qwen2.5-7B-Instruct 在 HotpotQA 上的 GRPO 训练奖励收敛。Training Step 索引 3,500 次更新日程中记录的 101 个训练点，相邻点之间间隔 35 次策略更新。曲线展示三次运行的未平滑均值，阴影区域表示一个标准差。Answer-Only 和 All-Returns 使用奖励章节中定义的监控映射；由于映射不同，其绝对水平不可直接比较。

表5：使用 Qwen2.5-7B-Instruct 在 HotpotQA 上的奖励函数消融。J_judge 表示 [0,1] 范围内的 LLM-as-a-Judge 分数，TN 是平均在线 token 数，MQ 是记忆质量，TC 是平均记忆工具调用次数。

Strategy	Jjudge ↑	TN ↓	MQ ↑	TC
Answer-Only	0.531	2,352	0.491	4.47
All-Returns	0.628	2,184	0.673	5.63

Reward function. The monitoring curves in Figure 4 suggest earlier stabilization and lower late-stage variability for All-Returns. Because the two strategies use different monitoring definitions, their absolute reward levels should not be compared directly. Table 5 shows that All-Returns raises $J_{\mathrm{judge}}$ from $0.531$ to $0.628$ and MQ from $0.491$ to $0.673$, while reducing TN by $7.14\mathrm{\%}$. TC increases from $4.47$ to $5.63$; together with the higher $J_{\mathrm{judge}}$ and MQ, this shows that the token reduction is not obtained merely by suppressing all memory-tool calls.

奖励函数。 图4中的监控曲线表明，All-Returns 更早稳定，后期变化也更小。 由于两种策略使用不同的监控定义，不应直接比较其绝对奖励水平。 表5表明，All-Returns 将 $J_{\mathrm{judge}}$ 从 $0.531$ 提升到 $0.628$，将 MQ 从 $0.491$ 提升到 $0.673$，同时把 TN 降低 $7.14\mathrm{\%}$。 TC 从 $4.47$ 增加到 $5.63$；结合更高的 $J_{\mathrm{judge}}$ 和 MQ，这表明 token 的减少并非仅仅通过抑制所有记忆工具调用获得。

5. Limitations

Our evaluation resets all memory states after each episode, so it measures within-episode persistence and cross-benchmark transfer of the learned policy rather than cross-session or cross-user memory retention. The memory operation policy is trained on constructed HotpotQA states and evaluated with two Qwen backbones; the findings may not transfer to other model families or domains. HotpotQA $J_{\mathrm{judge}}$ and MQ use a fixed proprietary Qwen-Max snapshot, which limits fully independent replication despite disclosure of the snapshot and prompts. The frozen DeepSeek-V3.2 teacher and verifiers also introduce offline training cost. Finally, learned verifier scores may inherit evaluator errors, and the present study does not establish robustness under adversarial, privacy-sensitive, or conflicting memory content.

我们的评估会在每个情景后重置所有记忆状态，因此它衡量的是学习策略在情景内的持久性和跨基准迁移，而不是跨会话或跨用户的记忆保留。 记忆操作策略在构造的 HotpotQA 状态上训练，并使用两个 Qwen backbone 评估；这些发现可能无法迁移到其他模型家族或领域。 HotpotQA 的 $J_{\mathrm{judge}}$ 和 MQ 使用固定的专有 Qwen-Max 快照；尽管公开了快照和 prompt，这仍限制了完全独立的复现。 冻结的 DeepSeek-V3.2 教师模型和验证器也会引入离线训练成本。 最后，学习到的验证器分数可能继承评估器错误，而且本研究尚未确立其在对抗性、隐私敏感或相互冲突的记忆内容下的鲁棒性。

6. Conclusion

We presented Verifiable Memory (VerMem), a unified framework for learning LTM and STM management in long-horizon LLM agents. VerMem maintains persistent memory, active context, and episodic history as distinct states, while one memory operation policy coordinates seven atomic tools for persistent-memory maintenance, active-context control, and historical-context recovery. Training combines an SFT warmup with a three-stage reinforcement learning curriculum. The local verifier evaluates realized atomic memory transitions, while the global verifier evaluates completed trajectories and terminal memory states. Both verifiers are removed during inference.

我们提出可验证记忆（VerMem），这是一个为长程 LLM 智能体学习 LTM 与 STM 管理的统一框架。 VerMem 将持久记忆、活动上下文和情景历史维护为不同状态，同时由一个记忆操作策略协调七种用于持久记忆维护、活动上下文控制和历史上下文恢复的原子工具。 训练将 SFT 预热与三阶段强化学习课程相结合。 局部验证器评估实际发生的原子记忆转移，全局验证器评估已完成轨迹和终端记忆状态。 两个验证器都在推理期间被移除。

Experiments across five benchmarks and two backbones show that VerMem consistently outperforms strong memory baselines and transfers from HotpotQA to interactive, planning, and instruction-following tasks. The budget-controlled evaluation further demonstrates a stronger efficiency--performance trade-off under limited online tokens. The ablation results confirm the contributions of unified LTM/STM control, stateful reinforcement learning, complementary verifier signals, and the complete reward design. These findings show that coordinated memory management and multi-granularity credit assignment provide an effective basis for reliable long-horizon agent behavior.

五个基准和两个 backbone 上的实验表明，VerMem 持续优于强记忆基线，并从 HotpotQA 迁移到交互、规划和指令遵循任务。 预算受控评估进一步表明，在有限在线 token 下，它取得了更强的效率--性能权衡。 消融结果确认了统一 LTM/STM 控制、有状态强化学习、互补验证器信号和完整奖励设计的贡献。 这些发现表明，协调记忆管理和多粒度信用分配为可靠的长程智能体行为提供了有效基础。