---

ECHO: Prune to Act, Trace to Learn with Selective Turn Memory in Agentic RL

MemoryAgentRL北京大学百度中国科学技术大学

Xie Zijun, Zheng Binbin, Gong Enlei, et al. ECHO: prune to act, trace to learn with selective turn memory in agentic RL[PP/OL]. V5. arXiv (2026-08-03)[2026-10-23]. DOI:10.48550/arXiv.2606.31650.

https://arxiv.org/abs/2606.31650v5

https://github.com/xiezijun714-lang/Echo

---

ECHO：行动时剪枝，学习时溯源——智能体强化学习中的选择性轮次记忆

Abstract

Long-horizon language agents must repeatedly interact with tools, accumulate evidence, and make decisions under bounded context windows. Context-management methods make such rollouts feasible by simplifying past interactions through deletion, folding, or memory editing. However, when useful history is collapsed into compressed states, the reconstructed context may no longer reveal which earlier observations support a successful final answer. This creates a mismatch between bounded-context acting and outcome-based reinforcement learning: the policy acts on reconstructed context, while the learner lacks source-level provenance for assigning credit to the evidence that mattered.

长时程语言智能体必须反复与工具交互、积累证据，并在有限的上下文窗口内做出决策。 上下文管理方法通过删除、折叠或编辑记忆来简化过去的交互，使这类轨迹采样得以进行。 然而，当有用的历史被折叠成压缩状态时，重建的上下文可能不再揭示哪些早先的观测支撑了最终的正确答案。 这使有限上下文下的行动与基于结果的强化学习之间产生错位：策略依据重建的上下文行动，而学习器缺少来源层面的溯源信息，无法将信用分配给真正发挥作用的证据。

We propose ECHO, a selective turn-memory framework for traceable context reconstruction in agentic RL. ECHO compresses each completed environment turn into a compact source-indexed memory record, reconstructs bounded policy contexts by selecting useful records, and reuses the selected source indices to route positive outcome credit to the final trajectory segment, reused evidence turns, memory findings, and memory-selection actions. On BrowseComp-Plus, ECHO reaches 43.4% held-out accuracy, outperforming GRPO at 28.9% and the rolling-summary baseline SUPO at 36.1%, while using fewer turns and lower trajectory volume than SUPO. The trained policy also improves zero-shot generalization across multi-objective QA, code generation, and deep information-seeking benchmarks on both dense and MoE backbones.

我们提出 ECHO，这是一个选择性轮次记忆框架，用于在智能体强化学习中实现可溯源的上下文重建。 ECHO 将每个已完成的环境交互轮次压缩为带来源索引的紧凑记忆记录，通过选择有用的记录来重建受限的策略上下文，并复用所选的来源索引，将正向结果信用传递给最后一个轨迹片段、被复用的证据轮次、记忆中的发现以及记忆选择动作。 在 BrowseComp-Plus 上，ECHO 的留出集准确率达到 43.4%，优于 GRPO 的 28.9% 和滚动摘要基线 SUPO 的 36.1%，同时所需轮次和轨迹规模均低于 SUPO。 在稠密与 MoE 两类骨干模型上，训练后的策略还改善了多目标问答、代码生成和深度信息搜寻基准上的零样本泛化表现。

图1：在 BrowseComp-Plus 上，以 Qwen3-32B-Instruct 为骨干模型训练 ECHO（紫色）、GRPO（橙色）和 SUPO（绿色）时，留出集准确率、每次轨迹采样的工具使用轮次以及轨迹规模的变化。ECHO 沿着左上方的前沿发展：准确率上升，却没有出现 SUPO 那样的轮次与轨迹规模增长。

1. Introduction

Large language models (LLMs) are increasingly deployed as multi-turn agents that interleave reasoning, tool invocation, and environment feedback. Reinforcement learning (RL) from verifiable final outcomes has become a central recipe for improving such agents in search, coding, function calling, and deep-research settings. As interaction horizons grow, however, history management becomes a bottleneck for both acting and learning. The policy must retain useful observations within a bounded context window, while the learner must decide which earlier decisions should receive credit from a sparse final outcome.

大语言模型（LLM）越来越多地被部署为多轮智能体，在推理、工具调用与环境反馈之间交替运行。 基于可验证最终结果的强化学习（RL）已成为提升这类智能体在搜索、编码、函数调用及深度研究场景中能力的核心方法。 然而，随着交互时程增长，历史管理成为行动与学习共同面临的瓶颈。 策略必须在有限的上下文窗口内保留有用观测，而学习器则必须决定应根据稀疏的最终结果，将信用分配给哪些早先的决策。

Context-management methods make long rollouts feasible by simplifying past interactions before the next decision. From the perspective of reconstructed history, these methods differ in what form the completed history becomes: some delete or omit distant turns, some edit an explicit memory state, and others fold long prefixes into compact summaries or internal states. The triggering mechanism may be rule-based or action-based, but this choice is orthogonal to the reconstructed form of history. For outcome-based RL, the central question is not only whether the shortened context is useful for acting, but also whether it remains traceable to the original environment turns for learning.

上下文管理方法在下一次决策前简化过去的交互，使长轨迹采样得以进行。 从历史重建的角度看，这些方法的区别在于如何表示已经完成的历史：有些删除或省略较早的轮次，有些编辑显式的记忆状态，还有一些将较长的前缀折叠为紧凑的摘要或内部状态。 触发机制可以基于规则，也可以基于动作，但这种选择独立于历史的重建形式。 对于基于结果的强化学习，核心问题不仅是缩短后的上下文是否有助于行动，还在于它是否仍能追溯到原始环境交互轮次，以支持学习。

This provenance gap affects credit assignment in outcome-based agentic RL. With final-outcome rewards, standard multi-turn RL often applies a trajectory-level advantage to many generated action spans. When the reconstructed context is not source-addressable, such dense assignment can reinforce redundant searches, incidental reasoning, or context-management behavior together with genuinely useful evidence-gathering steps. The issue is therefore not only whether context management enables longer bounded-context rollouts, but whether the reconstructed context exposes the source turns that the learner should credit. This suggests that a context manager should prune history for acting while preserving source-level provenance for learning.

这一溯源缺口会影响基于结果的智能体强化学习中的信用分配。 在使用最终结果奖励时，标准多轮强化学习通常将轨迹级优势应用于大量生成的动作片段。 当重建的上下文无法按来源寻址时，这种密集分配可能会把冗余搜索、附带推理或上下文管理行为，与真正有用的证据收集步骤一并强化。 因此，问题不仅在于上下文管理是否支持有限上下文下的长轨迹采样，还在于重建的上下文是否显式提供了学习器应赋予信用的来源轮次。 这表明，上下文管理器应为行动剪枝历史，同时为学习保留来源层面的溯源信息。

In this paper, we propose ECHO, a selective turn-memory framework for traceable context reconstruction in agentic RL. Instead of collapsing distant history into a single summary state, ECHO compresses each completed tool-use turn into a compact memory record with a source-turn pointer. When the context budget becomes binding, the policy selects useful memory records and reconstructs the next bounded context from these selected records together with recent interactions. The agent therefore acts with compact context, while the selected memories remain explicitly linked to their original environment turns.

本文提出 ECHO，这是一个选择性轮次记忆框架，用于在智能体强化学习中实现可溯源的上下文重建。 ECHO 将每个已完成的工具使用轮次压缩成带有来源轮次指针的紧凑记忆记录，而非将较早的历史折叠成单一摘要状态。 当上下文预算成为约束时，策略会选择有用的记忆记录，并结合这些记录与近期交互，重建下一段受限上下文。 因此，智能体能够基于紧凑上下文行动，而选中的记忆仍与其原始环境交互轮次保持显式关联。

The same source-indexed reconstruction trace is then reused for learning. Standard trajectory-level updates attach the group-relative advantage, whether positive or negative, to all generated tokens in a rollout. In contrast, ECHO first keeps only the positive part of the trajectory advantage, and then applies it only to traceable credit tokens: the final trajectory segment, selected historical source turns, their memory findings, and the memory-selection actions that constructed the reconstructed context. Tokens outside this trace receive no positive outcome credit. In this way, ECHO changes credit assignment from dense trajectory-wide supervision to provenance-guided positive credit routing, aligning context pruning for acting with outcome-credit routing for learning through a single source-indexed trace. Together, these design choices make context reconstruction not only a way to fit long rollouts into a bounded context, but also a learning interface for source-aware credit assignment.

随后，同一份带来源索引的重建溯源记录也被复用于学习。 标准轨迹级更新会将组相对优势——无论正负——附加到一次轨迹采样中所有生成的 token 上。 相比之下，ECHO 首先仅保留轨迹优势的正值部分，然后仅将其应用于可溯源的信用 token：最后一个轨迹片段、选中的历史来源轮次、其中的记忆发现，以及构建重建上下文的记忆选择动作。 这条溯源记录之外的 token 不会获得正向结果信用。 通过这种方式，ECHO 将信用分配从覆盖整条轨迹的密集监督，转变为溯源信息引导的正向信用路由，借助同一份带来源索引的记录，将面向行动的上下文剪枝与面向学习的结果信用路由对齐。 这些设计共同使上下文重建不再只是将长轨迹适配到有限上下文中的手段，也成为支持来源感知信用分配的学习接口。

Our main contributions are as follows:

我们的主要贡献如下：

• We formulate context-managed multi-turn RL with a unified reconstruction interface that links history transformation to source traceability. This view shows why context management can make long rollouts feasible while still removing the provenance needed for outcome-credit assignment.
• We introduce ECHO, a turn-memory method that stores each turn as a source-indexed memory record and reconstructs bounded policy contexts through learned memory selection rather than collapsed-history folding.
• We propose provenance-guided token-level credit assignment, which reuses source indices to route positive outcome advantage to the final trajectory segment, historical evidence turns, memory findings, and memory-selection actions.
• Experiments on BrowseComp-Plus and diverse zero-shot benchmarks show that ECHO improves accuracy while reducing turn proliferation and trajectory volume relative to the closest collapsed-history baseline, with consistent gains across dense and MoE backbones.

• 我们通过统一的重建接口形式化上下文管理下的多轮强化学习，将历史变换与来源可追溯性联系起来。这一视角说明，上下文管理虽然能让长轨迹采样得以进行，却仍可能丢失结果信用分配所需的溯源信息。
• 我们提出 ECHO，这是一种轮次记忆方法，将每轮交互保存为带来源索引的记忆记录，并通过学习式记忆选择来重建受限的策略上下文，而非对历史进行折叠压缩。
• 我们提出由溯源信息引导的 token 级信用分配方法，复用来源索引，将正向结果优势传递给最后一个轨迹片段、历史证据轮次、记忆中的发现以及记忆选择动作。
• BrowseComp-Plus 和多种零样本基准上的实验表明，相较于最接近的历史折叠基线，ECHO 在提升准确率的同时减少了轮次膨胀与轨迹规模，并在稠密和 MoE 骨干模型上都取得了一致收益。

2. Related Work

2.1. RL for Long-Horizon Language Agents

Language agents are commonly formulated as multi-turn decision makers that interleave reasoning, tool invocation, and environment feedback. Recent RL methods improve such agents with outcome-based or verifiable rewards in search, function calling, coding, and deep-research environments. These settings make context management necessary during rollout, because useful observations may span many tool-use turns and exceed the policy context budget. However, most agentic RL objectives treat the reconstructed policy context as given, and do not specify whether the information reused by the policy remains traceable to the original environment turns. ECHO addresses this missing interface between long-horizon rollout construction and learning by making reconstructed context source-indexed.

语言智能体通常被建模为多轮决策者，在推理、工具调用与环境反馈之间交替运行。 近期的强化学习方法利用基于结果或可验证的奖励，提升这类智能体在搜索、函数调用、编码和深度研究环境中的能力。 在这些场景中，有用观测可能跨越许多工具使用轮次，并超出策略的上下文预算，因此轨迹采样过程中必须进行上下文管理。 然而，大多数智能体强化学习目标将重建的策略上下文视为给定输入，并未明确策略复用的信息能否仍追溯到原始环境交互轮次。 ECHO 通过为重建的上下文建立来源索引，补上了长时程轨迹构建与学习之间缺失的接口。

2.2. Context Management and Agent Memory

Prior work addresses long-context limitations through compression, summarization, retrieval, and memory modules. These methods improve bounded-context acting by shortening or externalizing past interactions, but they usually do not define how final outcome rewards should be linked back to the original turns that supplied useful evidence.

已有工作通过压缩、摘要、检索和记忆模块应对长上下文限制。 这些方法通过缩短或外置过去的交互，改善了有限上下文下的行动能力，但通常并未定义如何将最终结果奖励关联回提供了有用证据的原始轮次。

Recent agentic RL methods make context management trainable. Some replace the distant prefix with updated summaries or compact states, while others expose memory or context editing as actions. Under the history-transformation view in Table 1, these methods differ in whether history is deleted, edited, folded, or kept source-indexed. ECHO differs by storing each completed turn as a source-indexed memory record and reusing the selected records both for context reconstruction and outcome-credit routing. This taxonomy motivates our empirical focus on SUPO-style rolling summarization: it is the closest collapsed-history RL counterpart to ECHO under the same agent-loop reconstruction setting, isolating the difference between collapsed folding and source-indexed selection.

近期的智能体强化学习方法使上下文管理变得可训练。 一些方法用更新后的摘要或紧凑状态替换较早的前缀，另一些方法则将记忆或上下文编辑显式设为动作。 从 表1 的历史变换视角看，这些方法的区别在于对历史进行删除、编辑、折叠，还是保留其来源索引。 ECHO 的不同之处在于，它将每个已完成的轮次保存为带来源索引的记忆记录，并将所选记录同时用于上下文重建和结果信用路由。 这一分类促使我们在实验中重点比较 SUPO 式滚动摘要：在相同的智能体循环重建设置下，它是与 ECHO 最接近的历史折叠强化学习方法，因此可以单独考察折叠压缩与来源索引选择之间的差异。

2.3. Credit Assignment in Agentic RL

Credit assignment is challenging when rewards are sparse and observed only after extended interactions. Many outcome-supervised RL methods reuse a trajectory- or response-level advantage across generated action tokens or spans. This dense assignment is simple, but noisy in long-horizon tool-use tasks where only a small subset of searches, observations, and intermediate decisions supports the final answer.

当奖励稀疏且只能在长时间交互结束后观测到时，信用分配就很困难。 许多结果监督的强化学习方法会在生成的动作 token 或片段间复用轨迹级或响应级优势。 这种密集分配虽然简单，但在长时程工具使用任务中噪声较大，因为只有少部分搜索、观测和中间决策真正支撑了最终答案。

Recent methods refine credit along temporal, hindsight, milestone, hierarchical, or uncertainty-based axes. These approaches improve how rewards are distributed over time or policy structure, but are mostly independent of how the agent's effective context was reconstructed. ECHO is complementary: rather than estimating a new temporal advantage, it uses the source indices exposed by context reconstruction to define a provenance-guided token mask for positive outcome credit.

近期方法从时间、事后回溯、里程碑、层次结构或不确定性等维度细化信用分配。 这些方法改进了奖励在时间或策略结构上的分配方式，但大多独立于智能体实际使用的上下文是如何重建的。 ECHO 与这些方法互补：它不估计新的时间维度优势，而是利用上下文重建所提供的来源索引，为正向结果信用定义由溯源信息引导的 token 掩码。

3. Preliminaries

3.1. Multi-Turn GRPO for Agentic RL

Let $x$ denote an initial task prompt. At interaction step $t$, the agent conditions on a policy context $c_t$ and samples an action $a_t\sim {\pi }_{\theta }(\cdot \mid c_t)$. The action may contain reasoning tokens, a tool invocation, an internal context-management operation, or a final answer. If a tool is invoked, the environment returns an observation $o_t$; otherwise $o_t$ may be null. The resulting multi-turn rollout is denoted by

令 $x$ 表示初始任务提示。 在交互步 $t$，智能体以策略上下文 $c_t$ 为条件，采样动作 $a_t\sim {\pi }_{\theta }(\cdot \mid c_t)$。 动作可以包含推理 token、工具调用、内部上下文管理操作或最终答案。 如果调用了工具，环境会返回观测 $o_t$；否则，$o_t$ 可以为空。 由此得到的多轮采样轨迹记为

$$\begin{array}{}\text{(1)}& \tau =(a_1,o_1,\dots ,a_T,o_T),\end{array}$$

and receives a sparse final reward $R(\tau )$.

并获得稀疏的最终奖励 $R(\tau )$。

For each prompt, GRPO samples a group of $N$ complete trajectories $\{{\tau }^{(1)},\dots ,{\tau }^{(N)}\}$. Let $r^{(n)}=R({\tau }^{(n)})$, and let $\overline{r}$ and $s_r$ be the mean and standard deviation of $\{r^{(i)}{\}}_{i=1}^N$. The group-relative advantage is

对于每个提示，GRPO 会采样一组包含 $N$ 条完整轨迹的集合 $\{{\tau }^{(1)},\dots ,{\tau }^{(N)}\}$。 令 $r^{(n)}=R({\tau }^{(n)})$，并令 $\overline{r}$ 和 $s_r$ 分别表示 $\{r^{(i)}{\}}_{i=1}^N$ 的均值与标准差。 组相对优势为

$$\begin{array}{}\text{(2)}& A^{(n)}=\frac{r^{(n)}-\overline{r}}{s_r+ϵ}.\end{array}$$

We use the following simplified policy-gradient form:

我们使用如下简化的策略梯度形式：

$$\begin{array}{}\text{(3)}& \begin{array}{rl}& {\mathcal{J}}_{\mathrm{MT}}(\theta )={\mathbb{E}}_{x\sim \mathcal{D}}[\frac{1}{N}\sum _{n=1}^N\sum _{t=1}^{T_n}\\ & A^{(n)}\log {\pi }_{\theta }(a_t^{(n)}\mid c_t^{(n)})].\end{array}\end{array}$$

This expression suppresses implementation-specific surrogate terms, such as old-policy ratios, clipping or gating, token normalization, and optional KL penalties. Its purpose is to make explicit that a trajectory-level advantage is attached to generated actions conditioned on the policy contexts actually provided to the model. In token-level implementations, each action $a_t$ is expanded into generated tokens, and the corresponding token log probabilities share the same trajectory-level advantage.

该表达式省略了与具体实现有关的替代目标项，例如旧策略比率、裁剪或门控、token 归一化，以及可选的 KL 惩罚。 其目的是明确说明：在以实际提供给模型的策略上下文为条件时，生成动作被赋予的是轨迹级优势。 在 token 级实现中，每个动作 $a_t$ 都被展开为生成的 token，而对应的 token 对数概率共享相同的轨迹级优势。

Here $c_t$ denotes the bounded policy context used for the current model call. Standard multi-turn RL objectives typically treat this context as given. The next subsection makes the context-management mechanism explicit by separating the complete environment-side history from the reconstructed policy context.

这里，$c_t$ 表示当前模型调用使用的受限策略上下文。 标准多轮强化学习目标通常将该上下文视为给定输入。 下一小节将区分环境侧的完整历史与重建的策略上下文，从而显式描述上下文管理机制。

表1：上下文管理与重建（ℳ，Φ）接口下的轨迹采样策略。我们依据各方法如何变换已完成的历史进行分类。示例中的上标表示重建的触发方式：R 表示规则触发，A 表示动作触发。来源溯源一列表示重建的上下文能否仍在原始环境交互轮次层面寻址。

History Transformation	Managed State	Reconstruction Form	Source Trace	Examples
Append-only	$H_j^{\mathrm{pre}}$	$H_j^{\mathrm{pre}}\oplus H_{j,t}^{\mathrm{loc}}$	Explicit	Vanilla prompting
Deletion / pruning	$C_j\subseteq H_j^{\mathrm{pre}}$	$\mathrm{Render}(C_j,H_{j,t}^{\mathrm{loc}};B)$	Partial / lost	Sliding windowR, Agent-OmitA
Edited memory state	$B_j$	$\mathrm{Render}(B_j,H_{j,t}^{\mathrm{loc}};B)$	Action-traced	MemActA, Memory-R1A
Collapsed folding	$z_j=C(z_{j-1},{\sigma }_{j-1})$	$\mathrm{Render}(z_j)\oplus H_{j,t}^{\mathrm{loc}}$	Collapsed	SUPOR, FoldGRPOA
Selective turn memory	${\mathcal{E}}_j=\{e_i{\}}_{i\le K_{j-1}}$	$\mathrm{Render}({\mathcal{E}}_j[{\hat{I}}_j])\oplus H_{j,t}^{\mathrm{loc}}$	Source-indexed	ECHOR

3.2. Context-Managed Multi-Turn Rollouts

In long-horizon tool-use tasks, the complete environment-side history may exceed the context budget used during rollout and training. We write one completed interaction turn as $u_t=(a_t,o_t)$, so that the full rollout can be equivalently written as $\tau =(u_1,\dots ,u_T)$. Let $B$ be the maximum policy context length. A context-managed rollout must construct a policy context $c_t$ satisfying $|c_t|\le B$ at every interaction step.

在长时程工具使用任务中，环境侧的完整历史可能超出轨迹采样和训练时使用的上下文预算。 我们将一个已完成的交互轮次记为 $u_t=(a_t,o_t)$，因此完整采样轨迹也可等价地写为 $\tau =(u_1,\dots ,u_T)$。 令 $B$ 表示策略上下文的最大长度。 采用上下文管理的轨迹采样必须在每个交互步构建满足 $|c_t|\le B$ 的策略上下文 $c_t$。

For clarity, we first describe one trajectory and omit the trajectory superscript. We partition the rollout into bounded-context segments:

为清楚起见，我们先描述单条轨迹，并省略轨迹上标。 我们将采样轨迹划分为上下文受限的片段：

$$\begin{array}{}\text{(4)}& 0=K_0<K_1<\cdots <K_J=T.\end{array}$$

The boundaries $\{K_j\}$ may be triggered by rule-based controllers, such as length or turn-count conditions, or by policy-generated context-management actions. The turns in segment $j$ are indexed by ${\mathcal{T}}_j=\{K_{j-1}+1,\dots ,K_j\}$. For segment $j$, the completed prefix and the local within-segment history before turn $t$ are

边界 $\{K_j\}$ 可以由基于规则的控制器触发，例如长度或轮次数量条件，也可以由策略生成的上下文管理动作触发。 片段 $j$ 中的轮次索引为 ${\mathcal{T}}_j=\{K_{j-1}+1,\dots ,K_j\}$。 对于片段 $j$，已完成的前缀和轮次 $t$ 之前的片段内局部历史分别为

$$\begin{array}{}\text{(5)}& \begin{array}{rl}{H}_j^{\mathrm{pre}}& =(u_1,\dots ,u_{K_{j-1}}),\\ H_{j,t}^{\mathrm{loc}}& =(u_{K_{j-1}+1},\dots ,u_{t-1}),\\ & K_{j-1}<t\le K_j.\end{array}\end{array}$$

A context manager maps the completed prefix into a managed state, and a reconstruction function renders the bounded policy context:

上下文管理器将已完成的前缀映射为管理状态，重建函数则将其呈现为受限的策略上下文：

$$\begin{array}{}\text{(6)}& \begin{array}{rl}{z}_j& =\mathcal{M}(H_j^{\mathrm{pre}}),\\ c_{j,t}& =x\oplus \mathrm{\Phi }(z_j,H_{j,t}^{\mathrm{loc}};B).\end{array}\end{array}$$

Here $z_j$ may be retained history, a summary, a compressed state, an edited memory, or a set of memory units. The function $\mathrm{\Phi }$ combines $z_j$ with the local history under the budget $B$, and $\oplus$ denotes prompt concatenation, or more generally the effective conditioning mechanism.

这里，$z_j$ 可以是保留下来的历史、摘要、压缩状态、经过编辑的记忆，或一组记忆单元。 函数 $\mathrm{\Phi }$ 在预算 $B$ 下结合 $z_j$ 与局部历史，$\oplus$ 表示提示拼接，或更一般地表示实际的条件化机制。

This interface separates two properties of a context manager: whether it keeps the policy context short enough for acting, and whether the reconstructed information remains source-addressable for learning. Traceable methods may expose metadata such as source indices of retained or selected turns, which can later be used for credit assignment.

这一接口将上下文管理器的两个属性区分开来：是否能将策略上下文保持得足够短以支持行动，以及重建的信息是否仍能按来源寻址以支持学习。 可溯源的方法可以提供元数据，例如保留或选中轮次的来源索引，供后续信用分配使用。

With context reconstruction, the reward and group-relative advantage remain unchanged. What changes is only the conditioning path from completed history to policy input:

引入上下文重建后，奖励和组相对优势保持不变。 变化的仅是从已完成的历史到策略输入的条件化路径：

$$H_j^{\mathrm{pre}}\stackrel{\mathcal{M}}{\to }{z}_j\stackrel{\mathrm{\Phi }(\cdot ,H_{j,t}^{\mathrm{loc}};B)}{\to }{c}_{j,t}.$$

At the objective level, this simply replaces the original policy context $c_t^{(n)}$ in Eq. 3 with the reconstructed segment context $c_{j,t}^{(n)}$. For trajectory $n$, let ${\mathcal{T}}_j^{(n)}$ denote the turn indices of segment $j$. Then

在目标函数层面，这只是将 式（3） 中原有的策略上下文 $c_t^{(n)}$ 替换为重建的片段上下文 $c_{j,t}^{(n)}$。 对于轨迹 $n$，令 ${\mathcal{T}}_j^{(n)}$ 表示片段 $j$ 的轮次索引。 则有

$$\begin{array}{}\text{(7)}& \begin{array}{rl}& {\mathcal{J}}_{\mathrm{CM}}(\theta )={\mathbb{E}}_{x\sim \mathcal{D}}[\frac{1}{N}\sum _{n=1}^N\\ & \sum _{j=1}^{J_n}\sum _{t\in {\mathcal{T}}_j^{(n)}}{A}^{(n)}\\ & \log {\pi }_{\theta }(a_t^{(n)}\mid c_{j,t}^{(n)})].\end{array}\end{array}$$

The segment index is only bookkeeping: the segments partition the same rollout, and the learning signal remains the trajectory-level advantage $A^{(n)}$. Thus, different context managers differ not by the reward estimator above, but by their choices of $\mathcal{M}$ and $\mathrm{\Phi }$, which determine the reconstructed context and its source trace. The key question is whether this trace remains addressable at the level of original environment turns, so that outcome credit can be routed to the evidence that the policy actually reused.

片段索引仅用于记录：各片段划分的是同一条采样轨迹，学习信号仍然是轨迹级优势 $A^{(n)}$。 因此，不同上下文管理器的区别不在于上述奖励估计器，而在于对 $\mathcal{M}$ 和 $\mathrm{\Phi }$ 的选择，它们决定了重建的上下文及其来源溯源记录。 关键问题是，这份溯源记录能否仍在原始环境交互轮次层面寻址，从而将结果信用传递给策略实际复用的证据。

4. Method

4.1. Why Traceability Matters

The reconstruction interface in Eq. 6 separates two questions that are often conflated in long-horizon agentic RL: whether the policy can act under a bounded context budget, and whether the bounded context remains traceable to original environment turns. Table 1 organizes context managers by how they transform completed history. Append-only prompting preserves explicit trace but exceeds the context budget; deletion and pruning remove turns; edited-memory methods maintain compact memory states; and collapsed-folding methods replace long prefixes with summaries or compact states. These transformations make long rollouts feasible, but differ in whether reconstructed context remains source-addressable for learning.

式（6） 的重建接口区分了长时程智能体强化学习中经常被混为一谈的两个问题：策略能否在有限上下文预算下行动，以及这段受限上下文能否仍追溯到原始环境交互轮次。 表1 根据上下文管理器如何变换已完成的历史对其分类。 仅追加式提示保留了显式溯源信息，却会超出上下文预算；删除与剪枝会移除轮次；记忆编辑方法维护紧凑的记忆状态；而折叠压缩方法则用摘要或紧凑状态替换较长的前缀。 这些变换使长轨迹采样得以进行，但重建的上下文能否按来源寻址以支持学习，各不相同。

This distinction matters because outcome-based RL assigns credit after the final answer is verified. If the policy conditions on reconstructed context whose source turns are no longer addressable, the learner cannot tell which earlier observations supported the successful decision. A dense trajectory-level update may then reinforce useful evidence-gathering turns together with redundant searches, incidental reasoning, or summary-generation tokens. Thus, context efficiency alone is not enough: the reconstruction process should expose a trace of the evidence that was selected and reused.

这一区别很重要，因为基于结果的强化学习会在最终答案被验证之后才分配信用。 如果策略所依据的重建上下文已无法定位其来源轮次，学习器便无法判断哪些早先的观测支撑了成功的决策。 此时，密集的轨迹级更新可能会将有用的证据收集轮次，与冗余搜索、附带推理或摘要生成 token 一并强化。 因此，仅有上下文效率还不够：重建过程应提供一份记录所选及所复用证据的溯源信息。

Collapsed summarization illustrates the limitation of history compression alone. By folding distant interactions into a compact summary state, it allows the agent to continue beyond the raw-context limit. However, the resulting context no longer exposes which original turns supplied the reused evidence. Figure 2 shows the practical consequence: summarization-based context management extends rollouts, but also produces more tool-use turns, longer responses, higher generation time, and larger trajectory volume. This supports the concern above: making history shorter for acting does not necessarily make learning easier. A context manager should therefore not only compress history, but also preserve explicit source traces that can support outcome-credit assignment.

折叠式摘要说明了仅靠历史压缩的局限。 通过将较早的交互折叠为紧凑摘要状态，它允许智能体在超出原始上下文限制后继续运行。 然而，得到的上下文不再揭示是哪些原始轮次提供了被复用的证据。 图2 展示了实际后果：基于摘要的上下文管理延长了采样轨迹，却也带来了更多工具使用轮次、更长的响应、更高的生成耗时和更大的轨迹规模。 这支持了前述担忧：为行动缩短历史，并不一定会让学习更容易。 因此，上下文管理器不仅应压缩历史，还应保留明确的来源溯源记录，以支持结果信用分配。

图2：长时程搜索中的训练诊断。基于摘要的上下文管理支持更长的轨迹采样，但也导致轮次膨胀、响应变长、生成耗时上升以及轨迹规模增加。

4.2. Selective Turn Memory

ECHO instantiates source-indexed reconstruction with selective turn memory. The goal is to keep the active policy context compact without folding the distant prefix into a single untraceable state. To do so, ECHO separates local turn compression from global context reconstruction: each completed turn is compressed independently into a source-indexed memory record, and the next policy context is reconstructed by selecting useful records from the resulting memory set.

ECHO 通过选择性轮次记忆实现基于来源索引的重建。 其目标是保持当前策略上下文紧凑，同时避免将较早的前缀折叠为单个不可溯源的状态。 为此，ECHO 将局部轮次压缩与全局上下文重建分开：每个已完成的轮次都被独立压缩为带来源索引的记忆记录，而下一段策略上下文则通过从得到的记忆集合中选择有用记录来重建。

Source-indexed memory. After each completed tool-use turn $u_i=(a_i,o_i)$, ECHO asks the policy to write a compact local finding:

带来源索引的记忆。 每个工具使用轮次 $u_i=(a_i,o_i)$ 完成后，ECHO 都会要求策略写出一条紧凑的局部发现：

$$\begin{array}{}\text{(8)}& \begin{array}{rl}{s}_i& \sim {\pi }_{\theta }(\cdot \mid c_i^{\mathrm{sum}}),\\ m_i& =\mathrm{Parse}(s_i).\end{array}\end{array}$$

Here $c_i^{\mathrm{sum}}$ contains the latest completed turn and a summary instruction that asks the model to summarize only that turn. This finding is generated by the same policy being trained, rather than by an external summarizer or a separate memory model. The memory record is

这里，$c_i^{\mathrm{sum}}$ 包含最近完成的轮次，以及要求模型仅概括该轮内容的摘要指令。 这条发现由正在训练的同一个策略生成，而非由外部摘要器或独立记忆模型生成。 记忆记录为

$$\begin{array}{}\text{(9)}& \begin{array}{rl}{e}_i& =(i,{\alpha }_i,m_i),\\ {\mathcal{E}}_j& =\{e_i{\}}_{i\le K_{j-1}}.\end{array}\end{array}$$

The index $i$ is the source-turn pointer, ${\alpha }_i$ is a compact rendering of the action or tool call, and $m_i$ is the parsed last-turn finding. Unlike a rolling summary, ${\mathcal{E}}_j$ is a persistent, non-collapsing archive. Selection changes only the active view rendered into the next policy context; it does not delete unselected records. Thus, every memory remains individually addressable at later reconstruction boundaries even after its raw observation is no longer placed in the active context.

索引 $i$ 是来源轮次指针，${\alpha }_i$ 是动作或工具调用的紧凑表示，$m_i$ 则是解析后的上一轮发现。 与滚动摘要不同，${\mathcal{E}}_j$ 是持久保存且不会折叠合并的存档。 选择过程只会改变呈现到下一段策略上下文中的当前视图；它不会删除未选中的记录。 因此，即使某条记忆的原始观测已不再置于当前上下文中，它在后续重建边界处仍可被单独寻址。

Memory selection. At a compression boundary before segment $j$, ECHO lets the policy select historical memories that are useful for continuing the task. Let $H_j^{\mathrm{bd}}$ denote the bounded local state available at the boundary. The selection context is

记忆选择。 在片段 $j$ 之前的压缩边界处，ECHO 让策略选择有助于继续完成任务的历史记忆。 令 $H_j^{\mathrm{bd}}$ 表示边界处可用的受限局部状态。 选择上下文为

$$\begin{array}{}\text{(10)}& \begin{array}{rl}{c}_j^{\mathrm{sel}}& =x\oplus {\mathrm{\Phi }}_{\mathrm{local}}(H_j^{\mathrm{bd}};B)\\ & \oplus \mathrm{RenderList}({\mathcal{E}}_j)\oplus x_{\mathrm{sel}},\end{array}\end{array}$$

and the policy generates a selection action

策略随后生成一个选择动作

$$\begin{array}{}\text{(11)}& a_j^{\mathrm{sel}}\sim {\pi }_{\theta }(\cdot \mid c_j^{\mathrm{sel}}).\end{array}$$

The instruction $x_{\mathrm{sel}}$ asks the model to output source indices of reusable evidence, constraints, failed attempts, or useful next-step plans. A repair operator converts the generated text into a valid bounded set:

指令 $x_{\mathrm{sel}}$ 要求模型输出可复用证据、约束、失败尝试或有用的下一步计划所对应的来源索引。 修复算子将生成文本转换为合法且规模受限的集合：

$$\begin{array}{}\text{(12)}& {\hat{I}}_j={\rho }_{B,S,K}(\mathrm{Parse}(a_j^{\mathrm{sel}}),{\mathcal{E}}_j).\end{array}$$

The operator ${\rho }_{B,S,K}$ removes malformed or out-of-range indices, deduplicates selections, keeps at most $S$ model-selected historical turns, and merges them with the latest $K$ turns retained automatically. The rendered view is then validated against the context budget; a rollout is marked overlong if the bounded view still cannot be constructed. Thus, both memory writing and memory selection are policy-generated and trained within the same RL loop.

算子 ${\rho }_{B,S,K}$ 会移除格式错误或越界的索引，对选择结果去重，最多保留 $S$ 个模型选中的历史轮次，并将它们与自动保留的最近 $K$ 轮合并。 随后，依据上下文预算验证呈现后的视图；如果仍无法构建受限视图，则将该次轨迹采样标记为超长。 因此，记忆写入与记忆选择都由策略生成，并在同一个强化学习循环中训练。

Context reconstruction. The selected indices determine which memories are used for reconstruction:

上下文重建。 所选索引决定了哪些记忆被用于重建：

$$\begin{array}{}\text{(13)}& {\mathcal{E}}_j[{\hat{I}}_j]=\{e_i\in {\mathcal{E}}_j\mid i\in {\hat{I}}_j\}.\end{array}$$

For any turn $t$ in segment $j$, ECHO reconstructs the policy context as

对于片段 $j$ 中的任意轮次 $t$，ECHO 将策略上下文重建为

$$\begin{array}{}\text{(14)}& \begin{array}{rl}{c}_{j,t}& =x\oplus \mathrm{Render}({\mathcal{E}}_j[{\hat{I}}_j])\\ & \oplus {\mathrm{\Phi }}_{\mathrm{local}}(H_{j,t}^{\mathrm{loc}};B),\end{array}\end{array}$$

with $|c_{j,t}|\le B$. The agent acts on a compact context composed of selected memories and recent interactions. At the same time, the selected set ${\hat{I}}_j$ preserves a source-indexed reconstruction trace, which is reused for credit routing.

其中 $|c_{j,t}|\le B$。 智能体基于由所选记忆与近期交互组成的紧凑上下文行动。 与此同时，所选集合 ${\hat{I}}_j$ 保留了一份带来源索引的重建溯源记录，并将其复用于信用路由。

4.3. Traceable Credit Routing

图3：ECHO 概览。ECHO 将已完成的轮次保存为带来源索引的记忆，选择有用记忆以重建受限上下文，并复用同一份来源溯源记录进行信用分配。

ECHO uses a final-trace approximation for outcome-based learning: it routes credit through the source turns selected into the final reconstructed context, rather than recursively tracking all intermediate reconstruction dependencies. This final reconstruction trace defines which generated tokens are eligible for positive outcome credit.

ECHO 在基于结果的学习中采用最终溯源近似：它沿着被选入最终重建上下文的来源轮次传递信用，而非递归跟踪所有中间重建依赖。 这份最终重建溯源记录界定了哪些生成 token 有资格获得正向结果信用。

For trajectory $n$, let $I_{\mathrm{src}}^{(n)}$ be the source turns selected into the final reconstructed context under this final-trace approximation. Let $j$ index saved trajectory segments and $q$ index a generated response-token position within segment $j$. For each token, we record four indicators: $g_{j,q}^{(n)}$ marks trainable policy tokens in the final saved segment, $d_{j,q}^{(n)}$ gives the source turn of a normal assistant/action token, $f_{j,q}^{(n)}$ gives the source turn of a generated memory-finding token, and $b_{j,q}^{(n)}$ marks memory-selection tokens. ECHO defines the hard credit mask

对于轨迹 $n$，令 $I_{\mathrm{src}}^{(n)}$ 表示在这一最终溯源近似下，被选入最终重建上下文的来源轮次。 令 $j$ 表示保存的轨迹片段索引，$q$ 表示片段 $j$ 中生成的响应 token 的位置。 对于每个 token，我们记录四个指示量：$g_{j,q}^{(n)}$ 标记最后一个已保存片段中可训练的策略 token，$d_{j,q}^{(n)}$ 给出普通助手/动作 token 的来源轮次，$f_{j,q}^{(n)}$ 给出生成的记忆发现 token 的来源轮次，而 $b_{j,q}^{(n)}$ 标记记忆选择 token。 ECHO 定义如下硬信用掩码

$$\begin{array}{}\text{(15)}& \begin{array}{rl}{\mu }_{j,q}^{(n)}=\mathbf{1}[& g_{j,q}^{(n)}=1\\ & \vee d_{j,q}^{(n)}\in I_{\mathrm{src}}^{(n)}\\ & \vee f_{j,q}^{(n)}\in I_{\mathrm{src}}^{(n)}\\ & \vee b_{j,q}^{(n)}=1].\end{array}\end{array}$$

The mask marks tokens that are eligible for positive credit: all trainable tokens in the final trajectory segment, action tokens from selected source turns, their memory-finding tokens, and generated memory-selection spans. The final segment begins after the last context reconstruction and includes its assistant reasoning and tool-call tokens together with the terminal response.

该掩码标记有资格获得正向信用的 token：最后一个轨迹片段中所有可训练的 token、所选来源轮次的动作 token、这些轮次对应的记忆发现 token，以及生成的记忆选择片段。 最后一个片段从最后一次上下文重建之后开始，包含其中的助手推理和工具调用 token，以及最终响应。

For binary verifier rewards, the selected trace is reliable only when the rollout is better than the group baseline. If the rollout fails, selected turns may be irrelevant, misleading, or useful but misused. We therefore route only the positive part of the group-relative advantage:

对于二元验证器奖励，只有当采样轨迹优于组内基线时，所选的溯源记录才可靠。 如果轨迹采样失败，所选轮次可能无关、具有误导性，或虽有用却被错误使用。 因此，我们仅传递组相对优势的正值部分：

$$\begin{array}{}\text{(16)}& \begin{array}{rl}{A}_{+}^{(n)}& =max(A^{(n)},0),\\ {\tilde{A}}_{j,q}^{(n)}& ={\mu }_{j,q}^{(n)}{A}_{+}^{(n)}.\end{array}\end{array}$$

Because credit is multiplied by $A_{+}^{(n)}$, only positive-advantage rollouts assign nonzero credit to the masked tokens. Memory-selection tokens are included in the eligible set because they choose the memories that later successful decisions rely on. The per-trajectory traceable surrogate is

由于信用乘以 $A_{+}^{(n)}$，只有优势为正的采样轨迹才会向掩码选中的 token 分配非零信用。 记忆选择 token 也被纳入可获信用的集合，因为它们选择了后续成功决策所依赖的记忆。 逐轨迹的可溯源替代目标为

$$\begin{array}{}\text{(17)}& \begin{array}{rl}& {\mathcal{J}}_{\mathrm{ECHO}}^{(n)}(\theta )=\sum _j\sum _q{\tilde{A}}_{j,q}^{(n)}\\ & \log {\pi }_{\theta }(y_{j,q}^{(n)}\mid c_j^{(n)},y_{j,<q}^{(n)}).\end{array}\end{array}$$

This surrogate has the same simplified policy-gradient interpretation as Eq. 3, but replaces dense trajectory-wide weighting with the trace-masked positive weight ${\tilde{A}}_{j,q}^{(n)}$. Thus, ECHO turns source-indexed reconstruction into a token-level credit route.

这一替代目标与 式（3） 具有相同的简化策略梯度解释，但将覆盖整条轨迹的密集加权替换为经过溯源掩码筛选的正向权重 ${\tilde{A}}_{j,q}^{(n)}$。 由此，ECHO 将基于来源索引的重建转化为 token 级信用传递路径。

5. Experiment

图4：以 Qwen3-30B-A3B-Instruct MoE 为骨干模型时，在 BrowseComp-Plus 上的训练动态。各子图展示留出集准确率、平均工具使用轮次和轨迹规模；ECHO 提升了准确率，同时没有出现 SUPO 那样的轮次与轨迹规模增长。

5.1. Experimental Setup

We use SUPO-style rolling summarization as the main context-managed RL baseline because it is the closest collapsed-history counterpart to ECHO. Both methods perform context reconstruction inside the agent loop and train the policy to generate the text used for context management, rather than relying on an external retriever or a separate memory model. The key difference is the form of reconstructed history: SUPO folds distant interactions into a collapsed summary state, whereas ECHO reconstructs context from selected source-indexed turn memories and reuses the selected source trace for credit routing.

我们使用 SUPO 式滚动摘要作为主要的上下文管理强化学习基线，因为它是与 ECHO 最接近的历史折叠方法。 两种方法都在智能体循环内部执行上下文重建，并训练策略生成用于上下文管理的文本，而不依赖外部检索器或独立记忆模型。 关键区别在于重建历史的形式：SUPO 将较早的交互折叠为合并的摘要状态，而 ECHO 根据选中的、带来源索引的轮次记忆重建上下文，并复用所选的来源溯源记录进行信用路由。

We evaluate on BrowseComp-Plus, a long-horizon tool-use QA benchmark, and compare GRPO, SUPO-style rolling summarization, and ECHO. All methods use the same search/open-page tool environment, verifier, 8 rollouts per prompt, and 32k-token working context budget. For context-managed methods, we allow at most 5 reconstruction rounds, which gives an effective interaction budget of up to 192k tokens while keeping each policy call within the 32k working context. ECHO automatically retains the latest 3 turns and selects up to 8 additional historical source turns at each reconstruction boundary. We report held-out pass@1 and rollout-efficiency metrics in the main text; dataset statistics, retrieval settings, and zero-shot benchmark details are provided in the appendix.

我们在长时程工具使用问答基准 BrowseComp-Plus 上评估，并比较 GRPO、SUPO 式滚动摘要和 ECHO。 所有方法使用相同的搜索/打开网页工具环境、验证器、每个提示 8 次轨迹采样，以及 32k token 的工作上下文预算。 对于采用上下文管理的方法，我们最多允许 5 轮重建，在将每次策略调用限制在 32k 工作上下文内的同时，提供最多 192k token 的有效交互预算。 ECHO 自动保留最近 3 轮，并在每个重建边界处额外选择最多 8 个历史来源轮次。 正文报告留出集 pass@1 和轨迹采样效率指标；数据集统计、检索设置及零样本基准的详细信息见附录。

5.2. Main Results

Figure 1 shows that ECHO reaches 43.4% held-out accuracy on the dense backbone, outperforming GRPO (28.9%) and SUPO (36.1%). GRPO remains short but plateaus early, indicating that raw bounded-context training limits the amount of evidence the agent can gather. SUPO improves accuracy by extending rollouts, but this comes with substantial turn proliferation: its final turn count reaches 62.5, with an 85.5% trajectory split rate and 4.18 trajectories per rollout. ECHO surpasses SUPO while ending with fewer turns (45.3), a lower split rate (57.8%), and fewer trajectories per rollout (3.13). These results suggest that ECHO's source-indexed selection and traceable credit routing help the agent reuse useful evidence more selectively, yielding more effective search paths under the same bounded-context setting.

图1 表明，在稠密骨干模型上，ECHO 的留出集准确率达到 43.4%，优于 GRPO（28.9%）和 SUPO（36.1%）。 GRPO 的轨迹保持较短，却很早进入平台期，表明直接使用受限上下文训练会限制智能体能够收集的证据量。 SUPO 通过延长采样轨迹提高准确率，但也伴随显著的轮次膨胀：最终轮次数达到 62.5，轨迹拆分率为 85.5%，每次轨迹采样产生 4.18 条轨迹。 ECHO 超过了 SUPO，同时最终轮次更少（45.3）、拆分率更低（57.8%），每次轨迹采样产生的轨迹也更少（3.13）。 这些结果表明，ECHO 基于来源索引的选择与可溯源信用路由，有助于智能体更有选择地复用有用证据，从而在相同的有限上下文设置下得到更有效的搜索路径。

Figure 4 shows the same ranking with the sparse Qwen3-30B-A3B-Instruct MoE backbone. GRPO reaches only 22.9% accuracy. SUPO initially benefits from rolling summaries, but its turn count and trajectory volume grow rapidly and accuracy drops to 13.3% by step 50. In contrast, ECHO remains stable and finishes around 35.0%. This suggests that source-indexed selection and traceable credit routing are not specific to the dense backbone.

图4 表明，在稀疏的 Qwen3-30B-A3B-Instruct MoE 骨干模型上，各方法的排名相同。 GRPO 的准确率仅达到 22.9%。 SUPO 起初受益于滚动摘要，但其轮次数和轨迹规模迅速增长，到第 50 步时准确率降至 13.3%。 相比之下，ECHO 保持稳定，最终准确率约为 35.0%。 这表明，基于来源索引的选择与可溯源信用路由并非仅适用于稠密骨干模型。

(a) 记忆组件消融。

(b) 信用分配消融。

图5：记忆组件与信用分配的消融实验。学习式来源选择效果最强，紧凑的局部发现已足够使用，而移除可溯源信用路由会降低准确率与稳定性。

5.3. Ablation Study

Memory ablations. Figure 5a studies how the memory component should be constructed and retrieved. Replacing learned source selection with static semantic top-k retrieval keeps the reconstructed context compact, but yields lower accuracy than full ECHO. This suggests that the benefit of turn memory does not come only from storing compressed historical records; the policy must also learn which records are useful for the current decision. Replacing compact turn findings with full observations also brings no clear gain, indicating that local findings are sufficient for preserving reusable evidence while avoiding unnecessary context growth. Together, these results support the design choice of storing compact source-indexed findings and performing policy-driven selection at reconstruction time.

记忆消融。 图5a 研究了记忆组件应如何构建和检索。 用静态语义 top-k 检索替代学习式来源选择，虽然仍能保持重建上下文紧凑，但准确率低于完整 ECHO。 这表明，轮次记忆的收益不只来自保存压缩后的历史记录；策略还必须学会哪些记录对当前决策有用。 将紧凑的轮次发现替换为完整观测也未带来明显收益，说明局部发现已足以保留可复用证据，同时避免不必要的上下文增长。 综合来看，这些结果支持如下设计：保存带来源索引的紧凑发现，并在重建时由策略驱动选择。

Credit-routing ablations. Figure 5b evaluates whether the selected source trace should also guide credit assignment. Removing traceable credit assignment lowers accuracy and leads to longer interaction paths with more turns and larger trajectory volume. This suggests that source-indexed reconstruction alone is not sufficient: if the final outcome advantage is densely assigned to all generated tokens, the learner may still reinforce redundant searches, incidental reasoning, and other tokens that merely co-occur with a successful rollout. Adding turn-level importance sampling does not solve this issue; in our training curves, it becomes unstable and collapses as turn count and trajectory volume grow. In contrast, ECHO routes positive credit only through the final trajectory segment, selected evidence turns, their memory findings, and the selection actions that exposed those memories. This acts as a denoising mechanism for outcome credit and helps compress successful search paths rather than merely extending them.

信用路由消融。 图5b 评估了所选的来源溯源记录是否也应指导信用分配。 移除可溯源信用分配会降低准确率，并导致交互路径更长、轮次更多、轨迹规模更大。 这表明，仅有基于来源索引的重建还不够：如果将最终结果优势密集分配给所有生成 token，学习器仍可能强化冗余搜索、附带推理，以及其他仅仅与成功轨迹共同出现的 token。 加入轮次级重要性采样也不能解决这一问题；在我们的训练曲线中，随着轮次数和轨迹规模增长，它会变得不稳定并发生崩溃。 相比之下，ECHO 仅沿最后一个轨迹片段、选中的证据轮次、其中的记忆发现，以及提供这些记忆的选择动作传递正向信用。 这起到了对结果信用去噪的作用，有助于压缩成功的搜索路径，而不只是延长它们。

表2：各类基准上的零样本泛化结果。粗体表示最佳性能，下划线表示次佳性能。CA 表示信用分配，Traceable CA 表示基于来源索引的信用路由，Turn-level IS 表示在轮次级聚合的重要性采样比率。

Method	Multi-Objective QA					Code Generation		Deep Information Seeking			Avg.
	2-obj.	4-obj.	8-obj.	16-obj.	Avg.	CodeGym	LoCoBench- Agent	GAIA	HLE	Frames
Backbone: Qwen3-32B-Instruct
GRPO	38.6	39.8	35.8	29.0	35.8	32.8	67.7	25.2	8.8	24.8	33.6
SUPO	40.9	36.4	36.4	34.7	37.1	35.4	68.1	25.2	9.2	26.8	34.8
ECHO w/ Top-K retrieval	47.7	42.0	39.2	27.8	39.2	40.7	69.3	29.1	10.6	37.3	38.2
ECHO w/ Top-K retrieval & w/o turn summary	40.9	44.3	35.8	35.5	39.1	40.3	69.5	23.3	10.0	31.3	36.8
ECHO w/o traceable CA	45.5	42.0	39.2	22.7	37.4	38.1	68.2	25.2	8.8	30.8	35.6
ECHO w/o Traceable CA & Turn-level IS	45.5	47.7	35.2	27.0	38.8	34.6	70.1	23.3	9.4	32.2	36.1
ECHO	47.7	45.5	41.5	36.1	42.7	41.4	70.4	29.1	11.4	39.1	40.2
Backbone: Qwen3-30B-A3B-Instruct
GRPO	27.3	26.1	27.3	16.2	24.2	20.3	65.7	23.3	7.8	19.1	25.9
SUPO	25.0	30.7	27.3	18.2	25.3	27.3	65.1	23.3	8.0	17.0	26.9
ECHO	34.1	36.4	30.1	18.8	29.9	29.7	66.8	24.3	9.2	25.0	30.5

5.4. Training Efficiency

图6：在 BrowseComp-Plus 上的训练时间效率。ECHO 的训练步耗时、生成耗时和工具调用耗时均低于 SUPO。

Figure 6 compares wall-clock cost during dense-backbone training. ECHO introduces early overhead because each completed tool turn writes a sum_last_turn finding and compression boundaries add selection actions. As training proceeds, however, SUPO's rolling summaries lead to rapidly growing interaction rounds, making tool calls the dominant cost. By tying reconstruction to a bounded set of selected source turns, ECHO limits this mid-to-late-stage turn proliferation and achieves higher accuracy with fewer splits and lower tool-call-dominated execution time.

图6 比较了稠密骨干模型训练过程中的实际时间开销。 ECHO 在早期引入了额外开销，因为每个已完成的工具使用轮次都要写入一条 sum_last_turn 发现，而且压缩边界处还会增加选择动作。 然而，随着训练进行，SUPO 的滚动摘要使交互轮数快速增长，工具调用因而成为主要开销。 通过将重建限定为使用一组数量受限的所选来源轮次，ECHO 抑制了训练中后期的轮次膨胀，以更少的拆分次数和更低的、由工具调用主导的执行耗时取得了更高的准确率。

5.5. Zero-shot Generalization Comparison

The learned context-management behavior also transfers beyond BrowseComp-Plus when the trained policies are evaluated in the same agent-loop framework without benchmark-specific further training (Table 2). These results therefore measure zero-shot agentic transfer, rather than bare-model generation performance. On Qwen3-32B-Instruct, ECHO achieves the best average score of 40.2%, outperforming GRPO (33.6%) and SUPO (34.8%), with especially large gains on evidence-heavy 16-objective QA and Frames. Under the MoE backbone, ECHO reaches 30.5% average score versus 26.9% for SUPO. The zero-shot ablation rows follow the same trend as the BrowseComp-Plus training curves: replacing learned selection with semantic top-k retrieval lowers the dense-backbone average to 38.2%, and removing traceable credit further reduces it to 35.6%. These results suggest that the benefits of source-indexed reconstruction and traceable credit routing transfer across tasks and backbone architectures.

将训练后的策略置于相同的智能体循环框架中评估，无需针对具体基准进一步训练，学到的上下文管理行为也能迁移到 BrowseComp-Plus 之外（表2）。 因此，这些结果衡量的是智能体的零样本迁移能力，而非裸模型的生成性能。 在 Qwen3-32B-Instruct 上，ECHO 取得最佳平均得分 40.2%，优于 GRPO（33.6%）和 SUPO（34.8%），其中在高度依赖证据的 16 目标问答和 Frames 上提升尤为明显。 在 MoE 骨干模型上，ECHO 的平均得分达到 30.5%，而 SUPO 为 26.9%。 零样本消融各行与 BrowseComp-Plus 训练曲线呈现相同趋势：以语义 top-k 检索替代学习式选择，会将稠密骨干模型的平均得分降至 38.2%；移除可溯源信用后，得分进一步降至 35.6%。 这些结果表明，基于来源索引的重建与可溯源信用路由的收益能够跨任务、跨骨干模型架构迁移。

6. Conclusion

We presented ECHO, a selective turn-memory framework that stores each completed turn as a source-indexed memory, reconstructs compact contexts through learned selection, and reuses the same trace to route positive outcome credit to the final trajectory segment, reused evidence, memory findings, and selection actions. Under a controlled agent-loop context-management setting, ECHO improves over the closest collapsed-history baseline, SUPO, while using fewer turns and lower trajectory volume. Ablations and zero-shot evaluations further show that learned source selection and traceable credit routing both contribute to robust transfer. These results suggest that long-horizon agentic RL benefits not only from shortening history, but from preserving a source-indexed trace of the history that the policy actually reuses. Looking ahead, we plan to extend ECHO from final-trace approximation to all-trace credit routing and train it on real software-engineering tasks with executable feedback, such as SWE-Bench. We will also compare its rule-triggered context management with action-based memory operations, including editing, omission, and folding, to further evaluate its generality.

我们提出 ECHO，这是一个选择性轮次记忆框架，将每个已完成的轮次保存为带来源索引的记忆，通过学习式选择重建紧凑上下文，并复用同一份溯源记录，将正向结果信用传递给最后一个轨迹片段、被复用的证据、记忆中的发现和选择动作。 在受控的智能体循环上下文管理设置下，ECHO 超过了最接近的历史折叠基线 SUPO，同时所需轮次更少、轨迹规模更低。 消融实验与零样本评估进一步表明，学习式来源选择和可溯源信用路由都有助于实现稳健迁移。 这些结果说明，长时程智能体强化学习的收益不仅来自缩短历史，还来自保留策略实际复用历史的来源索引记录。 未来，我们计划将 ECHO 从最终溯源近似扩展为全溯源信用路由，并在带可执行反馈的真实软件工程任务上训练，例如 SWE-Bench。 我们还将比较其规则触发的上下文管理与基于动作的记忆操作，包括编辑、省略和折叠，以进一步评估其通用性。