SKILL.state:可扩展的长时程智能体技能
Abstract
Large Language Models (LLMs) increasingly act as autonomous agents executing complex, long-running procedural skills. Existing agent runtimes maintain execution by continually appending observations, actions, and intermediate reasoning traces to an ever-growing conversation history, causing latency degradation and context-poisoning failures over long horizons. We present SKILL.state, a runtime architecture that replaces append-only conversational history with an explicit, mutable execution state. At each execution step, the model receives only the immutable skill specification, the current structured execution state, and the latest observation. Intermediate reasoning is discarded immediately after producing a validated state update, preventing prompt growth with execution history. Across diverse datasets, models, and execution environments, SKILL.state improves task accuracy while substantially reducing cumulative token consumption. Our results demonstrate that explicit execution state is an effective and architecture-agnostic abstraction for scalable long-horizon agent skills.
大语言模型(LLM)正越来越多地充当自主智能体,执行复杂且持续时间较长的程序性技能。 现有智能体运行时通过不断向日益增长的对话历史追加观测、动作和中间推理轨迹来维持执行,这会在长时程任务中导致延迟恶化和上下文污染引发的失败。 我们提出 SKILL.state,这是一种以显式、可变的执行状态取代仅追加式对话历史的运行时架构。 在每个执行步骤中,模型仅接收不可变的技能规范、当前的结构化执行状态以及最新观测。 一旦生成了通过验证的状态更新,中间推理便会立即被丢弃,从而避免提示词随执行历史增长。 在多种数据集、模型和执行环境中,SKILL.state 提升了任务准确率,同时大幅减少了累计 token 消耗。 我们的结果表明,显式执行状态是一种有效且不依赖特定架构的抽象,能够支持可扩展的长时程智能体技能。
1. Introduction
Large Language Models (LLMs) have rapidly evolved from passive language interfaces into autonomous systems capable of iterative reasoning, tool use, and interaction with external environments. Recent work further demonstrates that these capabilities can be encapsulated as reusable procedural skills, enabling agents to perform software engineering, workflow automation, web interaction, and scientific discovery through modular compositions of specialized behaviors. As agents increasingly execute long-running procedures, execution itself becomes a systems problem rather than purely a reasoning problem.
大语言模型(LLM)已迅速从被动的语言接口发展为能够进行迭代推理、使用工具并与外部环境交互的自主系统。 近期研究进一步表明,这些能力可以封装为可复用的程序性技能,使智能体能够通过对专门行为的模块化组合,完成软件工程、工作流自动化、网页交互和科学发现等任务。 随着智能体越来越多地执行长时间运行的过程,执行本身成为了系统问题,而不再仅仅是推理问题。
Modern agent runtimes almost universally adopt a conversational execution model. At every execution step, the language model receives the original skill specification together with an ever-growing transcript of previous reasoning, actions, observations, and tool outputs. Although memory systems alleviate context growth through summarization or retrieval, they preserve the same execution semantics: future decisions are conditioned on textual reconstructions of past execution rather than an explicit representation of the current execution state.
现代智能体运行时几乎普遍采用对话式执行模型。 在每个执行步骤中,语言模型都会接收原始技能规范,以及不断增长的记录,其中包含此前的推理、动作、观测和工具输出。 尽管记忆系统通过摘要或检索缓解了上下文增长,它们仍保留相同的执行语义:未来决策依赖于对过去执行过程的文本重建,而非对当前执行状态的显式表示。
This design introduces fundamental limitations for long-horizon procedural skills. Prompt size grows with execution length, increasing token consumption and inference cost. Historical observations and obsolete reasoning remain embedded in the context long after they cease to be relevant, requiring the model to continually distinguish current facts from historical artifacts. Consequently, execution correctness increasingly depends on reconstructing state from accumulated textual history.
这种设计给长时程程序性技能带来了根本性限制。 提示词长度随执行时长增长,增加了 token 消耗和推理成本。 历史观测和过时推理在失去相关性很久以后仍然留在上下文中,要求模型持续区分当前事实与历史遗留信息。 因此,执行正确性越来越依赖于从积累的文本历史中重建状态。
In this paper, we introduce SKILL.state, a runtime architecture that reformulates procedural skill execution as explicit state transitions rather than conversational history accumulation. Figure 1 provides an overview of the proposed runtime. At execution step
本文提出 SKILL.state,一种将程序性技能执行重新表述为显式状态转移、而非对话历史累积的运行时架构。 图1 展示了所提出运行时的整体结构。 在执行步骤
where
其中,
To evaluate this hypothesis, we evaluate SKILL.state across both synthetic and real-world benchmarks: SkillExecBench, a controlled benchmark designed for long-horizon procedural skill execution under scaling, noise, and state recovery; InterCode CTF, featuring interactive Linux terminal exploitation; and Sierra
为检验这一假设,我们在合成基准和真实世界基准上评估 SKILL.state:SkillExecBench 是一个受控基准,旨在从执行规模、噪声和状态恢复三个方面评估长时程程序性技能执行;InterCode CTF 以交互式 Linux 终端漏洞利用为特色;Sierra
Experimental results demonstrate that explicit execution state substantially improves the scalability of long-horizon procedural skills by maintaining bounded prompt sizes while cutting token consumption and outperforming history-based and compression-based baselines across multiple model families.
实验结果表明,显式执行状态能够维持有界的提示词长度,同时减少 token 消耗,并在多个模型系列上超越基于历史和基于压缩的基线,从而显著提高长时程程序性技能的可扩展性。
Our contributions are summarized as follows:
我们的贡献概括如下:
- We propose SKILL.state, a runtime architecture that executes procedural skills through explicit structured execution state where intermediate reasoning is discarded after each step, proving a strictly bounded
prompt footprint and cumulative token complexity.
- We present SkillExecBench, alongside evaluations on public benchmarks (InterCode CTF and Sierra
-Bench), for evaluating long-horizon procedural skill execution in sequential, stateful environments.
- Across multiple execution horizons and runtime baselines, we demonstrate that state-centric execution maintains competitive task performance while substantially reducing prompt growth and cumulative token consumption across both proprietary and open-weight models.
- 我们提出 SKILL.state,一种通过显式结构化执行状态来执行程序性技能的运行时架构,在每个步骤后丢弃中间推理,并证明其提示词规模严格有界,为
,累计 token 复杂度为 。
- 我们提出 SkillExecBench,并结合公共基准(InterCode CTF 和 Sierra
-Bench)上的评估,用于检验顺序、有状态环境中的长时程程序性技能执行。
- 通过多个执行时程和运行时基线,我们证明,以状态为中心的执行方式能够保持有竞争力的任务性能,同时在专有模型和开放权重模型上显著减少提示词增长与累计 token 消耗。
2. Related Work
2.1. Procedural Skills for LLM Agents
Existing research on reusable procedural skills primarily addresses skill discovery, representation, composition and and security threat modeling. Our work instead focuses on the largely unexplored mechanics of skill execution once a skill has been selected.
现有关于可复用程序性技能的研究主要关注技能发现、表示、组合以及安全威胁建模。 我们的工作则聚焦于技能选定之后的执行机制,这一方面尚未得到充分研究。
2.2. Memory Architectures for Long-Horizon Agents
Long-horizon agent architectures typically preserve conversational semantics through episodic retrieval or persistent storage. These methods leave execution state implicitly distributed across accumulated logs. SKILL.state instead isolates execution into an explicit, mutable runtime state, eliminating the need to repeatedly reconstruct world models from textual history. Frameworks like LangGraph use auxiliary structured state to orchestrate workflows across agent nodes. However, these systems still rely on conversational transcripts as the primary reasoning substrate. SKILL.state replaces this substrate by discarding intermediate reasoning traces immediately after producing validated state transitions.
长时程智能体架构通常通过情景检索或持久化存储来保留对话语义。 这些方法使执行状态隐式地分散在累积的日志中。 SKILL.state 则将执行过程独立为显式、可变的运行时状态,消除了从文本历史中反复重建世界模型的需要。 LangGraph 等框架使用辅助的结构化状态来编排智能体节点之间的工作流。 然而,这些系统仍将对话记录作为主要推理基础。 SKILL.state 在生成通过验证的状态转移后立即丢弃中间推理轨迹,以此取代上述推理基础。
2.3. Dialogue State Tracking
Dialogue State Tracking (DST) maintains user slot values across conversational turns in task-oriented dialogue. While both DST and SKILL.state maintain structured representations, they differ fundamentally in execution mechanics: DST tracks auxiliary state alongside full conversational transcripts in quasi-static dialogues, whereas SKILL.state treats the structured state as a sufficient statistic, discarding conversational history to execute autonomous skills in dynamic environments with bounded prompt footprints.
对话状态跟踪(DST)在面向任务的对话中跨轮次维护用户槽位的值。 虽然 DST 和 SKILL.state 都维护结构化表示,但二者的执行机制存在根本差异:DST 在准静态对话中,将辅助状态与完整对话记录并行维护;SKILL.state 则将结构化状态视为充分统计量,丢弃对话历史,以有界的提示词规模在动态环境中执行自主技能。
2.4. Context Management and Long-Context Reasoning
Language models exhibit degraded retrieval over long contexts, motivating streaming attention and prompt compression techniques. Rather than attempting to process or compress extended conversational histories, SKILL.state prevents history accumulation entirely by maintaining the canonical execution state required for the next computation.
语言模型在长上下文中的检索能力会下降,这推动了流式注意力和提示词压缩技术的发展。 SKILL.state 并不尝试处理或压缩冗长的对话历史,而是通过维护下一次计算所需的规范执行状态,完全避免历史累积。
3. SKILL.state
Current LLM agent runtimes execute procedural skills by repeatedly appending reasoning traces, actions, observations, and tool outputs to a growing conversational history. Consequently, the execution state is represented implicitly within natural language and must be reconstructed by the language model at every interaction. As execution horizons increase, both prompt size and the volume of obsolete information grow monotonically, making execution increasingly dependent on interpreting historical text rather than maintaining the current world state.
当前 LLM 智能体运行时通过将推理轨迹、动作、观测和工具输出反复追加到不断增长的对话历史中,来执行程序性技能。 因此,执行状态隐式地存在于自然语言中,语言模型必须在每次交互时重新构建它。 随着执行时程延长,提示词长度和过时信息量都单调增长,使执行越来越依赖于对历史文本的解读,而非对当前世界状态的维护。
SKILL.state reformulates procedural skill execution as an explicit state transition process. Instead of representing execution as an append-only conversation, every execution step is defined by:
SKILL.state 将程序性技能执行重新表述为显式状态转移过程。 执行不再表示为仅追加式对话,而是将每个执行步骤定义为:
where
其中,
Figure 1 illustrates the execution cycle. At each step, the runtime constructs a prompt from
图1 展示了执行循环。 在每个步骤中,运行时由
图1:SKILL.state 架构概览。
3.1. Execution State and Schema Authoring
Unlike conversational runtimes, SKILL.state treats execution state as a first-class runtime abstraction. The state contains only information required for future execution and is represented using a structured schema defined for the domain. Schemas are authored once per domain rather than per task; for example, across all 100 diverse challenge instances in the InterCode CTF benchmark, the agent reuses a single static 5-field schema (discovered_flags, tested_hypotheses, active_files, working_dir, cmd_summary).
与对话式运行时不同,SKILL.state 将执行状态视为运行时中的一等抽象。 状态仅包含未来执行所需的信息,并使用为相应领域定义的结构化模式来表示。 模式只需为每个领域编写一次,无需针对每个任务单独编写;例如,在 InterCode CTF 基准全部 100 个不同挑战实例中,智能体复用同一个包含 5 个字段的静态模式(discovered_flags、tested_hypotheses、active_files、working_dir、cmd_summary)。
3.2. Reasoning and State Transitions
Reasoning is used strictly as an intermediate computation for producing state transitions and selecting the next action. Given the current execution context
推理严格用作产生状态转移和选择下一步动作的中间计算。 给定当前执行上下文
where
其中,
Crucially, within-step multi-step reasoning is fully intact during generation to support complex deductive planning. However, once the state transition has been validated and applied, the reasoning trace
关键在于,生成过程中完整保留了单个执行步骤内的多步推理,以支持复杂的演绎规划。 然而,一旦状态转移通过验证并被应用,推理轨迹
where
其中,
- for t = 0, …, T do
- Receive latest observation Ot
- Construct prompt (P, Σt, Ot)
- Generate (Rt, ΔΣt, at) using the LLM
- Validate ΔΣt
- Σt+1 ← Σt ⊕ ΔΣt
- Execute at
- end for
Algorithm 1 summarizes the execution process.
算法1 概括了执行过程。
3.3. Complexity Analysis
Let
令
In contrast, SKILL.state maintains only the procedural specification, structured execution state, and latest observation:
相比之下,SKILL.state 仅维护程序规范、结构化执行状态和最新观测:
which is asymptotically bounded and independent of the number of previously executed turns
该规模在渐近意义下有界,并且与此前已执行的轮次数
The resulting runtime shifts execution from reconstructing history toward maintaining an explicit, validated representation of the current execution state.
由此得到的运行时将执行过程的重心从重建历史转向维护当前执行状态的显式、经过验证的表示。
4. Evaluation Benchmarks
4.1. SkillExecBench (Controlled Diagnostic Testbed)
SkillExecBench isolates execution mechanics from open-ended heuristic search by providing sequential procedural tasks with deterministic ground-truth world transitions:
SkillExecBench 提供顺序程序性任务,其世界状态转移具有确定性真值,从而将执行机制与开放式启发式搜索分离:
- Environment 1 (Warehouse Management): A discrete physical inventory domain tracking 500 independent shelves. Actions include
Store,Ship,Move, andWait. This environment tests the model's ability to maintain independent, non-overlapping state variables over extended horizons where early observations leave the context window.
- Environment 2 (Software Repository): A deeply nested, relational graph of Git branches, commits, Pull Requests, and CI test statuses. Actions include
CherryPick,Merge,RunTests,CreateRelease, andRollback. Features dense dependencies where a single action (e.g., merging a PR) fundamentally alters the state of the target branch and dependent PRs, testing complex structural reasoning over an entangled graph.
- 环境 1(仓库管理): 一个跟踪 500 个独立货架的离散实体库存领域。动作包括
Store、Ship、Move和Wait。该环境测试模型在较长执行时程中维护独立、互不重叠的状态变量的能力;在这种情况下,早期观测会离开上下文窗口。
- 环境 2(软件仓库): 一个由 Git 分支、提交、Pull Request 和 CI 测试状态构成的深层嵌套关系图。动作包括
CherryPick、Merge、RunTests、CreateRelease和Rollback。该环境具有密集的依赖关系,单个动作(如合并 PR)就会从根本上改变目标分支及依赖它的 PR 的状态,从而测试模型在相互交织的图上的复杂结构推理能力。
4.2. Public Interactive Benchmarks
To evaluate SKILL.state on real-world, non-deterministic tasks with complex search, generation, and tool use, we evaluate on two public benchmarks:
为评估 SKILL.state 在涉及复杂搜索、生成和工具使用的真实、非确定性任务上的表现,我们使用两个公共基准:
- InterCode CTF: A suite of 100 Linux bash Capture-The-Flag challenges spanning reverse engineering, forensics, cryptography, and binary exploitation. Agents execute bash commands in Docker containers and iteratively test hypotheses to discover hidden flags.
- Sierra
-Bench: A benchmark for tool-agent-user interaction in enterprise customer service (Retail and Airline domains). Agents interact with simulated users, query relational SQLite databases via tool calls, and execute transactional actions (e.g., flight rebooking, refunds) under business policy constraints.
- InterCode CTF: 一套包含 100 个 Linux bash 夺旗挑战的基准,涵盖逆向工程、取证、密码学和二进制漏洞利用。智能体在 Docker 容器中执行 bash 命令,并迭代检验假设,以发现隐藏的 flag。
- Sierra
-Bench: 面向企业客户服务(零售与航空领域)中工具、智能体和用户交互的基准。智能体与模拟用户交互,通过工具调用查询关系型 SQLite 数据库,并在业务政策约束下执行事务性操作(如机票改签、退款)。
4.3. Evaluation Metrics
We evaluate runtimes across three dimensions:
我们从三个维度评估运行时:
- Task Accuracy / Success Rate: In SkillExecBench, accuracy is measured continuously as the ratio of valid, correct actions matching the ground-truth deterministic simulation (
). In InterCode CTF, success is binary pass@1 (exact match on the binary-verified flag). In -Bench, success is scored by the official programmatic evaluator, which verifies that the final database state satisfies user intent without policy violations.
- Average Prompt Size: The mean string character length per LLM invocation.
- Total Token Cost: The cumulative token burn across the entire execution horizon.
- 任务准确率/成功率: 在 SkillExecBench 中,准确率通过与确定性仿真真值相符的有效、正确动作所占比例进行持续测量(
)。在 InterCode CTF 中,成功以二值 pass@1 衡量(与经过二进制验证的 flag 精确匹配)。在 -Bench 中,成功与否由官方程序化评估器判定,该评估器验证最终数据库状态是否在不违反政策的前提下满足用户意图。
- 平均提示词长度: 每次 LLM 调用的平均字符串字符数。
- 总 token 成本: 整个执行时程内的累计 token 消耗。
5. Experiments and Results
5.1. Experimental Setup
We evaluate SKILL.state against two families of baselines (see Appendix A for exact prompt templates):
我们将 SKILL.state 与两类基线进行比较(精确提示词模板见 附录 A):
Primary Runtime Paradigms:
主要运行时范式:
- Prompt (ReAct-style): Appends every observation, intermediate reasoning trace, and action to a continually growing transcript.
- Memory (Summarization-style): Maintains a rolling 3-step conversational window alongside a periodically updated natural language summary of past interactions.
- Stateful (LangGraph-style): Injects a structured state block into the context window alongside the full rolling conversational transcript.
- Prompt(ReAct 风格): 将每次观测、中间推理轨迹和动作追加到持续增长的记录中。
- Memory(摘要风格): 维护一个滚动的 3 步对话窗口,并定期更新关于过去交互的自然语言摘要。
- Stateful(LangGraph 风格): 在上下文窗口中注入结构化状态块,同时保留完整的滚动对话记录。
Budget-Matched and Compression Controls:
预算匹配与压缩对照:
- Truncated (Sliding Window): Retains only the most recent interaction turns that fit within a fixed token budget.
- Summary-capped: Strictly enforces a hard token ceiling on the natural language summary.
- ReAct + LLMLingua: Uses budget-aware small-model perplexity compression to prune tokens from the full history down to the target budget.
- Truncated(滑动窗口): 仅保留能够容纳于固定 token 预算内的最近若干轮交互。
- Summary-capped: 对自然语言摘要严格实施 token 数量硬性上限。
- ReAct + LLMLingua: 使用考虑预算的小模型困惑度压缩,将完整历史中的 token 剪枝至目标预算。
Underlying Models: Evaluations are conducted across proprietary and open-weight models: Gemini-3-Flash, Gemma-4-31B-it, and Qwen-3-8B-it. Decoding is controlled at temperature
基础模型: 评估涵盖专有模型和开放权重模型,包括 Gemini-3-Flash、Gemma-4-31B-it 和 Qwen-3-8B-it。 所有运行的解码温度均设为
Statistical Significance: All synthetic experiments are evaluated across 5 distinct procedural generator seeds. Results are reported as mean
统计显著性: 所有合成实验均使用 5 个不同的程序化生成器随机种子进行评估。 结果以均值
5.2. Experiment 1: Long-Horizon Execution Scaling
We evaluate runtime accuracy and context expansion across execution horizons scaling from
我们在从
Results: As shown in Table 1, SKILL.state matches or exceeds baseline accuracy across all horizons while maintaining a flat prompt size. In contrast, history-appending baselines suffer quadratic token accumulation
结果: 如 表1 所示,SKILL.state 在所有执行时程上的准确率均达到或超过基线,同时维持基本恒定的提示词长度。 相比之下,追加历史的基线存在二次增长的 token 累积,即
| Horizon (T) | Runtime | Score (Accuracy) | Avg Prompt (Chars) | Total Tokens |
|---|---|---|---|---|
| 10 | Prompt (ReAct) | 0.90 ±0.02 | 3,249 ±94 | 9,438 ±371 |
| Memory (Summary) | 1.00 ±0.00 | 3,300 ±123 | 9,972 ±204 | |
| Stateful (LangGraph) | 1.00 ±0.00 | 3,430 ±42 | 10,337 ±299 | |
| SKILL.state | 1.00 ±0.00 | 1,775 ±74 | 5,870 ±131 | |
| 25 | Prompt (ReAct) | 0.92 ±0.02 | 6,052 ±192 | 42,689 ±2,238 |
| Memory (Summary) | 0.99 ±0.00 | 6,357 ±203 | 43,067 ±1,948 | |
| Stateful (LangGraph) | 1.00 ±0.00 | 5,858 ±301 | 41,238 ±3,196 | |
| SKILL.state | 1.00 ±0.00 | 1,736 ±49 | 14,714 ±564 | |
| 50 | Prompt (ReAct) | 0.88 ±0.04 | 11,931 ±346 | 171,658 ±6,978 |
| Memory (Summary) | 0.93 ±0.03 | 7,582 ±283 | 131,455 ±6,841 | |
| Stateful (LangGraph) | 0.94 ±0.00 | 11,594 ±438 | 170,992 ±7,918 | |
| SKILL.state | 0.96 ±0.01 | 1,773 ±53 | 30,151 ±1,231 | |
| 100 | Prompt (ReAct) | 0.84 ±0.07 | 36,362 ±1,304 | 1,245,413 ±53,241 |
| Memory (Summary) | 0.87 ±0.05 | 29,607 ±978 | 1,082,154 ±83,212 | |
| Stateful (LangGraph) | 0.91 ±0.02 | 31,354 ±831 | 1,062,387 ±53,839 | |
| SKILL.state | 0.94 ±0.01 | 1,905 ±93 | 65,408 ±5,431 | |
| 200 | Prompt (ReAct) | 0.74 ±0.14 | 48,007 ±2,092 | 2,608,755 ±102,415 |
| Memory (Summary) | 0.84 ±0.09 | 84,364 ±3,446 | 6,175,509 ±294,089 | |
| Stateful (LangGraph) | 0.88 ±0.03 | 72,305 ±3,096 | 5,041,164 ±346,925 | |
| SKILL.state | 0.94 ±0.02 | 1,811 ±184 | 122,384 ±4,522 |
表1:使用 Gemini-3-Flash 的仓库管理长时程扩展实验。基线运行时出现
5.3. Experiment 2: Context Corruption (Noise Robustness)
Real-world execution environments emit dense background telemetry. We fix the horizon at
真实世界的执行环境会产生密集的后台遥测信息。 我们将执行时程固定为
Results: As shown in Table 2, the standard Prompt runtime degrades sharply from 0.68 at low noise down to 0.53 at high noise. In contrast, SKILL.state maintains robust task completion (
结果: 如 表2 所示,标准 Prompt 运行时的得分从低噪声下的 0.68 急剧下降到高噪声下的 0.53。 相比之下,SKILL.state 在所有噪声水平下都保持稳健的任务完成表现(
| Noise Level | Runtime | Score |
|---|---|---|
| 5 Events (Low) | Prompt | 0.68 |
| Memory | 1.00 | |
| Stateful | 1.00 | |
| SKILL.state | 1.00 | |
| 20 Events (Medium) | Prompt | 0.61 |
| Memory | 1.00 | |
| Stateful | 0.98 | |
| SKILL.state | 0.97 | |
| 50 Events (High) | Prompt | 0.53 |
| Memory | 0.96 | |
| Stateful | 0.98 | |
| SKILL.state | 0.98 |
表2:仓库环境中的噪声鲁棒性(
5.4. Experiment 3: State Recovery
We test runtime resilience to silent external environment drift where the true world state is modified outside the agent's action loop (e.g., an external actor moves an inventory item).
我们测试运行时对无显式通知的外部环境漂移的抵抗能力,即真实世界状态在智能体动作循环之外发生改变(例如,外部参与者移动了某件库存物品)。
Results: As shown in Table 3, history-based baselines hallucinate for 5 to 8 consecutive turns because obsolete facts in their prompt history overpower contradictory new observations. In sharp contrast, SKILL.state requires zero recovery steps: because its decisions depend on the current structured state, the state is updated immediately upon receiving the corrective alert.
结果: 如 表3 所示,基于历史的基线会连续 5 至 8 轮产生幻觉,因为提示词历史中的过时事实压过了与之矛盾的新观测。 与之形成鲜明对比的是,SKILL.state 无需恢复步骤:由于其决策依赖于当前结构化状态,收到纠正警报后,状态便会立即更新。
| Scenario | Runtime | Success | Recovery Steps |
|---|---|---|---|
| A: Secret Audit | Prompt / Memory / Stateful | Yes | 5–8 |
| SKILL.state | Yes | 0 | |
| B: Secret Barcode | Prompt / Memory / Stateful | Yes | 6–8 |
| SKILL.state | Yes | 0 | |
| C: Secret Move | Prompt / Memory / Stateful | Yes | 5–8 |
| SKILL.state | Yes | 0 | |
| D: Canceled Order | All Runtimes | No | N/A |
表3:仓库环境中的状态恢复(Gemini-3-Flash)。
5.5. Experiment 4: Public Interactive Benchmarks
To test generalizability on open-ended tasks with complex search, generation, and tool use, we evaluate SKILL.state on InterCode CTF and Sierra
为测试在涉及复杂搜索、生成和工具使用的开放式任务上的泛化能力,我们在 InterCode CTF 和 Sierra
Results: As shown in Table 4, SKILL.state achieves the highest task completion rates across all three benchmarks while substantially cutting cumulative token consumption. In InterCode CTF, maintaining explicit hypotheses and discovered flags in
结果: 如 表4 所示,SKILL.state 在全部三个基准上均取得最高的任务完成率,同时大幅降低累计 token 消耗。 在 InterCode CTF 中,在
| InterCode CTF (100 Tasks) | Sierra τ-Bench (Retail) | Sierra τ-Bench (Airline) | |||||||
|---|---|---|---|---|---|---|---|---|---|
| Runtime | Pass@1 | Prompt | Tokens | Pass Rate | Prompt | Tokens | Pass Rate | Prompt | Tokens |
| Prompt (ReAct) | 43.2% | 1,909 | 977k | 48.2% | 2,819 | 4.48M | 21.8% | 5,100 | 4.85M |
| Memory (Summary) | 46.4% | 1,797 | 1.03M | 29.9% | 2,737 | 4.24M | 23.6% | 4,700 | 4.65M |
| Stateful (LangGraph) | 41.8% | 1,946 | 1.13M | 51.7% | 3,065 | 3.92M | 28.1% | 5,400 | 5.28M |
| SKILL.state | 54.2% | 813 | 387k | 58.3% | 3,325 | 3.47M | 32.4% | 2,800 | 2.88M |
表4:使用 Gemini-3-Flash 在公共交互式基准上的评估。SKILL.state 取得最高的任务成功率,同时显著减少提示词长度和累计 token 消耗。
5.6. Experiment 5: Budget-Matched Controls and Statistical Compression
To determine whether SKILL.state's performance gains stem merely from shorter prompts or from structured state representation, we evaluate budget-matched baselines on Warehouse (
为确定 SKILL.state 的性能增益究竟只是源于更短的提示词,还是来自结构化状态表示,我们在仓库环境(
Results: As shown in Table 5, all budget-matched compression baselines suffer catastrophic failure. Sliding-window truncation drops to 0.18 because critical early inventory allocations are evicted. LLMLingua drops to 0.22 because statistical entropy filtering removes seemingly redundant slot identifiers that are semantically vital. In contrast, SKILL.state achieves 0.94 score, demonstrating that structured state maintenance preserves exact relational dependencies that statistical compressors destroy.
结果: 如 表5 所示,所有预算匹配的压缩基线都遭遇了灾难性失败。 滑动窗口截断的得分降至 0.18,因为关键的早期库存分配信息被移出了上下文。 LLMLingua 的得分降至 0.22,因为统计熵过滤删除了看似冗余、但在语义上至关重要的槽位标识符。 相比之下,SKILL.state 取得了 0.94 的得分,表明结构化状态维护能够保留被统计压缩器破坏的精确关系依赖。
| Runtime / Configuration | Score | Avg Prompt | Total Tokens |
|---|---|---|---|
| Full ReAct (Unbounded) | 0.84 | 36,362 | 1,245,413 |
| Truncated (Sliding Window) | 0.18 | 1,800 | 62,100 |
| Summary-capped | 0.52 | 1,840 | 63,400 |
| ReAct + LLMLingua | 0.22 | 1,810 | 62,350 |
| SKILL.state (Structured) | 0.94 | 1,905 | 65,408 |
表5:仓库环境中的预算匹配对照(
5.7. Error Taxonomy for Open-Weight Models
On open-weight models (Gemma-4-31B at
对于开放权重模型(Gemma-4-31B 在
- Premature State Overwrite / Deletion (68%): The model accidentally omits existing keys during state update rather than merging in-place.
- Schema Comprehension / Type Coercion (20%): Inconsistencies between expected nested lists and dictionaries.
- JSON Syntax / Formatting Slips (12%): Malformed JSON delimiters or trailing commas.
- 过早覆盖/删除状态(68%): 模型在状态更新时意外遗漏已有键,而不是就地合并。
- 模式理解/类型强制转换(20%): 预期的嵌套列表与字典之间出现不一致。
- JSON 语法/格式失误(12%): JSON 分隔符格式错误或存在尾随逗号。
This error distribution shows that small-model degradation stems from structured output adherence rather than reasoning capacity, motivating constrained decoding in future runtime iterations.
这种错误分布表明,小模型的性能下降源于对结构化输出要求的遵循程度,而非推理能力,这推动我们在未来运行时迭代中引入约束解码。
6. Conclusion
We presented SKILL.state, a runtime architecture that replaces append-only conversational history with explicit, structured execution state. By discarding intermediate reasoning traces after each validated transition, SKILL.state maintains a bounded
我们提出了 SKILL.state,一种以显式、结构化执行状态取代仅追加式对话历史的运行时架构。 通过在每次经过验证的状态转移后丢弃中间推理轨迹,SKILL.state 将提示词规模维持在有界的
7. Limitations
SKILL.state assumes that the execution state can be made a sufficient statistic for future execution: that everything in the past bearing on future actions can be projected into the structured state as soon as it becomes known. Where this holds, discarding intermediate reasoning and conversational history is lossless. However, this assumption fails in three distinct settings: (1) when no fixed schema is known in advance and the relevant state structure must be discovered dynamically during execution; (2) when a correct state update depends on an earlier observation whose relevance was not recognized when first observed, and was therefore never committed to state; and (3) when the task objective is defined over the historical trajectory itself (e.g., auditing, debugging provenance, or explaining past actions), where interaction history is the target output rather than operational overhead.
SKILL.state 假定执行状态能够成为未来执行的充分统计量:过去所有影响未来动作的信息,都能在获知时立即投射到结构化状态中。 当这一条件成立时,丢弃中间推理和对话历史不会造成信息损失。 然而,这一假设会在三种不同情形下失效:(1)事先不知道固定模式,必须在执行过程中动态发现相关状态结构;(2)正确的状态更新依赖于某个更早的观测,但初次观测时未意识到它的相关性,因此从未将其写入状态;(3)任务目标本身是针对历史轨迹定义的(如审计、调试溯源或解释过去的动作),此时交互历史是目标输出,而非运行开销。
Our current implementation focuses on single-agent procedural execution. While the explicit state abstraction extends naturally to multi-agent systems—where a shared execution state acts as the central coordination substrate instead of exchanging quadratic conversational transcripts—multi-agent environments introduce concurrent writes, requiring deterministic conflict-resolution semantics in the merge operator
我们当前的实现聚焦于单智能体程序性执行。 虽然显式状态抽象可以自然地扩展到多智能体系统——此时共享执行状态充当核心协调基础,而无需交换呈二次规模增长的对话记录——但多智能体环境会引入并发写入,要求合并运算符
Finally, SKILL.state relies on the language model to propose valid structured state patches. Because schema ownership and validation reside in the deterministic runtime rather than the model, malformed outputs cannot corrupt persistent state
最后,SKILL.state 依赖语言模型提出有效的结构化状态补丁。 由于模式的管理与验证由确定性运行时负责,而非由模型负责,格式错误的输出无法破坏持久化状态