在 2000+ 次运行中测试 AI 智能体上下文压缩得到的 6 条经验

工作上下文压缩实用指南A practical guide to working context compression

如果你把 AI 智能体接入了真实工具，就会遇到这个问题。你问一个简单的问题，智能体调用某个工具，返回的却是一大堆文本：包含 300 次提交的 git 日志、整张数据库表，或是长得像小说一样的浏览器页面快照。所有这些内容都会进入模型的上下文，而后续每一轮中，你都得为每个 token 付费。If you’ve wired an AI agent to real tools, you’ve seen the problem. You ask a simple question, the agent calls a tool, and back comes a wall of text: a 300-commit git log, a full database table, a browser page snapshot the size of a novel. All of it lands in the model’s context, and you pay for every token on every turn that follows.

显而易见的解决办法，是在模型看到工具输出之前先将其精简。这正是我们打造的方案：一个小型代理，位于智能体与其 MCP 服务器之间，可裁剪较大的输出，并保留完整原始内容，让模型在需要时只需一次调用即可获取。The obvious fix is to shrink tool outputs before the model sees them. So that’s what we built: a small proxy that sits between an agent and its MCP servers, trims large outputs, and keeps the full original one call away in case the model needs it.

它在输出上奏效了。以下是我们在五个热门的、未经修改的 MCP 服务器上测得的数据：It worked on the outputs. Here’s what we measured on five popular, unmodified MCP servers:

工具输出大小（token），原始 → 压缩Tool output size (tokens), raw → compressed

减少了 70%–94%，而且原始内容可以逐字节恢复。我们当时对自己的成果相当满意。Cuts of 70–94%, with the original recoverable byte for byte. We were pretty pleased with ourselves.

然后，我们衡量了真正重要的指标：智能体完成整个任务实际使用的总 token 数，以及它是否仍然给出了正确答案。在记录了 2,000 多次智能体运行之后，坦率地说，节省的 token 大多不见了。这篇文章将讲述这些节省去了哪里，以及这对任何构建或采购智能体上下文压缩方案的人有何启示。Then we measured what actually matters: the total tokens the agent used for a whole task, and whether it still got the answer right. Over 2,000 logged agent runs later, here’s the honest summary: the savings mostly disappeared. This post is about where they went, and what that teaches anyone building or buying context compression for agents.

实验设置如下，供你校准判断：通过 Ollama 运行本地开放权重模型（主要是 Qwen3 4B），温度设为 0；使用真实的 MCP 服务器（git、filesystem、SQLite），处理公开数据（Flask 仓库和 Chinook 示例数据库），并加入合成任务场景。问题的标准答案由代码计算得出。下面的每个数字都来自运行日志。任务都很短，每个只需调用几次工具，因此应将其视为经过充分测量的一个局部，而非全貌。Setup, so you can calibrate: local open-weight models via Ollama (mainly Qwen3 4B), temperature 0, real MCP servers (git, filesystem, SQLite) over public data (the Flask repository and the Chinook sample database), plus synthetic task worlds. Questions have code-computed gold answers. Every number below comes from logged runs. The tasks are short, a few tool calls each, so treat this as one well-measured corner of the space, not the whole map.

第 1 课：代理将其取回Lesson 1: The agent fetches it back

这里有一个真实案例。问题是：“根据 Flask 仓库中的 CHANGES.rst，版本 0.5.1 是在哪一天发布的？”Here’s a real episode. The question: “According to CHANGES.rst in the Flask repo, on which date was version 0.5.1 released?”

Agent → 代理 → 文件系统服务器Agent → Proxy → Filesystem server

1. 智能体调用 read_text_file(CHANGES.rst)。
   Agent calls read_text_file(CHANGES.rst).
2. 文件系统服务器返回 71,250 个字符。
   Filesystem server returns 71,250 chars.
3. 代理返回 10,639 个字符和一条注释：“已省略 1,614 行中的 1,377 行；展开(h…)”
   Proxy returns 10,639 chars + a note: “1,377 of 1,614 lines omitted; expand(h…)”
4. 它需要的那一行在省略的部分里，所以……
   The line it needs is in the omitted part, so…
5. 智能体调用 expand(h…)。
   Agent calls expand(h…).
6. 代理返回 71,250 个字符（完整文件）。
   Proxy returns 71,250 chars (full file).
7. 答案：2010-07-06 ✓
   ANSWER: 2010-07-06 ✓

答案是正确的。但这笔费用比什么都不做还糟糕：模型既为删减版付了费，也为完整文件付了费。在真实服务器测试集（108 项任务）中，结果如下：The answer was right. The bill was worse than doing nothing: the model paid for the trimmed version and the full file. Across the real-server test set (108 tasks), the result looked like this:

压缩反而让成本更高。看清之后，这个规律很简单：模型读取一大段输出，是为了寻找某个特定内容。如果压缩器删掉了包含该内容的部分，模型就会要求把它补回来，结果你就为此付了两次钱。Compression made things more expensive. The pattern is simple once you see it: a model reads a large output because it’s looking for one specific thing. If your compressor cuts the part containing that thing, the model asks for it back, and now you’ve paid twice.

课程 2：让检索更有针对性，而不是非此即彼Lesson 2: Make retrieval targeted, not all-or-nothing

显而易见的修复办法：不要让“把它取回来”意味着“把整个内容都取回来”。我们添加了一种搜索模式，这样模型就可以只请求它需要的行（这里用另一个版本来演示）：The obvious fix: don’t make “get it back” mean “get the whole thing back”. We added a search mode, so the model can ask for just the lines it needs (illustrated with another version):

展开(h…) → 71,250 个字符（整个文件）expand(h…) → 71,250 chars (whole file)

expand(h…, search="Version 2.0.1") → ~6 lines:

480: Version 2.0.1
481: -------------
482:
483: Released 2021-05-21

模型很快就上手了：16 次检索中有 14 次首次尝试就使用了搜索，无需额外提示。+17% 的开销消失了：The model took to it immediately: 14 of 16 retrievals used search on the first try, with no extra prompting. The +17% overhead disappeared:

在真实服务器上，它从消耗 token 变成了收支平衡。在输出量更大、重复性更高的合成任务中，它节省了约六分之一。这确实是进步，但远不及输出图表上显示的 70–94%。On real servers it went from costing tokens to breaking even. On synthetic tasks with bulkier, more repetitive outputs, it saved about a sixth. That’s a real improvement, but it’s nowhere near the 70–94% on the output chart.

课程 3：仅仅提供一种工具就可能让小模型表现变差Lesson 3: Just offering a tool can make a small model worse

这一点让我们很意外。在一次诊断运行中，我们完全没有进行压缩，只是把检索工具和智能体的常规工具列在了一起：This one surprised us. In a diagnostic run we compressed nothing at all and only listed the retrieval tool next to the agent’s normal tools:

同样的48项任务，没有任何精简：Same 48 tasks, nothing compressed:

一个 4B 模型看到了一个额外的工具，就分心了。它要么在不需要时调用了这个工具，要么转而思考这个工具，而不是专注于任务。样本量很小，所以不要引用具体差距，但趋势与我们看到的其他所有结果一致。A 4B model saw an extra tool and got distracted. It called it when it didn’t need to, or reasoned about it instead of the task. It’s a small sample, so don’t quote the exact gap, but the direction matched everything else we saw.

修复方案是延迟列出：只有在确实有内容被裁剪后，检索工具才会出现，代理会通过 MCP 的 notifications/tools/list_changed 通知来告知这一变化。在我们的测试中，Claude Code 能正确处理该通知，因此该工具会在恰好派上用场时出现。The fix was lazy listing: the retrieval tool appears only after something has actually been trimmed, and the proxy announces it with MCP’s notifications/tools/list_changed. Claude Code handles that notification properly in our tests, so the tool shows up exactly when it becomes useful.

第 4 课：廉价压缩暗藏代价：答案悄然丢失Lesson 4: Cheap compression has a hidden cost: silently lost answers

如果只是简单截断，只保留前 N 个字符呢？这是最简单的压缩器：What about plain truncation, keeping only the first N characters? It’s the simplest compressor there is:

合成测试（200 项任务）Synthetic test (200 tasks)

截断确实能节省四分之一的 token。但它是以丢弃大约六分之一任务的答案为代价的：模型甚至根本不知道这些数据存在。我们称之为“静默损失”，这是你应该向任何压缩服务商询问的指标。Truncation does save a quarter of the tokens. It does it by throwing away the answer in about one task in six: the model never even learns the data existed. We call that silent loss, and it’s the number to ask any compression vendor for.

（关于我们代理的 55.5%：它因一个错误而被拉低了。参见第 5 课。）(*About our proxy’s 55.5%: it was dragged down by a bug. See lesson 5.)

要点：衡量压缩器删掉了多少重要内容，而不只是删掉了多少内容。Takeaway: measure what your compressor deleted that mattered, not just how much it deleted.

课程 5：你的代理可能会破坏那些你从未想到过的东西Lesson 5: Your proxy can break things you never thought about

我们代理最差的成绩与压缩毫无关系。在一台测试服务器上——一台内存/知识图谱服务器——我们的代理得分为 0/16，而原始代理得分为 16/16。Our proxy’s worst number had nothing to do with compression. On one test server, a memory/knowledge-graph server, our proxy scored 0 of 16 where the raw agent scored 16 of 16.

原因是：代理将真实服务器作为子进程启动时，使用了 MCP SDK 的默认环境变量配置，其中只保留了 HOME 和 PATH 等少数变量。服务器的配置变量（MEMORY_FILE_PATH）并没有传过去，因此服务器启动时使用了一个空存储，对所有请求都回复“实体不存在”。在实际场景中，同样的错误可能会悄无声息地丢弃你的 GITHUB_TOKEN 或 API 密钥。The cause: when the proxy launched the real server as a subprocess, it used the MCP SDK’s default environment, which keeps only a few variables like HOME and PATH. The server’s config variable (MEMORY_FILE_PATH) never arrived, so it started with an empty store and answered “no such entity” to everything. In real life, the same bug would silently drop your GITHUB_TOKEN or API keys.

环境变量传递Environment variable propagation

MCP client
  └── env: GITHUB_TOKEN=…
        └── proxy
              └── env: HOME, PATH only
                    └── server ✗

MCP client
  └── env: GITHUB_TOKEN=…
        └── proxy
              └── env: everything
                    └── server ✓

只需一行修复。我们之所以发现这个问题，只是因为我们在几种服务器类型上测量了端到端任务成功率。压缩率基准测试永远发现不了它。It’s a one-line fix. We found it only because we measured end-to-end task success on several server types. A compression-ratio benchmark would never have caught it.

要点：如果在智能体与其工具之间加入任何东西（代理、网关、封装层），就要在真实服务器上测试完整任务，包括通过环境变量配置的服务器。Takeaway: if you put anything between an agent and its tools (proxies, gateways, wrappers), test full tasks on real servers, including ones configured through environment variables.

第 6 课：动工前先检查天花板Lesson 6: Check the ceiling before you build

这就是我们会采取不同做法的地方。在编写任何压缩代码之前，我们本可以先让普通智能体处理一些真实任务，并问问：提示词中的 token 到底有多少来自大型工具输出？Here’s the part we’d do differently. Before writing any compression code, we could have run the plain agent on realistic tasks and asked: what share of the prompt tokens even comes from large tool outputs?

在我们的任务中，粗略估算的答案约为 40%。只有大约一半的运行会产生大量输出，而 token 也来自指令、工具定义、问题以及模型自己的回复。因此，即使压缩器完美无缺，且模型从未还原压缩内容，最多也只能节省约 40%。每当模型检索某些内容时，其中一部分就会再次出现。花一个下午测量一下，就能预料到一个“0–25%”的现实结果。On our tasks, the rough answer was about 40%. Only about half the runs had a large output at all, and the tokens also come from instructions, tool definitions, the question, and the model’s own replies. So even a perfect compressor the model never undid could save about 40% at best. Every time the model retrieves something, part of that comes back. A realistic result of “0–25%” was predictable from one afternoon of measurement.

提示词 token 的去向（粗略估算，针对我们的任务）：Where the prompt tokens go (rough, our tasks):

要点：在优化上下文中的某一部分之前，先测量它在实际会话中占多大比重。如果上限很低，再巧妙的算法也无法带来显著提升。Takeaway: before optimizing a part of the context, measure how big that part is in real sessions. If the ceiling is small, no clever algorithm will make the result big.

那么，在智能体中节省 token 意味着什么？So what does it mean to save tokens in agents?

综合起来，以下是我们今天会给团队的实用检查清单：Pulling it together, here’s the practical checklist we’d give a team today:

1. 衡量会话，而不是输出。统计每个已完成任务的提示词元总数，并结合任务成功率。不要单独看压缩比。
   Measure sessions, not outputs. Total prompt tokens per completed task, plus task success. Ignore compression ratios on their own.
2. 先找出你的上限。如果大型工具输出只占 token 总量的一小部分，就去别处找：工具 schema、系统提示词和对话历史通常占用更多 token。
   Find your ceiling first. If big tool outputs are a small share of your tokens, look elsewhere: tool schemas, system prompts and conversation history are often bigger.
3. 如果要压缩，就让检索更有针对性。搜索或基于范围的检索把我们的 +17% 优势拉到了大致持平。
   If you compress, make retrieval targeted. Search or range-based retrieval turned our +17% into roughly break-even.
4. 只有在工具真正有用时才展示它们。按需列出工具，避免小模型分心。
   Don’t advertise tools until they’re useful. Lazy tool listing kept a small model from getting distracted.
5. 询问隐性损失。压缩器有多少次移除了任务所需的内容？
   Ask for silent loss. How often did the compressor remove the thing the task needed?
6. 通过真实服务器进行测试，使用真实配置，包括环境变量和身份验证。
   Test through real servers, with real configuration, including environment variables and auth.

压缩仍有用武之地：在非常长的会话中，早期冗长的输出会在上下文中保留许多轮，而使用检索能力出色的大型模型也适用于这种场景。我们的测试没有覆盖这种情况（任务较短，使用的是本地小模型）。要确定你的配置是否属于这种情况，唯一的方法就是测量会话表现。Compression still has a place: very long sessions where old, bulky outputs sit in context for many turns, with large models that use retrieval well. Our runs didn’t test that regime (short tasks, small local models). The only way to know whether your setup is in it is to measure sessions.

数据是开放的。The data is open

我们已将这些任务和 1,832 段完整记录的智能体运行轨迹（包括模型的每次调用、模型所见的每项工具结果、token 数量和正确性标签，涵盖 raw、我们的代理、Headroom 和截断）打包成一个开放基准：“超越压缩比：智能体上下文效率开放基准”，并发布在 Hugging Face 上。We’ve packaged the tasks and 1,832 fully logged agent episodes (every model call, every tool result as the model saw it, token counts and correctness labels, for raw, our proxy, Headroom and truncation) as an open benchmark: “Beyond Compression Ratios: An Open Benchmark for Agent Context Efficiency” on Hugging Face.

如果你构建了上下文压缩器，请在这些任务上运行它，并报告两个数字：会话令牌数和成功率。我们很期待看到一种在这两项指标上都优于原始方案的方法。If you build a context compressor, run it on these tasks and report two numbers: session tokens and success. We’d love to see a method that beats raw on both.