如何用 OpenWiki 与图谱构建有证据支撑的智能体记忆

关键要点Key Takeaways

在组织的政策、手册、维基、工单和代码中工作的智能体，必须在分散于不同系统和团队的知识中进行查找。找到相关内容只是第一步。它们还需要判断哪些规则适用、发生了哪些变化，以及一个来源会如何影响另一个来源。Agents working across an organization’s policies, manuals, wikis, tickets, and code must navigate knowledge scattered across systems and teams. Finding a relevant passage is only the start. They also need to determine which rules apply, what has changed, and how one source affects another.

持久化记忆层能够保留知识及其关联关系，供智能体在不同任务中重复使用。没有这一层，智能体往往不得不在每次回答问题时重新构建这些理解，导致工作成本高昂且容易出错。这在银行和保险领域尤其具有挑战性，因为相关政策和指南经常修订或被新规取代。A persistent memory layer preserves knowledge and relationships for agents to reuse across tasks. Without it, agents often have to reconstruct that understanding on every question, making their work expensive and error-prone. This is especially challenging in banking and insurance, where policies and guidelines frequently amend or supersede one another.

我们测试了 OpenWiki（我们的开源 Wiki 编译器），将其用作这一记忆层。对于需要跨来源建立关联的问题，使用该 Wiki 的智能体准确率约为仅使用搜索时的 5 倍。加入一个证据图，将 Wiki 中的陈述与支持它们的段落相连，并关联相关来源后，准确率又提升了 27%，引用一致性也提高了一倍。We tested OpenWiki, our open-source wiki compiler, as this memory layer. On questions requiring connections across sources, the wiki made the agent roughly 5x more accurate than search alone. Adding an evidence graph that connects wiki statements to supporting passages and links related sources improved accuracy a further 27% and doubled citation consistency.

为什么要使用编译型 Wiki？Why a compiled wiki?

RAG 流水线通常使用语义搜索或混合搜索来检索彼此孤立的文本块，留给智能体自行拼凑这些文本块之间的关系。我们曾探索通过智能体式 RAG 和基于图的元数据过滤来改进检索，但找到相关段落只是整个工作的一部分。跨来源问题需要追踪引用、解决规则冲突并理解依赖关系。记忆可以保留这些理解，让智能体不必每次遇到问题时都从头重建。RAG pipelines often use semantic or hybrid search to retrieve isolated chunks, leaving agents to piece together how those chunks relate. We’ve explored improving retrieval with agentic RAG and graph-based metadata filtering, but finding relevant passages is only part of the work. Cross-source questions require tracing references, resolving conflicting rules, and understanding dependencies. Memory preserves that understanding so agents don’t have to rebuild it on every question.

编译后的维基通过将源材料综合整理为相互关联的页面，解释关键概念及其关系，从而提供这一记忆层。随着来源内容的变化，维基也会随之更新，为智能体提供可重复利用的上下文。A compiled wiki provides that memory layer by synthesizing source material into linked pages that explain key concepts and their relationships. The wiki is updated as sources change, giving agents context they can reuse.

OpenWiki 是我们基于 Deep Agents 构建的开源 CLI，可创建带有来源引用的维基。我们最初构建它，是为了让编程代理在处理每项任务时都不必重新摸索架构和代码行为。OpenWiki, our open-source CLI built on Deep Agents, creates these wikis with citations back to the sources. We originally built it so coding agents wouldn’t have to rediscover architecture and code behavior on every task.

OpenWiki 中的每条陈述都建立在一项主张之上，该主张关联着作为依据的源文本行及其哈希值。当这些文本行发生变化时，哈希值也会随之变化，因此 OpenWiki 无需调用模型就能标记受影响的主张，从而节省 token 成本，并让过时检测更加确定。只有这些主张才会交由模型确认、修订或撤回，使记忆层能够随着来源变化而自我纠正。我们在《在 OpenWiki 中构建可自我纠正的记忆》中对此进行了说明。Every OpenWiki statement rests on a claim, an assertion linked to supporting source lines and their hash. When those lines change, the hash changes, allowing OpenWiki to flag affected claims without a model call, saving on token costs and making staleness detection more deterministic. Only those claims go to a model for confirmation, revision, or retraction, allowing the memory layer to self-correct as its sources change. We explain this in Building self-correcting memory in OpenWiki.

构建测试文档库Building the test document library

我们构建了一个包含 111 份长篇文档的合成保险文档库。我们选择保险领域，是因为保单和批单经常相互引用、修订和覆盖；正是这种跨文档依赖关系让检索变得困难。We built a synthetic insurance document library of 111 long-form documents. We chose insurance because policies and endorsements frequently reference, amend, and override one another; this is exactly the kind of cross-document dependency that makes retrieval hard.

为了评估这项基准测试，我们创建了一份包含 972 条事实的台账，并控制每条事实应出现的位置、文档之间如何相互引用，以及各版本之间发生了哪些变化。我们使用 LLM 生成相关叙述文本，并运行自动检查，以确保所需事实均已包含。To score the benchmark, we created a ledger of 972 facts and controlled where each fact should appear, how documents referenced each other, and what changed between versions. We generated the surrounding prose with an LLM and ran automated checks to ensure the required facts were present.

最后，我们刻意将语料库设计得杂乱无章，混入过时版本、相互矛盾的内容、间接引用链、不同的措辞以及超长文档，以更贴近智能体在真实企业资料库中面临的情况。Finally, we made the corpus intentionally messy, mixing in outdated versions, contradictions, indirect reference chains, varied wording, and very long documents to better match what agents face in real enterprise libraries.

我们如何配置实验How we configured our experiment

我们在两种条件下评估了同一个检索代理：Search（在原始文档中进行关键词搜索）和 Wiki（同一个代理，但可访问由 OpenWiki 汇编、并链接至来源的维基）。底层模型、指令、文档和问题均保持不变，唯一变化的是代理能否访问汇编后的记忆。We evaluated the same retrieval agent under two conditions: Search (keyword search over the original documents) and Wiki (the same agent with access to an OpenWiki-compiled, source-linked wiki). The underlying model, instructions, documents, and questions were held constant — only the agent’s access to compiled memory changed.

我们测试了 80 个简单查找问题，以及 50 个需要跨文档追踪依赖关系的关联问题（例如，判断某项规定依赖于什么，或在该规定变更时会受到什么影响）。每个问题我们都询问了三次，以衡量不同运行之间的波动。We tested 80 simple lookup questions and 50 connected questions that require following dependencies across documents (e.g., identifying what depends on a provision or is affected when it changes). We asked each question three times to measure run-to-run variation.

指标（“更好”的含义）：Metrics (what “better” means):

• 准确性（0–1）：评分者根据《972 条事实账本》中的清单核查了答案和引用的证据（对必需事实予以加分，对矛盾之处予以扣分）。
  Accuracy (0–1): a grader checked answers and quoted evidence against a checklist from the 972-fact ledger (rewarding required facts and penalizing contradictions).
• 一致性（0–1）：三次运行中引用证据的重合度（引用集合两两之间的 Jaccard 相似度平均值）。该指标衡量智能体是否始终依据相同来源作答，而非答案是否正确。
  Consistency (0–1): overlap of cited evidence across the three runs (average pairwise Jaccard similarity of citation sets). This measures whether the agent reliably grounds its answers in the same sources, not whether the answers are correct.
• 成本 + 延迟：每个问题的模型开销（美元）和实际耗时。
  Cost + latency: model spend (USD) and wall-clock time per question.

维基提供帮助的地方，以及遇到困难的地方Where the wiki helped, and where it struggled

OpenWiki 将包含 111 份文档的资料库编译成一个维基，大小约为原来的十分之一，费用为 4.15 美元。之后再添加 17 份文档，每份成本约为 0.09 美元。以下分析不包括编译和更新成本，仅报告检索和回答的成本。OpenWiki compiled the 111-document library into a wiki roughly one-tenth its original size for $4.15. Adding 17 documents later cost about $0.09 each. The analysis below excludes compilation and update costs, reporting only retrieval and answering costs.

我们使用简单查找作为对照。这些单段落问题几乎不需要跨文档的保险背景信息，因此我们预计 wiki 帮助不大。搜索和 wiki 的结果相近（准确率分别为 0.84 和 0.83；引用重合率分别为 0.72 和 0.74；每个正确答案的成本分别为 0.020 美元和 0.021 美元），但延迟从 9 秒增至 12 秒。We used simple lookups as a control. These single-passage questions require little cross-document insurance context, so we didn’t expect the wiki to help much. Results were similar for search and wiki, respectively (0.84 vs 0.83 accuracy; 0.72 vs 0.74 citation overlap; $0.020 vs $0.021 per correct answer), while latency increased from 9 to 12 seconds.

关联性问题需要基于领域特定关系对多个来源的信息进行推理。在这里，维基将准确率提高了约 5 倍，从仅使用搜索时的 0.10 提升到 0.52。它汇编的上下文帮助智能体运用领域知识并以更少的工具调用找到相关关系，从而将每个正确答案的成本从约 0.55 美元降至 0.19 美元。Connected questions required reasoning across sources using domain-specific relationships. Here, the wiki improved accuracy roughly 5x, from 0.10 with search alone to 0.52. Its compiled context helped the agent apply domain knowledge and find relevant relationships with fewer tool calls, cutting cost per correct answer from about $0.55 to $0.19.

尽管准确率有所提升、成本有所降低，维基智能体的证据检索仍不稳定。引用重合率从简单查询中的 0.74 降至关联问题中的 0.25。引用稳定性并不衡量答案是否正确，但证据发生变化可能导致答案改变，尤其是在来源相互矛盾时。对于关键业务职能，团队需要的是可复现、可审计的答案，而不只是更高的平均准确率。Despite these accuracy and cost gains, the wiki agent’s evidence retrieval remained inconsistent. Citation overlap dropped from 0.74 on simple lookups to 0.25 on connected questions. Citation stability doesn’t measure correctness, but changing evidence can change an answer, especially when sources conflict. For critical business functions, teams need answers they can reproduce and audit, not just better average accuracy.

该 wiki 帮助代理利用不同来源的领域知识。剩下的挑战是让查找支持性证据的过程更具可重复性。The wiki helped the agent apply domain knowledge across sources. The remaining challenge was making the path to supporting evidence more repeatable.

添加证据图谱Adding an evidence graph

为了让检索更加一致，我们添加了一个轻量级证据图谱：它将 wiki 引用转化为声明、wiki 页面、段落和主题之间明确且带类型的边。OpenWiki 引用是 Markdown 文件中的文本引用。图谱通过将这些引用物化为带类型的边，使其持久可靠。这样，智能体就能沿着稳定的路径查找，而不必每次都通过搜索重新发现相同的链接。To make retrieval more consistent, we added a lightweight evidence graph: a map that turns wiki citations into explicit, typed edges between claims, wiki pages, paragraphs, and topics. OpenWiki citations are text references in Markdown files. The graph makes them durable by materializing them as typed edges. That gives the agent a stable path to follow instead of rediscovering the same links with search.

我们分两步构建了这个图：We built this graph in two steps:

1. 将每个段落存储为一个新的原子单元。通过确定性流程将每份文档拆分为段落，并为每个段落分配唯一标识符。这些段落由此成为信息的原子单元，将各项主张与确切的支持性段落关联起来，而非与整份文档关联。
   Store every paragraph as a new primitive. A deterministic process splits each document into paragraphs and assigns each a unique identifier. These paragraphs become atomic units of information, linking claims to exact supporting passages rather than entire documents.
2. 使用带类型的边连接各段内容。我们用带类型的边连接段落、主张和 Wiki 页面：主张引用段落，较新的文档取代较旧的文档，而章节通过 refers_to 指向其他文档。每条边都标明了关系，而不只是连接。我们通过两种方式构建边。其中一些是根据文档引用确定性地提取出来的。由于 Wiki 主张已经包含文本引用，我们可以直接生成 refers_to 和 supersedes 这类带类型的边，无需再次调用模型。
   Connect the passages with typed edges. We linked paragraphs, claims, and wiki pages with typed edges: claims cite paragraphs, newer documents supersede older ones, and sections refers_to other documents. Each edge identifies the relationship, not just the connection. We built edges two ways. Some are extracted deterministically from document references. Because wiki claims already carry text citations, we can materialize typed edges like refers_to and supersedes without another model call.

图谱构建耗时不到一秒，并解析了测试库中的所有显式交叉引用，因此智能体可以沿着链接查找，而不必在每次提问时重新发现关联。The graph took under a second to build and resolved every explicit cross-reference in the test library, so the agent could follow links instead of rediscovering connections on each question.

图谱的职责是导航证据，而不是对组织所知的一切进行建模。综合整理仍由 wiki 完成。与依赖大量抽取的知识图谱流水线不同，这一层复用现有的参考资料和引文，而不是借助模型抽取并整合实体。构建图谱无需额外调用模型，不过，从 wiki 推导出的边仍取决于其模型生成引文的准确性。The graph’s job is evidence navigation, not modeling everything the organization knows. The wiki still provides the synthesis. Unlike an extraction-heavy knowledge graph pipeline, this layer reuses existing references and citations rather than extracting and reconciling entities with a model. Construction requires no additional model calls, though edges derived from the wiki still depend on the accuracy of its model-generated citations.

两种工具可让代理在地图中导航：locate 用于查找与问题相匹配的起点，expand 则沿着其连接关系追踪到支持性段落和相关文档。若只有一条路径，expand 会自动沿该路径前进；若路径出现分支，代理则选择下一步前往何处。这种先定位相关节点、再收集关联上下文的模式，也出现在 Microsoft 的 GraphRAG 等 GraphRAG 系统中。Two tools let the agent navigate the map: locate finds a starting point that matches the question, and expand follows its connections to supporting passages and related documents. Where there’s only one path, expand follows it automatically; where the path branches, the agent chooses where to go next. This pattern of locating relevant nodes and gathering connected context also appears in GraphRAG systems such as Microsoft’s GraphRAG.

@tool(parse_docstring=True)
async def locate(terms: str, within: str | None = None) -> dict:
    """Find where to start: match the question's words against document titles,
    section headings, wiki pages, and claim statements."""
    ...

@tool(parse_docstring=True)
async def expand(node: str, edges: list[str] | None = None, hops: int = 1) -> dict:
    """Open one node and list its typed edges (cites, refers_to, supersedes,
    amends, attaches_to, and their inverses). With hops > 1 and one edge type,
    walk that edge while there is a single path, and stop at a branch."""
    ...

准确度更高、成本更低、响应更快Better accuracy, lower cost, faster answers

添加明确的证据路径后，所有指标都比单独使用维基时有所提升。准确率提高了 27%，引用一致性翻了一倍，中位答题时间缩短了 40%，每个正确答案的成本降低了 58%。证据图谱通过更有针对性的检索补充了维基的上下文，从而提高了准确率和一致性，同时没有让智能体变得完全确定性。Adding explicit evidence paths improved every measure over the wiki alone. Accuracy rose 27%, citation consistency doubled, median answer time fell 40% , and cost per correct answer fell 58%. The evidence graph complemented the wiki’s context with more directed retrieval, improving both accuracy and consistency without making the agent fully deterministic.

什么时候值得付出代价来增加内存？When is adding memory worth the cost?

支持构建编译型 wiki 的最有力论据，是它能够反复推理并关联多个来源。在我们的实验中，wiki 帮助智能体运用领域关系，而不是通过搜索重新推导这些关系。但仅靠记忆层并不能让证据检索保持一致。添加显式的图边后，引用的一致性和准确性都得到提升，同时检索工作量也有所减少。The strongest case for a compiled wiki is repeated cross-source reasoning. In our experiment, the wiki helped the agent apply domain relationships rather than reconstruct them through search. But the memory layer alone didn’t make evidence retrieval consistent. Adding explicit graph edges improved citation consistency and accuracy while reducing retrieval work.

该 wiki 为构建轻量级证据图提供了坚实基础。证据图可根据现有引文和文档引用以确定性方式生成。这样，我们便能添加明确的证据路径，无需额外调用模型，也不必对组织掌握的所有知识进行建模。The wiki provided a strong foundation for a lightweight evidence graph, derived deterministically from its existing citations and document references. This let us add explicit evidence paths without additional model calls or the need to model everything the organization knows.

Wiki 是否值得投入，取决于内容能否重复利用。我们报告的问答成本不包括编译成本，添加文档的费用约为每份 0.09 美元。对于简单查询，搜索本身的准确度不相上下且速度更快；尤其是在企业级规模下，内容摄取和持续维护的成本可能会超过检索所节省的成本。Whether a wiki is worth the investment depends on reuse. Our reported answer costs exclude compilation, and adding documents cost about $0.09 each. Ingestion and ongoing maintenance may outweigh retrieval savings for simple lookups, where search alone was just as accurate and faster, especially at enterprise scale.

当多个智能体反复回答涉及同一批互不连通的信息源的问题时，这种权衡就会发生变化。共享 wiki 能保留这些信息源之间的关联，而图中的边则能引导智能体找到支持性证据。在我们的测试中，这种组合以更低的单个正确答案成本，带来了更准确的答案。只要复用次数足够多，避免反复重建上下文所节省的成本，就能抵消构建和维护记忆的成本。The tradeoff changes when multiple agents repeatedly answer questions spanning the same disconnected sources. A shared wiki preserves the relationships between those sources, while graph edges guide agents to supporting evidence. In our tests, that combination produced more accurate answers at a lower cost per correct answer. With enough reuse, savings from avoiding repeated context reconstruction can offset the cost of building and maintaining memory.

接下来是什么？What's next

接下来，我们希望以更动态的方式构建维基，只编译并保留智能体回答跨来源问题所需的上下文，而不是整个代码库。Next, we want to build wikis more dynamically, compiling and retaining only the context agents need for cross-source questions rather than entire repositories.

我们还想探索将主张和类型化边作为其他类型智能体记忆的基础。将知识与证据关联起来，并检测其变化，或许有助于为语义记忆（事实）、情景记忆（发生过什么）和程序记忆（如何做事）提供依据。We also want to explore claims and typed edges as a foundation for other kinds of agent memory. Linking knowledge to evidence and detecting when it changes could help ground semantic memory (facts), episodic memory (what happened), and procedural memory (how to do things).

想在你的智能体中添加维基内容吗？来看看我们的 OpenWiki 资源：Want to add wikis into your agents? Check out our OpenWiki resources:

• Openwiki 文档
  Openwiki Documentation
• OpenWiki 代码仓库
  OpenWiki Code Repository

致谢Acknowledgments

感谢 Colin Francis 和 Nick Hollon 分享他们对 OpenWiki 和评估的看法！Thank you Colin Francis and Nick Hollon for their thoughts around OpenWiki and evaluations!