为反洗钱构建具备记忆能力的智能体运行框架：Oracle 的 Nacho Martinez 实战分享

简要描述Brief Description

用于反洗钱的记忆感知型智能体运行框架 — Nacho Martinez，OracleMemory-Aware Agent Harnesses for Anti-Money Laundering — Nacho Martinez， Oracle

目录Table of Contents

• 工作坊设置
  Workshop setup
• 简介：Nacho Martinez
  Intro: Nacho Martinez
• 议程
  Agenda
• 应用场景：面向中型银行的反洗钱（AML）智能体
  The use case: an AML agent for a mid-sized bank
• 智能体技术栈
  The agent stack
• 智能体系统 = 自动化 + 自主性
  Agentic systems = automation + autonomy
• 什么是 AI 智能体？
  What is an AI agent?
• 为什么测试框架比模型更重要
  Why the harness matters more than the model
• 文件与数据库
  Files vs. databases
• 智能体记忆：短期记忆、情景记忆、程序性记忆、语义记忆
  Agent memory: short-term, episodic, procedural, semantic
• 上下文摘要与压缩
  Context summarization vs. compaction
• 智能体循环
  The agent loop
• 工具箱与技能箱模式
  Toolbox & skillbox patterns
• 技能蒸馏
  Skill distillation
• 冻结模型如何变得更好：持续学习
  How a frozen model gets better: continual learning
• 指令层级
  Instruction hierarchy
• Workshop 用户界面演示
  Workshop UI walkthrough
• 问答：情景记忆与冲突
  Q&A: Episodic memory & conflicts
• 问答：技能、缓存与上下文预算
  Q&A: Skills, caching & context budgets
• 问答：让你的记忆层保持简单
  Q&A: Keep your memory layer simple
• 问答：预构建 SQL 与动态查询
  Q&A: Pre-built SQL vs. dynamic queries
• 混合搜索（语义搜索 + 关键词搜索）
  Hybrid search (semantic + keyword)
• 构建小型自定义测试框架
  Build small custom harnesses

研讨会设置Workshop setup

大家好。谢谢大家来参加这场活动。我们这场活动的主题是智能体运行框架。时间不多，只有一个小时。希望我们有时间聊聊大家感兴趣的许多内容。活动大致会这样进行：我先花 10 到 15 分钟左右介绍我们将在工作坊中讨论和解决的主题。工作坊说明可以通过二维码或网站查看。如果你不想听我讲，觉得我有点无聊之类的，也可以在我讲的时候自行开始做工作坊。网站上基本就是这么操作：用你的 GitHub 用户名注册，然后你就会自动收到该代码仓库的邀请。Hi. Hi everyone. Uh thank you for coming to this session. Uh we're it's going to be a session about agent harnesses. We don't have a lot of time. It's only one hour session. So um hopefully we have time to talk about a lot of things that that you guys are interested in and the the session essentially is going to be uh like me talking like 10 15 minutes about the topics that we're going to talk about and solve in the workshop. You have the workshop instructions in a QR code or in a website and if you don't want to listen to me, you think I'm boring or something, you can start uh on your own on the workshop as I'm speaking. But essentially this is this is what's uh happening on the website. You just um sign up with your GitHub handle and then you will automatically receive an invitation to the repository.

这个研讨会托管在 Code Spaces 上。所以你只要从那里创建一个 Code Space，就会自动收到邀请，获得研讨会的访问权限。你只需要打开 GitHub Codespaces，在那里创建一个 Code Space，就可以开始参加研讨会了。好，我再放一次二维码，方便想要扫码的各位。我这就开始。好，太好了。如果你们有任何问题，我的同事 Richmond 也在那边。他会到大家的桌边转转。如果有问题，只要举手，他也可以帮忙。对，就是这样。另外，网站的话，你们可以访问 workshopwaiting room.com。我为那个域名花了太多钱。The workshop is hosted on code spaces. So you can just like create a code space out of that uh and you will receive an invitation automatically to get access to the workshop itself and you just have to create a a code face over there uh opening GitHub code spaces and you're good to go and and and begin with the workshop. Uh so I'll just put the QR code uh one more time for those of you who were uh who who want to take that and I'll I'll get started. Okay, perfect. Uh, if you have any questions, also my colleague Richmond is over there. He's going to go through um, you know, everyone's uh, table. If you have any questions, just raise your hand and he can help out as well. Yeah, there you go. Also, the website, I mean, you can go to the website workshopwaiting room.com. I paid too much money for that domain.

所以，呃，用它吧。So, uh, use it.

简介：Nacho MartinezIntro: Nacho Martinez

好的。那么，我是谁？我叫 Nacho Nacho Martinez，是一名 AI 开发者倡导者，骨子里也是一名数据科学家。如果你对我感兴趣，去 YouTube 上找我就行。我有播客、工作坊，什么都有。就不多介绍我自己了。嗯，不过有一件事：我们最近和 Andrew Ang 以及 DeepLearning.AI 合作推出了一门课程，讲的是如何构建自适应 AI 智能体，这也是我们今天要讨论的子组件之一。如果你特别想了解如何改进模型，让它随着时间不断适应、不断学习，不妨去看看这门课程。这是一门免费的课程，时长一小时，你可以在 Google 上找到。Okay. So, who am I? My name is Nacho Nacho Martinez. I am an AI developer advocate and a data scientist by heart. And if you're interested in anything about me, just find me on YouTube. I have podcasts, I have workshops, I have everything. Not going to go too much into myself. Um just one thing though uh we re recently released a course with Andrew Ang and deep learning AI about uh building adaptive AI agents which is one of the subcomponents that we're going to talk about today. And you know if you're particularly interested in the things that you can do on a model to make it better and adapt over time, learn over time, just check out that course. is a free course uh a 1hour course that you can check out on on Google.

议程Agenda

那么，今天我们要讲些什么呢？就像我说的，我们时间不多，所以我会尽量讲快一点。我们要聊的内容包括：首先，什么是智能体技术栈？智能体技术栈由哪些组件构成？AI 应用可以有哪些不同形态？什么是 AI 智能体？什么是智能体框架？与直接使用 AI 智能体或 LLM 模型相比，构建智能体框架或围绕它开展工作的主要动机是什么？此外，我们还会讲智能体框架的所有子组件，比如记忆、上下文工程、记忆工程等等。听说这次会有很多来自金融领域的人，比如很多在华尔街工作的人，所以我把这个研讨会的内容调整成了围绕银行展开。这个研讨会的核心设定是：如今我们常听说，OpenAI 正在推出一万个智能体，或者五千个智能体来解决某个问题；或者用一万个智能体来解决一个数学问题，而这个问题过去花了一百年、两百年才得以解决。So what are we going to go uh through today? Like I said, we don't have a lot of time. So I'm going to try to be quick. Um but these are the things that we're going to talk about. First of all, what is the agent stack? Uh what are the components of an agent stack? What are the different shapes that an AI application can take? What an AI agent is? And what is the agent harness? what is the main motivation of building or working around agent harnesses rather than working with AI agents or with an LLM model directly and you know all the subcomponents of an agent harness like for instance memory context engineering memory engineering and all these things. So because they told me that there's going to be a lot of people from uh financial uh like a lot of people that work in Wall Street uh I adapted this workshop to be around a bank right so the idea is that the main premise of the workshop is this we are hearing lots of the uh nowadays like open AAI is launching 10,000 agents or 5,000 agents to solve this problem 10,000 agents to solve this mathematical problem that was solved like in a 100 years 200 years.

问题在于，这些智能体是自主运行的。在它们得出结论或答案之前，整个过程中都没有人类参与，对吧？所以，智能体本身并不具备做到这一点的能力。为什么呢？因为上下文窗口是有限的。大型语言模型本身并不具备记忆组件。所以，我们需要自己实现所有这些东西，才能让这些智能体自主运行 10 个小时、100 个小时来解决所有这些问题，对吧？所以我的想法是，很多人都把重点放在“好，我们有这些前沿模型”上，但我认为，我们不必过于关注前沿模型，我们可以使用它们，但更应该着手改进 harness，也就是其中真正有意思的部分，以及那些你可以不断进行定制的东西。The thing is that these agents are autonomous. They don't have any human in the loop until they arrive to uh to the conclusion or the answers, right? So the agents themselves don't have the capacity to do this. Why? Because I mean the context window is limited. The memory components are something that a large language model doesn't have inherently. So all these things we need to implement ourselves to make these agents able to solve all these problems for 10 hours 100 hours autonomously right. So the idea is that a lot of people are making focus on okay we have these frontier models but my idea is that okay we don't need to focus so much on the frontier models we can use them but let's work on the harness which is the interesting part and the things that you can actually customize uh more and more.

应用场景：为一家中型银行提供反洗钱（AML）代理The use case: an AML agent for a mid-sized bank

这个应用的想法是：有一家中型银行，时不时会出现很多可疑交易，这些交易分为五种不同类型。举个例子，如果有一笔 100 万美元的交易进入银行，接着三天里你又自动地、神奇地通过 ATM 取款，每次取 5 万美元左右，这就很可疑，对吧？因为你为什么要存入 100 万美元，然后又这样把钱分散取出来呢？所以，我们的想法是，这家中型银行有大量交易，这些交易需要由系统自主判断，或者至少标记出来，这样银行里真正负责处理这些事务的人就能少花些时间查看这些交易。我们稍微讲讲理论吧。So on this application, the idea is that it's a medium-sized bank that has lots of uh you know suspicious transactions uh sometimes and they are categorized into five different types. So for example, if you get like a million dollar transaction that's coming into your bank and automatically and magically uh for three days you're making like $50,000 withdrawals on an ATM. That's suspicious, right? because why would you like put $1 million in and then just like distribute uh like that. So the idea is that we have this medium by uh medium-sized bank that has lots of transactions and these transactions they need to be kind of autonomously decided or at least flagged so that a human that actually works in the bank needs to spend less time actually looking at these things you know. So let's go to the theory a little bit.

什么是智能体技术栈？对吧？当我们与 AI 智能体交谈时，通常会……What is the agent stack? Right? When we talk with an AI agent, we typically are

智能体技术栈The agent stack

与应用层交互，对吧？比如我们使用的界面、聊天机器人、移动应用等。然后是模型本身，也就是推理核心——你知道，我们无法左右它会做出什么决策。除此之外，还有支撑这一切的基础设施，以及算力。不过，今天我们要重点讨论的是数据部分。数据非常重要，因为我们能够操控和处理的一切都在这一层。其他部分我们能控制得少一些，对吧？尤其是使用不属于我们自己的第三方模型时。所以，今天我们重点关注数据层，对吧？首先，我们来看看智能体循环，它可以说是 AI 智能体最重要的部分之一。talking with the application layer, right? Uh what we interact with either an an interface, a chatbot, an a mobile application etc. Then we have the model itself which is the reasoning core the the you know we don't get any say in what the decisions are. Uh and then we have also the infrastructure that supports it all and then the compute. But what we are going to focus about today is the data part of it. And the data is very important because everything that we can manipulate and work around is on this part. The other parts we get less control over them, right? Especially if we're using like a third party uh model that isn't ours. So the data layer is what we focus about uh today, right? First of all, we have the agent loop and this is arguably one of the most important parts of an AI agent.

嗯，智能体循环基本上负责驱动整个系统，而且它也是可定制的。在数据层中，你还可以使用网关和 MCP，将它连接到第三方外部数据源。你还有记忆层，我们稍后会讨论；此外还有一些上下文工程功能，可以自动总结、压缩并缩小模型的工作上下文窗口。最后，还有一些工具和技能，可以帮助我们完成一些有趣且智能的任务，从而尽可能限制上下文窗口的大小。这是本次研讨会的核心理念之一。通常，我们接触到的应用程序都是这四种类型的组合，对吧？LLM 聊天机器人是最简单的一类。很多人只是创建一个 AI 应用，做一个聊天机器人，让你可以和它对话。Uh the agent loop is in charge of essentially driving the whole thing. Uh and it's customizable as well. And inside the data, you also have like a gateway and MCP so that you can connect it to third party external data sources. You also have the memory layer which we will talk about later and essentially also some context engineering uh parts of it that let you automatically summarize and compact and shrink the working context window of a model and finally also some tools and skills uh which we can do some interesting and intelligent stuff so that we can essentially keep the context window limited. This is one of the main ideas of the workshop. Now, typically the applications that we interact with, they are either a combination of these four, right? LLM chatbots which are very easy. Lots of people just create an AI application, create a chatbot so that you can talk with it.

你们还有 RAG 应用，本质上就像是升级版的向量搜索。除此之外，还有由 LLM 驱动的工作流和 AI 智能体。顺便问一下，通常我们大多数人都会用这些——举手示意一下，谁不用 AI 智能体，比如 Claude Code 等等？当然，这些系统都是第三类和第四类的结合体，也就是由 LLM 驱动、提供自动化能力的工作流，以及提供自主性的 AI 智能体。因此，它们是自动化与自主性的结合。You also have rag applications which is essentially like a glorified uh vector search. And then you also have LLM driven workflows and AI agents. And typically what most of us use uh by the way raise of hands who doesn't use AI agents like cloud code etc of course uh so cloud codec etc all these systems are a combination of three and four which is um you know a combination of LLM driven workflows that provide automation and AI agents that provide autonomy. So it's a combination of automation and autonomy

智能体系统 = 自动化 + 自主性Agentic systems = automation + autonomy

这些被称为智能体系统。更优秀的系统，比如 Claude Code、Codex、Spy、OpenClaw、Hermes Agent 等，都是这两种类型的结合。我交流过的大多数工程师——这一点很重要——每天都会使用这些系统，对吧？但我们在构建 AI 应用时，往往会忽略这些问题。所以你可能有一个 AI 应用，但却不会像这样使用它。今天我们要重点讨论的就是这一点，因为我认为这是使用 AI 应用更好的方式：不只是通过聊天机器人，不只是通过 RAG 应用，还要结合自动化与自主性。and these are called agentic systems. The better systems like cloud code, codeex, spy, uh open claw, hermes agent etc. they are all a combination of these two types and most of the engineers that I talk to and this is important is that we use these systems every day, right? But we stop or a lot of people not a lot of people think about these problems when building their AI applications. So you might have an AI application but then you don't consume it like this. And this is something that we're going to focus on today because I think this is the better way of consuming um AI applications not only through chat bots, not only through rag applications but also through a combination of automation and autonomy.

什么是 AI 智能体？What is an AI agent?

那么，对于还不了解的人来说，AI 智能体是什么呢？它是一种由 LLM 驱动、能够进行推理的计算实体，有点像我们人类有大脑一样。它还通过数据库扩展记忆功能，也和我们的大脑类似；它通过工具来执行操作，就像人类有四肢，可以与世界互动一样。AI 智能体也能通过我们提供给它的工具与操作系统交互。它也以输入信息为基础。比如，对人类来说，这就相当于我们的感官，比如眼睛等，让我们能够感知信息。我们的注意力也有限，智能体也一样：我们没法同时看七部电影——那样的话，我们什么也理解不了，也没法专注于其中任何一部。所以，人的上下文窗口与 AI 智能体的上下文窗口类似。我们也在努力把其中一些概念转化为更贴近人类体验的说法。So what is an AI agent for those of you uh who don't know? It's a like a computational entity that is powered by an LLM by reasoning kind of like us humans we have our brain it's augmented by a database for memory also like our brain it's extended through tools for actions so the equivalent of a human would be like we have our extremities and we can interact with the world just like an AI agent can interact with an operating system through the tools that we give it they provide it and it's also grounded in inputs. So for for instance it would be like our senses for a human our eyes etc that let us perceive information and the context like our attention is also limited like our agents we can't watch seven movies at the same time like we will not understand any of the movies we will not focus on any of them so the context window for humans is similar to AI agents and kind of we try to assimilate or you know have some of these notions be closer to our human experience.

经验，而这是我们今天要一起研究的内容。其中，推理部分由前沿实验室提供，我们对此没有太多控制权。然后，还有一些我们能够多加控制的部分，也就是记忆、工具使用和感知，对吧？这就是脚手架本身。脚手架本质上就是构建这三个组件——工具使用、记忆和感知——所需的所有支撑结构。我们不会过多关注模型，因为正如我所说的，我们在这方面没有太多选择。experience and this is something that we'll we'll work on today. So the reasoning part of it uh is provided by the frontier labs and we don't get a lot of control over that and then we also have the things that we do have a little bit more control over which are the memory the tool uses and the perception right and this is the the harness itself. A harness is essentially all the scaffolding that you need to do to build uh these three components which is the the tool use the memory and the perception and we're not going to focus too much on the model because like I said we don't have a lot of possibilities on what to

为什么测试框架比模型更重要Why the harness matters more than the model

对，所以我认为，Harness（智能体运行框架）是 AI 领域最有意思的话题之一，甚至是当今 AI 领域最有意思的话题。因为这些运行框架实际上控制着 AI 智能体能做的一切。给它换一种配置，同一个智能体、同一个你正在使用的前沿大语言模型，表现就可能变好，也可能变差。有时你会发现，已经有证据表明，只要更换运行框架，同一个模型在特定任务上的表现就会变好或变差。所以，关注运行框架真的非常重要，但做这件事的人并不多。就像你说的，一个运行框架里有七个或八个不同的组成部分——如果把持续学习也算进去的话。你需要做很多选择，对吧？你要选模型、选存储方式——数据放在哪里；还要选一些编码和检索组件，比如如何计算两个向量之间的距离，才能正确地检索数据；还要考虑在记忆中存储什么？do right so the harness is actually one of the most interesting topics in AI I think is the most interesting topic in AI nowadays because these harnesses actually control everything that the AI agent can do. You give it a different configuration then the agent the same LLM frontier AI model that you're consuming becomes better or worse. Sometimes you know it's been proven that by just changing the harness the same model performs better or worse for specific tasks. So it's really really important to make focus on the harnesses and not a lot of people are doing this and like you said in a harness there are you know seven different or eight different components if you consider continue learning as well but there are lots of decisions that you need to make right you have the model you have the storage like where do you put the data uh you have some encoding and retrieval components like how do I make the distance between two vectors to actually retrieve the data properly uh what do I store in my memory?

我会把它看作类似人类的短期记忆加长期记忆吗？我会把它看作不同的区块或不同类型的记忆吗？还有语义层，这部分真的很有意思，因为作为人类，我们在这里，有一些彼此从未说出口的背景信息，对吧？但我们会共享这些信息。比如，我是发言者，你在听，你现在没有唱歌。你为什么不唱歌？因为按照我们当前所处情境的语义，这么做并不合适，对吧？所以，模型本身也能访问语义层，它会谈到一些模型从未被告知、但默认成立的事情。还有智能体循环，也就是驱动智能体、让它自动且自主地反复运行的一切。这是最重要的部分之一。还有上下文工程。我们会逐一关注这些内容，但不会深入探讨，因为我们会在工作坊里实际进行深入讲解。Do I think of it as short-term memory plus long-term memory kind of like humans have? Do I think of it in different uh compartments or different types of memory? uh also the semantic layer which is a part that's really um interesting because as humans we are here and we have some context that we have never said to each other right but we are sharing this for instance I'm a speaker and you are listening you are not singing right now why are you not singing because it's not appropriate in the semantics of the context that we are at right now right so the model itself also has access to the semantic layer which talks a little bit about all the things that haven't been set to the model but they are assumed. Also the agent loop which is like all the things that drive and automatically and autonomously make the agent work all over uh again and this uh is like one of the most important parts and also context engineering and we'll focus on each of them but I will not go d uh I will not deep dive into them because we'll actually do that in the workshop.

仅供参考，嗯，有这么一件事。So just for reference um there was this

文件与数据库Files vs. databases

关于文件和数据库有很多讨论，很多人都说我们应该什么都用文件：你只要把所有东西都放进一个文件里，一个月后，智能体就会神奇地变得更好。还有些人是数据库至上主义者，他们想把所有东西都放进数据库里，对吧？所以这个两难局面仍在持续。我想，嗯，举个例子吧。大家能不能告诉我，针对屏幕上这个例子，智能体会用什么变通办法来解决你们看到的这个具体问题？多个智能体同时写入同一个文件。在如今的 AI 智能体中，这种情况是怎么处理的？有什么想法吗？big discussion around files versus databases like lot of people were saying we need to use files for everything like you just put everything into a file and then like magically one month later the agent becomes better. Some other people are database maximalists and they want to put everything into a database. Right? So this dilemma is still going on and I would say that um well let let me give you an example like can can anyone tell me in this example that you see on the screen can anyone tell me what is the workaround that agents use to solve uh this specific problem that you're seeing on screen multiple agents writing to the same file. How does that work nowadays in AI agents? Any ideas?

主持人：工作树。没错，很好。那么，工作树就是目前的解决方案，对吧？而这其实是一种做法，或者说是一种变通方案，因为文件本身存在一个固有的问题：如果有多个写入操作同时针对同一个文件，就会出现一致性问题，比如哪个写入先发生？如果有人正在读取文件，那么在我修改文件之前就已经读取了它的人该怎么办？诸如此类的问题还有很多，对吧？所以，有些情况下我肯定会告诉你，文件很棒。文件非常适合存储大量非结构化数据。另一方面，数据库也用于其他一些类型的场景。我的看法是，在某些情况下，这些解决方案适用于文件；而在另一些情况下，它们适用于数据库。所以我们确实需要重新审视正在发生的这些事情，对吧？Host: Work trees. Exactly. Good. So, work trees is the current solution, right? And this is a a thing or a workaround because files have an inherent problem in themselves, which is that they they like if multiple writes are coming into the same file, then you have problems with consistency like which right came first? If someone is reading, what do I do with this person that read the file before I modified it, etc. lots of problems, right? So for some things I will definitely tell you like files are great. Files are great at storing lots of unstructured data. Database on the other hand is also used for some other types of use cases. But my solution is that for some cases some of these uh solutions work with files, for some other cases they work with databases. So we actually um need to reconsider some of these things that are going on, right?

嗯，工作树解决方案，或者说变通方法，数据库早在 30 年前就已经解决了。我们现在有些做法是在重复造轮子。其实我们不需要重新发明任何东西，因为已经有现成的解决方案了。比如说，如果你要写入一个文件，而且有多个代理同时写入，那么如果文件写入具备事务一致性或 ACID 一致性，也就是原子性、一致性、隔离性和持久性，就不会出现这个问题，对吧？所以，文件适合做某些事情，数据库则适合做其他事情。我们不必只选其中一种，对吧？所以，在智能体运行框架中，可以把这当作一种思路。Um, and the work tree uh solution or the workaround has been already solved by databases 30 years ago. And we're trying to reinvent the wheel with some of this. And we don't actually need to reinvent anything because there is an actual solution to this. For example, if you're trying to write to a file um at the same time multiple agents, if you had transactional consistency or acid consistency like atomic consistent isolated and durable um rights into the file, you would not get this issue. Right? So files are good for some things and databases are good for for other things. We don't have to go into only one of them, right? So in the agent harness just consider this as an as a thought.

智能体记忆：短期记忆、情景记忆、程序性记忆、语义记忆Agent memory: short-term, episodic, procedural, semantic

呃，另一个组成部分是所有的记忆组件，对吧？它涵盖了围绕 AI 智能体创建的所有机制和系统，让它能够使用信息；而且不只是使用这些信息，还能对其进行优化，并在需要的时候恰当地回忆起来，对吧？作为工程师，你们可以选择智能体的记忆层中要考虑哪些内容。有些人喜欢把记忆分为短期记忆和长期记忆。在这里，这是第二部分。我们有一些短期记忆组件。比如，如果我们在和 Claude 或 Codex 对话，我们就有短期记忆，因为上下文窗口本身就是短期记忆。对话结束后就会消失的内容，就是短期记忆，对吧？Uh another component is the all the memory components right and this comprises all the mechanics and the the systems that are created around an AI agent to make it use the information um and not only use this information but also be able to refine it and recall it appropriately when it's time right and agent memory you as the engineers you get to choose like what are the things in your agent memory layer that you're going to consider. Some people like the short-term versus long-term memory. And in this case, this is part two. We have some short-term memory components. For example, if we are talking with clot or with codeex, we have short-term memory in the sense that the context window itself is short-term memory. Anything that disappears after the conver like it disappears after the conversation, right?

上下文窗口里的大多数内容，呃，都会直接消失。然后你开启另一段对话，它就会从零开始。这就是短期记忆，非常有限，我们不可能把所有内容都放进去。除此之外，我们还有长期记忆。在长期记忆中，我们也提出了一些类别。呃，比如，情景记忆包含所有发生过的事情、我们对话中的所有片段、所有工具日志、之前的对话，以及你与智能体进行过的之前对话的摘要，等等。它们都会归入情景长期记忆这一类别。还有程序记忆，包含技能、工作流（也就是一连串依次执行的技能）等。这些内容都会归入程序记忆。most things uh in the context window just vanish. Then you launch another conversation and it starts from zero. That's short-term memory. It's very limited. We cannot put everything into it. And then we also have long-term memory. And inside the long-term memory, we also have some categories that we propose. Uh, for example, episodic memory would be all the things that happened, all the episodes of our conversations, all the tool logs, the previous conversations, um, summaries of previous conversations that you've had with the agent, etc. They all go into this episodic long-term memory category. You also have procedures or procedural memory, which are all the things like skills, workflows, which are like a sequence of skills that are being run etc. All of those go into procedural memory.

此外，我们还有语义记忆，它包含了所有这些内容。就像我说的，智能体会记住你是谁、你的偏好是什么、你最喜欢的框架是什么，等等，这些都会存入语义记忆。那么，问题是：为什么不能把所有内容都放进短期记忆？因为它是有限的，而且不只是容量有限，注意力矩阵越大——注意力矩阵本质上就是上下文窗口——模型的表现就越差。每个模型都会出现这种情况。许多研究都表明，如果上下文窗口从零个 token 开始，问题的解决准确度会高得多；而如果上下文窗口里已经有 256,000 个 token，表现就会差很多。目前上限是一百万个 token。之所以能达到一百万，是因为有 KV 缓存。And also we have semantic memory which is all the unset like I said the agent remembers who you are what are your preferences what's your you know favorite framework etc all these things go into semantic memory so the problem the main issue here why cannot we put everything into the short-term memory because it's limited and it's not only limited but actually the bigger the attention matrix is and the attention matrix is essentially the the context window. The bigger it is, the worse it performs. And this happens for every model. There are lots of studies that have shown that if you start at zero tokens on the context window, the problem is being solved much more accurately than if you have 256,000 tokens already and the limit is a million right now. We're at a million because of the KV cache.

所有这些因素都需要考虑。要知道，即使宣传的上限是 256k 或 100 万，继续往上用的话，性能也会变得非常糟糕。所以，上下文窗口越小，解决问题的效果反而越好。这适用于你能想到的人类面临的各种问题，对吧？而且有两种类型……All these things need to be considered and just know that even if the advertised limit stops at uh 256k or 1 million if you keep going the like the performance goes horribly. So the smaller the context window is the better it will solve a problem. And this applies for every type of problem that you can think of uh in humanity right and there are two types

上下文摘要与压缩Context summarization vs. compaction

解决这个问题的方法有几种。一种是上下文摘要：你只要让它生成一份简短摘要，概括上下文窗口里说过的所有内容，然后拿到摘要，并从这份摘要开始下一轮对话，这样就可以了。另一种是压缩，和它有点类似，但压缩时，你会把对话分成若干块，存进数据库或文件里，然后给每一块分配一个 ID。之后如果 AI 智能体需要用到某一块，它就有对应的 ID。它只要根据这个 ID 和偏移量读取内容，就像读取文件一样，对吧？这些都是上下文工程中会遇到的一些问题。接下来我们还有……of things that tackle this. One is context summarization which you just ask it hey just generate a summary a short summary for me with this uh with all the things that have been said in the context window and then you get the summary and then you start off from the summary the next conversation that is fine. Uh, another one is compaction and that's kind of similar, but in compaction you're just like putting chunks of the conversation into the database or into a file and then you just put it with an ID and if the AI agent needs it later, it has the ID. It just pulls from the ID with an offset kind of like reading from a file, right? Some of the these are some of the problems that happen in context uh context engineering. And then we have

智能体循环The agent loop

智能体循环就是驱动程序，或者说是那辆驱动我们谈到的所有组件的车。我们有一个上下文组装部分，它会从技能、工具、之前的对话、文件和数据库中提取内容。所有这些东西都会动态地带入上下文窗口。然后它会进行一些推理，调用 OpenAI（或你所说的任何模型）的 API，判断需要做什么。如果你愿意，这个过程就会自动、无限地进行下去。在我的例子中，这个为工作坊准备的智能体循环有一个最大耐心值，因为我不想耗尽 OpenAI 额度，或者我拥有的任何其他额度。不过，我们这里的最大迭代次数是八次，每次迭代最多进行 60 秒的推理，对吧？the agent loop which is the driver or the the car that that drives all the components that we have talked about. We have a context assembly part that pulls from the skills pulls from the tools pulls from the previous conversation from the files from the database. All of these things just brings in uh brings into the context window dynamically. Then it does some reasoning. it makes an API call to open AAI or whichever model you're talking to and decides on what needs to be done and this happens you know automatically and infinitely if you want in my case in this agent loop that that uh we prepared for the workshop it has a maximum patience because I don't want to run out of openi credits or any any credits that I that I have but uh yeah the the maximum number of iterations in our case will be eight and a maximum of 60 seconds of reasoning for each one of the iterations, right?

嗯，不过这完全是可配置的。就像我说的，作为工程师，我们必须决定哪些内容会进入智能体循环，Um, but this is completely configurable. Like I said, as engineers, we have to say in what goes on into the agent loop,

工具箱与技能箱模式Toolbox & skillbox patterns

对吧？还有工具箱模式、技能箱模式，我们提出用这种方式来存储工具和技能，而它们的数量有几百个。我不知道你们怎么样，但我已经安装了大约一百个技能。我不会把一百个技能同时放进上下文窗口，因为每个技能大约有 3,000 到 5,000 个 token。这样很快就会超出公布的限制。我就没有空间和智能体进行实际对话了。那么怎么解决这个问题呢？方法有很多，但我们提出的方案是对工具和技能进行向量化，生成它们的向量嵌入。借助这些向量嵌入，当智能体循环——我回到刚才的话题。智能体循环想要检索一些有用的技能或工具时，就会针对你拥有的这个向量存储进行向量相似度搜索。right? And also the toolbox pattern, the skillbox pattern, what we propose for this for storing tools and skills, which are hundreds. I don't know about you, but I have like a hundred skills already installed. I'm not putting a 100 skills at the same time into the context window because each skill is like 3,000 5,000 uh tokens. So that would automatically like go over the advertise limit in no time. I I would have no space for my actual conversation with the agent. So how do you fix that? Well, there are different ways, but what we propose is that you vectorize, you create vector embeddings out of the tools and out of the skills. And through those vector embeddings, once the agent loop, let me go back. Once the agent loop wants to retrieve some useful skills or tools, you're going to do vector similarity search against this vector store that you have.

你会调取最相关的技能或工具来解决问题。解决完问题后，你又会把它卸载掉。这也是你在某种意义上保持上下文窗口内容相关的方式之一。你知道，使用技能和工具是非常非常常见的模式，大家都在使用工具，比如 bash 命令或切换目录；这些都是通过 bash 使用的特定工具。所以我们一直在使用工具和技能。在某些情况下，这些内容占用了超过 50% 的上下文窗口，对吧？所以我只是想提醒你，要留意技能和工具实际上占用了上下文窗口的很大一部分。You're going to pull the most relevant skill or tool to solve it. You solve the problem and then you offload it again. And that's one of the ways in which you uh keep the context window relevant in this in this sense. And you know using skills and using tools is a very very uh common pattern like everyone's using tools like a bash command or a change directory all these are specific tools through bash right so we're using tools and skills constantly this is for some cases more than 50% of the context window that's being used right so I just want you to be mindful of the vast amount of context window size that is actually being kept by the skills and by the tools.

技能提炼Skill distillation

嗯，我们还有一个叫作“技能蒸馏”的想法：你可以拿之前的对话、运行记录，也就是你和 AI 智能体实际进行过的对话，然后试着从中提炼出有意义的东西。要怎么做呢？方法有好几种，其中最简单的一种就是创建一个 skill MD 文件，并把它放进之前放置所有内容的同一个技能框里。Hermes 智能体就是这样一种方式的例子；在这方面，它是创新者之一，创建了一些特别擅长解决特定问题的技能。比如说，你正在做前端开发。那么，你可能会想创建一个非常适合自己、包含所有个人偏好的技能。比如：“嘿，我喜欢这种配色风格。”等等。Um we also have this idea called uh skill distillation where you take previous conversations, runs and you know actual like conversations that you've had with an AI agent and then you try to create something meaningful out of that. How do you do that? Well, you can do it in several ways, but one of the easiest ways that you can do it is just like create a skill MD and put it into the same skill box that you had everything before. And this is one of the ways like a Hermes agent, one of the innovators in this sense was creating skills that are particularly good to solve specific specific problems. Let's say that you're working on a front end. Well, if you're working on front end, you might want to create a skill that has worked very well for you with all your preferences. Hey, I like this coloring style, etc.

下次你做前端工作时，由于你已经基于这些内容创建了向量嵌入，智能体就会自动选择更相关的技能。所以，如果你只是提一句：“嘿，我喜欢我在另一个技能里用过的这种配色风格”，它就会自动知道你指的是新技能，因为你用了“配色风格”这个词，比如说，对吧？所以，技能蒸馏是你可以用来随着时间推移改进模型学习效果的一种技术。这也和我刚才说的有关：这里有动态技能发现，也就是说，智能体循环在组装或构建上下文时，会在需要时提取合适的技能和工具。你知道，有些人会问，比如说怎么……And the next time you work on the front end, since you have created vector embeddings out of this, you can just like the agent will automatically choose the more relevant skill. So if you just mention, hey, I like this coloring style that I had in this other skill, it's automatically going to know that you're referring to the new skill because you use the word uh coloring style for for instance, right? So skill like skill distillation is one of the techniques that that you can use to improve the the learning the learning of a model over time. And it's the same uh this is uh related to what I what I was saying is that you have dynamic skill discovery in the sense that the agent loop when assembling or building the context will pull the right skills and the right tools when it needs and you know some people ask uh like how

冻结模型如何变得更好：持续学习How a frozen model gets better: continual learning

冻结模型会变得更好吗？我所说的冻结，是指我们无法控制前沿模型，对吧？但我们确实可以控制它的所有数据，也就是数据层，对吧？有些人会说：“嘿，我们来修改模型的权重。”另一些人会说：“我们来创建一个针对这个特定问题效果更好的嵌入模型，然后我再考虑如何检索更好的嵌入，等等。”实现这一点有不同的方法，但核心思路是：这是持续学习的主要组成部分。持续学习是你可以用来改进模型的几种方法之一。它分为三类，对吧？你可以处理参数化记忆。之所以称为“参数化”，是因为它会修改神经网络权重中的参数。does a frozen model get better and what I mean by frozen is that we we don't get any control over the frontier models right uh but we do get control over all the data the data layer of it right and some people say hey let's modify the weights of the model some other people say let's create an embedding model that works better for this specific problem and then I'll worry about you know just retrieving the better embeddings there uh etc so there are different ways to do that but the idea is that this is the main component of continual learning and continual learning is one part of of of the things that you can do to make a mo model better. And there are three categories, right? You either work on the parametric memory. It's called parametric because it modifies the parameters of the weights of the neural network.

你们还有表征性记忆，它涉及嵌入等内容。还有非参数记忆，也就是截至目前我们今天谈到的所有内容。不过，如果你们对这三类中的任何一类感兴趣，我们重点讲的是第三类。其他两类你们也可以了解，但我就不展开讲了。有很多相关技术，比如低秩适配、对模型进行微调、创建嵌入等。你们可以在我之前提到的那门课程中找到这些内容，那门课重点讲的就是这三类。所以，如果感兴趣，可以去了解一下那门课，或者在会后和我们聊聊。会后，我们也会在活动现场待到活动结束，对吧？最后我想提到的是，在调用任何 OpenAI 模型或其他你们正在使用的模型的聊天补全 API 时，所有发生的事情都内在地……You also have representational memory which is what it works with embeddings etc. And you also have nonparametric memory which is everything that we have spoken about today until this point. But um if you're interested in any of these three, we have focused on the third one. You also have the other two and I will not go into the uh into those two. There are lots of techniques like um you know creating low rank adaptation uh fine-tuning a model uh creating embeddings etc. And all these things you can find in that course that I I was referring to before which focuses on the three categories. So if you're interested just take out uh take that course if uh or chat chat with us after after the session. So after the session, we'll be in the event for the for the rest of the event too, right? And last thing I want to mention is that all of the things that happen during a chat completions API call to any uh OpenAI model or anything that you're using is that there is an inherent uh

指令层级Instruction hierarchy

指令层级。指令层级是什么意思？传递给模型的内容具有不同的权重。比如，我们用来交互的是用户，对吧？也就是请求。从这个意义上说，这是我们可以传递给模型的第三重要内容。最重要的是系统提示，对吧？大家都知道系统提示这个概念。嗯，系统提示本身没问题，但它不可能容纳所有内容，否则指令本身就不会有层级之分。呃，你需要把一部分内容放在系统提示里，一部分放在用户提示里；而所有可用的工具和技能，也都有各自的层级。这很好，因为这实际上能让我们，或者说让模型，把更多注意力放在某些事情上，而不是其他事情上。我们也会在研讨会上详细讲到这一点。instruction hierarchy. What do I mean by instruction hierarchy? There is a different weight on the things that are given to the model. For instance, what we use to interact with is the user, right? The request. In this sense, this is the third most important thing that we can pass to a model. The most important one is the system prompt, right? Everyone knows about this the system prompt concept. Um, and the system prompt is fine, but it cannot hold everything because otherwise things would not have a hierarchy in itself. uh you need to actually put some of the things in the system prompt, some of the things in the user prompt and all the tools and the skills that the things that are made available, they have this hierarchy. And this is good because it it's it actually um lets us or lets the model focus more on some things than others and we'll also go through that in the workshop.

所以So

工作坊 UI 演练Workshop UI walkthrough

仅供参考，呃，这是工作坊的界面。所以如果你已经在 Codespaces 里运行它，就会看到这个已经部署好了。我会带你们过一遍。不过，基本上，这就是我设想的一个在 AML 领域工作的人——比如银行里负责欺诈检测系统的人——与自主代理交互的方式。这个自主代理在后台 24/7 全天候运行。它不会睡觉，也不需要我们人类所需的那些东西，对吧？它通过一个运行框架连接到数据库，并具备我们提到的所有功能。除此之外，它还会自主地为公司做出决策，即使我们处于离线状态也一样。它不会仅仅因为觉得某个人可疑，就毁了别人的生活，比如说：“嘿，这个人很可疑。”just for reference uh this is the the UI of the workshop. So if you're running it in code spaces already, you'll see that you have uh this deployed. I will go through through it uh with you. But this is essentially the way in which I envision someone that works in AML like um a fraud detection system in a in a bank interacts with an an autonomous agent. The autonomous agent is running in the background 24/7. It doesn't sleep. It doesn't need any of the humanly uh necessities that we have, right? It's connected through a harness to the to a database and it has all the functionalities that we have mentioned but on top of that uh it's autonomously making decisions for the company even when you know we're offline. It's not ruining people's lives by just like saying, "Hey, this this guy is suspicious.

我们干脆把整家银行之类的都拦下来，你知道，因为那样我会毁掉那个人的生活。但我们真正要做的是让它做出这些决策，并把决策打印出来，供人之后检查。这样能在发现过程中节省很多时间，等等。所以到研讨会结束时，如果你完成了全部内容，就能拥有一个连接到数据库的智能体，它可以像任何一家公司的实际系统那样运行，并自主进行分析和决策，然后由人来接手并与之交互。这些就是我们刚才讨论过的所有智能体运行框架组件。废话不多说，我再放一次二维码，或者你可以访问 workshopwaitingroom.com。如果你想看看我们举办的其他研讨会，也可以看看这个 GitHub 仓库。我们一直在努力推进里面的各种内容，包括研讨会、课程、notebook 等。你可以从不同主题入手，比如智能体运行框架等等。Let's just like block the whole bank and everything, you know, because I would ruin ruin the person's life, but rather it's making these decisions and just like printing them so that a human can actually inspect them later on, but it's going to save a lot of time during the, you know, the discovery process, etc. So by the end of the uh of the workshop, if you complete it, uh you're going to be able to have an agent that connects to a database that is actually working as if it was uh any any company and it's going to do autonomous analysis and decision making that then a human is going to take and interact with. So these are all the the components that we have talked about about the harness and without further ado I will uh put the QR code just again or workshopwaiting room.com um and if you want to check out any of the other workshops that we have uh you can just check out this GitHub repo we're making a lot of uh progress and effort into all the things that go in there everything from workshops sessions uh notebooks So you can get started on different topics like agent harnesses etc.

所以，大家可以去看看。呃，我打开 Firefox，给大家演示一下这个应用是怎么运行的。首先，对于没有收到邀请的各位，如果你们访问 workshop waiting room.com，就会通过 GitHub API 收到一封邀请，使用你们的用户账号访问这个仓库。所以，你们只要点这里，然后在 GitHub 上打开 Codespaces，它就会提示你创建一个代码空间。当然，费用由我来付，所以不用担心。呃，只要注意你们部署时选择的硬件，因为费用很高。不过，整体思路是，你们会得到这样一个部署，其中包含几个笔记本文件。我会给大家展示一下工作坊本身，然后我打算几分钟后先退下来，接着我们会看看大家有什么问题之类的，然后一起解决。So just check it out. Uh and let me let me go to my Firefox and show you how this application works. So, first of all, let me just uh for those of you who didn't receive an invitation, if you go to workshop waiting room.com, you will get an invitation via the GitHub API with your uh user to this repository. So, if you just go here and then open on GitHub code spaces, it's going to ask you uh to create a code space. And of course, I'm paying for it, so don't worry about that. uh just be careful with um with the hardware that you deploy because it's expensive. But uh the idea is that you will get a a deployment like this with a couple of notebooks and I'll show you the workshop itself and then my idea is for me to step down in a couple minutes and then we'll go through the you know people who have questions etc and then we'll solve it together.

但它也会自动部署这个应用，本质上它会跟踪所有信息。当然，这些都是虚构数据。别担心，我没有暴露任何人的信息；它本质上是在生成新的交易记录。有些没问题，有些则有问题。所以，这个运行在 OCI 上的 Groq 模型自主智能体会自主做出决策，其中有些时候它会说：“嘿，这笔交易很可疑，把它加到列表里。”之后，你还可以在这里看到智能体的上下文窗口；也可以查看数据库里已有的所有数据，比如所有客户的信息等。此外，智能体还能自主实时分析所有这些数据，对吧？But it's automat automatically also deploys this application which essentially is tracking all the things and this is of course uh fake data. Don't worry I'm not exposing anyone here but it's essentially coming up with new transactions. Some of them are fine some of them are uh problematic. So the autonomous agent which is a grog model running on OCI it's essentially making autonomous decisions and some of them it's saying hey this one is suspicious let's add it to the list and then afterwards uh you also get like a way to visualize the context window of the agent here um you can also check out all the data that's already in the database uh here uh like all the customers etc and also an autonomous way in which the agent is actually going to make an analysis of all this um of of all this in real time, right?

所以，它会调用我们在工具箱中提供的工具。嗯，是的，不好意思，这里显示得有点小。嗯，你知道的，它会检查证据，并在我们通过向量存储创建的知识空间中搜索知识。它会调用很多工具。最后，它基本上会自动检查整个活动，也就是可疑活动报告。这里用一种比较直观的方式展示了银行最近发生的 30 起事件。工作坊本身就是这个笔记本，也就是学生版笔记本。我希望你们首先看看其中的各个组件，因为我们会在这里实现它们。接下来，我会讲第一个待办事项。So, it's going to run through the tool calls that we give uh that we have available in our tool um in our toolbox. Uh yeah, it's sorry, it's a little small. Um you know, so it's going to go through the evidence. It's going to search for knowledge in the knowledge spaces that we created through vector stores. It's going to make a lot of tool calls. And finally, it's going to essentially go through the whole activity uh suspicious activity reports automatically. And here is just a fancy way of visualizing the 30 most recent um events that happen in the bank. But the workshop itself is this notebook, the notebook student. And what I want you to do is first of all just check out the different components that we have there because we are implementing them uh here. And I'm going to go through the first uh the first to-do.

有几件事你需要在 workshop 上做一下，以确认你知道自己在做什么。首先，我先给你展示一下 workshop 的第一部分，好吗？你这里有一个 rock 模型 4.3，它运行在 Oracle Cloud 上。我们来连接数据库吧。当然，抱歉，我先重启一下内核。运行这个单元格。重启。不过，如果你在进入代码空间时有任何问题或疑问，请告诉我们，我会走过去和你一起操作，并回答你的问题。到目前为止有什么问题吗？好的。是的，那个 repo。There are a couple of things that you need to do on the workshop just to confirm that that you know uh what you're doing. So first of all, let me just go ahead and show you uh the first part of the workshop, right? Uh you have this rock model 4.3 uh which is running on on cloud Oracle cloud and let's uh connect to the database of course. And after sorry, let me restart the kernel. Let me run this cell. Restart. But yeah, uh let let us know if you have any questions or issues uh when getting into the code space and then I'll step down and and do this with you and answer any questions. Any questions by the way so far? Okay. Yes. Uh the the repo.

嘉宾：是的，你可以在 workshopwingroom.com 上找到它。我把二维码放在这里。如果你想扫码的话，可以扫这个二维码。这样就能通过 GitHub API 自动获取访问权限。好。那……Guest: Yes, you can find it on workshopwingroom.com. And let me just put the QR code here. Uh this QR code if you want to uh scan it. And this will automatically give you access to to it via GitHub API. Okay. So

问答：情景记忆与冲突Q&A: Episodic memory & conflicts

让我重新——呃，重新表述一下这个问题，给正在 YouTube 上观看的人听。嗯，他问的是情景记忆和所有这些记忆组件会发生什么。它会不会和之前与某个智能体的对话混淆？这取决于你赋予它多大的权重。这也是我提到指令层级的原因。要是某件事非常非常重要，而且你确实想从某个地方调取信息，你可能会想把它放进去。呃，你完全可以自己决定记忆层的架构，对吧？比如在某些情况下，情景记忆非常重要，而且你确实想调取之前的对话，那么你可能会想把它向量化，而不是只把它保存为原始文本，对吧？这在很大程度上取决于具体的使用场景。let me re uh reframe the question for the people who are on YouTube. Um so he's asking about what happens with the episodic memory and all these memory components. Does it get it confused with previous conversations with a with an agent? Well, it depends on how much uh weight you give it. And this is why I was mentioning the instruction hierarchy as well. You might want to put this if it's a really really important thing and you actually wanted to pull from some place or the other place. Uh you can actually decide your memory layer architecture yourself, right? So you might want to for instance in some cases where the episodic memory is really important and you actually want to pull from previous conversations you might actually want to vectorize that rather than just save it as raw text right depends a lot on the on the use case.

我会说，在这种情况下，它不会，呃，不会；这里不存在冲突，因为架构中的记忆组件有明确的区分。是的。所以这些技能可以……I would say in this case it doesn't uh it doesn't there is no confliction there because we have a clear separation on the memory uh components of the architecture. Yes. So the the skills can

问答：技能、缓存与上下文预算Q&A: Skills, caching & context budgets

添加到系统提示词中。呃，它们通常……哦，抱歉，它们通常是不同的。呃，对，在指令层级中，它们的权重是四者中最低的。如果你这么想的话，它通常就在这里，所以权重最低。我会把它看作一个即插即用的地方，你可以从中获取各种东西。我们正在尝试这么做，实际上是我和 Richmond，因为我们确实想创建一种方式，让你可以接入外部、内部或第三方的技能中心和工具中心。比如说，一个 MCP 集合，你可以通过预先计算所有这些东西的嵌入向量，让你的智能体能够使用它们。等预先计算好这些嵌入向量后，你就可以直接说：“嘿，针对这个第三方技能组或工具组进行向量搜索。”不过，与其把它看作是被包含在其中的东西——顺便说一下，这当然包含在你发送给 OpenAI 或任何其他模型的每个 OpenAI 或 Chat Completions API 请求中。be added to to the system prompt. uh they typically um oh sorry they typically have a different uh yeah in the instruction hierarchy they have the least weight out of the four uh if you were to think about it it um it's typically here so it has the least weight I would think of it as a plugandplay place in which you can pull things and we are experimenting with this uh actually Richmond and I because we actually want to create a way in which you can plug into external internal or third party skill hubs and tool hubs for instance a collection of MCPS that you can give your agent availability by precomputing the embeddings of all of these things and once you have these precomputed embeddings you can just like say hey do vector searches against this third party uh group of skills or group of tools but rather than thinking of it as being included and by the way this is of course included in every open AAI or every chat completions API request that you're making to open AAI or any other model.

这是附加在底部的，通常大概占了其中的 80%，也就是说，整个请求中大约 80% 是工具和技能。但是，为什么你没有被收取现在费用的五倍呢？原因是，OpenAI 和所有这些系统都有语义缓存。如果其中 80% 和上一个请求相同，它就不会重新计算，而是直接使用语义缓存。这让你，也让我们所有人，都能以一种经济实惠的方式使用 Frontier Labs 的模型。当然，它仍然非常昂贵。但我希望你明白，如果没有这个语义缓存来缓存这些工具和技能，费用就会贵四到五倍。因为要记住，当你与前沿 AI 模型交互时，本质上整个过程都是通过 HTTP 请求完成的，而像这样的内容，例如可能会有 10K 或 5K，作为系统提示词。This is attached at the bottom and this typically is like 80% of it like 80% of the whole request is uh is is tools and skills. But what what happens or like why don't you get charged like five times more than what you're charged now is because uh OpenAI and all these systems they have a semantic cache and they put uh if 80% of it was the same as the previous request it doesn't recomputee it just goes into the semantic cache and this allows you and this allows all of us to have um an a way in which to consume the Frontier Labs models in an economical way and of course it's still very expensive. But what I want you to understand is that it would be four or five times more expensive if we didn't have this semantic cache that just um puts the tools and these skills into it. Because in the response like remember that when you're interact with interacting with a frontier AI model, you're essentially making the whole thing through an HTTP request and everything like this would be for example 10K or 5K as a system prompt.

你是一个 AI 助手，然后你需要帮我解决这个问题、那个问题之类的。你会收到开发者提供的具体应用指令、用户查询（也就是我们在聊天中输入的内容，这部分微不足道，大概 100 到 1,000 个 token），然后还有工具输出和检索到的文档，这些内容占请求的 80%，每次请求中已经有大约 35 到 100,000 个 token。这样回答你的问题了吗？you're an AI assistant and then you need to help me solve this this and that. You have the developer like the specific app instructions, the user query which is what we type in the chat which is negligible like 100 tokens thousand tokens maybe and then the tool outputs and the the retrieve docs which is 80% of the request like 35 to 100,000 tokens is already this in every request. Does that answer your question?

主持人：最后一个问题。Host: One last.

访客：是的。Guest: Yes.

主持人：好的。他的问题大概是：你们如何管理预算？因为 Skillbox 和 Toolbox 里有这么多东西和模式，你们究竟是怎么让这一切运转起来的？嗯，我们对上下文窗口有明确的划分。我们把它看作一个从零开始的 n 维矩阵，但它实际上从来不是零，更接近 10,000。我之前读过一篇论文，比较了各个 harness 的效率，考察它们在最开始时分别放入了多少内容，包括第一个系统提示词和首次 Completions API 请求。Pi，你知道的，Pi 这个 harness 是最精简、效率最高的，因为它在每次请求开始时只包含大约 6,000 个 token。但换成其他 harness，内容就更多了，大约有 30,000 到 40,000 个 token。嗯，我们如何……我们对上下文窗口有明确的划分？Host: Okay. So his question was like how do you manage the budget and since there are so many things in the skillbox and the toolbox patterns like how do you essentially make this work right um we have a clear separation the context window we think of it as or think of it as an n-dimensional matrix that starts at zero but it's never zero it's closer to like 10,000 I was reading a paper that was comparing the efficiency of each harness on how much they put into the first system prompt and the first uh completions API request at the beginning. Pi, you know, Pi, the harness, that's the most minimalistic and the most efficient uh harness because it includes about 6,000 tokens at the beginning on every request. But if you go to other harnesses, they have more like 30 40,000 tokens. Um how do we we have a clear separation the context window?

我们就让所有东西都围绕着 agent loop 运行，对吧？那么，我打开 agent loop，给你们看一件事。agent loop 会控制它连接的所有东西，对吧？嗯，比如这里的 agent loop。[哼了一声] 哦，我在最底下。稍等。嗯，先把 agent loop 当作参考。所以这里，嗯，在你使用它时，上下文组装这部分，你基本上会构建……你可以在笔记本中了解这一过程。我来解释一下笔记本，不过，上下文组装是其中一个环节。首先，我们会定义要检索的工具或技能的最大数量，我想默认数量是三个。所以每次有新请求进来时，它都会判断——这是由模型本身完成的——判断要做什么，然后尝试组装执行任务所需的上下文，对吧？we just let the everything goes around the agent loop, right? So, let me go to the agent loop to show you uh one thing. The agent loop will control everything that it connects to, right? Um so here, for example, the agent loop. [snorts] Oh, I'm at the bottom. Hold on. Uh just as a reference um the agent loop. So here right uh the context assembly when you're working with it uh you will essentially build and you can go through this in the in the notebook. explain the notebook but um the context assembly is one part that first of all we have a maximum number of defined uh tools or skills to retrieve I think the defi the default number is three uh so every time a new request is coming it's determining and this is done by the model itself it's determining what is the thing to do then it's going to as try to assemble the context with the things that it needs to perform Right?

其中一项就是检索出最相关的三个工具、最相关的三个技能。我们没有使用重排序器。不过，你也可以在相似度搜索结束后，再加上一个嵌入模型和一个重排序器，让它从这三个结果中判断出哪个最重要。这样实际上可以进一步优化结果，因为你可能不希望每个任务都加载三个技能。任务也许只用一个技能就能解决，对吧？所以，关键是你可以采用自己想要的任何策略，只要选定之后就坚持使用，并始终保持一致。在这个工作坊的具体案例中，我们要对一个存放工具的向量存储和另一个存放技能的向量存储分别进行向量搜索，k 等于 3；然后从这三个结果中通过重排序器选出最相关的一个，接着当然会分派工具并构建上下文窗口。但这完全由智能体循环来决定和负责。And one of these things is retrieving the three most relevant tools, the three most relevant skills. And we are not using a reranker. But what you could also do is like at the at the end of uh the similarity search, you can also put an embedding model plus a reranker that will take those three and tell you out of these three, this is the most important one out of the three. And you could actually have a better refinement out of that because you you might not want to have a total of three skills loaded for every task. The task might be solvable by only one. Right? So the idea is that you can put uh whichever strategy you want just uh stick with it and uh stay consistent with it until the end. Uh so the idea in this case in the particular workshop is that we have a total number of uh vector searches that we're doing against one vector store which holds the tools another vector store that holds the skills k equals three and then out of the three you can retrieve the most relevant one with a reranker and then it's going to dispatch the tools of course and build a context window but this is entirely the decision and the job of the agent loop.

是的。Yep.

问答：让你的记忆层保持简单Q&A: Keep your memory layer simple

好的。那么，你与内存层交互时，最重要或最有意思的方式是什么？Okay. So what is the most uh important or interesting way in which you you can interact with the memory layer of

嘉宾：我的意思是，效率完全取决于你创建的实体数量，对吧？所以我的首条建议就是尽量保持简单，呃，不要创建太多抽象层，也不要给记忆层划分太多组件。比如，首先可以尝试只使用短期记忆和长期记忆这两种结构，对吧？如果你认为，或者你的智能体认为，短期记忆或上下文窗口中的某些内容值得保留，就把它提升到长期记忆中，并以原始文本的形式存放在某个地方。这是最简单的做法，对吧？等你把它作为文本存放在长期记忆组件或对象的某处之后，就可以对它进行向量化，然后让智能体循环以某种方式检索和探索它。Guest: I mean the efficiency comes just from the number of entities that you create right so my first recommendation would be just like keep it simple uh so don't create too many abstractions or too many components of the memory layer so for instance try to experiment first of all with only short-term memory and long-term memory and just have two structures Right? If you think or if you have an agent that considers that something that was in the short-term memory or in the context window is kept is is worth keeping, you promote it to the long-term memory and you just put it somewhere as raw text. That's the easiest thing that you can do, right? Once you have it in text somewhere else in a long-term memory component or object, then you can take that vectorize it and then let it be explored by the agent loop in some way.

所以比如说，在这里，你只需要在观察和推理之间加一个额外的小模块，让智能体能够回忆记忆之类的。你可以把它叫作“回忆长期记忆”。这样一来，它就能在再次解决问题之前进行自省。So for instance, here you would just have like an extra thingy in the middle of observing and reasoning that lets the agent recall memories or something. You can call it like recall recall long-term memory. And this is going to make it uh be introspective before solving the problem once more.

呃，所以它会进入你的向量存储，或者你用来存放长期记忆的其他地方。比如说，你在之前的一次对话中记录下来的这段经历。然后你可以让它判断：好，如果它觉得这段内容有意思，或者向量相似度合适——也就是说，查询向量和长期记忆中的内容之间的距离足够小、相关性足够高——那我就把它纳入上下文窗口；如果距离太大，或者相关性不够高，那我就直接丢弃它。这样一来，在观察和推理之间就多了一层抽象。我们这里没有用 MCP，不过使用 MCP 的好处之一是能解决……我想说的是，这种情况下 MCP 其实没那么相关，但这取决于你的数据库里有什么。比如说，如果你有一个数据库，并且像我们现在这样要与数据库交互。Uh, so it's going to go to your vector store or whatever you have this long-term memory uh, episode that you recorded in a previous conversation and from that you can let it see okay I'm going to let it pull from it if it considers it to be interesting or if the vector similar like if the distance between the vector of the query and what I have in my long-term memory is relevant enough and the distance is small enough then I will consider it into the context window if it's too big or it's not relevant enough then I will just discard it so it will be like a different an additional abstraction between the observing and the reasoning part there are ways we are not doing MCPS right here but one of the benefits of of using MCPS would be to solve um I would say that MCP is not really that that relevant in this case um but it depends on what you have in the database so for example If you have a database, you're interacting with a database like we are doing right here.

拥有 MCP 挺有意思的，但我经常会问别人一个主要问题：MCP 到底解决了什么问题？尤其是在数据库领域，因为我是做数据库的，我在 Oracle 工作。嗯，所以你可以做的一件事是……Having an MCP is interesting, but one of the main problems that I like to say to people like what does an MCP solve, especially in the database, because I'm a database guy because I work in Oracle. Um, so one of the things that you can

问答：预构建 SQL 与动态查询Q&A: Pre-built SQL vs. dynamic queries

Solve 让 AI 代理能够执行 SQL 查询，同时避免产生幻觉，或每次都试图重新编写一条 SQL 查询。这是 AI 代理使用 SQL 时面临的较大问题之一。SQL 有大约 15 种引擎，比如 PostgreSQL、Oracle Database、VSQL，还有许多不同的 SQL 查询；但这些查询中有一些特定的提示或细微差别，可能导致它无法正常运行，对吧？所以在这种情况下，MCP 解决的主要问题是：好，我会预先创建一些 SQL 查询，然后让代理访问这些查询。这样这些查询就已经经过验证、能够正常运行，而不是让代理每次都动态地尝试编写 SQL 查询，对吧？没错。好问题。呃，我举个例子。最重要的应该是响应速度，以及句子的含义，还有技能本身代表的意思。solve is letting AI agents make SQL queries, but by not hallucinating or trying to come up with a new SQL query every time. This is one of the bigger problems that that AI agents have with SQL. SQL has like 15 times like 15 engines like posgres, you have Oracle database, you have VSQL, you have like lots of different queries that work on SQL, but they have specific hints or subtleties in the query that makes it not not work well. Right? So the main problem that MCP solves in this case would be okay, I'm going to pre-create some SQL queries and then let the agents access those. So those queries are proven to work um rather than having an agent try to come up with a SQL query dynamically every time, right? Yep. Good question. Uh let me give you an example. So the most important thing would be the rapidness and that the meaning of the sentences and what the skill means itself.

呃，与其把它作为文本存储在 SQL 文件里——抱歉，是存储在文件系统中的文件里，也就是操作系统中——这样代理就有很多方式可以进行正则表达式匹配。比如，所有搜索基本上要么是关键词搜索，要么是在该技能中进行词语搜索，这没问题。但文件越大，对代理来说就越难处理。首先，对于超大文件，像 grep 这样的正则表达式匹配效率很低；有时，随着文件系统规模不断线性增长，正则表达式匹配的性能会呈指数级下降。嗯，不是指数级，应该说是二次方级别。但这就是问题之一：它无法很好地适应文件大小的增长。Uh rather than having it as text and if you were to store this just as text in a SQL file it would be able sorry in a in a file in the file system right in the operating system then you would have uh lots of ways in which an agent can do regular expression matching. So for instance, all the search would basically be either keyword search or like word search based uh into that skill which is fine. Um but the bigger that gets the less in like the the harder it gets for an agent. First of all, regular expression matching like GP is inefficient for huge files and sometimes if uh like the the file system grows and grows linearly, the performance of the regular expression matching decreases exponentially. Well, not exponentially, quadratically, let's say just quadratically. But this is one of the problems um that it doesn't scale as well as the size of the files.

呃，另一个例子是，如果你预先计算某项技能的嵌入向量，实际上也会捕捉到这些嵌入向量中的含义，也就是我们人类无法理解的隐藏含义。但它不只是用文本来描述这些内容（文本对我们来说很容易阅读），它还会将其计算为至少 384 维、甚至 1,000 多维的向量。而这个嵌入向量实际上会包含嵌入矩阵中每个词元之间的信息，使其更容易被发现，例如通过语义……Uh, another one would be that if you premp compute the embeddings of a skill, you're actually also capturing the meaning in these embeddings, the hidden meaning that we as humans don't cannot comprehend. But rather than just describing it in text, which is fine for us to read, uh, it also has a premputation of 384 minimum to a,000 plus dimensions. And this embedding will actually hold uh things between each one of the tokens in the in the in the embedding matrix to be much more discoverable for instance by semantic

混合搜索（语义 + 关键词）Hybrid search (semantic + keyword)

搜索。接下来，这就促使你尝试混合搜索，也就是语义搜索（使用嵌入）与关键词搜索的结合。这称为 RRF，是一种混合检索方式。它的效果要好得多，因为它本质上同时使用了两者。我可以在工作坊上给你看这个例子，因为我在这里制作了这个可视化图表，能准确说明我所说的混合搜索是什么意思，对吧？你看，这张图本质上展示了关键词搜索和语义搜索能力的结合。你看到中间的这个交集了吗？在进行混合搜索时，落在这里的结果会比另外两种搜索各自得到的结果都更相关。search. And then this is what leads you to try hybrid search which is a combination of semantic searchs uh which is embeddings plus keyword search and and this is called RRF uh which is a type of hybrid retrieval. it performs much better because it will essentially just have both of them. Uh and it will essentially and this is uh we can I I can show you uh the example in the in the workshop because I I created this visual visualization here that explains exactly what I mean with hybrid search, right? Uh you see yeah so depending on the on the this essentially is a plot that uh combines the search capability of keyword search plus semantic search and this gap that you see at the middle is anything that's that's plotted here when doing hybrid search it will be more relevant than any of the other two.

所以，呃，总结一下，呃，你可以有两种不同类型的搜索。每一种都有自己的优势，但如果把它们结合起来，虽然会多消耗一点计算资源，但随着时间推移，检索质量会更好。嗯，我想这些大多是从 AI agent 的角度来考虑的。所以我不指望有人会从这个……So uh as a summary uh the more like the more you can have two different types of of search. Each one has its advantages but if you combine them both you will uh effectively do a little bit more compute but then the quality of the retrieval will be better over time. Um these are like most of these I'm thinking of it from the AI agent perspective. So I'm not expecting anyone uh from this uh

构建小型自定义测试框架Build small custom harnesses

这样大家就有空间开始构建自己的定制化执行框架，因为市场上已经有很棒的解决方案了，对吧？我们有 Claude Code、Codex，每个人都有自己的偏好。顺便问一下，喜欢 Pi 的举个手？比我预想的少，不过好吧。那么，我会从 AI agent 的角度来思考这件事。我想让你们明白的是，你们可以针对特定问题构建定制化执行框架。而构建定制化执行框架，不一定要像打造一个全新的 Claude Code 那样复杂，然后把所有工作都放进那个定制框架里运行。相反，我们可以把你们试图解决的问题简化，然后构建能够接入这些问题的定制化执行框架。就这个例子来说，我想的是，如果你是一名顾问，那你就是我理想的应用场景，也是参加这场工作坊的理想用户，因为你可以创建一个能很好地服务于金融代理的定制化执行框架，并配齐金融应用场景所需的一切。room to just like start building their own custom harnesses because we already have great uh solutions in the in the market right we have cloud code we have codeex each one has their own um preference by the way a show of hands for who likes pi okay less than I expected by but uh good so um I would say that I'm thinking of it as from the AI agent perspective. This what I want you to understand is that you can build custom harnesses for specific problems. And building custom harnesses doesn't to be doesn't have to be as sophisticated as creating the new clo the new clo code and then you're running all of your work in that custom harness. Rather, let's simplify the problems that you're trying to solve and then build custom harnesses that attach on to onto that. So in this case I'm thinking of um you know if you're a consultant you are my perfect uh use case or my perfect user for this workshop because you could be creating a custom harness that is working fine for a financial agent and has all the things that it needs to a financial uh use case.

但如果你有另一类客户，或者数据类型、约束条件不同，那么如果你想解决下一个 Navir Stocks 问题之类的，嗯，你知道，这些问题的约束条件也不一样。比如 OpenAI 的那些问题，也就是数学问题，它们需要无限的自动压缩，或者自动进行上下文摘要，因为这些智能体在运行过程中没有人类参与。所以，围绕解决这些问题的一切，默认都必须具备自动的上下文摘要功能。它还必须有无限的耐心，或者说接近无限的耐心。在这个案例中，这个智能体循环被简化为最多只有八个步骤。所有这些考量都取决于具体问题。是的，我会从头开始，这样你就能去掉 90% 不需要的东西。But then if you have another type of customer or with different types of data or with different types of constraints if you want to solve the next navir stocks problem or something uh you know those have different constraints for for instance the open AI problems the mathematical problems they need an infinite an automatic compaction or automatic context summarization because these agents don't have any human in the loop in the middle. So the like everything that goes around solving these problems have to have by default automatic uh automatic context uh summarization. It also has to have infinite patience or near infinite patience. In this case this agent loop is simplified to only eight steps maximum. All these considerations right depends on the on the problem. Yeah, I would start from scratch because that way you can you get to uh strip 90% of what you don't need.

嗯，所以我肯定……对，我的意思是，一切都是围绕 Chat Completions API 构建的。Anthropic 和 OpenAI 都创建了各自的标准来对抗它，但一切都还是围绕着这个 API。所以，如果你把与模型通信的方式看作是 harness 的一个额外插件，就会更清楚地看到，整体越精简，最终效果就越好。这样回答了你的问题吗？好。那么我们来做这个。你们先从代码库开始，用一个代码空间。你们自己把笔记本设置好，然后我们就开始。我会巡视一圈，如果有人有问题，我们可以轻松地聊聊，不用那么紧张——我觉得自己好像正被人盯着，而你们可能也觉得有压力。所以我先离开，把压力留给你们。那我们继续吧。Um so I would definitely yeah I mean everything is built around the chat completions API and both Anthropic and OpenAI have created their own standards to fight off against this but everything goes around that API. So if you just think of uh the way to communicate with a model just as an additional plugin to the to the harness, it becomes much clearer that the smaller the everything is, the better it is at the end. Does that answer the question? All right. So let's do uh this. You guys uh start with the code base um with a code space. just set that uh notebook uh by yourselves and then we're gonna go I'm gonna go through and do a round and if anyone has any questions we can chat in a less uh aggressive way because I feel observed and you probably feel no pressure. So I'm going to put the pressure on you by stepping off and if you Yeah, let's uh keep going.

我是说，我们时间不多了。天哪。嗯，时间过得真快。嗯，不过，是的。谢谢。谢谢大家的掌声。嗯，谢谢，谢谢。呃，我还以为才过了15分钟或者20分钟，没想到已经过了一个小时。呃，有件事……我准备好了，你们可以在家里的酒店创建代码空间。我会自己付费。所以你们可以使用这个研讨会的内容，随意动手尝试，费用由我来付。别担心。要是想聊聊代理运行框架的一些概念，就来 Oracle 展台找我们。我们会待到周三晚上。谢谢大家来参加这场讲座。好。I mean we don't have a lot of time. Oh my god. Um I time flew. Um but yeah. Yeah. Thank you. Thank you for the clap. Um thank you. Thank you. Uh so I thought like 15 minutes or 20 minutes went by, but one hour went by. So uh one thing uh so I'm going to leave I set this up so that you can create the code space in the hotel at home. You can I will pay for it myself. So you can, you know, use this workshop and and play with it as as you wish and I will pay for it. So don't worry about it. Just um if you want to chat about some of the agent harness concept, just come to the Oracle booth. We're going to be here until Wednesday evening. And thank you for coming to the to the session. Yeah. Okay.