Oracle 的 Wojciech Pluta：如何决定 AI 智能体应该记住什么

简要描述Brief Description

如何决定你的 AI 智能体应该记住什么——Wojciech Pluta，OracleHow to Decide What Your AI Agent Should Remember - Wojciech Pluta， Oracle

目录Table of Contents

• 简介
  Intro
• 从命令到意图
  From commands to intent
• 数据层就是你的优势所在
  The data layer is your edge
• RAG 负责读取，记忆负责写入
  RAG reads, memory writes
• 记忆提升：哪些内容值得保留
  Memory promotion: what's worth keeping
• 演示：一位有记忆的抵押贷款代理人
  Demo: a mortgage agent that remembers
• 护栏：永远不会被提升的内容
  Guardrails: what never gets promoted
• 什么是智能体记忆
  What agent memory is
• 短期记忆、长期记忆和共享记忆
  Short-term, long-term and shared memory
• 将记忆存储在哪里
  Where to store memories
• 将记忆构建作为一门学科
  Building memory as a discipline
• 上下文卡片
  The context card
• 模型是共享的，内存属于你
  Models are shared, memory is yours

简介Intro

呃，我叫 Wojtek。嗯，Wojtek Pluta，在 Oracle 工作。过去 10 年来，我一直在 Oracle 工作。我参与过各种项目，包括与英国的一些银行合作，开展客户体验、抵押贷款等用例项目，也参与过早期的非接触式支付项目——当时这类支付还处于兴起阶段——以及开放银行相关项目。现在，我负责 Oracle AI Database 的技术营销团队。此前，我也花了一些时间研究云原生技术，以及如何在我们的云基础设施上应用 AI。今天我要谈的是从 RAG 转向记忆系统，以及这样一个理念：RAG 为我们提供了一条有用的读取路径，而记忆系统则能让我们基于自己的数据创造独特的业务价值。Uh my name is Wojtek. Uh Wojtek Pluta, and I work at Oracle. And over the last 10 years, I've been working um at Oracle. I worked at the various projects um including working with some banks in the UK on projects like customer experience mortgage use cases or things like early contactless payments when um when it was still uh sort of emerging and open banking initiatives. Now, I am uh heading up the technical marketing team uh at Oracle AI database. And before, I spent a bit of time also um working on cloud native and AI on our cloud infrastructure. So, what I'm going to talk about today is the um the moving from rack to memory systems and the notion how rack gives us a useful read path, but memory system allows us to create unique business value based on your data.

从命令到意图From commands to intent

因此，多年来，我们一直在探索界面优化这一理念，主要是让界面在交互方式上更加直观。作为社会的一员，我亲眼见证了我们与系统的交互方式如何从命令驱动的点击和输入，转变为如今这种基于意图的系统。这显然为按需或即时软件创造了机会。在这样的背景和期待下，我认为建立某种连续性很重要。So, over the years, we went through this notion of interface optimization uh predominantly making interfaces more intuitive in terms of how we interact with them. And as a society, and I experienced that seeing sort of how our interactions with the systems moved from sort of this command-led uh click and and type into what we have right now, this intent-based uh uh systems. And um which obviously creates opportunity for this sort of on-demand or just-in-time software. And within that context and within those expectations, I think uh creating a some sort of continuity is important.

数据层就是你的优势所在The data layer is your edge

所以，对我来说，过去几年里，AI 技术栈一直相当稳定。嗯，模型层和基础设施层可能已经商品化了，但在我看来，数据层才是你获得竞争优势的地方，因为它属于你。在 Oracle，我们采用的模式和其他所有人一样：我们用“模型加执行框架”来思考智能体，并对它们进行定义。显然，在我们能够访问模型的方式以及访问范围上，模型本身相对比较固定。嗯，然后模型会提出该做什么。不过，最终由执行框架来决定并掌控运行时的决策，以及围绕这些决策的检查。所以，嗯，执行框架显然就是模型周围的代码。为了今天的讲解，也为了让内容不那么抽象，我会用一个小头像。So, for me, the AI stack has been pretty consistent for the last few years. Um there is probably models and infrastructure layers are commoditized, um but the data layer, in my opinion, is where you gain competitive advantage cuz it's, you know, it belongs to you. And at Oracle, we adopted the same model as everybody else in terms of how we think about agents and how we define them with model plus the harness. And obviously, the models we sort of are relatively static in the way that we have access to them and to the certain extent. Um and then they suggest what to do. However, the harness in the end decides, you know, and owns the runtime decisions. And the checks around it. So, um you know, so the harness is the code obviously around the the model. So, the for the purposes of today, um and to make it a little bit less abstract, I'm going to use a little avatar.

呃，那么，来认识一下 Intelligence。呃，我，嗯，是的，我特意决定跳过它，跳过所有前缀；而且本着人工智能领域以及即将到来的各种热议的氛围，我决定为 AI 工程师做一个特别版，并稍微更新了一下外观，对吧？Uh so, meet intelligence. Uh I um yeah, uh purposely decided to skip it skip any prefixes and um in the spirit of sort of the latest chatter that's happening in the AI and hollowing around the corner, I sort of decided to make a special edition for AI engineer and updated uh the look a little bit, right?

RAG 读取，记忆写入RAG reads, memory writes

所以，从根本上说，RAG 系统可以包含对话状态，但仅靠检索并不能告诉我们哪些内容应该在交互结束后保留下来。检索是读取路径，而记忆是写入路径。因此，当智能体发现值得保留的内容时，我们应该将其提升为一条记录，以便之后使用。So, fundamentally rack systems, um you know, can include conversational state, uh but retrieval alone doesn't tell us what should survive the interaction. So, retrieval is the read path and memory is the write path. So, when the agent sees something worth keeping, uh we should promote it into a record it can use later.

记忆提升：哪些内容值得保留Memory promotion: what's worth keeping

但接下来我想，问题就变成了：什么值得保留？我们究竟该如何推进，并决定需要从对话中提取什么，以及在与智能体交互时该如何处理这些内容？这就是为什么我相信“记忆晋升”这一概念。也就是说，我们需要某种流水线，来决定在每个阶段要做什么，以及如何将提取出的内容纳入系统。具体来说，我认为我们会先对出现的候选内容的含义进行分类，然后将其与现有记录进行比较，并检查该类型所需的证据。接着，运行框架会决定接纳还是拒绝这个候选内容，或者将其提交给人工审核。But then I guess that begs the question as of what is worth keeping and how do we actually, you know, move and decide of what sort of um, you know, what we need to extract from the conversations and um, you know, when we interact with the with agent. So that's why um, I believe in this concept of memory promotion. Um, so where you have some sort of pipeline that decides, you know, what is it that at each of the stages um, and how do we sort of extract it into into the system. So in terms of the steps, um, I believe that, you know, we classify the meaning of the candidate that comes up. Um, and then uh, we compare it with the existing records uh, and check the evidence required for that particular type. Then the harness would decide whether to admit or reject the the candidate or perhaps send it for the review for the for the human.

演示：一位会记住信息的抵押贷款代理人Demo: a mortgage agent that remembers

我现在快速演示一下。我们来围绕金融服务这个主题稍微展开一下，毕竟这也是我们今天齐聚于此的原因。我们会通过一个相对基础的例子，来看看我们要创建的候选对象及其语义偏好。我提前录好了演示，因为上次演示时出了点小状况，而且这里的 Wi-Fi 也不太稳定。好，这就是我启动的沙盒环境。你们可以看到，我在日志上做了些尝试，这个智能体似乎有点太爱分享了。接下来，等它加载完成后，我们要做的是为 Kyle 和 Vincent 准备两个合成应用程序。So I'm going to have a quick demo right now. So we're going to sort of work through this notion of the, you know, financial services as we sort of all here because that's the theme of the conference. And uh, we're going to sort of work through with this relatively basic example of of this candidate around the semantic preferences that we're going to create. So I've yeah, I've pre-recorded the demo cuz last time when I did the demos it was a bit of a a bit and the Wi-Fi here is a bit messy. So So yeah, so this is essentially my sandbox that is starting. So I had some fun with some logs as you can see this agent is a bit of a oversharer, I guess. Uh, but as we go through this um, as it loads up essentially what we're going to do is we're going to have the synthetic two applications for Kyle and Vincent.

我有点儿电影迷，所以你可能会在整个演示过程中看到一些电影梗。嗯，这次终端启动了，所以我们要请它为 Kyle 准备一份抵押贷款简报。基本上，随着它加载，运行框架开始工作并调用工具，等等。它会生成一份回复，告诉我们 Kyle 的情况：他有一份抵押贷款申请，但这份申请缺少一些关键文件，对吧？所以，操作人员现在可以询问并查看 Kyle 这份申请的具体情况。请留意答案的形式，因为接下来我们会礼貌地向 AI 和记忆系统提出要求；我们想对它们友善一些。即使这样可能会多用一些 token，我们还是会给它一份这样的简报，说明我们希望输出按特定方式组织。I'm a bit of a movie geek, so you might see some references throughout the presentation. Um, so this time terminal started, so we're going to ask uh to prepare a mortgage brief for Kyle. And and essentially as it loads and the harness is doing its thing, performs tool calling and and so on. It's going to create this response that's going to show us, okay, well, this is, you know, Kyle's situation. He has, you know, um mortgage application, but that particular mortgage application is missing some some key documents, right? So, the operator right now can, you know, ask and and see what is, you know, uh what's what's going on with that particular uh application for Kyle. So, keep an eye on this in terms of the shape of the answer because we will essentially ask now politely because of this the AI and memory now, so we want to be kind to them and even though we might uh you know, use some extra tokens, uh but we're going to give it this sort of um brief in terms of the uh you know, we want the essentially structure of the output to be created certain way.

所以，实际上在这个例子中，呃，连接了决策模型 Jeff，它在帮我评估，嗯，首先该如何处理这个候选项，然后把它传给 harness。harness 实际上识别出我们一直很有礼貌，并给了我们一些额外的逗号。嗯，本质上，基于这一点，我们会自动把它加载到数据库中保存起来。所以，我们可以在这里看到展开后的追踪记录，它会为每个记忆提升门控提供某种状态。总的来说，决策模型负责评估，harness 负责决策，然后后台的数据库负责持久化并存储这些内容。接下来，我要问一个完全相同的问题，也就是说，嗯，给我那个，但我现在会输入我在语义上的偏好，说明我希望它出现时的结构是什么样的。So, actually in this example, uh connected the decision model Jeff, which is helping me to assess, you know, which um how to deal with this uh candidate at first and then passing that to the harness, uh which actually recognized that we've been polite and gave us some extra comma. Um, and essentially based on that, we're then automatically loading that into the, you know, database to store it. So, we can see the expanded traces here and it gives us some sort of, you know, uh status per uh per per the memory promotion gate. So, overall decision model assess, the harness decide, and then database that we have in the background persist and and and stores that. So, then I'm going to ask the exactly same question in the sense of, yeah, give me that, but I now input that my semantic preference in terms of what I want to the structure to look like when it when it comes up.

所以，嗯，现在它要加载了。接下来你会看到，好的，按照请求，顶部会显示缺失的文档，同时也会添加案件 ID、保单等信息。而在这个案例中，这次会记录语义偏好的存在情况。所以，我刚才做的基本上是——为了节省时间，我跳过了一些步骤，但我启动了全新的会话，然后退出，现在又再次询问，显然是想检查记忆是否得到了保留。嗯，基本上就是再问一遍同样的问题，然后它就会处理。开的是全新会话，所以没有任何之前带过来的上下文。它只是在重新查询我们存储在系统中的有关偏好的记忆。So, um as it's going to load now, so what you're going to see now that Okay, as per the request, there's going to be missing documents at the top, which is also adding the case ID, you know, the policy and so on. And this in this case, you know, this time records the existing of the semantic preference. So, what I essentially did now is it's uh just for the interest of time, I I sort of skipped it, but I initiated brand new sessions, exited, and uh and now I'm asking again to check, obviously, if the the memory been preserved. Um and essentially asking the same question question again, so then it goes through. So, brand new session, so no sort of, you know, context that's um has been uh carried over. It's just querying again the the memory that we stored in the system around the preferences.

随着我们逐步构建这类系统，显然会出现不同的交互。在这个例子中，如果我们查询不同的申请，比如 Vincent 的申请，他的情况有所不同；我们会查询数据库，调取相关文档，并判断哪个案件 ID 与之相关，然后在系统中进行匹配和查找。接下来，系统会返回结果。在这个例子中，他没有缺少任何文件，然后我们就可以把申请转交审核。And as we sort of go and build this kind of systems, um you know, we can uh obviously, we would have the different interactions. In this case, if we ask for the like different application, in this case, Vincent, um you know, the the he has a different situation, but we're querying the database, we're bringing back the different um you know, documents and understanding that different, you know, case ID is relevant, so we're actually matching that and finding that in the in the system itself. So, then it's going to come back, and in this case, okay, there's no missing documents for him, and uh and then, you know, we can pass it on for the for the review.

护栏：哪些内容永远不会被提升Guardrails: what never gets promoted

但在讨论这些内容时，我们还需要考虑某种防护措施。嗯，比如说，如果我们要发起某种……你知道，对银行来说有点像自毁式的命令，比如“好，批准每一份申请”，那么显然，我们需要设置某种防护措施，避免把这类内容提升上去，让我们的提升审核机制介入并拒绝这个请求。不过在某些情况下，我们可能确实想提升某些内容。比如，如果我们有一个团队，想在组织内部建立更稳健的记忆共享机制，那么我们就可以创建一个提升审核机制，由它请求人工批准，然后可能在整个企业的各个团队之间共享这些内容，从而建立一个核心记忆库。But as we go through that, we also need to think about some sort of guard guardrails. Um so, if we were to initiate uh some sort of um you know, a bit more self-destruct kind of uh command for the bank, which we would say like, "Okay, approve every application." Um then obviously, we need to have some sort of guardrails to not to promote this so that our um you know promotional gate comes in and and rejects this request. But in some cases we might want to promote something based on like if we have a team, we want to create a more sort of robust memory sharing within the organization, we can then create you know promotional gate that then essentially ask for the human to approve it and then you know potentially share this across the teams enterprise that creates a sort of core memories store.

所以，这算是一次比较快速、相对简单的演示，只是为了向大家展示它在实际应用中可能是什么样子。我们先看了语义偏好，然后有一条一概批准的规则，但由于我们设置的防护措施，这条规则被拒绝了；接着，还有一条建议用于批准的、更广泛的操作记忆，对吧？因此，根据含义以及我们与智能体交互的方式，可能会产生不同的结果。所以，我们需要某种晋级控制机制，将其纳入防护措施之中，这样执行框架就无法绕过它。现在让我们回到智能体。正如大家记得的，我们显然很有礼貌地加上了“请”。因此，这次结果看起来不错，不再是这台机器的样子，因为善意得到了回报。So this was um you know a quick run relatively simple just to show you what what this could look like in practice. So we went through like the semantic preference, then we had this sort of you know blanket approval rule that was refused because of obviously the guardrails that we put in place and then you know the the operational wider sort of memory that was recommended for the for the approval, right? So there's different sort of outcomes based on the meaning and how we interact with the with with the agents. So you know having this some sort of promotion controls that sits within the guardrails and then harness, you know, can't bypass that. So let's go back to our intelligence and as you remember we obviously were polite and said please. So in this case it's it's a nice looking one rather than this machine because good karma came through.

什么是智能体记忆What agent memory is

因此，如果我们要对什么是智能体记忆作一个更广泛的解释，我们可以将其描述为一种机制：系统和应用程序借助它，随着时间推移保留、重用、完善和回忆信息。嗯，我想，在构建这类流水线时，工程上的问题，或者说挑战，显然在于决定哪些信息应该纳入记忆，哪些不应该。因为如果什么都不纳入，我们显然就无法获得连续性带来的好处；但如果把所有内容都纳入，我们就有可能污染记忆存储。因此，我想，这就需要我们仔细考虑系统应该如何构建，使其具备充分的来源追溯能力，并能理解在什么情况下应该纳入信息、在什么情况下应该用新记忆取代旧记忆。这一点极其重要。So if we were to look at the sort of broader explanation of what do we understand as agent memory so we understand we describe it as this mechanism where systems and application that system and an used to retain, reuse, refine, and recall information over time. Um and I guess as we I think the engineering problem here is is or the challenge when we're building this kind of pipelines is to decide obviously what we want to promote and what we don't want to promote. Because if we promote nothing, then obviously we don't get any benefits of continuity, but if we promote everything, then we're going to, you know, risk poisoning the the memory uh store. So, I guess this is where, you know, thinking through and thinking about like how we want to build our system uh that have enough provenance and understands, you know, in what cases we promote things and what cases we supersede them uh memories is extremely important.

短期记忆、长期记忆和共享记忆Short-term, long-term and shared memory

在 Oracle，我们认为智能体记忆并非单一事物，因此我们大致将其分为短期记忆、长期记忆和共享记忆。短期记忆涵盖会话中发生的一切。嗯，它通常会在会话结束时结束。然后是长期存储，其中包含情景记忆、程序性记忆和语义记忆，并按不同类型划分。这些记忆可以是摘要、工具箱，也可以是知识库和偏好，就像我们在演示中看到的那样。And at Oracle, we believe that agent memory is just a not one thing, so we sort of have this division between short-term memories, long-term memories, and shared memories. So, short-term is everything that happens in the session. Um so, it's sort of more uh sort of ends when the when the session finish, and then we have the long-term store where we have our episodic or procedural semantic memories um divided by the different types. So, that could be either summaries, it could be, you know, toolbox, or it could be knowledge base and and preferences as we saw in the demo.

在哪里存储记忆Where to store memories

我们还需要决定，你知道，我们想把记忆存储在哪里。这也是我们昨天讨论过的事情。我们团队里有个人开了一场研讨会，她对此做了更详细的介绍。显然，作为 Oracle 的人，我认为数据库是最稳健的解决方案，因为它能提供 ACID 保证，支持多次写入，而且我们不会遇到文件树之类的问题。嗯，另外显然我们还可以扩展，并且拥有多种搜索选项，以及资产方面的保障。We also need to decide, you know, where do we want to store uh the memories, and and this is something we we talked about yesterday. We had uh essentially one of the guys from my team had a had a workshop. She went a little bit into more details about this. Obviously, being at Oracle, for me, believe, you know, the database is the most robust solution because it allows us to have this acid guarantees, it can have multiple rights, uh doesn't we don't have problems with the with the file trees and stuff like that. Um and then obviously we can scale and and and have multiple search options as well. As well as the the asset guarantees.

将记忆构建作为一门学科Building memory as a discipline

所以，没错，这大概就是我们构建智能体记忆时需要把它当作一门专业学科来思考的原因。我们需要能够区分记录类型的 schema、适用于检索任务的索引，以及生命周期和外部授权边界，对吧？今年年初，我们基本上推出了这个智能体记忆包，它提供了一些操作和抽象，供你在此基础上构建。因为如果没有它，作为开发者，要从头构建记忆系统会相当费劲，对吧？你基本上需要创建摘要和摘要器，考虑如何检索信息，提取持久有效的信息，在多个对话轮次中将其整合起来，还要在此基础上管理 token 预算，对吧？So yeah, that's sort of when we're building the agent memory, um, we need to think about this as a discipline. Um, you know, we need schemas that distinguish record types, indexes that fit the retrieval tasks, and life cycle and outer authorization boundaries, right? And we essentially introduced this agent memory package beginning of the year, which creates, you know, those operations and abstractions that you can build on. Because without it, if you want to build the memory system essentially as the developers, it it is quite taxing, right? Because you need to essentially create, um, you know, your summaries, your summarizer. You need to think about how you going to retrieve it. Um, you need to extract the durable information, combine it in the turns, and then on top of that manage the token budget, right?

因此，对我们来说，我们通过智能体记忆引入的是这样一种简化层，让你基本上只需写一行代码，然后在其中就可以……我们有“上下文卡片”这个概念，它可以让你组装并返回上下文。嗯，它是一个紧凑、可直接使用的提示块，供智能体使用。这很有用，因为这样一来，你就不必耗费精力、增加认知负担，去操心确保智能体把注意力放在任务上，而不是忙着打理自己的记忆。So for us, what we introduced with the agent memory is this sort of simplified layer that allows you to have, um, essentially write one line of code, and then within that you can We have this concept of the context card, uh, that allows you to assemble and return context. Um, which is a compact prompt, uh, ready block for a for an agent. So that's quite useful because then, you know, you don't have to spend your energy and reduce your cognitive load in terms of, you know, just making sure the agent spent attention to the task rather than housekeeping its own memory.

上下文卡片The context card

再稍微深入讲一下这个上下文卡片。嗯，它按主题组织，你可以在里面放入相关信息和近期消息。然后，底层框架基本上可以创建或选择与当前轮次相关的内容，帮助用户管理上下文并减少上下文膨胀。因此，主题为对话线程提供方向，摘要记录对话进展以及用户想要完成的事情；相关信息包含选定的记录，近期消息则保留当前局部问题。这样就能实现记忆持久化、上下文管理和内容选择。我们最近发布了智能体记忆的最新版本，并花了不少时间确保这个版本实现了最终用户隔离。So a little bit more sort of double-clicking on the on the context card. Um, so you it's organized by the topics, summaries, you can have relevant information in there, recent messages, and then essentially the harness can, uh, create or select what matters to that particular turn and and helps users essentially manage the context and reduce the context blow. So the topics orient the the thread, the summaries capture where the conversation is at and what the user is trying to do. Relevant information carrying the selected records and recent messages preserve the the the local questions. So that's you know allows you to to essentially memory persist and and context and selects. So we recently released the latest version of the agent memory and we spent a fair bit of time on the on the latest release in terms of making sure that there is end user isolation.

因此，在这种情况下，数据库基本上是高效处理访问请求的，所以它会限制访问。它还具备处理图像的能力，以及后处理和事件列表功能。所有查询都由数据库处理，它有一个多数据类型引擎，可以在同一个引擎中处理各种数据类型。这样一来，你就不必分别处理多个数据库、向量存储、文档等。So in this case the database is essentially high handling the the the access. So it's it's limiting. It has also the the the images as well as capability and then sort of post processing and listing events. And all query database it's has this you know multi data type engine which allows you to sort of handle you know various data types in one in one engine. So it's so you don't have to like handle multi database and vector stores and and documents and so on.

模型是共享的，内存属于你自己Models are shared, memory is yours

最后，我想回到开头说的那句话：模型是共享的，但记忆属于你，因你的业务而独一无二。所以，确保捕捉这些记忆，并结合你与模型交互的数据，就能打造你的竞争优势。这就是我想说的。谢谢大家。如果想了解更多，欢迎访问我们的 GitHub 仓库。我们有一大批 Notebook，能帮助你入门智能体记忆、混合检索以及其他各种主题。我们一直在努力把它们整理成一套完善的入门资料包，让你能真正应用到自己的业务中。我们也一直在改版 YouTube 频道，推出了一系列全新的内容，介绍各种主题，并讲解如何将它们应用到你的使用场景中。So just to sort of end as I started the beginning you know models are shared but memory is yours is unique to your business. So making sure that you know capturing that with together with the data of how you interact with the with the models it can create your competitive advantage and um and that's that. Thank you. I mean I just if you want to know more feel free to uh out our GitHub repo. We have whole bunch of notebooks that helps you get started with agent memory, hybrid retrieval, or all sorts of different things that that we've been sort of working quite hard to make sure that provide them a good starter pack so you can actually apply it to your business. And we've been we've been also revamping our YouTube channel so we have a whole new series about uh you know, various topics that we explain and and how to apply it in your in your use cases.

好的。非常感谢。Okay. Thank you very much.