Spotify：AI 广告平台的多智能体架构

转录稿Transcript

了解如何在 QCon AI 大会上实现 AI 的规模化运行。将 AI 从实验阶段推向生产环境，会给平台和团队带来更大压力。聆听资深同行讲解如何控制推理延迟、为智能体构建安全防护机制，以及标准化内部平台。此外，5 月 12 日前报名可享早鸟价。立即注册。"Learn how to run AI at scale at QCon AI conference. Moving AI from experiments into production increases pressure on platforms and teams. Hear senior peers explain how to control inference latency, build safe agent guardrails, and standardize internal platforms. Plus an early bird price until May 12th. Register now".

Pratik Rasam：你刚才听到的那则广告，包括文案、受众和定向，都是由我们在 Spotify Ads Manager 中自建的制作平台生成的。如果你从事广告行业，就会知道制作一则广告需要撰写文案、生成音乐。而现在，这些都不再需要了。广告主只需描述他们想要什么样的广告就行。这不是一场研究分享，而是介绍 Spotify Ads Manager 中实际投入生产并大规模运行的智能体。我还会谈谈我们在开发这个平台时犯过的错误。Pratik Rasam: The ad that you just heard, the script, the audience, the targeting, all of that was built by an in-house production platform that we have within Spotify Ads Manager. If you're into advertising, you'll know that creating an ad involves copywriting, music generation. All of that is now not needed. All we need is just the advertiser to describe what they want in an ad. This is not a research talk, but these are actual production agents at scale running at Spotify Ads Manager. I will also be talking about the mistakes that we did while developing this platform.

背景Background

我叫 Pratik，是 Spotify Ads 的高级工程师。今天我将介绍构建 Ads AI 的平台。Ads AI 本质上是一个面向 Spotify 广告解决方案的多智能体平台。先来看看我们在生成式 AI 领域的表现规模。自去年推出以来，目前超过 70% 的广告都在使用 AI 工具。我们已为 7,000 多家广告主生成了约 20,000 个创意素材。这些都是真实的广告活动、真实的资金投入和真实的受众。这次演讲讲的不只是研究成果，而是正在运行并创造价值的实际生产环境智能体。如今，我们的大部分创意生成核心流水线都由 Spotify Ads AI 驱动。在深入了解架构之前，先来看一个贯穿本次演讲、用于讲解架构设计不同阶段的示例。假设我们要为 QCon AI 制作一则广告。My name is Pratik. I'm a senior engineer at Spotify Ads. Today I will be going over the platform that builds Ads AI. Ads AI is basically a multi-agentic platform for Spotify's advertising solutions. Let me give you a scale at how we are performing within the GenAI space. Since we launched last year, more than 70% of ads currently are using AI tools. We have around 20,000 creatives that have been generated for over 7,000 advertisers. These are real ad campaigns, real money, and real audiences. This talk is not just research. These are actual production agents that are running and delivering value. Today, we have most of our creative generation primary pipelines being generated by Spotify Ads AI. Before we dive into the architecture, this example is something that we'll be using throughout the talk for different phases of the architecture design. Let's imagine that we are creating an ad for QCon AI.

品牌本身是 QCon。我们想传达给听众的信息是：如果你想为资深工程师获取生产级 AI 实战指南，以便大规模交付 AI 解决方案，那么 QCon AI 就是不容错过的会议。我们的目标受众是资深工程师、架构师和工程领导者。我们希望覆盖美国主要科技中心。我们的 CTA（行动号召）是“立即注册”。The brand itself is QCon. The message that we want to have our listeners is, let's say if you want to have production AI playbooks for senior engineers, for shipping AI solutions at scale, QCon AI is the conference to be. Our target audience is senior engineers, architects, engineering leaders. We want to target major U.S. tech hubs. Our CTA, which is the call to action, is, Register Now.

“广告 AI”究竟意味着什么What 'Ads AI' Actually Means

我想确保我们达成共识。当我说 Ads AI 实际上意味着什么时，可以想象广告主说：“我想面向美国各大科技中心的工程领导者投放广告。”这是自然语言。我们要做的是将这种自然语言提取为具体的受众意图。在这里，我们要将受众确定为科技行业，地理定位则设为美国。目标是提升知名度。我们通过 Vertex AI 和内部 GCP 平台使用这个 LLM 网关。提取出的意图随后会被提供给我们的多智能体编排层，该层运行在 Google ADK 上。我们使用的是 Google ADK Java。我们有多个并行智能体，分别负责不同的职责。对于刚才提到的广告，我们有一个广告脚本生成智能体。我们还希望确保广告符合我们的政策，因此设有一个广告护栏智能体。我们还有一个目前正在试点运行的受众推荐智能体。这些智能体的综合输出最终会生成两个独立对象：推荐的受众和创意内容。I want to make sure that we are on the same page. When I say what Ads AI actually means, imagine the advertiser says, I want to run an ad for engineering leaders across the U.S. tech hubs. This is natural language. What we do is extract this natural language into specific intent for audience. Over here, we are talking about the audience to be resolved to technology, and the geo-targeting should be for U.S. The goal is for awareness. We are using this LLM gateway using Vertex AI and our internal GCP platform. When it comes to the extracted intent, it is then provided to our multi-agent orchestration layer, which runs on Google ADK. We are using Google ADK Java. We have parallel agents that are composed per responsibility. For the ad that you just heard, we have an Ad Script Generation Agent. We also want to make sure that the ad conforms to our policies, so we have an Ad Guardrail Agent. We have an Audience Recommendation Agent that we have currently running in pilot. A combined output of these agents eventually results in two discrete objects, which we have the audience recommended and the creative that is created.

一个代理、一个软件包、一个负责人One Agent, One Package, One Owner

在深入探讨这些模式之前，我想先向大家展示我们最初是如何构建这一架构的。我说的 agent，指的是一个 agent、一个 package 和一个 owner。我们明确强制执行了这一要求，因为当 agent 以独立 package 的形式组合起来时，每个 package 本身又会围绕所有权、职责、监控和提示词等不同方面进行拆分。你们可以看到，在 agent ad script 文件夹中，我们有 Bazel package，用于管理依赖关系，规定哪些人可以访问这个 agent。我们还有 lib-info。lib-info 用于确定所有权，也就是哪个团队负责这个 agent。我们还有一个内部组件，用于处理 PR 审核。比如，如果你修改了这个 agent 中的某个属性，相关团队就会介入处理所有权变更。Before diving into the patterns, what I wanted to show you guys was how we built this architecture in the first place. When I talk about agent, I'm talking about one agent, one package, and one owner. We have explicitly enforced this requirement because when you have the agents composed as individual packages, that package itself is composed into different aspects around the ownership, the responsibilities, the monitoring, and the prompt itself. If you see in the agent ad script folder, we have the Bazel package, which involves the dependency management of who is allowed to access this agent. We have the lib-info. The lib-info is something that determines ownership with regards to what team is owning that agent. We have an internal component that resolves the PR reviews. Let's say if you are changing a particular attribute within this particular agent, that is the ownership change that the team will go to.

Monitoring-info 本质上是一个 YAML 文件，其中包含有关仪表板和 PagerDuty 告警的信息。这些内容都由我们的平台提供，因此在为代理指令编写哪怕一行代码之前，指标和告警就已经全部就绪。AgentFactory、Adapter、Service 和 AgentModule 是依赖注入脚手架，为代理本身提供一套完整的封装。请注意，代理的 LLM 配置以 front matter 的形式存储。这让架构变得更有意思了。当我说不同团队负责不同代理时，指的是我们在水平层面强制执行这种 Bazel 可见性。所谓水平层面，就是说，广告脚本生成代理不能使用受众解析器，除非受众解析器允许它被导入。这种约束是在编译时强制执行的。不作评论。Monitoring-info is basically a YAML file that has information about your dashboards and your PagerDuty alerting. You have all of this provided by our platform where before even writing a single line of code for the agent instructions, you have metrics, you have alerting, all in place. AgentFactory, Adapter, Service, and AgentModule are dependency injection scaffolding that provide a holistic package for the agent itself. Note that the LLM configuration for the agent is stored as a front matter. This is what makes the architecture a little more interesting. When I say different teams are owning different agents, we are enforcing this Bazel visibility at the horizontal layer. When I say horizontal layer, I'm saying that the ad script generation agent will not be allowed to use the audience resolver unless and until the audience resolver allows it to be imported. This enforcement is at the compile time layer. No comments.

如果没有专门导入那个特定的 agent，基本上就会导致构建失败。除了由团队负责的组件之外，还有共享平台。共享平台提供指标生成、可追溯性等功能，而我们的大部分工具使用都围绕 Spotify Ads 的开发者 API 展开，该 API 会调用 Ads API。我们还有一个名为 kutest 的额外审核层，用于管理护栏策略。You basically get a build failure when you are not importing that particular agent specifically. Other than the team-owned components, you have the shared platform. The shared platform provides capabilities such as metrics generation, traceability, and most of our tool usage evolves around our developer API for Spotify Ads, which calls Ads API. We also have an additional moderation layer called kutest, which provides guardrail policy management.

智能体架构栈Agent Architecture Stack

从团队层面缩小视角来看，整体技术栈如下：我们有一个 gRPC 服务和多个智能体。运行时上下文由 Google ADK 控制。A 团队拥有自己的智能体和工具，B 团队拥有自己的智能体和工具，C 团队也可以拥有自己的智能体和工具。最终，所有这些智能体都可以在共享上下文和共享的 ADK 运行时上运行。工具调用的可追溯性、工具上下文以及运行时循环都由平台管理。 从架构角度来看，整个流程如下：客户端应用向我们的 gRPC 服务发起服务调用，gRPC 服务也负责管理会话。每当客户端发起调用时，都需要负责与智能体创建会话，随后请求最终会进入 AI 智能体层。When you zoom out at the team level, this is what the overall tech stack looks like. We have one gRPC service. We have multiple agents. We have the runtime context, which is controlled by Google ADK. Team A owns its own agent and its own tools. Team B owns its own agents and its own tools. Team C can own its own agents and own tools. All of these agents can eventually run on a shared context and a shared runtime ADK. The tool traceability, the tool context, and the runtime loop is all managed by the platform. From an architecture point of view, here's how the flow looks like. You have the client applications making the service calls to our gRPC services, which also manage the session management. Whenever a client is making a call, it is responsible for creating a session with the agent, which eventually goes to the AI agents layer.

AI 智能体层是各个智能体实际驻留的地方。我们在单独的层中实施安全机制和编排，而不将它们放在智能体内部。这是因为我们希望在整个智能体体验中采用统一的编排和统一的防护机制。最后一层是模型和工具平台，基本上包括 LLM 运行时、插件，以及为所有智能体运行的可观测性配置。在本次演讲中，我们介绍了这一平台的三个特性：安全内容、模块化智能体平台和灵活集成。这是我们希望在整个智能体界面中贯彻的三大支柱。The AI agents layer is where individual agents actually reside. We have enforced safety and orchestration at a separate layer and not within the agent. This is because we want to have common orchestration and common guardrails to be consistent throughout the agentic experience. The last level is the model and the tool platform, which basically is the LLM runtime, the plugins and the observability configurations that are running for all of the agents. Throughout this talk, we have three attributes of this particular platform, safe content, modular agent platform, and flexible integration. These are three pillars that we want to have throughout the agentic interfaces.

论点与播放列表Thesis, and Playlist

这或许是整个演示上半场最重要的一页。在处理多智能体架构时，你总会问一个问题：什么归谁负责？智能体负责情感倾向、LLM 判断以及推理部分。在我们的示例中，用例包括受众意图。我们想了解广告主说“我想为我们的运动鞋广告定向触达 Z 世代”时，究竟是什么意思。我们希望根据从这些信号中获取的所有信息来生成广告简报。此外，我们还想识别敏感话题。比如，如果某个广告描述不符合 Spotify 的广告政策，我们确实希望阻止生成该广告。至于数据，我们绝不希望 LLM 猜测地理兴趣是什么。在广告领域，我们有特定的兴趣细分人群，也有特定的 DMA 定向投放。所有这些数据实际上都来自我们的 API 端点。This is perhaps the most important slide of the first half of the presentation. When you are dealing with multi-agent architecture, the question you always ask is, who owns what? Agents are owning the sentiment, the LLM judgment, the reasoning aspect. In our example, the use cases are like audience intent. We want to understand what the advertiser means when they say I want to target Gen Zs for our sneaker ads. We want to create the ad brief from all the signals that we get from these. Also, we want to identify sensitive topics. Let's say if a particular ad description is not conforming to Spotify's ad policies, we do want to prevent that ad to be generated. When it comes to data, we really don't want the LLMs to guess what the geo-interest would be. Within advertising, we have specific interest segments, we have specific DMA targeting. All of this data actually comes from our API endpoints.

Spotify 拥有公开的 Web API，我们的工具目前正在调用这些 API。所有这些信息依据都建立在工具层。第三个方面是应用程序代码本身。当你问我，是否所有事情都应该由智能体来控制？我的回答是否定的。这是因为凡是能够通过确定性方式编写代码实现的，都应该始终以确定性方式实现。在这场演讲中，我会不断举例说明，为了实现我们的目标，我们必须设置一些防护措施和确定性的门控机制。Spotify has publicly owned web APIs that our tools are today accessing. All of this grounding happens at the tool layer. The third aspect is the application code itself. When you ask me, should everything be controlled by agents? My answer would be no. This is because whatever can be deterministically coded should always be deterministically coded. I'll be providing examples throughout this presentation where we had to establish some guardrailing and deterministic gating in order to achieve the goals that we wanted to have.

这就是今天的议程。我想先从为什么要采用多智能体架构谈起。我们不应该只是因为智能体很酷就使用它们。采用多智能体的动机在于控制影响范围，并为每个智能体划分职责。接下来，我们会讨论如何划定智能体的边界。这或许是最长的一场讨论，因为它也是最难展开的。我们必须弄清楚，智能体应该负责什么，以及哪些部分应该在代码中通过确定性逻辑或工具来实现。 第三个主题是工具设计。工具设计本质上就是提示工程。我这么说，是因为定义工具时，我们也会定义它的架构。架构中包含对工具功能的描述。这段描述不只是写给我们这类开发者看的文档，它也会传递给 LLM，帮助 LLM 理解工具的用途，以及该如何准确调用它。 之后，我们会谈到如何通过评估和追踪来实现可靠性。要进行评估，我们首先需要追踪，而这正是我们会重点强调的内容。最后，我会谈谈哪些地方我们会采取不同的做法，其中会提到我们最初探索智能体时犯下的一些错误。This is what today's playlist looks like. I want to start with why multi-agent at all. We don't want to just have agents because they are cool. The motivation behind having multi-agent is controlling the blast radius, having responsibilities shifted for each and every individual agent. We are then proceeding towards where to draw the agentic boundary. This is perhaps the longest discussion because it's also the most difficult to have. It is very important to understand where the boundary lies between what the agent should be doing and what should be deterministic in our code versus tools. The third is tool design. Tool design is basically prompt engineering. I say that because when we define a tool, we also define its schema. The schema has description about what the tool is doing. This description is not just documentation for developers like us, but it's also something that is going to the LLM itself and helping the LLM understand what and how to exactly call the tool. Then we proceed towards having talked about reliability through eval and tracing. In order to achieve evaluation, we need to have tracing first, and that's what we'll be stressing upon. Last but not least, I'll be talking about what we would do differently. This highlights some of the mistakes that we had done initially in our agentic journey.

为什么需要多智能体？Why Multi-Agent at All?

让我们把时间倒回18个月前，想想当时我们对大语言模型（LLM）会有什么期待。比如，去年要从某个特定的 LLM 模型解析 JSON，并不是那么简单。它返回的可能是类似 JSON 的内容，也可能是带有三个反引号的 Markdown。随着时间推移，模型变得越来越好。如今我们使用的模型已经能够很好地处理 JSON 解析、模式检查和业务验证。第三点，业务逻辑验证非常重要，因为无论某个模型返回什么值，我们现在都可以添加额外的确定性检查，判断这些值是否合理。后面的幻灯片中我会进一步讨论这一点。再把时间倒回一年以前，如果你要实现智能体的并行执行，并自定义会话管理，很可能就得自己动手编写相关代码。Let's rewind 18 months back in time and think about the things that we would expect from an LLM. If we see, like last year, parsing JSON from a particular LLM model was not that straightforward. It would have been a JSON-ish response or it would have been like a Markdown with those three codes. Over time, models have gotten better. The JSON parsing, the schema checking, and business validation is now very much adopted by the model that we are using today. The third, the validation of the business logic is very important because whatever values are returned by a particular model, we can now add additional deterministic checks, whether those make sense or not. This is something which I will be talking more about in the later slides. Rewind back again to one year. If you had to write a parallel execution of agent with custom session management, you would probably have to write something by yourself.

如今，像 LangChain4j、Google Cloud ADK 或 CrewAI 这样的框架，都为你提供了这些基础构件。过去的问题是：我们该如何设计它？现在的问题则是：我该怎么实现它？我说运维变得可观测了，指的是供给侧发生了变化。可观测性方面的变化在于，OTel GenAI conventions 为 GenAI 工作流确立了一些标准化的可观测性约定，从而支持采用通用约定进行追踪。过去，团队通常会编写自己的注解和追踪机制。如今，这些语义约定让 GenAI 工作流能够实现标准化的可观测性。有了这些标准化的可观测性追踪，你现在可以追踪并重放评判模型。重放生产环境中的追踪数据，使我们能够实际测试生产环境中正在发生的情况，而不必创建合成测试。最终，这能避免系统陷入黑箱状态，让我们不至于对智能体系统中究竟发生了什么一无所知。Today, frameworks like LangChain4j, Google Cloud ADK, or CrewAI, these are frameworks that give you those primitives. The question used to be, how do we design it? Now, what should I do to implement it? When I say operations got observable, it's a supply side change. What changed in observability is the OTel GenAI conventions established some standard observability conventions for GenAI workflows. These have enabled to have generic conventions for tracing. Earlier, what used to happen was teams used to write their own annotations, teams used to write their own tracing. Now these semantic conventions allow standardized observability for GenAI workflows. With these standardized observability traces, you can now trace and replay the judge models. Replaying production traces enables us to actually test what is going on in production and we do not have to create synthetic tests. This eventually prevents us from having that black box behavior where we don't really have any idea what is going on in our agentic system.

第四点很有意思，它属于需求侧。前三项举措更多着眼于供给侧，也就是技术得到了改进。第四项则是需求侧提高了自身的要求。传统软件开发中一直存在多步骤管道。变化在于，如今我们会在每个步骤中加入 LLM 交互，而这些交互可以涉及多入口产品。我所说的多入口产品，是指广告主可以在移动应用上开始创建广告系列，然后在广告管理工具或 Ads API 中继续创建。每个入口都可能有不同的意图和上下文。只有通过多智能体流程，才能将这些上下文串联起来。最后一点是闭环衡量。我们在利用 GenAI 创建广告的同时，也有能力衡量这些广告的表现。当这些表现数据回流到系统中，我们就能知道哪些做法有效，哪些无效。The fourth one is interesting. It's on the demand side. The first three initiatives were more on the supply side where the technology got better. The fourth one is where the demand side increased its requirements. The multi-step pipelines have always been there in the traditional software development. What changed was, within every step, we now involve LLM interactions which can have multi-entry products. When I say multi-entry products, an advertiser can start their campaign creation on their mobile app and then continue the creation on ads manager or Ads API. Each of these entry points can have different intent and different context. Stitching this context together is only possible when it comes to a multi-agentic flow. The last one is the closed-loop measurement. When we have ads created by GenAI, we also have the ability to measure the performance of those ads. When this performance is funneled back into the system, we do know what works and what does not work.

这种闭环衡量被视为广告活动整体成功的一个因素。多智能体一直都存在，只是这四项功能——三个供应侧功能和一个需求侧功能——随着时间推移变得更加完善。我们有三个供应侧功能和一个需求侧功能。直到 2026 年初，多智能体还只是一个研究演示；如今，它已经成为一种架构。That closed-loop measurement is something that is attributed to an overall success of an ad campaign. Multi-agents were always there. It's just that these four, three supply side and one demand side features got more better over time. We had three supply side and one demand side. Until early 2026, multi-agent was a research demo, now it is an architecture.

我们来谈谈广告系列创建流程是什么样的。每当广告主登录 Spotify Ads Manager 时，他们都会设定自己的目标。比如，我的广告系列目标是什么？我的目的是什么？我的 KPI 是什么？衡量成功的指标有哪些？接下来是定向设置：我想触达哪些人群？作为广告主，当我指定想要触达的人群时，也就确定了广告的展示位置。有哪些兴趣人群细分？比如说，我想针对参加 QCon 的科技爱好者投放广告。此外，我还想添加人口统计筛选条件。接下来是预算和排期设置：我想花多少钱、投放日期是什么时候，以及采用哪些投放节奏规则。然后是上传创意素材，也就是你们在第一页听到的内容，广告主会提供创意素材的相关信息。这里的问题显而易见：到了创意素材这一环节，整体上下文就丢失了，因为每个步骤都在各自的流程中彼此孤立，无法为广告系列和创意素材的创建提供整体信号。Let's just talk about how a campaign creation process looks like. Whenever an advertiser logs into the Spotify Ads Manager, they are providing their objectives. Like, what is my objective for my ad campaign? What is my goal? What is my KPI? What are the success metrics? This is followed by the targeting. Whom do I want to target? When, as an advertiser, I'm specifying whom I want to target, I'm specifying where I want my ad to display. What are the interest segments? Let's say I want to target technology enthusiast for QCon. Also, I want to add demographic filters. I then proceed towards the budget schedule. How much I want to spend, the flight dates, and the pacing rules. Then comes the creative uploading, which is what you guys heard in the first slide, where the advertiser provides details about what they are providing as their creative. The problem is clearly visible over here, where at the creative level, the context really dies here, because each step is isolated in its own resolve, but it does not provide the holistic signal for creating and creative.

我来简单演示一下今天在 Ads Manager 中的操作流程。我们创建了一个 QCon Boston 大会广告系列，广告主可以设置年龄定向，详细定向设置为科技行业，语言则选择所有语言。我希望这个广告系列覆盖所有设备。这是一个音频广告系列。我希望针对展示量进行优化。假设我希望这个广告系列的每日预算为 15 美元。现在进入下一步，这里需要填写广告详情：广告主名称是什么？标语是什么？我还应该上传图片，但我还没有上传。接下来我们会重点介绍本次演示中的音频生成部分。你在第一页听到的广告就是这样制作的。广告主提供主要信息和行动号召。这个行动号召与我们在前一页中提供的内容类似。Let me just walk over how this would look with ads manager today. We have this QCon Boston conference campaign created, where the advertiser provides their age targeting, the detail targeting has technology industry, and the languages for all the languages. I want to have this campaign for all the devices. This is an audio campaign. I want to optimize for impressions. I want to have this campaign for a daily budget of $15, let's say. Now I go to the next step. This is where I give my ad details. What is my advertiser name? What is my tagline? I should also be uploading images, which I have not. We are going to focus on the audio generation aspect of this presentation. This is exactly how the ad was created that you heard in the first slide. The advertiser provides the main message, the call to action. The call to action is something which is similar to what we provided in the slide before.

当广告主选择“生成”时，就能看到生成的脚本。我已经有一个生成好的脚本，但我们允许广告主多次生成，这样他们就可以为自己的营销活动选择想要使用的脚本。正如你所看到的，在广告主创建广告营销活动的过程中，有一些信号与广告创作密切相关。如今，广告主在提供广告脚本创作所需的输入信息时，会提供品牌、信息、目标和广告类别。这些是用于创建营销活动的五个叶节点信号。相比之下，你可以看到右侧的可组合系统不仅可以使用这五个信号，还可以使用已解析的受众、区域设置、营销活动的过往表现、地理位置与年龄、性别组合、广告类别以及投放目标组。所有这些都能为创作更好的广告提供额外的背景信息和资料。这种能力的解锁，正是多智能体系统所带来的。When the advertiser selects generate, they can see the script that is generated. I already have one script that is generated, but what we do allow for advertisers is to have multiple generations so that they can choose which script they want for their campaign. As you saw, while the advertiser was performing the task of creating the ad campaign, there were certain signals that were pretty much relevant in the ad creation. Today, when the advertiser is providing the inputs for creating the ad script, they are providing the brand, the message, the goal, and the ad category. These are five leaf node signals that are used for creating the campaign. On the contrary, on the right side, you can see a composable system where not just these five signals, but you can also use the resolved audience, the locale, past performance of the campaign, geo-age, gender combination, ad category, delivery-goal group. All of this can definitely provide additional context and additional information for creating a better ad. This unlock is definitely something that multi-agent systems promote.

在哪里划定智能体的边界Where to Draw the Agent Boundary

我们先从核心层面做个比较。在函数式风格中，广告主执行提供品牌信号这一任务时，会提供素材，并获取品牌语调和行动号召。在生成脚本时，他们会提供品牌名称和 CTA，正如你在之前的演示中看到的那样。而在这里，实际上并没有品牌语调。同样，在进行推荐定向时，你会提供品牌名称和目标，但这里没有品牌语调和内容主题。 采用智能体技术风格的界面时，各个流程会提取各自的信号和会话上下文。完成推荐定向后，所有智能体只需使用同一个共享会话状态，上下文会在每个步骤中不断累积。流程结束时，智能体便能获取品牌语调、行动号召、主题等信息，让广告创作更加全面。Let's just do a comparison at a core level. At a function style, when the advertiser is performing the task of providing the brand signals, they are providing the asset and they are getting the brand tone and the call to action. When they are doing the generate script, they are providing the brand name and the CTA, as you saw in the previous demo. Over here, you don't really have the brand tone. Similarly, when you're doing the recommend targeting, you're providing the brand name and the objective, but the brand tone and content themes, they are not present. When we have an agent tech-style interface, you have the processes extracting their own signals and their own session context. After recommended targeting, all the agents have to do is just use the same shared session state, where the context accumulates across every step. At the end of the process, you have the agent getting a handle to the brand tone, the call to action, the themes, everything makes the ad creation more holistic.

这就是我们最终希望实现的那张幻灯片。如今，我们有了广告活动上下文，其中的品牌语调和目标由广告主提供。我们也有创意产出，包括通过 UI 界面展示的脚本、视觉标语和格式。广告活动提交后，就会开始实时投放。与此同时，我们会获取定向信息、地理属性（例如广告表现良好的地区），以及各项效果信号，包括点击率、完成率、转化率等。设想一下，如果闭环反馈能真正作为输入进入我们的漏斗，我们就可以发现，带有偏置提示的对话脚本和受众选择实际上表现更好。我们可以调整定向策略，也可以调整模型，让反馈回系统的漏斗能够提供关于哪些做法有效、哪些无效的有价值见解。多智能体技术无疑能让我们实现这种双向流动。This is the slide where we eventually want to go. Today we have campaign context, where the brand tone and objective are fetched from the advertiser. We also have creative outputs where scripts, visual taglines, and formats are provided from the UI display. When the campaign is submitted, it is running live. At that same time, we get the targeting information, the geo attributes, where the ad performed well, and we have the performance signals. We have the click-through rate, the completion, conversion rate, all of these signals. Imagine the closed loop actually performs as an input to our funnel, where we can say that conversation scripts with bias prompts and audience selection are actually performing better. We can tune our targeting and we can tune our model in such a way that the funnel that goes back to the system is actually giving meaningful insights about what works and what does not work. This bi-directional flow is something that multi-agent is definitely enabling us to do.

在把事情做对之前，先让我从我们做错的地方说起。和所有热衷于尝试 AI 的人一样，我们一开始只用了一个智能体，还给它下达了一条庞大的指令。这个智能体负责生成脚本、设置防护措施、确定受众、执行搜索、进行验证、处理 JSON schema，以及把控语气和地区设置。它拥有所有工具，一切都塞在一个大块里。想象一下，一个有 1,000 行代码的函数。它确实能用，但同时也很脆弱。随着指令不断变化，这个智能体的行为开始出现一定程度的幻觉。我们没有采用隔离式的方法来识别实际出了什么问题，而问题确实正在发生。这让我们意识到，如果想让专门的智能体各自承担专门的职责，就需要在 POC 之后的初期阶段把这些工作拆分开来。这就是我们最终采用的拆分方案。基本上，我们有三个智能体。Before we got things right, let me begin with what we got wrong. As all enthusiastic AI experimenters, we started off with one single agent and a mammoth of an instruction. This agent was responsible for script generation, guardrails, audience resolution, search behavior, validation, JSON schema, and the tone and locale. It had all the tools. It had everything in one large block. Imagine something like a function with 1,000 lines of code. It worked, but it was brittle at the same time. As instructions changed, the behavior of this agent got somewhat hallucinated. We did not have an isolated approach towards identifying what is actually wrong, which is happening. This led us to determine that if you want to have dedicated responsibility for dedicated agents, this is something that we need to split as a part of our initial post POC. This is the decomposition we landed on. We are basically having three agents.

我们有一个目前正在试点的受众解析器，它负责确定地理定向、人群特征、兴趣和排除条件。基本上，如果广告主简报要求面向纳什维尔地区、18 至 34 岁、对健身感兴趣的女性，这个代理就会确定兴趣和地理定向。与此同时，我们还有广告脚本生成流程，由两个代理并行运行：一个是广告脚本安全护栏代理，负责执行特定的品牌安全检查；另一个是广告脚本生成代理。这里有一点需要注意：代理的拆分也取决于各团队的职责归属。在广告业务中，广告创建和提交流程的不同领域由不同团队负责。我们有受众管理团队，也有创意管理团队。创意管理团队负责广告脚本生成代理，受众定向团队则负责受众解析器。We have the audience resolver, which is in pilot right now, but what it's responsible for is resolve the geo-targeting, resolve demographics, resolve interest and exclusions. Basically, if an advertiser brief has women 18 to 34 in Nashville into fitness, this agent is resolving the interest and the geo-targeting. At the same time, we have the ad script generation, which is running two agents in parallel. We have the ad script guardrail agent, which does specific brand safety checks. We have the ad script generation agent. One key thing to note here, that this agent bifurcation is also dependent on the ownership teams. Within ads, we have different teams for different domain aspects of the ad creation and submission process. We have a team for audience management. We have a team for creative management. The creative management team owns the creative ad script generation agent and the audience targeting team owns the audience resolver agent.

这是一个值得遵循的经验法则。我们总会被问到一个问题：什么时候该把某件事交给智能体处理？什么时候又该写代码？在我看来，模型正在变得越来越好，但它们仍然具有非确定性。如果你行为描述的措辞语义会发生变化，那显然应该由智能体或 LLM 来处理。像资深工程师、技术领导者这类事情，都可以交给智能体处理。相反，如果你的规则、目录或数据会发生变化，那就应该用代码来处理。一个很好的类比是：如果某件事可以通过代码计算，并且可以用单元测试来测试，那就一定要写成代码。我们是通过一种很有意思的方式学到这一点的。在广告管理平台上进行受众定向时，我们目前支持以下年龄段。This is a good heuristic to follow. A question that we always get asked is, when do you use something as an agent? When do you actually have code? If you ask me, models are getting better, but they are still non-deterministic. If the semantic of the wording of your behavior is changing, that is definitely the agent will handle or the LLM would handle. Things like senior engineers, things like technology leaders, all of this is something the agent can handle. On the contrary, if your rules change or if your catalog changes, or if your data change, that is something which is code. One good analogy is, if something that can be computed in code is computational and can be tested using unit tests, definitely have it as code. We learned it in a very interesting way. When we have targeting of audiences, on the ads manager platform, we currently support these age brackets.

我们遇到过这样一个案例：输入条件之一是纳什维尔地区 21 岁及以上的女性。在这个案例中，智能体直接输出了 21 到 99 岁。我希望有人能活到 99 岁，但遗憾的是，我们的下游系统拒绝了这个结果，因为它不属于年龄区间选项。所需的修复方式相当明显：我们必须明确列出七个年龄段，并排除某个不受支持的特定年龄。 第二个错误更有意思。如果提示要求的是 13 到 17 岁青少年的父母，智能体却推荐了 13 到 17 岁的青少年，而不是他们的父母。这里需要修复的不是人口统计条件，而是角色画像。这些都是你在测试智能体时可能会遇到的情况：即使使用最新的大语言模型，也可能发现一些令人意外的细微行为，导致智能体出错。We had a case where one of the inputs was women 21-plus in Nashville. In this case, the agent just emitted 21 to 99. I hope someone lives 99 years, but this was unfortunately rejected by our downstream systems because it was not part of the age bucket. The fixes that were required were pretty much obvious. We had to enumerate the seven age brackets explicitly that rejected a specific non-supporting age. The second bug was even more interesting. If a prompt says parents of teen 13 to 17, the agent actually recommended 13 to 17 instead of the parents. The fix that was required here was not the demographic, but actually the persona. These are the cases that you will find as you are testing your agents that even if you're using the latest LLM, there will be cases where you will find niche behaviors that you might find surprising that the agent gets wrong.

另一个有趣的案例是地理定向。用户可能只会说：“我想定向美国南部和中西部。”这些都是非常笼统的说法。美国南部指的是 13 个州，中西部也包括 12 个州。如果没有把国家代码作为附加属性传给工具，Georgia 就可能被解析为国家，而不是州。我们从中学到的是，如果在工具本身中传入这类额外上下文作为护栏，就能为代理提供更多上下文，帮助它确定应当采用哪种具体定向。我们最终采用的另一项优化，是审视哪些工作可以在调用 LLM 之前完成。在为广告脚本生成采用多代理模式之前，我们会分别调用 LLM。问题在于，无论是否进行生成，护栏都会运行。Another case which is interesting is the geo-targeting. A user can just say, I want to target the South and the Midwest. These are very generic terms. The South in the U.S. means 13 states. The Midwest means also 12 states. If you're not providing the country code as an additional attribute to your tool, Georgia can be resolved as a country and not the state. This is something that we learned that when you have that additional context being passed in the tool itself as a guardrail, that gives more context to the agent to resolve what specific targeting is to be provided. One more optimization that we eventually landed upon was we looked into anything that can be done before calling LLMs. Before we actually had like a multi-agent pattern for our ad script generation, we were having individual LLM calls. The problem was the guardrail was actually running irrespective of the generation.

每当护栏给出错误结果时，都会浪费我们的 token，因为没有为用户生成任何输出。我们有一个内部审核分类器，可以实际完成分类。我们的做法是先调用这个内部审核分类器。如果失败，就提前返回。我们会先进行把关检查，之后才调用 LLM。当你发现组织内部可能已有无需调用 LLM 就能给出结果的解决方案时，就使用这些方案，因为从更大规模来看，否则会浪费大量 token。Whenever the guardrail was giving us a false result, that was wasting token usage for us because there was no output being written for the user. We have an internal moderation classifier that actually gives us classification. What we did was we called this internal moderation classifier first. If that fails, we return early. We are doing a gatekeeping check first and only then calling the LLMs. Whenever you find cases where you might have existing solutions within your organization that can give you the output without calling an LLM, do that, because on the larger scale, you will be wasting a lot of tokens.

原型化的模式Prototyped Patterns

在正式深入介绍智能体的关键要点之前，我想先谈谈我们已经原型化的模式。我们经常被问到一个问题：如果我想编写智能体，应该怎么写？或者说，我应该使用哪些模式？在我们的项目中，Google ADK 自带一个开发者界面。我们为工程师搭建了一些现有模式，今天会和大家分享。这些都是基础组件，可以根据需求进行组合和修改，从而实现不同的行为。我们有基础路由器、条件路由器，还有非常常见的顺序流水线。扇出、扇入模式用于并行执行。最后一种是生成器—评审器循环，适用于持续优化智能体的输出，直到得到最合适的结果。Before we actually dive into the key takeaways for agents, I wanted to talk about the patterns that we have prototyped. This is a very common question that we get asked that, if I want to write agents, how should I write them? Or, what are the patterns that I should be using? Within our project, what we have done is Google ADK comes with a dev UI. We have scaffolded certain patterns that we currently have for our engineers, which I'm going to share with you guys today. These are primitives which are composed and modified to give different behavior based on your requirements. We have the basic router. We have a conditional router. We have the sequential pipeline, which is a very common workflow that we have. Fan-out, fan-in is where we want to have parallel execution. The last, which is the generator critic loop, where you want to refine an agent output continuously till you find the best output suitable.

从基础智能体和工具开始。这是你会看到的最简单的智能体调用：只有一个智能体，并且 LLM 可以访问一组特定工具。它没有编排开销，易于进行单元测试和推理。需要注意的是提示词负载。你可能会发现，调用工具 3 到 10 次都还可以，但达到 8 到 10 次时，LLM 可能会产生幻觉。只要有可能，如果你觉得同时调用的工具太多，就考虑将智能体拆分成多个智能体。接下来是顺序模式。这是我们大多数工作流中非常常见的一种模式。顺序模式本质上是一系列智能体运行：一个智能体接收另一个智能体的输出，并将其作为输出键。从核心层面来看，这些步骤之间没有任何衔接代码。Starting with the basic agent and tools. This is the simplest agentic call that you'll see, where you have a single agent and you have specific tool sets that the LLM has access to. It has zero orchestration overhead, easy to unit test and reason about. Things to watch out for is the prompt load. Eventually, you might see that 3 to 10 tool calls might be fine, but 8 to 10 is where your LLM might get hallucinated. Wherever possible, think about splitting the agent into multiple agents if you feel that you're calling too many tools at the same time. The next one is a sequential one. This is a very common pattern that we have in most of our workflows. Sequential patterns are basically a series of agent runs where one agent gets output from the other agent as a form of the output key. At the core level, you have zero glue between the steps.

这些可以由顺序代理中的子代理来确定。会话状态本身通过一个输出键来维护。每个代理运行后，其输出都会通过输出键参数提供。添加和移除阶段都很容易。需要注意的是，延迟会累加。显然，工作流中添加的代理越多，等待时间就越长。如果按顺序运行所有这些代理，并行处理的空间就会更小。如果你知道这些代理确实可以并行运行，就尽量让它们并行执行。我也会谈到并行扇出。还需要注意的一点是，上游故障可能会拖垮整个链条。比如在这种情况下，如果研究员代理失败，写作者代理就无法获得输出。这就像顺序流程中的单点故障。These can be determined by the sub-agents for a sequential agent. The session state itself is maintained via an output key. After each agent runs, the output is provided by the output key parameter. It's very easy to add and remove stages. Things to watch out for is additive latency. The more agents that you will add in your workflow, obviously, the wait time increases. There is less scope for parallelism if you are sequentially running all these agents. Try to have the agents run in parallel if you know that these can actually run in parallel. I'll be talking about the parallel fan-out as well. One more thing to watch out for is the upstream failures can poison the entire chain. Let's say if the researcher failed in this situation, the writer will not be getting an output. It's like a single-point failure for your sequential flow.

这是我们用过的一种很有意思的智能体模式。这里，Java 提供了类似 Flowable 的事件机制。如果你要扩展基础智能体，可以随时重写 `runAsync` 函数。我们在这里做的是创建一个分类智能体，将查询分为账单问题或技术问题。如果是账单查询，就交给账单智能体；如果是技术查询，就交给技术智能体。我们还有一种默认模式，会将请求交给通用智能体。 在确定使用哪条分支或哪条路由时，这是一种值得关注的有趣模式。在这个例子中，路由器是单点故障。如果路由器本身无法确定要执行什么任务，就需要考虑采用分流方案，或者为默认路由准备一个备用方案。This is an interesting agentic pattern which we have used. What you have over here is Java provides like Flowable for events. If you are extending the base agent, you can always override the runAsync function. Over here, what we have done is we have a classifier agent that classifies a query into either a billing or a technical question. If it's a billing query, it goes to the billing agent. If it's a technical query, it goes to the technical agent. We also have like a default pattern that goes to a general agent. When it comes to determining which fork to use or which route to use, this is a very interesting pattern to watch out for. The router in this case is a single point of failure. If your router itself is not able to determine what task to-dos, that is when you want to check a bifurcated approach or a backup approach for your default routing.

路由器本身确实会带来额外延迟。模型也可能过度分类意图。比如，技术代理和计费代理实际上处理的是相同的事情。在这种情况下，代理本身很难准确判断用户的意图。顾名思义，并行扇出与汇集就是让多个代理并行运行。当你非常确定代理 A 和代理 B 可以同时运行时，就可以使用这种模式让这些代理并行运行。在我们的案例中，广告脚本生成采用的方式是让广告脚本护栏和广告脚本生成并行运行。这个例子中有一个审计代理和一个风格检查器。这两个代理并行运行，另有一个审核代理负责审核最终输出。You do have an additional latency that adds to the router itself. Models can also over-classify intent. Let's say you have this example wherein a technical agent and a billing agent are actually doing the same things. In this case, it's very difficult for the agent itself to really determine what the intent of the user is. Parallel fan-out and gather, as the name suggests, is agents running in parallel. When you absolutely know that agent A and agent B can actually run together, these are the patterns we can use to run these agents in parallel. In our case, what we are doing with the ad script generation is we are running the ad script guardrail and the ad script generation in parallel. In this example, we have an auditor agent and a style checker. You have these two agents running in parallel, and you have the reviewer agent that reviews the eventual output.

并行智能体需要注意的一点是，每个智能体都不了解其他智能体实际运行过什么。运行并行智能体时应该牢记这一点：顺序执行的智能体能够获取的信息，并行智能体无法获取。显然，随着并行智能体数量按线性规模增加，Token 消耗也会更多。最后一种是我们采用的另一种改进方法，称为生成器-评审器循环模式。在这个例子中，你可以看到一个 SQL 生成器和一个 SQL 评审器。SQL 生成器的工作基本上就是生成 SQL，而评审器则负责判断生成的 SQL 是否通过。你要做的是持续检查，直到评审器表示可以为止。One thing to watch out for for parallel agents is each agent does not have context of what the other agent has actually run. That is something that you should keep in mind when running parallel agents, that what was available for a sequential agent is not available for parallel agents. Obviously, token consumption is more once you have a linear scale for your parallel agents. The last one is more of a refinement approach that we have. It's called a generator-critic loop pattern. In the example, you can see that you have a SQL generator and you have a SQL critic. What the SQL generator is doing is basically generating a SQL, and the critic, what it's doing is it's providing whether that generated SQL is actually a pass or a fail. What you want to do is you want to keep on checking until you have the critic give its go ahead.

评审器可能会一直运行下去。循环代理有一个名为 maxIterations 的属性，可用于防止令牌消耗失控，有点像一种退出策略。需要注意的是，每次迭代都要承担一次完整的 LLM 往返调用成本，因此成本可能会成倍增加。如果没有明确的退出策略，模型可能无法收敛。即便如此，也可能遇到竞态条件之类的问题。评审器本身也是一个 LLM。实际上，每个细化步骤都要付两次费用。There is a possibility that the critic might just run indefinitely. What loop agents have is this attribute called maxIterations. These allow us to prevent runaway token consumption, kind of like an exit strategy. Things to watch out for is like, each iteration is a full LLM round-trip cost, so costs can multiply. Models may fail to converge without a clear exit strategy. Even in this case, you can have situations where you can run into a race condition or something like that. The critic is also an LLM. You're actually paying twice per refinement step.

边界要点Boundary Takeaways

回到要点总结。关于我们在智能体方面所做的工作，本次智能体相关的分享就到这里。推理应由智能体负责。你应该使用智能体来识别意图、范围和上下文，而不是依赖算术运算。凡是确定性的工作，都应该交给代码；凡是涉及语义或意图的工作，都应该交给智能体。 在验证智能体输出时，我们目前会对每个智能体进行 schema 验证。模型如今完全能够处理缺失字段防护、业务规则验证和空值检查。只要条件允许，就应该识别适合串行执行和并行执行的模式。如果你的用例需要在智能体运行过程中根据条件提前退出，这如今也是可以实现的。 所有这些都封装在 gRPC 服务边界内。最终，你会调用 gRPC 服务，获得包含智能体输出的完整响应。最后一点是：只要条件允许，就应避免让一个智能体同时处理多项任务。就像单一职责原则一样，智能体越精简、职责越明确，性能就越好。Going back to the takeaways. This concludes our agentic talk for what we have done with regards to agents. Agents own reasoning. You want to identify the intent, the range, the context using agents and not arithmetic. Anything that is deterministic should be code. Anything that is semantic or intent based should be agents. When it comes to validating agent outputs, schema validation is something that we currently have on each of the agents. Missing field guards, business rule validations, and null checks, those are the things that models are completely capable of doing today. Wherever possible, you want to identify the patterns for sequential versus parallel. If you have a use case where you want to conditionally exit out of your agentic runs, that is also possible today. All of this is encapsulated within a gRPC service boundary. Eventually you get your final call for your gRPC service that has the agent output, which is your holistic response. As a last takeaway, wherever possible, you should avoid an agent doing multiple things. Just like a single responsibility principle, the smaller and more concise your agents are, the better the performance is.

工具设计就是提示工程Tool Design is Prompt Engineering

现在我们来谈谈工具。工具就像智能体的手和脚，可以这么理解。工具模式有一点经常被严重低估。什么是工具模式？它是一种注释，用来告诉 LLM 应该如何使用这个工具，以及这个工具的用途是什么。它本质上是 LLM 的契约，而不只是共享文档。工具描述非常重要，接下来我们会讨论一些例子，看看这些工具描述实际上是如何帮助我们确定地理定位的。我们目前的工具配置方式是将工具和 API 区别对待。从更宏观的层面来说，我们有 Ads API，它包含标准的 Web API；但我们的智能体在调用不同工具时各有不同的意图。现在你应该明白，为什么我们把智能体和工具捆绑在一起，而不是将它们作为共享实体了。We'll now move towards tools. Tools are the hands and legs of the agents, that's a way to describe it. One of the things that is very underestimated is the tool schema. What is a tool schema? It's an annotation that provides the LLM how the tool should be used and what the description of that tool is. It's basically the contract for the LLM, and not just like shared documentation. The descriptions of the tool are something that are very important, and we'll talk about a few examples how these tool descriptions have actually helped us to determine geo-targeting. The way we have currently our tool setup is we treat tools and APIs differently. On a broader level, we have Ads API, which has standard web APIs, but we have agents that have different intents for calling the different tools. Now you'll understand why we have bundled agents and tools together and not as a shared entity.

如果一个团队拥有自己的智能体，也就拥有了自己的工具。在这个例子中，我们有两个工具，分别叫作 searchGeoTargets，以及一个假设未来可能会有的智能体，叫作 lookupGeoById。这些工具的用途完全不同，但 API 是相同的。一个关键要点是，你的工具不应该只绑定到 API 封装上，而应该与智能体的意图以及智能体实际提出的请求相匹配。根据智能体的需求来扩展工具，比它提供的是哪个 API 更重要。和其他软件一样，工具也可能发生故障，因此让工具以得体的方式失败非常重要。所谓得体的方式，基本上就是工具发生故障时，智能体能够理解究竟发生了什么。在这页幻灯片的例子中，我想向大家展示的是，假设我们调用了兴趣细分接口，而兴趣细分 API 因某种原因没有返回兴趣 ID，我们可以通过两种方式处理。一种是完全不处理这个错误，结果得到一个看似不错的内部服务器错误，但智能体不知道该怎么办。另一种方式是返回结构化响应，让智能体了解究竟发生了什么，并得体地退出。If a team owns its agents, it also owns its tool. In this example, we have two tools, which are called searchGeoTargets, and a hypothetical agent that we may have in the future called lookupGeoById. The intents of these tools are completely different, but the API is the same. A key takeaway is that your tools should not be tied to just API wrappers, they should be tied to the agent intent and what the agent actually requests. Augmenting the tool with the agent requirements is more important than what API it is providing. As any other software, tools are subject to failure, and it's very important to have tools fail in a very gracious way. When I say gracious way, basically, when a tool fails, the agent should be able to understand what exactly happened. In the example for this slide, what I'm trying to show you guys is, let's say if there was a call to the interest segments, and the interest segments API, for some reason, is not providing the interest ID. There are two ways we can handle this. One is, you don't even handle that error, and you get this nice internal server error where the agent does not know what to do. Or, on the other hand, you can have a structured response, which gives the agent an idea of what exactly is happening, and exit gracefully.

工具委派有不同的模式和组合方式，我想在这里讨论一下。首先，你可以将工具映射为一对一的上游 API。当你希望针对某个特定的客户端搜索调用特定的 API 时，就可以采用这种方式。它的优点是几乎没有额外开销，延迟也很低，而且有点像直接的契约。 第二种方式是将多个 API 调用组合成一次工具调用。当你希望在一个工具功能中聚合某些 API 行为时，这种方式就很有用。请记住，工具调用本身也是 LLM 调用，因此成本也不低。每当你在追踪记录中发起工具调用时，都会看到这次额外的 LLM 调用也增加了延迟。有时，工具需要会话状态，或者工具实际上可以上传会话状态。Tool delegation has its different patterns and different compositions that I want to talk about. First, you can map tools as a one-to-one upstream API. This is where you want to invoke a particular API call for a particular client search. The pros of this can be like, you don't really have any overhead, you have minimal latency, and it's kind of like a direct contract. The second approach is, tools can compose multiple API calls into one single tool call. This is useful when you want to aggregate certain API behaviors in one tool functionality. Keep in mind that tool calls are also LLM calls, so these are also expensive. Whenever you are making a tool call in your trace, you'll see that that additional LLM call is also adding to your latency. There can be times where tools need a session state, or tools can actually upload the session state.

工具有自己的上下文，称为 ToolContext，它可以传递到会话作用域。如果工具想要更新或读取此上下文，就可以获取该上下文。在我刚才举的例子中，我们从工具上下文中获取国家代码。回到第一页幻灯片，我们使用的是相同的上游服务，但面向不同的智能体需求。你可以让两个不同的智能体使用两个不同的工具，但调用同一个 API。工具设计的关键要点其实很简单。对于调用的组合或拆分，需要判断哪些操作应该合并，哪些应该分开。工具可以在描述中加入模式层面的依据，以避免对数据进行静态引用，并避免昂贵的调用。正如我提到的，将错误作为恢复提示，对于处理智能体工具故障并为智能体提供更多上下文、帮助其优雅退出来说非常重要。多智能体封装是指多个智能体可以用不同的工具封装对同一上游 API 的调用。Tools have their own context called ToolContext, which can be threaded to the session scope. If a tool wants to update or read from this context, you can get that context. In our example that I mentioned, we are getting the country code from the tool context. Going back to the first slide, we have the same upstream, but different agentic needs. You can have two different agents using two different tools, but the same API. The key takeaways of the tool designing are pretty much straightforward. Whenever it comes to call bundling or splitting, you need to determine what needs to be clubbed together or what needs to go separately. Schema-level grounding is something that tools can have in the description itself to avoid static references to the data, and expensive calls can be avoided. Errors as recovery prompts, as I mentioned, are very important in terms of handling agentic tool failures and giving the agent more context around how to exit gracefully. Multi-agent wrapping is something that multiple agents can wrap the same upstream API calls with different tools.

通过评估与追踪提升可靠性Reliability Through Eval + Tracing

既然我们已经谈了很多关于工具的内容，接下来就谈谈可能出错的地方、需要测试的内容，以及我们一开始采用的糟糕模式。作为传统的软件工程师，我们喜欢编写单元测试。这些测试运行起来很顺利，让我们很有成就感，觉得一切都运行得非常好。为了进一步增强信心，我们还会编写使用模拟 LLM 的集成测试，确保流水线各环节的连接以及每一项 gRPC 集成都正确无误。那 LLM 调用本身呢？LLM 调用本身具有非确定性，而任何非确定性的东西都容易产生幻觉。如果我们的做法是等到有人投诉了才意识到出了问题，那就是一个非常糟糕的信号；应该避免这样的说法。谈到评估时，我想重点关注评估的前提条件，那就是追踪。Now that we have talked a lot about tools, let's talk about the things that can go wrong, the things that need to be tested, and the bad pattern that we initially started with. Being traditional software engineers, we love to write unit tests. These are nice functions that run and give us satisfaction of, things are running amazingly well. To boost our confidence even more, we have integration tests with mocked LLMs, where the pipeline wiring and every gRPC integration is made sure that we are wiring everything correctly. What happens to the LLM call itself? The LLM call itself is something non-deterministic, and anything that is non-deterministic is susceptible to hallucination. The approach where we say, we know it's bad when some complaints come in, that is a very bad sign, and comments like this should be avoided. When it comes to evaluation, what I want to focus on is the prerequisite for evaluation, and that is tracing.

如果某项评估只验证输出，却不检查追踪记录，那不过是做做样子。我这么说是因为追踪记录不仅包含智能体的输出，还记录了生成该输出的整个过程。所谓输出的生成过程，是指你可以通过不同的指标了解智能体的运行时长、工具调用、内容审核和延迟；我们还会添加自定义的 GenAI span，记录原始输入和输出。ADK 提供了一个独立的指标插件，我们可以用它将这种行为整合到每个智能体中。收集到这些遥测数据后，我们可以将其导出到 OTel，用于监控 Grafana 仪表板；也可以使用名为 Braintrust 的第三方服务，它能以非常直观的图表展示智能体调用的情况。评估分为三个层面：工具调用轨迹，用于验证智能体是否执行了正确的步骤；确定性资产，例如我们这里用到的架构验证、Geo ID 结构和年龄段；以及 LLM-as-judge。像语气、品牌契合度和语义连贯性这样的特征，都可以通过范围限定的 LLM-as-judge 来识别，这充分说明我们有必要设置最后这一步评估。If you have an evaluation that just verifies the output without the trace, that's mere theater. The reason I say that is the trace not only has the output of the agent, but also the journey of that output. When I say journey of that output, you have different metrics for the agent that tells you the duration, the tool call, the moderation, latency, and we also add our own custom GenAI span that records the raw input and the output. ADK offers a separate metrics plugin that we can use to bundle this behavior to each and every agent. Once we have those telemetry, we can export those telemetry to either OTel for monitoring our Grafana dashboards, or we use this third-party vendor called Braintrust that shows us a very nice demographic display of how the agent calls look. The three levels of evaluation are the tool trajectory. Tool trajectory is verifying that the agent is performing the right steps. Deterministic assets, things like schema validation, Geo ID structures, and age brackets in our case. LLM-as-judge, so identifying the tone, the brand fit, the semantic coherence is something that scoped LLM-as-judge provide a very good reason for us to have like a last evaluation step.

最后我要谈的是实时评估。事实证明，实时评估非常有用。当你在 trace 上运行智能体时，实际上可以让评估器在实时流量上运行。让实时评估覆盖 100% 的流量并不是个好主意，因为这也会增加成本。具体取决于你的使用场景。如果某个智能体刚刚进入市场，那肯定应该先设置一个较高的比例。等你对它有了相当的信心之后，就可以考虑调整这个比例。这些值都完全可以配置，我们可以随着时间推移进行调整。我们在评估方面做的一件重要的事是，没有将评估与合并构建绑定。我们发现，每次你向代码仓库提交一个合并构建时，实际上都会发起 LLM 调用。The last thing which I'm going to talk about is live evaluation. Live evaluation is something that is proving to be very useful. When you have an agentic run on the trace, you can actually have an evaluator which is running on live traffic. It's not a good idea to have live evaluation run on 100% traffic because that also adds to your cost. It depends on your use case. If an agent is new to the market, then definitely start with a higher value. Once you have pretty much confidence around it, then you can probably change the value. These are completely configurable values which we can change over time. One important thing that we have actually done with evaluation is we have not attached evaluation to merge builds. What we have figured out is every time you have a merge build to your repository, you are actually making LLM calls.

这是一个我们想要避免的额外成本环节。开发者编写自己的智能体时，会反复在离线环境中运行评估。我们确实有一个类似夜间定时任务的流程，会对整个测试套件运行评估。下面展示的是一条 trace 的实际样子。这是广告脚本生成智能体运行过程中每次调用及其成本的情况。在这里，你可以看到两个并行的智能体调用正在运行。在底部，你还能看到广告质量评分器在最后运行。它会定义输出，并判断智能体的响应是好是坏。正如你所看到的，评估的关键要点显然是：我们希望为每次生产环境调用都保留 trace。我们可能不会对每次生产环境调用都进行实时评估，但追踪肯定很有帮助。评估的三个层面——轨迹评估、确定性评估和裁判评估——是我们希望具备的三大支柱。That is an additional cost step that we wanted to avoid. When developers are writing their own agents, they are iteratively running their evaluations offline. We do have like a nightly process that runs the evaluation for the entire suite. This is how a trace actually looks like. This is what each call and cost of the run agent for the ad script generation looks like. Over here, you can see that the two parallel agent calls are running. At the bottom, you can actually see that the ad quality scorer is running at the end. It's defining your output and determining whether the agent response is good or bad. The key eval takeaways, as you can see, are obviously, we want to have trace for every production call. We may not have live evals for every production call, but tracing is definitely something that helps. The three levels of evaluation, the trajectory, the deterministic, and the judge are the three pillars that we want to have.

我们不希望把评估作为 CI 门禁来运行。我们希望按需运行，可能像夜间构建那样。实时评估确实能提供非常重要的信号，因为它们基于生产流量。需要明确的一点是，判定器的作用是检测，而不是预防。每当判定器检测到异常时，你要做的就是针对该异常逐步收紧规则，并添加一条新的断言。最终断言越多，判定器的成本就越低。We don't want to have the evaluation run as a CI gate. We want to have this on-demand and probably like a nightly build. Live evaluations definitely carry a very important signal because these are production traffic. One key thing to identify is like judge is equal to detection and not prevention. Every time you have the judge detect an anomaly, what you want to do is you want to ratchet on that anomaly and add a new assert. The more asserts you eventually have, the less expensive your judge becomes.

结论Conclusion

最后总结一下，从智能体架构开始，我们会做出一些不同的选择：不采用单一智能体架构，也避免大量消耗 token。基本上，在工具设计、可观测性和提示词路由方面，我们不希望依赖一个全局系统提示词。今天我希望大家记住的是：不同团队各自负责不同的智能体，推理过程经由编排层流转，再回到漏斗中。这就是目前整个多智能体架构的样子。To wrap things a bit, what we would do differently, starting with the agent architecture, is not have a mono-agent, a big, large token usage. Basically, tool design, observability, and prompt routing is something that we want to not have like a global system prompt. What I want you to take away today is you have different teams owning different agents. You have reasoning flowing through the orchestration and reasoning coming back to the funnel. That's what the whole multi-agentic architecture currently looks like.

 

查看更多带文字稿的演示文稿See more presentations with transcripts