如何判断 AI 智能体是否可靠：Arize AI 的 Laurie Voss 用 Phoenix 与 Claude 实操评估

简要描述Brief Description

如何判断你的 AI 代理是否优秀：使用 Phoenix 和 Claude 进行评估——Laurie Voss，Arize AIHow to Know If Your AI Agent Is Any Good: Evals With Phoenix & Claude — Laurie Voss， Arize AI

目录Table of Contents

• 简介与设置
  Intro & setup
• “我觉得不错”不是测试套件。
  "Looks good to me" isn't a test suite
• 追踪记录与评估：心智模型
  Traces vs. evals: the mental model
• 代码评估、LLM 评审与人工审核
  Code evals, LLM judges & human review
• 智能体为何会自信地出错
  Why agents are confidently wrong
• 第 1 步：使用 Phoenix 进行追踪
  Step 1: Tracing with Phoenix
• 步骤 2：读取你的跟踪记录
  Step 2: Reading your traces
• 116%的分析师共识错误
  The 116% analyst consensus bug
• 步骤 3：代码评估
  Step 3: Code evals
• 步骤 4：将 LLM 用作评判者
  Step 4: LLM-as-a-judge
• 为什么正确性评估的每份报告都失败了
  Why the correctness eval failed every report
• 步骤 5：编写自己的评判器
  Step 5: Writing your own judge
• 步骤 6：测试你的判定器
  Step 6: Testing your judge
• 第 7 步：实验
  Step 7: Experiments
• Upgrading to Claude Haiku 5.5
• CI 中的评测
  Evals in CI
• 周一从哪里开始
  Where to start on Monday

简介与设置Intro & setup

我叫 Lauri Voss，是 Arise AI 的开发者关系负责人。呃，很久以前我曾在 npm, Inc. 做 JavaScript，所以你可能还记得那时的我。呃，不过如今我主要思考 AI 应用，以及如何让它们运行起来、真正发挥作用。嗯，我之前 15 分钟一直在放这张幻灯片。希望你已经看过了。如果还没看，现在正是开始的时候。嗯，这是一个需要适度动手实践的工作坊。我已经为你构建好了一个智能体，它就在那个代码仓库里。呃，你可以试着调整提示词并运行实际评估。嗯，整个过程设有多个检查点。所以如果你迷路了，只要切换到相应的检查点，就能跳到已经可以正常运行的部分。嗯，你需要使用 Node 22 或更高版本。所有这些示例都会使用 TypeScript。My name is Lauri Voss. I'm head of developer relations at Arise AI. Uh I used to do JavaScript at npm, Inc. way back in the day, so you may remember me from then. Uh but these days I think about AI applications and how to make them run and how to make them actually work. Um, I had this slide up for the 15 minutes before. I hope you've seen it already. If not, now is the time to get started. Um, this is a moderately hands-on workshop. There's I've already built an agent for you. It's sitting there in that repo. Uh, you will get to play with the prompts and run actual evals. Um, but it's got checkpoints all the way through. So if you get lost, you can just switch to the right checkpoint and uh uh jump to the part where it's already working. Um you're going to need node 22 or later. All of this all of these examples are going to be in Typescript.

嗯，因为你知道这是我，所以只要可以，我就会用 JavaScript。呃，你要在自己的笔记本电脑上安装并运行 Phoenix。呃，在会议的 Wi-Fi 上可能会有点麻烦，不过我今天早上试过，速度还挺快的。所以，呃，UVX 很神奇。嗯，你不需要在这里运行任何 Python。所有 Python 代码都会由 JavaScript 替你运行，这挺方便的。嗯，设置方面我想就这些了。嗯，今天我们要做的是……哦，对了，抱歉，还有一项设置。安装好 Phoenix 后，你应该在浏览器中打开 localhost 606，这样就能看看它是什么样子。现在里面还是空的，不过我们会运行一些命令，把数据填进去。嗯，你还需要一个 Anthropic 密钥。Um because you know it's me, I'm going to use JavaScript if I can. Uh and uh you're going to be installing and running Phoenix on your own laptop. uh on conference Wi-Fi that can be a challenge, but I tried it this morning and it came through pretty quickly. So, uh UVX is magic. Um you're not going to need to run any Python in this thing. All of the Python is going to be run for you on your behalf by JavaScript, which is nice. Um I think that's it for the setup. Um so, what we're going to do today is um uh oh yeah, sorry, one more bit of setup. Once Phoenix is installed, uh you should open up your browser on localhost 606 so that you can see what it looks like. Um it's going to be empty right now, but we are going to run the commands that will fill it with data. Um you will also need an anthropic key.

我的所有示例都用的是 Claude。呃，你可以用世界上任何一家 LLM 服务提供商来做这些事情，不过我得选一个例子，所以就选了我每天都在用的这个。所以，这个例子用的是 Claude。嗯，有谁需要帮忙设置吗？举个手。直接喊我就行，没关系。好，前十位。怎么了？All of my examples are using Claude. Uh you can do all of this stuff using any LLM provider in the world, but I had to pick an example, so I picked the one that I use every day. So, this one is using Claude. Um, who needs help with the setup? Show of hands. Just shout at me. It's fine. All right. So, the first 10. What's up?

主持人：刚刚开始。Host: Just getting started.

嘉宾：好的，太棒了。Guest: Okay, great.

主持人：嗯。好，前10分钟是理论内容，我讲的时候你们可以趁机把设置弄好。Host: Yeah. So, the first 10 minutes are theory, which will give you some time to catch up on the setup uh while I'm talking.

“看起来不错”不算测试套件。"Looks good to me" isn't a test suite

那么，我们先从理论讲起。嗯，我假设你们大多数人现在遇到的情况是这样的：你们已经开发了一个内置 LLM 的应用。呃，也许是聊天机器人，也许是智能体。你们反复测试了很多次，觉得效果相当不错。呃，接着你们把它发布了，结果有人用了一种你们没预料到的方式使用它，然后它就出问题了。嗯，或者你们为了修复一个问题修改了提示词，呃，心里却一直隐隐觉得，可能又把别的东西弄坏了。呃，可你们无从得知。这就是今天要讨论的问题，也是评估要解决的问题。你们怎么知道自己的智能体好不好？怎么知道一次改动让效果变好了还是变差了？呃，又该如何在进行改动时防止效果变差，而不必亲自阅读每一条输出呢？So, let's get started with the theory. Um, here's the situation I'm assuming most of you are in. You have already built an application that has an LLM in it. Uh, maybe a chatbot, maybe an agent. You tried it a bunch of times and it looks pretty good. Uh, then you shipped it and somebody used it in a way that you weren't expecting and it broke. Um, or you changed the prompt to fix one thing. uh and you have this nagging feeling that you broke something else. Uh and there's no way of knowing. That is the problem that today is about. That is the problem that evals are here to solve. How do you know if your agent is any good? How do you know if your change made things better or worse? Uh and how do you stop it getting worse when you make changes uh without having to read every output yourself?

我承诺，今天结束之前，你们会有一个可运行的 agent，并且已经用代码评估、LLM 裁判评估，以及你们自己编写的自定义评估，对它进行了端到端追踪。呃，你们还会检查自己的裁判是否可信。你们会做一次实验，证明所做的改动确实有效。呃，你们还会把这一切都做成一个测试套件，用 npm tests 运行，这样就能把它放进 npmci。嗯，这正是我最期待的部分。所以，这是我们最后要做的事。我看到有些人迟到了才进来。那我就快速回到介绍设置的那张幻灯片，让他们不至于完全摸不着头脑。好，记住这个代码仓库，因为你们会用得上。大家的 Wi-Fi 连接怎么样？能连上网吗？My promise by the end of today is that you're going to have a working agent traced end to end with code evals with LLM judge evals and with a custom eval that you wrote yourself. Uh and you'll have checked whether your judge can be trusted. You'll have run an experiment to prove that your changes worked. Uh and you'll have have all of it running as a test suite with npm tests so you can put it into npmci. Um, that is the bit that I'm most excited about. So, it's going to be the last thing that we do. I see some people have walked in late. So, let me just go back to the setup slide very quickly so that they're not completely lost. Right. Remember this repo because you're going to want this repo. How is the Wi-Fi working for people? Is it Are you getting places?

嘉宾：太好了。好吧。我在世界博览会上做过这件事，呃，当时我不小心弄了个仓库，里面全是产品图片，在演示时看起来特别棒，但那些产品图片占了 400 MB，结果彻底失败了。没人能把仓库下载下来。所以我吸取了教训，今天的内容里没有图片。呃，Guest: Excellent. Okay. I did this at World's Fair and uh every and I accidentally had a repo that was full of product images that looked really great in the demo, but it was 400 megabytes of product images and it just complete failure. Nobody was able to get even the repo down. So, I've learned my lesson and today has no images. Uh,

追踪记录与评估：心智模型Traces vs. evals: the mental model

那么我们先来讲讲基础知识。什么是 eval？最简短的说法——我跟有一定经验的工程师交流时总是这么解释——是这样的：有一种东西叫 trace，基本上就是日志；还有一种东西叫 eval，基本上就是测试。eval 就是在 trace 上运行测试。嗯，eval 和单元测试的区别在于，eval 用于测试那些每次都不完全相同的输出，也就是说，LLM 生成的非确定性输出。嗯，trace 会以极其详尽的程度记录 AI 做了什么：每一步 agent 操作、每一次工具调用、每一次 LLM 调用，以及每一步的输入和输出。trace 由我们称为 span 的部分组成。每个 span 都是执行过程中的一个步骤。你可以把它看作日志中的一行。so let's cover the basics. What is an eval? The shortest version, the one that I use whenever I'm talking to somebody who's been an engineer for a while is that uh there are these things called traces and they are basically logs and there's these things called evals and they're basically tests and evals are using are running tests on your traces. Um the difference between an eval and a unit test is that uh evals are for outputs that are never exactly the same twice, which is to say the non-deterministic outputs that you get from LLMs. Um traces record what your AI did at an incredible level of detail. So every agent step, every tool call, every LLM call, the inputs and outputs at every single step. A trace is made up of what we call spans. Each span is one step in the execution. You can think of it as just a line in the log.

嗯，所以一个完整的智能体轮次是一个跨度，里面嵌套着其他跨度。呃，每个跨度都会记录输入、输出、耗时、使用的 token 数量，有时还会记录成本。所以，追踪记录会告诉你发生了什么，评估则会告诉你效果好不好，这就是今天要讲的整体思路。这完全不是什么高深的科学。“我觉得看起来不错”并不等于测试套件。嗯，但大多数 AI 功能都是这样发布的：你把它构建出来，试几次，看看效果对不对？嗯，看起来不错。然后就发布了。呃，你试过的那三种情况可能确实都表现不错。嗯，但试三次并不等于测试套件。通常的解决办法是编写单元测试。嗯，但单元测试在这里行不通，因为每次运行应用程序时，同一个提示都会生成不同的答案文本。Um, and a so a whole agent turn is a span that has other spans nested inside of it. Uh, and each one records its input, its output, how long it took, how many tokens it used, uh, and sometimes how much it cost. So traces tell you what happened and evals tell you whether it was any good and that is the whole mental model for today. This is very far from being rocket science. Looks good to me is not a test suite. Um, but it's how most AI features get shipped. You build it, you try it a few times, does it look right? Yeah, that looks good. You ship it. Uh, and it probably does look good on the three things that you tried. Um, but three tries is not a test suite. And the usual fix is unit tests. Um, but unit tests don't work here because the same prompt is going to give a different answer text every single time you run your application.

呃，所以两个截然不同的答案可能都正确。一千种不同的答案也可能全都正确。呃，只是正确的方式不同。所以没有一个预期字符串可以用来断言结果。呃，因此人们往往会退而依靠人的判断。他们看着输出的字符串，心想：嗯，这看起来没问题。[嗤笑] 但是，这种做法无法规模化。它发现不了回归问题，而且肯定也无法在 CI 中运行，因为那里没有人盯着。呃，实际情况是，你修改系统提示词来调整语气，语气确实变好了。呃，可现在机器人开始编造产品功能了。等客户发现时，你才知道出了问题。每次修改提示词，都可能影响用户发送的各种输入。呃，你不可能手动测试所有这些情况。于是你就陷入了打地鼠游戏：修好一处，别处又坏了；修好那里，另一处又出问题。Uh, so two completely different answers can both be right. a thousand different answers can both can all be right. Uh just differently right. So there's no expected string to assert against. Uh so humans tend to fall back on human judgment. They look at the strings being output and they're like well this looks fine. [snorts] Um but that doesn't scale. It doesn't catch regressions and it certainly doesn't run in CI where there are no humans to watch things. Uh what that looks like in practice is you change your system prompts to fix the tone and the tone gets better. Uh but now the bot is making up product features. Uh and you don't find out until a customer does. Every prompt change can affect every kind of input your users send. Uh and you can't test all of those by hand. So you end up playing a game of whack-a-ole where you fix one thing and something else breaks and you fix that and something else breaks.

嗯，评估可以解决这个问题：它会给你一个可以跟踪、比较并据此采取行动的数字。这一点每个月都变得越来越重要，因为嗯，差不多每隔几周就有人发布一个新模型，然后你就会想：“哦，我想换用新模型。”嗯，但除非你有一套可以用来测试的评估，否则你根本不知道新模型在哪些方面会更好，又在哪些方面会更差。嗯，这正是我们今天要做的。我们先从 Claude 4.5 开始，然后把它升级到 Claude Haiku 5.5。嗯，这个模型刚好在一周前发布，所以做这个演示特别容易。我当时想，演示里要升级模型，那就干脆把这件事做成演示吧。嗯，有了评估，你就可以运行整套测试、比较数字，到下午结束时，你就知道自己能不能升级模型了。uh evals fix that by giving you a number that you can track and can compare and act on. This matters more every month because uh every couple of weeks somebody releases a new model and you're like, "Oh, I want to switch to the new model." Uh but you have no way of knowing which things the new model is going to be better at uh and which things the new model is going to be worse at unless you have a set of evals that you can run things through. Uh, and that's exactly what we're going to do today. We're going to start with uh Claude 4.5 and then we are going to upgrade it to Claude uh Haiku 5.5 um uh which was conveniently released like a week ago making this demo extremely easy to put together. I was like I want to upgrade the model for this demo. So I'm going to make that the demo. Uh so with eval you run the suite you can compare the numbers and by the end of the afternoon you know uh whether you can upgrade your model.

代码评测、LLM 评判与人工审核Code evals, LLM judges & human review

嗯，评估有几种。第一种是代码评估。代码评估速度快、免费，但有点脆弱。它们和单元测试非常相似，就是普通函数，基本上是对字符串进行断言。你可以使用很多技巧，让它们比那种只断言一个简单字符串的普通单元测试更灵活。不过从根本上说，它们做的就是这件事。代码评估可以测试的内容包括：输出能否解析为 JSON？是否少于 500 个词？是否提到了我询问的内容？它们的优点是只需几毫秒就能运行，而且运行成本为零。不过，有很多有效答案是字符串匹配无法识别的。要评估这类答案，就需要第二种评估方式，也就是由 LLM 担任评判者的评估。Um and there are a few kinds of evals. The first kind is code evals. Code evals are fast and free and a little bit brittle. They are very very similar to unit tests. They are just ordinary functions where you basically assert a string. There are a bunch of techniques you can use to make them more flexible than a straight unit test that asserts a basic string. Uh but fundamentally that's what they're doing. So the things that you can test with a Kodi valal are stuff like did the output parse as JSON? Is it under 500 words? Did it mention the thing that I asked about? The advantage of them is that they run in milliseconds and they cost nothing to run. Um, but there are lots of valid answers that a string match can't tell you about. Um, and for that you need the second kind of eval which is the LLM is a judge eval.

呃，这就是大多数人想到评测时会想到的东西。在这种评测中，你会用第二个 LLM 来给第一个 LLM 的输出打分。[哼笑] 嗯，你会根据所谓的评分标准来评估它。在评测领域，有不少术语是从机器学习里借来的，我觉得这些术语反而模糊了事情的本质。其中一个是“轨迹”，一个是“评测”，还有一个是“评分标准”。我就想，评分标准不就是规则吗？为什么非要专门造个词？总之，LLM 评审能理解含义，而不只是字符串。所以可以评估诸如“这准确吗？”“它忠实于源文档吗？”“语气适合客户吗？”之类的问题。嗯，问题在于，LLM 评审本身并不是一个 LLM，却有着 LLM 的所有缺陷。所以它可能难以预测，而且每次表现都不一样。Uh, which is what most people think about when they're thinking about evals. In this one you use a second LLM to grade the output of the first LLM. [snorts] Um, you grade it against what is called a rubric. There are a bunch of terms in uh eval land that are borrowed from ML uh that I find obscure the meaning of what's going on. One of them is traces and one of them is evals and the other one is rubric. I'm like rubric just means rules. Why do we have a special word for it? Anyway, uh L&M judges understand meaning not just strings. So stuff like is this accurate? Does it stick to the source documents? Is the tone right for a customer? Um, the catch is that because the LLM judge isn't itself an LLM, it has all of the flaws that LLMs have. So, it can be unpredictable. It is never the same twice.

呃，它也可能会出错。呃，因为你用的是 LLM，所以也要花钱。呃，而且它们可能会自信满满地给出错误答案。所以，你必须检查它们。这也是我们今天要做的事情之一。然后还有第三种评估，呃，也就是人工评估：由了解该领域的人阅读输出，并以专家的身份进行评分。呃，人类是质量评估的黄金标准。嗯，但他们速度慢、成本高。呃，而且他们会感到厌倦，开始略读。所以，呃，你要同时使用这三种方式。它们不是相互竞争的方法，而是互补的方法。稍后我们会稍微谈谈如何互补地使用它们。那么，什么时候该用哪种评估呢？当答案是确定性的，就用代码。比如格式、长度、必填字段和禁用短语等。Uh, and it can make mistakes. Uh, they also cost money because you're using an LLM. Uh, and they can be confidently wrong. So, you have to check them. And that is one of the things that we're going to do today. Then there's a third kind of eval uh, which is the human eval. somebody who knows the domain, who can read the output and grade it as an expert. Uh, humans are the gold standard for quality. Um, they're also slow and expensive. Uh, and they get bored and start skimming. So, uh, you want to use all three at the same time. These are not competing approaches. They are complimentary approaches. And we're going to talk about a little bit about how you use them in a complimentary way later. So which eval do you use when? Use code when the answer is deterministic. So things like formats and lengths and required fields and band phrases.

呃，不过在需要理解的场景下，要用 LLM 评审器。它准确吗？有依据吗？有用吗？呃，对于你以前没见过的那类失败，也就是你没预料到的情况，要让人来评估。还要检查你的 LLM 评审器是否判断正确。嗯，这些都适用于任何 LLM 应用，但对智能体来说尤其棘手，因为智能体更复杂。单个 simp……单次 LLM 调用相对简单：你给它一个输入，就会得到一个输出。呃，但智能体往往会把一大串步骤串联起来。因此，可能出错的一种情况是：在流程很早的时候，智能体就犯了一个错误，然后这个错误会在智能体后续的所有步骤中层层放大，让每一步都变得越来越偏离正轨。嗯，最大的问题之一就是……Uh but use an LLM judge when you need understanding. Is it accurate? Is it grounded? Is it useful? Uh and use humans for the kinds of failures you haven't seen before. Stuff that you haven't anticipated. And to check that your LLM judges are judging correctly. Um, all of that applies to any LLM app, but agents in particular make all of this stuff harder because agents are more complicated. A single simp a single LLM call is relatively simple. You give it an input, you get an output. Uh, but an agent tends to chain a whole bunch of steps together. So, one of the things that can go wrong is very early on in the chain, the agent makes a mistake and it cascades through all of the rest of the steps in your agent and each one gets progressively more off the rails. Um, and one of the biggest problems is being

为什么智能体会自信地犯错Why agents are confidently wrong

自信地犯错。自信地犯错比明显地犯错更糟。呃，我刚才讲了级联故障。嗯，举个例子：假设你写了一个金融分析智能体，也就是我今天要给你们展示的这类智能体。嗯，你让它写一份关于特斯拉的报告。它搜索特斯拉，结果搜到的不是这家汽车公司，而是关于一位 18 世纪发明家的信息。它写出了一份关于这位 18 世纪发明家的报告，内容出色，格式也完美。呃，而你的所有检查都不会失败，因为你会觉得它格式规范、语气很好、语法也没问题。呃，只是主题完全错了。呃，这就是智能体的一种隐蔽故障。智能体也可能朝相反的方向出错。嗯，开发 Claude 的 Anthropic 做过一项名为 TA2 Bench 的基准测试，用来模拟客户服务对话。他们给智能体安排了一项任务：用户要求改签航班。confidently wrong. Confidently wrong is worse than obviously wrong. Uh, so I talked about the cascading failure. Um, one example of that is say you've written a financial analysis agent, which is the kind of agent that I'm going to show you today. Um, and you ask it for a report on Tesla. It searches for Tesla and instead of getting results about the car company, it gets results about the 18th century inventor. It writes an beautiful, perfectly formatted report about an 18th century inventor. Uh, and none of your checks fail because you're like, it's well formatted. It's in a great tone. Everything is grammatical. Uh, it's just about completely the wrong subject. Uh, and that is the kind of failure that is uh um insidious with uh agents. Agents can also fail in the opposite direction. Um, Anthropic uh, who make Claude ran a benchmark called Ta2 bench which simulates customer service conversations and they uh, gave it this task where the user asked to change their flight.

呃，按照这个测试，用户本来应该通不过测试，呃，或者说，智能体本来应该失败，因为呃，这个人坐的是经济舱，而经济舱是不允许换座位的。呃，但 Claude 发现，可以把这个人升舱到头等舱。在头等舱里，你可以换座位，然后他们在换完座位后又把这个人降回经济舱。呃，结果通过了测试，而且就像是完成了用户要求的事情；但测试却说：“不，你失败了，因为你本来不应该能做到这一点。”嗯，但这个智能体比它所处的规则更聪明。所以你在构建评估时必须预料到，呃，有时智能体会想出一种你没预料到的办法来解决问题。因此，你的测试必须非常灵活。Uh, and the test according to the test the user was supposed to fail the test uh, or rather the agent was supposed to fail because uh, the person was in economy and you're not allowed to switch seats in economy. Uh but what cloud found is that they can upgrade the person to first class. You're allowed to change your seat in first class and then they downgraded them back after changing their seat back down to economy. Uh and pass and like did the thing that the user asked for and the test said no you failed because you weren't supposed to be able to do that. Um but the agent had been smarter than the rules that it was in. Um so you have to expect that uh when you are building eval that sometimes the agent is going to come up with a solution to the problem that you did not expect. So your tests have to be very flexible.

嗯，评估结果通常大致是这样的：它包含一个分数、一个标签和一段解释。分数是一个数字，通常只有 0 或 1。标签则是一个便于人类阅读的词，与分数所代表的含义相对应，比如“通过”和“失败”、“可执行”和“不可执行”。LLM 评审还会多提供一项内容，也就是解释你为什么得到这个分数、这个标签。嗯，这也是 LLM 评审最有价值的部分，因为价值就在解释里。它会告诉你哪里出了问题、缺少了什么，而且很多时候还会告诉你该如何修改提示词，才能得到正确结果。嗯，当你大规模运行评估时，这一点尤其有用，因为你可以从 LLM 那里获得 50、100 甚至 1000 条解释，然后再让另一个 LLM 阅读所有这些解释，并指出：“这些解释里似乎存在某种规律。”Um here's what an eval result generally looks like. Uh it comes with a score, it comes with a label and it comes with an explanation. The score is a number usually just zero or one. Uh and the label is just a word that the human can read that matches what the score means. Uh so stuff like pass and fail, actionable and not actionable. Uh and LLM judges add a third thing which is an explanation of why you got the score you did, why you got the label you did. Um and it is the most valuable part of an LLM judge. Uh because the explanation is where the value is. It tells you what was wrong and what was missing and it tells you a a lot of the time what to fix in your prompt to get this right. Uh this is especially useful when you're running your evals at scale because you can get, you know, 50, 100, a thousand explanations from your LLM and you can get another LLM to read all of those explanations and be like, it looks like there is a pattern in these explanations.

看起来我们遇到了一个系统性问题。你只要把这些问题交给你的智能体，它就会说：“好的，我看到有 50 条追踪记录都在做同一件事，而且对出错原因的解释也都一样。我能修复它。”嗯，这就是我们今天要构建的循环。我们会追踪你的智能体，查看它做了什么，为重要事项编写评估，检查这些评估是否可信，然后修改智能体，并证明修改确实有效。接着，我们会把这些修改纳入 CI，确保问题不会再出现。那么，还有谁需要帮助完成设置？有没有人来晚了，还需要设置？举手示意一下。好，我回到设置页面，看看你们能不能完成。对，这就是那个神奇的网址。It looks like we have a systematic problem. And you can just feed them to your agent and your agent goes, okay, I see 50 traces all doing the same all with the same explanation of what went wrong. I can fix it. Um, so this is the loop that we're going to be building today. We're going to trace your agent. We're going to read what it did. We're going to write evals for what matters. Uh, we're going to check that those eval can be trusted. Uh, then we're going to change the agent and we're going to prove that the change worked. Then we're going to lock that in so that it stays fixed by putting it into CI. So, uh, who still needs help getting set up? Who walked in late and needs set up? Show of hands. Okay, let me head back to the setup screen and see if you can get it. Right, this is the magic URL.

如果你还没有这个，呃，你只要读一下 README，然后按照说明操作就行；或者，你知道的，直接把 README 粘贴给你的智能体，它就会阅读说明并帮你全部搞定。2026 年做教程有件挺奇怪的事：我们就会想，好吧，我可以教你怎么做这些事，也可以直接告诉你怎么让智能体帮你做这些事。呃，我们很多时候就是这么做的。好吧，我现在就假设大家都有那个 URL 了。很好，那我们从第一步开始。第一步If you haven't got this one, uh, all you have to do is read the readme and, uh, run the instructions or just, you know, paste the readme into your agent and your agent will read the instructions and get it all done for you. One of the weird things about doing tutorials in 2026 is that we're just like, well, I could show you how to do all of this stuff, or I could just tell you how to tell your agent to do all of this stuff. Uh, and that's what we do a lot of the time. All right, I'm now going to assume that everybody has that URL. Cool. So let's start with step one. Step one

步骤 1：使用 Phoenix 进行追踪Step 1: Tracing with Phoenix

就是追踪。在评估任何东西之前，你需要先获取数据，才能运行评估。而这就是追踪。今天我们要使用 Phoenix。希望到现在为止，这对大家来说都不是什么意外。呃，Phoenix 是 ARIS 的开源 AI 可观测性平台。它会从你的 AI 应用中收集追踪数据，不管你用什么框架来构建应用。它会把评估结果和追踪数据存储在一起。呃，它还提供了一个 UI，方便你深入查看所有内容。它也支持数据集和实验，我们今天会聊到这些。呃，你可以在自己的笔记本电脑上运行它。所以，呃，你可以让它完全只对自己开放，也可以在服务器上运行并与团队共享。现在，我要冒险试试看我是不是真的能……哦，这也太小了。我们把它调大一点。is tracing. Before you can evaluate anything, you need to get the data to run the to run the evaluations on. And that is what that is traces. Today we are going to be using Phoenix. I hope that's not a surprise to anybody by now. Uh, Phoenix is ARIS's open-source AI observability platform. Uh, it collects traces from your AI AI app with in whatever framework you use to build it. It stores your evals next to those traces. Uh, and it gives you a UI to dig through all of it. It also does data sets and experiments, which we're going to be talking about today. Uh, and you run it on your laptop. So, uh, you can keep it entirely private to yourself, or you can run it on a server and share it with your team. So now I'm going to dangerously see if I can actually Oh, that's so that is so small. Let's make that bigger.

好，今天我们要用的是 TypeScript。呃，你会看到很多 Python 示例，而 Fenix 也提供了很棒的 Python SDK。不过我喜欢 JavaScript，所以我们会用 TypeScript。嗯，这个仓库里有一小组脚本。Instrumentation.ts 用于设置追踪。Agent.ts 是智能体本身。Prompts.ts 里放着提示词。呃，这些文件都在哪儿？找到了。嗯，你要在这个文件夹里编辑提示词。呃，抱歉，是在这个文件里编辑。嗯，evals 文件夹用来存放我们的评测。Queries.json 是我们的测试数据。呃，希望今天的每一步都只需要运行一条 npm 命令。每一步都是独立的。所以如果你跟不上进度，可以直接跳到下一步。呃，还可以使用我放在仓库里的种子数据，这样你就不会落下。今天我们要构建的——或者说，我已经替你构建好的——是一个金融分析师。Okay, so what we're working in today is Typescript. Uh you'll see a ton of examples in Python and Fenix has great Python SDKs as well. Uh but I like JavaScript so we're going to use TypeScript. Um the repo is a small set of uh scripts. Instrumentation.ts sets up tracing. Agent.ts is the agent itself. Prompts.ts has the prompts. Uh where are all of these things? There we go. Um and you'll be editing the prompts in this folder. Uh sorry, in this file. Um and the evals folder is where our evals go. Queries.json is our test data. Uh, and hopefully every step today is just going to be one npm command. Uh, and each one stands on its own. So if you fall behind, you can skip straight to the next one. Uh, and use the seed data that I put in the repo uh, so that you don't get left behind. So what we're building today, or indeed what I've already built for you, is a financial analyst.

它的工作方式是，你给它一个股票代码，以及一个关注重点，比如特斯拉汽车交付量。呃，然后它就会去网上搜索实时数据，呃，再为你写一份相关报告。还有很多初创公司都是基于这个想法创立的：为你提供财务分析，告诉你该不该买某只股票。呃，但我们的智能体刻意做得很简单，因为我需要一个会犯错的东西，这样我们才能修正这些错误。嗯，今天不是要讲如何构建智能体。我已经替你把智能体搭好了。我假设你已经在某个地方运行着一个智能体，而你想要的是在如何让它更可靠这方面得到帮助。嗯，我们的智能体分两轮运行。第一轮是研究阶段，它会搜索网络；接着是撰写报告阶段，它会根据研究结果撰写报告。How it works is you give it a stock ticker and something to focus on like Tesla vehicle deliveries. uh and it will go and it will search the web for live data uh and write you a report about that. Uh there are whole startups built on this idea of giving you you know financial analysis of whether or not you should buy a stock. Uh but ours is deliberately simple because I needed something that would make mistakes so that we could fix the mistakes. Um today is not about how to build an agent. I've just built the agent for you. I assume you've already got an agent running somewhere and your what you want is help on making it reliable. Um, our agent works in two turns. It has a research turn where it searches the web and then it has a writing a report writing turn where it takes the research and it writes the report about it.

嗯，我是用 Claude Agent SDK 构建的。如果你没用过它，它是 Anthropic 的智能体框架。呃，它和 Claude Code 底层使用的是同一套引擎，后面这一点会变得很重要，尤其是当你用 Claude Code 来处理它时——因为两个版本的 Claude Code 往往会试图相互配合，结果就是它们会在评测中作弊，这就不太有帮助了。嗯，它提供了工具调用、网页搜索，以及可以跨多个轮次持续进行的对话。我们还兼容许多其他选项，包括 Vercel AI SDK、Mastra、LangChain 和 OpenAI 的 Agent SDK。呃，Phoenix 为你能想到的所有框架以及所有 LLM 提供商都提供了适配器和插桩支持。无论你用什么构建智能体，Phoenix 都能与之配合使用，而且会自动完成插桩。Um, I'm I've built it using the claude agent SDK. If you haven't used it, that is anthropics framework for agents. Uh it is the same engine that runs underneath cla code which is going to become relevant later especially if you're using claude code to work with it uh because the two versions of cloud code tend to try to cooperate with each other and that makes them cheat the eval which is not very helpful. Um uh it gives you tool use web search uh and conversations that carry on across turns. There are also lots of other options that we are compatible with. The Versel AI SDK, Mastra, Langchain, OpenAI's agent SDK. Uh, Phoenix has adapters and instrumentation for every possible framework that you can think of and every LLM provider that you can possibly think of. Wherever you have built your agent in, Phoenix will work with it and the instrumentation will happen automatically.

呃，不过演示时我得选一个，所以就选了这个。嗯，[哼笑] 跟踪只需要大约五行代码。嗯，首先我们要调用 Phoenix hotel 包里的 register，它会设置跟踪并将其指向 Phoenix。嗯，这个行业的一大优点是，大家基本上都选用了同一套检测标准。嗯，整个行业的一切都构建在 OpenTelemetry 之上，而在 OpenTelemetry 之上还有一个名为 OpenInference 的标准，由 Arize 负责维护。嗯，大家已经把检测功能集成到了各自的框架中，这就是为什么我们能兼容所有这些框架。你只需要使用这个 register 命令，告诉它：“嘿，把你已经集成到框架里的检测功能打开，并把它指向这里。”在这个例子中，你只需要把它指向你的笔记本电脑，嗯，就这么简单。Uh, but I had to pick one for a demo, so I picked this one. Um, [snorts] tracing takes about five lines. Um, first we're going to call register uh from the Phoenix hotel package that sets up tracing and points it at Phoenix. One of the good things uh about the industry is that everybody has pretty much picked uh the same uh instrumentation standard. uh everything is built on open telemetry in the industry and on top of open telemetry there is a standard called open inference uh which arise takes care of um and everybody has already built the instrumentation into their frameworks that is why we are compatible with all of those frameworks all you have to do is use this registered command to say hey that instrumentation that you've already built into your framework turn it on and point it over here in this case you're just going to point it at your laptop uh and That's all you have to do.

现在，我们的代理发起的每次代理调用都会被追踪。嗯，哦，我刚才已经说过了。呃，这样做的好处是，我们不必为每一次调用都写一条日志，也不必为代理执行的每一件事都编写追踪记录。它会自动完成。有时我们会想添加额外信息，而这个框架完全支持这样做。嗯，呃，但我们不必对每件事都这么做。嗯，TypeScript 版本有一个你应该知道的注意事项：你应该手动使用 manually instrument，它会返回一份 instrument 的副本，而你应该使用这份副本。因为 Cloud Agent SDK 是一个 ES 模块，而 ES 模块的导出是只读的。因此，instrumentation 无法像在 Python 中那样替换你的 instrument，从而改变所有内容。Every agent call that our agent makes is now going to be traced. Um, oh, I said that already. Uh, so the good thing about this is that we don't have to write a log line for every single call. We don't have to write a trace for every single thing our agent does. It's just going to do it automatically. Sometimes we will want to add additional information and the framework absolutely supports that. Um, uh, but we don't have to do it for every single thing. Um there's one gotcha in the TypeScript version that you should know about uh which is that manually you should use manually instrument which will hand you back a copy of the instrument uh and you should use the copy of the instrument uh because the cloud agent SDK is an ES module uh and ES module exports are readon. So the instrumentation can't do the thing like it can do in Python where it swaps out your instrument and just changes everything.

你必须使用它提供给你的那个副本，其中已经替换了那个 instrument。嗯，如果你不使用它提供的副本，不会导致程序出错，只是不会记录任何内容。所以在这个仓库里，我们都从 instrumentation 文件导入 claude agent SDK，而不是直接从包中导入。现在，如果你打开 agent.ts，就能看到我们的 agents 选项。我们要开始使用 Claude Haiku F 4.5，因为它速度快、价格便宜，而且算是有用地“愚笨”。即使是相对简单的 agent，它也会犯一些很不错、很容易预测的错误。相比之下，前沿模型肯定也会犯错，但你得设计相当复杂的 agent 和相当复杂的任务才行，这会让演示变得更难。如果你以前没用过它，那么 allowed tools 会为它启用网络搜索，让它可以查找最新的金融数据。you have to use the copy that it gives you which has that instrument swapped out. Um it won't break it just won't log anything if you don't use the copy that it gives you. Uh so everywhere in this repo we import claude agent SDK from our instrumentation file never straight from the package. Uh now if you open agent.ts you'll see our agents options. Uh we're going to start using Claude Haiku F 4.5 uh because it is fast, it is cheap, and it is sort of usefully dumb. Um it makes uh mistakes that are nice and predictable even with a relatively simple agent. Um unlike the frontier models which will still definitely make mistakes, but you have to have quite a complicated agent and quite a complicated set of tasks. Uh and it makes it harder for the demo. Um if you've not used it before then uh allowed tools is giving it web search so it can find the current financial data.

呃，权限模式“不要询问”意味着它[哼笑]不会进行交互。它不会暂停下来问你问题，呃，因为没有人在那里回答“是”。嗯，这个选项之后会和一种故障模式有关。[哼笑]嗯，就像我提到的，另外三个选项会让你的笔记本电脑与代理隔离开来。Cloud Agent SDK 使用的引擎和 Claude Code 一样。呃，所以如果你是在 Claude Code 里运行它，它往往会使用你给自己的 Claude Code 副本配置的选项、工具和技能，而不是它自己的 Cloud Code 副本中的配置。呃，这往往会让事情变得极其不确定，而且毫无益处。呃，所以我在那儿加了几行代码，确保 Cloud SDK 运行在它自己隔离的版本中，而不是从你的数据里偷用你的密钥之类的东西。Uh and the permission mode don't ask means that it's [snorts] uh going to not be interactive. It's not going to pause and ask you questions uh because there's nobody there to say yes. Um that option is going to become relevant uh to a failure mode later. [snorts] Um, and the other three options, like I mentioned, they keep the your laptop out of the agent. The cloud agent SDK, uh, runs the same engine as cloud code. Uh, so if you're doing this in cloud code, it tends to pick up the options that you and the tools and the skills that you've given to your copy of Claude code instead of its copy of Cloud Code. Uh, which tends to make things extremely non-deterministic in an unhelpful way. Uh so I've uh thrown in a couple lines of code there that make sure that the cloud uh the cloud SDK is running in its own isolated version of uh the cloud SDK and not uh stealing your copies of of you know keys and things from your data.

嗯，[叹气]对了，顺便说一下，这些都不是 Phoenix 特有的。嗯，这只是同时使用 cloud SDK 和 cloud code 时会遇到的风险。在这个 SDK 中，一轮就是一次查询调用。你给它一个提示词。嗯，它会在智能体工作时流式返回消息。它会思考、调用工具、读取结果。嗯，它可能还会调用另一个工具，等它完成后，你会收到一条结果消息，其中包含最终文本和一个会话 ID。会话 ID 是我们继续对话的方式。如果我们把它作为 resume 参数传回去，下一轮就会记住上一轮看到的所有内容。嗯，所以我们可以先做研究，然后说：“嘿，根据这项研究写一份报告。”因为它知道我们还在同一个会话中。嗯，它也能找到上一轮刚刚完成的研究。然后我们把这两轮串起来。Um [sighs] right uh none of that is Phoenix specific by the way. Uh that is just a danger of using the cloud SDK at the same time as you're using cloud code. Um in this SDK one turn is one query call. So you give it a prompt. Uh it streams back messages as the agent works. It thinks it calls a tool. It reads the result. Uh it maybe calls another tool and when it's finished you get a result message with the final text and a session ID. The session ID is how we carry on the conversation. If we pass it back in as resume, the next turn remembers everything that the last turn saw. Uh so that's how we can do the research and then say, hey, write a report on this research because it knows that it's in the same session. Uh and it can find the research that the previous turn just created. Then we put the two turns together.

嗯，这样我们就能在一条 trace 中得到两个回合。嗯，先是研究回合，然后是写作回合，恢复研究回合的会话，这样写作者就能看到研究人员找到的所有内容。嗯，然后我们把整个流程包在 trace agent 里。嗯，我们这么做的原因是，它能为我们提供一个包含所有数据的父 span。如果不这么做，我们在 Phoenix 里就会得到两个、呃，两个同级 span。嗯，这也没问题，但读起来会稍微困难一些。呃，所以我现在只是在整理数据，让之后的工作更轻松。现在，打开 prompts.ts。你会看到里面有一堆提示词。它们都是故意写得很差的。嗯，研究指令只有一行，写作指令也只有一行。呃，我是故意这么做的，因为这样以后就有很多可以改进的地方。Uh so we get the two turns in one trace. Um first the research turn, then the writing turn, resuming the research turn session so the writer can see everything the researcher found. Um, and then we wrap the whole thing in trace agent. Um, the reason we do that, uh, is because it gives us one parent span that encompasses all of the data. If we didn't do that, we would get two, uh, two, uh, sibling spans in our Phoenix. Um, which would be fine, but slightly harder to read. Uh, so I'm just, uh, shaping the data now to make our lives easier later. Now, if you open up prompts.ts, ts. You'll see that uh there are a bunch of prompts in there. They are deliberately bad. Um there's one line of research instructions, one line of writing instructions. Uh and I did that on purpose because it gives us lots to improve later.

嗯，如果你在跟着一起编程，现在就可以开始试着改改东西了。尽管修改它们。可以加上“不要犯任何错误，否则我就给你 100 美元小费”之类的话。嗯，据说这类提示挺有帮助的。我希望在场的每个人最后都能得到一个略有不同的智能体，这样大家的体验就各不相同，交流心得也会更有意思。现在，如果你一直跟着做，已经安装好了所有东西，而且一切顺利，那你应该就能运行它了。你应该运行 `npm run agent -- Tesla "growth outlook"`。嗯，你可以用特斯拉，也可以用其他公司。它需要一两分钟，因为它确实得实际搜索网络，而你用的是会议 Wi-Fi。模型还会进行几轮推理。Um if you're coding along, now is the time to start playing with things. Feel free to change them. Add stuff like make no mistakes or I will tip you $100. Um allegedly those help. Uh what I want is everyone in the room to end up with a slightly different agent uh so that we get different experiences uh which will make comparing notes more interesting. So now if you've followed along and you've installed everything and everything is going fine uh you should be able to run it. You should run npm run agent dash uh Tesla and then growth outlook in quotes. Um, you can do it on Tesla, you can do it on some other company if you like. Uh, it will take a minute or two because it really does have to actually search the web and you're on conference Wi-Fi. Um, and the model will do a few rounds of reasoning.

嗯，在它为你运行时，发生的事情是：智能体会收到研究提示，然后判断自己需要搜索——这会触发一次工具调用。它读取搜索结果，判断结果是否足够，还是需要再次搜索。这就是让它成为智能体的关键。我们没有规定它需要搜索网络多少次，也没有规定它需要搜索哪些内容。智能体会自行做出这些决定。因此，我们的所有智能体在这些方面做出的决定都会略有不同。它们会使用略有不同的搜索词，也可能会搜索不同的次数。然后，完成搜索后，写作阶段就会开始，它会把所有内容整理成一份报告。所以，智能体采取的路径本来就是非确定性的。Um, what's happening here while that's running for you uh is the agent gets the research prompt. It decides that it needs to search uh which is a tool call. It reads the results uh decides whether it has enough results or whether it needs to search again. And that is the thing that makes it an agent. We haven't specified how many times it needs to search the web. We haven't specified what it needs to search the web for. The agent is making those decisions by itself. Uh so all of our agents are going to make slightly different decisions about those things. They're going to use slightly different search terms. Uh they're probably going to search a different number of times. Um, and then once it's done, the writing turn starts and it turns all of that into a report. So the path the agent takes is nondeterministic by by design.

嗯，如果你运行一次……如果你运行两次，这两份报告可能都没问题。嗯，也可能一份很好，另一份却是错的，而且你不能……你无法根据输入预测输出。呃，这就是为什么你需要评估。嗯，有人已经有实际的报告了吗？有人有吗？Um, both reports might be fine if you run it a if you run it twice. Um, or one might be great and one might be wrong and you can't but you can't predict that uh output from the input. Uh, which is why you need evals. Um, who's got an actual report yet? Anybody?

主持人：还在运行。[哼笑]Host: Still running. [snorts]

嘉宾：好，我们还有时间。我鼓起勇气，自己试着运行一下。我们会看到智能体遇到的一个问题是，我们让它写一份报告，却没告诉它要把报告写到哪里。于是它就默认应该把报告写入文件。好了，你看到了吧。我这边显示的是：我需要获得写入财务报告文件的权限。要我在这里显示报告吗？这完全帮不上忙。智能体没意识到它运行在一个没有权限的环境中，也没明白我其实是想让它把报告直接输出到命令行。嗯，这是因为我的提示词有歧义。不过有时它不会这样。有时它会自己弄明白，然后把报告给你。这就是我们所说的非确定性。Guest: Okay, we've got time. Let me be brave. And actually try running it myself. One of the problems that we're going to see that the agent runs into is that we told it to write a report, but we didn't tell it where to write a report. And so it assumes that it should be uh writing the report to a file. There we go. So you can see mine. I need permission to write the financial report file. Would you like me to display the report here? This is completely unhelpful. The agent does not understand that it is running in a no permissions environment and that I wanted the report actually output to the command line. Uh and that is because my prompt is ambiguous. Um but sometimes it doesn't do that. Sometimes it does just figure it out and give you the report. Uh, so that is the nondeterminism that we're talking about.

那么，我们来看看 Phoenix 从这一切中捕获了什么。这是我的 Phoenix 101。这是我的财务报告。这就是一条 trace 的样子。你可以看到财务报告代理正在运行，也可以看到 Claude 代理被调用了两次。其中一次是研究报告。你可以看到它内部进行了工具搜索，以了解它有哪些可用工具。接着你可以看到它发现自己有一个网页搜索工具，于是进行了三次不同的网页搜索。你可以看到输入和输出：具体的查询内容，以及每次工具调用返回的结果。然后你可以看到，这次它搞砸了。它试着写报告，写完后又想把报告写到我电脑上的一个文件路径中，但它说这行不通。这里重要的是，我们不需要在代理中添加任何日志记录。So let's see what Phoenix captured from all of this. Here is my Phoenix 101. There is my financial report. This is what a trace looks like. So you can see the financial report agent running and you can see the two invocations of the claude agent. One is the this is the research report. You can see tool searches inside of it to find out whether what it's got available to it. You can then see it finds that it's got a web search tool. So it conducts three different web searches. You can see the inputs and the outputs. Uh what the exact query is, what output it got from each tool call. Uh and then you can see uh that it messed up in this case. uh it tried to write it it wrote a report and then it tried to write it to a file path on my computer and said that it couldn't work. Um so uh the important thing here is that we didn't have to add any logging to our agent.

这里的重点是，所有这些关于运行情况的非常有用的信息，都自动呈现出来了，我们什么也没做。我们没有说“嘿，你们的工具需要埋点”，也没有说“嘿，你们的网络搜索需要埋点”。这一切都是 Cloud Agent SDK 自带的，因为它已经替我们做好了埋点。呃，你可以在 Phoenix 中点开任意一个跨度，查看大量关于运行情况的数据。嗯，之所以会发生这一切，也就是它尝试写入磁盘却失败的原因，是因为我们给它设置了“不要询问”选项。我们告诉它，你将以无人值守的方式运行，呃，不要问我任何问题。它确实没有询问任何问题就运行了，然后就搞砸了。嗯，但如果你没法查看这条追踪记录，你就不会知道这一点，对吧？The important thing here is that all of this showed up all of this very useful information about what's going on without us doing anything. We didn't say hey your tools need to be instrumented. We didn't say hey your web searches need to be instrumented. All of that just happens as part of the cloud agent SDK because it's already instrumented for us. Uh and you can click into any span in Phoenix and you can see an enormous amount of data about what's going on. Um uh and the reason that all of this happened, the reason that it's trying to write to disk and then failing is because we gave it that don't ask setting. We told it you're going to be running unattended. Uh don't ask me any questions. it ran without asking any questions and then it messed up. Um, but you wouldn't know that if you hadn't been able to look at the trace, right?

你只会得到一个神秘的输出，上面说出了问题。你可以在 traces 中看到工具调用失败了，也可以看到它试图执行某些未经授权的操作。嗯，这类信息就记录在 span 里。span 的类型会告诉你这是哪种步骤，比如 agent、工具调用、LLM 等。嗯，在工具调用、网页搜索之类的操作中，你可以看到我们的输入实际上是 JSON。这会成为我们第一个 eval 的依据，因为它为我们提供了结构化数据。嗯，你还会看到计时和 token 数量之类的信息。嗯，而且你的 agent 通常还会记录调用成本。稍后我们编写 eval 时，就会提取这些字段并对其进行评分。You just would have got a mysterious output where it says something was wrong. The traces are where you can see that the tool call is failing. The traces are where you can see it's trying to do something that it's not been given permission to do. Um, and that is the kind of stuff that lives in a span. Uh, so the span kind tells you what type of step it is. So it's an agent tool, an agent, a tool call, an LLM and so on. Um, and in a tool call and a web search and things like that, you can see that our input is in fact JSON. Um, and that's going to make our first eval uh because it gives us some structured data. Um, you also get things like timing and token counts. Um, and uh, often your agent will record what it costs. Um, when we write eval in a bit, these are the fields that we're going to pull out and grade.

所以，呃，你已经运行了一次，赢了一次，到目前为止一共运行了一次测试。呃，这点数据不足以得出任何结论。所以我现在想让你运行 `npm run batch`，它会连续运行 15 个查询。呃，这要花上一会儿。我会趁你运行的时候讲一讲。嗯，它一次运行五个。在 Wi-Fi 状况乐观的情况下，大概要三四分钟。嗯，现在就开始运行吧。[哼了一声] 嗯，如果因为任何原因失败了，不用担心。你之前加载的种子项目叫作 Phoenix 101 golden，里面有我早些时候在稳定的 Wi-Fi 环境下进行的 16 次运行记录。所以，如果你的批处理跑不起来，可以用那些代替。嗯，下面是一些查询示例。呃，有一些很明显的，比如 Apple、Nvidia 和 Amazon。然后还有一些我特意放进去的边缘案例，目的是让程序出错。So, uh, you've run one, you've won one, you've run one test so far. Uh, that is not enough data to draw any conclusions from. So what I want you to do now is run npm run batch uh which will run 15 queries in a row. Uh that will take a little while. So I'm going to talk while you're doing that. Um it runs five at a time. It takes three or four minutes optimistically on Wi-Fi. Um so start it now. [snorts] Um if it fails for whatever reason, don't worry about it. The seed project that you loaded earlier uh contains is called Phoenix 101 golden. Uh, and it has my 16 runs that I did earlier on reliable Wi-Fi on it. So, you can use those instead if you can't get your batch to run. Um, here are some examples of the queries. Uh, there are obvious ones like Apple, Nvidia, and Amazon. And then there are some edge cases that I put in there to make things fall over.

呃，所以有人让它比较苹果和微软，这比只分析一家公司要难。呃，有一个问题是关于 Rivian 的，呃，它不是上市公司。所以，关于它的数据比上市公司少得多。嗯，可口可乐和科技公司是截然不同类型的公司。呃，ZZQ 根本不是一个真实的股票代码，是我编出来的。嗯，这样做是为了看看智能体看到自己不认识的股票代码时会怎么做。它会不会凭空编出一家公司？会不会猜测你指的是哪家公司？还是会说出我希望它说的话，也就是这个股票代码不存在？嗯，其中一个问题是关于特斯拉的，但重点是：它现在值得买入吗？呃，因为现实中人们就是这样向智能体提问的。他们会用俚语，而且往往不会明确说明自己到底想要什么。Uh, so one asks it to compare Apple and Microsoft, which is harder than just company covering one company. Uh, one is about Rivian. uh which is not a public company. So there's much less data out there than it does about uh public companies. Um Coca-Cola is a very different type of company from a tech company. Uh and ZZQ isn't a real ticker at all. I just made it up. Um and that is to see what the agent does when it sees a ticker symbol that it doesn't recognize. Does it, you know, invent a company? Does it guess what company you mean or does it say what I'm hoping it says, which is that ticker doesn't exist? Um, one of them is uh about Tesla, but the focus is yo is it a buy right now? Uh, because that is how real people send queries to agents. They do it in slang. Uh, and they don't often specify exactly what it is that they wanted.

嗯，所以这些看起来都像是真实的查询。嗯，正如我说的，你要做的是在衡量之前先明确写下什么才算好。如果你还没定义“能正常工作”是什么意思，就不能说你的智能体不好用。那么，对我们的分析师来说，一份好的报告是什么样的？嗯，它应该使用真实的最新数据，应该针对正确的公司，而且应该忠实于它实际查到的内容。呃，就我而言，我希望它能告诉我们该怎么做。我想让它给出买入、持有或卖出这只股票的建议，而不只是提供分析。嗯，把这些要求写下来之后，它们听起来似乎显而易见；但在此之前，它们可能只是一些模糊的想法，或者团队里的不同人对一个好的金融分析师智能体应该做什么有不同的理解。Um, so these are all, you know, realish looking queries. Um, and what you have to do is, like I said, you have to write down what good means before you measure it. You can't say that your agent doesn't work if you haven't specified what working means. So, for our analyst, uh, what does a good report look like? Um, it should use real current data. It should probably be about the right company. It should stick to what it actually found. Uh, and for my purposes, I want it to tell us what we should do. I want it to say buy or hold or sell this stock. I don't just want it to give me analysis. Um, those sorts of requirements sound obvious once you've written them down, but before that, they could just be vague or different people on your team could have different ideas about what a good financial an analyst agent would do.

主持人：[嗤笑]Host: [snorts]

嘉宾：呃，把它们写下来，就能把模糊的失望转化为具体、可测试的事项，而今天我们要把每一项要求都转化为一个评测。界定什么是“好”，不只是工程师的工作。如果你要为自己的产品界定什么是“好”，就必须和组织里的非技术相关方沟通，因为他们是领域专家，知道什么样才算好。作为工程师，你会对什么造就一个好的智能体、什么样的输出才算好，有自己特定的猜测和偏见，而这些不一定总会和领域专家一致。所以，界定你的智能体应该是什么样子时，面向客户的同事也需要在场。呃，如果你没有现成的用户，也可以自己编一些出来。Guest: Uh so writing them down turns vague disappointment into specific things that you can test and we are going to turn every single one of those requirements into an eval today. Defining good is not solely uh an engineering job. If you are defining good for your product, you have to talk to the non-technical stakeholders in your organization because they are the domain experts who are going to know what good looks like. You as an engineer are going to have specific guesses about what uh guesses and biases uh about what makes uh a good agent and what makes good output and you are not always going to agree with the domain experts. So uh the customerf facing people need to be in the room when you define what your agent looks like. Uh it's also great uh if you don't have users that you can make some up.

嗯，你可以让 LLM 生成大量测试查询，并有意让它们有所变化。比如，“研究特斯拉的财务表现”是一种问法，“特斯拉股票现在是什么情况？”是另一种问法，“特斯拉现在值得买入吗？”则是第三种。然后你可以让人来查看这份列表，因为 LLM 生成的查询是为初始数据提供种子的有效方式。它们并不完美，主要是因为它们相对保守，不像人类用户实际提出的问题那样天马行空。人类用户会问：“你知道，告诉我我孩子的健康出了什么问题”，而不是问你要一位财务分析师；而你的智能体必须能够应对这类问题。嗯，不过合成数据是很好的起步方式。等你开始运作，并有了真实用户和真实流量之后……uh you can get an LLM to generate lots of test queries and vary them on purpose. So research Tesla's financial performance is one way to ask what's going on with Tesla stock is another and yo is Tesla by right now is a third. Um then you can get a human to look at the list because LLM generated queries they are a useful way of seeding your initial data. They are not perfect u mostly because they are relatively conservative. They don't go as far a field as human users tend to actually go. Human users are like, you know, tell me what's wrong with, you know, my child's health instead of asking you for a financial analyst and you your agent has to have something to do with that. Um, but synthetic data is a great way to get started once you've got started and you have real users with real traffic.

真实流量是测试智能体的黄金标准。你要尽可能快地把它纳入数据集。嗯，在我们查看数据之前，我先补充一点——我希望数据现在正传到你们的笔记本电脑上。我刚才手动演示了追踪，因为你应该了解它的工作原理；但就像我说的，你也可以让你的编程智能体完成这一切。嗯，Phoenix 有一个非常好用的 CLI。如果你在项目中运行它的 setup 命令，它会让你的编程智能体为应用添加追踪。呃，它会一直运行，直到你的应用安装好追踪功能；这样你就不用费心弄清楚如何添加注册命令之类的东西了。它适用于每一个……Real traffic is uh the gold standard for testing an agent. you want to get it incorporated into your data set as fast as you possibly can. Um, one aside before we look at the data that I'm hope hoping is arriving in your laptops right now. Um, I walked through tracing by hand because you should understand what it's doing, but you can, like I said, you can also get your coding agent to do all of this. Um, Phoenix has a really good CLI. Uh, and if you run it setup command in your project, it will give your coding agent um, it will get your coding agent to add tracing to your app. uh and it will work until your your app uh has tracing installed and you don't have to figure out how to add the register command and all of that stuff. It will work in every

步骤 2：读取你的追踪记录Step 2: Reading your traces

语言。呃，真的很棒。所以第二步是读取你的数据。现在有谁手头有数据？举个手。一个人。还有别人吗？哦，有。不错。好吧，好几个人。太棒了。呃，在写任何评测之前，你必须先读读你的调用轨迹。把十几条甚至更多的轨迹从头到尾读一遍。呃，输入是什么？输出是什么？具体哪里出了问题？这听起来很手动、很老派，但这是你能做的一件价值非常高、非常高的事情。呃，在构建智能体时，你必须亲自检查正在发生什么，这样才能发现那些你没想到、甚至不知道自己不知道的问题。呃，Anthropic 团队会定期坐下来，手动阅读评测记录，也在内部构建了大量工具，让这件事更容易进行。[哼笑] 呃，如果你还没弄清楚智能体会如何失败就先把一切自动化，最后你衡量的就会是那些容易衡量的东西，而不是那些真正重要的东西。language. Uh it's really neat. So step two is reading your data. Who has any data at this point? Show hands. One person. Anybody else? Oh yeah. Cool. All right. Several people. Fantastic. Uh you have to read your traces before you write a single eval. Um reading a dozen or more of them end to end. Uh what was the input? What was the output? Um what specifically is wrong? This sounds manual and old-fashioned, but it's an incredibly high value high value thing that you can do. Um, when you're building an agent, uh, you have to manually inspect what is going on so that you can find things that you the unknown unknowns that you weren't expecting. Uh, Anthropics team regularly sits down and reads eval transcripts by hand uh, and have built a bunch of tooling internally to make that easy to do. [snorts] Um, if you automate before you understand how your agent fails, you're going to end up measuring what's easy to measure instead of what actually matters.

嗯，所以你应该阅读 traces，而我们马上就要这么做了。但阅读 traces 无法规模化。自己读 16 条 trace 完全没问题。呃，但读 1600 条 trace 会给你带来一大堆非常棒的数据，也会耗掉你一周的时间。嗯，不过事实证明，编码代理特别擅长阅读 traces。呃，所以你可以让编码代理查看你的 traces，找出其中奇怪的地方，让它们编写一些代码。[嗤笑] 呃，但它们不一定总能找到你预期的东西。因此，把手动检查作为一种抽样方法，始终是个好主意。所以，嗯，如果你在用编码代理帮你做事，就需要安装 Phoenix 插件或 Phoenix 技能，具体取决于你使用的是哪种编码代理。Um, so you should read traces and we're about to do that. But reading traces doesn't scale. Reading 16 traces yourself is absolutely fine. Uh, but 1600 traces uh is going to give you a bunch of really great data and it's also going to take a week out of your life. Um, but it turns out that coding agents are aces at reading traces. Uh, so you can point your coding agent at your traces and get them to look for things that are weird in your traces. Get them to code stuff up. [snorts] Uh, but they're not always going to find what you expected. So manual inspection as a sampling technique always a really great idea. So, um, if you are using if you're getting the, uh, agent to do stuff for you, you want to install the Phoenix plugin or the Phoenix skill depending on which coding agent you're use you use.

这是设置 README 里的最后一步，如果你还留着它的话。呃，所以你可能已经装好了。嗯，对于 Cloud Code，我们有一个插件，包含两个命令。呃，Cursor 和 Codeex 也有对应的插件。呃，对于其他任何编程代理，还有一个 skill 文件。嗯，这个插件会把你的编程代理连接到 Phoenix。呃，它已经指向 localhost 606，所以不需要进行任何配置。嗯，Phoenix 本身会向你的代理提供一个名为 Phoenix error analysis skill 的 skill，里面有关于如何正确进行错误分析的说明。嗯，安装应该只需要大约 10 秒。呃，我就不在这儿等你安装了。嗯，现在打开你的编程代理，让它进行错误分析。呃，如果你已经装好了的话。之前谁装过这个 skill？It was the last step in the setup readme if you've still got that. Uh, so you may have it already. Um, for cloud code, we have a plugin which is two commands. Uh, there are also plugins for cursor and codeex. Uh, and for any other coding agent, there is a skill file. Um, the plug-in connects your coding agent to your Phoenix. Uh, and it already points at localhost 606, so there's nothing to configure. Um, and Phoenix itself will hand your agent a skill called the Phoenix error analysis skill with instructions on how to do error analysis properly. Um, it should take about 10 seconds to install. Uh, and I'm not going to hang around waiting for you to install it. Um, now open your coding agent and ask it to do the error analysis. Uh, if you've got it. Who's got who's got the skill installed from previously?

太棒了。我又要来为难你了。你有没有让你的编码代理帮你做过事？Fantastic. I'm going to pick on you again. Can you have you asked your coding agent to do things?

主持人：好，太棒了。嗯，提示词就在幻灯片上。告诉它 Phoenix 在哪里。让它读取这些 trace。归纳出失败类别，嗯，统计各类别的数量，然后把每个类别写回 Phoenix 中对应的 trace，作为注释。注释只是一段附加在 trace 上、便于人类阅读的文本。嗯，它们可以为 trace 添加额外信息，帮助你或你的智能体弄清楚当时发生了什么。嗯，你的插件会请求权限来使用 Phoenix 的工具，所以你应该点“是”。嗯，我运行时花了大约 9 分钟。嗯，所以我暂时还不打算点你来回答。嗯，另外，如果它询问是否要把笔记写入代码库，嗯，没问题。让它把内容写入磁盘就好。现在，我们再看看 Phoenix。嗯，我打算冒险试着回忆一下搜索的语法。Host: Okay, great. Um, the prompt is on the slide. Tell it where Phoenix is. Tell it to read the traces. Come up with categories of failures. uh count them up and write each category back onto its trace in Phoenix as an annotation. An annotation is just a human readable string of text attached to a trace. Uh they are a way of adding additional information to traces so that either yourself or your agent can figure out what was going on. Um your your plug-in is going to ask permission to use Phoenix's tools. So you should say yes. Uh when I ran it, it took about 9 minutes. Uh, so I'm not going to pick on you quite yet. Um, uh, also if it asks to write notes into the repo, uh, that's fine. Just let it write stuff to disk. Now, let's look at Phoenix again. Um, I'm going to very dangerously try and remember my syntax for search.

哈。好吧，这些都是我的运行记录，呃，也就是财务报告。呃，我们可以点开其中任意一份。看看能不能找到有意思的。呃，ZZQ，我对 ZZQ 的输出是什么来着？ZZZQ 似乎不是有效或活跃的金融股票代码。太好了。智能体做到了我们预期它会做的事。嗯，那可口可乐呢？首先，它先道了歉。它正在受限模式下运行，无法直接写入文件。我们肯定得解决这个问题。呃，接着它得出了结论：可口可乐是一项高质量、低风险的股息投资。KO 对于风险厌恶型投资者来说，是一项出色的长期收益型投资。这下可有意思了。举个手，谁觉得这是一份可供采取行动的报告？好吧，我已经先给你们打好预防针了。Ha. Okay, so these are all of my runs uh that are the financial reports. Uh we can click into individual ones of them. Let's see if we can find the interesting ones. Uh ZZQ, what was my output for ZZQ? ZZZQ does not appear to be a valid or active financial ticker symbol. Hooray. The agent did the thing that we were expecting it to do. Um how about uh Coca-Cola? First, it starts with the apology. It's running in restricted mode and it cannot write files directly. We're going to have to fix that definitely. Uh then it comes to a conclusion. Coca-Cola represents a high quality, lowrisk dividend investment. Um KO is an excellent long-term income vehicle for riskaverse investors. This one's going to be fun. Show of hands. Who thinks that this is an actionable report? Okay, I primed you already.

嗯，它没有说买入、卖出还是持有。它只说这是一项不错的投资，但没有说明你应该怎么做。嗯，我会把这些报告的可执行性标准定为：它应该告诉我这只股票该买入、卖出还是持有。我不会让它含糊其辞。这组报告里有不少都各有各的问题。嗯，你可以看到这份也有同样的问题。它试图写入磁盘，但失败了。嗯，关于 Rivian 的跟踪记录，我记了一笔，说 Rivian 很有意思。就在这里。不知道为什么，里面全是表情符号。嗯，它一开头就问：“你想要……是的。你想要创建报告文件吗？”之前我实时运行时就看到过这种情况。嗯，它在要求我做事，但它不能这么问。我告诉过它不允许这么做，可它还是在要求我做事。Uh it does not say buy, sell, or hold. So it just says that it's a good investment. That doesn't say what you should do. Uh I'm going to make my standard of actionability for these reports that it should tell me whether I should buy, sell, or hold the stock. I'm not going to leave it to be ambiguous. Um a whole bunch of them do different things in this set. Uh you can see this one had the same problem. It tried to write to disk and it was and it was and it failed. Um, uh, the Rivian trace, I made a note that Rivian was interesting. There it is. Uh, it's full of emojis for some reason. Um, it opens by asking, "Would you like to Yes. Would you like to create the report file?" We saw that earlier when I ran it live. Uh, it's asking me to do things and it can't ask. I told it I told it it's not allowed to and it's asking to do things anyway.

嗯，这些都是不同的失败方式。这里面很多都算是失败，也有很多还算可以。不过，嗯，然后还有特斯拉。Um, all of these things are uh different ways of failing. Lots of these are failures. Lots of these are sort of okay. Um, but then uh there's Tesla.

分析师共识达到 116% 的漏洞The 116% analyst consensus bug

特斯拉尤其令人着迷。呃，具体来说，是那个“特斯拉现在值得买入吗？”的问题。在这个例子里，呃，它说分析师的一致意见是买入。它说，50%的人建议买入，52.6% 6%的人建议持有，13.1%的人建议卖出，加起来就是116%的分析师在表达意见。它完全没有发现自己的数学计算错得离谱。它做的是两次不同的网页搜索：一次搜到50%的人持某种看法，另一次搜到13%的人持某种看法。它把这两次网页搜索的结果拼在了一起，尽管它们并非来自同一组数据，于是得出了116%。但它还是信心十足地写下了“116%的分析师持某种看法”。这正是你会看到的那种智能体式失败。呃，它算错了，却还信心十足地给出错误答案。嗯，如果你点开这些网页搜索结果，就能看到它具体从哪些地方获取数据，以及它是如何把这些内容拼到一起的。Tesla is particularly fascinating. Uh specifically the yo is Tesla a buy right now one. So in this one uh it says that the analyst consensus is a buy. Uh it says 50% say buy, 52.6% 6% say hold and 13.1% say sell, which adds up to 116% of analysts saying things. And it did not catch that its math was completely wrong. What it did was it did two different web searches. One where it found 50% of people saying something, another one where it found 13% of people saying something. It took those two web searches and it munched them together even though they were not from the same data set and it came up with 116%. But it confidently wrote down that 116% of analysts say something. This is exactly the kind of agentic failure that you see. Uh it is confidently wrong having done uh bad math. Um if you go into these web searches, you can see the specific places where it got the data and how it uh put those things together.

所以，当你发现一个失败时，必须追问它为什么失败。“答案错了”只是症状，不是原因。通过追踪记录，你可以看到实际给出的答案是什么。它是不是检索到了糟糕的搜索结果？那就是检索失败。但这里并非如此，每条搜索结果都没有问题。它是不是拿到了正确的数据，却得出了错误的结论？那就是推理错误。这里确实是这种情况，对吧？它看到了两条不同的数据，却没有正确地推理。它有时也会完全编造数字，那就是幻觉。有时它回答的并不是你提出的问题，那就是范围问题。每种原因都有不同的修复方法。但如果你不知道原因，就不知道该如何修复。这就是为什么阅读追踪记录很有用。Um so when you find a failure, you have to ask why it failed. the answer was wrong is a symptom. It's not a cause. Um with the traces you can see what the actual answer was. Did it get bad search results? That would be a retrieval failure. That is not what happened here. This each search result was fine. Uh did it have the right data but the wrong conclusion? Then that is a reasoning error. That is uh definitely what happened here. Right? It saw two different bits of data and it did not reason correctly about them. Um sometimes it will make numbers up completely. that is a hallucination. Uh and sometimes it will answer a question that isn't the question you asked. So a scope problem. Each cause has a different fix. Uh but if you don't know the cause, you don't know the fix. Which is why reading traces is useful.

嗯，这件事有个特别好的方法，那就是我们切回幻灯片。呃，你应该先写笔记，再归类别。所以，通读你所有的跟踪记录，用人类能理解的语言记下你看到的情况。比如，这样不好，因为它会做这件事；这样好，因为它会做那件事。呃，这叫作开放式编码。呃，这是从机器学习借来的另一个术语。你只是在说，呃，你看到了什么。完成这一步之后，呃，你就可以进行所谓的轴心编码：通读你写下的所有笔记，并把它们归纳成不同类别。作为工程师，你非常容易忍不住提前猜测会有哪些类别，先想出一些类别，然后一开始就把内容归进这些类别里，呃，因为这样省事一些；但你会做得更差，因为你无法事先知道会出现哪些失败节点。Um there's a really great way to do this which is let's switch back to slides. Uh you should write notes first and categories second. So look through all of your traces and just write down in human language what you see happening. So this is bad because it does this. This is good because it does that. Uh that is called open coding. Uh another term borrowed from ML. You're just saying uh what you saw. Once you've done that uh you can then do what is called axial coding where you look through all of the notes you've written to yourself and you group them into categories. It is very very tempting as an engineer to g try and guess in advance what the categories are going to be and come up with categories and just put things in categories from the get-go uh because that is less work but you will do worse at it because you will not know in advance what your failure nodes are going to be.

嗯，所以你要选的类别，要么太狭窄，要么太宽泛，要么根本不是实际出错的那类东西。[嗤笑] 嗯，既然说到这个，我之前讲过如何把代码评测、LLM 评审评测和人工评测叠加起来。嗯，这是一种从安全领域借鉴来的模型，叫作“瑞士奶酪模型”。你拥有的每一层防御都会有漏洞；你做的每一项评测也都会有盲点。你要做的是尽可能多地叠加评测，让错误很难一路穿过所有评测。嗯，因为它们都有漏洞，但漏洞的位置并不相同。所以，如果你有足够多的评测，也有足够多种不同类型的检查，它们就能层层叠加，让整体可靠得多。Uh so the categories you're going to pick are going to be either too narrow or too broad or just not what not the kind of stuff that actually went wrong. [snorts] Um, while we're talking about this, I talked about how you stack up uh um Kod valves and LLM judgy valves and human evals. Uh this is uh a model barred from the security world called the Swiss cheese model. Um every layer of defense that you have is going to have holes in it. your your every eval you have is going to have a blind spot. What you want to do is uh stack as many eval as you can on top of each other such that it is very hard for an error to get all the way through all of them. Um because they've all got holes but the holes are not in the same place. So if you can uh if you have enough evals, if you have enough different types of check going on, they will stack up on top of each other and be much more reliable.

嗯，代码评测可以发现格式问题，但会漏掉语义问题。LM 评审可以发现推理上的漏洞，但会漏掉一些细微的幻觉。人类通常能发现那些细微问题，但不可能查看每一条轨迹。所以，没有任何一种方法能发现所有问题。嗯，今天我们要构建几个小型评测，而不是一个庞大的超级评测；后者本身也是一种常见的失败模式。刚开始接触评测时，人们往往会想：“我要把所有可能出错的地方都考虑到，然后全塞进一个大型 LLM 评审里，这样它就能发现所有问题了。”而这是一种非常不可靠、也非常复杂的做法，根本解决不了问题。好了。嗯，这个智能体可能会调用……这个智能体会做很多事情。这就是智能体的输出看起来是什么样的。Um so code evals can catch formatting problems, but they will miss problems with meaning. LM judges will catch reasoning gaps, but they will miss some subtle hallucinations. Humans will often catch the subtle stuff, but they can't look at every single trace. So no single method uh is going to catch everything. Um, so today we're building several small evals, not one giant super eval, which is also another failure mode. When people first get into evals, they're like, I'm going to think of every possible thing that can go wrong and I'm going to cram it into one big LLM as a judge, and it's going to find everything. And that is uh a very unreliable and very complicated way of not solving the problem. So, whoops. Uh so the agent may call uh the agent is going to do a whole bunch of stuff. This is what the agent output looks like.

我已经在 Phoenix 里给你看过了。嗯，我已经给你看过它得出的各种结果了。嗯，在这个例子里，我的编码智能体替我查看了所有追踪记录。它得出了开放式编码和主轴编码的结果，还归纳出一个类别，叫作“框架泄漏”。它决定用“框架泄漏”来称呼这样一种情况：它不停地要求我们允许它做一些它本来无权做的事。嗯，所以，这种情况发生的次数实在太多了。[哼笑] 嗯，它还发现了不少其他问题。它发现了幻觉，也发现了推理错误，比如特斯拉那个例子，它把百分比加总时算出了过高的结果。嗯，如果你自己运行过这份数据，并且你的编码智能体也给出了自己的分析，那么你得到的分析和标签集会有所不同。I showed you already in Phoenix. Um we've I've already showed you uh the various things that uh it came up with. Um, in this case, uh, my coding agent looked through all of my traces for me. It came up with, uh, both, uh, open coding and aial coding, uh, and it came up with a category, uh, called harness leakage. Uh, harness leakage is what it decided to call that thing where it's constantly asking us to give it permission to do things that it doesn't have permission to do. Um, so, uh, that's a ton of that's a ton of times for that specific thing to happen. [snorts] Um, there are also a bunch it also found a bunch of other things. It found hallucination. It found uh, reasoning errors like the Tesla one where it had too many it added up percentages. Um if you're doing if you've run this data yourself and you've come up with your coding agent has uh produced its own analysis, you're going to get a different analysis and a different set of uh labels.

呃，因为我们的智能体都不一样。谁已经让自己的编程智能体完成分析了？还有人在运行吗？Uh because all of our agents are different. Who has got their coding agent to complete an analysis yet? Anybody still running?

嘉宾：好吧，我们继续。希望你能收集到一些数据；如果没有，也可以直接用我的种子数据。呃，拿到数据之后，你就可以统计各类问题出现的次数。最常见的失败类型会是你首先关注的对象，但最常见的失败未必就是你要优先修复的。你要考虑的是出现频率和严重程度。比如，呃，如果它有 40% 的时候会产生幻觉，那就是个问题。如果它有 1% 的时候会给用户提供制造炸弹的说明，那么即使这种情况只占 1%，你也要立刻修复，因为这绝对不是金融分析师该做的事。呃，所以要用出现频率乘以严重程度，来确定优先级。好，希望你们的智能体能赶上我们讲到的进度，不过我接下来要……Guest: All right. Well, we'll keep going. Hopefully, you'll get some, but if not, you can just use my seed data. Uh once you've got data, you can count the categories. The most common failure is going to be where you look first, but the most common failure is not necessarily the first thing that you're going to fix. What you want to do is address uh frequency and severity. So if uh you know 40% of the time it hallucinates, that's a problem. If 1% of the time it gives your user instructions on how to build a bomb, you fix that immediately even though it happens only 1% of the time because that's absolutely not what a financial analyst is supposed to be doing. Uh so you multiply frequency by severity to get your priority. All right, so hopefully your agents will catch up to where we are, but I'm going

第 3 步：代码评估Step 3: Code evals

继续往下。第三步是代码评估。代码评估比较简单。不需要模型，也不需要调用 API，只需要 TypeScript。嗯，我能想到的最简单且实用的评估，就是检查报告里是否提到了我们询问的股票。我们的智能体应该分析一家特定公司。如果它确实在做这件事，就应该提到所分析公司的股票代码。嗯，如果我问的是特斯拉，它写的却是微软的内容——它至少有一次确实这么做过——那么这份报告就完全没有用。所以，报告里要么有股票代码，要么没有。用 LLM 来检查这一点完全是小题大做。嗯，要运行这项检查，我们首先需要从 Phoenix 中取出数据，这样才能进行分析。to keep going. Step three is Kodi valves. Kodi valves are the simple ones. There's no model, there's no API call, there's just TypeScript. Uh the simplest useful eval that I could think of was does the report actually mention the stock that we asked about. Our agent is supposed to analyze a specific company. If it's doing that, it should mention the ticker symbol of the company that it is analyzing. Uh if I ask about Tesla and it writes about Microsoft, which is a thing that it really did on at least one occasion, um that is a completely useless report. So either the ticker report, the ticker is in the report or it isn't. and using an LLM to check that would be complete overkill. Uh so to run that check, the first thing that we need to do do is get our data out of Phoenix so that we can run the analysis.

具体就是这样做的。嗯，我们会为项目查询 spans，并将父级 ID 设为 null，这意味着只获取没有父级的 spans。嗯，所以这些就是顶层 spans，也就是我在 traces 中展示给你看的那些。嗯，代码库会通过一个名为 get agent runs 的小型辅助函数来执行这项操作，它会返回每个运行的 span ID、输入和输出，以及研究轮次的输出，呃，后面我们会用到。呃，接下来是 ticker 检查。嗯，create evaluator 来自 Phoenix evals 包。你只要传入一个普通函数就行，不需要自己构建 evaluator 函数。呃，你只需要告诉它这是一次代码评估，因此不会涉及 LLM。这个函数会接收输入和输出。它会从中提取我们询问的 ticker，然后检查每个 ticker 是否都出现在报告中。This is how that is done. Um we ask for spans for our project and parent ID null means that only spans that have no parent. Um so those are the top level spans. So the ones that I was showing you in my traces. Um, the repo runs this in a little helper called get agent runs, which hands back each run span ID, its input, and its output, plus the research turns output, uh, which we're going to need later. Uh, and then this is the ticker check. Um, create evaluator comes from the Phoenix evals package. You can just give it an ordinary function. You don't have to build an evaluator function yourself. Uh, all you have to do is tell it that this is a code eval so that no LLM is involved. The function gets the input and the output. It uh splits out the tickers we asked about. Uh it checks that each one appears in the report.

呃，它会返回一个标签和一个分数。如果缺少什么，它也会指出是哪个股票代码。所以失败原因一目了然。嗯，我之前提到的结构化输入——我们发送给工具的输入是 JSON——在这里帮了我们很大的忙，因为这样编写代码评估就很容易了，毕竟输入本来就是 JSON。这样我们已经知道哪个字段是股票代码，哪个字段是报告的描述，也就是报告应该写些什么。[哼笑] 嗯，如果能把输入整理成结构化数据，你的评估就会很容易处理。但如果做不到，你就得再调用一个 LLM 来读取输入、弄清楚输入是什么，然后把它传给评估。呃，接下来我们运行它，并把结果记录回 Phoenix。Uh and it returns a label and a score. And if something's missing, it also it also says which ticker. So the failure explains itself. Um the structured inputs that I talked about earlier, how we the input being sent uh to our tool was JSON, that is helping us out here because it's making it really easy to write this Kodi val because it's already JSON. So we already know uh which thing is the ticker and which thing is the uh description of the uh report that it should be writing. [snorts] Um so if you can get uh your inputs to be structured data that's going to make your evals really easy to work with. But if not you're going to have to get another LLM involved to read your input figure out what the input was and then pass it to your eval. Uh then we run it and we log the results back to Phoenix.

所以，这个脚本会在每次智能体运行时执行评估器，然后调用 log span annotations，将每个结果关联到对应的 span。嗯，我们今天运行的每项评估都会采用同样的模式：获取运行记录、评估它们，然后把结果记回 Phoenix。呃，所以这里的命令是 `npm run eval code`。我看看能不能运行成功。好了，运行成功了，而且几乎是瞬间完成的，因为这是代码评估。它们不需要等 LLM 做任何事，只会 [喷鼻息]，呃，然后它报告了 15 项通过和两项失败。一项是 Tesla 的报告里漏掉了 Tesla，另一项是 Amazon 的报告里漏掉了 Amazon。呃，所以这就是我们能做出的最简单的检查：LLM 有没有真的针对我们要求它分析的内容写报告？写报告……而它已经发现了一些错误。So the script runs the evaluator on every agent run and then it calls log span annotations which attaches each result to its span. Um that's going to be the same pattern for every single eval we run today. You get the runs uh you evaluate them and then you log the results back to Phoenix. Uh so uh the command here is npm run eval code. Let me see if I can get that to work. All right, it worked and it ran almost instantly because it's Kod valves. They don't have to wait for an LLM to do anything. They just go [snorts] uh and it has reported 15 fastes and two failures. One is Tesla is missing Tesla and one is Amazon is missing Amazon. Uh so uh this is the dumbest possible check that we could have made. Did the LLM actually write a report about the thing that we asked it about? Write a report about and it has already found some errors.

它确实发现过一些写错主题的情况。比如我知道，Amazon 的问题是，它询问的是 AWS 的盈利能力，结果智能体就把注意力都放在 AWS 上，写了一份只谈 AWS 及其业务的报告，而不是谈 Amazon——也就是那家购物公司——做什么的。嗯，所以这个测试值得做，而且运行起来完全不花钱。所以，代码评测可不是玩具。那么，输出能解析为 JSON 吗？有没有超过长度限制？有没有避开你绝对不想看到的措辞，比如“作为一个 AI 语言模型”？嗯，而且检查方式不一定非得是字符串匹配。所谓代码评测，就是评分过程具有确定性。所以，你的代码评测可以查询数据库，确认状态是否正确；也可以检查工具调用是否完成了预期的工作。It has found cases uh where it wrote about the wrong thing. I know for instance that the am the problem with Amazon is uh that it asked it about AWS profitability and the agent got focused on AWS and wrote a report that was only about AWS and what AWS was doing uh and not about what Amazon, you know, the shopping company does. Um so this is a test worth doing and it cost you absolutely nothing to run. So Kod valves aren't toys. So, did the output parses JSON? Is it under the length limit? Does it avoid phrases you never want to see like as an AI language model? Um, and it doesn't have to be a string check. Kodval just means that the grading is deterministic. So, uh, your Kodval can be querying your database to make sure that the state is correct. It can make sure that the tool calls did what the tool calls were supposed to do.

呃，它可以调用 API，再核实一下智能体认为的股价是否确实如此。嗯，对于相同输入每次都能给出相同答案的任何东西，都可以成为代码评估。因此，代码评估可以做得相当高级。不过，有一点需要注意：评估结果，而不是评估过程。这里有个很容易犯的错误，就是检查智能体是否采取了特定步骤。它是不是先调用了搜索工具，然后又调用了另一个工具？嗯，Anthropic 建议不要采用这种过于死板的方法。正如我们之前看到的，智能体可以很有创造性，也可能以意想不到的方式完成任务。你不应该假设它总会完全按照你的做法来。因此，你不应该评估它采取的过程，而应该评估最终结果是否良好。uh it can call an API and double check that the stock price uh that the agent thinks is happening is really the stock price. Um anything that gives the same answer for the same input every time can become a Kodi val. So your Kodi valves can be quite advanced. Um but one thing you should be wary of is grading the outcome and not the path. Uh there's a tempting mistake here um to check that the agent took particular steps. Did it call the search tool first and then this other tool? Uh, Anthropic advises that this is too rigid an approach. Uh, like we saw earlier, the agent can be creative. It can do things in an unexpected way. You don't want to assume that it's always going to do exactly what you would have done. Uh, so you don't want to grade the path that it took. You want to grade whether or not the outcome was good.

嗯，我们的工单检查并不在乎智能体是如何找到数据的，它只关心报告中是否包含正确的公司，这对很多评估来说已经足够了。之后我们会讨论轨迹级评估，届时我们确实会想要评判执行路径，但那是完全不同的一回事。um our ticket check doesn't care how care how the agent found its data right it just cares that the right company is in the report and that is the right level for a lot of evals we will talk later about trajectory level evals where we do actually want to grade the path but that is a very different uh kettle

步骤 4：将 LLM 用作评判者Step 4: LLM-as-a-judge

……鱼类方面的内容。这就把我们带到了第四步：让 LLM 充当评审。我们将从 Phoenix 内置的评估功能开始。代码可以检查报告中是否包含股票代码，但无法判断分析质量如何。数字准确吗？报告完整吗？推荐理由充分吗？这些问题关乎报告的含义，而非报告的文本内容。要回答这些问题，你需要某种具备判断能力的东西。LLM 评审由三部分组成：评审模型，也就是负责评判的语言模型；评分标准，也就是告诉它什么样的结果才算好的提示词；以及数据，也就是它要评分的示例。Phoenix 将这三部分分开，因此你可以自由组合搭配，也可以用不同的模型尝试同一套规则。of fish so that brings us to step four the LLM as a judge uh and we're going to start with Phoenix's builtin evals Um code can't check whether code can check whether the ticker is in the report but it can't tell you if the analysis is any good. Uh are the numbers accurate? Is the report report complete? Is the recommendation well reasoned? These are questions about the meaning of the report not the textual content of the report. And for those you need uh something that has judgment. Uh an LLM judge has three parts. It has a judge model which is the LM that is doing the judging. Uh it has a rubric which is the prompt that tells it what good looks like. And then it has the data which is the examples that it's grading. Phoenix keeps those three separate so you can mix and match. You can try the same rules with different models.

你可以用不同的规则试试同一个评估器。对你来说，好消息是 Phoenix 内置了许多评测，所以你不必总是自己编写评测，至少一开始不用。比如，有准确性评测，用来检查答案是否准确；还有忠实度评测，用来检查答案是否忠于源材料。我们马上就会看到这两者的区别。还有完整性、幻觉、拒绝回答、PII 和毒性等评测。许多智能体需要检查的事项都已内置。对于智能体，还有用于评估工具使用情况的评测：它有没有选对工具？有没有传入正确的参数？有没有合理地处理结果？这些评测并不是什么神奇的东西，它们只是我们已经编写、经过实战检验，并确认能为你所用的提示词。You can try the same judge with different rules. Uh the good news for you is that Phoenix ships a bunch of built-in evals. So you don't always have to write your own evals, at least not right off the bat. Uh there's correctness which checks if the answer is accurate. Uh there's faithfulness which checks if it sticks to the source material. Uh we're going to see the difference between those two in a second. Uh there's completeness, hallucination, refusal, PII, and toxicity. Lots of things that lots of agents need to check are built in. Uh and for agents, there are evals for tool use. Did it pick the right tool? Did it pass the right arguments? Um did it do something sensible with with the result? Uh these eval are not magical. They are just uh prompts that we've already written and battle tested and made sure work for you.

好，我们来看看评审模型。嗯，Phoenix 的评估在 TypeScript 中使用 Vercel AI SDK 与模型交互。因此，AI SDK 支持的任何模型都可以作为评审模型。嗯，这里我用的是 Claude Sonnet 5.5。智能体运行在 Haiku 上。正如我说的，它便宜又快。我们的评审模型运行在 Sonnet 上，因为你希望评审模型比智能体本身更强大。所以，可以用 Sonnet，也可以用 Opus。或者，更好的做法是，使用完全不同服务提供商的模型。如果你有 OpenAI 密钥，就让 OpenAI 来评审 Claude 的工作。因为 Claude 有一种所谓的“同模型偏差”。所以，Claude 往往更喜欢其他版本 Claude 的输出，而不是 OpenAI 或其他服务提供商的输出。嗯，最后这一行是为了关闭评审模型的追踪，以防你想知道原因，因为我们已经在所有地方都启用了追踪。Uh so let's find the judge model. Um uh Phoenix evals in Typescript uh use the Versel AI SDK to talk to models. So any model that the AI SDK supports can be a judge. Uh in this case I'm using Claude Sonnet 5.5. Um, the agent runs on Haiku. Like I said, it's cheap and fast. Our judge is running on Sonnet because you want your judge to be more capable than the agent itself was. So, give it Sonnet or give it Opus. Uh, or give it uh even better um give it a model from uh a different provider entirely. If you've got an OpenAI key, get OpenAI to judge the work of Claude. Uh because Claude has what is known as uh same model bias. So Claude tends to like the output of other versions of Claude uh more than it does the output of uh of OpenAI or some other provider. Um the last line here uh in case you're wondering is suppressing tracing on the judge because we turned on tracing across the board.

评判器本身也是 Claude 的一个副本。呃，默认情况下——事实上，你会在我的测试数据中看到——我忘了开启遥测抑制。所以我的追踪记录里全是评判器运行自身的记录，还能看到评判器的所有追踪信息，而这些都是不需要的。嗯，这个仓库中的每个评估器都会应用这项遥测设置，这样评判器就不会混进我们的智能体里。嗯，我们先从正确性开始。这显然是首选。这个报告正确吗？嗯，只需导入一次，再写一行设置代码。你什么都不用做，只要说：“嘿，我正在运行正确性评估器。用它评估我的内容，看看效果如何。”然后调用 `evaluate`，就会得到一个标签、一个分数和一段说明。脚本会在每次智能体运行时都执行评估，并将结果记录回 Phoenix。The judge is itself a copy of Claude. Uh so by default and in fact in my test data you will see that I forgot to turn on uh uh telemetry suppression. So my traces are full of the the judge running itself and and seeing all of the traces from the judge which you don't need. Um so every evaluator in this repo gets it gets this telemetry setting uh so that the judge stays out of uh our agent. Um let's start with correctness. Uh which is the obvious first choice. Is the uh is this report correct? Um that is one import and one line of setup. You don't have to do anything. You just say hey I'm running the correctness evaluator. Run it on my stuff and let's see how it goes. Uh then you call evaluate and you get back a label, a score, and an explanation. And the script runs it on every agent run and logs the results back to Phoenix.

嗯，就跟我们做 ticker 检查的方式一样。嗯，我也应该能运行那个。这要花点时间，因为它实际上得做一些 LLM 相关的处理。嗯，如果能跑通就行；如果不行，我已经内置了相关内容。所以你可以看到，呃，在我的 annotations 列里，你可以看到……的结果。Uh the same way we did the ticker check. Uh, I should be able to run that as well. That's going to take a little while because it has to actually do some LLM stuff. Uh, if it works, it works. If it doesn't, I have stuff already built in. So you can see uh in my annotations column you can see the results of

为什么正确性评估在每份报告中都失败了Why the correctness eval failed every report

我已经运行了各种评估。在这个例子里，它展示的是可操作性评估，不过你也能看到正确性、忠实度、对人类的可操作性，以及提及贴纸。我们会逐一讨论这些评估的含义。但首先来看正确性。这些评估的正确性得分全都是零。16 份报告都被判定为不正确。它们被判定为不正确的原因是，Claude 会根据它所掌握的知识来判断正确性，而这就是它唯一可用的依据。目前，Claude 的知识截止时间大约是 1 月。所以，我的智能体在谈论 10 月的股价时，Claude 就慌了，心想：“你说的都是未来的事。这显然是幻觉。这份报告不可能正确。”various evaluations I've already run. In this case it's showing you the uh actionability one but you can see uh correctness, faithfulness, human actionable uh and mention sticker. We're going to talk about what all of those are. But first you're going to see correctness. Uh correctness on all of these is zero. Uh, all 16 reports have been judged as incorrect. And the reason that they've been judged as incorrect is because Claude judges correctness by what it knows. That's the only thing that it has available to it. And Claude's knowledge cut off uh at the moment is in something like January. So uh my agent is talking about stock prices in October and Claude is freaking out and going, "You're talking about stuff that's in the future. That's obviously a hallucination. This report cannot possibly possibly correct.

所以，我们内置的正确性检查因为这个原因，判定每一份报告都不合格。嗯，这其实没什么用。嗯，所以这个评估一半错、一半对。在我的数据中，有 16 条解释里有 7 条抱怨说，评判模型无法核实最新数据。呃，这显然说明评估本身有问题。嗯，但其中很多也指出了一些真实的问题，比如数字对不上，或者日期互相矛盾。嗯，所以这里的零分什么也说明不了。它说明的是，整体来看，这个评估可能不是合适的评估方式。嗯，我们想要的是一个评判模型确实能回答的问题。所以，哦，我刚才已经说过了。好吧，我们接下来要改用忠实度。忠实度提出了一个好得多的问题。它问的是：根据你掌握的数据，你的报告讲得通吗？So our built-in check for correctness fails every single report for that reason. Um which is not terribly useful. Um so the eval is half wrong and half right. Seven of in my data, seven of the 16 explanations complain that the judge can't verify fresh data. Uh which is the eval definitely being wrong. Um, but lots of them also point out something real. Things like the numbers don't add up or dates that contradict each other. Um, so a score of zero is telling us nothing here. It's telling us across the board that this that this eval is probably not the right choice of eval. Um, what we want is a question that the judge uh can actually answer. So uh oh, I told you that already. All right. uh what we're going to use instead is faithfulness. Faithfulness is asking a much better question. It's saying given the data that you had, did your did your report make sense?

答案是否基于我们拿到的数据？这样效果好得多，因为它可以使用与智能体拿到的完全相同的数据，并检查它是否能生成一份合理的报告。呃，所以这是评审器确实能够回答的问题。呃，而且上下文已经在追踪记录中了。正如我们看到的，每次网页搜索都是一个独立的跨度。嗯，每次研究调用的输出都是智能体自己记下的发现。所以我的第一个版本，也就是我的第一个评估，呃，只给评审器提供了研究笔记。后来我读了它的解释，发现评审器把一些数字标记为编造的，呃，因为我的评估版本没有告诉它：哦，我其实搜索了网页，而且确实找到了这些数据。呃，所以，呃，在你们看到的这个版本中，我们提供了完整的上下文。嗯，我们的 get agent runs 助手会同时提取原始搜索结果和笔记，这样它就能看到研究智能体发现了什么，以及它是如何找到这些数据的。Did is the answer based in the data that we were given? This works much better because it can work with data this exactly the same data that the agent was given and see if it gets a report that makes sense. Uh so that's a question the judge can actually answer. Uh and the context is already in the trace. Every web search is its own span like we saw. Um, every research turns output is the agent's own notes on what it found. So my first version, my first eval uh only gave the judge uh the research notes and then I read the explanations and the judge was flagging numbers as made up uh because my version of the eval hadn't told it that oh I I searched the web and actually found this data. Uh so uh in the version you see we've given it the full context. um our get agent runs helper is going to pull both uh the raw search results and the notes so that it can see uh what the research agent found and how it found that data.

呃，就像我说的，忠实度评估需要三样东西：输入，也就是用户提出的要求；输出，也就是报告；还有上下文，也就是搜索结果和我刚才提到的笔记。[哼笑] 呃，运行它的方法和之前一样：创建评估器，评估每次运行，并记录结果。嗯，就像我说的，它比正确性评估慢，因为评判器要阅读每一次搜索结果。呃，到目前为止，有谁的评估得出了什么结果吗？再说一次，你们的 Wi-Fi 连接比其他人都快。还有别人吗？好。嗯，我们来看看 Phoenix 里的忠实度报告。嗯，我的忠实度得分有很多个零，但也不是全都是零。嗯，我们来读读 Microsoft 的说明。这是我读过的、最有意思的一条。Uh so like I said faithfulness takes three things. It takes the input which is what the user asked. It takes the output which is the report and it takes the context which is the search results and the notes that I was just talking about. [snorts] Uh and you run it using the same pattern as before. You create the evaluator, you evaluate every run, you log the results. Um, like I said, it's slower than correctness uh because the judge is reading every search. Uh, who has got any results out of their uh eval so far? Again, you've got a faster Wi-Fi connection than everybody else. Anybody else? Cool. Um, so let's look in Phoenix at my faithfulness reports. Um, I got a lot of zeros on faithfulness, but not all of them. Um, let's read the Microsoft explanation. That was the one that I read that was most interesting.

嗯，正如你所见，每一项旁边都有相应的解释。嗯，在这个例子中，回答补充了几项上下文并不支持的说法：Azure 是第二大云服务提供商，积压订单让人能够提前三年掌握情况，而且还隐含了 FY202 的目标。嗯，它在告诉我它不喜欢这段内容的哪些地方，也在解释它为什么认为这段内容不忠实。嗯，它指出，2027 年的一些目标是捏造的，因为搜索结果中根本没有这些信息。嗯，英伟达那个例子也挺有意思的，如果我能找到的话。对，在这个例子中，它说回答把两个不同的数据点混为一谈。这是一个推理错误，而评判代理准确地识别了出来。嗯，它说上下文信息是捏造的。这些结果有用得多。Uh, as you can see, next to each one, you get the explanation. Um, so in this one, the response adds several claims that the context does not support. Azure is the number two cloud provider. The backlog gives three years of visibility and there are implied FY202 targets. Uh, it is telling me what it doesn't like about uh this thing. It does it's telling me why it thinks this thing is unfaithful. Um, it's complaining that uh some of the 2027 targets are fabricated because they're not in any search results. Uh, the Nvidia one is also interesting if I can find that one. Right. In this one, it says the response conflates two different data points. This is a reasoning error that the agent that the judge has uh accurately found. Um it says the context information this is fabricated. These are much more useful results.

这里有 16 份报告，其中 14 份被标记为不忠实，不过这些解释要有用得多。嗯，而且它确实在做我们希望它做的事。对于智能体，你也可以评估工具调用。我们看到了每条追踪记录中的网页搜索片段。Phoenix 也内置了针对这些内容的评估。工具选择评估智能体有没有选对工具，或者在没必要时进行了搜索。工具调用评估它传给工具的参数是否合理，比如搜索查询是否确实与这家公司有关。工具响应处理则评估它有没有对调用结果做出合理的处理。嗯，这些决策可能会成就智能体，也可能会毁掉它。它们全都发生在报告真正被阅读之前。我们也知道，智能体有时会做出奇怪的工具选择，比如它会尝试把报告保存到文件里。We've got uh 14 out of 16 uh reports marked as unfaithful here, but the explanations are way more useful. Uh and it's actually doing the thing that we wanted it to do. Um so for agents, you can also grade the tool calls. Uh so we saw the web search spans inside each trace. Phoenix has built-in evals for all of those as well. Tool selection asks did the agent pick the right tool uh or search when it didn't need to. Tool invocation asks did it pass sensible arguments to those tools like a search query that's actually about the company. And tool response handling asks did it do something sensible uh with the result of that call. Uh these are the decisions that can make a break make or break an agent. Um, and they all happen before the report actually gets read. Uh, and we already know our agent makes strange tool choices sometimes, like it tries to save the report to a file.

呃，所以我们可以看看内置评估能不能捕捉到这一点。呃，我不会运行这个，不过你可以运行那个。嗯，工具选择评估需要三样东西：给智能体的指令、它可以使用哪些工具，以及它最终选择了哪些工具。呃，脚本会遍历智能体的每一轮对话，并分别检查每条轨迹中的研究和写作部分，然后收集每个部分下面的工具调用片段。嗯，轨迹会记录智能体调用了哪些工具，但不会记录它本来可以调用哪些工具。所以我们得在 tool selection.ts 里自己写出那份列表。呃，接下来就和其他运行一样。呃，运行 npm run eval tools，然后就完成了。呃，这个大约需要 15 秒。好，我们来看看 Phoenix 里的工具选择。工具选择。它正在帮我处理。好。那么，这些就是它认定我的工具选择有误的案例，至于原因嘛，各不相同。Uh, so we can see if our built-in eval catches that. Uh, I'm not going to run this one, but you can run that one. Um, tool selection takes three things. What the agent was asked, which tools it could have used, and which ones it picked. Uh and the script goes through every agent turn and every trace research and writing separately and collects the tool spans underneath each one. Um the trace records which tools the agent called but not which ones it could have called. So we have to write that list ourselves uh in tool selection.ts. Uh then it is uh the same pattern as every other run. Uh so npm run eval tools uh and then you're done. Uh that one should take about 15 seconds. So, let's look in Phoenix at tool selection. Tool selection. It's helping me out. All right. So, these are ones where it has decided that my tool selection was incorrect for whatever reason.

呃，用户要求根据前面的研究撰写一份简洁的报告。使用 write 将报告写入文件，是合理且恰当的，因为 write 工具存在，而且使用安全。呃，不过，LLM 又多调用了一次 bash，纯属重复。真是太棒了。呃，确实如此。嗯，评审器判定六个尝试了两次的案例不通过，判定七个通过。其中，六个是只尝试了一次的案例。呃，呃，总体而言，我的 16 次写作尝试中有 13 次试图保存文件。嗯，这些解释中，只有一个提到用户从未要求将内容保存到文件里。那，呃，这个评审器是不是判错了？评审器判定这次工具调用正确，到底准不准确？嗯，我们的写作提示是：“根据以上研究撰写一份简洁的财务报告。”我们的提示确实存在歧义。我们说了“撰写”，却没说要把报告写到哪里，也没说要把它写来做什么。Uh, the user asked for a concise report based on prior research. Writing it to a file with write is plausible and appropriate since write exists and is safe. Uh, however, the LLM made a second redundant bash call. Awesome. Uh, yeah, it sure did. Um, the judge failed six that tried twice. It passed seven. Six of the seven that tried once. Uh uh overall 13 of my 16 writing turns tried to save a file. Um and only one of these explanations said that the user never asked for a file to be saved. So uh is this judge incorrect? Is the judge uh accurately judging that the tool call was correct? Um, our writing prompt said, "Write a concise financial report based on your research above." Our prompt is genuinely ambiguous here. We said, "Write, we didn't say where to write it to. We didn't say what to write it for.

呃，我们没说受众会是谁。”所以，工具选择知道这一点，并查看输入，然后想：“嗯，似乎让这个 agent 尝试写入磁盘是合理的。”呃，所以再说一次，你告诉 agent 做什么，以及你告诉 eval 做什么，两者会相互影响。呃，你不一定能相信工具调用的执行会是正确的，因为在这个例子里，工具调用评估站在 agent 一边。它的意思是：“不，agent 是对的。agent 给了我们指示，呃，agent 收到了指示，并且已经尽力了。”这基本上就是 eval 的立场。呃，eval 结果也会保存在 trace 中。嗯，如果你在 Phoenix 中查看 traces，呃，就能在这里看到它们附加到了哪个 trace，以及 trace 中的哪个 span。Uh, we didn't say who the audience was going to be." So, the tool selection knows that and is looking at the input and is going, "Well, it seems a reasonable thing for the agent to try and write to disk here." Uh so again what you're telling the agent to do and what you're telling the eval to do interact with each other. Uh you can't necessarily trust that the tool call invocation is going to be right because in this case the tool call is citing the tool call eval is siding with the agent. It's like no the the agent is right. The agent gave us instructions uh the agent received instructions and did its best is basically the eval's position. Uh, eval results live in the trace as well. Um, so if you're looking at the traces in Phoenix, uh, you can see them here attached to which trace they got, which span in the trace they got attached to.

所以你会看到，actionability 和 correctness 都关联到了根节点。而 tool selection 则关联到了发起该工具调用的特定轮次。你可以按任意评估项筛选追踪记录，也可以按分数排序，找出表现最差的运行结果。你可以直接从失败的评分点击进入导致问题的步骤，也就是说，一键从评分追溯到原因。以上这些都是内置评估功能。这里的经验是，选对评估项比调优评估项更重要。选择 correctness 完全是选错了评估项，它没有告诉我们任何有用的信息，只是告诉我们有很多东西不对。tool selection 确实提供了一些信息，而 faithfulness 告诉了我们不少东西。所以，检查评估项是否选对，比调优更重要；弄清楚自己真正要问的是什么，才是把这件事做对的关键。So you see actionability and correctness got attached to the root. Uh, whereas tool selection is getting attached to the specific uh, turn that made that tool call. Um, you can filter your traces by any eval. Uh, you can sort by score to find your worst runs. You can click straight from a failing score into the step that caused it. Uh, from score to from score to cause in one click. Uh, so all of that is the built-in eval. Um, and the lesson here is that choosing the right eval beats tuning it. Choosing correctness was entirely the wrong eval. It didn't tell us anything. It just told us a bunch of stuff was wrong. tool selection did tell us something uh and faithfulness told us quite a lot. Um so checking the right eval beats tuning it uh and knowing which question you're actually asking is really the core of getting this right

步骤 5：编写自己的评审器Step 5: Writing your own judge

这就引出了第五步：编写你自己的评审器。这一步会让评估针对你的应用变得具体起来。呃，再说一次，人们想到评估时，大多数人想的都是这个，但我想先确保你掌握了基础知识。你不必一上来就编写自定义评分标准。你可以编写代码评估，先把一大堆事情处理掉。你也可以用内置评估完成很多显而易见的检查，不必自己发明一个用于评估工具调用正确性的评估器。嗯，我希望我的报告能给出买入、卖出或持有的建议。没有内置评估能检查这一点，因为这确实非常具体。所以，我们要自己编写一个。呃，我见过的效果不错的评估提示词都有五个不同的部分，我们会逐一讲解。which brings us to step five which is writing your own judge. This is where the eval get specific to your application. Uh, and again, this is when people are thinking about evals, this is what most people are thinking about, but I wanted to make sure that you got the basics in first. You don't have to jump straight to writing a custom rubric. You can write code evals to get a bunch of stuff out of the way. You can write uh you can use built-in eval to do a bunch of obvious stuff without having to invent your own tool calling correctness uh uh evaluator. Um, what I wanted my report to do was give me a buy, sell, or hold recommendation. There is no built-in eval that checks for that because that would be a very specific thing indeed. So, we're going to write our own. Uh, every eval prompt I've seen that works well has five different parts and we're going to go through them one at a time.

第一部分是赋予评判者一个角色和背景。嗯，你经常会看到这样的说法：“你是一位专家，等等等等。”这确实管用，所以大家才都会把它写进去。比如，告诉智能体它是什么身份，会有一点帮助。嗯，在并排比较时，告诉它自己是专家，似乎也比单纯告诉它正在运行某种其他东西更能产生效果。呃，这就是大语言模型的一些奇怪之处。你得告诉它自己很聪明，这样它就会更有信心，也会努力去做。嗯，不过真正重要的是背景信息。嗯，这是一份财务报告。我们期待的是一份分析财务数据、提供真实见解的报告。嗯，我们尤其关心的是，它能不能让人据此采取行动。The first part is you give the judge a role and context. Uh, a thing you'll see all the time is you are an expert blah blah blah blah blah. That does actually work. That's why everybody puts it in there. Like it helps a little bit to tell the to tell the agent what it is. Uh, telling it it's an expert as opposed to just telling it that it is running up something else also seems to make a difference when they do sideby-side comparisons. Um, it's just one of the weird things about LLMs. You need to tell it it's really smart and it gets like confident and tries. Um uh but what really matters is the context. Um this is a financial report. Uh we are expecting a report that that analyzes financial data and tells us real things. Um and specifically what we care about is whether or not you could act on it.

所以，呃，这就给出了我们纳入评分标准的明确准则。嗯，很多人在这方面下的功夫不够。所以，不要写“好的回答是有帮助且准确的”，因为评审者并不知道在这个语境下“有帮助”指什么，也不知道“准确”是什么意思。嗯，评审者无法据此做出是或否的判断。要明确写出一份报告怎样才算具有可操作性。比如，它会提出具体建议，会指出具体风险，而且着眼于未来而不仅仅回顾过去。这些都是评审者可以观察并实际核查的内容。嗯，还要注意，我们并不是凭空编造这些标准。嗯，你的准则来自错误分析。“只总结数据，不作解读”是我们阅读追踪记录时发现的一个问题。我们把它纳入标准，是因为我们想避免这个问题。So uh that gives us the explicit criteria that we include in our rubric. Um so this is where a lot of people don't put in enough effort. So don't write a good response is helpful and accurate because uh a judge doesn't know what helpful and it doesn't know what accurate means in this context. Um a judge can't make a yes or no decision from that. Write down exactly what makes a report actionable. Uh so it makes a specific recommendation. It names concrete risks and it looks forward not just back. Those are things that a judge can look at and actually check. Uh, and note that we're not just making up these things blind. Um, your criteria come from your error analysis. Uh, only summarizes data with no interpretation is a problem that we saw when we were reading our traces. We are putting that in there because it's a problem that we are trying to avoid.

嗯，这正是先阅读数据的全部意义所在。具体的标准能让你的判断保持一致，而要制定具体的标准，就得提前了解数据会是什么样，因为你亲自读过它。好的评分准则的第三部分，是清楚地标记数据。把每一条数据都分别包在自己的 XML 标签中。Claude 是用数十亿、数百亿个 XML 标签训练出来的，因此非常擅长处理它们。嗯，如果你查看我们自己的评分准则代码，就会发现我们也是这么做的。呃，我们自己的提示词[嗤笑]呃，会把所有内容都包在 XML 标签中。嗯，双花括号是模板变量，Phoenix 会用每个片段的输入和输出来填充它们。呃，这么做的原因是，你希望清楚地区分输入、输出和指令。Um, that is the whole point of reading the data first. Specific criteria give you consistent judgments and you get specific criteria by knowing in advance what the data is going to look like because you read it yourself. Part three of a good rubric is to mark the data clearly. Wrap each piece of data in its own XML tag. Claude was trained on billions and billions of XML tags and it's really good at working with them. Um, if you look into the code at our own rubrics, that's what we're doing. uh our own prompts [snorts] uh wrap everything in XML tags. Um and the double curly braces are template variables that Phoenix fills in uh with each span's input and outputs. Uh the reason you do this is because you want clear boundaries between what is the input, what is the output, and what is your instructions.

呃，这有助于防范提示注入，也有助于避免智能体单纯因为困惑而出错。呃，尤其是如果你包含示例——我们接下来会看到——把示例标记为示例，而不是数据，这会有帮助；否则它会对此感到非常困惑。嗯，我很喜欢加入带标签的示例，但这实际上是一种颇有争议的做法，呃，在评估领域尤其如此。嗯，评判标准会抽象地告诉评判者你的意思，而示例则能向它准确展示你的意思。作为一个人，我发现别人给我演示了事情应该怎么做之后，我就容易得多；相比之下，如果他们只是给我一份操作说明清单，就没那么容易了。呃，显然，很多大语言模型也有同感。嗯，所以，在测试你的评分标准时——我们下一步就会做——用有示例和没有示例两种方式分别测试，看看哪种效果更好。Uh this helps against prompt injection, but it also helps uh the agent just getting confused. uh in particular it helps you get con uh if you include examples which we're going to see mark the example as an example and not the data otherwise it will get very confused about that um I love to include labeled examples but it is in fact a controversial proc uh a controversial practice uh in the eval world um criteria tell the judge what you mean in the abstract but examples show it exactly what you mean as a human, I find it much easier to do something when somebody has shown me an example of how it's supposed to work than if they've just given me a list of instructions of how to work. Uh, and, uh, a lot of LLMs feel that way as well, apparently. Um, so when you test your rubric, uh, which we're going to do in the next step, uh, test it with examples and without examples and see which one does better.

呃，因为有时候，呃，示例能帮助它做得更好。有时候，呃，示例也会让它过拟合。它会看着你的示例说：“嗯，这看起来跟我的示例不完全一样。”或者它会试图让每个输出都看起来像你的输出示例。呃，这样反而没那么有帮助。嗯，我在自己的模型上做的正是这个测试。呃，我用的评分标准里有个结果特别有意思，简直是 nerd 才会觉得有趣的那种，所以我就把它放进来了。呃，我去掉了示例；有示例时，评审器在三次不同的运行中，分别有 16 次里的 15 次、15 次里的 15 次和 16 次里的 15 次同意我的判断。呃，而去掉示例后，它有两次对 16 个示例全部同意我的判断，还有一次对 15 个示例同意。呃，所以我就很好奇，为什么去掉示例反而让结果变好了。呃，具体来说，它遇到的问题是，有一份报告里没有包含，嗯，呃，没有包含风险清单。Uh, because sometimes, uh, examples are going to help it do a better job. And sometimes, uh, an example will make it overfit. It will look at your examples and say, well, it doesn't look exactly like my example. Or it will try and make every output look like your output example. Uh, which is less helpful. Um, I ran exactly that test on mine. Uh, so in the rubric that I used, um, this was fascinating in a really nerdy way, which is why I included it. Uh, I threw out my examples and with the examples, the judge agreed with me, uh, 15, 15, and 15 times out of 16 in three different runs. Uh, and if I threw out the examples, it agreed with me 16 times twice, all 16 examples, and 15 times once. Uh and so I was like I was very curious as to how throwing out the examples made it better. Uh and the specific problem that it was having is that in one of the reports it did not include um uh it did not include a list of risks.

作为人类，我读了这份报告，觉得它的意思是“买入微软”，而且没有提到任何风险，我当时想，这没问题，仍然具有可操作性。报告给出了买入建议。呃，所以我把它标记为一份可操作的报告。呃，但我的示例说必须包含风险。于是，智能体——也就是 LLM 评审器——根据规则和示例进行判断，认为报告里必须提到风险。因此，由于没有风险提示，这份报告不具有可操作性。说实话，我的人类判断究竟比评审器更好还是更差，这仍然是个悬而未决的问题。评审器的标准非常严格，而我在人为标注黄金数据时则比较随意。So as a human I read the report and it was like Microsoft is a buy and it didn't include any risks and I was like that's fine. It's still actionable. It included a buy. Uh so I marked it as uh an actionable report. Uh but my examples said you must include risks. And so the agent uh the LLM judge was looking at the rules and and looking at the examples and saying there must be risks in here. So this is not actionable because there's no risks. So that's genuinely an open question as to whether or not my human judgment was better or worse than the judges. The judge was being very strict and I was being loosey goosey when I as a human gave myself golden data.

主持人：不好意思，我可以问个问题吗？Host: Sorry, can I ask a question?

嘉宾：当然。Guest: Sure.

主持人：那么你的假设是什么？为什么没有这些例子时，它会同意你的看法，并认为微软是可以采取行动的对象？Host: So what's your hypothesis then as to without the examples it agreed with you so it saw that Microsoft as being actionable?

嘉宾：是的。Guest: Yeah.

主持人：所以，你的假设是，是不是仅仅因为你没说风险是一个必要要素，它就让人觉得这是个不错、可执行的方案？我的意思是，这并不是一个假设，比如 LLM 评判器会附带解释。所以它会说……Host: Is that is that your hypothesis then that like simply because you didn't say risk was a necessary element it felt that that was a good actionable? I mean, it's not a hypothesis like the LLM judge comes with an explanation. So, it says

嘉宾：所以，它说那是可以采取行动的，Guest: So, it said that that was actionable,

主持人：对吗？Host: right?

嘉宾：是的。就像我说的，在这种情况下，确实很难说到底是我做得对，还是评审员做得对。评审员决定非常严格地解读评分标准，而我在把这些内容标记为人类可执行或不可执行时，呃，标准没那么严格。嗯，下面是一个可执行建议的例子。它给出了具体数字，包括营收增长、相对于所属行业的估值，还指出了一项具体风险。最后，它给出了一个你确实可以照着做的建议，其中还包括在什么情况下你会改变主意。嗯，下面则是一个不可执行建议的例子。它并没有错，里面的内容都是真的，只是没什么用。“投资者应考虑各种因素”并不是一项建议。嗯，现在评审员已经看过这两种例子，希望它在评估那些介于两者之间的报告时能更加一致。Guest: Yeah. So, like I said, in this case, it's genuinely unclear whether or not I was doing the right thing or the judge was doing the right thing. The judge decided to interpret the the rubric very strictly and I when I was marking these things as human actionable or not, uh was not being as strict. Um, here's an example of an actionable one. Uh, it has specific numbers. It has revenue growth. Uh, it has valuation compared to its sector. It names a concrete risk. And it ends with a recommendation you could actually follow, including when you'd change your mind. Um, and this is what a not actionable one looks like. It isn't wrong. Everything in it is is true. It's just not very useless. Investors should consider various faction uh factors is not a recommendation. Um, now that the judge has seen both, the hope is that it's more consistent on reports that fall somewhere in between.

嗯，至于它实际上是不是可执行的，你要通过测试才能知道。嗯，制定评分标准的第五部分是约束答案。你要求它判断某件事是否可执行。以前你得说“输出一个字符串，而且这个字符串必须是‘可执行’或‘不可执行’”。嗯，现在已经不用这么做了。现代评估器会使用工具调用来输出结果。嗯，工具调用可用的选项是在工具调用中定义的，而不是在评分标准里定义的。嗯，这样就可靠得多了。现在轮到你来编写评分标准了。嗯，打开 evalactionability.ts。里面有我写的评分标准，包含全部五个部分。嗯，你可以把它改成自己的。你可以修改评判标准，也可以修改示例。嗯，这些报告中，什么情况会让你采取行动？Uh, and whether it actually is, you find out by testing. Uh, part five of putting together a rubric is you constrain the answer. You ask it for things that are actionable or not actionable. A thing you used to have to do is you used to have to say output a string and the string must be the word actionable or not actionable. Uh, you don't have to do that anymore. what uh modern evaluators use tool calls uh to put their output. So those uh the choices available for that tool call are defined in the tool call and not in your rubric. Uh which makes them a lot more reliable. So now it is your turn to write the rubric. Uh open evalactionability.ts. Uh it has my rubric in it with all five parts. Um and you can make it yours. You can change the criteria. You can change the examples. Uh what would make you act on one of these reports?

你在自己的 traces 中看到了什么？把它记下来，这就应该成为你的评分标准。嗯，运行需要三到四分钟。到目前为止，谁还跟得上我？比如说，谁跟到这里了？太好了，我看到好几只手。呃，就像我说的，上次我办这个工作坊时，大家都完全听懵了，因为那个 repo 有 200 MB。所以现在竟然有人弄好了，这对我来说真是太棒了。嗯，这就是它的接线方式。你创建一个分类评估器，它会处理所有底层配置。呃，你给它一个名称，指定模型。呃，再给它模板。呃，就像我说的，你还要在带外配置它的输出选项。所以你要给它一些选项：可执行的，分数为 1；不可执行的，分数为 0。嗯，我们来看看 Phoenix 里的可执行性。What did you see in your own traces? Write that down and that should become your rubric. Um it will take three or four minutes to run. Who has managed to stay up with me so far? Like who is at this point? Excellent. I have multiple hands. Uh like I said, the last time I did this workshop, everyone was completely lost because the 200 megabyte repo. So the fact that anyone's got it this is just great for me. Um this is how you wire it up. You create a classification evaluator which does all the plumbing. Uh you give it a name, you give it the model. Uh you give it your template. Uh and like I said uh out of band you give it its out its options. So you give it its choices which are actionable with a score of one and not actionable with a score of zero. Um, so let's look at actionability in Phoenix.

[哼哼] 这里冷得我手指都麻了，没法好好打字。好吧。这份报告分析了 JPM 的利率风险敞口和贷款增长，呃，还有前瞻性输出，但没有给出买入、持有或卖出的建议。太棒了。它完全按照预期在做。呃，它在抱怨缺少具体的建议。嗯，在看这些建议之前，我们应该先看看建议的分布情况。呃，在我的数据中，16 份报告里有 5 份被判定为可执行，11 份被判定为不可执行。呃，这是最有用的评估类型之一。所有结果都完美的评估并没有太大帮助。所有结果都错的评估同样没什么帮助。而结果各占一半的评估，则像是一座你可以攀登的山。[snorts] It's so cold in here that my fingers have gone all numb and I can't type properly. Okay. So, the report analyzes JPM's rate exposure and loan growth uh and a forward-looking output, but makes no buy, hold, sell recommendations. Fantastic. It's doing exactly what it's supposed to do. Uh it is complaining about uh the lack of recommendation of concrete recommendations. Um before we look at all of these recommendations, we should look at the split in the recommendations. Uh in my data, five of 16 reports were actionable and 11 were counted as not actionable. Uh and that is one of the most useful types of evals you can get. An eval where everything is perfect is not particularly helpful. An EVA where everything is wrong is also equally unhelpful. An EVA where you've got an even split that is a hill that you can climb.

你突然就有了，呃，一组确实能正常运行的示例，还有一组无法正常运行的示例，你可以开始找出它们之间的差异。嗯，如果你逐个查看这些追踪记录——我就不一一打开了——在 Rivian 的案例中，它说自己从不提出买入、持有或卖出建议，也不会给出任何同等性质的下一步操作建议。对于 Nvidia，它说最终投资论点的风险等级为中等，这是模糊的评级，不是指导意见。至于 Meta，结论仅仅是该持仓稳健。所以，这个问题一再出现：它拒绝给出具体的买入、卖出或持有建议。嗯，我们可以打开 Phoenix 提示模板看看。我现在就不打开了。嗯，不过关于评分标准还有一条规则：你应该把评估提示当作代码来处理。You've suddenly got uh some you've got a def a set of examples where things are actually working, a set of things where uh stuff isn't working and you can start figuring out the differences between them. Um so if you click through these traces, I'm not going to bother clicking through all these traces. In Rivian it says it never makes a buy, hold, sell recommendation or any equivalent next step. For Nvidia, it said the closing investment thesis risk level is moderate, is vague rating, not guidance. And uh for Meta, the verdict is the is only that the position is solid. So this is the same problem over and over. It is refusing to give concrete buy, sell, hold recommendations. Um, if we uh we can open the Phoenix prompt template. I'm not going to do it now. Um, but uh there's one more rule for rubrics, which is that you should treat eval prompts as code.

你放进自定义评估中的评估提示词，对措辞上的细微变化极其敏感。嗯，我在做这个的时候就遇到过一个例子：我告诉你应该用 XML 标签把内容括起来。可一开始我忘了给示例加上 XML 标签。嗯，当示例是纯文本时，我得到了一串无法采取行动的内容，因为它搞不清哪些是数据，哪些是我的示例。而当我用 XML 标签把它们括起来后，所有内容突然都变得可执行了，因为它能分辨我在说什么。嗯，除此之外没有任何变化。我每次给它的都是相同的数据、相同的指令、相同的输入和相同的输出。嗯，只是评估器对发生的事情没那么困惑了。嗯，所以你必须把评估当作代码来对待。Your eval prompts going into your into your custom eval are extremely sensitive to small changes in wording. Um, so, uh, one example of that I ran into while making this was, uh, I told you that you should put XML tags around stuff. I forgot to put XML tags around my examples to begin with. Uh, and with my examples as plain text, uh, I got a string of not actionable stuff, uh, because it was getting confused about what the data was and what my examples were. And when I wrapped them in XML tags, everything suddenly became actionable because it could tell what I was talking about. Um, this is there was no other change. I'd given it the same data, the same instructions, the same input and the same output every time. Uh, it just got less confused in the evaluator as to what was happening. Uh, so you have to treat your evals as code.

你应该对它们进行版本管理、检查和测试。嗯，正如我之前提到的，使用评估器时有个容易踩的坑，就是试图编写一个包办一切的评估器。这种模式很常见，因此它被称为“上帝评估器”。你会试着让它在一个提示词中检查准确性、语气、完整性、格式和政策合规性。嗯，这简直是一场噩梦，因为一旦它失败了，到底是哪一项出了问题？是不准确？语气不对？忠实度有问题？你让这个上帝评估器评估的那么多项中，究竟是哪一项出了问题？然后你试图通过给它 15 个标签，让它从中输出一个，来解决这个问题。嗯，结果整个事情就变得很难追踪。嗯，最佳实践是把它拆开：测试多少个维度，就将评估器拆分成多少个部分。You should version them, you should check them, and you should test them. Um, as I mentioned earlier, uh, there is a trap to fall into with evaluators, which is to try and write write one evaluator that does everything for you. It is called the god evaluator because it is such a a common pattern. Uh, you try and get it to check accuracy, tone, completeness, formatting, policy all in a single prompt. Uh, and it is a nightmare because if it fails, what thing failed? Was it inaccurate? Was the the tone wrong? Was the faithfulness wrong? which of the many many many things that you've told this god evaluator to evaluate is it actually wrong and you try and solve that by giving it 15 labels that it can spit out uh and the whole thing becomes very hard to track. Uh the best practice is to split this up split your evaluators into as many dimensions as you are testing.

所以，一个评估忠实度，一个评估正确性，一个评估可执行性。嗯，你还应该知道哪些评估是护栏，哪些是北极星指标。护栏指的是应该阻止发布的标准。比如，如果智能体编造股价，就应该停止发布。呃，而北极星指标是你努力追求的目标。比如，推荐其他值得关注的股票，会是一个不错的功能，值得加入。你的评估可以检查这一点，但不必以此作为阻止发布的门槛。So one for faithfulness, one for correctness, one for actionability. Um, and you should also know which one of your evals is a guardrail and which one is a north star. So, a guardrail is something that should block the release. So, if the agent is making up a stock price, that should halt shipping. Uh, but north stars are things that you're aiming for. So, suggests other stocks worth a look would be nice to have. That would be a good thing to include. Your eval can look for that, but it doesn't have to gate shipping on that.

步骤 6：测试你的裁判Step 6: Testing your judge

这就引出了第六步：测试你的测试。我们已经构建了一个评判器，它指出我们有些报告不具备可操作性。呃，但我们能相信这个评判器吗？怎么知道它的判断是对的？呃，记住这个诀窍：从经典机器学习的意义上说，你的评判器是一个分类器。呃，它会查看一份报告，并预测它属于“具备可操作性”还是“不具备可操作性”。预测呃可能正确，也可能错误。所以，和任何分类器一样，你要通过将它的预测与事实进行比较来评估它。呃，这里的事实就是人类专家会怎么判断。你可以来评判——呃，你可以给评判器的作业打分。呃，在这个例子里，人类专家就是我。我来判断股票应该买入、卖出还是持有。呃，我未必是你理想中的专家。呃，但我们会尽力。Which brings us to step six, which is testing your tests. We've built a judge and it says that some of our reports aren't actionable. Uh but can we trust the judge? How do we know that the judge is right? Uh the trick to remember is that your judge is a classifier in the classic ML sense. Uh it is looking at a report and it is predicting a label as actionable or not actionable. Predictions uh can be right or wrong. So like any classifier, you measure it by comparing its predictions with the truth. Uh, and here the truth is what a human expert would say. You get to judge uh you get to mark the judge's homework. Uh, in this case, the human expert is going to be me. I'm going to be deciding whether or not stocks are buy, sell, or hold. Uh, not necessarily the expert that you want. Uh, but we're doing our best.

这只是个演示。嗯，操作非常简单。打开一条 trace，呃，点击那个小铅笔按钮，就可以添加标注。我提前创建了一个名为 human actionable 的标注，可以将它设为 actionable 或 not actionable。然后我可以很快地逐条查看所有 trace，并为每条设置好这个标注——我已经提前做完了。呃，这时就可以请领域专家参与进来。你可以让他们在 Phoenix 里查看。你可以说：这里有 10 条 trace，这里有 20 条 trace，总共有 50 条 trace。请逐条阅读输出，然后告诉我这些结果好不好，并为我整理一份黄金数据集。嗯，[哼笑] 我坐下来，按照给评判器的同一套标准，把我的 16 份报告都正确地标注了一遍。呃，这些都放在你的种子项目里了，这样你就不用自己做了，因为那会花很长时间。It's just a demo. Um, how you do this is very simple. You open up a trace. uh you click the little pencil button and you can add annotations. So I have created an annotation in advance called human actionable uh and I can set it to actionable or not actionable and I can very quickly uh click through all of my traces and set that on each one which I've done in advance. Uh this is where you would get uh a domain expert involved. You would throw them at Phoenix. You'd say here are 10 traces, here are 20 traces. there's 50 traces. Go through and read the output and give me give me your golden data set about whether or not these things are good. Um, [snorts] I sat down and labeled all 16 of my reports properly against the same criteria that I gave the judge. Uh, and those are sitting in your seed project so that you don't have to do that because that would take you a long time.

嗯，像我说的，你这样做是在构建一个黄金数据集。呃，里面的例子都是根据人类所知、答案确实正确的例子。嗯，这需要投入大量工作，所以你应该让这些工作发挥应有的价值。因此，你也应该给自己设定和给评审模型相同的具体标准。这就是我们之前谈到的那个错误，对吧？我没有严格按照自己设定的评估标准来审阅。我违反了自己的一条规则：规则要求必须包含风险，但我却说这份报告没问题，尽管它并未包含风险。结果，我和评审模型的判断不一致，而且我觉得评审模型其实比我更准确，因为它更严格。嗯，所以你在这里的敌人就是疲劳。如果你给自己安排了一百条轨迹来评判，评到第 50 条时，你就会对评判轨迹感到非常厌烦，也会变得越来越敷衍。Um, what you're building when you do this, like I said, is called a golden data set. Uh it's examples where to the best of human knowledge you know that the answers are right. Um it is a lot of work so you should make that work count. So you should give yourself the same specific criteria that you gave the judge. That was the error that we were talking about earlier. Uh where I was not exactly reading the rules that I had put into my eval criteria. Right? I was disobeying one of my own rules. my rule said uh must include risks and I said that this report is fine even though it doesn't include risks. Uh and as a result me and the judge disagreed and I feel like the judge was actually more correct than I was. It was being stricter. Um so uh your enemy here is fatigue. Uh if you give yourself a hundred traces to judge by trace 50, you're going to be really bored of judging traces and you're going to get lazier about it.

呃，所以，与其有一百个只是随手点过去、觉得“嗯，看起来没问题”的例子，不如有五十个标注得非常好的例子。呃，黄金数据集中的数据质量越高，一切就会越可靠。嗯，你能做的最好的事情之一，就是不断向黄金数据集中添加数据。当你遇到棘手的例子，或者在生产环境中发现异常情况时，弄清楚正确答案应该是什么，并把它加入黄金数据集。呃，构建优质黄金数据集时，有几条关于人工评估的好规则：你应该编写人类能够通过的测试任务。呃，如果你的智能体每次都无法完成某项任务，那几乎总是说明任务本身有问题，而不是智能体有问题。嗯，对于每项任务，你都应该有一个确定正确的答案，以此证明该任务确实可以完成。Uh so it is better to have 50 really well-labeled uh examples than a hundred where you were just clicking through and saying ah this looks fine. Uh the the higher the quality the data in your golden data set uh the more reliable everything is going to be. Um and one of the best things that you can do is that uh is continuously add to your golden data set. when you see a tricky example, when you see something strange happen in production, figure out what the answer should have been and put it into your golden data set. Uh, a few good rules for human go for a good golden data set is that you should write test tasks a human could pass. Uh, if your agent fails a task every single time, that is almost always going to be a broken task, not a broken agent. Um, and for each task, you should have an answer that you know is good, which proves the task can be done.

呃，你还应该对两个方向都进行测试。比如说，如果你只检查智能体在应该搜索网页时是否会搜索网页，那你就会在不知不觉中把它训练成每次、时时刻刻都搜索网页，即使它其实不需要这么做。呃，这里有一条同样适用于评判器的机器学习规则：不要用测试集来调优。呃，把你的标签分成两组，一组用于调优评估，另一组用于测试评估是否有效。呃，如果你的黄金数据集中只有 16 个答案，那测试集就会非常小。呃，因为通常的规则是：75% 放在调优集中，25% 放在测试集中。所以我们的测试集中只有 4 个答案。呃，但如果你有 100 个答案，那就没问题了。嗯，Phoenix 数据集内置了数据划分功能。Uh, you should also test in both directions. If you only check, for instance, that the agent searches the web when it's supposed to search the web, you're going to accidentally train your agent to search the web every single time, all the time, even when it doesn't need to. Uh, and here's a rule from machine learning that applies to judges as well. Don't tune on your test set. Uh, split your labels into two groups. One that you use to tune your evals and one that you use to test that they are working. Uh when you only have 16 answers in your golden data set, that makes it your test set very small. Uh because the rule is usually 75% in the tuning set, 25% in the test set. So there's only four in our test set. Uh but if you were up to 100, that would be fine. Um Phoenix data sets have splits built in.

所以这很容易做到。呃，看看效果如何。呃，首先，它会构建带有划分的数据集基准。它会找出每一份带有人类标注的报告。呃，呃，每一份报告都会成为 Phoenix 数据集中的一个样例。输入是报告，预期输出是人工标注。我们按 span ID 排序。然后每第四个样例进入测试集，其余的进入开发集。呃，先排序意味着无论运行多少次，同一份报告总会被分到同一个集合中。呃，span [吸鼻声] ID 会将每个样例关联回它所属的追踪记录。呃，这样你就可以随时点击查看原始运行记录。然后我们用人工标注来评估裁判器。这是一个实验，我们稍后会深入探讨。嗯，不过现在，呃，一个实验需要一个数据集、一个要在每个样例上运行的任务任务，以及用于评估结果的评估器。So this is very easy to do. Uh see what that looks like. uh so first it builds the golden data set with splits. It finds every report with a human label. Uh uh and each one becomes an example in a Phoenix data set. The input is the report. The expected output is the human's label. We sort them by span ID. Then every fourth example goes into the test split and the rest go into dev. Uh sorting first means that a report always lands in the same split however many times you run this. Uh, and span [snorts] ID links each example back to the trace it came from. Uh, so you can always click through to the original run. Then we run the judge against the humans. This is an experiment which is something that we're going to dig into in a little bit. Um, but for now uh an experiment takes a data set. It takes a task task to run on every example uh and evaluators to score the results.

这里的任务是我们的裁判，而评估器是一个小型的 Kodi val，用来检查裁判给出的标签是否与人工标注一致。注意，我们只在开发集拆分上运行它。你可以运行 `npm run meta` 来亲自试试。[叹气]看看能不能跑起来。它会运行一个 LM。所以，我先假设它会和昨晚一样正常运行。脚本应该会在最后打印一张小表格。在我的 12 份开发集报告中，裁判和我每次的判断都一致，这很不错，但毕竟也只有 12 份报告。所以，应该把它理解为：目前还没有发现问题。你在生产环境运行时，需要使用更大的测试集。嗯，不错。它确实跑完了。更有意思的是，我觉得最难标注的那两个端口——我是说，那两份报告。The task here is our judge and the evaluator is a tiny Kodi val which checks whether the judge's label matches the humans. Uh notice that we only run it in the devs split. Uh you can run this yourself by running npm run meta. [sighs] Let's see if we can get it to go. It's going to run an LM. So, uh, I'm going to assume that it does the same thing that it did last night. Um, the script should print a little table at the end. Uh, and on my 12 dev reports, the judge and I made the same call every time, which is great, but it's also only 12 reports. Uh, so read that as, you know, no sign of a problem yet. you're going to want a larger test set when you're running this in production. Um, cool. It actually completed it. Uh, the more interesting part is the two ports that I the two reports that I found hardest to label.

嗯，所以在我的 Phoenix 数据中，有可口可乐和微软。嗯，第一个是可口可乐。可口可乐是那个在我从没有 XML 标签改为使用 XML 标签后结果发生反转的案例。嗯，这就是加入示例后效果变好的那个。对我来说属于边缘情况的这个，结果也确实是边缘情况。报告称 KO 是保守型、注重收入的投资组合的理想选择，这听起来像是一种推荐。所以作为人类，我反复权衡自己该不该把它算作推荐。最后我决定不算，但裁判也在反复权衡这是否算得上足够明确的推荐。嗯，第二个是微软，我就不点进去看了，它的情况正好相反。Um, so in my Phoenix data, uh, there's Coca-Cola and there's Microsoft. Um the the first one is Coca-Cola. The Coca-Cola is the one that flipped uh when I changed from the XML tags to the uh from not having XML tags to having XML tags. Uh this is where when it got the examples it got better. Um borderline for me turns out to be uh borderline as well. the it the report calls KO ideal for conservative income oriented pol uh portfolios uh which sounds like a recommendation. So as a human I went back and forth on whether or not I should count that as a recommendation. Ultimately I decided it did not uh but the judge also goes back and forth on whether or not that counts as enough of a recommendation. Um and the second one was Microsoft uh which I'm not going to bother to click into which is the other way around.

这份报告给出了买入建议，并说明了理由和时间范围。嗯，而且它完全没有提到任何风险。这就是我之前说的那份报告。嗯，我的评分标准把列出具体风险作为一项评判标准。嗯，我当时觉得这样可以，但评审者认为不行。评审者说它在风险方面表现薄弱。它没有风险部分。它勉强达到了可执行性的门槛，不过也只是勉强。所以，评审者看了这些内容后，对这份报告是否可执行的判断，和我这个人类的判断完全一致，包括其中的模糊之处。嗯，这种模糊性仍然存在于我们的评分标准中，因为我们还没有修改它。嗯，所以真的很想把它改掉。于是我试着加了一句话：只有满足全部四项标准，报告才算可执行，因为之前这一点有歧义。The report says buy with reasons uh and a time horizon. Uh and it never mentions a single risk. This is the one that I was talking about earlier. Uh my rubric lists name concrete risks as a criterion. Uh and I decided that was fine and the judge decided that that was not fine. It said it was weak on risks. There is no risk section. It clears the bar for actionability but though not by much. So, the judge looking at this stuff came up with exactly my human judgment on whether or not this was an actionable report, including the ambiguity. Um, and that ambiguity is still in our rubric because we haven't changed it yet. Uh, so it's very tempting to fix it. So, I tried I added one sentence uh which was a report is actionable only if it meets all four criteria because previously that was ambiguous.

我说的是，报告应该包含所有这些内容。我没说它必须包含。呃，所以你可以修改评分标准，把要求说得更具体，比如明确规定：是的，这四项标准都必须包含。呃，在开发集上，这让 Microsoft 的结果变差了，我原本 12 项全都一致，变成了 12 项里只有 11 项一致。嗯，而且在测试集划分中，它也让 Apple 的结果变差了，因为在评审代理看来，Apple 的风险部分不够充分。嗯，这说明你可以修改评分标准，反而让评估变得更差。呃，我可以让评估变得没那么有帮助，也可以让 EVA 变得不那么像人类会做出的判断。呃，尽管我是在努力把标准说得更具体。嗯，这个脚本也会输出精确率和召回率。呃，精确率和召回率都是非常重要的概念，同样是从机器学习领域借鉴来的，或者更准确地说，是从统计学领域借鉴来的。I said a report should include all of these things. I didn't say that it must. Uh so you can change the you can change the rubric and be more specific and say yes this absolutely must contain all these four criteria. Uh and on dev uh that broke Microsoft my 12 out of 12 agreement became 11 out of 12 agreement. Um and uh it also in the test split it broke Apple uh because Apple didn't have enough uh a big enough risk section as far as the agent was the judge was concerned. Um what that tells you is that I could change my rubric and make my eval worse. Uh I can make my eval less helpful. I can make my EVA less like what a human would do. Uh even though I'm trying to be specific. Um the script also prints precision and recall. Uh precision these are very important things again borrowed from the ML land or rather from stats land.

嗯，精确率问的是：当裁判说某份报告不需要采取行动时，这个判断有多大比例是正确的？召回率问的是：在所有不需要采取行动的报告中，裁判捕捉到了多少？这两个概念听起来很相似，但衡量的是不同的东西。 嗯，在大多数情况下，你会想优先考虑召回率，因为召回率高就能捕捉到所有情况，但也会带来误报。误报的代价是：你花一两分钟查看一份报告，最后发现其实没问题。呃，但漏掉一次故障，就可能让一份糟糕的报告送到客户手上。嗯，所以你必须在两者之间权衡。召回率提高，嗯，精确率就会下降，反之亦然。因此，你得根据自己的应用来决定：是更看重准确判断——也就是丢弃一些实际上正确、但你判定为不正确的内容——还是更看重避免误报，也就是你说某件事很重要，结果却发现并非如此。Um precision asks when the judge says not actionable how often is it correct? And recall asks of all the reports that aren't actionable how many did the judge catch? Those two sound very similar but measure different things. Um, in most cases, you're going to want to prioritize recall because what recall gives you is it catches everything, but it gives you high recall catches everything, but it will give you uh false alarms. A false alarm costs you a minute or two when you look at a report that turns out to be fine. Uh, but a missed failure uh would give you a bad report that reaches a customer. Um, so you have to trade one for the other. If recall goes up uh um then uh precision will go down and vice versa. So for your application you have to decide whether being accurate uh as in throwing away some things that were actually correct uh but but uh you decided they weren't is better or worse than getting false alarms where you say something is important and it turns out not to be.

嗯，评判模型也有偏见。我刚才已经提到过其中一种：位置偏见。给出两个选项时，它们往往会根据模型的不同，更偏好排在前面的那个，或者排在后面的那个。这取决于模型的调整方式。所以，你应该调换选项的顺序，避免它总是以相同的顺序看到这些内容。 它们还有长度偏见，会偏好较长的回答；也有自信偏见，会偏好听起来流畅、自信的回答。我刚才提到的另一种偏见是自我偏好：模型评判自己的输出时，往往会给出更高的评分。这就是为什么我们用 Sonnet 来评判 Haiku。理想情况下，你应该用 OpenAI 的某个模型来评判 Haiku。 不过，在你太担心评判模型有多值得信任之前，值得一提的是，人类评判者之间也经常意见不合。Um judges also have biases. Uh I mentioned one of them already. Um one is position bias. Given two options, they tend to prefer depending on the model either which one comes first or which one comes last. It depends which way the model around which way around the model was changed. Uh but you should swap your options so that it's not always looking at those uh things in the same order. Um they have length bias. Uh, judges prefer long answers and they also have a confidence bias. Judges prefer answers that sound fluent and confident. Um, and the one that I mentioned already is self-preference. A model judging its own output tends to rate it higher. Uh, which is why we judged Haiku with sonnet. Uh, and ideally you judge haiku with something from open AAI. Um, but before you get too worried about how much you can trust a judge, it's worth noting that human judges also disagree with each other a lot of the time.

呃，如果你把相同的输出交给两位专家，按照相同的评分标准进行评估，并测量他们意见一致的频率，就有一个标准分数来衡量这一点，叫作 Cohen's Kappa。呃，对于这类任务，这个分数往往低至 0.2 或 0.3。也就是说，他们彼此一致的程度并没有比随机碰巧一致高多少。嗯 [吸鼻子]，所以，对人类判断来说，门槛——呃，你的评判器的门槛就是人类的表现；而人类的表现其实没那么好，尤其是在人与人相互比较的时候。所以，评判器出现分歧……你不需要一个百分之百都和你意见一致的评判器。呃，你的评判器永远不可能百分之百和你一致。你想要的是一个表现胜过另一位人类评判者的评判器。呃，因为你的评判器可以评判数千个例子，并借此发挥优势。Um, if you give two experts the same outputs in the same rubric and measured how often they agree, uh, there is a standard score for that called Cohen's Kappa, uh, and for this kind of task, it's often as low as 02 or.3. Uh, which is to say they agree with each other not very much more often than just random chance. Um [snorts] so the bar for your human judgment is h for the bar for your judge is human performance and h human performance uh is not actually that good when you compare humans to each other. So having a judge that disagree you don't do not need a judge that agrees with you 100% of the time. Uh your judge is never going to agree with you 100% of the time. What you want is a judge that does better than another human doing the judge would be. uh because your judge can run across thousands of examples and help itself out that way.

嗯，关于评测器，最后一个原则是：失败应该看起来公平。当某项测试失败时，应该能明显看出智能体做错了什么。如果你看到失败结果后觉得“嗯，其实我觉得这看起来没问题”，那问题很可能出在你的评测上。嗯，这里有个著名的例子：Claude Opus 4.5 最初在一个名为 Corebench 的基准测试中只得了 42 分。这个测试考查智能体能否复现科学论文。嗯，Anthropic 对这个低分很不满意。于是有位研究人员查看了实际的对话记录，发现 Claude 只是以另一种方式答对了。嗯，测试要求精确到六位有效数字，而 Claude 给出的答案精确到两位有效数字，而不是四位。嗯，但测试却判定答案错误。任务描述本身也存在歧义。Um one last principle on judges uh is that failures should seem fair. When something fails, it should be obvious what the agent did wrong. If you look at the failure and go, "Well, actually that looks fine to me," then the problem is probably your eval. Um there's a famous example of this uh when Claude Opus 4.5 first scored 42% on a benchmark called Corebench uh which tests whether agents can reproduce scientific papers. Uh anthropic were upset about that low score. Uh so a researcher ran the actual transcripts and found uh that uh Claude was just being uh right in a different way. Uh it the test expected uh six digits of precision and Claude gave the right answer to two digits of four digits of precision rather. Um and the test said that was wrong. There were ambiguous task descriptions.

有些任务根本无法复现。嗯，Anthropic 就逐一检查并修复了所有测试，还表示这些测试可以做得更好。嗯，结果他们神奇地在评测中拿到了 95%。这种情况一直在发生。SWEBench 以及你看到的所有其他基准测试，都在不断地重新评估并由人工调整。嗯，这就是为什么那些著名的基准测试会有第二版、第三版、第四版，因为人们不断发现这些基准测试实际上还有很大的改进空间，而你的评测也会遇到同样的问题。嗯，所以你需要不断重新评估你的测试集。Some of the tasks couldn't be reproduced at all. Uh so Anthropic went through and they fixed all of the tests and they said these tests could be better. Uh and magically they got 95% on the eval. Um this happens all the time. SWEBench and all of the other uh benchmarks that you see are constantly being re-evaluated and manually tweaked and uh that is why you see version two, version three, version four of of uh the famous benchmarks uh because people are constantly finding that the benchmarks are not actually as good as they could be and your evals are going to end up with the same problem. Uh so you need to be constantly re-evaluating your test set.

步骤 7：实验Step 7: Experiments

这就引出了第七步：实验。我们现在有了值得信赖的评估。接下来要用它们来真正改进智能体，并证明修复确实有效。我接下来会讲快一点，刚才节奏一直很悠闲。我开始有点赶时间了。问题是这样的：我们知道有些报告不具备可操作性，也知道原因。所以我们修改提示词，然后呢？怎么知道修改有效？怎么知道没有破坏之前正常运行的部分？如果你只拿几个例子跑一下，再凭感觉看看结果，那就又回到凭感觉了。没有经过测量的提示词修改，只是凭感觉。你真正需要的是受控比较：使用相同的输入，在修改前后进行相同的评估。这就是实验。Uh which brings us to step seven which is uh experiments. We've got evals that we trust. Now we are going to use them to actually improve that the agent and prove that the fixed worked. Uh I'm going to go a little faster now as I've been going on a very leisurely pace. I'm getting beginning to run out of time. Um so the here's the problem. We know that some reports aren't actionable and we know why. So we change the prompt and then what? How do you know that it worked? Uh, how do you know that you didn't break something that was working before? If you run it on a couple of examples and eyeball it, then you're back to vibes. A prompt change that you haven't measured is just a vibe. What you want is a controlled comparison. You want the same inputs, the same eval before and after. That is an experiment.

嗯，所以在 Phoenix 中，你可以把一批经过筛选、标记为不可操作的评估结果选出来，然后将它们添加到一个数据集中。这样你就有了一组数据，可以针对特定内容进行测试。嗯，我这里就不操作了，因为我已经在这边做好了。嗯，有了数据集之后，你可以把它创建成一个实验，然后只针对这个数据集，也就是只针对那些失败的评估结果运行实验，而不必每次都针对整个数据集运行。如果你的数据拆分和测试集规模都不小，这样每次就能省下成千上万次测试。嗯，接下来我们要修复评估发现的问题。所以，如果你还在跟着一起编写代码，恭喜你。打开 prompts.ts。Um so uh what you can do in Phoenix is you can take a bunch of your evals that you have filtered down to not actionable say uh and you can select them and you can add them to a data set. Um that gives you a set to test against uh against specific things. Um I'm not going to do that since I've already done it over here. Um Um once you have a data set you can create turn that into an experiment and run uh your experiment against only the data set only against the evals that were failing as opposed to against your entire set every single time which you know if you've got a decent sized split and a decent sized test set is saving you thousands and thousands of tests every time. Uh now we're going to fix what the eval found. So, uh, if you're still coding along, congratulations. Open prompts.ts.

每个提示词都有一个改进版本。呃，或者你也可以自己编写。嗯，每项修改都对应一项发现。因此，评审器说没有提出任何建议，我们就要求它明确给出建议。它说提到了风险，却没有证据支持，所以我们要求它提供有数据支撑的风险。忠实度评估发现了编造的数字，于是我们要求它只使用研究中出现的数字。呃，阅读追踪记录发现，代理一直在尝试保存文件。呃，所以我们告诉它改为直接回复报告。我们不是在瞎猜，而是根据之前评估的结果，为代理制定具体的操作要求。呃，这就是实验脚本的样子。一个实验由数据集、任务和评估器组成。这个数据集就是我们刚刚创建的那个。There is an improved version of each prompt. Uh, or you can write your own. Um, and every change maps to a finding. So, the judge said, uh, makes no recommendation, so we asked for an explicit recommendation. It said mentions risks with no evidence. So, we asked for risks with supporting data. Faithfulness found invented numbers, so we said use only numbers that appear in your research. uh and reading traces found that the agent uh was trying to save files all the time. Uh so we tell it to reply with the report instead. We're not guessing here. We are coming up with specific things for the agent to do as a result of the previous results of our evals. Uh this is what the experiment script looks like. An experiment is a data set, a task and evaluators. The data set is the one that we just created.

嗯，这个任务是一个函数，会在每个示例上运行智能体。嗯，在这个例子中，我们运行的是整个智能体，但如果需要，也可以只针对智能体行为的特定部分或某个特定子任务运行。嗯，我们的任务会返回报告和研究内容。因此，每个评估都有一个小型包装器，用来挑出它所需的字段。嗯，接着，忠实度评估会将研究内容作为上下文。和之前一样，Phoenix 会在每个示例上运行任务，对每个结果进行评分，并存储所有内容，以便你进行比较。如果你想运行这个任务，可以运行 `npm run experiment`，大约需要四分钟。不过我时间不多了，所以不会等这四分钟。嗯，我已经提前跑过了。你需要多少个示例？嗯，这是个非常好的问题。Uh the task is a function that just runs the agent on each example. Uh in this case, we're running the entire agent, but we could be running it on a specific subset of the agents behavior or a specific subtask uh if we wanted to. Uh our task hands back the report and the research. So each eval gets a small wrapper that picks out the fields that it needs. Uh and then faithfulness gets the research as its context. The same as before, uh Phoenix runs the task on every example. It scores every result and it stores everything so you can compare. If you want to run this, you can run npm run experiment and it will take about four minutes to run. Um, but I'm running out of time so I'm not going to wait for four minutes. Uh, I ran mine ahead of time. How many examples do you need? Uh, is a really good question.

呃，10 到 20 个示例能让你大致了解事情的发展方向。在开发阶段非常有用。嗯，如果要投入生产环境，你会需要几百个示例。嗯，这需要很长时间，但为了提高准确率，非常值得。嗯，要了解准确率的误差范围，你需要把示例数量增加到四倍。所以，从 25 个示例增加到 100 个示例，确实非常有帮助。呃，而且要付出的额外努力并不算多，就能带来这么大的提升。再往上就得增加到 400 个示例，这就开始让人觉得你大概根本不会费心去做了。呃，不过示例越多越好，呃，把越多生产环境中的失败案例加入数据集也越好。嗯，我把自己的测试跑了三遍，原因是代理的运行结果并非确定性的，每个示例每次运行都可能把你带向错误的判断。嗯，理解代理多次运行时的表现，有两种很有用的思路。Uh, 10 to 20 examples gives you a rough direction of where things are going. Is very useful in dev. Um, if you're getting into production, uh, you're going to want hundreds of examples. Um, that is going to take a long time, but is, uh, very worth it for accuracy. Um, for every, uh, to have your error bars on your accuracy, you need to quadruple the number of examples. So, uh, going from 25 examples to 100 examples is really, really helpful. uh and is not that much effort going to get that big a jump again you'd have to go to 400 examples which begins to get into the territory of you're never going to bother to do this uh but the bigger the better uh and the more production failures you throw into your data set the better um so I ran mine three times and the reason I did that uh is because the agent is non-deterministic each run of each example uh can mislead you in either direction um And there are two useful ways of thinking about how an agent runs uh when it runs multiple times.

第一个叫 pass@K，也就是说，它在 K 次尝试中至少成功一次吗？然后是 pass^K，也就是说，它每次都会成功吗？我运行 K 次时，它能不能每次都通过？随着 K 增大，这两个数值会逐渐分化。给它的尝试次数越多，pass@K 成功的可能性就越大，因为它只需要至少成功一次。给它的尝试次数越多，它至少失败一次的可能性也越大，因此 pass^K 的数值会下降。编程助手第二次尝试就答对，这就够好了，所以 pass@K 就可以。支持机器人每五次有一次答错，那就是个大问题。金融分析师给出不准确的建议，也会造成很大问题。所以，对于这些场景，你需要看 pass^K，也就是它每次都能通过。The first is called pass at K which is uh does it succeed at least once in K tries and then there is pass to the K which does it increase every single time uh does it pass every single time I run uh K times. Um so as K goes up those two numbers diverge. Uh the more tries you give it, the more likely pass at K is to succeed because it succeeds at least once. The more tries you give it, the more likely it is to fail at least once and then pass it K pass to the K will go down. Um a coding assistant gets that gets it right on the second try is fine. So pass it K is fine. Um a support bot that's wrong runtime in every five is a huge problem. And a financial anal analyst uh that gives inaccurate recommendations is also a huge problem. Uh so you want pass to the K. You want it to pass every single time for those.

嗯，改进智能体时，应该按照调整的优先顺序来做，依据是每项调整能带来多大帮助。首先是数据质量。如果底层数据质量很差、难以阅读，或者智能体或评估器难以理解，那么你做的其他任何事情、进行的任何调整都不会有用。数据质量达标后，就可以转向提示词。这也是我们今天大部分时间都在做的事：通过修改评分标准和智能体，在提示词层面改进效果。然后是模型选择。模型选择能带来很大的影响，但同时也会改变许多东西。而且，通常你更换模型时，会换成更大的模型，这样成本也会更高。Um when you're improving an agent, there's an order of operations based on how useful making a change is going to be. The first one is data quality. If your underlying data is bad, if it's hard to read, if it's difficult to interpret for the agent or for the judge, uh then nothing else you do, nothing else you tweak is going to be useful. Once you've got good quality data, then you move to the prompts. Uh that is what we've been messing with mostly today. Uh is to make things better at the prompt level by changing our rubric, by changing our agent. Then there's model choice. Uh model choice has a lot of leverage, but it also changes a whole bunch of stuff at the same time. uh and also the usual way that you're going to change your model is you're going to change it to a bigger model and that makes things more expensive.

所以这就会带来一种权衡，而你不一定总是想做出这种权衡。最后还有超参数。超参数就是 temperature 之类的东西。它们实在没什么用，以至于现代模型都不再允许你调整它们了。你不能在现代 Claude 模型中传入 temperature。所以，如果你正打算这么做，这是你能传入的参数里最没用的一个。看看我的实验的话，你可以查看你的种子数据，看到同样的实验。你会发现，随着我的运行效果不断改善，我的结果从 16 次中成功 15 次，提升到了 48 次中成功 48 次；另一个结果则从 16 次中成功 5 次，提升到了 48 次中成功 48 次。而忠实度从 16 次中成功 2 次，提升到了 48 次中成功 3 次。这不太理想，对吧？我的提示词都改进了，忠实度本应该做得更好才对。So that is uh creates a tradeoff uh that you don't always want to make. Uh and finally there's hyperparameters. Hyperparameters are things like temperature and stuff like that. Uh they're so useless that uh people have that modern models have stopped allowing you to tune them. You can't pass temperature to model to modern claw models. Um so if you're tempted to it is the least useful thing that you can pass. Um so if we're looking at my experiments you can look in your seed data and get see see the same experiments. Um you'll see that uh as I improved as I improved my runs I went from uh 16 out 15 out of 16 to 48 out of 48. and went from 5 out of 16 to 48 out of 48. And faithfulness went from 2 out of 16 to 3 out of 48. Uh, which is not that good, right? Like we really should have done better on faithfulness with my improved prompt.

嗯，重要的是，之前能通过的东西有没有被弄坏？我们可以筛选出通过项的细分结果。我现在不会在 UI 里操作。不过，我改进提示词后，之前通过的五份报告仍然都通过了。所以，可操作性直接达到了完美。但忠实度很差。忠实度从 16 项中得 2 项，变成了 48 项中得 3 项。尽管提示词里明明写着只能使用研究中出现的数字。所以，我们只用提示词中的一行就修复了测量结果。但这并没有解决智能体编造内容的问题。这就是我们接下来必须为这个智能体解决的问题。我们已经解决了它能否做出合理判断的问题，但还没有解决它有时会捏造数据的问题。如果愿意，我们可以查看每次运行的情况，你会发现，由于它具有非确定性，每次运行给出的建议都不一样。Um, the uh important thing is did anything that used to pass break? We can filter down to the passes split. I'm not going to do it in the UI right now. Uh, but all of the five reports that passed previously as I improved my prompt still passed. Uh, so actionability went straight to perfect. Um, but faithfulness was bad. Faithfulness uh went from 2 out of 16 to 3 out of 48. Uh, even though the prompt literally says only use numbers that appear in your research. Um, so we fixed what we measured in a single line of the prompt. Uh, didn't fix the agent making things up. That is the next thing that we have to fix for this agent. Uh we have fixed whether or not it is making a good judgment call. Uh but we have not fixed uh that it is uh inventing the data some of the time. Um if we want to we can look into uh each of the runs and you'll see that each of the because it's nondeterministic each of the runs are coming up with different advice.

嗯，不过我不会这么做，因为我时间不多了，而且还想讲讲其他内容。呃，我们来看看……好，太棒了。嗯，一个不错的评测，比如我刚才提到的，在我的 16 个数据集上采用 56 和 511 的划分。这就叫能力评测。能力评测指的是我们的智能体目前还做不到的事情。它为我们设定了一个可以衡量自己攀登进度的目标。呃，当它能够稳定通过时，也就是当某项测试每次都能通过时，你不会把它丢掉，而是把它变成所谓的回归评测。这种评测可以放进 CI 里，然后告诉大家：它过去每次都能通过。如果它突然不再每次都能通过，我们就可以把这视为失败，并且应该停止发布。嗯，问题是我们该怎么攀登这个目标。呃，我们可以再改进提示词，或者……Um, but I'm not going to do that because I'm running out of time and I want to get to some other stuff. Uh, let's get to Okay, cool. Um, a good eval like the 56 the 511 split that I was talking about on my 16 data sets. That is what is called a capability eval. A capability eval is a thing that our agent can't do yet. And it gives us a hill that we can measure ourselves climbing. Uh when it passes reliably, when you get a test that passes 100% of the time, you don't throw it away. You turn it into what is called a regression eval. That is the kind that you can put into your CI and say this used to pass 100% of the time. If it suddenly stops passing 100% of the time, then we can call that a failure and we should stop release. Um how do we climb this hill is the question. uh we could work on the prompt again or uh

Upgrading to Claude Haiku 5.5

我们可以做我之前说过要做的事，也就是升级模型。呃，你可以传入一个新模型 Haiku 5.5，再运行 npm experiment，重新进行实验。嗯，我今天一开始就说过，不做评估就不能切换模型。而这就是在有评估的情况下切换模型的样子，因为现在我们知道自己正在衡量所有这些指标。嗯，重复三次需要 13 分钟，比我们剩下的时间还长。所以我就直接跳到结果吧。呃，这是 Haiku 4.5 和 5.5 的对比。忠实度提高了，效果比之前好三倍多。不过它还是很糟糕。这个智能体还是一直在胡编乱造。呃，但调用的一致性变好了。所以至少它会一遍又一遍地做出相同的调用。这算好吗？we could do the thing that I said that we would do which is we can upgrade the model. Uh so you can run the experiment again by passing in a new model haiku 5.5 and then run npm experiment. Um, I said at the start of today that you can't switch models without eval. And this is what it looks like when you can switch a model with an eval because we know now that we are measuring all of these things. Um, with three repetitions, it takes 13 minutes, which is more time than we've got left. So, I'm just going to skip to the results. Uh, here is Haiku 4.5 versus 5.5. Uh, faithfulness improved. It got more than three times better. It is still terrible. This agent is still making things up all the time. Uh, but consistent calls got better. So, it is at least making the same call over and over. Is that good?

呃，这也让成本变成了原来的四倍。值得吗？呃，这种事在生产环境中要不要做，需要你自己判断。嗯，它变得更一致的一个原因——它在不同调用之间不再一会儿建议买入、一会儿建议卖出的原因之一——就是它变得更保守了。哈，结果 Haiku 5.5 就是一直说持有。呃，所以这是更好还是更糟？大概是更糟。呃，我们可能需要继续迭代，解决 Haiku 5.5 对很多事情过于保守的问题。呃，但至少现在我们知道有些东西变了，而且保守过头是 Haiku 5.5 的一个问题。嗯，这是我发现的问题。它变成了在 45 次调用中，有 41 次说持有——41%的时候，也就是 41 次。嗯，不同运行之间完全一致。Uh, it's also costing four times as much money. Is that worth it? Uh, is a question that you need to make for yourself in production when you're doing this kind of thing. Um, one of the ways that it got more consistent, one of the ways it stopped varying between buy and sell between calls, uh, is it just got more conservative. Haiku 5.5 uh it turns out just says hold all the time. Uh so uh is that better or worse? Probably worse. Uh probably we need to iterate on Haiku 5.5 being too conservative uh about things. But at least we know now that something has changed and that the conserviveness is a problem with Haiku 5.5. Um that is the problem that I found. It went it went to 41 of out of uh 45 calls it said uh hold 41% of the on 41 of the times. Um which is perfectly consistent between runs.

嗯，所以它在我刚才提到的那个一致性检查上做得更好了，也就是一致性调用。不过，它是靠每次都选同一个答案才做到这一点的。Uh so it it got better at the uh consistency check that I was talking about over here the consistent call. Uh but it got there by just picking the same answer every single time.

主持人：[嗤笑]Host: [snorts]

嘉宾：呃，不过这就是评测迭代周期。呃，价值在于这个周期，而不是任何人的评分。你运行评测，阅读失败案例和解释，发现其中的规律，修改提示词、工具或数据，然后再次运行实验。效果变好了吗？呃，现在你的评测可以替你回答这个问题。你可以从“我觉得它有效”变成“我可以证明它有效”。呃Guest: Um but that is the eval iterate cycle. Uh and the cycle is where the value is not anyone's score. You run your evals, you read the failures and the explanations, you spot the pattern, you change the prompt or the tools or the data and you run the experiment again. Did it get better? Uh your evals can now answer that question for you. You can go from I think it works to I can show you it works. Uh

CI 中的评测Evals in CI

我刚好有足够的时间来讲讲最后一步怎么做，也就是在 CI 中运行评测。嗯，越来越多快速交付 AI 产品的团队会先编写评测，再开发功能。这叫作评测驱动开发。比如说，你希望客服代理在退款前一定先核实客户身份。你可以先编写评测来检查这一点。这样一来，「完成」就有了明确的定义，也有了一个从一开始就可以衡量、逐步攀登的目标。我们已经讲过能力评测和回归评测。下面来看看如何创建一个在 CI 中运行的评测。如果你从未用过 VEST，它是一个 JavaScript 测试运行器，而 Fenix 可以直接接入它。打开 agent.eval.ts，你会看到它看起来就像一个普通的测试文件，因为它确实就是。里面有一个 describe 块，并且针对回归测试集中的每个查询都编写了一个测试。I have just enough time to talk about how to do the last step which is eval in CI. Uh more and more teams that ship AI quickly write the eval before the feature. That's called eval driven development. Um say you want your support agent to always check who the customer is before giving a refund. Uh you can write the eval that checks for that first and now done has a definition that you can measure a hill that you can climb from the start. Um, we talked already about capability capability evals and regression evals. Um, here's how uh to create a eval that runs in CI. Um, if you've never used VEST, VEST is a JavaScript testr runner and Fenix can plug straight into it. Uh, if you open agent.eval.ts, you'll see that it looks like an ordinary test file because it is one. There is a describe block and a test for each query in our regression set.

所以有可口可乐、苹果对微软、Rivian，还有假的 ZZZQ。嗯，每项测试都会运行代理，将报告记录为输出，然后在报告上运行我们的评估。也就是我们之前写的同一个股票代码检查和可操作性评判器。CI 不需要添加新的评估。嗯，然后在底部，我们根据验收标准来决定是否通过构建。这就是我之前提到护栏和北极星指标的地方。提及股票代码是一个护栏，必须是在讨论正确的公司。嗯，可操作性是一个目标，平均分必须保持在 0.8 或更高。因为 LLM 评判器对单份报告的评分可能会有些波动，所以我们根据平均分来决定是否通过，而不是只看一份报告。然后你可以运行 `npm test`，它的运行方式就像任何测试套件一样。嗯，最后，报告器会打印出一份评分卡。嗯，这大约需要四分钟，比我剩下的时间还长。So Coca-Cola, Apple versus Microsoft, Rivian, uh, and the fake ZZZQ. Um, each test runs the agent, logs the report as the output, uh, and then runs our evals on it. And the same ticker check and actionability judge that we wrote earlier. We don't have to add new evals for CI. Uh, then at the bottom, we gate the build on acceptance criteria. This is where I talked about guardrails and northstars. Mention sticker is a guardrail. It has to be talking about the right company. Um actionability is a target. The average has to stay 08 or better. Uh because an LLM judge can be a bit noisy on a single report. So we gate on the average uh not on just one. Uh and then you can run npm test uh and it runs like any test suite. Um and at the end the reporter prints out a scorecard. Um, and that'll take about four minutes, which is more time than I've got left.

嗯，构建结果是红色的。如果构建失败，就会看到这样的结果。四项测试都运行了，每份报告都提到了股票代码，但 Rivian 的报告不具备可操作性。因此，可操作性平均分是 75。呃，这并不是说你失败了，只是说你没达到要求。嗯，绿色运行则表示所有项目都通过了。我就不展示实验视图了，因为我又快没时间了。构建失败时，你得到的不只是一枚红叉，还会有一段说明，告诉你结果是好是坏，以及出错的原因。这才是最重要的。这个测试套件也可以在 GitHub Actions 中运行，它会将 Phoenix 作为服务容器启动。现在，我们已经完成了我之前说过你可以做的所有事情：运行评估、测试各项功能、确保结果符合你的人工判断、将其纳入 CI，并确保构建是否通过取决于 LLM 对你所编码的“良好”定义所作的判断是否确实有效。Um, a red build. If a failed build, if your build fails, this is what you'd get. All four tests ran and every report mentioned its ticker, but Rivian's report wasn't actionable. So, the actionability average came out at 75. Uh, it doesn't say you failed. It just says that you missed. Um, but, uh, a green run, uh, is where all of those pass. Um, I'm not going to show the experiment view because again, I'm running out of time. Uh, but when a build fails, you don't just get a red cross. You get an explanation that says whether things are good or bad and why they went wrong. And that is the important thing. Uh, this same test suite can be run in GitHub actions. Uh, it starts Phoenix as a service container. Uh, but now we've done everything that I've said that you can do. uh you can run your evals, you can test things, you can make sure uh that it matches your human judgment, you can put it into CI and you can make sure that your builds are gated on whether or not uh the LLM's judgment of what you've encoded as the definition of good uh really works.

嗯，我特别喜欢这一点：现在，任何知道什么样的结果才算好的人，都能编写评测了。比如你的产品经理、支持团队，或者那个总是听到同一个投诉的销售人员。他们不需要写 TypeScript。嗯，他们只要描述什么样的回答才算好，然后把描述交给你；更好的做法是直接交给编程智能体，让它为他们编写所有评测。嗯，你不应该在每次提交代码时都运行整个智能体，因为那会花很长时间，还会让你多花一大笔钱。嗯，不过现在你已经拿到了我答应给你的代码仓库，也有了所有代码和示例。你可以看看每件事是怎么做的。嗯，你可以让智能体参考它，然后说：“照着这个做，但要针对我的代码库。”嗯，智能体会自己弄明白。Um and there's something I love about this which is that anyone who knows what good looks like can write evals now. uh your product managers, your support team, your person in sales who keeps hearing the same complaint. They don't need to write TypeScript. Uh they just need to describe what a good answer looks like uh and give it to you or even better just give it to a coding agent which will then write all of the evals for them. Uh you shouldn't run your whole agent on every single commit because uh it's going to take a really long time and cost you a whole bunch of money. Um, but now you've got the repo I promised you. You've got all of the code. You've got all of the examples. You can see how everything is done. Uh, you can point your agent at it and say, "Do this, but for my codebase," uh, and your agent will figure it out.

嗯，今天有一件事我们没来得及构建，那就是上线之后，你应该持续进行评估。你应该在一部分真实流量上运行评估，并在分数下降且持续低迷时发出警报。嗯，不是每次失败都报警，因为单次失败只是噪声。嗯，但如果失败呈现出趋势，生产环境监控就非常有用，而评估也能帮你做到这一点。这就是我们构建的完整闭环。我们追踪、阅读、评估、检查、改进，并通过 CI 将其固化下来。嗯，不过你不必……Um, there is one thing that we didn't get time to build today, uh, which is, uh, once you're live, you should keep evaluating. You should run your evals on a sample of real traffic, uh, and alert when scores drop and stay down. um not on every single failure because one failure would be noise. Um but if you get a trend in failures, uh production monitoring is incredibly useful and eval can get you that as well. So this is the full cycle that we built. We traced, we read, we evaluated, we checked, we improved, uh and we locked it in with CI. Uh but you don't have to

周一从哪里开始Where to start on Monday

把你今天学到的所有内容都用到周一的工作中。你可以从小处着手，比如先给你的智能体添加监测。也可以先读几条追踪记录。阅读 12 条追踪记录，你会获得仅仅查看输出时得不到的信息。所以，运行起来看看结果，然后在此基础上继续完善。试着在自己的应用上用一下吧。这就是我说过包含所有内容的代码库。里面有 Phoenix 文档。Phoenix 本身是开源的，所以你可以折腾代码，并告诉我们哪里写错了。时间也差不多了。非常感谢大家。do all of the stuff that you've learned today on Monday. You can start small. You can just instrument your agent. you can just read some traces. You will get information reading 12 traces that you wouldn't have got uh by just looking at the output. Um so run them. Look at the results and build from there. Uh so go try it on a app of your own. This is the repo that I said that everything was in. Uh the Phoenix docs are there. Uh Phoenix itself is open source. So you can mess with the code and tell us that we got stuff wrong in there. Uh and that just on time is it. Thank you very much everybody.