如果你的 AI 评估拿到 100 分，评估就错了：Ramakrishnan Lokanathan 谈测试设计

简要说明Brief Description

关于高风险 AI 应用评估的实践探讨，涵盖宏大目标、工作流测试、游戏、回归检查，以及从真实失败中汲取的反馈。A practical discussion of evaluation for high-stakes AI applications, covering ambitious goals, workflow testing, games, regression checks, and feedback from real failures.

目录Table of Contents

• 引言
  Intro
• eval 到底是什么？
  What is an eval, really?
• 关于我
  About me
• 为什么评估现在如此重要
  Why evals matter now
• 1. Define the most ambitious success
• 2. Test the whole workflow
• 3. One score is not the system
• 作为评测的游戏：eSlams
  Games as evals: eSlams
• 4. Keep testing what used to work
• 5. Turn real failures into tests
• 回顾
  Recap

简介Intro

我觉得我们大家之所以主要聚在这里，是因为目前的评测问题很多，对吧？我们想要探讨的，也正是如何着手解决这个问题。我认为，评测之所以常常出问题，往往是因为我们把事情搞得太复杂了，而且其实并没有真正弄明白，对吧？我们会听到诸如“评测比模型更重要”或者“评测比难度之类的东西更重要”这样的说法，但评测究竟是什么呢？I think why all of us are here primarily is because evals are currently very broken, right? And like that is what we're trying to explore and understand how to go about it. More often than not, I think evals are broken because we overcomplicate it and we don't really understand, right? We hear things like the eval is more important than the model and the eval is more important than the hardness or whatnot, but what even is an eval, right?

评估到底是什么？What is an eval, really?

这是一个我们多少都会遇到的问题。我和构建高风险应用的人交流时，总会碰到这样一种情况：他们甚至需要从第一性原理的层面理解 eval 到底是什么，对吧？而 eval 只是 evaluation（评估）这个词的缩写，对吧？我们对评估的理解，就像我们知道考试——也就是测验——一样；这是一个我们在人类身上都能理解的概念，对吧？如果你仔细想想，作为人，我们一生中会参加各种各样的考试，比如 SAT、GMAT、律师资格考试之类的，不管我们从事什么。It's a question that we all sort of come to and like this is something that I consistently hit when I speak with people building high-stake applications to even understand to the first principle level of what an eval is, right? And what is an eval is just a abbreviation of the term evaluation, right? And how we understand evaluation is like we know exams, which are examinations and this is a concept that we understand in human beings, right? And if you think about it, as people we write a variety of exams through our life, like we write the SATs, the GMAT, the bar exam and like regardless of what we do.

这就是同一种思路，可以平行地应用到评测中。你可以把评测看作是对 AI 系统的考试，对吧？比如 SWE-bench、人文学科考试、法律考试等等。从多个角度来看，这些评测都是在尝试评估并了解模型或智能体在某项任务中的表现，对吧？That is the same sort of idea you can apply as a parallel into evals and then you can think about evals as those examinations for AI systems, right? You have the Sweet Bench, the humanities law exam and all of this. In multiple ways, this is trying to evaluate and try to understand how a model or an agent will be working in something, right?

归根结底，考试和评估不过是一些任务、标准和证据，用来评估和了解某件事在某方面的表现是否符合预期，对吧？我认为这也引出了一个非常关键的区别。考试必须回答一个范围明确的问题。以 SAT 为例，它是一项大学入学考试，能告诉你学生的语言和数学水平如何，但这只是衡量学生是否为上大学做好准备的一个指标，对吧？我来自印度，即使在今天，印度的大学录取也只通过一次考试来决定。At the end of the day, examinations and evals are just tasks, standards, and evidences to sort of evaluate and understand how something is to perform on something as you expect, right? And I think that brings us to a very key distinction here as well. An exam must answer a bounded question and let's take the example of the SATs. it's a college entrance exam and it tells you how good a student is on language and math, but it's just one signal of a college readiness, right? I come from India and even today in India your college admissions are done with a single examination.

不过，假设你看看美国的考试，SAT 是衡量大学入学准备程度的一个信号，对吧？我认为这一点极其关键。你不应该只看某一项评估或某一次考试；它应该只是衡量你是否已准备好进入平台的一个信号，而不是过度看重你的模型在文科课程考试或 W bench 上表现得特别出色，对吧？However, let's say you look at the US exam, SAT is one signal of college readiness, right? And I think that's extremely key. You should not index on one eval or one test and it should just be one signal of your readiness to the platform rather than trying to sort of over index on your model being really good at humanities class exam or at W bench, right?

所以，没有任何评估或测试能够衡量一个人的全部能力，这一点从一开始就应该明确。归根结底，我们通过评估试图回答的是：在任何给定时刻，我们的表现是否符合我们向消费者或用户作出的承诺，对吧？任何评估的职责，就是确保我们所提供的产品或智能体兑现对消费者作出的承诺，达到相应的性能水平。So, no eval or no test is a measure of someone's complete abilities and that should always be clear going into that. So, at the end of the day, what we try to answer with evals is is our performance matching the promise that we're making to the consumer or the user at any given point in time, right? And that's the job of any eval to make sure that we go ahead and ensure that the promise of the product of the agent that we're providing is meeting its performance to the consumer.

关于我About me

在继续之前，我想先简单介绍一下自己。我叫拉马克里希南，来自印度南部名为科钦的城市。我很早就开始编程，高中时就开始构建机器学习分类器之类的东西。我的职业生涯始于一家印度教育科技独角兽企业。2021 年，这家公司让我有机会观看一款基于 GPT-3 的产品演示，了解如何改进我们的内容流程。这次经历改变了我的想法——毕竟我从小就非常热爱机器学习。Before we move ahead, I wanted to sort of give a small brief on who I am. I'm Ramakrishnan. I come from the city called Cochin in South of India. I started coding very early on and started building machine learning classifiers in high school and whatnot. And I started my career with an Indian EdTech unicorn which gave me the opportunity in '21 to get a demo of a GPT-3 based product on how to better our content flow which changed my mind being someone who really loved machine learning from an early age.

我当时就觉得，这真的会改变世界，对吧？我最引以为傲的事，或者说我当初做对的事，是在 2022 年 9 月，我们决定会出现一波新应用，它们将基于 GPT-3 API 构建，对吧？而我们全职投入大约两个月后，OpenAI 发布了 GPT-3.5 的研究预览版，名叫 ChatGPT。这几乎彻底改变了我们的生活，验证了我们的判断，同时也让我们遭受了重创，对吧？And I was like, this is going to really change the world, right? And my claim to fame or like what I was right in doing was in September 2022, we decided that there is a new wave of applications that are going to be possible that are going to be built on the GPT-3 API, right? And about 2 months after we went full-time, OpenAI released their research preview for GPT-3.5 called ChatGPT and like that literally changed life for us and it validated us and killed us as well, right?

所以，我们经历了多次转型，最终做出了一个宠物护理 AI 助手，并在此基础上通过电商和推荐引擎实现了变现。后来我们在印度被收购，这段旅程也就此结束了，对吧？之后，我一直对了解这些系统如何演进非常感兴趣。由于我们很早就开始基于 GPT-3 构建产品，评测对我们来说至关重要。因此，我后来成为了 Nematron 3 模型的红队测试员。目前，我在劳伦斯伯克利国家实验室从事 AI 安全研究，也参与伦敦商学院的数据科学与 AI 倡议。总体来说，我们是想探索如何审视我们正在开发的新技术，以及如何看待并探索它，对吧？So, went through multiple pivots and then we eventually landed on a pet care AI assistant and we monetized it using e-commerce and recommendation and engines on top of it and we got acquired in India and that was the end of that journey, right? after that, I've been super interested in understanding how these systems evolve and because we were super early on building on GPT-3, evals were critical for us to build. So, that took me to being a red teamer at for the Nematron 3 models. I work at the Lawrence Berkeley Lab right now on a AI safety research and also with the London Business School data science and AI initiative. Broadly to explore how to sort of look at the new technology that we are working on and then how we see and explore, right?

所以，我并不偏向这里的任何人，也不打算为任何供应商背书。So, I am not aligned or like, you know, with anybody here, so I'm not going to sponsor a vendor.

我只是想了解你如何看待评估，以及如何构建这些高风险应用，对吧？还有，抱歉，我想把这些年来我进行过的对话，以及我们为数百万用户使用 AI 产品所做的多个实现方案整合起来。It's just to understand how do you think about eval and like build these high-stake applications, right? And the consolidation of the conversations that I've had over time with multiple implementations that we have done with millions of users using our AI product, sorry.

为什么评估现在很重要Why evals matter now

所以，这是我深信的一点：十年后，评测将不再那么重要。因为如果我们相信模型能力会持续提升，我们就不必像现在这样做那么多评测。不过，在中期阶段，生产环境等不了。So, this is something I strongly believe. So, evals are not going to be relevant in a decade's time. Because if we are to believe that model capability is to keep improving, we will not have to have as much eval as we do. However, in the midterm, like production cannot wait.

问题在于，如果没有恰当的评估体系，就会出现类似 MIT《生成式 AI 鸿沟》报告中的情况：95% 的组织无法获得任何回报，因为你们在生产环境中并没有兑现自己希望借助 LLM 实现的承诺，对吧？这就是我们今天必须通过评估来解决的问题。十年后，评估可能只是一种验证手段。但今天，我们要确保你们在生产环境中兑现自己所追求的承诺，对吧？The problem is if you don't have the right amount of eval structure, you get reports like the MIT GenAI divide where 95% of organizations are not able to get any returns because you are not delivering the promise in production that you're trying to make with the LLM, right? And that is what we have to solve with evals today and in 10 years time, evals is just going to be a validation of this. But today, it is to make sure that in production you are delivering the promise that you're going ahead for, right?

1. 定义最宏大的成功目标1. Define the most ambitious success

所以，在为高风险应用构建评测时，首先要考虑的是定义最宏大的成功目标，对吧？在设计任何测试之前。比如，假设是在 AGI 的情况下，你的 AI 智能体会做什么？或者说，你能从中获得的最佳结果是什么，对吧？So, the first thing that you have to think about as you build evals for high-stake applications is to define the most ambitious success, right? Before designing any test. Like, assume in in of AGI, what is your AI agent going to do? Or what is the best outcome you can get out of it, right?

而设定这个目标，确实能让你明确接下来可以进一步设计的范围，对吧？你可以从这项工作入手。首先要问的一个简单问题是：必须发生什么？绝不能发生什么？什么时候应该停止？你还可以拓展到其他以疑问词开头的问题，并从第一性原理出发，思考你能从中实现的最具雄心的成果是什么，对吧？And setting this goal really gives you the scope of what you can go ahead and design further, right? And you start with the job. And simple the first question that you ask is what must happen. And what must never happen? And when should it stop? And you can extend it into other WH questions and first principles to go ahead and think about what could be the most ambitious outcome that you can get out of it, right?

要着眼于你需要完成的工作，而不是你能拿到的分数。如果某项评测得了满分100分，那这项评测就有问题，对吧？And aim at the job you need and not the score you can get. If you have an eval that is scoring a 100 of a 100, your eval is wrong, right?

而且你的目标还不够远大，测试也做得不够，对吧？看看如今任何模型评测，大家的成绩最多也就 85、86 分。没人能拿到 100 分。所以你得确保评测标准和目标都超出模型当前的能力，这样才能真正对它进行压力测试，而不是把测试弄得太简单，对吧？And you're not ambitious enough and you're not testing enough, right? If you look at any model eval today, people are at like 85, 86 max. Nobody gets to 100. So, you have to make sure that the evaluation and your ambition is beyond what the model ability today is for you to make sure that you're putting it to the stress test and not making it easy, right?

2. 测试整个工作流2. Test the whole workflow

而且要始终测试整个工作流程，对吧？不只是测试最终答案。而这是原始图片，我的要求只是更换背景，对吧？我现在只是在看输出结果。输出把背景换成了奶油色，但现在多了两个把手，对吧？And always test the whole workflow, right? And not just the final answer. And this is the original picture, and my request would be only change the background, right? And I am just looking at the output. And the output has changed the background to a cream background, but it has two handles now, right?

所以，必须明确一点：你必须始终对工作进行测试。这只是一个非常简化的例子。假设你的工作流程有 100 个步骤。每一步都必须评估其输入和输出，以及预期结果是什么。So, it has to be very clear that you have to always test the work. And this is a very simplified example of this. Let's say you have a 100 steps in your workflow. Each step has to be evaluated for what goes into that step and what comes out of that step and what is the expected outcome.

你不能只看起点和终点，然后说：“好吧，如果看起点和终点，背景变了。”但这就是你想要达成的目标，对吧？You can't look at the start point and the end point and be like, "Okay, if you look at the start point and the end point, the background has changed." But is that the objective that you're trying to achieve, right?

所以，在处理事情时，总是要着眼于整个工作流程，对吧？So, always for look at the whole workflow when you're approaching things, right?

3. 一个分数并不是整个系统3. One score is not the system

第三点，我也想稍微谈谈我一直在这个领域着手做的一件事：单一分数并不等于整个系统，对吧？我们的想法是构建一个全面的评估体系。2022 年和 2023 年刚开始使用模型时，你评估的是编程能力，或者写作能力，等等。但如今，你面对的是能够持续运行、具备智能体能力的系统，它们必须完成多种任务，对吧？And the third thing, and I'm going to talk a little bit about something I've been working on in this space as well, is one score is not the system, right? And the idea is to build a holistic evaluation system. When you started off with models in 2022 and 2023, you were measuring for coding ability or let's say you were measuring for writing ability, etc. But today you have agentic persistent systems that have to do multiple things, right?

因此，你必须确保始终在设计一套全面的评估系统。首先，要把自动化检查和评估纳入 CI 流水线，对吧？每次你的智能体经历生产环境部署流程时，都要向它提出评估问题，对吧？将这部分纳入自动化检查，可以确保你不会只在决定采用哪个模型时才看一次评估分数。So you have to make sure that you're designing a holistic evaluation system at all points in time. And the first element of that is that automated checks and eval being part of your CI pipeline, right? And your eval questions have to be asked to your agent every time they are going through a production deployment process, right? So that being part of the automated checks makes sure that it's not like one eval score you are taking at the time you're deciding the model.

持续进行人工审核，确保结果合理。任何时候都不要依赖你的智能体来审核评估结果，对吧？你可以借助它加快评估流程，但要确保进行人工审核。第三部分是对抗性测试，对吧？你可以在哪里打破工作流，把它置于具有挑战性的情境中，从而进行对抗性测试？我最近一直在研究一个想法，也很想聊聊：如何选择能够揭示潜在故障的测试，对吧？Continuously human review and make sure that the results make sense. Don't rely on your agents to be the reviewer of your eval at any point always, right? You can fast-track your eval process with it, but make sure you're doing human review. And the third part is adversarial test, right? Where can you break the workflow and like put it into challenging situations like where you can have adversarial testing? And one thing I've been recently working on and would love to talk about is the idea of how you can choose the test for the failures it can reveal, right?

作为评测的游戏：eSlamsGames as evals: eSlams

还有一个想法，就是让测试能够反击。如今，你在 SW bench 上有一项任务，智能体或模型会被要求执行一项任务并给出结果。但它不会持续思考或评估，对吧？而且，越来越多地得到一个正确答案，并不等同于做出一系列明智的决策；你需要智能体式行动、长链推理，以及所有这些能力。And the idea of a test that can fight back. Today you have a task on SW bench, the agent or the model is asked to go ahead and perform one task and give an output. But there is no continuous thinking or evaluation, right? And getting one right answer more and more is not the same as making a good sequence of decisions and you need agentic action and long-chain reasoning and all of these things.

长期以来，棋类游戏，比如国际象棋和围棋，一直是衡量人类自主行动能力和长链推理能力的最佳方式。即便你进一步去评估一个模型，AlphaGo 也是 AI 发展史上非常重要的一步，但它并没有专门针对围棋训练模型。如今，我们不会用围棋或国际象棋来测试通用大语言模型，而这正是我想通过一个名为 E-Slams（即 electronic slams）的开源代码库解决的问题：让 AI 玩游戏，对吧？For the longest time the best measure of agentic action and long-chain reasoning in human beings have been board games like chess and go. And even when you go ahead to like like evaluation of like a model, AlphaGo is a very important step to AI development, but it never very specifically trained model for playing Go. Today's general-purpose LLMs we don't test on Go or chess, and that is what I'm trying to solve with an open-source repo called E-Slams, which is electronic slams, and we make AI play games, right?

所以，你只要安装这个包就行了。它是一个 Python PyPI 包，你可以让你的智能体和模型玩游戏，看看它是怎么运作的。那要怎么测试它呢，对吧？你有……你正在用一个改进后的模型替换一个智能体。你可能会想让旧智能体和新智能体来一盘国际象棋。So, you can just install the package. It's a Python PyPi package, and you can make your agent and your model play games and see how it works. And how can you put it to test, right? You have You're replacing an agent with an improved model. You might want your old agent to play with the new agent on a chess game of chess.

比如说，我不是说那场比赛的赢家就是更好的模型，但它如何进入并进行那场比赛，会让你更了解该如何推进这个模型。因为下棋时，你需要不断调整策略、摆脱失误，而你走出的棋也可能成为日后的负担。所以，你的智能体如何进行推理，是你可以采用的整体评估之外的另一种评估范式。这里的想法是，进行编程评估、写作评估、人文学科期末考试评估，并尽可能采用多种评估范式——这就是我想通过 E-Slams 构建的东西，对吧？And let's say I don't I'm not saying the winner of that match is a better model, but how they enter that match and play that match will tell you a lot more about how to sort of go ahead with that model. Because playing chess, you are changing your strategy, you have to get out of blunders, and move that you make can become liabilities for you in the past. So, how is your agent reasoning through that is an additional paradigm to the holistic evaluation you can have. And the idea here is do a coding evaluation, do a writing evaluation, do humanities last exam, and try to have as many paradigms as possible, and that's what I'm trying to build with E-Slams, right?

所以，我们支持大约 50 款游戏，以及 90 家不同的游戏供应商。我们有国际象棋和围棋，这些游戏考验对不可见状态的规划能力。我们有扑克，这是一种基于隐藏信息做决策的游戏。还有基于共同目标和协作的合作类游戏。我们也有街机游戏的试验，对吧？So, we support about 50 games and 90 different like providers. We have chess and Go, which looks at planning invisible state. We have poker, which is decision-based on hidden information. And we have cooperation games based on shared goals and cooperation. And we also have trials for arcade games, right?

所有这些都能让你深入了解模型及其思考方式。在我们的平台上，你可以在 eslams.com 上预览如何使用这个代码库。这是 Fable 5.1 与 GPT-6 Astra 玩井字棋的一个例子，对吧？你可以看看 Fable 和 Astra 如何观察对方的每一步，以及它们如何继续下棋。And all of this tells you a lot about models and how they think. And on our platform, we have preview on eslams.com of how you can use the repo. And this is an example of Fable 5.1 playing tic-tac-toe against GPT-6 Astra, right? And you can look at how Fable and Astra are looking at each other's turn, and how they go ahead and play this.

有趣的是，这场对决最终打成了平手。大多数情况下，总有一个模型胜出，但 Fable 和 Astra 最后势均力敌，对吧？假设你要在生产环境中部署一个智能体。And this match, interestingly, ends in a tie. And most cases, a model wins, but Fable and Astra ended up knocking their heads to each other, right? And let's say you are deploying an agent in production.

你可以——你想的话，就直接和这些 AI 模型对战，看看它们表现如何。你可以访问 Islam.com，和这些 AI 模型玩任何你想玩的不同游戏。你还可以用开源代码仓库完成这一切，让它和你自己的智能体一起运行，并评估你的系统，对吧？我们将在二月举办 AI 大满贯赛，设想是为 AI 模型举办一场奥运会，让它们展开较量，看看谁能脱颖而出，对吧？You can You want to go ahead and play against these AI models and see how they are. You can come on Islam.com and like play against these AI models on any different game that you want to. And you can all do all of this using the open source repo and make it play with your own agents and like evaluate your systems, right? And we are going to host the Grand Slams of AI on in February and the idea is to put an Olympics of for AI models to go ahead and compete and know who will stand out of all of this, right?

所以，请务必了解一下伊斯兰教，看看我们打算如何推进它；这可以成为你纳入技术栈的一种模型评估范式。So, please do check out Islam to know where we're going with it and this could be one paradigm of model evaluation you can add into your stack.

4. 继续测试过去有效的内容4. Keep testing what used to work

那么，回过头来说，经历了这一切之后，最关键的部分是什么呢？也就是说，你要不断迭代，不断加入新的范式，同时确保持续测试评估流程。最重要的是让流程中行之有效的部分得以保留。我最常举的例子就是 Google Assistant 转向 Gemini。So, now coming back, the main key part after all of this, right? Like you keep evolving, you keep adding paradigms, is to make sure that you keep testing the evaluation workflow, right? And the most important part of that is keeping what works in the flow. And my best example of this is Google Assistant moving to Gemini.

我以前所有提醒事项都用 Google Assistant 来设置。有一天，它迁移到了 Gemini，之后就不再提醒我了，还说它是一个无法设置提醒事项的语言模型，对吧？所以，在迁移到 Gemini 之前，他们原来的 Google Assistant 模型用起来非常出色；迁移之后，它就不好使了。这就是 Google 在推出 Gemini 产品时，没有对过去能正常使用的功能进行评估的一个例子，对吧？I used to use Google Assistant for all of my reminders. And one day it moved to Gemini and it stopped giving me reminders and it said it's a language model that cannot set up reminders, right? So, it was excellent in the old Google Assistant model they had and before they moved it to Gemini and it stopped working and this is an example of Google not doing eval on what used to work, right? When they went ahead and launched the Gemini product.

而且非常重要的是，要确保你继续测试回归用例，对吧？And that's extremely important to go ahead and make sure that you keep testing like the regression cases, right?

以及已知任务和过去的失败，因为一个比旧模型好得多的模型，很可能无法完成旧模型非常擅长的某件事，比如设置提醒，对吧？而在各个阶段加入新的挑战案例，又回到了我刚才所说的设定最具雄心的目标。关键是，你必须不断推动模型进步。如果你觉得你的模型在评测中已经能拿到85分或90分，那就该改进评测、推动能力边界了，对吧？And known task and past failures because a model that is extremely better than the old one could very well be incapable of something the old one did very well, for example, setting a reminder, right? And fresh challenge cases at all points goes back to what I was saying in terms of setting the most ambitious target. The idea is that you have to keep pushing your model. If you feel that your model your is getting to an 85 or a 90 on your eval test, it's time that you better your eval and like push the frontier, right?

并确保你的模型受到挑战，也有无法达到预期表现的情况。要贯穿整个过程，对吧？And make sure that your model is challenged and has cases where it is not able to perform to expectations. And at all points, right?

5. 将真实故障转化为测试5. Turn real failures into tests

我认为，纵观软件发展史，情况一直如此：你必须把现实世界中的故障转化为未来的测试，对吧？你的客户和用户才是评估你的智能体和模型最重要的人。因此，每次故障都应该有一套完善的工作流程，让它进入你的系统，并确保这类问题得到处理，对吧？首先要持续观察，而确保能够进行观察的方法有很多。And I think this has been true in software throughout history is that you have to turn real-world failures into future tests, right? Your customer and your user is the most important evaluator of your agent and your model. So, every failure should have a proper workflow for it to enter into your system and make sure that like be that, right? So, one is to keep observing and there are multiple ways of making sure that you can observe.

有多种可观测性工具可以自动为你提供数据，让用户通过简单地点赞或点踩向你反馈，并尽可能多地收集客户指出流水线故障所在位置的数据，对吧？这就像传统的软件质量保证：复现问题，并尝试弄清楚模型究竟在哪个环节出错。There are multiple observability tools that give you automatic data, give option for your users to report it to you with your simple thumbs up or a thumbs down, and collect as much data where your customers are telling you where there is a failure in the pipeline, right? Classic QA on software, go ahead and reproduce that issue, and try to understand where is the model failing?

这是一次工具调用吗？是吗？是吗？你的评估套件中，哪一部分在继续执行并完成这件事时出了问题，对吧？然后审查它，把它加入流水线，再把它加入评估流水线。持续保持这个反馈循环极其重要，这样你才能不断探索新的前沿，发现自己在每个时刻错过的东西。Is it a tool call? Is it Is it And what part of your eval suit failed in going ahead and doing that, right? And review it, add it to the pipeline, and add that to the eval pipeline. This becomes extremely important to keep this feedback loop going so that you go ahead and access new frontiers of things that you are missing out in every point in time.

我认为，要尽可能全面地开展评估，有两种做法：一是持续对流程进行人工审核，二是确保将客户反馈纳入评估流程，对吧？因为真正显现出来的往往是最后那 5% 的边缘案例，模型或现有的评估框架可能都无法捕捉到这些情况。And I think the two ways to go ahead and get to eval that is as holistic as possible is one to consistently human review a process, and then ensure that your customer feedback is part of your eval pipeline, right? Because these are the last 5% edge cases that really come out which a model or an existing eval framework might not catch.

回顾Recap

总之，测试的是工作本身，而不仅仅是答案，对吧？这对整个流程至关重要。最后，我想总结一下我们讨论过的内容：第一，始终明确你最宏大的成功目标。想想 AGI 时代的生活会是什么样，对吧？So, to conclude, test the job and not just the answer, right? And that's extremely key to the whole process. And I'll just wrap up with all the things that we've discussed is one is always define your most ambitious success. Think about what life would be in AGI, right?

再想想，在一个没有任何能力缺陷、能够完全像你在这个过程中试图模仿并尝试定义的那种人一样行事的模型中，生活会是什么样，对吧？第二，要在工作流的每个环节持续测试。不要想当然地看待工作流中的事情，也不要假设某个输入和输出就是任何事情的结果，对吧？测试工作流时，我觉得我刚才稍微漏掉了一点：要确保你的测试是针对工作流中该环节所执行的任务而设计的。And think about life in a model that has no inabilities and can absolutely perform like the human that you're trying to sort of emulate and try to define in this process, right? Second is continuously test the workflow at every part of the workflow. Don't assume things in the workflow or like assume for an input and output to be the outcome of anything, right? And when you test the workflow, I think something I slightly missed on here is to make sure that your tests are designed for what that element of the workflow does.

例如，假设你的工作流中有一个特定环节会持续调用工具来获取信息，那么你的评估就必须针对工具调用来设计，而不是单独针对其他流程设计，对吧？因为该工作流的目标是了解你的工具调用能力和调用准确性，对吧？所以，每次测试工作流时，都要确保测试是针对工作流中的那个特定环节设计的，并衡量该环节的成功情况。For example, let's say in that in your workflow there's a specific part that is doing tool calls consistently to get the information, your eval has to be designed for tool calling and not for any different process separately, right? Because the objective in that workflow process is to know what is your tool call ability and the tool call accuracy, right? At that point. So, every time make sure that when you're testing the workflow, you design the test to that specific part of the workflow and measure what is the success in that element of the workflow as well.

并且要持续、全面地收集证据。在智能体化的世界里，智能体行动评估所依据的证据也在不断演变。游戏是我正在探索的一个方向，想看看效果如何；不过还有很多不同的前沿领域值得你去看看，对吧？And continuously gather evidence holistically. In an agentic world, what evidence for agentic action eval is something that keeps evolving. and games is something I'm exploring to see how it works out, but there are so many different frontiers for you to go ahead and look at, right?

我在这里时发现一件特别有意思的事：并行搜索模型索引，对吧？模型很擅长从各种并行渠道（比如 AI 搜索索引）获取搜索结果。而当你设计一个需要实时数据的模型时，评估这一点就变得非常重要，对吧？所以，无论你是公司还是个人，只要在构建这类产品，就要确保持续寻找更多证据、更多应用场景和更多评估范式，来评估你的模型。这是确保覆盖大多数边缘情况的一个可靠方法。Something super interesting that I came across while here I'm here is the parallel search model index, right? And how models are good at fetching search from parallels like AI search index. And that becomes very important for you to evaluate when you are designing a model that needs real-time data, right? So, make sure that you as a company or a person building these products is consistently looking for more evidence and more arenas and more paradigms for you to evaluate your models and that is one sure shot way to making sure that you cover most edge cases.

第四点是随着时间推移不断改进测试，并确保测试涵盖过去和未来的所有方面。最简单的做法是：如果测试得分很高，比如 85 分或 90 分，那就该调整测试，并把难度提高。The fourth one is to evolve the test as time flow and making sure that you cover all aspects from the past and the future into the test and the simplest thing is if your test is scoring high is it 85 90 that is the time you change your test and you have to make it harder.

对吧？最简单的想法是，如果你觉得所有人的 SAT 成绩都处于第 99 百分位，那就说明测试本身是时候在互联网环境中做出改变了，对吧？而且要始终向现实学习。现实会是你最好的老师。我一开始谈到评估这个概念，是为了确保我们在实际生产环境中兑现对消费者和用户作出的承诺，而这最能体现在客户反馈和真实发生的情况中，对吧？Right? The simplest thing is if you think everyone is scoring like in the 99th percentile for the SATs, it is time that the test have to change in the internet of itself, right? And always learn from reality. Reality is going to be your best teacher and the idea of eval that I started off with is to make sure that we deliver the promise that we are making to the consumer and the user in production and that is most defined in the customer feedback and the reality that really happens, right?

因此，在开始推进时，要持续思考如何将真实用户反馈和真实场景测试纳入工作流程，以及如何加入人工审核和人工测试。我认为，具备这五种思维方式，并牢记要测试的是任务本身，而不只是答案，你就能建立起一套良好的思维框架，用于为高风险应用构建评估体系。So, consistently think about how you can incorporate real user feedback and real world test and how you can have human review and human testing into the workflow as you go ahead and get started. I think with these five thought processes and the idea that you go ahead and test the job and not just the answer, you will have a good mental framework to go ahead and build eval for high stake applications.

感谢大家前来。之后我很乐意和大家交流并回答问题。Thank you for coming. I'm happy to catch up for questions afterwards.