为什么构建评估平台比看起来难得多 — Braintrust

简要说明Brief Description

为什么构建评测平台比看起来更难 — BraintrustWhy Building an Eval Platform Is Harder Than It Looks — Braintrust

目录Table of Contents

• 简介
  Intro
• Braintrust 的作用
  What Braintrust does
• 评测与可观测性
  Evals and observability
• 为什么评估至关重要
  Why evals matter
• 它不就是个电子表格吗？
  Isn't it just a spreadsheet?
• 冰山之下有什么
  What's under the iceberg
• 改进循环
  The improvement loop
• 阶段 1：电子表格
  Stage 1: spreadsheets
• 第 2 阶段：自定义 UI
  Stage 2: a custom UI
• 阶段 3：人人都能进行实验
  Stage 3: experiments for everyone
• 阶段 4：连接生产环境
  Stage 4: connecting production
• Agent 轨迹庞大而杂乱。
  Agent traces are huge and messy
• 使用 BTQL 查询
  Querying with BTQL
• 运行评测的智能体
  Agents running evals
• 发现未知的未知因素
  Finding the unknown unknowns
• 智能体负责迭代，人类负责决策
  Agents iterate, humans decide

简介Intro

主持人：感谢大家参加这场关于为何构建智能体质量平台如此困难的分享。我叫 Hussein，负责 Braintrust 西部地区的解决方案工程团队。嗯，我在 Salesforce 和 Databricks 做了 15 年解决方案相关工作。我一直都是个技术爱好者，从 5 岁起就开始组装电脑。所以我对技术有着近乎永不满足的热情，这也让我在 Braintrust 踏上了 AI 领域的前沿。对于在座各位，我知道有些人刚才听了 Jess 的分享。我看到有人举手提到了评估。大家有多少人知道 Braintrust 是什么，以及我们公司是做什么的？请举手示意。Host: Thank you for joining this session on why building agent quality platforms is hard. My name's Hussein. I lead the solution engineering organization on the West for Braintrust. Um, I spent 15 years in solutions between Salesforce and Databricks. I've been a lifelong technologist. I've built computers since I was 5 years old. So, I have this like insatiable appetite for technology, which finds me kind of on the bleeding edge of AI here at Braintrust. For those of you, I know some of you were in the room earlier when Jess did her session. I saw some hands about evals. How many of you know what Braintrust is and what we do as a company? Just show of hands.

Braintrust 的作用What Braintrust does

很好。我看到了一两个。所以，概括地说，Braintrust 这个平台专注于智能体质量。其核心理念是帮助团队对他们正在交付的 AI 功能或 AI 智能体建立并保持信心。Perfect. I see one or two. So, at a glance, Braintrust as a platform is focused around agent quality. And the core idea is helping teams build and maintain confidence with the AI features or the AI agents that they're shipping.

评估与可观测性Evals and observability

代理质量有两个关键支柱。第一个是评估，我们之前已经谈过。评估是在代理进入生产环境之前进行的。团队会在这个阶段进行实验、测试代理的行为，并建立信心，确保达到预期后再将其部署到生产环境。第二个支柱是可观测性。可观测性指的是代理进入生产环境并开始与真实用户交互之后的情况。代理会产生交互记录。我们的目标是，将你在离线开发过程中形成的假设，通过持续监控在生产环境中加以验证和强化。评估和可观测性密切相关，都是为了解决同一个问题：一个发生在生产环境中，另一个发生在开发阶段，而二者都旨在帮助我们理解并提升代理质量。And there are two key pillars of agent quality. The first is evals, which we talked about earlier. Evals is what you do before your agent reaches production. This is when teams experiment, they test the behavior, and they build the confidence that they want to have before they ship to production. The second pillar is observability. Observability is once your agent is in production and is interfacing with real-life users. It's creating interactions. And the goal is to take that hypothesis that you built in your offline development process and then reinforce that in your production scenario through continuous monitoring. Evals and observability are closely related to the same problem. One happens in production, the other one happens in development, and both is the understanding to improve agent quality.

现在，我就不在这里多花时间了。让我把它关掉。我们来谈谈我们为什么会在这里，Now, I won't spend much more time here. Let me close that. Let's talk about why we're here,

为什么评估很重要Why evals matter

这就是评估为何重要。你知道，评估很重要，因为大型语言模型（LLM）本质上具有非确定性，表现变化很大。正是这种多变性让它们如此强大。它们能够跨多个不同领域进行推理，解决不同类型的问题，并满足各种各样的用户需求。但这种灵活性也会带来风险。智能体以 LLM 为大脑，而智能体式体验正逐渐成为用户与公司互动的主要方式。因此，团队需要确信自己的智能体能够可靠地运行，并产生预期结果。没有评估，公司就会面临切实的风险。如果行为不一致，品牌就会面临风险；如果智能体说错话或做错事，就会产生合规风险；如果系统难以调试、难以控制或不够可靠，还会带来成本和维护方面的风险。which is why are evals important? You know, evals matter because LLMs by nature are non-deterministic. They are highly variable. That variability is exactly what makes them so powerful. They can reason across many different domains. They can solve different kinds of problems, and they can handle a wide range of user needs. But that same flexibility also creates risk. Agents use LLM as their brain. And agentic experiences are starting to become the primary way in which users will interact with companies. So teams need confidence in their agents that they will behave reliably and produce the outcomes that they expect. And without evals, companies face real risks. Brand Brands become a risk if there's inconsistent behaviors. Compliance if the agent says or does the wrong thing. And there's a cost and maintenance risk if the systems are hard or unreliable to debug or control.

评估的目标是在发布前降低这种不确定性，从而确保客户获得良好的体验，并让智能体按照你的预期运行。The goal of evals is to reduce that uncertainty before launch so that customers have a good experience and the agents behave the way that you expect them to.

它不就是个电子表格吗？Isn't it just a spreadsheet?

所以你可能会想：“好吧，Evals 不就是电子表格上的一个 UI，对吧？”我想不如问问在场的各位。请举手示意：有多少人跑过 Evals，并把结果记在电子表格里？有人这样做过吗？好，我看到一两位。顺便说一句，这完全不丢人。事实上，这是很好的第一步。重要的是，团队要承认这确实是个问题，并且需要一种方法来了解智能体在不同输入下的行为。最简单来说，评估系统需要具备三类要素。第一，能够使用测试输入来运行智能体；第二，能够查看输出或评分，即使这些结果是记在电子表格里；第三，一组能够调用智能体的测试输入或示例。So you might be thinking, "Okay, well, Evals is just a UI on a spreadsheet, right?" And I guess I should ask the audience. Just show of hands, how many of you have run Evals and logged the results into a spreadsheet? Has that happened? Okay, I see one or two. So, there's no shame in that, by the way. It's actually the a good first step. The important thing is that teams acknowledge that this is a real problem and they need a way to understand how agents behave based on different inputs. At the simplest level, an eval system tests against or it has three different kind of criteria. It has a way to execute agents against test inputs. Second, a way to view the outputs or the scores, even if those are being logged in a spreadsheet. And third, a set of test inputs or examples that can invoke the agent.

输入示例是启动智能体运行所需的所有信息。它可以是提示词、请求，或促使智能体采取行动的上下文。如今，基于电子表格的评估并没有错。不过，它们往往是构建实用评估工作流的第一步。An input example is whatever information is needed to start the agent run. It could be a prompt, a request, or context that causes the agent to act. Now, spreadsheet-based Evals are not wrong. However, they're often the first step for a useful version of eval workflow.

冰山之下是什么What's under the iceberg

不过，当然，正如你们所想，冰山远不止我们刚才谈到的那些。如果评测只是运行智能体、查看输出，然后在电子表格里打分，那这场演讲现在就结束了。但在冰山之下，还有很多幕后工作。许多支持团队最终需要构建更好的数据集、评分系统、审核流程和调试工具，还要设法将预生产测试与生产环境中的行为联系起来。今天我们会谈到其中一些内容；没来得及讲的部分，欢迎会后来找我，我们可以聊聊。事情也正是从这里开始变得复杂起来。底层技术很复杂。LLM 并非简单的确定性系统。智能体的质量是一个涉及多种角色的问题。But, of course, as you can imagine, there's more to the iceberg than just what we just talked about. And if Evals were only, you know, run the agent, view the output, and then score it in a spreadsheet, this talk would be over right now. But, there's a lot more that happens behind the scenes or underneath the iceberg. There are many supporting teams that eventually need to build better data sets, scoring systems, review process, debugging tools, and the way to connect pre-production testing with production behavior. We'll touch on some of these today, and anything I don't get to, you feel free to come up afterwards and we can chat about. And this is where things start to get complicated. The underlying technology is complex. LLMs are not simply deterministic. Agent quality is a multi-persona problem.

这不仅仅是软件工程师和 AI 工程师的事。以前负责编写 PRD 的产品经理，现在也在运行评测。还有一些 SMEs，他们对你正在构建的产品拥有领域专长。而评测本身只是开发和运营工作流中的一环。那种只构建一次、编写单元测试和回归测试、发布到生产环境，然后就不再过问的日子已经一去不复返了。评测能让你持续朝着一个目标不断改进，那就是智能体的质量。接下来，我们来聊聊我们看到的评测平台所经历的不同阶段。It's not just software engineers and AI engineers. There are PMs who used to build PRDs that are now running evals. There are SMEs who have the domain expertise of the product that you're building. And evals themselves are just one part of the development and operating workflow. And gone are the days where you build once with unit tests and regression tests, you ship to production, and you don't think about it. Evals are a way for you to continue hill climbing against one target, which is agent quality. So, let's talk about the different stages of the the eval platforms that we see.

改进循环The improvement loop

在此之前，这也是许多团队的北极星目标。他们希望建立一个持续改进的闭环：我有一个已投入生产的功能或应用，可以根据我定义的维度观察发生的失效模式。我希望能够收集这些失效模式并创建测试用例，以便离线迭代，从而提升智能体的质量，同时避免引入新的回归问题。然后随着时间推移继续迭代，因为就像传统机器学习中一样，漂移确实会成为一个问题。用户与智能体交互的方式会随着时间而改变。构建智能体的方式也会改变。And before we do that, this is the North Star for many teams. They want to build the improvement loop, which is I have a feature or application that's in production. I can observe the failure modes that happen based on dimensions that I define. I want to be able to grab those failure modes and create test cases so that I can iterate offline, so I can improve my agent quality without introducing new regressions. And then you continue iterating over time, because just like in classical ML, drift becomes a real thing. The way users interact with your agents will change over time. The way you build your agents

阶段 1：电子表格Stage 1: spreadsheets

会随着时间而变化。那么，第一阶段是什么样的？有人举手提到了电子表格，这没什么可羞耻的。很多人都是从这里开始的。基本设置相当简单：一个 for 循环、一组输入示例，以及一种运行智能体的方式。有了这些，你就可以运行同一组示例，并根据需要进行调整，观察智能体的输出如何随时间变化。最大的优势是易于上手、节省时间。基本上，你可以零门槛地开始使用。但这种做法的收益很快就会递减。在这个阶段，你基本上只是在做记录，无法真正进行实验，更不用说随时间开展任何类型的分析了。常见的局限包括：分析很困难，需要手动完成；人工评分很有价值，但实施起来不容易；协作能力很弱；领域专家通常也被排除在这类成果之外。will change over time. So, what What phase one look like? There were some hands that were raised around spreadsheets and it is not There's nothing to be shameful about. This is where many people start. And as the basic setup is pretty simple. It's a four loop with a set of input examples and a way to execute the agent. And with that, you can run the same examples that you tweak and see how the agent responds with the outputs over time. The big biggest advantage is accessibility and time. You can start this with basically zero barrier to entry. But the the returns diminish pretty quickly. At this stage, you're pretty much only documenting. You can't really do true experimentation. And let alone do any kind of type of analysis over time. Common limitations are analytics are hard, they're manual, human scoring is valuable but difficult to do, collaboration is very weak, and domain experts are usually boxed out of these type of results.

所以，还是那句话，电子表格是一个很好的起点，但很难进一步演进。然后有人——通常是产品工程师——看到了这个问题，他们……So, again, spreadsheets are a great starting place but difficult to evolve. Then someone, usually a product engineer, you know, they see this problem and they

阶段 2：自定义 UIStage 2: a custom UI

想一想：“好吧，我可以构建一个不错的 UI，一个定制的 UI，来帮助解决这个问题。”乍看之下，这可能挺不错，但它能帮你引入不同的用户角色，而这些人原本是你无法接触到的。这让 PM 们现在也能参与到评估中来。但你可能已经想到的局限依然存在：文档和评审流程虽然看起来更美观了，协作仍然很困难。迭代周期虽然缩短了，但用户仍然不容易持久化这些数据，也难以基于这些数据开展长期分析。它终究还只是一个报告工具。think, "Okay, I can build a nice UI, a bespoke UI to help serve to solve this problem." And while it might be nice at a glance, um it helps you bring in the different personas that you would otherwise not have access to. This allows PMs to now kind of get into the loop of doing evals. But the limitation that you might have thought about, it continues. That the documentation, the review process, though it might visually look nicer, still collaboration becomes difficult. And though iteration cycles become quicker, it is still not easy for users to persist that data and do long horizon analytics on. Still just a reporting tool.

阶段 3：人人都能进行实验Stage 3: experiments for everyone

接下来的演进方向是：我希望非技术用户也能拥有一个操作界面，不仅可以记录结果，还能进行实验。我希望能够调整系统提示词、更改底层模型，并反复尝试不同的参数值。因此，能够进行这种并排对比，便成为我们所看到的评估周期的下一步演进。Then the next evolution becomes hey, I want my non-technical users to have an interface where not that they just log the results, but they should be able to experiment. I want to be able to tweak a system prompt. I want to change the underlying model. I want to iterate on some parameter values. So, being able to do this type of side-by-side comparison becomes the next evolution of the eval cycle that we see.

第 4 阶段：连接生产环境Stage 4: connecting production

不过，仍然存在一个主要的缺口。那就是，用来进行这类比较的测试用例，仍然是从离线评估中挑选出来的一组精选用例。但在生产环境中会发生什么呢？我已经在这里进行了多轮迭代，对生产环境中可能发生的情况有了不错的预判，但一旦将它部署到生产环境，我就只能盲目运行了。我完全不知道我的智能体采取了哪些步骤，也不知道这些故障究竟何时在我的线上环境中发生。回到飞轮这个概念，想想什么样的系统最能支持这一过程：你应该能够从顶层开始观察故障模式，记录智能体执行过程中的每一个输入、输出和每一步轨迹，分析并弄清楚出了什么问题。你可以据此构建失败和成功的维度，因为你知道最终结果。But, there's still one primary gap. And that is that the test cases in which you use to do these kind of comparisons are still a curated set that arise in your offline evals. But, what happens in production? I've done my iterations here. I have a good idea of what I can expect to happen in prod, but once I ship it to production, I'm operating in the blind. So, I have no idea what steps my agent is taking and when those failures are actually happening in my live environment. And when we think kind of going back to the flywheel, when we think about what are the best systems that enable this, well, you should be able to starting at the top observe the failure modes, log every input, output, every step of your trace execution that your agent takes, analyze, understand what went wrong. You create the dimensions of failure and success because you know the outcomes.

然后收集这些故障模式，也就是用户体验不佳的各种场景，并在离线流程中针对它们创建评估，不断迭代，从而在实现改进的同时避免引入新的回归问题。接着将其发布到生产环境，并以此为目标持续爬坡优化。想想这一切最终会带来什么：团队开始构建飞轮，整个开发周期也由此形成。Then grabbing those failure modes, those different scenarios where your users had poor interactions, and creating evals in them in your offline process, iterating on them so you can create the improvements without developing new regressions. And then shipping to production and hill climbing against this. And when you think about the final outcome of this, well, you get teams that are building the flywheel. You get that entire development cycle

Agent 跟踪记录既庞大又杂乱。Agent traces are huge and messy

在你的平台内部，但接下来的问题是，你必须维护它。你拥有自己构建的这个产品。尤其是在规模较小的场景，或者 POC 和演示环境中，这样做很容易。但智能体追踪数据很棘手。它们是半结构化的 JSON。有些情况下，我们看到团队每次交互记录的数据量相当于一部电影，达到数百兆字节。因此，查询这类数据可能很困难。常见的云数据仓库在这种数据量和负载下往往会不堪重负。这也是我们遇到的同一个问题。我不会花太多时间介绍 Brain Trust，但当生产环境中的追踪数据不断涌入时，你会希望数据一落地就能实时查询。因为如果用户的体验不佳，你就要能够实时了解发生了什么、他们为什么会有这种糟糕的体验，以及如何尽快解决问题。inside of your platform, but then the problem becomes you have to have you have to maintain it. You own this this product that you've built. And especially this is easy to do when it comes to smaller scale or POC and demo environments. But agent traces are nasty. They are semi-structured JSON. In some cases we see teams that are logging a movie's worth of data in each interaction, hundreds of megabytes. So being able to query that type of data can be difficult. And your typical cloud data warehouses kind of collapse under this type of volume and load. And this is the same problem that we ran into. I won't spend much time talking about Brain Trust, but when you have production traces coming in, you want to be able to query that data in real time as it lands in there. Because if your users are having poor experiences, you want to be able to know in real time what is happening, why are they having these poor experiences, and how can I remedy that as soon as possible.

你还希望能够执行长时间运行的查询。如果你考虑微调代理体验，或者让人类通过标注评判器的结果来进行对齐，这就是两种不同的工作负载。You also want to be able to do long-running queries. If you think about being able to fine-tune your your agent experience or even have humans who are doing alignment by annotating your your judges' outcomes. Two different workloads.

使用 BTQL 查询Querying with BTQL

因此，我们构建了一个名为 BTQL 的抽象层，这又带来了一层复杂性，因为你需要提供一个接口，让人们能够查询这些数据。所以，问题不一定只是 UI 或 UX 问题；真正构建一个评估平台，会变成一个系统层面的问题。几年前 ChatGPT 热潮兴起时，还出现了一系列新问题：我需要进行实时数据摄取。我处理的负载非常大，有时达到数十 MB，甚至数百 MB；而传统的心跳监控数据只有几 KB。数据的结构和形态都不一样。深度嵌套的半结构化全文数据很难查询，读取模式也有所不同。你既希望能够对海量数据进行聚合，也希望能够对某个时间点的数据进行快照。So we built an abstraction layer called BTQL, which is a another form of complexity because you want an interface for people to be able to query that data. So, it's not necessarily that it's just a UI or UX problem, but building an eval platform truly becomes a systems problem. And there are novel set of issues when the chat GPT boom happened a few years ago where I need to do real-time ingest. I have huge payloads that are tens of megabytes, hundreds of megabytes in size sometimes, compared to traditional heartbeat observability, which are just kilobytes in size. The structure and the shape of the data is different. Having deeply nested semi-structured full text is difficult to query, and the read patterns are different as well. You want to be able to aggregate in large volumes, but also be able to do snapshots of of of in-time data.

而构建合适的系统，不仅能让 AI 工程师受益，也能让产品经理和领域专家受益。最近，And building the right system should allow you to not only empower the AI engineer, but also the PMs, as well as the SMEs. And more recently,

正在运行评测的智能体Agents running evals

我们已经看到，智能体已成为这些评估平台中的一等公民。在无头体验中，你可以使用自然语言告诉自己选择的编码智能体：“帮我找出过去 24 小时内用户体验不佳的所有追踪记录，并替我运行评估。”由于这些编码智能体能够访问你的底层基础设施和代码库，它们便成为运行评估并记录结果的执行机制。接下来就会有人问：所以呢？这为什么重要？嗯，到目前为止我向你展示的所有事情，都需要人类——工程师、产品经理——深入参与这个过程，有时可能会很费力。在 BrainTrust，我们的想法是帮助你实现规模化运营。we've seen that agents have become a first-class citizen of these eval platforms where you want to be able to use natural language in a headless experience where you can tell your coding agent of choice, "Find me all traces in the last 24 hours where the user had a poor experience. Run the evals for me." And because these coding agents have access to all of your underlying infrastructure and your code base, they become the mechanism to run the evals and log those results. And then it becomes a question of like, so so what? Why is this a Why is this important? Well, the goal is everything that I've shown you so far, it requires the humans, the the engineer, the PM to be deeply involved in this process and it could be laborious at times. I BrainTrust, the way we think about this is we want to help you operate at scale.

与其让你不得不思考这些维度，以及 AI 代理成功与失败的定义，BrainTrust……Rather than you having to think about these dimensions and of success and failure for your AI agent, BrainTrust

发现那些未知的未知因素Finding the unknown unknowns

可以自动呈现这些洞察。因为我们会将你的追踪数据记录到平台中，所以可以基于这些数据进行推理，并告诉你那些你尚未意识到的问题。比如，哪些情况下我的智能体会悄无声息地失效？又有哪些情况下，用户会因为一遍又一遍地向智能体提出同一个请求而感到沮丧？还有很多我们没谈到的事情，比如支撑它运行的底层数据库，或者必须将后端构建到系统中，以管理控制项、权限或数据屏蔽。你知道，这些功能中的每一项都表明，这不只是一个 UI 和一个电子表格，而是一个支撑其运行的系统问题。can surface these insights automatically. Because we log your tracing data into our platform, we can run inference on them and we can tell you the unknown unknowns. Hey, what are what are scenarios and when my agent is failing silently or what are scenarios that my users are experiencing frustration by prompting my agent for the same request over and over again. And it's a lot of things that we didn't talk about. Things like the underlying database that powers it or having to build our back into the system to manage controls and permissions or data masking. You know, each one of these features proves that this is not just a UI and a spreadsheet, but rather it is a a systems problem that powers it.

智能体负责迭代，人类负责决策。Agents iterate, humans decide

当我们思考改进循环的演变时，我之前提到过，以前是由人来执行改进循环；而现在，我们看到编程智能体可以反复进行修改，并提出应该如何改进应用程序。归根结底，人的职责是审核结果。如果我的编程智能体运行了这些评估的不同迭代版本，我就可以查看结果，并决定哪个版本最适合我发布到生产环境。好了，我的演讲到此结束。感谢大家前来了解构建评估系统为何如此困难。谢谢。And as we think about the evolution of of the improvement loop, I said this earlier where humans were doing the improvement loop and now we see coding agents where they could iteratively make changes and suggest what kind of improvements you should be making to your application. And ultimately, the humans responsibility is to review the outcome. If I have different iterations of these e-vals that are run by my coding agent, I can look at the outcome and make the decision of which version of it is the best for what I want to ship to production. So, with that, that is the end of my talk. I appreciate you all coming out and learning about why it's hard to build eval systems. Thank you.