为什么 80% 的可靠性仍不足以赢得信任｜Amazon AGI Lab Filipe Blanes

简要说明Brief Description

为什么 80% 的可靠性还不够——Felipe Blanes，亚马逊 AGI 实验室Why 80% Reliability Isn't Good Enough — Felipe Blanes， Amazon AGI Lab

目录Table of Contents

• 简介
  Intro
• Nova Act 是什么
  What Nova Act is
• 直接与客户合作
  Working directly with customers
• 基准测试的假象
  The benchmark illusion
• 静态评测与希望
  Static evals and hope
• 完成客户反馈闭环
  Close the customer loop
• 评估飞轮
  The eval flywheel
• 第 1 步：定义成功
  Step 1: define success
• 步骤 2：捕获信号
  Step 2: capture signals
• 与客户沟通
  Talk to your customers
• 步骤 3：诊断差距
  Step 3: diagnose gaps
• 步骤 4：输入决策
  Step 4: feed decisions
• 客户旅程的四个阶段
  Four stages of the customer journey
• 第 1 课：信任悬崖
  Lesson 1: the trust cliff
• 课程 2：坦诚说明自身局限
  Lesson 2: be open about limits
• 第 3 课：出人意料的应用场景（Amazon Leo、Hertz、Sola）
  Lesson 3: surprising use cases (Amazon Leo, Hertz, Sola)
• 四项原则
  Four principles
• 评估应当每周都变得更智能。
  Evals should get smarter every week
• 两个要点
  Two takeaways

简介Intro

大家都能听到我说话吗？好的，太好了。首先感谢大家参加。我叫菲利普，是亚马逊 AGI 实验室的一名技术人员。作为工作的一部分，我协助团队打造下一代产品，并负责管理实验室下一代产品的研发项目。今天我将和大家聊聊如何设计能够赢得客户信任的评测。在谈评测之前，我先简单介绍一下实验室过去一年的工作。那……Can everybody hear me? Okay, perfect. So, first thanks for joining everyone. Uh, my name is Filipe and I am a member of technical staff at the Amazon AGI lab. Um, as part of my role, I help us to build the next generation. I manage the programs to build the next generation of products our lab is building. And today I will talk to you about designing evolves that earn customer trust. Before talking about evolves, I just need to give you some context about what our lab was doing the past year. So

Nova Act 是什么What Nova Act is

一切始于去年三月，当时我们推出了 Nova 研究预览版。简单介绍一下，Nova 是我们实验室开发的一款工具，可以帮助你构建浏览器代理。也就是说，你在网页浏览器中执行的所有操作，都可以用 Nova 自动化。作为研究预览版的一部分，我们的主要目标是推出一款产品，尽可能多地收集客户反馈。我们在三月推出了它。随后在七月，我们对产品进行了一项改进：全面打造全新的开发者体验。这是因为在最初几个月里，我们注意到开发者体验中存在一些不足，因此希望确保客户在构建代理时能获得最佳体验。最后，在十二月，我们在 AWS 上正式推出了这项服务。everything started in March last year where we launched our Nova research preview. for context like Nova is a tool that we build as part of our lab and it's a tool that helps you to build browser agents. So basically everything that you do in a web browser you can use Nova to automate those tasks. Uh and as part of the research preview our main goal was we wanted to ship a product to get as much feedback as we could from our customers. Uh in March we launched that. Then in July we made one additional improvement in our product which was basically develop a whole new developer experience and this was because basically during the first couple months we noticed like some gaps in the developer experiment experience and we wanted to make sure customer had the best experience building agents. Then in December we finally launched our service on AWS.

所以如今，Novax 是 AWS 上的顶级服务，任何人都可以用它来构建生产级智能体。在这段时间里，我基本上……So today Novax is a top tier service on AWS that anyone can use to build production level agents and during this period I was basically

直接与客户合作Working directly with customers

负责客户赋能这部分工作。简单介绍一下，什么是客户赋能？基本上就是我每天都和我们的客户一起工作，了解他们遇到了什么问题，以及我们如何帮助他们解决问题。这就是我去年主要负责的工作。接下来我们来谈谈今天要讨论的问题。我觉得……responsible for the customer enablement piece of that. What's customer enablement just for context? It's basically I was working daytoday with all of our customers to understand what was their problem and how we were helping them to solve that. So that was my main uh role uh during last year. And then let's go to uh the problem that we will be discussing today. So I like

基准测试的假象The benchmark illusion

这就叫“基准测试幻觉”。你们都在做 AI，所以很可能正在基于基准测试开展工作、构建东西，对吧？可能是大家都在用、都在努力优化产品表现的公开基准测试；也可能是你们自己创建的基准测试：生成合成数据，然后评估自己的产品。嗯，这里的主要目标是优化，对吧？你想在这些基准测试中尽可能做到最好。但当你把产品部署到生产环境，客户开始使用时，情况就变成了这样：他们开始遇到问题。基本上，他们会尝试所有你没想到他们会做的事，然后把你的产品弄坏。之所以会这样，是因为我们大多数人做的评估都是静态的。to call this is the benchmark illusion. So you all working on AI. You are probably working on and building things based on benchmarks, right? It could be public benchmarks that everybody's using and everybody's trying to optimize the products on that and also benchmarks that you are creating your own synthetic data that you are creating and you are evaluating your product. Uh and the main goal here is you want to optimize right you want to be the best as you can there in those benchmarks but then when you ship that to production and your customers start using that what happen is it's this right they start seeing problems and it's basically everything that you were not expecting them to do they try to do that and they break your product and this is happening because you basically most of us we are doing we are working on evals that that are static.

静态评测与希望Static evals and hope

那么，什么是静态评测？简单来说，我们先来聊聊产品的标准开发流程。首先，你会从产品团队那里收到需求，然后开始构建产品，对吧？希望大家都是这么做的。等产品成形之后，接下来要做的就是再次运行评测：使用公开基准测试、生成的合成数据，并针对这些数据进行优化。当你觉得差不多稳妥了，就会想：“好，现在可以发布了。”于是你把产品交付给客户。但如果你做的是静态评测，发布之后你唯一能做的是什么？基本上就是抱着希望——希望你的评测能准确反映客户会如何使用产品。否则，产品就会开始出问题，对吧？这就是差距，也是我们今天要讨论的内容，以及我们在实验室里为解决这个问题所做的工作。So what are static evals? Basically let's talk a little bit about what's the standard development process of a product right first thing that you do you receive uh the requirements from your product chain and you start building right everybody does that I hope then when you have a product in a good shape the next thing that you will do is running your evals again public benchmark synthetic data that you generated and you optimize for that when you start feeling comfortable you're like okay now I will ship that and you ship that to the customers but if you have a static evals what's the only thing that you can do after that it's basically this is hope right you just hope your evals were reflecting exactly what your customers would do with your product and if not it will start failing right and this is the gap this is exactly what we will be discussing today and what we were doing in our lab to solve this problem.

先透露一下，这不是关于……的事情。And just a spoiler, so this is not about

闭环客户反馈Close the customer loop

更好的基准。这关乎闭合客户反馈环路。因此，考虑到这一点，我们在自己的……中所做的就是……better benchmarks. It's about closing the customer loop. So what we want you to do here is get the feedback from your customer and close the loop. So with that in mind, what we did in our

评估飞轮The eval flywheel

实验室基本上是在提出一个评估飞轮。所以，如果你不了解这个术语，飞轮基本上就是一个你一直重复的循环，而这一演进的第一步是定义成功，并且非常……lab was basically propose an eval flywheel. So if you are not aware about the terminology but flywheel is basically a loop that you keep repeating like all the time and the first step of this evolve is define success and very

步骤 1：定义成功Step 1: define success

这一点很重要。所以，要在这里定义成功：成功不是你认为的成功，而是你的客户认为的成功。好，下一步，一旦你理解了……important here. So define success here it's not what you think it's success it's what your customer think it's success. Okay. So next step as soon as you understand

步骤 2：捕获信号Step 2: capture signals

了解客户真正需要什么，以及你试图解决的问题后，你就会去捕捉信号。作为工程师，我们首先想到的是什么？就是埋点，对吧？获取指标，尽可能多地收集客户行为数据，但还有一件非常重要的……what your customer really needs and the problem you are trying to solve you go to capture signals and we as engineers what's the first thing that comes to your mind it's like instrumentation right let's get metrics let's get data as much as you can about what your customer is doing but one very important

与您的客户沟通Talk to your customers

我们缺少的那一部分是，你应该和客户交流，比如和他们沟通、安排会议、登门拜访，准确了解他们想做什么。你的数据可能有点误导性，你可能没有准确把握客户想要解决的问题。piece of that that we are missing is you should talk to your customer like talk to your customer go schedule meetings uh visit them and understand exactly what they are trying to do. Your data might be a little bit misleading. You might be missing exactly what's the problem that your customer is trying to solve.

步骤 3：诊断差距Step 3: diagnose gaps

第三步是诊断差距。这很简单。基本上，你会从客户那里收集到各种信号，可能来自埋点，也可能来自你们的输入，或是你和客户开会时获得的洞察。然后你只需要开始对这些信息进行分类。这里说的分类，主要是从这些讨论中找出三类信息。第一类是关于模型的反馈，也就是模型在哪些地方表现不佳，这些反馈会用于模型评测。第二类是给工程团队的反馈，也就是你们的工程框架有哪些问题需要修复。这属于比较传统的 bug 修复。第三类则是产品方面的反馈：从产品角度来看，你们还缺少什么？比如，客户是否确切理解你们的产品想要解决什么问题？Third step, you go to diagnose gaps. That's easy. That's basically you get all these signals from customers. Could be instrumentation but also your inputs, your insights from the meetings that you had with them. And you just start categorizing those. And when I say categorizing there are basically very high level but there are three things that you could find from those discussions. First it's basically inputs about your model. So where your model is failing there and this feeds into evals right to your model. The the second point it's inputs to your engineering team. So basically what uh is the gap in your hardness that you should fix. This is more traditional bug fixing. And the third one is also on the product side. So what are you missing in terms of product like do your customer understand exactly what your product is trying to solve?

你的产品定位是否恰当？这非常重要。所以，一旦你把这些差距——把这些差距分类好，就把它们输入到……Is your product like properly positioned? So that's very important. So as soon as you have the gaps di the gaps categorized you just feed that into

步骤 4：提供决策反馈Step 4: feed decisions

决策。所以把这些纳入决策，基本上就是优先处理这些问题，并让你的科学、工程和产品团队去解决它们。这很基础，对吧？然后你只要一遍又一遍地重复这个循环。考虑到这一点，比如在我们的实验室里，从去年三月以来，我们已经好几次重复这个飞轮了。我还注意到，基本上在我作为产品人的历程中，我的每一个……decisions. So feed that into decisions is basically you prioritize that and you ask your science, your engineering, your product team to address those. That's basic, right? And then you just continue repeating this loop over and over. So with that in mind, like in our lab, we repeated this flywheel several several times since March last year. And what I also noticed is basically as part of my journey as a product, each of my c

客户旅程的四个阶段Four stages of the customer journey

每个客户使用我的产品时都会有自己的历程。这意味着什么呢？也就是说，客户会随着使用我的产品而不断发展，对吧？客户使用产品的过程中，你基本上可以了解四件事。第一件事，是你第一次与客户接触时要做的事，基本上就是发现使用场景。在这第一步，你要做的就是了解他们面临的问题。也就是你的客户有什么问题，并确保你正在构建的产品能够解决这个问题，对吧？之后，你会有几个早期采用者，对吧？你想从这些早期采用者那里获得的主要信号，基本上就是了解哪些事情是有可能实现的。这时客户会尝试一些疯狂的做法，你会发现有些事情你根本帮不上忙，对吧？each customer will have their own journey using my product. What that means? So my customer evolves as they use my product, right? So there are basically four things that you can learn while your customer use your product. The first one is that's the first thing that you will do the first time that you meet a customer. It's basically use case discovery. So what you want to do here at this first step it's basically understand what's their problem. So what's the problem of your customer and make sure what you are building solves that. Right? Then after that you will have couple early adopters. Right? from these early adopters the main signal that you want to get. It's basically what's possible. So that's the time where customer will try crazy things. You will find things that you cannot help them at at all, right?

但你也会发现，客户正在用你的产品解决各种问题，而且很多时候你会感到惊讶，因为这些用途是你完全想象不到的。第三点，就是你开始扩大规模的时候，对吧？你获得了早期采用者，但现在开始扩大规模了。客户越来越多，你也能够识别出一些规律，对吧？这些规律是什么？基本上就是你的核心用例，对吧？也就是每个人都在用你的产品解决的那个问题。最后一点，着眼于未来，就是你的产品差距。在服务早期采用者的过程中，以及开始扩大规模之后，你也会发现产品存在一些不足。你的产品团队可以帮助你排定这些问题的优先级，而这些就是你未来的筹码——接下来应该开发什么。But you will also find things that your customers are using your product to solve and many times you will be surprised because it's something that you would never imagine. Third point, this is when you start scaling, right? You get your early adopters but now you start scaling. you are getting more customers and you are able to identify some patterns, right? What are the patterns? The pattern is basically your hero use case, right? That's the case, that's the problem that everybody is using your product to solve. And last one, this is more future looking, but these are your product gaps. So when you were working with early adopters and also after you start scaling you will also be finding some gaps on your product. Your product team can help you to prioritize those and these are the bats for your future right what you should build next.

嗯，正如我所说，在这个过程中我与好几位客户合作，有一些经验想分享。有些可能显而易见，但另一些则非常非常有启发性。所以……Um as I said I work with several several customers during this journey and there are a couple learnings that I want to share that some might be obvious but others it's very very insightful. So the

第 1 课：信任悬崖Lesson 1: the trust cliff

第一个是我称之为“信任悬崖”的现象。什么是信任悬崖？想象一下，你开发了一个产品，准备发布了。你测量了这个产品的可靠性，发现是 80%。这听起来还不错，对吧？你会想：“好，我可以把它交给客户，他们可以开始用了，对吧？”但当你告诉客户这个产品的可靠性是 80% 时，他们会想：“我没法信任这个产品。”为什么呢？因为当你告诉他们可靠性是 80% 时，他们会想：“哦，好吧，既然只有 80%，那我还是得监控这个智能体。它要是出错了，我还得手动处理。”所以事实上，这反而给我增加了工作量，而不是减少了。我不仅要部署这个智能体、监控它，还得在之后做一些手动处理。但当可靠性达到 90% 左右时，传达给客户的信息就变了。他们会想：“好，90%，我确实可以放心让它处理我的任务，把整个工作流程完全交给它。”first one is the the trust cliff that I I like to call. So what's the trust cliff? So imagine you build a product, you are ready to ship and you and you measure the reliability of this product and you found 80% reliability. That seems fine, right? You're like, okay, I can give that to my customer and they they can start using it, right? But what your customer thinks when you say you are 80% of reliability is I cannot trust this product. I cannot trust. Uh and why? Because when you show 9 80% reliability, what they will think is oh okay so if it's 80% I still have to monitor this agent. If they fail I will need to do some manual work. So in fact it's actually more work for me not less work. I need to build this agent plus monitor that and also do some manual work after but when you get to a point around 90% reliability the message changes to your customer and what they think it's okay 90% I can actually trust this right to handle my tasks and I completely hand over like a workflow to my customer.

所以，可靠性基本上就是从零到一，对吧？要么信任，要么不信任。不存在什么中间状态。So it's basically reliability is a zero to one, right? Or you trust or you don't. There is nothing like in the middle there.

第 2 课：坦诚面对局限性Lesson 2: be open about limits

第二个经验教训是评测透明度。每次开始与客户合作时，基本上都需要向他们透明地说明产品的三件事。第一件事很简单，大家都乐于谈这个：哪些方面做得好？你基本上就是向客户展示：我很擅长做这件事。这就是可靠性。你可以信任我的产品。只要使用我的智能体，它就能达到你所需要的可靠程度。 第二点是，我正在做哪些事情来改进产品。好吧，有些地方我知道还存在不足，但我目前正在投入时间，让它变得更好。 第三点就比较棘手了。没人愿意谈这一点：哪些事情不在产品的适用范围内？所以你确实需要做到透明。如果你的产品无法胜任某件事，就直接告诉客户。Second learning, it's the eval transparency. So every time that you start working with a customer, you basically need to be transparent about three things that your product does. First thing is that's easy. Everybody likes to do that. So what works? So you basically show to your customer I am really good doing this. That's the level of reliability. You can trust my product. Just use my agent and it will give you the reliability that you need. Second point is these are the things that I'm doing to improve my product. Okay. So there are some things that I know it's a gap but I am investing time right now to make this better. And the third thing that's the tough one. Nobody wants to do that that it's what is out of scope. So you really need to be transparent. Like if something doesn't work for your product, just go ahead and tell your customer.

别让他们去尝试。否则，现在市场上有太多工具了，如果你做一些……如果你向他们展示某些不起作用的东西，他们就会忘记我们的产品，转而使用下一个。所以要保持透明。这里最重要的经验是，坦诚说明产品的局限性，比追求更高的基准测试成绩更能赢得信任。这是个很有意思的发现，对吧？要坦诚说明你的产品能做什么，更重要的是，不能做什么。这样才能赢得客户的信任。第三个经验是，如果你工作、工作得非常……Don't let them try. Otherwise, like there are so many tools in the market right now that if you do some if you show something for them that doesn't work, they will just forget about our product and they will move to the next. So be transparent. And the main learning here is transparent on product limitation increase trust more than higher benchmarks. That's a very interesting finding, right? Be transparent about what your product is capable of doing and in more important what's not capable of. That's how you earn customer trust. Third learning is if you work work very

课程 3：令人惊讶的应用案例（Amazon Leo、Hertz、Sola）Lesson 3: surprising use cases (Amazon Leo, Hertz, Sola)

与客户密切合作时，你可能会发现很多意想不到的用例。这里有几个例子。第一个是 Amazon Leo，这是亚马逊旗下的卫星互联网公司。与他们合作时，我们了解到一些关于开发轨迹缓存功能的想法。因此，他们使用我们的产品时，不必每次都进行推理，而是缓存轨迹。如果缓存失败，我们就回退到模型。这样我们帮他们优化了成本。第二个经验来自 Hertz，一家汽车租赁公司。他们也在使用我们的产品进行 QA 自动化。我们了解到，他们公司里有一些 QA 工程师技术能力很强，能够用 Python 等编程语言编写脚本。不过，QA 团队里也有一些人没那么懂技术。close with customers, you might find a lot of use case that you would never expected. So some examples here is Amazon Leo. It's the satellite internet satellite company from Amazon. Working with them, we actually learn some inputs uh about creating features to do caching of trajectories. So instead of doing inference all the time that they use our product, we cache trajectories. If it fail, we just fall back to the model. So we optimize the cost for them. Second learning is with Herz, the car rental company. So they were also using our product for QA automation and we learned that as part of their company they had some QA engineers that were very technical and were capable of building like scripts with Python and other coding languages. However, some technical uh some people from QA they are not that technical.

这就是我们获得的输入，它让我们能够为产品打造一套完整的非技术人员使用体验，从而让更多客户能够使用我们的产品。最后一个例子是一家名为 Sol 的初创公司。Sol 基本上是一家 RPA 公司。如果你不熟悉这个术语，RPA 指的是机器人流程自动化，他们提供的工具可以帮助其他公司实现流程自动化。想想看，他们可能会有哪些各式各样的使用场景：种类非常多。我们从中了解到，可以让产品更加灵活，尤其是让他们能够定制我们工具的执行栈。这里的具体经验教训没那么重要，重要的是要记住，你会在这个过程中深入了解客户想做什么，对吧？以及接下来应该为产品开发哪些功能，然后让这个飞轮持续运转。So that was the input that we needed to actually build a whole non-technical experience for our product that would enable more customers to use it. Last one, this is a startup called Sol. Solar is basically a RPA company. If you are not familiar with the term, RPA is repetitive process automation and they offer tools to automate process for other companies. So imagine the variety of use cases that they might have. It's a lot of different use cases. And what we learned from then is we could give some additional flexibility in our product for them to customize especially the actuation stack of our tool. The learnings here are less important than keeping in mind that you will learn a lot about what your customers is trying to do here, right? and what should be the next features you should build for your product and then like running this flywheel

四项原则Four principles

我们多次总结出了这个飞轮的四项原则。第一，从生产环境中提炼评估场景，而不是凭空想象。再说一遍，创建合成数据在流程开始时可能非常重要，但一旦开始收到客户反馈，就要确保将这些反馈纳入评估。第二，将问题归类到可采取行动的领域。比如产品问题、工程问题、研究问题，并确保向团队妥善传达这些问题的优先级。第三，坦诚地分享能力。说到底，透明才能建立信任。确保与客户分享信息，尤其是产品不擅长的方面。第三，尽可能快地运转飞轮。基本上，飞轮转得越快，产品改进得也越快，对吧？这几乎是显而易见的，对吧？但请牢记这一点：尽可能快地运转飞轮。several times we came up with four principles for this flywheel. First one is derive eval uh eval scenarios from production not imagination. So again like creating synthetic data might be very important at the beginning of the process but as soon as you start getting customer signals make sure that feeds your eval second categorize into actionable areas. Again product issues, engineering issues, research issues, make sure you prioritize that well to your team. Third one shared capabilities honestly. Again, transparency builds trust. Make sure you share to your customer, especially what your product is not good at. Third one is run the flywheel as fast as you can. So here it's basically the faster you run that flywheel, faster your product will improve, right? That's almost obvious, right? But keep it that in mind like run your flywheel as fast as you can.

然后，关于这四项原则，你会看到，呃……And then like with those four principles what you will see is uh

评测应该每周都变得更智能。Evals should get smarter every week

你的评测应该每周都变得更智能。但非常重要的一点是，如果你发现评测没有每周变得更智能，那就意味着它们已经过时了。好。最后，为这场演讲收个尾，接下来……your evals will should get smarter every week. But very important if you are not seeing your evals getting smarter every week that means your evals are getting stale. Okay. And just to finalize the talk here there

两点启示Two takeaways

有两个想法我希望大家能带回去。就两个。第一个想法是，评估必须反映客户真正关心的事情，而不是你以为他们关心的事情。关键是要听取客户的意见，确保你的评估能反映他们面临的问题。第二，评估不是一次性建成的。它是一个随着客户不断变化而演进的飞轮。这里要记住一点：客户旅程会不断推进，而随着客户走过这段旅程，他们想用你的产品做的事情也会发生变化。比如，这些事情会变得更复杂，对吧？所以，你的评估需要及时更新，准确反映客户此刻想要做的事情。为此，你要做的其实很简单：建立这个循环，交付真正有意义的评估，然后赢得客户的信任。今天我就讲到这里。are two ideas that I want you guys to take home. Just two. So first idea is eval must reflect what customer cares about not what you think they care about. Okay. All about getting customer input here and make sure your evals reflects what's their problem. Second is evals are not built once. It's a flywheel that evolves as your customer evolves. One note here, remember that customer journey, as your customer moves through that customer journey, what they are trying to do with your product will also change. Like it will get more complex, right? So your evals needs to be up to date and reflecting exactly what your customer is trying to do right now. And with that what you have to do is simply like build the loop ship evolves that matter and then you earn your customer trust. And that's all that I have for today.

非常感谢大家的参与。会后我会留在我们的展位，欢迎大家继续交流。再次祝大家会议愉快。Uh thanks very much for joining. Uh I will stay in our booth after for more discussions. But again enjoy the conference.