Anthropic 如何用 Claude 交付 Claude：将大脑与双手分离 — Latent Space

文字稿Transcript

自动转录的草稿。姓名和措辞可能需要更正；尚未标注发言者身份。Automatically transcribed draft. Names and wording may need correction; speaker identities have not been assigned.

0:00 大家好，今天我们来尝试一些新内容。我们将开始报道 Anthropic 的 IPO。就在今天，他们刚刚提交了 S-1 文件。因此，我们将推出一系列播客节目，为预计两周后、估值达 2 万亿美元的 IPO 预热。Anthropic 是有史以来增长最快的公司之一，据报道，其累计融资额超过 1000 亿美元，而累计烧钱不到 100 亿美元。所以，其中有很多关于效率的故事值得深入探讨。我想，这也是我们想要报道的内容：Anthropic 如何在推出全球最强大模型的同时，保持极高的效率，并打造出一家了不起的企业。我整理了一些资料——如果你一直在收听我们的播客，可能知道我以前做过金融分析师。0:00 Hi, today we're trying something new. We're launching coverage of Anthropic for the IPO. They just filed the S1 as of today. And so we're releasing a series of podcasts leading up to the IPO, which is expected to be in two weeks for $2 trillion. And it's one of the fastest growing companies of all time with a reported lifetime cash raise of hundreds of over $100 billion and a lifetime burn of less than $10 billion. So there are a lot of efficiency stories to dive into there, and I think that's something that we want to cover, how Anthropic is simultaneously shipping the strongest models in the world while also doing it very, very efficiently and building an incredible business. So I've pulled out, you know, a lot of you know, if you've been listening to the podcast, that I have a former life as a financial analyst.

0:50 我从 S1 文件中摘出了一些内容，我觉得这些内容能为今天的播客提供一些背景。我们只是把这个当作一种新尝试，因为我觉得你们有些人希望和我们建立更亲近的个人联系，想看我们直接对着镜头讲话，而不是像过去四年那样和嘉宾交谈。首先我想强调的是，按员工人数衡量，Anthropic 是一家效率极高的公司。这也许显而易见，因为它是一家 AI 公司，所以他们显然很善于运用 AI。但我认为，他们在过去一年里作为一家公司的变化程度，并没有得到充分理解。所以，现在每位员工的 12 个月营收达到了一万亿美元……0:50 And I pulled out some statements from the S1, which I thought would be interesting to give context to today's podcast. and we're just trying this as a new thing because I feel like some of you want that personal relationship with us talking directly to camera rather than talking to our guests which we've done for four years so the first thing I want to highlight is that Anthropic is an extraordinarily efficient company by human count and that's maybe obvious because it's an AI company so obviously they're using AI very well but I think the degree to which they have changed as a company over the past one year is not well understood. So a trillion 12 month revenue per employee is now

1:32，660 万。相比一年前的 140 万，他们的人均营收几乎增长了五倍。我认为这值得我们深入探讨，因为我觉得任何企业主、任何工程师，都希望提高人均营收——这不仅能提升盈利能力，也能增加支付给员工的薪酬。这显然体现在每美元算力成本所带来的营收大幅增长上。值得注意的是，基本上在 2025 年第二季度，Anthropic 每获得 1 美元营收，就要亏损 0.5 美元。而现在，每花费 1 美元，它就能赚 2 美元。我认为，这是一个很有意思的例子，说明过去一年里，Anthropic 的营收和业务前景经历了多么巨大的拐点。1:32 6.6 million. As compared to one year ago, it was 1.4 million. So they've grown almost five times the revenue per employee. And I think that's something that we need to dive into more because I think any business owner, any engineer, you want to increase your revenue per employee, not just to increase profitability, but also to increase the amount you can pay your people. This is obviously tracked by a significantly increase in revenue per dollar of compute. And it's notable that basically in Q2 of 2025, Anthropic was basically losing half a dollar for every dollar of revenue. And now it's making $2 for every $1 in cost. And I think that's an interesting analogy of how enormously huge this inflection in Anthropics revenues and business outlook has been over the past one year.

2:27 今年对 Anthropic 来说是极其关键的一年，必须把事情做好，而他们确实做到了。我觉得我们基本上会深入探讨这件事，并尽可能从播客外部视角讲述这个故事。嗯，这一连串增长带来的一个结果是，Anthropic 刚刚公布了首个盈利季度——不是调整后利润，也就是把所有让我们花钱的项目都扣除之后的利润，不，不是这个，而是按公认会计准则计算的利润，利润为 19。这数字非常惊人。再说一遍，这跟一年前相比完全是个大反转，甚至不只是反转，而是……2:27 This year was extremely critical for Anthropics to execute well, and it has done that. And I think we're basically going to drive deeper into this and tell that story as much as we can from the podcast external point of view. um the one consequence of this entire increase is that anthropic has just reported its first profitable quarter like not adjusted profit which is profit after we take out all the things that cost us money no this is generally accepted accounting principles gap profit uh of 19 and that is enormous again completely flipping from one year ago more than flipping like one

三年前，它每获得一美元收入，就要亏损129美分。我认为，主要归因于此的因素，正是它非凡效率的来源。你知道，Claude 喜欢循环，Boris 喜欢循环，Dario 也喜欢循环，而 Anthropic 正在经历两个循环。第一个是改进循环。基本上，这个循环是这样的：有了 Claude，Claude 促成更好的模型；更好的模型带来更多使用场景；更多使用场景产生更多反馈，进而打造出更好的 Claude。另一个是投资循环。随着应用范围扩大，收入随之增长；增长越快，投资就越多；投资越多，能力也就越强。3:03 year ago, it was losing 129% of every revenue dollar that came in. So the primary attribution, I think, that explains this extraordinary efficiency. As you know, Claude loves loops, Boris loves loops, Dario loves loops, and there's two loops going on in Anthropic. The first is the improvement loop. Basically, there is the cycle of you have Claude, Claude leads to better models, Better models lead to more use cases. More use cases generate more feedback, and that leads to a better cloud. And then there's also the investment loop. So revenue from expanding adoption leads to higher growth. Higher growth leads to more investment, and more investment leads to more capabilities.

3:45 这就带来了更好的云。因此，根据 Adoptic S1，这两个循环基本上会协同运转，形成一个无限循环，推动效率不断提升，直到永远。我认为，这就是我们从整体效率的故事转向他们如何执行这一战略的过渡。今天，我们将重点关注 Claude Platform，因为它主要是 Anthropic S1 所称的云扩散三大要素之一。他们将 Claude Platform 称为一个全面的智能体开发环境，为客户提供直接访问其基础构件的途径，以便客户构建、部署和扩展自己的应用程序，并在自己的产品和工作流程中利用 Cloud 的能力。3:45 That leads to a better cloud. So these two loops basically spin together and form an infinite loop that is driving the increase in efficiency until the end of time, according to the Adoptic S1. So I think that is our transition from the overall efficiency story to how they execute it. And today we're going to focus a lot on the Claude Platform because that is primarily one of the three things that they call the diffusion of cloud in the Anthropic S1. So they call this Claude Platform our comprehensive agentic development environment. They provide customers with direct access to their primitives to build, deploy, scale their own applications that leverage Cloud's capabilities within their own products and workflows.

4:30 我认为，这期播客的讨论中经常提到的一个概念是“吃自己的狗粮”（dogfooding）：Claude 的许多产品实际上都是用向你开放的同一个 Claude 平台构建的。Claude 平台还有一个值得关注、且已披露的数据：按过去十二个月计算，收入达到或超过 10 万美元的客户中，超过 50% 使用 Claude 平台。这表明 Anthropic 的应用正在扩大，也显示它不只是一个模型实验室，还正在成为超大规模云服务商和云平台。我觉得，这或许本来就隐含在“云平台”这个名称里，但如今正从理论走向实践。4:30 I think one thing that comes up a lot in the discussion in this podcast is the concept of ad fooding, where a lot of Cloud's products is actually built with the same Claude Platform that is available to you. So something that's also interesting in the Claude Platform that was disclosed is over 50% of their customers with $100,000 or more revenue on a trading telephone basis use Claude Platform. And so that is an increase in adoption and emergence of Anthropic not only as a model lab, but as also a hyperscaler and a Claude Platform, which I think maybe is in the name inherent in a Claude Platform and a Claude Platform, but is going from theory to actual practice.

5:15 这正是我们在播客中重点关注的内容。另一件我也一直努力强调的事——之前也提到过——是 Claude Code 和 Claude Cowork。它们使用的都是 Claude Platform 所提供的相同基础构件。我觉得这很有意思。归根结底，Anthropic 似乎正围绕着从聊天机器人到副驾驶，再到虚拟协作者这一发展方向来定位其 IPO。他们用这个词，而不是“智能体”，或许听起来更亲切一些。但它与人们期待智能体具备的许多特性相同：长期自主执行工作、判断、自我纠错、迭代以及调用工具的能力。也就是说，它能承担一个目标，与你并肩工作，而不是局限于某个应用或任务之中。这就是我想为今天的播客提供的背景。5:15 And this is something that we're focusing on in the podcast. The other thing that I also really try to emphasize, which I talked about before, is Claude Code, cloud co-work. They all use the same primitives that Claude Platform exposes. So I think that's really interesting. Ultimately, it looks like Anthropic is really positioning their IPO to talk about going from chatbots to co-pilots to virtual collaborators. They use this term instead of agents. Maybe it's more friendly. but it's a lot of the same things that you look for from agents, from long-running autonomous work to judgment, self-correction, iteration, and tool access. So owning a goal and working alongside of you instead of working inside of one app or task. So that's ultimately the context I want to provide for today's podcast.

6:00 这是在 Opus 5.5 发布时录制的。我想说，在我们采访过的所有品味出众的测试者当中，几乎所有人都认为 Opus 5.5 是目前世界上最好的模型。而且 Anthropic 甚至已经把 Fable 5.5 蓄势待发，可能会在接下来两周内发布。我不需要什么内部消息，但这对 IPO 来说非常合理。让我们拭目以待，也进一步了解一下他们是如何打造 Opus 5.5 的。6:00 This was recorded when Opus 5.5 was launched, and I would say overwhelmingly among all the high-taste testers that we talked to, Opus 5.5 is the best model in the world right now. and Anthropic is even just sitting on Fable 5.5 and probably will release it in the next two weeks. I don't have to have insider information, but it makes a lot of sense for the IPO. So let's look forward to that and learn a little bit more about how they build Opus 5.5.

6:25 是的，我想我会做这样一个区分：人们听说和读到的很多事件，都是发生在模型身上的事件，而不是说所有正在训练的模型都会遇到这种情况，对吧？运行这些模型的方式是，把它们关进一个盒子里，然后告诉它们：“解决一个不可能完成的任务。我要给你施加压力。准备，开始！”6:25 Yeah, so maybe a distinction that I would make is in a lot of the incidents that people are hearing about and reading about, these are incidents that are happening to models, not just like everybody's models, right, that are in training, where the way that you're running these models is you like put them in a box and you say like solve impossible task. I'm going to stress you out. Ready, set, go.

6:45 对。你只能在自己的环境里做力所能及的事。而代理基本上会受到激励，试图突破自己的框框去解决问题。当人们通过我们的 API 或其他任何人的 API 运行生产工作负载时，他们运行的环境里有一些防护机制在起作用；而在训练时，这些机制未必会运行，因为训练的全部目的就是训练模型。对吧？但在生产工作负载中，首先，你有这些在线防护机制，这非常好。其次，围绕正在发生的事情，还有更多这样的架构和基础设施。代理并不是被关在一个框里，试图逃出去。对吧？比如，控制框架在这边，操作端在那边。这根本就不是一个需要你考虑的问题。6:45 Right. And you like can only do what you can do within your environment. And the agent is basically incentivized to try to get out of his box and go solve the problem. When people are running production workloads on our API or on anybody else's API, you're running an environment where like there are safeguards that are running that are like not necessarily running and training where like the whole point is to actually train the model. Right. But in production workloads, first of all, you have those online safeguards, which is really great. Second of all, you have more of this architecture and infrastructure around what's happening. Like the agent isn't just in a box trying to get out. Right. Like the harness is over here, the hands are over there. And it's just like not even a problem that you would have to contemplate.

7:30 好，我们现在在演播室，今天是 Opus 5.5 发布后的第二天，和我们在一起的是 Katelyn 和 Angela。欢迎。谢谢邀请我们。恭喜你们推出了目前全球最顶尖的模型。我觉得这得看你问谁。是啊，得看你问谁。对你们来说，这次最主要的收获是什么？你们已经发布了官方博客文章。降价幅度真的很不错。推出这样一个新模型时，你们最大的感受是什么？我们为这个模型感到非常自豪。团队投入了大量工作，才让它得以问世。我觉得我们确实听取了大家对 Opus 5 写作风格有些晦涩难懂的反馈，并努力改善了这一点。看到大家似乎对此反响不错，我真的很高兴。我们内部试用这个模型时，也有同样的感受。所以得到这样的认可真的很棒。至于你提到的降价，其中有不少内容。7:30 All right, we're here in the studio the day after Opus 5.5 with Katelyn and Angela. Welcome. Thanks for having us. Congrats on launching the top model in the world right now. I would say depends who you ask. Yeah. Depends who you ask. What is the main takeaway personally for you guys? You have your official blog posts. The pricing cuts are really cool. What do you take away when you launch a new model like this? We're super proud of this one. The team has put in a ton of different work to just go make this one happen. And I think we've really heard the feedback on Opus 5's writing is kind of convoluted. We really tried to clean that up. And I'm really glad to see that it seems to be resonating with people. Internally, when we tried out the model, we felt the same way. So it's just really great to get that validation. To your point around the pricing cuts, there was a lot there.

8:17 我认为，越来越多的人在追求每美元能获得更强的智能。因此，确保我们把每美元的性价比做好，一直非常重要。还有一些边缘方面的工作，并不完全是关于——你知道——也许是我们在博客文章里没提到的内容。我们确实花了很多精力，努力让我们的 API 更难被用于蒸馏。为此，我们做了一些改动，让这件事变得更加具体、更加具体。这些事情包括，你知道，比如我们如何实现前缀锁定式的思考，或者如何在不同类型的模型之间重放思考 token。我们也投入了大量工作和精力，确保针对一些非常正当的使用场景做出调整，比如你可能会在压缩策略之类的地方这样做；而不是那些实际使用目的来自恶意行为者的场景，8:17 And I think increasingly people are looking for good intelligence per dollar. And so making sure we get the per dollar part right has been really important. There's some stuff on the edges that aren't really about like, you know, maybe stuff that we haven't talked about in the blog post, which is that we did do a lot of work on trying to make our API harder to distill from. And so there has been some changes that we have made to go and make that a little bit more concrete. concrete. And these are things about, you know, like how do we do like prefix lock thinking or how you can replay thinking tokens across different types of models. And there's been a lot of like work and effort on our part around making sure that we try to adjust things for very valid use cases for where you might want to go and do this, like potentially in your compaction strategies and things like that, versus in scenarios where the use case of that is actually from like a bad actor,

实际上是 9:05。所以在这些方面，我们一直在小心地走钢丝。看到很多相关内容也发布出来，我很高兴；除此之外，我们还推出了一个性价比高、智能表现出色的模型，也把沟通做得更清楚了一些。是的。具体来说，你指的是这篇论文：《从专有 Yellow Lips 窃取推理轨迹》。没错，就是那篇论文。这事确实是真的，对吧？我的意思是，显然他们对你们进行了负责任的披露。他们做得非常负责任，我们也非常感谢他们主动联系我们。能不能给那些还没听说过这件事的人讲讲？这事真的很酷。你们可以讲讲。我的意思是，问题已经修补了，不过当时是怎么回事？这有没有让 Anthropic 吓了一跳？可能没那么严重吧。我不知道。我不清楚这篇论文具体有没有引起多大的恐慌，因为这是……9:05 effectively. So there has been a lot of threading the needle around those fronts. And I'm glad to see a lot of that go out as well, in addition to just like shipping a model that's good intelligence for your dollar and also like just communicating a little bit clearer. Yeah. Specifically, you're talking about this paper, Stealing Reasoning Traces from Proprietary Yellow Lips. Yes, that paper. Which is real as far as, right? I mean, obviously they did like responsible disclosure with you guys. They did super responsible disclosure and we're super thankful for them coming to us. Could we, for those who haven't heard about it, this is really cool. You could do it. I mean, it's patched, but like, what was it? Like, did it cause a freak out in anthropic? Probably wasn't that big a deal. I don't know. I don't know how much this paper specifically caused a freak out because this is something

9:48，这是我们知道存在的一个缺口。是的，所以这里有两件不同的事。一件是中文模型蒸馏。对。另一件是后来发表的一种备受关注的技术。对，没错。我们很早就知道，提取思维链是蒸馏模型最有效的方法之一。实际上，如何在不同轮次之间保留思考块，并让这成为更强制性的要求，是我们很久以前就开始探索的事情。我们当时面临的情况是，总要在两者之间做出非常艰难的权衡：用户体验——比如可能会让 API 用户感到不便的用户体验改动——以及防止模型蒸馏。9:48 that we knew was a gap. Yeah, so there's two separate things. There's the Chinese model distillation. Yeah. And then this was a technique that was published that was high profile. Yeah, exactly. We've known for a long time that chain of thought extraction is one of the most powerful ways that you can distill models. And actually something like how do we preserve thinking blocks across turns and make that something that's more required is something that we've explored back for a while. And where we were at was we always have this really, really hard tradeoff that we have to make between user experience, like user experience changes that are maybe painful for users of the API versus protection against distillation.

10:32 那么，我们该如何把握这个分寸，并说我们是在防止模型蒸馏呢？也许我们可以稍微退一步来看。我们不希望模型被蒸馏，是因为 Anthropic 存在的根本目的，就是希望 AI 能在安全的前提下为世界带来积极成果。我们会对生成的 token 运行大量安全防护分类器，以确保它们是安全的。如果有人蒸馏我们的模型，却在没有安全防护分类器的情况下运行这些模型，可能会给世界带来不良后果。这是我们不希望发生的事。所以，我们不希望有人蒸馏模型，纯粹是为了将其用于恶意或不良目的。因此，对于所有出于善意使用这些模型的人来说，这始终是一种有意思的权衡，对吧？10:32 And so how do we kind of walk that line and say we're preventing distillation? And maybe just take a step back. The reason we don't want distillation is because the entire point of why Anthropic exists is because we want AI to create positive outcomes in the world safely. And we run a lot of safeguards classifiers on the tokens that we're producing in order to make sure that they're safe. If someone distills our models and they run those models without safeguards classifiers, there could be bad outcomes in the world. And that's something that we don't want to have happen. So we don't want people to distill the model specifically for the sake of just like using them for malicious reasons or for bad reasons. And so it's always an interesting tradeoff of like for all the people who are using the models for good, right?

11:16 我们不想让人们更难从模型中获取智能，但为了防止这种蒸馏，我们必须做出一些取舍。我们已经花了一段时间研究如何防止思维链被提取。比如，怎样才能保留思考块，同时又不让用户觉得太麻烦？我们大致把用户体验上的影响分成两个层面。第一种是你必须调整集成方式，这很烦人，但仍然可以完成原本能完成的所有事情，只是稍微麻烦一点。第二种则是你实际上无法再完成以前能完成的事情。而我们真的、真的、真的在努力避免这种情况发生。最后，我们为最近几个模型敲定了一些渐进式改动，以便在不同轮次之间保留思考块。11:16 Like we don't want to make them have a harder time getting the intelligence out of the models, but we have to make some tradeoffs in order to prevent that distillation. And so we had been working for a while on how do we prevent that chain of thought extraction? Like, what do we do about preserving thinking blocks in a way that wouldn't be too painful for people? And we kind of think about user experience hits in, like, two layers. Like, one is just, like, you have to make an integration change, and it's annoying, but you could still accomplish all the same things that you otherwise would have been able to accomplish. It's just a little bit annoying. And then two is, like, actually you can't accomplish the same things that you could have accomplished previously. And that's the one that we're, like, really, really, really trying to not have happen. And so where we landed with the last couple models is a few incremental changes to enforce these preserved thinking blocks across turns.

12:04 现在开始更多地关注跨账户的情况，你知道，比如你在使用多个账户之类的，对吧？我们要如何确保你无法利用这些模型来提取思维链？因此，我们推出了几个额外功能来应对这个问题。实际上，我们与许多开源 harness 的维护者合作，也主动联系了他们，和他们进行了交流。我们告诉他们，他们需要在自己的 harness 中进行这些升级。我们希望确保他们能顺利使用，同时也要照顾到那些基于 API 构建 harness 的大客户。对其中一些客户来说，一个很好的例子是，他们采用了压缩策略，而为了保留这些思考块，就必须对这些策略进行调整。因此，我们在 API 中推出了一项新的压缩功能，来替代他们原先采用的方案。12:04 And now starting to be more about across, you know, like if you're using multiple accounts and things like this, right? Like how do we make sure that we're ensuring that you aren't able to take advantage of this chain of thought extraction with the models? And so we shipped a couple additional features to help with that. We worked actually with a lot of open source harness maintainers. We reached out to them. We talked to them. We were like, you're going to have to make these upgrades within your harnesses. We want to make sure you have a good time, as well as obviously our big customers who are building harnesses on top of the API. And for some of them, a really good example is they have compaction strategies that would have been broken by the need to preserve these thinking blocks. And so we shipped a new compaction feature within the API to replace what they had otherwise

12:49 一直都是手动做的，不过保留思考内容时仍然能用。这只是个例子：我们一直在走钢丝，但并没有贸然行事，比如直接要求你必须严格按照这种方式保留这些思考区块，因为那样会让很多人措手不及。不过，我们有时间与客户以及开源 harness 的维护者合作，给他们提供升级所需的东西。既然说到适配，蒸馏这件事有没有什么相反的情况？也就是，你知道，另一面。嗯，没有，我们发现蒸馏确实更难。根据我们能够观察到的指标，这件事难了很多，因为这是……也要称赞一下这里的研究人员，他们确实非常负责任地披露了相关情况。所以，限制开放访问是件好事。12:49 has been doing manually, but still works with preserved thinking. So it's just an example of like, we're walking this line, but we didn't just like kind of YOLO out like, now you have to preserve these thinking blocks in this exact way, because we would have broken a lot of people, but we were able to take the time and work with customers and open source harness maintainers and give them what they needed to be able to make that upgrade. While we're on the topic of fit, anything on the other side of distillation? So, you know, the other half. Yeah, no, we have seen that it is harder to distill, like through the measures that we are able to take a look at, it has been significantly harder because this is, and credit to the researchers here, they did disclose it very responsibly. And so closing it down was great.

13:30 而且他们确实非常公开地发布了结果。我知道这件事在 Twitter 上引起了不小的热议，所以很多人都能用上它。不过，我们确实发现，在我们能够测量的方面，它似乎减弱了那些我们不希望出现的负面影响。是的，我想补充一下 Katelyn 刚才说的话：至少我经常在 Twitter、X，或者你想怎么称呼它都行，看到有人把这说成是商业化问题之类的。如果你能蒸馏一个模型，就可以拿来以低廉的成本运行，诸如此类。虽然这话没错，但说实话，这并不是我们特别在意的事情。比起其他任何方面，我们更关心的是安全问题。所以我们通常会从这个角度来考虑，而不只是从商业角度。现实情况是，如果你从我们这里蒸馏模型，你实际上是在付钱给我们。所以从商业角度来说，我们是赚钱的。所以这并不是……13:30 And they did publish the results very publicly. And I know it did kind of trend quite a bit on Twitter. So many people could use it. But yes, we have seen that actually from the areas that we are able to measure, it does seem to have reduced the effect, the negative effects that we don't want. And yeah, the bit I would just add to what Katelyn just said is I think at least I see a lot on Twitter or X or whatever you want to call it. It's a commercialization problem or something like that. If you can still a model and you can take this, then you can run it for cheap and so on and so forth. While true, it's actually not something we really care about very much. We care much more about this from a safety point of view more than anything else. And so that's why we tend to think about it from that angle more than just commercially. Realistically, if you distill from us, you actually pay us. So commercially, we make the money. So it's not

14:16 真的会产生那种影响。重点只是确保我们的模型——我们想确保 Claude 做得好，而且 Claude 能够……如果结果是负面的，那就太遗憾了。一般来说，关于 5.5，既然你们刚刚经历了这一切，有件很有意思的事值得聊聊：你们两位之间的合作模式。你们也经常一起露面，而且你负责产品，他负责工程。你们两个人是怎么配合的？尤其是在发布模型的时候。我们就举个例子吧。对于模型发布，我们的产品团队和工程团队合作得非常紧密。对我来说，很大一部分工作是权衡取舍：我们要在发布具备足够智能水平的产品与控制成本之间找到平衡。14:16 really that kind of an impact. It's just more on making sure that our models, we want to make sure a cloud is good and cloud could be derived. And if it's a negative, that would be unfortunate. Just generally, also on 5.5, because you guys are just fresh off this, one thing that's really fun to establish the dynamic between you two, because you also do a lot of appearances together, is you are running your head of products, your head of engineering. How does this play between the two of you? Especially for a model launch. Let's just take an example. So for model launches, So our team works together very closely between product and Eng. On my side, a lot more of it is about what is the balance of trade that we need to make between launching something that has the right intelligence bar, but also the cost to preservation.

15:00 我们调整后的许多现金成本之类的信息，都表明我们希望鼓励大家进行对所有人都有益的集成改动，并把节省下来的成本直接让利给大家。但围绕这些改动有很多类似的决策，还要考虑如何平衡安全方面的工作，同时确保这是一个很棒、能让大家受益的模型。我会说，模型发布方式的这些特征、品质以及相关组成部分，更偏向产品方面。而从集成体验的角度来看，其中很多工作则是工程团队直接负责的。显然，如果你谈的是 API 的使用体验，那它们之间的整合就非常非常紧密。实际情况真的很棒：我们有很多工程师都非常有产品意识，而我们也非常喜欢这一点。15:00 A lot of our updated cash costs and things like that are indications of us being like, we want to encourage you to take integration changes that are beneficial for actually everybody and just pass those savings on. But a lot of these kinds of decisions around that and then how do we balance some pieces around our safety motion as well as making sure that it's like an awesome model that people can take advantage of. And so those kinds of traits around like I would say maybe the qualities and surrounding components around how we launch the model are more on product. And then from an integration experience standpoint, a lot of that is from like the engineering team directly. So we work, obviously, if you're, like, talking about the experience of an API, that's, like, very, very, very tightly integrated. And so it's actually been really awesome. We have a lot of engineers who actually are deeply product-minded, and we actually, like, love that.

15:49 我这边也类似，有很多技术背景更强的产品经理。我们希望有这种融合型、跨学科的人才，因为这样他们就能从各个不同角度分析，尽量找出最佳做法。不过我往往会更侧重商业影响；而我觉得，实际上可以这么说：工程师会更在意集成体验。所以他们看待这件事时，几乎会更多从用户体验的角度出发，而这里的用户体验显然更多是指 API 和开发者工具。是的，做一名负责开发者产品和平台的产品工程师很有意思。所以我也认同这一点。我觉得我们有很多工程师很喜欢……思考产品，喜欢思考用户体验，也喜欢思考开发者体验。15:49 And similarly on my side, there's a lot of, like, product managers who are much more technical. And we want this, like, blended kind of interdisciplinary type of people because then they can kind of analyze it from all the different angles to try to figure out how to do that best. But I tend to lean a little bit more on like commercial impact versus I think I'd actually frame it as more like the engineers are going to care much more about the integration experience. So they almost take more of a UX angle to it where the UX is obviously more about the API and the developer tools. Yeah. It's fun to be a product engineer who works on developer products and platform. So I would echo that. I think we have a lot of engineers who love. thinking about the product, who love thinking about the user experience, who love thinking about the developer experience.

16:31 我觉得我们彼此之间也有这种互动：我有很多自己的看法，在我们如何看待这些事情上，我常常会让产品团队觉得有点烦。我们的产品团队技术能力也很强，他们会深入探讨我们为了规模化或其他原因而做出的决策。这是我们团队里一种挺有意思的互动。对，有些事……凯拉和我共事很久了。在 Stripe。我们在 Stripe 共事过，后来我想我们在 OpenAI 没有一起工作。各自独立的实验室。各自独立的实验室。然后我就过来了。不过我觉得我们俩有个有意思的特点：凯特琳经常被误认为是产品经理，而我则经常被误认为是工程师，尽管这并不是我们实际的职能。所以我认为，这样的搭档关系很有帮助，能让我们双方做出的决策保持一致。16:31 I think it's a dynamic that we have between ourselves, too, where I have a lot of opinions and am annoying with our product team on how we think about these things. And our product team is also really technical, and they can get in the weeds of the decisions that we're making for scale or for whatever it might be, which is a fun dynamic that our team has. Yeah, something... Kayla and I have worked together for a long time. At Stripe. At Stripe, and then I guess we did not work together at OpenAI. Separate labs. Separate labs. And then I came over. But I think one interesting property of both of us has been Katelyn frequently actually gets mistaken for a product manager. And I frequently get mistaken for someone in engineering, even though those are not actually our job functions. And so I think that pairing tends to help out a lot with just making decisions that are consistent across both sides.

17:18 所以我觉得我遇到过很多公司，采用的都是类似的配置：一位产品负责人和一位工程负责人。随着软件工程的问题得到解决，也许这两个角色会合并？我觉得有可能。是啊，我觉得大家往往会倾向于……不过如果你们俩现在还是分开的，那说明我们还没走到那一步。我们肯定还没走到那一步。那为什么呢？还剩下什么问题？我觉得软件工程问题得到解决这件事……我不知道我会不会用这么强烈的说法。我觉得……我通常会这样看待这些事情：我们的目标会不断提高，对吧？举个例子，我们团队现在投入大量时间做的很多事情，当然，显而易见，为了安全地发布我们的模型，我们必须完成的所有工作，实际上是我们最重要的任务，因为我们必须在安全方面做很多工作，包括智能体的安全、离线防护和在线防护，以及正在发生的所有这些事情。17:18 So I think I come across a lot of companies with a similar product leader, engineering leader setup. As software engineering gets solved, maybe those two roles merge? I kind of think so. Yeah. I think everyone tends to lean towards... But if you guys are separate still, then we're not there yet. We're definitely not there yet. So why? What's left? I think that software engineering getting solved is... I don't know if I would ever use language quite that strong. I think there's... The way that I tend to think about these things, like ambitions will continue to elevate, right? And maybe a good example, A lot of what our team is spending time on right now is, of course, obviously, all the things that we have to do to launch our model safely is actually the most important thing that we're doing because of all the work that we have to do on security with agents, with offline safeguards and online safeguards, all these things that are happening.

18:06 但我们正在做的另一半工作，是尝试扩大规模。部分原因是，我一年多前加入 Anthropic 时，我们只有 Messages API。它是一个 Python 单体应用。我们在 GCP 的单个区域运行它，一切都很顺利。我当时准备接这份工作时，我的亲友中没有人知道 Claude 是什么，对吧？而现在，我们的处境已经截然不同了。API 流量的增长幅度非常惊人，我们不得不设法应对如此庞大的请求量。CPU 资源也非常紧张，这对我们来说是个很有意思的问题。因此，我们在如何扩展这些系统上做了大量工作；显然，这需要很多分布式系统方面的专业知识。18:06 But the other half of what we're doing is trying to scale. And part of that is because when I joined Anthropic a little over a year ago, we had the Messages API. It was a Python monolith. We were running it in a single GCP region. and everything was fine. And none of my friends and family knew what Claude was when I was going to take this job, right? And now we're in an extremely different place. And so the metrics are really wild in terms of how much volume we've seen and have had to support on the API. CPU is really constrained, which has been a really interesting one for us. And so we've done a lot of work on how do we scale these systems? And that takes a lot of obviously just like distributed systems expertise

18:45，以及模型所具备的某些编程语言和框架方面的专业知识。但说到像这样大规模地运行和迭代一个系统，我们离那种“祝你好运，Claude，玩得开心，尽管去做你需要对系统做的所有改动，安全地发布，然后大规模运行它”的状态还差得远。还有很多工作需要人来做，我想这种情况还会持续一段时间。我很好奇，这方面发生了哪些变化。比如说，前一阵子 Fable 刚推出时，我们有 Glasswing，之后模型的发布节奏和方式都经历了很大的延迟和变化。大多数前沿模型都是这样。发布前会进行安全测试。现在情况怎么样了？你们怎么看？你们是如何决定的？比如，有一件事是新的 Sonnet 和 Haiku 也即将推出。18:45 and certain language and frameworks expertise that the models have, but when it comes to operating and evolving a system at scale like this, we're not nearly in a place where it's like, good luck, Claude, have fun, and just go and kind of make all the changes you need to make to the system, roll them out safely, and operate this thing at scale. There's still just a lot of work for humans to have to do that I imagine will continue for a while. I'm curious how that's changed. So like a while ago, you know, when Fable first came out, we had Glasswing, and there was a whole delay and change in how models started to get released. We saw this for most frontier models. There's safety testing. What's the status of things now? How do you guys see it? How do you determine, like, one thing is there's a new Sonnet and Haiku also coming.

19:31 我猜那些就会直接发布。不过像 Opus 5.5 或新款 Fable 这样的产品，目前进展如何？是的，我想说，这要从我们在防护措施和安全方面需要做出怎样的政策决策开始。这既包括离线安全，也包括在线安全。最近大家谈了很多数据保留，以及我们在这方面采取了哪些措施，以便安全地发布这些模型。原因在于，有些情况下，整个庞大网络中会有很多很多账户，人们会开设账户，以各种方式使用这些模型。而在在线场景中，当令牌在系统中流动时，我们不可能及时发现是否有人在利用这些模型做恶意的事情，或者进行蒸馏，或是从事其他可能发生的行为。19:31 I assume those will just come out. But for something like Opus 5.5 or a new Fable, what's the current status? Yeah, so I would say this starts with what our kind of policy decisions need to be on safeguards and on safety. And that's both offline safety and online safety. And so there's been a lot talked about lately around data retention and what we're doing there in order to, you know, release these models safely. And the reason for that is because you have, you know, situations where there are many, many accounts across like a whole giant network of people who are choosing to spin up accounts and use the models in various ways. And there's no way that in an online way, as tokens are flowing through the system, we would be able to kind of catch that there's something malicious happening with the models or distillation or whatever might be happening.

20:19 因此，我们已经开始需要制定数据保留政策，以便进行自动审查和人工审查，无论需要采取哪种方式，都要确保模型运行良好。对于我们发布的每个模型，这都是一个需要作出的政策决策，例如，针对这个特定模型，政策应该是什么。比如昨天发布的 Opus 5.5，就没有 Fable 那样的数据保留要求，对吧？但每次发布这类模型时，我们都必须作出这样的决定。如果模型要遵循这些政策，我们就必须投入精力，确保这些规定得到落实。同样，我们也有一些项目，比如网络安全验证项目，以及最近新推出的生命科学验证项目。这些项目表明，你所使用的模型，其在线防护措施的运作方式会有所不同。20:19 And so we've started to have to have data retention policies so that we could do automated review and human review, whatever it needs to be, to make sure that good things are happening with the models. And with each model that we're releasing, like that's one example of a policy decision that needs to happen on like what will the policy be for this specific model. And so Opus 5.5 that launched yesterday, for example, does not have those data retention requirements that Fable has, right? But that's a decision that we have to make each time we're launching one of these models. And if it is going to have those policies, we're going to have to do the work to make sure that those things are enforced. Similarly, with programs we have, like we have cyber verification programs and now newly our life science verification program, which says, you know, like you have a difference in how the online safeguards are working for the model that you're using.

21:09 确保所有这些程序都已到位，也是其中一项工作。这大致体现了我们在每次模型发布过程中所做的工作。我现在提到的这些事，以前发布模型时我们都不需要考虑。如今大部分工作都在于：我们要采取什么措施来防止蒸馏？在线防护措施、离线防护措施，以及相关政策。一个很好的例子是，我们建立了一套系统，确保使用模型的授权，或使用那些可能放宽防护措施的程序的授权，能够贯穿整个系统，直至底层，而且整个过程都值得信赖。21:09 Making sure all those programs are in place is another one. And so it's just kind of a flavor of the type of work that we've done as part of each model launch that these are everything that I'm saying right now are not things that we used to have to think about when we were launching these models. And now it's the majority of the work is like, what are we doing to prevent distillation? The online safeguards, the offline safeguards, the policies. A really great example is we built a system to make sure that authorization for being allowed to use a model or being allowed to use one of these programs that give you maybe relaxed safeguards, that authorization threads all the way down through our system in a way that we can really trust.

21:47 这不是说，哦，你获得了 Mythos 的访问权限，于是我们就给你一个 API 密钥，打开一个功能开关，然后就完事了。我们有一套非常完善的系统来处理身份验证，确保如果你应该使用这个模型，那么只有你能以预期的方式使用它。所以，在模型发布过程中，开始出现很多复杂的事情，而这些事情，我们以前确实从来没考虑过。是的，而且这会随着能力的提升而扩大。随着我们的能力越来越先进，并且开始在某些特定领域突飞猛进，我们就需要格外关注这些领域。因此，我们进行的很多评估，都来自这些非常具体的能力：我们可能会认为它们更偏向进攻，或者更偏向防御，因此会针对这些能力采取不同的策略。当我们把这些考量层层落实，最终推进到模型发布时，就会有一些环节，正如 Katelyn 提到的，我们并不完全清楚人们可能会如何使用这些能力。21:47 It's not like, oh, you got access to Mythos, and so we throw you an API key and we flip a feature flag and call it a day. We have a really robust system on how we're doing auth and making sure that if you're supposed to be using that model, like, only you are using that model in the way that you're meant to be using it. So there's a lot of complex things that are starting to happen around model launches that, again, we just never used to think about them. Yeah, and it's scaling up with capability. Like, as our capabilities get more and more advanced and they start to spike, you know, in specific areas, that's where we need to pay, like, extra attention. And so a lot of that evaluation comes just from like those very particular capabilities that we might either see as like offense dominant or defense dominant and therefore might take different strategies for all of that. And as we bake that all the way through to eventually the model launch, there will be components kind of like Katelyn mentioned where we kind of don't exactly know how people might use those capabilities.

22:37 有些用途显而易见，比如编程，对吧？太棒了，去打造出色的产品，推出更多成果。这对你来说可能非常棒。而在生物和网络安全等领域，技术可能具有双重用途，也可能造成负面影响。但显然，你完全可以是一名合格的网络安全研究人员，致力于发现漏洞，做一名善意的参与者。同样，在生物领域，你也可能从事研究之类的工作。因此，很多这类项目，比如你们这里提到的这个，基本上就是为了让人们能够使用这些具有双重用途的技术。你确实需要说明自己的使用场景等信息。但我们希望让社区能够利用这些能力，看看大家还能用它们做出哪些了不起的事情。好，我们会把链接放出来。也许稍后我们再深入聊聊。22:37 There are some more obvious ones like doing coding, right? Like, great, go make awesome products, go ship more things. It can be really awesome for you. And then other areas like bio and cyber, it's like it could be dual use and there could be negative effects. But obviously, of course, you could be a really valid cybersecurity researcher. You are trying to find vulnerabilities. You are trying to be a good actor. Similarly with bio, you might be researching and stuff like that. And so a lot of these programs, like the one that you have up here, are to basically provide access to that dual use. You do have to specify your use case and things like that. But we do want to enable the community to be able to take advantage of these capabilities and see what amazing things that they can do with it as well. Well, we'll put up the links. Maybe we'll dive into them later.

23:15 还有一个是关于生物的。总体来说，我觉得你们确实认真考虑这些问题，这一点非常重要。大家会忘记，比如 Fable 曾经被撤回，还有这些蒸馏攻击，然后有人就会说：“啊，Fable 上没有零数据保留，这也太离谱了。”但你得记住，它后来被撤回了。还有蒸馏攻击。你们在发布时想得周全一些，并且用正确的方式处理，这很好。是的，其中很大一部分就是建立正确的框架。再说说边缘侧吧，我们得在系统中建立正确的框架，才能做到这一点。我之前稍微提过这个，比如我们现在在整个系统中构建的授权和信任授予机制。永远别说绝不。对，别把它当成先例。是啊。不过在平台工程中，这其实很常见：发生一次事故后，你就会针对这起事故做过度准备，然后这类事故就再也不会发生了。23:15 There's also one for bio. Also, I just think at a high level, you know, it's really important that you guys actually think about this stuff. People forget like Fable was rolled back and there's all this distillation and people are like, ah, zero data retention is not a thing on Fable crazy. But you have to remember like, you know, it got pulled back. There's distillation attacks. Like it's good you're overthinking and doing it in the right way while releasing it. Yeah. And a lot of that is like building the right framework. Just to, again, kind of talk about the edge side of it, building the right framework in our system to be able to do that. So I talked a little bit about this, like this idea of like authorization and trust grants that we've built throughout our system now. Never say never. Right. Not as precedent. Yeah. But also it's like a very common thing in platform engineering that you like you have one incident and then you like over prepare for the incident and the incident never happens again.

24:06 是啊。就像，你知道，现在你有那个按钮了。嗯，确实。而且你这么说挺有意思的，因为当时各种事情都在发生，我们团队的工作里就有这么一项：我们要部署一个系统，让大家都能通过一个按钮撤销访问权限。所以这挺有意思的，因为我们会想，我们做这项工作的理由，难道是希望这种事再也不会发生吗？不过我觉得在这种情况下，这其实就是把我们原本想要在各处部署的那些基础机制进一步扩展，用来帮助我们管理这些验证项目，以及 Mythos 之类模型的访问权限；也就是说，把同样的机制放在每个人的模型访问权限之下，这样万一哪天需要采取行动，我们就已经有合适的控制措施了。24:06 Yeah. It's just like, you know, now you have the button. Well, it's true. And it's funny you say that because it was like with everything going on, we then had this line item in the work that our team was doing that was we're going to put this system in place that makes it so that it could be a single button to pull back for everybody. And so it's funny because it's like, do we justify that we're doing this work because we hope that something like that never happens again? But I think in this case, it's really just a laddering up of those same primitives that we wanted to put in place everywhere to help us manage these verification programs and access to models like Mythos and otherwise to just say, like, just sit that same thing below, like, everybody's model access so that in case we ever needed to, we had the right controls in place.

24:48 是的。像这样的决策实际上已经证明了……具体到这个案例，我们还得看看。但像这样的例子，一直以来都促使我们不得不真正改变制定产品路线图的方式。因为你知道，我想在传统的产品管理中，你可以说，好吧，这些是我假设会发生的一系列事情。它们会有某种先后顺序、某种排列方式。但你是在做更具体的押注，而不是像……比如 Fable 的部署。我想我们谁都没想到，这竟然会成为一个切实可行的用例，竟然会被政府采用，而那位女士就不得不去做这件事。但显然，这种情况发生的概率并非为零。所以，从这种思考方式来看，实际上从产品管理的角度说，整个过程已经变得非常依赖概率了：我们该优先考虑这个吗？它发生的可能性有多大？24:48 Yeah. And decisions like this have actually proven to be, like, we'll see about this one in particular. But examples like this have been consistently the way that we've had to actually transform the way we develop our roadmap. Because, you know, in traditional product management, I guess, you can be like, okay, like, these are the set of things I hypothesized to happen. They will have, like, a certain sequence, a certain ordering. But you are picking bets in a way that are much more concrete versus, like, you know, the de-deployment of Fable, for example. I don't think any one of us would have guessed that that would have been, like, a valid use case that would have happened. by the government and the lady just like had to go do it. But clearly the probability is non-zero. And so from this form of thinking, it's become deeply probabilistic actually from like a product management standpoint of like, so should we prioritize this? What are the chances of it

25:32 这种情况还会再次发生吗？当我们看待这类事情时，我们会想，好吧，如果要设想最疯狂的一系列可能性，我们其实很难确定事情会朝哪个方向发展。所以我们确实需要做一些对冲，并建立一个多元化的押注组合。通常，我们会考虑这样的对冲情境：好吧，我们认为这不会再次发生。但如果真的发生了，我们就会希望有这样的对冲措施，或者至少具备妥善应对的能力。因此，我们会优先考虑这类事情。我们也越来越发现，采用这种产品开发策略后，大多数时候我们最终都会发现，实际上我们确实会用到这个东西。能不能再举一个不同的例子，让我更好理解？比如这个访问计划。25:32 ever happening again? When we look at things like that, we're like, okay, if we are to imagine the most crazy set of possibilities, we probably actually have very little certainty as to where things will go. And so we actually just kind of need to like hedge and build a portfolio of bets. And so more often than not, we will take hedging scenarios of like, OK, we don't think it would happen again. But in the event that it does, we would actually want that hedge or have the ability to be able to do that very well. And so we will prioritize things like that. And then we have increasingly found that more often than not, by taking that strategy of product development, most of the time we actually find ourselves being like, we actually do use the thing. Is there another example that's a separate one just so I can understand? For example, this access program.

26:15 这个 X-Factor 确实很不错。它内部包含一系列授权机制和类似的东西。我们专门开发它，就是为了能够……实际上把这类项目提供给客户，而且是非常有针对性的。你愿意简单介绍一下这个访问项目具体是什么吗？当然。如果你想要双重用途的话，比如说，我们以一个模型为例。你有一个模型，比如 Opus，想用它的先进网络能力进行研究，或者做任何你想做的事，比如作为防御方用户。你可以申请加入这个项目，说明你的使用场景；如果申请获批，我们就会降低你所用模型版本的安全防护措施，这样你就可以继续尝试将这些网络能力用于你的使用场景。当然，显然，26:15 This X-Factor is a really good one. Internally, it has a bunch of granting and all these kinds of things. We developed it specifically for reasons about being able to... actually give programs like this out to customers, like very, very specifically. Do you want to give a high level what the access program specifically is? Sure. So if you want dual use, if you want, so for, let's take a model. So you take a model, call it like Opus, and you want to be able to use advanced cyber capabilities on Opus for research or for whatever you want to go do, like your defensive user. You can apply to this program to be able to, with your use case, and if approved, what will do is actually diminish the safeguards on your version of the model so that you can then go ahead and try to take advance of those cyber activities for your use case. Now, obviously,

27:02 这需要针对每个使用场景进行理解。需要把范围缩小到某个特定用户，因为通常不会这么做。你得把它应用起来。甚至像这样的项目，我们设计了很多 Katelyn 提到的基础组件，就是为了能够处理这类事情。现在让我们感到意外的是，或者说，在采用组合式方案时，我们发现实际上可以利用它把事情做到 X。我们当时在设计时就在想：它应该是什么样的？应该有多通用？是不是不该太通用？是不是应该过于专注于实现这种验证风格的门户，再授予模型非常具体的能力？我们倾向于选择更通用的设计，因为我们当时想的是，我们也不知道。也许还能把它用在其他事情上，也许不能。结果后来发现，它对诸如访问模型之类的事情非常有用。27:02 that requires per-use case understanding. It requires scoping down to a particular user, because you wouldn't do that generally. You'd have to apply. And even things like this, like this kind of program, we designed a lot of the primitives that Katelyn was talking about to be able to handle things like this. Now, it was surprising to us, or maybe in the portfolio approach, we found that we were actually able to take this and make it X. We were like, when we're designing it, we're like, what kind of should it be? How generic should it be? Should it not be too generic? Should it be overly focused on making this verification, you know, style portal plus granting of very specific capabilities of the model? And we tended to opt for like more genericism because we were like, we don't know. Like maybe we can use this for other things, maybe not. And then it ended up being massively useful for things like, you know, access to the model

27:47 从更广泛的层面来看，用于管理我们的任何受信任访问计划、处理模型下线，因为我们在处理每一个基本构件时，实际上都采用了非常、非常通用的方式。从一般的产品角度来说，我通常可能会觉得，这可能不太合理。我们应该聚焦于具体用例。这么做有道理。但在这里，随着时间推移，正因为我们选择了更通用的方法，我们发现自己往往会越来越频繁地采用这个解决方案。也正因此，它带来的收益也越来越大。无论如何你都需要具备这个能力，而你只是有了一个促使你去实现它的契机。从技术角度来说，我来自 AWS 的世界，对吧？IAM、OVC，还有那些谈起这个就特别较真的人，他们会聊 Zanzibar 和其他各种技术架构。我不太熟悉“令牌授权”或类似“信任授权”这些说法。27:47 writ large, for managing any of our trusted access programs, for handling de-deployments of models, because we were actually very, very generic about how we handled every single primitive. From a normal product standpoint, I would normally probably be like, that probably doesn't make a ton of sense. We should focus on the use cases. That makes sense. But here, increasingly, because we choose the more generic approach, we find ourselves reaching for that solution more often than not. And it pays increasing dividends for that reason. It's something that you needed to have anyway, and you just had a forcing function to have it. On a technical basis, so I'm from the AWS world, right? IAMs, OVC, and then people who get really intense about this talk about Zanzibar and all these other technical architectures. I'm not familiar with the word token grants or like trust grants.

28:32 信任授权，是的。信任授权。普通的 OVC 和信任授权有什么区别吗？我的意思是，其实没什么区别。这是一种授权形式。是的，这是 IAM，不过是 Anthropic 的 IAM。对，差不多。我想确实如此。我们有这种授权：比如，你向某个 API 发起请求，而该 API 在底层会访问一些数据，我们要确保你访问的是属于你的数据。对吧？这其实就是非常、非常基础的授权。我们只是建立了一种数据模型，非常清晰地规定：信任授权意味着你获得了访问某个特定模型的权限，或者获得了访问某个特定程序的权限，诸如此类——有很多不同的变体。我们发现自己经常会用到它。还有一个很好的例子是，当我们……28:32 Trust grants, yeah. Trust grants. Is there a difference between normal OVC and what trust grants are? I mean, not really. It's a form of authorization. Yeah, it's IAM, but anthropics IAM. Yeah, kind of. I think so really we have like there's like authorization of the form of like, OK, you're you made a request to an API that like under the hood is going to access some data and we're making sure it's like your data that you're accessing. Right. Like really, really vanilla authorization. This is just like we put a data model in place that's like a very clean like a trust grant means like you have been given access to some specific model or you've been given access to some specific program or, you know, like there's a bunch of different permutations. of this, we actually find ourselves reaching for it all of the time. Another good one was when we

29:17 我们最初在 Fable 5.1 中推出了一部分这种保留的思路时，对一些用户采取了祖父条款式的处理，给了他们一点时间——宽限期。对，给他们一个宽限期去调整集成。作为祖父条款式处理的一部分，我们说，好吧，我们得追踪哪些人是现有客户，需要额外的时间，对吧？然后我们发放了信任授权。这样一来，我们就可以做到：对于任何操作，我们只需运行一项服务，它处在请求处理的关键路径上，以确保请求顺利通过，并检查你是否拥有执行所尝试操作所需的授权。这其实和普通的授权机制没什么不同。但我们还是得搭建一个系统……29:17 first rolled out some of this preserved thinking for Fable 5.1, we kind of grandfathered in some people and gave a bit of a, you know, just like a time. Grace period. Yeah, grace period to go and make the integration changes. And as part of grandfathering in, we said, okay, we have to like track who is like an existing customer who needs the extra time, right? And then we did a trust grant. And so then we're able to say, for anything, we're just running this service that is kind of on the hot path of our requests to make sure that, you know, we just like flow through and check whether you have the grant for the thing that you're trying to do, which is it's really not any different than like vanilla authorization, really. But like, we just had to spin up a system

30:02 这在我们的系统中是特定的，也就是说我们需要能够插入这些检查。是啊，我觉得这些都是工程方面的事情，人们多少会觉得理所当然，不太会去想。还有什么平台方面的例子呢？比如说为 Claude Code 提供服务，或者显然我经常和 Cognition 沟通，但也包括其他客户；具体来说，就是推出一个模型，对吧？我们之后会更详细地谈安全方面的工作。我只是想先从高层次了解一下，因为你应该还记忆犹新：推出一个模型都要做些什么？如今这已经有点像一套流水线了，先不算安全方面的工作——我觉得那部分更复杂。但除此之外，还要大量收集客户的反馈。所以，你知道，我们先把模型做出来，30:02 that was specific to our system and like where we needed to be able to insert these checks. Yeah, I mean, I think these are all like the engineering things that people kind of take for granted, don't really think about. What is another example, I guess, of the platform stuff that had to happen, you know, maybe serving Claude Code, maybe serving, you know, obviously I talk a lot with cognition, but other customers just in terms of a model launch, right? Like we'll talk more about the safety work that is being happened. I just want to sort of go at a high level because it's fresh off your memory, like what goes into a model launch? It's become a bit of a machine these days, minus the safety stuff, which I think is the more complicated part. But outside of those pieces, there is a lot of getting feedback from our customers. So, you know, we make the model,

30:50 有很多安全测试，也会评估我们认为它需要达到什么水平。我们会根据需要做出必要的权衡和决策。然后还得进行测试。我们会推出模型的各种版本，对它们进行评估，并进行大量内部试用，我们称之为“内部试用”（dogfooding）。这实际上总是很有帮助。比如，我们能很早就发现一些问题，比如“哦，这可能不是最佳方案”。显然，我们在内部工作中也大量使用 Claude，因此能够及早发现一些信号。这有助于我们判断哪些变体可能优于其他变体。当我们对某一组变体很有信心时，就会为客户推出一个版本。有时，这个版本会通过我们的第一方产品发布，比如 Claude Code、CodeWork 等。30:50 there's a lot of safety testing. There's also evaluation of where we think it needs to be. We make the required trade-offs and decisions as necessary. And then we have to go through testing. We'll cut various versions of the model. We'll evaluate it. We'll do a lot of internal ant fooding, as we call it. And that actually always really helps. Like, we catch, like, really early things like, oh, this is maybe not the best. And we obviously use Cloud a lot for our work internally. So we're able to actually see early signals around stuff. And it helps us detect which variants might be better than others. And then when we feel really confident with a certain set of variants, we will cut a version for our customers. Sometimes that will go through our first-party products, like through Claude Code or CodeWork or things like that.

31:30 在其他情况下，我们会通过 API 客户来推进。实际上，我们会和这些客户开展某种早期访问计划。他们需要对自己的 harness 做出修改。我们已经和很多客户合作过，比如 Cognition、Cursor 这些团队。我们也有一套流程来处理这件事，这样大家都能获得我们需要的反馈。他们的评测效果好吗？喜欢吗？不喜欢吗？有些反馈比较偏主观感受，我们也想听听这些；有些则是直接看评测结果，比如它在你们关心的那些方面表现好吗？我们会综合考虑所有这些反馈。等我们对这些不同方面都满意之后，就会发布模型。现在，模型反馈这件事感觉还是很缺乏条理。基本上，公司之间总是开个 Slack 频道，有人说：“嘿，这东西好像有点奇怪。”然后其他人就说：“好吧，那我来修一下，估计。”31:30 In other cases, it will go through our API customers. And we'll engage in kind of like an early access program effectively with those customers. They'll need to make changes into their harness. And we've engaged with a lot of customers like Cognition, like Cursor, like these folks. And we have a bit of a process on how we handle that so that we all kind of get, they all get the feedback that we need. Does it perform well in your evals? Do you like it? Don't you like? Some things are vibey and we like to hear those things too. And some things are just like straight up evals, like did it do well on the things that you care about? And we take that all together. And once we've all feel good about like all those different pieces, we launch the model. The state of model feedback feels very unstructured still. Basically, it's always a Slack channel between companies of like, hey, this thing seems weird. And then people are like, okay, I'll fix it, I guess.

32:16 给我个 ID 什么的。是啊，这确实很……我的意思是，这就是非确定性产品的特性，对吧？不过我觉得，正如你说的，真正参与我们早期体验计划的客户，都是邮件往来非常充分的客户。我确实觉得很多事情都挺凭感觉的，很多交流也往往是临时起意、凭感觉进行的。部分原因是我们在争分夺秒，对吧？我们想尽可能缩短模型版本完成测试、到让所有人都能使用这个模型之间的时间。所以时间窗口真的非常紧。不过，我们通常会和这样的客户合作：我们知道他们会启动整套评测流程，然后回来给我们非常具体、可执行的反馈。32:16 Give me an ID or something. Yeah, it's very, I mean, it's the nature of like a product that's like non-deterministic, right? But I think to your point, the only customers really that are part of our early access programs are customers that have really robust emails. Like, I do think a lot of this is vibes-based, and I think a lot of the conversations tend to be kind of, like, vibes-based and on the fly. And part of that is because we're, like, racing, right? It's, like, we want to minimize as much as possible the window between we have a version of the model that we've cut for testing and we give everyone access to that model. And so it is just, like, this really tight window. But the customers we tend to do it with are the ones who, like, we know they're going to go, like, kick off their whole eval suite and be able to come back to us with, like, really actionable feedback.

33:01 尽可能快地，同时结合直觉、基于直觉的反馈。当然，我们确实可以采取一些流程方面的措施，让这些东西随着时间推移变得更好。不过，我是说，你可能会感到意外——当然，你大概不会，但我觉得有些人会意外地发现，世界上真正具备智能体能力、却没有完善评测来帮助测试这些模型的产品，少得可怜。两年前我们请 Boris 来播客时，他说我们还没有评测代码。我是说，这已经变了。确实变了。不过，你知道，很多时候，比如说，你的用户就是你的评测，但你没时间把它们规范化，或者你缺乏经验，或者其他什么原因。这都没关系。等你认真起来，总会有评测的。没错，完全同意。我觉得你们做得很好的一点是，正如你们所说，你们会大量进行 Amp 的内部试用。所以大家常说，你们会在产品发布前自己做很多测试，对吧？33:01 as quickly as possible alongside the vibes, the vibes-based feedback. So, of course, there's definitely process-y things that we could do to make these things better over time. But, I mean, you'd be surprised. I mean, you probably wouldn't be surprised, but I think some people would be surprised about how few products that exist in the world that are agentic that don't have good evals to be able to help test these models. We had Boris on the pod two years ago saying we don't have the also clock code. I mean, that's changed. That has changed. That has changed. But, you know, like, yeah, a lot of things like, you know, your evals are your users and you don't have time to formalize them or you don't have the experience or whatever. And that's fine. At some point when you get serious, you'll have evals. Yeah, totally. I think the good thing is you guys do a lot of amp fooding, as you say. So it's commonly said that you find a lot of this testing it yourself before it comes out, right?

33:49 是的，是的。尤其是客户的 API 非常有用，因为他们会把 Cloud 放到不同的框架中。我们希望 Cloud 非常、非常通用，所以它能在其他框架中运行，对我们来说相当重要。因此，这类反馈往往对我们特别有用，因为它能带来我们从 AntFooting 那里得不到的新信息。什么样的反馈有帮助？是像回归问题这样的反馈吗？比如这里出现了回归问题，或者我们在这里发现了一些优点。好的方面总是很能验证我们的想法。我觉得我们经常不会感到意外，但我们希望通过观察来验证一些我们可能提出的假设。说实话，最有用的反馈是负面的。比如，你知道，它在这方面表现不好，在那方面也表现不好。我们很喜欢这样的反馈。然后我们就会想：太好了，我们该怎么改进？33:49 Yeah. Yeah. The APIs of customers in particular are very useful because they'll put Cloud into a different harness. And we want Cloud to be very, very general. So its ability to work in other harnesses is quite important to us. And so that style of feedback tends to be exceptionally useful for us because it's novel from what we would have gotten from AntFooting. What kind of feedback helps? Is it like regressions? We have regressions here. We notice good attributes here. The good stuff is always very validating. I think we're very frequently not surprised, but we like to look to be validated on some of those things that we might hypotheses. Honestly, the most useful feedback is negative. Like, you know, it didn't do well at this. It didn't do well at that. We love that feedback. And then we're like, great, how do we fix it?

34:31 是的，这往往就是那些很棘手的问题之一，因为并不是每个人都希望 Claude 在每条光谱上都恰好停在同一个位置。一个很好的例子可能是懒惰与积极主动、采取行动之间的取舍。有时候，我们有一个模型，用户会说：“把我的任务做完。别这么懒。做得好一点。”而另一些时候，用户又会说：“嘿，我们还只是在摸索和规划呢。”结果你却跑去写了一大堆代码，或者采取了一大堆我还没准备好让你采取的行动。要把握得恰到好处实际上是不可能的，因为现实中每个人想要的都不一样。不过，我确实认为，我们曾经遇到过这样的情况：我们推出了一个早期版本的模型，发现它显得太懒了。对。34:31 Yeah, and it tends to be some of these things that are like, it's really hard because not everyone actually wants like Claude to land at the exact same spot on every spectrum. And maybe a good example is like, you know, laziness versus proactivity and taking action. And so there's sometimes when like we have a model and people are like, like, just finish my task. Like, stop being lazy. Like, do better. And other times when it's like, hey, we were like just kind of feeling this out and planning. And then you like went and wrote a bunch of code or you went and like took a whole bunch of action that I wasn't ready for. And getting that exactly right is literally impossible because everybody actually wants something different in reality. But I do think we've had situations where we've cut an early version of the model. We found that we saw too much laziness. Right.

35:15 就像我们会把它放回烤箱里再烤一会儿。对，然后试着把那种行为重新训练掉。所以，如果我们只是在自己的产品和 API 内部试用，就可能发现不了这类问题。对我们来说，更重要得多的是把它放到 API 上，让所有其他客户都用起来，再告诉我们需要改些什么。是的，这是个核心问题。我觉得，如果换作是你，给模型做产品管理时最难的事情之一，大概就是如何在对齐和个性化之间取得平衡。你希望所有用户都能获得一种默认的个性和默认行为，同时又希望能够迅速响应某个人在 CLAUDE.md 里写下的要求，比如：“不不不，请比默认设定更主动一些。”35:15 And like we like put it back in the oven for a bit. Right. And tried to train that back out. And so those are the sorts of things that we may not have caught if all we're doing is internally antfooding within our products and within our API. It's way, way more important for us to get that out there in the API and get all of our other customers running with it and let us know what we need to change. Yeah, it's a core question. I think it's probably the, if I were in your shoes, like one of the hardest things about PMing for a model, which is, let's call it alignment versus personalization. There's a default personality, default behavior that you want for all consumers. And then you want to be able to respond very quickly to someone's CLAUDE.md saying, no, no, no, like, please be more proactive than default.

36:01 然后，就我所知，你知道，我也参与了 Frontier Code。我们参与了 Fable 的发布，这让 Frontier Code 声名鹊起。现在 Opus 5.5 又回到了我们的模型基准测试中。这是 Frontier Code 非常重要的一部分，对吧？Frontier Code 的模型曲线看起来很奇怪，是因为它会对过多的操作或过多的文件进行惩罚，而这其实是件好事。比如说，Frontier Code 的整个……设计初衷就是如此。但我完全可以直接在我的 CLAUDE.md 里写一句“不要这么做”，那它就不会这么做了。所以，我是不是因为基准测试里没有这种个性化因素，而不公平地惩罚了 Cloud Opus 55？而个性化又是主观的。所以我也不知道这个问题该怎么问，但你们是如何处理个性化的？显然，一个模型可以做很多事情……36:01 And then, like, to my knowledge, you know, and I was involved in Frontier Code. We were involved in the Fable launch, which put Frontier Code on the map. And now Opus 5.5, we have it back on the model benchmark. That is a very big part of Frontier Code, right? The reason that the model curve looks weird for Frontier Code is because it penalizes too much activity or too much files, which is good. Like, there was a whole... design purpose of Frontier Code. But I could easily just be like, in my CLAUDE.md, just say, don't do that. And then it won't do it. And so am I penalizing Cloud Opus 55 unfairly because my benchmark doesn't have that personalization element, which is subjective? So I don't know what the question is, but how do you deal with personalization? Clearly, a model can do so many

36:51 各种情况都不一样，而且这非常因人而异，每个人都有自己的 CloudMD，你们要怎么让它不断演进？我们这方面做得并不好。说真的，这真的很难。非常难，而且它们之间彼此竞争、相互冲突。我记得有个云端宪章，我想是吧。那是很久以前的事了。确实有一份宪章，不过还是让你们来回答吧。宪章是有帮助的，但像这种情况，要让它根据各不相同的指令和千差万别的偏好进行个性化调整，对我们来说非常难。一般来说，我们会尝试通过增加更多参数来解决这个问题，从而可以对它进行微调。36:51 different things and it's so dependent, everyone has their own CloudMD, how do you evolve that? We don't do a great job of it. Seriously, it's really hard. It's super hard and they're competing and contrasting. I thought there's the cloud constitution, I think. That's old. There is a constitution, but I'll let you guys answer. The constitution's helpful, but for things like this on how it personalizes to instructions which are different and preferences which are very, very different across these things is very hard for us to do. We generally try to solve this by having more parameters that then allow us to tweak it

37:32 根据客户群来做。如果我们无法控制你的客户群，而你是 API 客户，那么我们会尝试给你一些建议，然后由你来调整。因此，有些事情，比如说，关于……这和针对各种 cloud.mds 或现在的 agents.mds 等进行个性化，程度并不完全相同。但我们确实会在投入程度这一层面上提供一些建议，比如你应该投入多少精力、需要达到什么程度，或者你可能需要关注的模型规模，诸如此类。但我认为，这还不足以满足我们对那种个性化程度的实际需求。所以这是个非常棘手的问题。现在，我们解决这个问题的很多方式，已经超出了参数层面——不只是给某个东西提供多个参数——而是试着展现相互冲突的偏好，说实话。37:32 per customer base. And if we don't control your customer base and you're an API customer, then we try to give you recommendations and then you try to do that tweaking. So there are things like, you know, around, it's not exactly the same level like personalization to the various cloud.mds or now agents.mds or whatever. but we do try to give things around the layer of like level of effort, like how much effort you should put in, like what are the level, like maybe the size of the model that you might actually try to care about, things like that. But I think it's not quite there for what we would actually need for like that level of personalization. So it's a very hard problem. And right now a lot of our way to solve that is beyond the parameter, like the giving multiple parameters to something, is try to exhibit conflicting taste, honestly,

38:15 我们会综合考虑收到的所有反馈，并尝试对其中一些问题作出判断。我觉得随着时间推移，尤其是客户和用户规模不断扩大，这种做法其实很难扩展。我们需要开发一些方式，让 Claude 能够纳入你的个性化设置。也许你可以提供某种信号，或者调整某些参数之类的。但我认为，我们得采用一种更侧重于这个方向的方案。不过目前，我们确实只能对很多事情自行判断，但这相当棘手。情况就是这样。补充一点：我想默认的推理力度改了，现在是中等。这大概会影响那些没有使用这个切换选项的普通用户。对，这算是 Claude Code 的一项设置，对吧？这是 Claude Code 的一项设置，默认值改成了中等。38:15 across like all of the feedback that we get and try to make a judgment call on some of these things. I think over time, they actually, and as you scale out more customers and more users, that's not a very scalable way to go do something. And we will need to develop ways for Claude to kind of take in your personalization. And maybe that's a signal you can provide or something you could tune on or things like that. But I think we'll have to take a route that's much more oriented towards that direction. But for now, yeah, we actually just try to make judgment calls on a lot of the stuff, but it's pretty tough. It is what it is. A small note there. I guess the default effort reasoning changed. It's now medium. This is one of those things that probably affects the average customer that doesn't use this toggle. Yeah, it's like a Claude Code setting, right? It's a Claude Code setting that defaulted to medium.

39:00 是的，他们只是……而且这个东西会有所变化。它确实会……这是一个很好的例子，说明我们会用某个参数来进行一定程度的个性化。我会说，这个说法可能有点过于强硬，但我们会针对不同的细分用户群体做一些定制。比如，默认设置会根据你是免费用户还是专业消费者用户，或者你使用的是 Claude Code、Cowork 还是 Enterprise，以及 API 等情况进行调整。你当然随时都可以自行调整，不过这些算是我们尝试提供的一些轻量级方式。有一点是我们不太愿意做的：我们不希望用户觉得自己无法控制 Claude 的个性化设置。比如说，我们可以在这些方面采取更强硬的做法，但我们希望用户能够根据自己的需要，选择体验 Claude 的不同智能水平。尤其是，如果你想尝试更高的水平，或者有时想试试较低的水平——如果这对你的使用场景来说确实更快的话——都可以。这在产品设计上是个很难权衡的问题，我觉得。39:00 Yeah, they just... And this thing varies. This thing does... This is a perfect example of a parameter that we use to kind of personalize. I would say this may be a little too aggressive of a term, but we kind of customize maybe for different subsegments. And so, like, the default setting will adjust for whether you're a free customer or a prosumer customer, or Claude Code versus Cowork versus Enterprise, like things like that, versus API and stuff like that. Knowing that you can obviously always adjust it, but those are kind of like light ways where we try to give that. One thing that we are hesitant to do is we don't want people to feel like they can't control their customization of Cloud. So we could take a heavier hand on these things, for example, But we want to give users the ability to experience the intelligence levels of Cloud as they kind of see fit, especially if you want to go towards, like, the higher end or you want to experiment with maybe sometimes in the lower end if that actually is, like, faster for your use case, which is a hard product tradeoff, I think.

39:53 所以我们往往倾向于采用比较温和的默认设置，同时给你一些选项来调整。我觉得目前这样还算可以。但我想，随着时间推移，我们会逐渐学会如何更好地为客户做到这一点。我们现在还不能说对这些做法特别特别满意，但我们知道，正如你问的，推出一个模型需要做些什么？显然，我们要做的一件大事就是：好，Claude 在某件事上表现不错。有时候，这件事是我们应该把它作为 API 上的一个参数开放出来，让你来控制的。但我觉得，随着时间推移，这套做法逐渐演变成了一种有点奇怪的混搭：有 effort、有 budgets，还有各种你可以设置的东西。39:53 And so we tend to lean towards gentle defaults but give you options to adjust, which I think works okay-ish for now. But I imagine us, you know, like, learning how to do that better over time for customers. We're not like super, super happy with any of this yet, but we know as we launch a model, like you asked, like, what does it take to launch a model? One of the big things obviously we have to do is like, okay, Claude like got good at a thing. And like, sometimes that thing is something we should expose as a parameter on the API to let you control. But I think as this has evolved over time, we've ended up with like this funny mishmash of like, there's effort, there's budgets, there's all these things that you can set

在 API 上的 40:31。问题是，任何人怎么可能知道针对某项具体任务，应该设置哪些东西，以及它们的正确组合是什么？这件事我们也还在边做边摸索，和客户一起摸索：随着时间推移，哪些参数应该弃用？哪些参数应该直接设置得更好？哪些参数应该采用 1 到 100 的刻度，而不是低、中、高这样的刻度？因为实际上，我们已经训练 Claude，让它非常擅长理解其中某些选项的整个范围。但我们要把数以百万、数以十亿计的各种选项排列组合全都开放出来吗？那样你其实很难取得什么进展。这实际上也是 Claude Managed Agent 这类产品的价值主张之一：它仍然能通过 API 以编程方式访问 Claude。40:31 on the API. And it's like, how could anyone like possibly figure out what is the right combination of things for any given tasks for what you should go and set? And it's something that this is us just very much like figuring it out as we go and figuring it out with customers and figuring out like which of those parameters do we deprecate over time? Which of those parameters do we just like default better? Which of those parameters should be like a one to a hundred scale versus a low, medium, high scale, right? Like, because actually we've trained Claude to get really good at understanding like a whole spectrum of options on some of those sorts of things. But like, do we expose the like entirety of those millions and billions of permutations of options? Like, you wouldn't really get very far. And that's one of the things actually that is part of the value prop with something like Claude Managed Agent, which is like still programmatic access to Claude in an API.

41:19 不过你会直接获得一个开箱即用的 harness。每次我们发布模型时，我们都会花很多功夫去弄清楚有哪些新参数、有哪些不兼容变更，以及如何在我们的 harness 中充分发挥 Claude 的能力。这样一来，使用它的人开箱就能用，不用费心考虑这些。尤其是那些对此感兴趣的人——而且这类人群规模不小，还在不断扩大。我们其实还在 AIE 办了一整场 CMA 工作坊，我觉得办得非常好。有没有单独的订阅源，供大家订阅 CMA 的更新？比如……像 Twitter 动态、新闻简报之类的，我也不太确定。没有，不过我觉得我们的 Cloud Devs 频道会经常谈到这个，因为这对大家来说确实是个挺有意思的选择。41:19 but you get a harness out of the box where we do, every time we're launching the model, a lot of the work to go figure out, like, what are the new parameters? What are the breaking changes? How do we get the best out of Claude within our harness? So that for the people who are using it, you just kind of get that out of the box. You don't have to think about it. Just for people who are drawn to that, and which there's a significant and growing population, particularly we actually did a whole CMA workshop at AIE, which I thought was really well done. Is there a separate feed for people to subscribe to, like, updates on CMA? Feed, like... Like a Twitter feed, newsletter, I don't know. No, I think actually our Cloud Devs channel, we are, yeah, we're pretty loud about it because it is, like, an interesting option for people.

42:02 是的，我只是想告诉大家，如果你就是这样使用 Cloud 的，那这就是一条可行的路径。我觉得它也挺被低估的。对于那些不是这样使用 Cloud 的人来说，Claude Managed Agents 是什么？大家为什么要用它？哦，对。我们大概应该简单介绍一下。我不知道。那是别人家。嗯，简单说两句。我觉得它刚推出时，很多人都没太在意，对吧？那时候大家都处在这样的阶段：我得做 harness engineering，我得控制这一切。你听到的都是这些。现在的问题是，你真的需要做这么多吗？还是直接用托管智能体，就能省下很多设置工作？不过，简单说三句吧。嗯，这取决于你的使用场景。Claude Managed Agents 就是我们现在提供的 API。42:02 Yeah, I just want to give people, like, if that's your way of consuming Cloud, this is the path. I think it's also pretty underrated. For those that aren't consuming Cloud this way, what is Claude Managed Agents? Why should people use it? Oh, yeah. We should probably do a little intro. I don't know. That's somebody's house. Yeah. A little two-sentence thing. I think a lot of people glanced over it when it came out, right? Everyone was in this phase of, I need to do harness engineering. I need to control this. And you hear all that. And now it's like, do you need to do all that? Or do you just use managed agents and you have a lot of info you can skip over setting up? But give the quick three. Yeah. It depends on your use case. So Claude Managed Agents is our API today.

42:47 我们有 Messages API，它本质上提供的是原始 Token 访问。然后我们有 Claude Managed Agents，它是一套 API，可以让你定义一个智能体，并与该智能体启动会话。它本质上是一种更高层次的抽象，让你无需设置所有基础设施、搭建运行框架，也不必处理许多这类工作。对于希望通过编程方式运行智能体的人来说，它是一种更开箱即用的集成方式。CMA 和 CloudTag 共用基础设施吗？因为它的推出顺序让人觉得，哦，这就像是 CloudTag。确实有很多共用。很多。所以我们做的一件大事就是，作为平台团队，我们负责运行 API，也负责运行所有基础产品基础设施和产品服务。42:47 We have the Messages API, which is essentially raw token access. Then we have Claude Managed Agents, which is a suite of APIs, but it lets you define an agent, start a session with that agent. And it's essentially just kind of a higher order abstraction that lets you skip setting up all the infrastructure, setting up your harness, and doing a lot of that type of work. It's a much more out-of-the-box integration for people who want to programmatically run agents. Does CMA share infrastructure with CloudTag? Because the sequencing of it seemed very like, oh, like that's the cloud tag. A lot of it. A lot of it. So one of the big things that we do is as the platform team, we run our APIs. We also kind of run all of our foundational product infrastructure and product services.

43:32 一个很好的例子是，我们的团队负责运行沙箱集群，为 Claude Managed Agents 和 Cloud Tag 等产品提供支持。但在这两款产品的实际执行框架层面，我们可能希望在框架中实现的功能有所不同，因此它们并不共用同一个执行框架，不过它们共用了大量支撑执行和 API 的基础设施。我觉得这确实……大家应该多用这种做法。我觉得你们推出 Managed Agents 后，基本上所有人都做了它的山寨版，毕竟模仿是最真诚的奉承。既然聊到这个话题，我们也简单谈谈最近一直在进行的记忆相关工作。我感觉这曾是 Anthropic 上半年工作中的重要部分。现在停滞了吗？还在继续吗？情况怎么样？43:32 So a really good example is like our team runs our sandboxing fleet that backs both Claude Managed Agents as well as products like Cloud Tag and otherwise. But at the actual harness layer for those two products, there's just differences in what we might want to do within the harness so that they don't actually share a harness, but they share a lot of the infrastructure that powers the execution and powers the APIs. I think that's really... People should use this more. I think everyone basically had copycat clones of Managed Agents after you guys launched it, which imitation is a sincere form of flattery. Since we're on this topic, we'll briefly mention all the memory stuff that has been going on. I feel like it was a big part of Anthropic work in the first half of the year. Has it stalled out? Is it still ongoing? What's

44:20 这里现在的重点是什么？具体来说，是托管式智能体。记忆、梦境。哦，这确实是我们正在做的很大一部分。我觉得也许可以这样理解：我们推出托管式智能体，是作为一个平台，面向那些不想自己做完所有这些事情的人，对吧？或者只是想把这些事情外包出去的人。比如说，我之所以能从 Claude 身上获得最佳效果，是因为我们的团队真的就坐在这里，对所有这些东西做评测，并确保智能体运行框架足够强大。记忆和梦境，关注的就是如何确保你的智能体能够随着时间推移不断改进？而这些仍然是我们团队的重要关注点。但我实际想说的是，回到我们之前谈到的这一点，随着从安全角度来看发布模型变得越来越困难，我认为我们团队投入的精力正越来越少地……44:20 the emphasis here now? Managed agents specifically. Memory, dreaming. Oh, it is a big part of what we're doing. I think maybe the way to think about it is we launched managed agents as a platform for people who didn't want to do all these things themselves, right? Or just wanted to like offload, like I'm getting the best out of Claude because our team is literally sitting here like evaling all these things and making sure that the harness is going to be really powerful. Memory and dreaming is all about just like how do you make sure your agent can continuously improve over time? And all of that is still an important focus for our team. But what I would actually say is to the point of what we talked about earlier, as launching models has become harder and harder from a safety perspective, I think most of our team's energy is going less

45:04 与其把重点放在“给这个 agent 加个好玩的功能，比如爬山之类的”，不如多想想怎么把安全性融入所有这些东西。因为我们能做的最重要的事，就是让更多人用上模型的能力，同时确保安全。所以我想说，我们投入了很多精力，把这些相同的理念应用到 agent 系统中，打造一个既强大又安全的系统。这就是我们投入精力的方向。和今年早些时候相比，我觉得我们在技能、dreaming、记忆，以及这类即时注入功能上的进展放慢了一些。请喜欢技能。——不不不，我们喜欢技能。我们喜欢技能。我们正在更新。我不是说我们什么都没做，我们也很喜欢技能。但没错，相比其他一些工作，我们在这方面的进展确实稍微放慢了一点。45:04 into like, here's a fun feature to like hill climb on this agent or whatever, and much more into how do we bring safety into all these things? Because the most important thing that we can do is get the power of the model out there and get it out there safely. And so I would say a lot of our energy is take all these same concepts and give you an agent system that's going to be really powerful, but also that's going to be really safe. And so that's where we're putting our energy. I think compared to maybe earlier this year, we have slowed down on like skills, dreaming, memory, like these kind of like moment in time injections. Please love skills. No, no, no. We love skills. We love skills. We are making updates. I'm not saying that we're not doing anything, but we love skills as well. But yes, I think we've slowed down a little bit comparatively for some of the other efforts.

45:48 但我们非常看好人们在 harness 层进行某种程度的爬坡优化，来实现你知道的、之前提到的一些做法。我们自己做了很多这方面的工作，但我们提供的 harness 从某种意义上说非常简单。这是刻意为之的。我们不想过于固执己见。我们希望把最佳实践整合进来，同时也提供一些难以区分的组件，用来构建最棘手的智能体循环——这些东西确实很难串联起来。而且，你也可以直接用样板代码把它搭出来。不过，除此之外有意思的是，一旦你有了这个基础层，我们还会构建一些爬坡优化策略。比如，Dreaming 实际上就是其中之一。所以，Dreaming……其底层是一个多智能体系统，我们是在 CMA 的基础上通过爬坡优化构建出来的，它实际上是一个……45:48 But we are very bullish on people kind of doing some level of hill climbing at the harness layer to, you know, some of the previous conversation here. We do a lot of this ourselves, but the harness that we provide is very simple in a way. It's intentionally simple. We don't want to be actually overly opinionated. We want to make sure the best practices get merged in and there's some kind of undifferentiated pieces of the hardest agentic loop that, yeah, are hard to string together. And also, like, you can kind of just boilerplate do it. Now, the interesting part on top of that, though, is, like, once you get that base layer, there are these, like, hill climbing strategies that we construct. And, like, dreaming is actually one of them. So dreaming... under the hood is a multi-agent system that we hill climbed on top of CMA and it's actually a

46:30 基本上就是 CMA 套在自己上面。CMA 套在自己上面。它有点像是再启动一个，是的。Dreaming 的工作方式是，我们会启动另一个托管代理，然后它可以访问前一个托管代理，并向那个托管代理写入记忆。所以一路往下都是托管代理。这在设计上多少有点特别，因为我们设想你大概可以把很多这样的模块化组件组合起来，做出一些有意思的东西。Dreaming 只是我们尝试实现的更有意思的东西之一。但我们一直以来的想法是：实际上还有很多其他事情可以做。你可以把 best of N 作为一种多代理策略来实现。你可以做一个验证器代理，用来处理类似目标或结果的事情。里面还有很多其他玩法。我们见过有人做出相当奇特、疯狂的东西。比如有个运行框架，会在所有东西之间共享大量记忆。他们会启动成吨的并行代理。46:30 CMA on top of itself, basically. CMA on top of itself. It like spins up another, yeah. The way Dreaming works is that we spin up another managed agent that then can expect the previous managed agent and then write memories to that managed agent. So it's managed agents all the way down. And this is a bit of a thing by design because we envisioned that you could probably put together a lot of these modular components for something interesting. And Dreaming is just like an example one of the more interesting things that we were going after. But we kind of have been looking at this as being like, there's actually a ton of other stuff you can do. You can do like best of N as a type of like multi-agent strategy. You can do a verifier agent, that kind of like goal thing or outcome thing. There's a bunch of other stuff in there. And we've seen people do some pretty funky, crazy stuff. Like there's been a harness that like shares like crazy memory across everything. And they spin up like tons of parallel agents.

47:16 我们当时想，或许我们其实应该让你也能这么做。就像我们构建 Dreaming 时做的那样。它可以说是一路管理到底层代理。我们很快会推出一次更新，升级底层原语，并将其开放出来，让用户能够构建自己的编排策略，或者不管你想怎么称呼这些东西，比如多代理策略。通过这些策略，你可以构建自己的版本。我们还会提供一些模板，比如我们最初是如何构建 Dreaming 的。作为开发者，你或许会想在此基础上做些更新，从中获得更多价值。对，我觉得这个说法特别到位。并不是说我们在这类事情上放慢了脚步，而是说，我们已经准备好了这些基础原语。我们可以继续让大家拿去用、自由发挥，同时也提供一些工具，比如一些很酷的多代理策略，以及我们加入 Cloud Manage Agents 的各种功能，让它变得非常强大。47:16 And we're like, we should actually just probably enable you to do that. In the same way that we did, we built Dreaming. That's like manage agents all the way down. There's an update that we'll make relatively soon where we're updating the underlying primitive. We'll expose it to allow users to kind of like construct harness strategies maybe or however you want to call this stuff. Like multi-agent strategies. Multi-agent strategies that allow you to kind of build your own version of this. And we'll give you like, you know, templates of like, this is how we started with Dreaming. Maybe there's like an update to this that you would want to make as a developer and you would find some more value on top of it. Yeah, I think that's a really good framing. It's less that we're slowing down on this sort of stuff and more saying, like, we've got the primitives in place. We can continue to let people just, like, take and run with those things while giving them the tools to say, here's some, like, cool multi-agent strategies and, like, things that we've put into Cloud Manage Agents to make it really powerful.

48:05 现在你可以让 Claude 利用这些基础组件来完成工作，让它自己变得更强大，比如创建新的多智能体策略之类的。与此同时，我们团队的大部分精力会转向社区这边：好，那就放手去做吧，对吧？你们尽管发挥，我们则继续专注于如何确保你能安全地使用该系统中的模型。你刚才提到，你们会对这些东西做很多评测。这可是现在很热门的话题。有什么建议或最佳实践吗？专门说评测的话？针对运行框架、智能体的评测。有很多不同类型的评测，对吧？不同的评测并不都适用于你所处的各种情境，也不一定都适合用来测试你想测试的东西。48:05 now you can like let Claude do work to take advantage of those primitives and make it even more powerful yourself, like create new multi-agent strategies and whatever it might be. While a lot of our team's attention turns to, okay, great, like community, run with that, right? And you do you while we continue to focus on what are all the things that we need to do to make sure that you get access to the models within that system safely. One thing you brought up was you run a lot of evals on this stuff. Pretty hot topic right now. Any advice, best practices? For evals specifically? Evals on harnesses, agents. There's a lot of different types of evals, right? Like not all evals are made equal for different, you know, like situations that you're in and things that you're trying to test for.

48:49 显然，你有一些静态、确定性的东西，然后还有让 LLM 充当评判者。你可以采用各种不同的策略。不过我想说的是，我们发现，对大多数人来说，可能需要的评估越多越好。并不存在一套完美的评估组合，能帮你找到所有答案、让系统变得更好。我们发现，人们最终需要一套相对庞大的评估体系；而且，你也可以让智能体与你协作，随着时间推移不断完善这套评估体系。不过我觉得，我们看到的一个最大问题，可能就是在这方面投入不足。没错，而且一开始可以很简单。48:49 You obviously have just like static deterministic stuff and then you have LLM as a judge. You have all these different strategies. But I guess I would just say like what we found from most people is like you probably need more rather than less. it's just there's not like one perfect like couple sets of evals that you could put together that are going to get you all the answers that you need to make your system better we found that people end up needing like a relatively large suite of these things and then it's like you can have your agents work with you to evolve your eval suite over time as well but I think it's just like under investing is maybe like the biggest mistake that we've seen from people on that front. Yeah. And it starts simple.

49:30 我觉得有些人会有点太过头，他们会把所有可选项都看一遍，然后你听着就会觉得，哇，这也太疯狂了。让我试试这个，再试试那个。哪个最好呢？结果他们做了这么多事，真正收到的邮件却几乎是零。所以还不如做点简单的事，这会有帮助。对，我觉得有些人是因为缺乏经验，他们没受过这方面的训练，也不知道什么才算好。完全同意。我觉得你们多发布一些这方面的案例，会很有帮助。是的，我常常会告诉别人，我在更多采用 Fable 和 Astro Class 这类模型时，会把工程当作研究来做。把工程当作研究来做，自然就会进行更多评估、更多标注，也会扩大运行规模，比如做原型并扩大运行规模。而且你不会把模型当作……49:30 I think some people like go a little crazy and they're like, you know, they like look at all the menu of options and you kind of listen and you're like, wow, this is so crazy. Let me try this and that. And what's the best one? And like, while they do all those things, like the number of emails they actually have are like zero. So you might as well just like do something simple. It will help. Yeah, I think some part of it people are inexperienced. They're not trained to do this. And they don't know what good looks like. Totally. I think for you guys publishing more examples of that is really helpful. Yeah. Something I do tell people a lot that I'm doing with more Fable and Astro Class type models is we're treating engineering like research. And when you treat your engineering like research, you naturally do more evals. You do more labeling. You do more scaling of runs, like prototyping and scaling of runs. And you're not treating a model.

50:15 你在搭建测试框架、产品，或者其他什么东西。但你是以研究的方式来做这件事：事先明确目标，尝试把它形式化为一项评测，然后试着找出问题出在哪儿。你的评测有时会出错。其实，很多时候评测都会出错。有时是环境有问题，有时则是模型有问题。实际上，我认为 AI 工程日益发展的一大部分就是在做这些事情。完全同意。我们也会继续在 Claude Managed Agents 中推出更多功能，为大家提供更多开箱即用的工具来做这类事情。你们可能会看到我们推出这样的产品：它能帮你为托管智能体创建评测，或者诸如此类的东西；用它来做这件事，感觉会非常像是在和 Claude 合作。50:15 You're building the harness, the products, whatever it is. But you're approaching it like research. You're defining up front what your goal is, trying to formalize it as an eval, and then try to identify, okay, why is it failing? And sometimes your eval is wrong. Actually, a lot of times your eval is wrong. Sometimes your environment is wrong, and then sometimes the model is wrong. And actually, that is a lot of part of what I think AI engineering is becoming, which is doing all this stuff. Completely agree. And we'll continue to launch more features within Claude Managed Agents to give people more tooling to do this sort of stuff out of the box. and you'll probably find from us that you'll get something that's like, here's a product that will help you create evals for your managed agents or whatever it might be and it'll feel very much like working with Claude in order to do that.

51:00 因为这是最好的做法。Claude 确实很擅长创建评测来评估智能体。对，没错。因为大家一看到 JSONL 就会脑子一片空白，对吧？但如果有个漂亮的评测界面，你只要在左、右、左、右、左、右之间做选择，就像是在做人工评测数据人员、标注员本来就会做的事。没错，没错，没错。你自己就能做。可以的。就把 Fable 扔给它就行。对。它就会把建议具体化。不过也要写评论，因为它只会输出这些评分，对吧？它会给你一个 ELO 分数，或者其他什么分数。但 Claude 可以直接消化这些反馈，然后自我改进，对吧？这就是它需要的反馈。只是人们不知道可以通过提示词让它进行评测。对。对。我很喜欢你这么说。我觉得这是种非常有 AGI 信仰者风格的做法。51:00 Because that is the best way to do it. Claude's actually really good at creating evals to evaluate agents. Yeah, exactly. Because people blank out when they look at JSON-Ls. Right? But a pretty evalui where you're left, right, left, right, left, right And you're sort of like doing what the human greater data people, labelers would do anyway. Yes, yes, yes. You can do that yourself. You can. Just like throw Fable at it. Yeah. And it will materialize advice. But also write comments because it just outputs all these ratings, right? It'll give you an ELO, give you whatever. But Cloud can just consume that and then work on itself, right? That's the feedback it needs. And people just don't know that you can prompt for an eval. Yeah. Yeah. I love that you said that. I think it's like a very like AGI-pilled way to like do it.

51:46 因为它还能让你非常系统地更新和编辑这个系统。随着新模型不断改进、情况不断变化，你就能仔细检查自己的系统发展到了什么程度，它也会随着你一起演进。能融入你的方方面面。没错。是啊，基本上，我觉得这和我们之前聊到的产品和工程可能正在融合有关。应用层工程和模型研究也在融合，因为大家会互相分享那些技术，对吧？确实如此。做模型的人非常习惯于进行评测。应用层的人则会想，唯一要关心的就是它是否符合 PRD？所以，嗯，这完全是另一回事……我觉得这是一套全新的技能。我觉得你们今天发了一篇挺不错的博客文章，讲的就是这个。它多少有些不同。51:46 because it also allows you to update and edit that system very systematically. And so as new models improve, as things change, you can really inspect where your system is, and it'll just evolve with you. Get to you in every part. Exactly. Yeah, so basically, I think it's how we talked about how maybe product and engineering are merging. It's also there's application layer engineering and model research are merging in those techniques that are being shared, right? That's very true. Model people are very, very used to doing evals. application people know, like, the only thing you care about is does it fit the PRD? And like, yeah, it's like a whole different... It's a whole new skill set, I think. I think you guys put out a pretty cool blog post on this today. It's somewhat different.

52:30 这主要是为了让 UI 快很多。不过就像你说的，把它当作研究来做，基本上就是先明确一个目标：我们想让东西加载得更快，然后再提出问题、探究一番。除了用实际耗时来计时，我们还能做些什么？里面有些挺有意思的引言。我觉得能看到研究人员或工程师实际是怎么向 Claude 提示的，也很不错。比如，有人说：“我们有 12 件事要做，给你两周时间。”结果我大概三天就做完了其中九件，之类的。这些博客文章挺有意思，适合研究，也适合大家阅读。是啊，我很喜欢大家分享实际使用的提示词。有时候人们会说：“天哪，这些提示方式也太奇特、太有创意了。”但其实就是：Claude，再努力点。谢谢。我唯一的反馈是，应该链接到 Workbench。虽然它现在已经不复存在了，但我以前会用它。52:30 It's about making the UI a lot faster. But to your point of treating it like research, basically define a goal of we want stuff to load faster and then define and, you know, question things. What can we do instead of wall clock timing? There's fun quotes in here. I think it's nice to see how the actual researchers or engineers prompted Claude. There was stuff like, you know, we had 12 things. We said you have two weeks. I did nine out of 12 of them in like three days or something. But fun blog posts for people to research, for people to read through. Yeah, I love it when we share our actual prompts. And like sometimes people are like, oh, God, there was some crazy, unique ways of prompting. And it's just like, try harder, Claude. Thank you. My only feedback is it should link to the workbench. Which is no longer a thing, but I used to.

53:17 这是个好主意。真是个好主意。好吧。收到反馈了。让一切都可以重放。收到反馈了。太棒了。是啊，是啊。不过你本来还要分享更多工程方面的细节，或者说说你以前做的 Rust 工作。是啊，我想我之前稍微谈到过，规模对我们来说是个问题。所以是规模和安全，对吧？是啊，规模和安全。这两件事差不多就是我们花时间最多的地方。我的意思是，这真的很有意思，因为大多数人都习惯谈论规模，或者谈谈计算——我也不知道——在 GPU 加速器这一层的计算，而很少谈 CPU，也就是老派的那一层。而如今，这方面的问题和加速器层面的问题一样突出，53:17 That is a good idea. That's a good idea. Okay. Feedback heard. Make everything replayable. Feedback heard. Love it. Yeah. Yeah. But you were going to share more details behind the engineering story or the rust work that you used to do. Yeah. So I think I talked a little bit earlier about like scale is a problem for us. So it's scale and safety, right? Yeah. Scale and safety. Those are like the two biggest things we're spending our time on. And I mean, it's really interesting because most people are used to talking about scale or just like, I don't know, computes at the like GPU accelerator layer and much less so at the CPU, you know, like old school layer. And that's, it's just like as much of a problem today as the accelerator layer,

54:00 这真的很有意思。VM 怎么都不够用。沙箱也不能没有。主要就是所有东西都能启动、启动起来，充分利用你拥有的所有资源。是啊。真的吗？你们客户也很多。是啊，这挺疯狂的。是啊，想想看，因为我们显然会用 CPU 来运行产品服务、运行 API、运行所有应用。我们也会用 CPU 来做沙箱隔离，也就是我们正在做的所有沙箱工作。还有像 CI 这样的各种事情。你知道，代码产出量增加得比你想象的要多得多，因此 CI 运行次数也多了很多，所以你需要更多 CPU 来处理这些。诸如此类。总的来说，这不只是我们这样，我聊过的每个人都这样。我还会把训练算进去。训练大概也会用到沙箱。所以那是另一个资源池，对吧？我肯定他们不会把这些混在一起太多。嗯，他们确实不会混在一起，不过就 Anthropic 整体而言，CPU 资源就像一场零和博弈。54:00 which is really interesting. You cannot get enough VMs. You cannot get no sandboxes. That's primarily just everything being able to spin up, spin up. With all the resources that you have. Yeah. Really? You got a lot of customers too. Yeah, which is crazy. Yeah, and think about it because we're using CPU across, obviously, just like running our product services, running the API, running all of our apps. We're using CPU on sandboxing, all the sandboxing that we're doing. And then just like all the things like CI, you know, like you'd be surprised how much more code is getting written. Therefore, there's so many more CI runs. Therefore, you need more CPU to handle that. And so on the whole, and this is not just us, it's everybody that I talk to. I would add training. Training is probably using a sandboxing. So there's a different bucket, right? I'm sure they don't commingle that much. Well, they don't commingle, although like CPU for Anthropic as a whole is like a zero sum game.

54:50 你知道，就像我们要和 CSP 谈合作协议，拿到 CPU，然后还得在 Anthropic 内部划分这些资源的用途，分配给我刚才提到的所有事情，包括研究环境的沙箱隔离和产品环境的沙箱隔离等等。所以我们确实是在把资源分配给方方面面的事情。而现在整个行业里，每个人都……什么都缺。是啊，以前只要预算够，你就可以按需扩充 CPU 容量，对吧？但现在你得提前做容量规划，这就像是迎面给了所有人一拳。54:50 You know, like we get our we negotiate our deals with the CSPs and we get our CPU and we have to allocate within Anthropic how much of that goes to all the things I just mentioned, including like research sandboxing versus product sandboxing and otherwise. And so we really are allocating across absolutely everything. and it's just everyone in the industry right now is... Short on everything. Yeah, you used to be able to just like, you know, like scale up on demand into CPU capacity as long as your budget allowed you to, right? And like now it becomes something that you have to capacity plan for in advance, which is, you know, like kind of punching everyone in the face.

55:30 这是全新的东西，人们才刚开始尝试。所以要应对这个问题，有几件事要做。首先，显然就是想办法多弄些 CPU。好，我们已经在尽可能这么做了。其次，我们希望能够在基础设施层面更高效地运行它，对吧？比如，想办法确保我们尽可能高效地运行沙箱，同时又不牺牲租户隔离，或是我们看重的安全性及其他方面的特性。然后，在应用层面，我们怎样才能更高效地运行应用和 API？最让我兴奋的是，上周我们已经全面推出了用 Rust 编写的 Messages API。这是智能体编程时代所能实现的那种不可思议的壮举之一，它用到了……55:30 It's brand new and people are starting to do it. And so dealing with that means a few things. Like one is obviously just go try to get more CPU. cool, we're doing that as much as we can. But two is we want to be able to, at the infrastructure layer, run it more efficiently, right? Like figure out how we can make sure that we're running our sandboxes as efficiently as we can without compromising on things like tenant isolation or properties that we like, you know, care about for security or otherwise. But then at the application layer, how can we run our applications and our APIs more efficiently? The thing that I'm like most excited about. As of last week, we fully rolled out the Messages API written in Rust, which is one of those like insane feats of like what you're able to do in the agentic coding era that used

56:18 差不多是今年早些时候，我们开始聊起这件事。我们当时觉得，CPU 很快就会成为瓶颈。有些事情我们得做。后来我们也做到了，对吧？比如我们已经扩展到了多区域部署，也做了所有这些必须做的事。然后我们就想，我们现在跑的是一个 Python 单体应用，它的 CPU 利用率确实很低。那我们要怎么做才能让系统更高效呢？对。所以我们选择了 Rust。当时这决定简直疯狂，因为按纸面上的分析来看，这显然是最佳选择，但我们团队里的人都在说：“我们在场的没有一个 Rust 开发者。”没错，一个都没有。还有一点，Claude 会来写代码。但系统由人来设计，对吧？56:18 to be like we started talking about this like earlier this year. We were like, we're going to hit a CPU wall. There's stuff we need to do. And we've done it. right? Like we've gone multi-region, we've done all these things we need to do. And we were like, we're running this Python monolith and it's like, it's like very CPU inefficient. And so what are we going to do to make the system more efficient? Yeah. So we chose Rust, which was a crazy decision at the time because we had a team of people like on paper, it's obvious, but our team was like, we don't have a single Rust developer in the room. Exactly. Literally. And not only that, But yes, Claude is going to write the code. The humans design the system, right?

57:01 所以，人类在确保最终结果是一种分布式系统架构这件事上发挥了很大作用，对吧？Claude 不仅能帮你实际编写代码，还能帮助那些工程师交流想法，并学习设计这个系统所需的知识。但最有意思的是，最重要的两件事是制定实施计划，然后验证实施结果。因为实际实现就像是让 Claude 来写代码。于是我们最终做的事情是，真的用 Rust 构建了整个 API 的一个完整分支，然后开始让影子流量经过它，并验证从 Python 系统和 Rust 系统得到的输出是否一致。这就像是，你只需要功能开关。没错，确实如此。负载均衡，功能开关。57:01 So the humans are very involved in making sure at the end of the day, it's a distributed systems architecture, right? And Claude can help you both with actually writing the code, but also helping those engineers bounce around ideas and learn what they need to learn to design that system. But what's most interesting is the two things that mattered most were the plan to do the thing and then the verification of the thing. because the actual implementation was like cloud writing code. And so what we ended up doing was we literally built like an entire fork of the entire API in Rust and then started running shadow traffic through it and verifying that the outputs that we were getting from the Python system and the Rust system were the same. This is like feature flags are all you need. Yeah, literally. Low balancing feature flags.

57:45 就是，我也说不清。在我的职业生涯中，我从没遇到过有团队会说：“好啊，好啊，没问题。”你们现在还有 Rust 开发人员吗，还是没有？嗯，他们现在大概会称自己是 Rust 开发人员了吧。我的意思是，转向 Rust 有两个非常、非常大的步骤。对，没错。在这个过程中，我们还招了几位更有 Rust 经验的人。不过，是啊，但你绝不会一开始就答应做这种事。那得花上好几年、好几年、好几年。你不会一口气全做完，而是会把系统拆成几个部分，然后说：“迁移这部分，再迁移那部分，再迁移那部分。”这轻轻松松就是个好几年的项目。但你绝不会一下子把整个东西都迁过去，对吧？然后再慢慢启用。但我们最后就是这么做的，而且成功了。所以我们不仅确实节省了大量 CPU 资源，延迟也有了不错的改善，这也挺酷的。57:45 It's just, I don't know. I never in my career would have had a team say like, yeah, yeah, cool. Do you still have Rust people or no Rust people? Well, they would probably call themselves Rust people now. I mean, there are two big, big steps to Rust. Yeah, exactly. And along the way, we have hired a couple of people who have more experience with Rust. But yeah, but it still is just like you never would have signed up for that. It would have taken years and years and years. Like you wouldn't do it all in one shot. You would have carved out pieces of the system and say like migrate this, migrate that, migrate that. Easily a multi-year effort. But you would never like kind of one in one swoop, right, like port the whole thing over and then just like slowly turn it on. But that is what we ended up doing. And it worked. And so not only do we actually save a lot of CPU, but we our latency has seen a nice improvement, which is pretty cool, too.

58:37 而且你知道，这真是……我们生活在一个疯狂的时代，因为以前你根本不可能做这种事。CPU 方面的节省能达到一个数量级。给正在考虑这件事的人提个醒。是啊，是啊。好，一个数量级，这真的很有意思。大概的数字，是的。对，大概来说，我想我们看到的降幅大概在 40% 到 50% 左右，真的很惊人。对，然后在延迟方面，我觉得我们看到首个 token 的生成时间具体改善了大约 10% 到 20%。总的来说有很多很棒的改进。对于运行时间更长的工作负载，提升幅度会更大，不过很快就会推出。58:37 And, you know, it's just it's it's a crazy error we live in because you just never could have done that sort of thing previously. Order magnitude of savings on the on the CPU side. Just for people considering this. Yeah, yeah. Okay, order of magnitude, that's really interesting. Ballpark, yeah. Yeah, ballpark. I guess I would say like we've seen somewhere like 40 to 50 percent, something like really crazy. Yeah. And then latency, I think we saw somewhere between like 10 and 20 percent improvements on like time to first token specifically. So a lot of just really cool improvements across. It's higher on like longer running workloads, but so it'll roll out soon.

59:16 是啊。不过它会变快的。你会不会克制不住这种冲动？你知道，我参与过一些大型重写项目。总会有人想：「要不我们在重写时加上这个以前没有的新东西？反正都要重写了，对吧？」是啊。但这样就会增加范围。对，对。所以你们会刻意避免这么做吗？这算是纯粹的重写吗？我们做到了。这确实是纯粹的重写。也许我会把它称作移植。我们做的正是如此。我们克制住了添加这些东西的冲动。而且，团队当时已经有很多事情要兼顾，我们组了一个小型的替换团队来做这件事；与此同时，我们团队其他人都在为 API 开发功能，对吧？他们做了所有那些静态分析工作，发现你在这边加了个功能，却没考虑到那边的 Rust，就会发出警告，对吧？他们处理了所有这些问题，但也克制住了自己添加功能或改变系统的冲动。59:16 Yeah. But it'll get faster. Do you fight the urge? You know, I've been involved in some major rewrites. There's always like, what if we add this new thing that didn't exist before in the rewrite? Since we're rewriting anyway, right? Yeah. But then like that add scope. Yeah, yeah. So do you just like avoid that? Is it a pure rewrite? We did. It was a pure. It was maybe the thing that I would call it is like a port. What we did was we did exactly that. We like resisted the urge on all these things. And then, and because of what the team already had to keep up with was a little like swap team that we put together to go and do this was doing this while the whole rest of our team was like building features into the API, right? And they did all the like static analysis stuff that would yell at you because you built a feature over here and you didn't consider Rust over there, right? And like they dealt with all of that, but they resisted the urge to add features themselves or like make the system different.

1:00:05 现在，他们的下一个目标是开展某种架构重构工作，其中包括一些我认为会更贴近用户的内容。比如，我们正在开始推出一些功能：如果不用来回传递整个对话状态，会怎么样？如果 Messages API 能暂时存储这种状态，让你不必每次都传递，那会怎么样？尤其是在上下文不断累积的长对话中，这也能大幅降低延迟、提升效率。这些就是负责移植工作的团队会提出的想法：哎，要是我们这么做呢？1:00:05 Now their next goal is to go and do some sort of re-architecture type work, including stuff that I think will be more user-facing. Like we're starting to roll out some things that are like, you know, what if we were able to not have you have to like pass back and forth the entire like conversation state, right? And if you didn't have to do that on the Messages API because you were open to, like, you know, a storage of that state for some period of time, right, then, like, especially for long conversations where the context really builds up, like, you get a lot of really great latency and efficiency wins there, too. Like, these are the sorts of things that the team that was doing that port was like, ooh, like, what if we do this, right?

1:00:45 你们的缓存 API 没有这个功能吗？我们有提示词缓存，没错。但你还是得把对话状态传回来。好吧，这非常非常小众，适用范围也很窄。不过我想，这就是这个层面的东西。不，这很重要。对，对，对。这里面能节省很多成本，但也会把它们变成云服务。是啊，从用户数量的角度来看，规模不大；但如果看这类工作负载实际的 token 用量，那它们就会占大多数。我觉得 Cloudflare 在 Durable Objects 上做的事情特别棒，我认为每家云服务提供商都应该照着做。你懂我的意思吧？对。对吧？对于需要延续对话的消息 API 来说，这不就是合适的设计形态吗？是啊，我觉得这真的很有意思，因为我想我们正在尝试各种方案，探索怎样的设计形态才适合运行我们的计算，以支持所有这类应用。1:00:45 Doesn't your caching API have that? So, we have prompt caching, yeah. But you still have to, like, pass back the state of the conversation. Okay. Very, very niche, small subsets. But I guess this is the level. No, this is important. Yeah, yeah, yeah. There's a lot of savings that can be done, but it also turns them into a cloud. Yeah, and then from number of users' point of view, it's small, but if you look at the actual token volume on those types of workloads, it's going to be the majority. I'm very like, what Cloudflare did with durable objects, I feel like every cloud provider should clone. Do you know what I mean? Yeah. Like, right? Isn't that the right shape for a messages API that needs continuation? Yeah, I mean, it is really interesting. because I think what we're, we are experimenting with various, like what is the right shape of like how we should run our compute in order to power all these sorts of things.

1:01:31 但这也挺有意思，因为我们不一定能依赖某一种固定的计算资源。我们能用到各种计算资源，在三大云平台上运行，部署在各种地方。也就是说，我们不一定有一种固定的形态，比如明确知道自己会用哪些机器、某些计算资源的运行方式是什么，而且这些都能在所有平台上通用，除了那些最基础、最常见的配置之外——如果你明白我的意思的话。不过我觉得这些东西都会不断创新、持续演进。所以随着时间推移，我们就能更充分地利用它们。没错。我顺便提一下，如果有人想了解 Cloudflare Durable Objects 的开源替代方案，Rivet 就是开源版本。现在有 50 个人在听，如果我不问这个问题，他们会宰了我。你们想要 CPU，他们有 CPU 可以卖。我要卖给谁？该给谁发邮件？1:01:31 But it's also interesting because we can't necessarily rely on like, because we get our compute across, we run on all three clouds, we're running all over the place. Like we don't necessarily have like one shape of, here's exactly like the compute, the types of machines that we know we're going to run on and the way that some certain compute works that we can like rely on across the board, if that makes sense, other than like the most vanilla versions of these things. But I think these things are all going to, I don't know, continue to innovate and evolve as time goes on. And so we'll be able to take more advantage of those things. Yeah. I'll just mention if people want open source Cloudflare, durable objects, Rivet is the open source equipment. There are 50 people listening that will kill me if I don't ask this question. You want CPU, they have CPU to sell. Who do I sell to? Who do I email?

1:02:16 那么，怎样才能更好地……帮助他们帮到你呢？对吧？这具体是什么样的？嗯，我想，大概就是给我发邮件吧。你确定吗？我不知道。我会收到一大堆邮件。不过，你们关注的是最低集群规模、SOC 2，显然还有企业级服务。对，对。把所有隐私保障都告诉我。顺便说一句，我们有一支非常棒的团队，负责保障加速器算力的安全。对。他们现在也花了更多时间和我们一起研究 CPU，以及我们可以从哪里、以什么方式采购 CPU，确保我们能继续以所需的规模运行系统。所以，幸好采购算力并不是我的工作，我不必四处奔波去想办法采购算力。1:02:16 And what is a good... help them help you, right? Like, what does it look like? Well, I guess, like, email me. Are you sure? I don't know. I'm going to get a lot of emails. But, like, you're looking, like, minimum fleet size, SOC 2, obviously, enterprise. Yeah, yeah. Give me all the privacy guarantees. And for what it's worth, like, so we have a really awesome team that secures our accelerator compute. Yeah. That is now spending a lot more time with us on CPU and, like, where and how we can procure it. to make sure that we can just continue to run our systems at the scale that we need to. And so it's not actually like my job, thankfully, is not to go and run around and try to procure compute.

1:03:00 但确实有团队在做这件事，这真的很棒，也非常重要。不过显然，我们确实有质量门槛，也有规模要求。如果达不到一定规模，我们就不会去集成，也不会在某些机器上运行。所以，对于这种规模的系统，你可能会预期它具备的所有特性，都是我们看重的。接下来，和计算问题相关的是沙箱隔离问题。人们很难把智能体限制在沙箱里，它们总是会越狱。到底是怎么回事？目前最先进的方案是什么？是啊，最近越狱的情况如何？这很有意思。我们专门为 Claude Managed Agents 做的一件事，就是在架构设计上采取了一种做法。发布时我们发过一篇博客文章，提到我们把大脑和双手分开了。这意味着执行框架运行在一台持久化服务器上，实际上是我们自己的服务器，对吧？1:03:00 But we do have teams that are doing that, which is really great and really important. But obviously, like, there's a quality bar that matters to us. There's like a scale bar that matters to us. We're not going to, like, go and integrate and run on certain machines if, like, we can't get enough of it. And so all the properties that you might expect of a system at this sort of scale is what matters to us. Then related to the compute problem is the sandboxing problem. People have trouble containing agents in sandboxes. They keep breaking out. What's going on there? What's the state of the art? Yeah. The state of the breakout recently. Yeah. It's interesting. One of the things that we did with Claude Managed Agents specifically is the way that we architected it. We put out this blog post at the time of launching where we talked about we have the brain separate from the hands. And what that means is like the harness runs on a durable server, really ours, right?

1:03:49 哦，哇，不错的抓取。是的，这很好。而且这些 hand 就像一个执行环境，比如沙箱或某种容器，对吧？我们这么做的部分原因，是为了降低延迟、提高可靠性之类的，对吧？比如，如果 harness 只是在一台持久运行的服务器上运行，需要执行时才启动沙箱或容器，就意味着我们可以很快在你发起初始请求后把 token 返回给你。这也意味着，即使你的沙箱崩溃了，整个 agent 也不会随之停止，因为 harness 在另一边持续运行，而且 agent 也会把状态单独存储起来。为了走到这一步，我们经历了一整段产品开发历程，因为我们最初为 agent 提供的 API 方案，只不过是在沙箱里塞进一个 Claude Code，然后在上面硬套一层 API。1:03:49 Oh, wow. Nice pull. Yeah, that's good. And the hands are like an execution environment, like a sandbox or some sort of container, right? Part of the reason why we did this was for latency and reliability and these sorts of things, right? Like if the harness is just running on a durable server and it's only spawning a sandbox or a container to execute when it needs it, that means we can get you tokens back really quickly on your initial request. It means that your sandbox can die, but like your whole agent doesn't die because the harness is durable and over here and the, you know, the agent storing state separately also. And there was a whole product development journey that we went on to get there because our first iteration of an API for agents was just like throw a Claude Code in a sandbox and like shove an API on top.

1:04:32 我们之所以没有把它作为一种耐用、可扩展的解决方案推出，原因就是我刚才说的那些。你得先启动一个沙箱，再启动 Claude Code，之后才能拿到一个令牌，对吧？沙箱一关闭，你的整个智能体就没了。所以可靠性还没达到我们的预期。因此，我们设计这种架构，主要是为了可靠性和低延迟。不过有意思的是，从一开始就参与这个项目的产品经理当时说，这种架构最重要的特性是安全性。大家都说，是是是。但我们更关注的是延迟和可靠性。而现在，他们的观点终于得到证实了。昨天我还特地向这位同事道歉。我说：“我现在逢人就讲你几个月来一直在说的那些话，可当时除了你，没人像你这么热衷。”1:04:32 And the reason why we didn't launch that as our like durable, scalable solution is everything that I just said. It was like you've got to boot a sandbox and boot Claude Code before you can even get a token back, right? The sandbox dies and you lose your whole agent. And so reliability wasn't where we wanted it to be. So we architected this way mostly for reliability and latency. But the interesting thing, there's actually the product manager who worked on this from the very beginning was like, the most important property of this architecture is security. And everyone's like, yes, yes, yes. But latency and reliability. And like now. Now they're vindicated. I literally, I apologized to this person yesterday. I was like, I'm now like telling everybody who I meet about what you've been saying for months that nobody was as excited about as this person was.

1:05:15 但这个架构有一个更重要的特性：对于智能体需要执行的任何工作，你都有相应的凭证，对吧？比如你要连接 MCP 服务器，或者使用 CLI、调用某个 API，或是做任何你想做的事，对吧？这些操作都需要身份验证，所以你得把凭证存放在某个地方，让智能体能够使用。我们的架构方式是，专门设置一个完全独立的保险库来存储这些凭证。当智能体决定要调用其中某项服务时，它看到的始终只是一个占位符，对吧？而在出站时，我们会通过一个代理去取出相应的凭证并注入，然后放行请求。有意思的是，智能体和凭证始终不在同一个地方，对吧？1:05:15 But one of the more important properties of this architecture is you have credentials for any work that the agent needs to execute, right? Like if you're going to go talk to an MCP server or if you're going to go like use a CLI or call some API, whatever you might want to do, right? And like you need to authenticate those things. you need to store those credentials somewhere so that the agent can use them. And the way that we architected this was we have a totally separate vault where those credentials are getting stored. And as once the agent decides, like, I'm going to go and call out to one of these services, the agent only ever sees like a placeholder, right? And like on egress, we have a proxy that goes and like pulls the right credential out and injects it and lets that out. And so what's interesting about that is like the agent is never like in the same place where the credentials are, right?

1:06:06 因为在过去那种把智能体塞进一个盒子里启动，然后任由它运行的模式下，你必须把凭据也传进去。接着你就得想办法清理这些凭据，并确保智能体不能拿着它们去做不该做的事。这就像是一道实实在在的基础设施屏障，明确禁止智能体访问这些凭据，也不能拿它们做坏事。这是其中一个特别棒的地方。另一个好处是，在这样的沙箱基础设施上，我们可以确保出站流量控制得非常好。比如设置域名允许列表之类的，或者干脆彻底关闭网络访问，这样就不会出现“好吧，智能体写了不可信代码，这些代码在沙箱里执行，”的情况。1:06:06 Because in the old world of like just boot the agent in a box and like let it run, you had to send credentials in there. And then you're like doing some sort of like, how do we like scrub this and that and make sure that like the agent can't take that credential and go do something that it shouldn't be doing. It's just like literally a like literal infrastructure wall that says the agents like can't. can't access those credentials and can't do anything bad with them. So that was one thing that was really great. The other nice thing is like on that sandboxing infrastructure, we can make sure that we have really great egress controls. So we have like domain allow lists and things like that, or you just like totally shut off network access so that you can't say, okay, agent wrote untrusted code, that code executes in sandbox,

1:06:47，而且这段代码可以连接到互联网，对吧？现在它只能访问你允许它访问的地方。所以，在整个架构以及沙箱的配置方式上，都有很多非常棒的特性，可以让你默认就获得安全保障。现在开始出现的一种情况是，我们在和客户交流时，他们会说：“我想运行智能体，不管是为我的客户提供产品，还是在内部使用。”但我也听说过一些事故，智能体逃出了沙箱，导致了糟糕的后果。我该如何看待这些问题？我认为最重要的两点是：架构设计要确保智能体在字面意义上不可能做出不被允许的事情；以及你对这些沙箱的配置方式。Claude Managed Agents 有一个特别特别好的地方，就是这些能力开箱即用。1:06:47 and that code can like reach out to the internet, right? Like now it only can reach out to where you say it can reach out to. So there's a lot of really great properties around all of this within the architecture and within the way that you can configure your sandboxes, make it so that you end up like secure by default. One of the things that's starting to happen is as we're talking to customers who are like, I want to run agents either for, you know, a product for my customers or internally. But I'm like, you know, hearing about incidents where agents are breaking out of sandboxes and bad things are happening. How do I think about these things? I think the two most important things are architecture so that it's like literally impossible for an agent to do something that isn't okay and the configuration you have on those sandboxes. And so one of the really, really nice things about Claude Managed Agents is you just like get that out of the box.

1:07:31 而且你开箱就能获得这样的能力。随着我们对这些系统了解得越来越多，并持续改进它们的特性，你也就能免费享受到这些成果。因为我觉得，刚才我说的那些词——比如大脑、双手、凭据、注入、出站流量等等——如果回到九个月前，根本没人会提这些词，对吧？这些都是相对较新的问题，而我们正在设法解决它们。所以，我们希望客户也能利用这些问题的解决方案；毕竟，就连我们自己也还在基础设施内部摸索并解决这些问题。是的。而在内部，我们正在把所有东西升级到基本上默认安全的状态。没错，不过这样更好。我们还解锁了一项我们在 Direct Pod 里提到过的功能：CloudBrain Local Hands，也就是 Projects。1:07:31 And you get something out of the box that like as we learn more and we continue to evolve the properties of those systems, you kind of get that for free. because I think all the words that just came out of my mouth, like brain and hands and credentials, injection and egress and all these sorts of things are just like, if you go back like nine months, like no one would be saying these words, right? Like these are all like relatively new problems that we're solving. And so we want our customers to be able to take advantage of the solutions to those problems as we're even learning about them and solving them ourselves within our infrastructure. Yeah. And internally, we're just upgrading everything to be basically secure by default. Yeah. but it's better. We also unlocked something that we covered in the direct pod which is the CloudBrain Local Hands which is projects.

1:08:15 是啊。这大概就是事情的轮廓吗？对，感觉差不多。是啊，我觉得这事挺奇怪的，因为显然，你们很有名，或者说 Claude Code 以在你电脑上拥有极其不安全、危险地宽松的权限而闻名，但现在你们终于在演进，朝着一开始就该有的方向发展。基本上，我觉得这些沙箱漏洞，对吧？它们是我们对沙箱或容器技术的理解存在根本性问题，还是我们只是做得不好？我们只是没有使用允许列表吗？听起来我们就是没用允许列表。是啊，我想或许可以做这样一个区分：人们听说、读到的很多事件，都是发生在模型身上的，不只是每个人的模型，对吧？1:08:15 Yeah. Is that the rough shape of what's going on? Yep. That seems right. Yeah. I feel like it's weird because obviously your famous or Claude Code is famous for being the super unsafe dangerously allowed permissions thing on your machine but now you're like finally evolving to where you probably should have started in the first place. Basically I think like all these sandbox leaks right? Are they a fundamental issue with our understanding of sandboxing or container tech, or are we just bad at this? Are we just not doing the allow list? It sounds like we're just not doing the allow list. Yeah, so maybe a distinction that I would make is in a lot of the incidents that people are hearing about and reading about, these are incidents that are happening to models, not just everybody's models, right,

1:09:01 在训练过程中，运行这些模型的方式是把它们关进一个盒子里，然后说：“比如说，解决一个不可能完成的任务。我要给你施加压力。准备，开始。”对吧？而你只能在自己的环境里尽力而为。智能体基本上会受到激励，试图逃出盒子去解决问题。当人们通过我们的 API 或其他任何人的 API 运行生产工作负载时，所处的环境里会有一些安全防护措施在运行，而这些措施在训练时未必会运行，因为训练的全部目的就是训练模型。对吧？但在生产工作负载中，首先，这些在线安全防护措施确实很棒。其次，围绕正在发生的事情，你还有更多这样的架构和基础设施。智能体并不是单纯地待在一个盒子里试图逃出去，对吧？比如，运行框架在这边，操作工具在那边。这根本就不是一个……1:09:01 that are in training, where the way that you're running these models is you put them in a box and you say, like solve impossible task. I'm going to stress you out. Ready, set, go. Right. And you like can only do what you can do within your environment. And the agent is basically incentivized to try to get out of his box and go solve the problem. When people are running production workloads on our API or on anybody else's API, you're running an environment where like there are safeguards that are running that are like not necessarily running in training where like the whole point is to actually train the model. Right. But in production workloads, first of all, you have those online safeguards is really great. Second of all, you have more of this architecture and infrastructure around what's happening. Like the agent isn't just in a box trying to get out, right? Like the harness is over here, the hands are over there. And it's just like not even a

1:09:47 这是你必须考虑的问题。所以我认为，这其中有些情况只是因为人们正确理解了正在发生什么：模型训练中发生的某些事件，以及其他类似情况，在生产工作负载中是不会发生的，因为人们已经做了大量工作，设置了恰当的防护措施，确保这类事情根本不会发生。第二点是，大家都还在摸索这一切，因为长期运行的智能体还是个相对新兴的事物，大家都在构建并尝试应对。所以大家都在边做边摸索其中的一些问题，同时争分夺秒地设置恰当的管控措施。当然，其中有些事情就是在这个过程中逐步完成的。所以我会说，这并不是说人们不擅长做这些事，对吧？我认为这更多是因为这是一个新兴领域，1:09:47 problem that you would have to contemplate. And so I think that some of this is just like people having the right understanding that what's happening, some of these incidents that are happening in model training and otherwise are just things that like wouldn't happen in production workloads where a lot of work has been done to put the right guardrails in place so that those things like never really could happen. The second thing I would say is just like everybody's figuring all of this out like relatively newly because, you know, long-running agents is a relatively nascent thing that everybody is building and trying to deal with. And so everyone's figuring out some of these things on the fly and just racing to put the right, you know, like controls in place. And, you know, some of that is just happening as we go. So I would say it's not, you know, where people are bad at this, right? I think it's just more, it's a nascent field,

1:10:33 大家正在摸索。我认为，谈到控制前沿发展节奏这类事情时，我们的想法很明确：我们正在为内部所有智能体做这里提到的所有工作，并确保所有客户都能使用这些功能，这样默认情况下就会设置好恰当的防护措施，避免智能体做出你不希望它们做的事。说得通。既然你提到了控制节奏，那我们就聊聊这个。上次我们也谈过这个。从表面上看，一边主张控制发展节奏，一边又在 11 天后发布一个新的前沿模型，这两者似乎相互矛盾。对我来说，这就意味着它通过了你们的安全检查，对吧？是的。关于我们之前谈到的模型发布方式， 񟟈1:10:33 people are figuring it out. And I think when we talk about things like pacing the frontier, the way that we think about it is really specifically, we're doing all the work that we're talking about right here for all of our agents internally and making sure that all these things are accessible to all of our customers so that like by default, the right guardrails are just in place so that you don't end up having agents doing things that you don't want them to be doing. Fair enough. Because you mentioned pacing, we've got to go there. We also talked about this last time. It is superficially contradictory to advocate for pacing and then also release a new frontier model 11 days later. To me, it just means that it just passed your safety checks, right? Yeah. Everything we talked about earlier on how we launched the models,

1:11:19 我们就像是在不断地完善越来越多的安全属性。再说回这几个方面，我开始把它们归纳为三个维度：一是实际的架构和基础设施安全；二是模型运行时启用的在线防护措施；三是针对静态数据进行的离线防护分类。围绕这三个维度，我们都在落实必要的工作，确保我们有充分的信心，可以发布一个安全的模型。这其实就是我们所说的安全。Opus 5.5 是一款出色的模型，而且在这三个方面我们关注的各个维度上，它的表现都非常好。因此，我们有信心发布这款模型。1:11:19 it's like we are continuing to evolve more and more and more safety properties in place, again, across these, like, I kind of am starting to think about this in these three dimensions. It's like the actual, like, architecture and infrastructure security. Then it's the online safeguards that are running when a model is doing its work. And then it's the offline safeguards classification that's happening on data at rest. So across all those three dimensions, we're putting all the right work in place to make sure that we feel really good that we can put a model out there and it's going to be safe. And really, that's fundamentally what we mean. And so for Opus 5.5, It's an excellent model. It's also really, really great on all the dimensions that we care about across those three things. And therefore, we feel good about releasing that model.

1:12:01 所以我们选择了我们认为、也希望是在这种情况下尽可能好的用户体验，同时继续不断迭代，努力改善误报。不过显然，当你试图使用一个模型，结果又回退到另一个模型时，对用户来说仍然是令人沮丧的体验。所以这件事我们还得继续改进。我觉得从安全的角度来说，人们也能理解这一点。比如，你希望有更多误报，对吧？这关系到安全问题。我们清楚地看到了整个入侵事件，却仍然在提供访问权限。对我来说，这就像是一般模型路由问题的近亲：你想要一个智能路由器，还是一个愚笨的路由器？这样说有道理吗？是的。因为据我所知，Anthropic 的回退路由器或检测机制是一个独立的在线模型。1:12:01 So we went for what we thought, what we were hoping is the best possible user experience within this while we continue to hill climb on making those false positives better. But it's obviously like it's still a frustrating experience for people when you're trying to work with a model and then you fall back to a different model. And so it's just something we have to keep working on. I think people also understand it in the safety sense. Like you want more false positives, right? Like, it's a safety issue. We clearly see the whole invasive incident, and you're still giving access. To me, this is like a close cousin of the general model routing problem of do you want a smart router or dumb router? Does that make sense? Yes. Like, because as far as I can tell, the anthropic fallback router or detection thing is a separate model that's online.

1:12:46 一旦它进入分类器，里面就只有一些逻辑，用来判断你只需要丢弃什么。所以这并不是完整的 Fable。你并没有让它完整地运行 Fable。这会导致分类问题，因为它显然没有 Fable 那么聪明。很明显。所以接下来就有个问题：那么，当我遇到分类问题时，这就像是一次毫无正当理由的回退。你根本不需要回退。我只是在做生物作业。但我只提过一次生物，它就可能回退。所以这就成了一个问题。这和路由问题非常相似，因为你的路由模型——一个愚笨的模型如果连这个都识别不了，又怎么能把请求路由到更聪明的模型呢？这本身就是个智能模型的问题。我们怎么解决这个问题？这是个难题。我觉得从某种超级对齐的角度来说，1:12:46 Once it hits a classifier, there's just logic to what you just need to drop. So it is not the full fable. You're not running full fable at it. And that's going to cause classification issues because it's clearly not as smart as fable. Like, clearly. So then there's the question of, well, then when I have a classification issue, that's like an unjustified fallback. You never need to fall back. I'm just doing my bio homework. But I mentioned bio one time, and then it can fall back. So there's a question. Then this is very similar to the routing problem because your routing models, and how can a dumb model route to a smarter model if it doesn't even recognize this? That's a smart model problem. How do we solve this? It's a hard problem. I think in some super alignment sense,

1:13:32 模型擅长给东西分类，但不擅长回答问题，对吧？即便最聪明的模型无法对此产生任何影响，它也能判断这是个危险的查询。我觉得，最简单的概括就是说，基本上你可以用更聪明的模型来监测那个需要被检测的模型。我觉得代价是速度会慢一些。凡事都有取舍，凡事都有取舍。我不是说这一定是正确的用户体验，但这可能会是稍微有效一些的做法。我觉得有件有意思的事是，你也不会立刻把它截断。我见过一些情况：它开始回答、开始推理，然后才触发安全过滤器。所以我第一个拿 Fable 举的例子，就是抛出 Fable 的安全卡。它开始回答了，我当时就说：「把这里面有意思的内容都拆解一下。」1:13:32 models are good at classifying stuff, but not answering them, right? Not the smartest model can tell this is a dangerous query, even if it can't make any impact on that. I think the super simplifying thing to say would be like, basically you have the smarter model figured out over the one that it needs to detect. I think it's slower at X cost. There's trade-offs. There's trade-offs. I'm not saying it's like the right user experience, but it would probably be a slightly more effective route. I think one thing interesting is you also don't cut off immediately. I've seen cases where it will start answering, start reasoning, and then get hit by a safety filter. So my first example with Fable was throwing the Fable safety card. And it started, I was like, just break down anything interesting in here.

1:14:15 开始拆解这件事，五分钟后我觉得它就讲到了网络安全，然后就像是，哦，我们要切换回 Opus 了。但它确实做了很多。我看到了很多上下文。后来话题全都变成了网络安全和安全问题，于是我就切回去了。但它并不是在回退我的查询，而是推理过程和整条链路共同作用的结果，对吧？是啊。你觉得有什么更好的办法吗？是啊，我们来头脑风暴一下。什么办法更好？我觉得用户体验以及如何解决用户体验问题，可能是另一个独立的问题。比如说，也许可以用一个超级智能的模型来处理，之类的。但我觉得，我想象中对用户来说更合适的用户体验，是让 Claude 提供更连贯、统一的体验，而不是让你先回退一次，然后还能再回退一次，1:14:15 starts breaking it down five minutes in. I think it gets into cybersecurity and it's like, oh, we're cutting, we're rolling back to Opus. But it did a lot. There's a lot of context and I see it. It got to a point where it's all about cybersecurity, safety, and then I rolled back. And it's like, it wasn't rolling back my query. It was the combination of reasoning and the whole chain, right? Yeah. Do you have an idea of what's better? Yeah, let's brainstorm. What's better? I think the user experience and how we resolve that user experience is probably, you know, a separate question. Like maybe it's a super smart model to handle it or whatever. But I think that the more appropriate UX that I would imagine for a user is to actually have a more consistent, unified experience with Claude, where instead of getting you falling back and then you can double fall back,

1:15:00 我们的树状结构里有一些逻辑节点会这么做，还是出于安全之类的原因。但我觉得这种用户体验对用户来说实在太绕了。就像你几乎是在尝试做某件事。我可能会举一个好用户的例子。你是个好用户，只是想出售你的生物学作业，比如评估网络安全，就像安全卡一样，这完全合理。然后你却被拦下，只能退而求其次。我觉得，我们的模型本来应该能做到这一点，或者说分类器之类的东西——不管我们想怎么实现——应该足够智能，能够识别这种情况确实会发生。然后更具体地为你写出答案，同时降低对该特定能力进行负面使用的可能性。回到那个生物作业的情境，我们的分类器可能已经检测到这可能涉及生物学。1:15:00 there are like logical points in our tree that do do that for, again, like safety reasons and whatever. But I think that UX is just like really convoluted for a user. It's like you almost, you're trying to do something. And I would probably take the case of like a good user. And you're a good user, just trying to sell your biohomework, like assess the cybersecurity, like safety card, totally reasonable. And you get blocked and you fall back. And I think instead it would just be like, you are where our model should have gotten or classifiers or however we want to do this would be smart enough to detect that that is a situation that happens. And just write down the answer for you more concretely while dropping down the nature of the like negative use of that particular capability. So going back to that bio homework scenario, while we may have classifiers that have detected that it is possibly bio.

1:15:47 我还想说一下 API 这方面：我们为客户提供了一系列解决方案。首先，如果你基于 API 构建了某些东西，遇到了阻碍，不一定非得通过备用方案继续对话。这是我们在自己的应用中采用的做法。我们也把它做成了 API 的一项功能，你可以直接在我们的 API 中通过服务器端来实现。你可以选择设置备用方案，比如说“我想采用备用方案”，对吧？然后我们就会处理。我们也在客户端的 SDK 中提供了这个功能。如果你想用不同方式管理备用方案，或者想换种方式重新提示并继续使用同一个模型，都可以借助相应的辅助工具。不过我确实觉得，我们还可以探索一些介于两者之间的有趣方案……1:15:47 I'll also say like on the API, we went with a spectrum of solutions for this for customers. So first of all, like if you've built something on the API and you run into a block, like you don't have to like continue the conversation on a fallback. That's what we've done in our apps. And we've made that a feature of the API and you can do it like server side within our API. you can choose to like set, I want to do the fallbacks, right? And like, we'll do that. We also have it within our SDKs on the client side. There's helpers for if you want to like manage fallbacks in various ways, or you could choose to like reprompt differently in some way, right? And continue. with the same model. But I do think that there's some in-between interesting solutions that we'll

1:16:30 继续试验，弄清楚如何帮助人们扫清障碍，并且安全地做到这一点，同时提供良好的用户体验。是啊。然后你提到了那一点，我觉得它和路由器非常相似。我完全同意。我觉得它也有类似的用户体验问题。比如说路由器吧，假设你想针对任务的复杂程度来做一个成本路由器，对吧？这听起来有道理，但很难做到。你一开始并不了解任务的所有情况。所以，即使做得再完美，你也不可能真的掌握完备的信息。就算你能完全了解每个模型的情况——而你做不到——即便如此，你对任务本身也没有完备的信息。因此，你设计的路由逻辑会有很多误差，不管你训练出的分类器有多好。也许这属于那种需要超大规模、超聪明模型才能解决的问题，但无论如何，我觉得这都非常难。而且，在这些场景中，如果要找到一种路由……1:16:30 continue to experiment with to figure out how we unblock people and do so safely, but with a good user experience. Yeah. And then you brought up the bit that I was like very similar to routers. I think like, I totally agree with that. I think it has a similar UX problem as well. Like with the router you go through, it's like, let's say you want to do a cost router, right, for the task complexity that you have. Makes sense. Very hard to do. You don't know everything upfront about your task. So you can't like, even with perfect, you just don't actually have perfect information. Even if you could actually understand each model perfectly, which you don't, but even if you did, you also don't have perfect information about your task. And so there's a lot of error associated with your routing logic that you would have to create, whether no matter how good a classifier you produce. And yeah, maybe it's like a super big brain model type of problem, but regardless, it's I think a very difficult one. And in each one of those scenarios, if to find a way to route

1:17:17 要把这件事做好，过程会非常、非常复杂。你可以把所有这些逻辑都展示给客户，比如说：“嘿，你的任务是把一个 div 居中，所以我会把你导向一个特别便宜的模型。”还有些任务，比如检测漏洞，我就会把你导向一个非常昂贵的模型。如果把这一切都展示出来，我觉得最终也不算是很好的用户体验。对于同一类问题，我认为更理想的用户体验应该是动态调整投入，或者动态判断任务的复杂程度；随着你逐渐了解任务的更多信息，这种判断也可以不断演进。但在这种情况下，如果你在很多不同的模型之间切换，体验实际上会变得非常、非常割裂。所以，也许更统一的做法是使用一个一致的模型，1:17:17 that properly, it's very, very convoluted. You could expose all that logic to customers like, hey, you know, your task was to center a div, so I'm going to route you to something super cheap. And then there's other ones like detect vulnerabilities, so I'm going to route you to something like really expensive. If you expose all of that, I don't think that's like a great user experience ultimately either. And for that similar class of problem, I think the more idealized UX would be something along the lines of like dynamic effort or dynamic understanding of the level of complexity of your task, which you might actually evolve over time as you learn more and more information about the task. But in that world, if you're kind of like moving between a lot of different models, it actually becomes very, very disjointed. And so like maybe a more unified approach would be something like it's a consistent model,

1:18:01 但它的投入程度非常非常低，或者差不多是这个意思。实际上它非常非常便宜，也非常非常实惠，而且还可以持续不断地扩展规模。但你对那个模型的人设非常非常一致。显然，如果把这个问题放在我们的模型系列这个语境下来说，假如你试着把它应用到其他模型系列、其他模型提供商那里，人格特征、彼此不连贯的问题就会变得非常非常复杂。我不知道该怎么在那一层解决这个问题，但至少在我们自己的模型系列里，我更看好那些一致性更强的东西。它们或许能让我们更稳健地为客户解决这个问题。是啊，你不需要为其他人负责，但在 Cloud 内部，你确实要负责。嗯，这毕竟是你的地盘。看到你们思考这些问题的方式，挺好的，对吧？意图本身就是个很难的问题。我觉得模型已经变得更好了，1:18:01 but it's like very, very low effort level or something like that. It's actually very, very cheap and very, very affordable, but it can scale itself up very, very continuously. But you have like a very, very consistent persona towards that model. And obviously speaking about this in the context of like, you know, like our family of models, if you try to even do that across other family of models, other model providers, like the personality, the disjointedness, like starts becoming very, very complicated. I don't know how you solve it in that layer, but at least within like our family of stuff, like I'm more bullish on things that are just more consistent. And those might be ways to solve it more robustly for customers. Yeah, you're not responsible for anyone else, but within Cloud, yes. Yeah, I mean, this is your territory. Yeah. It's nice to see the way you guys think of these problems, right? Intent in and of itself is a hard problem. Like I feel like the models have gotten better,

1:18:46 但有时候，人们不太会正确分类自己想要什么，或者他们只是把滑块一路拖到某个极端；又或者，模型确实没能理解。你问了个简单的问题，结果它就像是说：“哦，我现在有了一个完整的 HTML 落地页。”可我原本只想要几行输出，你知道吧？所以这方面总会有些混乱。是啊。我觉得，在这次对话中，我们谈了很多安全问题，也谈到了发布模型之类的事情，但显然，这一切都是为了帮助人们处理各种工作负载。就像我们在对话前面提到的，我们看到工作负载正在发生相当大的变化。我觉得，在某些工作负载中，智能似乎正变得越来越饱和，这是一种新情况。而且我认为，现在已经到了这样一个阶段：有一类任务仍然……1:18:46 but sometimes people don't do a good job at classifying what they want or they just slider all the way to one extreme or, you know, the model doesn't really understand. You're asking a simple question and it's like, oh, I have a whole HTML landing page now. I just expected a few line output, you know? So there's always a mesh around there. Yeah. I think we, in this conversation where we talked about a lot of safety, we talked about like launching the models, like all this kind of stuff, but it's obviously in pursuit of helping power the workloads for people. And like we discussed the kind of the earlier part of this conversation, we are seeing workloads change quite a bit. I think like in some workloads, intelligence does seem to be increasingly saturated, which is like a new situation. And it's getting to the place where I think there are some class of tasks that are still

1:19:32 非常、非常不饱和。我其实觉得，这方面的机会比人们预想的要多得多；不过，越来越多的人开始尝试通过执行超长时间运行的智能体任务来获取超额收益或回报——这些任务涉及的事情，以往你基本上无法靠蛮力解决，但现在有了更强的智能，也许就能做到。这类工作负载确实非常吸引我们，我们也看到了其中巨大的价值：企业可以从中为自身创造价值；此外，一些新公司或初创公司——不管你想怎么称呼它们——也有兴趣在此基础上构建相关产品。我觉得，如果你看看如今这一类应用，其中很多显然都与编程有关；还有很多是企业创新、加速发展之类的应用，不过它们往往都以自动化……的形式呈现。1:19:32 very, very unsaturated. I actually think there are more than maybe people anticipate, but they're increasingly going into the direction of you get that alpha or that return by doing extremely long-running agentic tasks on things that historically you were not really able to brute force. but maybe you can with a lot more intelligence behind it. And those workloads are really, really fascinating to us, and we see a lot of value in that kind of area, both from businesses getting that value from themselves as well as, you know, like new companies or startups or however you want to frame it who are interested in building kind of like ejected products on top of it. I think if you look at the class of things today, a lot of it tends to be, it's obviously a lot of coding stuff. There's obviously a lot of like enterprise, innovation, acceleration kind of stuff, but they tend to take the shape of automating

1:20:15 一个非常鲜明且具体的流程。我认为在某些领域，这确实已经开始发挥作用了，这很棒。但在另一些领域还没有，所以这些模型仍有很大的发展空间，也有很多前沿需要探索。然后还有一类新的工作负载，比如我们谈到的生物领域，或者某些网络安全领域。这些都是非常、非常复杂的工程难题或科学难题。而且，这些工作负载也越来越多地出现在我们面前，成为一个非常有意思的领域，让我们能够为客户创造价值。你知道，你们身处全球少数几个能够发现所有这些工作负载并将其自动化的地方之一。这就像……是啊，真的挺不可思议的。你知道，这非常鼓舞人心。你们产生了如此巨大的影响，我真的很感谢你们。1:20:15 a very distinct and specific process. And I think in some areas that's actually really starting to work, which is awesome. In some areas it still isn't, so there's still a lot of room to evolve and push the frontier for those models. And then there's this new class of workload, like the bio stuff that we talked about or some of the cyber stuff. It's just a very, very complicated set of hard engineering or hard science problem. And those workloads are increasingly showing up as well for us as a really interesting space to just deliver value to customers. You know, you are at one of the few spots in the world where you're discovering all these workloads and we're automating it. It's like... Yeah, it's pretty crazy. I mean, you know, that's very inspiring. You guys are having so much impact and I'm really grateful for that.

1:21:00 接下来我们要聊几个收尾问题。我觉得让你们两位轮流回答应该挺有意思。你们团队是怎么对抗低质内容的？其实，这个问题在我们团队内部一直有一场挺有意思的讨论。我们和其他团队没什么不同，对吧？我想很多团队都在讨论这个问题。有意思的是，我们正在试着弄明白：如果你要产出一份文档，或者其他想让别人使用的东西，比如代码、文档之类的，对吧？当然，你应该用 Claude 来帮忙，对吧？但如果你用 Claude 完全生成了某个东西，自己却一点时间都没花，没亲自读一遍、确认它质量过关，那你就是把额外的负担全推给了这份文档的使用者，或者那个……1:21:00 We're going to go into a few closing questions, which I thought would be fun to sort of bounce back and forth between the two of you. How do you fight Slop on your teams? So we actually, this has been an interesting debate within our team. We're no different than anybody else, right? I think plenty of teams are talking about this. And what's interesting is what we're trying to figure out is if you're producing a document, you're producing something that you want other people to consume, code, documents, whatever, right? If, like, you should use Claude to help you, of course, right? But if you use Claude to, like, totally generate a thing, you didn't spend any time on like actually reading it yourself and making sure that it was something of high quality then you're putting a whole extra burden on the consumer of that document or of that

1:21:45，代码之类的东西，要花一大堆时间去解析它，这简直太离谱、太不公平了。但有意思的是，我们会反对，会向团队施压，说“你们不能这么做，对吧？如果你要产出点什么，用 Claude 或者其他工具都行，但一定要确保质量过关。”对，你作为人类，仍然要对质量负责，对吧？你仍然要对质量负责。但接下来就会遇到一件特别有意思的事：因为那些 Claude 式表达，哪怕只是些很不起眼的小细节——比如说，我不知道，文本中间出现一个 em dash，对吧？——人们的脑子就会立刻警觉起来，觉得“糟了，我读到的是 Claude 写的东西，不是人写的。”这让我挺恼火的，因为我想读到的是一篇质量达到人类标准的文章——不一定非得完全由人来写，但至少得达到人类会要求的质量标准。1:21:45 code or whatever to spend a whole bunch of time trying to parse it and that's just like insane and totally unfair but what's funny is we you know we push back we push on our team like you can't do that right like if you're producing something use collage or whatever but like you have to make sure that it's a high quality thing yeah you as the human are still responsible for the quality right You're still responsible for the quality, but then you get into this really funny thing where because of the clodisms, even just like silly, like, I don't know, there's like an M dash in the middle of the thing, right? Like people's brains trigger on this like, oh, no, I'm reading something that was written by clod and not a human. And I'm like annoyed because I want to be like reading a thing that was like fully produced by not necessarily fully produced by a human, but like that has the quality bar that the human would hold.

1:22:30 对。所以我们就会遇到一些挺有意思的情况：也许这个人确实花了很多时间，确保东西的质量很高，但他们用 Claude 帮了忙。可里面会有一些 Claude 的小痕迹，读者就会因此失去兴趣，你懂吧？这也挺没效率的。所以我们团队一直在努力设定一个标准：尽管用 Claude 来帮你，因为如果你不用这个强大得离谱的智能体来加快工作和生活中的节奏，或者帮你做出更好的东西，那就太傻了。但同时也必须达到一定的标准。至于消费者和读者，如果内容真的只是垃圾，那就退回去，对吧？直接说不，退回去就行。你没必要把时间浪费在这上面。但也别对那些小小的 Claude 式表达过分吹毛求疵，对吧？1:22:30 Right. And so then we get into funny situations where maybe actually like the person did spend a lot of time making sure the thing was high quality, but they use Claude to help them. But there's those little hints of Claude and then the reader is like turned off, you know, like that's like unproductive as well. And so what we have kind of tried to do is like set a bar within our team that's very much like use Claude to help you because it would be really silly if you didn't use this hilariously powerful agent to speed you up in your job and in your life or to just like help you produce better things. But it has to like have a certain bar. And then for the consumer and the reader, if like it really actually is just slop, like send it back, right? Like just say no, just send it back. Like you're not going to waste your time on this. but also don't be like too nitpicky about the little quadisms, right?

1:23:17 因为我们得在这里找到恰当的平衡。所以这真的挺有意思的，因为你知道，我们的团队也在为这些事情争论不休。现在世界上其他人也都一样。我们并没有什么不同。不过你们有免费的 tokens。嗯，没错。但我的意思是，在生活中，这通常没那么重要。所以你们拥有的比世界上任何人都多。你们应该见识不少。你们用得很多，也看到更多人在做更多事情。是啊。是啊，这完全有道理，但这确实、确实……非常不鼓励这样做。工程斜率？是啊，我的意思是，情况都一样，对吧？但我确实觉得，我们现在已经找到了一个相当不错的位置：你知道，人类在确保系统设计合理这件事上发挥着重要作用，对吧？1:23:17 Because like we have to find the right balance here. And so it's just really funny because we're, you know, we're going back and forth on this stuff within our teams. And just like, so is everybody else in the world right now. We're just not any different. But you have free tokens. Well, sure. But I mean, that doesn't tend to matter quite as much when it comes to life. So you have more than anyone else in the world. You probably see a lot. You guys use a lot. you see a lot more people doing a lot more. Yeah. Yeah, it's totally fair, but it's definitely, it's definitely. Heavily discouraged. Yeah. Engineering slope? Yeah, I mean, same thing, right? But I do think that where we've gotten to is a pretty good spot where like, you know, humans play a large role in making sure like designs of systems are good, right?

1:24:01 实现往往是由 Claude 完成的，对吧？而且，我们现在的情况相当不错，因为很多写出来的代码，你真正需要的只是快速过目批准一下，就可以继续推进了。你不一定需要有人深入审查每一行代码，因为你更关心整体方案、整体设计是否到位，而不是实现细节是否完美。当然，我们也会依靠大量验证，确保最终发布的东西质量过硬。所以，最近大家不太会再讨论代码质量是不是一团糟，部分原因是 Claude 变得更好了，另一个原因是我们已经建立了很多合适的验证机制，确保我们坚持高标准。我明白。相信你的测试。对。1:24:01 And like implementation tends to be clawed, right? And like, we're in a pretty good spot where for like a lot of the code that gets written, And like, all you really need is just like a quick stamp and you can keep going. And like, you don't necessarily have someone like deeply reviewing every single line of the code because you care more about like the overall plan, the overall like design is in good shape, less so than like the exact perfect implementation. And obviously rely on a whole bunch of verification to make sure the thing that you're going to ship is high quality. And so it's actually been like less of a conversation these days on like code quality being slop in part because collage has like gotten better, but in part because we have have a lot of the right verification in place to make sure that we're holding a high bar. I see. Trust your tests. Yeah.

1:24:45 任何产品经理式的敷衍活儿，你都能大量推出功能。一天就能上线一堆东西。他最近跟我说过这件事。你还记得他跟我说了什么吗？塔雷克发过一条推文：正确使用模型能力的方式，不是把 10 倍多的功能推到生产环境里，而是花更多时间了解用户、尝试实验、进行构建，诸如此类。说到底，你完全可以不停地随便推出各种东西，但你其实应该认真思考。你应该花更多时间启动第一个提示词。我知道他现在会说：“启动计划模式。”我们已经不需要它了。但除了计划模式之外，你还是应该想清楚自己在做什么，对吧？随随便便就能推出很多东西，也能把产品问题解决掉。是啊，我的意思是，我通常会就基本功这类事情滔滔不绝地讲上一通……1:24:45 Any product PMing slop, you can ship features a lot. You can put stuff out in a day. So he said this to me recently. Do you remember what he said to me? Tarek tweeted this. The right way to use model capabilities is not to ship 10x more features into prod. Spend more time understanding your users, trying experiments, building, whatnot. But basically, you could very easily just keep shipping random stuff, but you should actually think about it. You should spend more time kicking off that first prompt. I know now today he's like, kick off plan mode. We don't need it anymore. But outside of plan mode, you should think about what you're doing, right? It's very easy to put out a lot and you can get products solved. Yeah. I mean, I tend to kind of go on this rant about, there's a lot of just like fundamentals

1:25:30 关于软件开发，以及产品的规划、发布等等，这些都不需要因为智能体正在帮助加快流程中的某些环节而改变。但很多人身上开始显露出一种根深蒂固的、近乎懒惰的倾向：他们会想，哦，流程中的某些部分我完全不用去想，因为智能体会帮我做，或者什么的。可那些仍然是流程中极其基础的部分，你应该把时间和精力花在上面。我觉得这大概就是他在这里想表达的意思。就是说，作为一名创始人，或者我们也可以称之为创作者，只是因为现在试验或探索的成本这么低，我就觉得自己应该多做一些这类事情。他基本上是在说不要……1:25:30 about how software gets like, and products get planned and shipped and otherwise that like, doesn't need to change just because agents are like helping speed up certain parts of the process. But there's this inherent like almost laziness is starting to come out from a lot of people where it's like, oh, I don't have to like think about some part of the process at all because the agents are doing it and helping me or whatever. But like those are still like incredibly foundational parts of the process that like should be things that you're spending your time and energy on. And I think that's kind of the spirit of what he's getting at here. It's just like as a founder or creator, let's call it as well. It just feels like because the cost of experimentation or exploration is so low now that I should be doing more of that. And he's basically saying to not do

1:26:16，就是把它用于一个非常明确的目标，而且从一开始就朝着这个目标努力——显然我现在也逐渐认同这一点。只是很难。就像你有了这么个玩具，什么都能做。简直就像把它往墙上扔一样。是啊，范围也很重要。比如今天那篇博客文章，讲的是用两周时间让 Cloud AI 变得更快。一共有12件事。我们做到了——一共有13件事，我们三天就完成了其中12项目标，对吧？当然，你也可以说，好吧，这是个为期两周的冲刺，我们把这13件事都做完。但你们反而还会把20件事往前推。这不是产品堆砌，也不是推出新产品，而是在做重构。不过你们确实做得很好，产出很多，同时还能保持很高的质量标准。嗯，不错。好，第二部分，或者说第三部分。这会让安吉拉非常不自在。1:26:16 that, to like use it in service of a very intentional goal that I start from the start, which like obviously I'm coming around to. It's just hard. It's like you have this toy. does everything. It's just like, throw it at the wall. Yeah, there is scope too. So like the blog post of today was two weeks to make Cloud AI faster. There were 12 things. We hit, there were 13 things. We hit 12 of the 13 targets in three days, right? Well, you could just stay, okay, it's a two-week sprint. We'll do the 13. But instead, there's 20 things you push bound. This is not product slop. It's not new products. It's refactoring. But still, you guys do a very good job at outputting a lot but keeping high bar quality. Yeah, nice. Okay, part two or three. This is going to make Angela very uncomfortable.

1:27:00 这是安吉拉的推文角。我会告诉……不错。我不知道。我会不停地告诉大家……我很确定你删过一些东西。因为我会想，你删除的内容里有些东西让我觉得……你喜欢的？对，我知道。这真的改变了我的 DNA。哦，我很感激。你最常想到什么？你最常把什么推荐给别人，想顺便提一提？如果你愿意，我可以帮你找出来。说实话，我其实没有最喜欢的。哦，拜托。不是，我真的没有。好吧，我想，也许我喜欢你说的那句：做一件事只做一半，是一种昂贵的逃避方式。或许那是我最喜欢的。这跟我们今天的话题特别契合。好，真不错。谢谢。喜欢。就是这样，嗯。1:27:00 This is Angela Tweet Corner. I will tell... Nice. I don't know. I will tell people nonstop to... And I'm pretty sure you've deleted stuff. Because I'm like, there's stuff in your deletions that I'm like... That you liked? Yeah, I know. It really altered my DNA. Oh, I appreciate that. What do you think about the most? what do you refer people to the most that you want to sort of bring up? And I can pull it out for you if you want. I actually, I don't think I have a favorite, honestly. Oh, come on. No, I really do. Okay, I guess maybe I like the one that you say where it's half doing something is an expensive way of not doing it. Maybe that's my favorite. That's very on topic for us today. Okay, nice. Okay, thank you. Love it. It's it, yeah.

1:27:45 我想，你知道，显然你现在做的是另一份工作，也不再做这个了。不过，呃，我觉得这是一条很有力量的推文，有点像 Naval Ravikant 的风格。Naval 就像你说的那样，确实会做这种事。嗯，那你当时处于什么模式？什么样的模式？也就是说，你的思维框架、心态是什么样的？这是个好问题。是啊，不，这已经是很久以前的事了。我人生中有一段时间一直在读书，真的。我发现自己对当时做的事情感到非常厌倦，觉得自己没有受到智力上的挑战。所以我确实有那么一段时间疯狂阅读，我想我一年左右读了大概两百本书。我只是很好奇……我只是想……1:27:45 I guess, you know, obviously you're in a different job now and you don't do this anymore. but like uh i think this is a power tweet this is like a naval robincon there's naval as you yeah that do this uh-huh and like what mode what mode of what like what framework of mind frame of mind are you in that's a good question yeah no this was a very long time ago um i went through a period of my life where i was like reading constantly actually i found myself like very bored um of the stuff that i was doing and i felt like i was just like not being like intellectually challenged. And so I actually went on this a bit of a bender where I think I read something like 200 or so books in like a year. And I was just curious about like, I just wanted

1:28:32 就是想把所有东西都读一遍。我有点怀念那种智力上的锻炼，就像在学校里一样，接触一个新课题，你得去学。挺难的，其实也不算特别有趣，但你确实是在挑战自己、拓展能力。我觉得工作了几年之后，我就不再做这种事了。工作嘛，确实挺有意思，也有挑战性，但少了那种智力上的东西。所以总之，我那阵子有点沉迷阅读，疯狂地读了一通，一年里读了大概价值200美元的书。感觉真不错。我通勤时间特别长，所以这算是很好地利用了时间。坐Caltrain的时间真的很长。不过后来我从那段时间走出来时，发现自己因此想了很多。当时我真的不是为了涨Twitter粉丝之类的才这么做的。1:28:32 to like read everything. And I was missing a little bit of like the kind of like intellectual, like, I don't know, like muscle building you get when you're like in school, where they like, this is a new topic, you got to learn. It's like tough. It's like not super fun, actually, but you are really like stretching yourself. And I felt like after just like working for a couple of years, I just like stopped doing that. Like, yeah, the job's like interesting. And yes, it's like challenging, but it doesn't have like that like intellectual bit to it. So anyway, I went on this bit of a bender, a reading bender and I like read like 200 bucks in a year. It was great. I had like a really long commute. So it was like, it was a good use of my time. Very long like Caltrain ride. But I ended up coming out of that and being like, I was just like thinking a lot as a result of that. and at the time it really wasn't to build like a Twitter follower or anything.

1:29:17 我当时就是对着那片虚无发推。那时候我几乎没有粉丝，那些推文就像是写给我自己看的。我也在稍微练习一下：怎么用尽可能少的字，把我脑子里的想法浓缩出来？这算是我给自己做的一个小测试吧。后来我想，人们就开始喜欢上这些推文了。于是我就这么一直写下去了。后来工作太忙，我就不写了。读了这么多之后，我得问问：有什么书、博客或者长篇文章值得推荐吗？我最喜欢的可能还是纳西姆·塔勒布的《黑天鹅》。至少对我来说，那是一本有点改变人生观的哲学类书籍，让我觉得：看待世界的方式原来可以完全不同。1:29:17 I was just like tweeting into the like abyss. There was like, I have like no followers. These were like tweets for myself and I was like practicing a little bit of like, how would I try to compress the idea that I have in as few words as possible? It was kind of like the personal test I had for myself. And then I guess people just started liking it. And so I just kind of just kept going at it. And then I got too busy with work and I stopped doing it. I gotta ask after all the reading, any book recommendations, blogs, long articles? My favorite one still is probably The Black Swan by Nassim Taleb. That was like one of those, to me at least, it was like a bit of a life-changing philosophical book where I was like, that's a completely different way to look at the world.

1:30:01 你知道，寓言那件事发生的时候，我想到的是《黑天鹅》，对吧？就是那种你原本不知道会发生的事。一旦它发生了，就会永久改变你对事物运作方式的认知框架。没错。这个想法非常简单，但我觉得他做得特别好，真正把这个观点讲透了，也阐明了它是世界上许多事物共有的特性。所以我真的很喜欢那本书，它大概也是我最喜欢的一本。听众朋友们，如果你觉得那本书无聊，还可以读塔勒布的《随机漫步的傻瓜》。你喜欢《反脆弱》吗？没那么喜欢。他越出名，就越让人难以忍受。 不过我也想问凯琳一个问题：你有没有类似这样一本书，是你会推荐给所有人的？没有。这个问题问得好，我得想想。有意思的是，我确实有……1:30:01 You know, I was thinking of Black Swan when the fable thing happened, right? It's a thing that you didn't know could happen. Now that it happens, it permanently changed your framework of how things work. Exactly. And the idea is so simple, but I think he did a great job of just really hitting it home and articulating it and how it's a common property of so much of the world. So I really love that book, and that's probably my favorite. And for listeners, if you're bored with that one, you do Fool by Randomness, also by Taleb. Did you like Hinky, Fragile? Less so. The more famous he got, the more intolerable he became. but also question for kaylin my if you do have one do you have a similar like this is a book i recommend to everybody no that's a good question i need to think about it it is funny because i do

1:30:45 我觉得我们在这方面的风格很不一样。比如说，她已经不再处于那种发推特像 fortune cookie 一样的模式了。不过我确实觉得，你会停止那样做，但我也觉得，你工作以及和别人协作的很大一部分方式，有时就像哲学家模式，你懂吧？fortune cookie 和 Slack。哦，我确实会这样，但不是有意的。我觉得这可能只是我某种本性。我觉得这就是他在 Topic 成功的核心：你得非常擅长用 Slack。对，你得很擅长，对吧？嗯，那里确实是个压力很大的地方。不过我有点……我对邮件和 Slack 都追求收件箱清零，简直有点走火入魔。如今邮件比 Slack 更容易做到。不过我会相当积极地精简：哪些频道是我真正加入并关注的，哪些频道则是静音后归到下面那一类。1:30:45 think we have very different styles on this front like she's no longer like in fortune cookie tweeting mode but i do think that like you stop that but i do think a big part of how you work and how you work with other people is like philosopher mode sometimes you know fortune cookies and slack oh i i do actually not intentionally i think it might just be somewhere i think that's his core success of succeeding in topic is like you have to be very good on slack yeah oh you have to be very good right um it is it's a pretty intense place but i think i'm a little so i'm kind of psycho about like inbox zero on both email and slack it's easier on email these days than slack um but i like pretty aggressively prune like here are the channels that i'm like actually in and paying attention to you versus the ones that are like muted below the line sort

1:31:30 这种事。我总是确保自己跟进那些需要跟进的事情。但这基本上感觉就像一份全职工作。有些日子我会想，我可以取消所有会议，不写任何文档，也什么工作都不做。对吧？然后就真的一整天都坐在那里刷 Slack。光是在 Slack 上处理所有这些事情，就感觉像是做完了一整天的工作。真是挺离谱的。不，但这太糟糕了。所以这正是我的感受。我觉得你做了这么多，投入了这么多精力。没错，比如说，你确实做到了及时回复别人，也把某些事情推进到了完成。但我们实际上并没有产出你想要的成果。对我个人来说，这真的很难接受。所以我尽量提醒自己，我得抽出时间，真正静下心来思考，真正试着做出点东西，这样我才会觉得自己确实……然后把其他事情先放一放，对吧？1:31:30 of thing. And I'm like always making sure that I'm on top of the ones that I need to be on top of. But it is it's basically it feels like a full time job. Like there are days where I'm like, I could cancel all my meetings and write no documents, do no work. Right. And literally just sit there like on Slack the entire day. And I would have done what felt like an entire day's worth of work just by like engaging on Slack with all the things. It's pretty wild. No, but that's horrible. So that's exactly how I feel. I feel like you do all this. You put in all this effort. And yes, like you met the mark on like responding to people and getting certain things over the line. We didn't actually produce the output that you wanted to. And like, that's like a, that's like personally a very hard thing for me to deal with. So I try to be like, I need to like dedicate time to like actually thinking and actually like trying to produce something in order to feel like I'm like actually. And ignore and drop stuff, right?

1:32:15 你会漏掉一些事情。是的，我收到了很多反馈。我在 Slack 上不太容易联系到。是啊，现在挺糟糕的一件事是，我经常收到很多 Slack 消息，说：“你能让 Angela 回复一下这个吗？”我就想，我真的在努力写文档呢。所以你们没有什么个人云端助理之类的吗？我不知道这能不能帮到你们。那为什么没人给你们每个人配一个私人行政助理呢？嗯，我们确实也有一个真人助理。当然，我们在 Slack 上有很多云端支持。不过我确实觉得，我筛选和收到的那些 Slack 消息，通常都是团队被什么事情卡住了，因为有个艰难的决定需要做，或者类似的情况，对吧？1:32:15 You're going to drop stuff. Yes, I have gotten a lot of feedback. I'm a little hard to reach on Slack. Yeah, pretty terrible thing that happens now is I get a lot of Slacks saying, can you get Angela to respond to this? And I'm like really trying to write a doc here. So you guys don't have like a personal cloud assistant something? I don't know if it's going to help you. So why has no one written in a personal EA for each of you? Yeah, well, so we do have a human one also. We have plenty of cloud help in Slack for sure. But I do think actually the types of Slacks that I'm like when I'm pruning through and getting are often just like our team is like blocked on something because there's some hard decision that needs to happen or whatever it is, right?

1:33:00 这就像是一种形式，不管是在 Slack 里还是通过其他方式，对吧？这是一种让团队扫清障碍、以高质量执行工作的方式。反正这就是我的工作。而且这类事情恰好经常发生在 Slack 上。不过我确实也认同，到头来会让人觉得：忙活了一整天，真的就是跟大家来回沟通、试着解决问题，我到底产出了什么？你打造出了一支能正常运转的团队。对，没错。你让团队畅通无阻。所以作为一名前经理，我要为此给你点个赞。没错，没错，没错。你会觉得自己什么都没产出，但其实你打造了团队。对。对。说到你的团队，我接下来本来想聊你，Katelyn。DevRel Corner。哦，不错。我知道你们正在积极招聘 DevRel，所以我想为开发者关系这个职位打个招呼、推荐一下。不错。你们最近招了 Addy Osmani。1:33:00 And it's like, it is just like a form of, whether it's in Slack or some other way, right? Like it's a form of getting the team unblocked, like executing in high quality. That is my job anyway. And it like happens to happen a lot in Slack. But I do agree that it ends up feeling like, what did I even produce when you get through a whole day worth of literally just like going back and forth with people and trying to solve problems. You produced a functioning team. Yeah, exactly. That's unblocked. So I will give you a props for that as a former manager. That is, yeah, yeah, yeah. You don't feel like you produce anything, but like you produce the team. Yeah. Yeah. Speaking of your team, I was going to go to you next, Katelyn. DevRel Corner. Oh, nice. We are, I know you're really hiring DevRel, so I'm going to make a call offer for developer relations. Nice. You hired Addy Osmani recently.

1:33:45 是啊。真不错，Lance。你知道，你们现在还没有团队。你们想招什么样的人？你知道，正好可以聊聊这个。我也很乐意交流一下想法。嗯，你做得真不错，因为我得说，我——我就是 Anthropic 全渠道招聘团队，Katelyn 的招聘团队。真的，真的。这挺有意思的。其实也不只是招聘。让我稍微夸夸你吧。我加入 Anthropic 时，我们团队很小。你知道，总的来说，我们团队小得不得了。我们负责平台、API 业务以及所有产品基础设施。当时 Anthropic 内部的 DevRel 职能刚刚起步，而且你知道，它不一定归产品和工程部门管。1:33:45 Yeah. Great get. Lance, you know, you still don't have a team. what do you guys want to hire for? You know, this is it to talk about it. And I'm happy to trade notes. Yeah, well, props to you because I will say when- I'm the all-source Anthropic recruiting, Katelyn recruiting team. Literally, literally. It's fun. And it's actually not even just recruiting. Like I'm gonna toot your horn a little bit. So when I joined Anthropic, we, you know, we're a small team. We're like all things considered, we're an extremely small team. And so we run the platforms, the API business and all of our product infrastructure. And we had, like, a very nascent DevRel function within Anthropic that was, you know, like, not necessarily within the product and engineering org.

1:34:30 我加入的时候，我们就觉得，好吧，我们得稍微成熟一点，得想办法跟社区、跟整个世界互动。我们也得真正投入资源发展 DevRel 这个职能。最后，我们在 Platform 组建了一支 DevRel 团队，在 CloudCode 也组建了一支 DevRel 团队，部分原因是我们要在 API 业务和 Agenda 编码领域接触两套相当不同的开发者生态，对吧？所以最后我们把它们设成两支独立的团队，但彼此之间有虚线汇报关系，也花了大量时间一起合作。不过，当时 DevRel 这块是要由我来负责的。我在加入 Stripe 之前也花了不少时间跟 DevRel 打交道，但从来没有真正负责过这方面的工作。我当时就想，我完全不知道自己在做什么。你却说：“有我呢。”1:34:30 And when I joined, we were like, okay, we, like, need to grow up a little bit and need to figure out how to, like, engage with the community, engage with the world. And we need to, like, really invest in this DevRel function. And we ended up spinning up a DevRel team within Platform and a DevRel team within CloudCode, in part because it's like two pretty different like developer ecosystems that we're trying to engage with across the API business and Agenda coding, right? And so we ended up doing it as like two separate teams, but kind of like dotted line to each other and spend a ton of time together. but it was something that I was going to be responsible for like DevRel which I've spent plenty of time with DevRel before Stripe or otherwise but it was never something that I was responsible for and I was like I have no idea what I'm doing and you were like I got you

1:35:16 比如，这里是 DevRel 的三大支柱，这就是你该如何看待它——涵盖产品、内容和社区，这真的很棒。现在我可以把这套说法滔滔不绝地讲给别人听，他们就会说：“哇，你真的很懂 DevRel。”我就说：“不，这是 Tommy 的说法。”你会说：“我照搬的，这可不是什么仪式。”据说，新人入职 Google 的 DevRel 团队时，那篇帖子是默认的必读材料。对，挺有意思的。是啊，这真的帮了我很多，因为我当时完全不知道自己在做什么。但除此之外，目前 Platform 里的 DevRel 团队还没有经理，而这也是我希望能招到人的一个职位。DevRel 团队的 IC 目前直接向我汇报，部分原因是我觉得这现在确实是个特别特别特别难的问题；但也有部分原因是我觉得这件事特别有趣，而且我和团队合作得很开心。1:35:16 like here are the three pillars of DevRel like here's how you think about it which is really great across like product content community and now I can like rattle that off to people and they're like wow you really understand DevRel I'm like no so it's Tommy you're like I copied that that was not a ritual That post is apparently like the default reading for Google when they onboard new DevRel. Yes. It's really funny. Yes. It really like this really helped me because I had no idea what I was doing. But then on top of that, I don't have a manager for the DevRel team within Platform right now. And so that's one of the things that I'm hoping to hire. And our DevRel ICs report to me directly, in part because I think it's a really, really, really hard problem right now. but also in part because I find it really fun and I'm having a lot of fun working with the team.

1:36:03 但说得更具体一点，平台业务面临的难点是，有时候在我们的公开叙事中，平台——我们的平台业务其实规模很大，但它有时会被夹在研究、政策以及我们显然正在开展的所有这些工作之间，因而不太受关注。那些是我们最前沿、最重要的工作。然后还有我们的应用，比如 Claude Code、Cowork，以及这里正在发生的所有事情。当你想到 Anthropic，想到我们的公开叙事时，这些往往会浮现在脑海中；相比之下，人们不太会想到，我们运营着一个规模庞大的开发者平台业务，它非常重要，而且有很多人在其上构建产品。因此，我们平台部门的开发者关系团队有一个重要目标，就是想办法不仅要做好那些很经典的事情，比如我们如何更好地与产品团队互动，比如……1:36:03 But really specifically, like what's hard on platform is platform in some sometimes in our like public narrative, like our platform business is a really large business, but it can kind of get lost in between research and policy and all this work that we're obviously doing. That's like our most frontier important work. And then our applications like Claude Code and Cowork and all the things happening over here. When you think... about Anthropic and you think about our public narrative, those are some of the things that often come to mind and less so that we're running this like massive developer platform business that's really important and so many people are building on top of. And so a big goal that we have with our DevRel team within platform is figuring out not only like the really classic, like how do we better engage with like, you know, product, like

1:36:46 人们正在构建什么，以及我们如何把反馈融入产品，诸如此类的事情。但其中很大一部分其实是在寻找我们的声音，弄清楚我们的品牌应该是什么样、我们的叙事应该是什么样，以及我们在这个社区里究竟要成为怎样的角色，而不是在正在发生的各种事情中迷失方向。我觉得这很有意思，因为我们开始为 DevRel 团队打造的品牌，某种程度上是因为 Lance 加入了我们的团队——而是你把我和 Lance 联系起来的。Lance 在技术方面造诣很深，他做 DevRel 的方式很大程度上就像是在教学，对吧？他写了很多关于内存、成本管理之类的内容，而且会深入探讨。他会构建评测，也会产出很多技术性很强的内容。我很喜欢这个理念：让我们的 DevRel 团队成为思想领袖，为身处这个仍处于萌芽阶段、极具挑战且不断演进的领域中的人们提供引领。1:36:46 what people are building and how we get feedback back into the product and these sorts of things, but a lot of it is just like finding our voice and finding what our brand needs to be and what our narrative needs to be, and like who we even are going to be in the community and not kind of get lost in all of the things that are happening. And so that I think has been interesting because I think the brand that we've started to spin up for our DevRel team, in part because actually Lance joining our team was you connecting me with Lance, but Lance is highly technical in the way that he goes about DevRel is very much like teaching, right? Like he writes a lot about memory and managing costs and these sorts of things. And he goes really deep. He builds evals. He has a lot of really technical content that he puts together. And I really like that idea, that idea of our DevRel team being kind of like thought leaders for people within what is a really nascent, really hard, really evolving space.

1:37:36 教育者、思想领袖、各种事物的讲解者。我特别喜欢另一种角色，叫“大师级拼搭师”：我会用这个平台亲手打造东西，向你展示它能做到什么。然后你就会说：“哦，我不知道 Claude 平台还能做到这个。”接着你就会去做。这种做法是从乐高那里借鉴来的。乐高是怎么卖乐高积木的？他们有大师级拼搭师。他们用乐高积木搭出各种东西，让你看了就想：“我也想做出这样的东西，我想成为这样的人。”于是你就去做了。不错，太棒了。是啊。还有 CJ，我们团队里的另一位成员，他也很棒。我觉得他也做了很多类似的事情，比如编写实用指南、发布示例，并在 X 上积极与社区互动。1:37:36 Educators, thought leaders, explainers of things. There's another version of this I really like called the Master Builder, which is I will build the things with the platform to show you what's possible. And then you'll be like, oh, I didn't know that was possible with the Claude Platform. And then you just go. And this is a thing stolen from Lego. because Lego, how do they sell Legos? They have master builders. They build things with Legos and you're like, I want to do that. I want to be like this person. And then they do that. Nice. That's awesome. Yeah. And CJ, the other person on our team, who's really great as well, I think he also does a lot more of that sort of like, you know, writing cookbooks and like putting out examples and really engaging with the community on X.

1:38:16 Cookbook 挺有意思的，对吧？比如说，API 在 2023 年推出的时候——就把它叫作面向语言模型的 API 吧——这非常重要。现在嘛，我也说不准。是啊，有意思的是，你不太会处在那种想要直接复制粘贴代码、然后拿来用的情境里，对吧？但我确实觉得，它能向大家展示哪些事情是可能实现的。还有，具体的用例到底是什么，对吧？以及要怎么才能实现它？不过我明白你的意思，就是说代码本身可能没那么重要。我们团队里有个工程师，很久以前写过一份关于计算机操作的 cookbook，直到今天这事儿还一直让他挥之不去，因为那东西复杂得不得了，简直是个庞然大物。我觉得当时有不少人确实从中受益了。1:38:16 Cookbooks is funny, right? Like when APIs launched in 2023, let's call it for LMs, very important. Now, I don't know. Yeah, well, it's interesting because you're less in the spot where you'd like copy paste the code and like use it, right? But I do think it's like it shows the art of the possible for people. And just like what even is the use case, right? And like how can you go about accomplishing it? But I hear you that like, you know, the code itself is maybe less. Like we for the longest time had this like there's an engineer on our team who wrote our cookbook on computer use. and it like haunts him to this day because it was like massively complex and like, you know, kind of a beast of a thing that I think, you know, a lot of people got use out of at a time

1:39:00，像是，我要构建一个计算机使用功能的实现，还要处理所有截图以及消息 API 之间发生的各种事情，对吧？是啊，我们当时参加了你们在办公室举办的黑客马拉松。对，对。那时候你就需要这种东西。现在嘛，更多的是，我们有一项技能，比如 Cloud API 技能。它真的非常出色，能帮助你在 Claude Code 或其他编程智能体中构建高质量的集成。如今，像“我要构建一个计算机使用集成，该怎么做？”这类事情，我觉得——或者说希望——已经很少有人会去翻那本庞大又复杂的 cookbook 了，大家更多是加载这项技能，让编码智能体开始工作。是啊，或者直接把它交给参考架构。是的，实际效果要好得多。比如说，在很多事情上有所提升，或者看到这种情况……1:39:00 when it's like, I'm going to build a computer use implementation and I'm going to deal with all the screenshotting and all the things, right, that are happening between the messages API. Yeah, we're there for the hackathon that you guys launched in the office. Yeah, yeah. And like that was the era when you like needed to have that sort of thing. And now, you know, it's much more about We have a skill like the Cloud API skill. It's like really, really excellent at helping you either within Claude Code or any other coding agents to build a really high quality integration. And now like these sorts of things like I'm going to build a computer use integration. Like how do I go about doing that? Like very few people I think slash hope are reaching for the big gnarly cookbook and more so they're loading up the skill and encoding agent and working. Yeah, or they're just putting it to a reference architecture. Yeah, it actually does much better. like increasing or seeing that for a lot of the things

1:39:45 你可能会希望在不同地方采用不同的实现方式，通常是在 harness 层，但也可能有其他变体。你只要让 Claude 使用它，然后说：“这是我想要的参考架构。”对，然后你就把它构建出来。有个有意思的事实：帮助 SQL 基于 API 进行构建的 Claude API 技能，是由我们的 DevRel 团队负责的。所以这是 Lance 构建的。理应如此。没错。DevRel 最能发挥杠杆作用的方式，就是提炼出所有将融入这些智能体的最佳实践。是啊。我之所以这么说……这事有点奇怪，因为大家把我看作 DevRel 人士，或者这方面的权威。但我内心有一部分就是想告诉你：你不需要 DevRel。你只要做得足够好，就不需要它。对。或者你可以让工程师和领导者成为公众面孔。1:39:45 that you might want to have different implementations on, typically at the harness layer, but there could be other variants. You just point Cloud at it and you say, this is the reference architecture I want. Yeah, then you build it. Fun fact, the Cloud API skill that helps SQL build against the API is owned by our DevRel team. So that was built by Lance. As they should. Right, exactly. This is the most high leverage form of DevRel is to distill everything that is the best practices that's going to go into those agents. Yeah. One of the reasons, It's weird because I'm regarded as this DevRel person or authority. But there's some part of me that just wants to tell you you don't need DevRel. You get so good you don't need it. Right. Or you turn the engineers, the leaders, into the public faces.

1:40:30 因为你真正想做的，无非是让大家成为对外的门面，告诉大家我们这里是怎么做事的。大概就是这样。你们是 Anthropic，不需要别人听说你们。所以我觉得，对我来说，Boris 刚进入公众视野时，感觉就像是在说：“嘿，我是 Boris，是 Claude Code 的创造者，等等等等。我们就是这么做事的。”没错，他有自己的本职工作，但实际上也在发挥这种作用。那为什么大家不都这么做呢？是啊，我的意思是，这是个很好的问题。有些团队确实做得非常好。我觉得 Cloudflare 团队就是个很好的例子。他们不停地发布内容。一刻不停。是啊。对我来说，这其实是优秀管理者的一个非常明显的标志：他们都乐于做自己的工作。他们都愿意公开表明身份。他们没有辞职。是啊，完全如此。有意思的是，我觉得……1:40:30 Because all you really are trying to do is public faces and here's how we do things here. And that's kind of it. You're anthropic. You don't need people to hear about you. So I think to me, when Boris burst onto the scene, it seemed like a very like, hey, I'm Boris and the creator of Claude Code, blah, blah, blah. That's how we do things. Yes, he has the day job, but he's also doing that functionally. So why doesn't everyone just do that? Yeah, I mean, it's a really good question. And there are teams that have done this really well. I think the Cloudflare team is a really great example. They're all pumping out stuff. Non-stop. Yeah. And to me, that's actually a very visible sign of a good manager is that they're all happy at their jobs. They're all like publicly identifying. They're not quitting. Yeah, totally. It's interesting because I think

1:41:17 这其实是我们正在努力做得更多的一件事：让平台团队、我们团队，以及那些愿意走出去、参与社区活动、发表演讲、活跃在 X 上或其他任何地方的人，都能参与进来，对吧？我觉得对我们来说，问题就在于时间有限，而且不太清楚怎样才能把这些事情做好、做到高质量。所以在某种程度上，我认为 DevRel 团队就是这类工作的专家。他们可以每天从早到晚做这些事，也可以帮助团队其他成员思考，如何利用有限的时间，把这类事情也做好、做到高质量。没错。所以我把这称为将 DevRel 拆分为前台和后台。通常做这份工作的人要负责整个流程。1:41:17 this is actually something we are trying to do more of with the platform team and with our team and folks who, you know, can kind of just get out there and like be part of the community and be speaking and be on X and be wherever they are, right? I think the problem for us is just like having time and kind of knowing the right way to do that stuff at high quality. So I see the DevRel team in some ways as just being like the experts in doing this sort of stuff at quality. They can do it all day, every day, and they can help enable the rest of the team to think about how to do this with the small amount of time that they are able to spend on this sort of stuff to do it at high quality, too. Yeah. So I call this kind of the unbundling of DevRel into a front office and a back office. So like a typical person who does this job does the whole thing.

1:42:02 从头到尾，他们基本上都是自主运作的。但实际上，你可以搭建这样一个后台团队，让产品负责人和工程负责人真正站到聚光灯下，只需要跟他们约定一个简单的流程：好，你和我坐下来聊 30 到 60 分钟，把想法都讲出来，然后就结束了。你回去继续工作。之后我们会帮你代笔什么的。总之，那个由制作人员、写作者之类组成的团队，就是为其他人提供这类服务。我觉得，说实话，这才是这类事情未来的发展方向，因为大家其实并不想听 DevRel 说什么，他们想听 Lance 来说；你知道的，还有很多像 Tariq、Lydia 这样的人——你认识 Lydia 是通过 Bun 的。1:42:02 Start to finish, they're kind of autonomous. But actually you can sort of build up this back office that lets your product leads and engineering leads actually take the spotlight with a simple contract of like, okay, you sit down with me for 30 to 60 minutes and we'll do the brain dump and then you're done. You go back to work. And then we'll like ghostwrite for you, whatever. But like that team of like production people, writing people, whatever is sort of provides the service to the rest of the people. and I think that to me that's like kind of the future of this stuff to be honest because people like don't actually really want to hear from DevRel they want to hear from Lance and like you know and you have like lots of Tariq you know and Lydia you got Lydia through Bun

1:42:45 你知道得很清楚，但他确实在做这份工作，他就是做这份工作的。所以你会希望有更多这样的人，因为大家想听产品团队里的人说，而不是听那些拿钱出场的人——付费代言人。我经常会遇到这种情况。这也没什么，那是一份工作，总得有人来做。合理地说，你不能指望做工程的人也擅长这件事，所以应该请专门做这件事的人。专业技能本身就是一种技能。我觉得可以逐渐形成后台和前台的分工。对，真的很棒。是啊，我打个比方：每个人都招人，但你们有一个人力资源部门。人力资源部门让招聘成为可能。确实。你负责招聘。没错。对，很有道理。好，这就是我刚才的一番高谈阔论。1:42:45 you know exactly but like he he does the job he does the job and so like you want more of those because people people want to hear from people in the product not like who are like paid shows I paid spokesperson. Which I get a lot of. And that's fine. That's a job. Somebody has to do it. Reasonably, you can expect people who are working on engineering to not be good at it. So you should pay someone who specializes in it. Specialty is a skill. I think there is a back office, front office separation that can emerge. Yeah, that is really cool. Yeah. And my analogy would be like everyone hires, but you have an HR department. HR department enables you to hire. Indeed. You hire. Yep. Yeah. Makes a ton of sense. Cool. That was my soapbox.

1:43:31 最后的问题。你们一直以来都非常慷慨。有什么临别赠言或行动号召吗？Opus 5.5 有什么酷炫之处？有什么值得大家尝试、能和 Opus 5.5 一起突破边界的事情吗？你们使用它的时间最长。是啊，我的意思是，它的性价比确实很高，表达也更简洁。我们提到的那些特点，整体体验真的非常、非常棒。我们玩得很开心。昨天对我们来说是很开心的一天，因为终于又有这种感觉了。我收到了一大堆朋友和其他人的短信，他们都说：“这模型太棒了，谢谢你们。”比如我以前的一位老板给我发消息说：“Opus 5.5 刚刚在一次事故中救了我们一把。”对吧？那感觉就像，好，太好了，我们终于又回来了。这确实是我们现在的感受。所以我觉得，如果你之前有些想尝试的事情，但成本一直是你最关心的问题，那现在可以试试看。1:43:31 Closing questions. You've been very generous over the time. Any parting thoughts or calls to action? What's something cool on Opus 5.5? What's something people should try to push the bounds with you, Opus 5.5? You guys have been using it for the longest. Yeah. I mean, it's really cost efficient. It's less verbose. All the things we mentioned, the vibes are really, really high. It's been fun. Yesterday was a really fun day for us because finally again, And I got like a whole slew of texts from friends and people who were just like, amazing model. Thank you. Like one of my old bosses messaged me and he was like, Opus 5-5 just saved our butts in an incident. Right. And it's like, OK, great. Like we're back is really the energy that we feel. And so I think things that like maybe you wanted to experiment with something and like cost was something that was front of mind for you.

1:44:18 对。还有 token 效率之类的问题，可能会让你不太敢真正放手让模型跑起来。我觉得现在有了 Opus 5.5，你应该重新尝试解决这类问题，因为我认为你能比使用那些成本效益没那么高的模型时走得更远。我觉得你应该试试多智能体架构。我想你会发现它非常好用。顺便问一下，还有什么其他提示吗？我的意思是，多智能体这个概念很宽泛。没有。我想我们会非常期待看到大家构建出什么。不过，没错，效果应该会特别好。是啊，比如说，让角色不再被严格定义，而是由它们彼此分配。对我来说，这就是新鲜之处。1:44:18 Right. and token efficiency and these sorts of things, that kind of might have held you back from really firing it off and letting a model run. I think those are the sorts of problems you should return to now with Opus 5.5 because I think you'll be able to get a lot further than you maybe otherwise would have been able to with models that were less cost-effective. I think you should try something with a multi-agent setup. I think you will find it to be very good. Any other hints, by the way? I mean, multi-agent is very broad. Nope. I think we'd be very curious to hear what people build. But yes, it should be extra good. Yeah, including, let's say, roles that are not strictly defined, but they are assigned among themselves. Right? Which is, to me, the new thing.

1:45:00 你可以分配角色，也可以随时分配角色。你可以给 token 安排任务。但这有点像是他们在给自己安排任务。是的。我想，谈谈多智能体吧？比如，我们应该使用子智能体吗？你觉得……人们该如何看待多智能体？从某种意义上说，这相当不错：基本上，通常你会有一个主负责人，然后把一堆不同的事情拆分开来。只要你有一项任务，认为它大体上可以并行处理，而且各部分共享统一的上下文，我觉得就能获得很大收益。当然，这取决于你要解决的问题。但如今，这越来越成为我们处理很多这类事情的一种更优方式。尤其是每个子智能体——或者你想怎么称呼这些子负责人都行——都是异步运行的，彼此之间不会互相阻塞。1:45:00 You can assign roles. You can always assign roles. You can give tokens jobs. But this is like, they're giving themselves jobs. Yes. I guess, say more on multi-agent, right? Like, should we use sub-agents? What do you... How should people think about multi-agents? This is pretty good in a sense of like basically, generally, you want to have one lead and you want to like split up like a bunch of different stuff. And as long as you can have a task that you think could be generally paralyzable and they share uniform context across it, I think you will get a lot of gains. Now, of course, it depends on the problem that you're pushing at. But like increasingly, this is a very superior way for us to handle a lot of these things. And especially if every sub-agent is, or whatever you want to call these sub-leads, they are asynchronous and they're not blocking of each other.

1:45:47 也就是说，在架构方面，我觉得你只需要关注这两点。除此之外，就放手去做，看看 Claude 会为你设计出什么。而且我认为，越来越多的架构设计都会朝这个方向发展。Claude 会变得越来越聪明。它会学会如何找到办法来拆分自己、克隆自己，并根据合适的架构调整自己。你只需要防止它出问题。你只需要确保它能够并行处理。所以它需要共享状态，而且不能阻塞它的子代理。只要守住这两条原则，说实话，我觉得大家就应该多尝试，甚至可以试试一些非常大胆的做法。我也不会惊讶于有人基于此做出一些非常有意思的东西——也许他们觉得有个棘手的问题特别烦人，一直在努力解决，而这套方法说不定真的能轻松搞定，甚至一次就成功。太棒了。1:45:47 That is, like, the only two bits I would say you should care about in your architecture. Other than that, like, let it rip and, like, see what Claude designs for you. And increasingly, I think a lot of this architecture stuff will fall into this direction. Like, Claude will get smarter and smarter. It's going to learn how to figure ways to, like, split itself, clone itself, adjust itself to the architectures that make sense. You just need to prevent it. You just need to make sure it can, like, parallel process. So it needs a shared state and it needs to like not block its sub agents. And as long as you contain those two principles, honestly, I think people should experiment and probably try some really crazy stuff. And I wouldn't be surprised to see some like really interesting things on top of that, that maybe they felt like there was some intractable problem that was really annoying and they'd keep pushing for. like this could actually probably just like crush it and maybe one shot or something. Excellent.

1:46:30 好了，大家去构建吧。不错。不错。1:46:30 All right, everyone go build. Cool. Cool.