微软杰出工程师 Stephen Toub：为何以及如何将整个代码库迁移到 Rust

简要描述Brief Description

微软杰出工程师：为什么以及如何将整个代码库移植到 Rust | Stephen ToubMicrosoft Distinguished Eng: Why And How To Port Your Entire Codebase to Rust | Stephen Toub

目录Table of Contents

• 简介
  Intro
• 为什么 Copilot 运行时需要移植到 Rust
  Why the Copilot runtime needed a Rust port
• 他为什么选择 Rust
  Why he chose Rust
• 讨论发布策略
  Discussing rollout strategies
• 他如何发现回归问题
  How he caught regressions
• 指导智能体
  Guiding the agents
• 在上线前发现回归问题
  Catching regressions before prod
• 他使用了哪些模型
  Which models he used
• 最终收益与成本
  The end benefit and costs
• 何时进行全面重写
  When to do a full rewrite
• 迈向杰出工程师之路上的职业经验教训
  Career learnings on the way to Distinguished
• 最佳技术书籍推荐
  Top technical book recommendation
• 给年轻工程师的建议
  Advice for young engineers
• 结尾
  Outro

简介Intro

总计约有 1360 亿个 token，费用约为 12 万美元。这位是微软杰出工程师 Steven Tob，他独自用 Rust 重写了一个 40 万行的代码库。我向他详细请教了他从这个过程中学到了什么。过去负担不起的项目，现在已经完全可行了。All told, it was about 136 billion tokens. It came out to approximately $120,000. This is Steven Tob, distinguished engineer at Microsoft, who single-handedly rewrote a 400,000line codebase in Rust. And I asked him all about what he learned from the process. These projects that were once unaffordable are now quite doable.

你是如何在过程中发现并修复回归问题的？How did you discover and and fix regressions along the way?

我逐行阅读并不能避免 bug。事实上，让智能体逐行阅读要有效得多。如果让我重来，有件事我会做得不一样——这是完整的一期节目。我想把这次对话的重点放在你为 GitHub Copilot 运行时所做的移植上，也就是基本上把它重写成 Rust，因为我认为，随着近期这些模型的发展，这类新型项目现在已经可行了。所以我想，业界会有很多人尝试类似的事情，比如把他们的遗留代码库迁移到性能更高的语言上。Me reading every line would not prevent bugs. In fact, agents reading every line is a lot more effective. Something I would do differently if I had to do it again was here's the full episode. I wanted to focus this conversation around the port that you did for the GitHub copilot runtime and basically rewriting it into Rust because I think this is a new type of project that is now feasible with the the recent models. So I imagine a lot of people in industry are going to uh attempt something similar maybe with their legacy code bases into some more performance language.

为什么需要将 Copilot 运行时移植到 RustWhy the Copilot runtime needed a Rust port

我想先问问你，这个项目是什么？你们为什么要用 Rust 把所有东西都重写一遍？你们当时要解决什么问题？ 是啊，这是个很好的问题。尤其是我觉得，人们有时会以为，最近把东西重写成 Rust 是一种潮流。所以他们会想，呃，或者我也被问过一些带有这种假设的问题，比如：“哦，你这么做只是因为……”，或者“你能这么做，所以就这么做了”。但事实完全不是这样。So I want to start off with asking you what was the project and why did you rewrite everything in Rust? What was the problem that you were solving? Yeah, that's a great question. Especially because people sometimes I think assume that rewriting things into Rust is kind of all the rage these days. And so people assume uh or I've had questions that assume like oh you're doing this just because or like you can and so you will and uh that is not at all what this was.

你知道，当前这个项目本来可以以某些形态存在，在那些情况下，重写它根本不会成为一个需要讨论的话题。但主要的推动因素是，众所周知，AI 一直是个变化极快的目标。过去几年里，它所带来的能力飞速增长，所以在 GitHub 和微软各处，有许多项目从零开始发展起来，将 AI 能力融入其中。There were you know there are certain shapes the current the project could have existed in where rewriting it wouldn't have been a conversation at all but the impetus is largely that AI has been this incredibly fastm moving target as we all know and over the last few years the the capabilities that it has enabled has skyrocket skyrocketed and so [clears throat] across GitHub across Microsoft there are all of these projects that have um kind of blossomed from the ground up to incorporate AI capabilities into them.

它们基本上都是从“哦，我们可以调用这些 LLM”开始的。接着又想到：“哦，我们得在外面包一层东西。还得不断往上加功能、加东西。”就这样逐步构建起来，最终形成了如今所谓的智能体框架、智能体循环，或者智能体运行时——随你怎么称呼。嗯，最后我们有了很多这样的东西，因为每个人都多少有自己的内部实现。这不是个好局面，尤其是当你希望这些实现都能做到业内最佳、始终走在前沿时更是如此。在 AI 这个领域，新模型每周都会推出，甚至每天都有。And all of them basically started from oh we can make these LLM calls. Oh, we have to wrap a little bit of you know goo around it. We have to add more around it and more around it and building up effectively what is now called agentic harnesses or aentic loops or aentic runtimes whatever you want to call them. Um, and we ended up with a lot of them because everyone kind of had their own homegrown implementation. And that's not a good place to be, especially when you're trying to have all of them kind of be best of breed and be on the forefront. And in this world of AI, like there are new models coming out weekly, if not daily.

你知道，就在几周前，OpenAI 和 Anthropic 都接连发布了一堆东西，彼此较劲、争着压对方一头。嗯，一个月前还是研究成果的新技术，如今每个 harness 都得具备，否则马上就会过时。Like, you know, a couple weeks ago, there were all these releases from both OpenAI and Enthropic, you know, oneuping each other. And um there are new techniques that a month ago were research and today every harness has to have or they're immediately out of date.

而且你肯定不希望有无数种不同的实现都需要持续更新。所以你真正需要的是少数几个——最好是一个——可扩展的工具框架或运行时，让所有人都能使用，并根据自己的需求进行定制和扩展。不过，当时 Copilot CLI、GitHub 应用，以及 Copilot 云代理所使用的 Copilot 运行时——比如你在 GitHub 上让 Copilot 帮你做事或进行代码审查时用到的运行时——是用 TypeScript 编写的，使用 Node 和 V8。这就影响了它作为可嵌入库的适用范围。And you don't want this myriad of different implementations all needing to stay up to date. And so you really want a small number, ideally one, but a small number of extensible harnesses or runtimes that everyone can use and customize and extend to their needs. But the the Copilot runtime that was being used for the Copilot CLI and the GitHub app uh and uh Copilot cloud agent like when you're on GitHub and you ask Copilot to do stuff or code review like um that was written in Typescript um using Node using V8 uh and that has implications for where you can use that as an embeddible library.

所以，要让这个运行时能在我希望它可用的所有地方运行，主要障碍之一从根本上说在于它的实现方式，以及它所使用的框架和运行时等等。也就是说，重点不是转向 Rust，而是从 TypeScript 和 Node 转向一种更容易嵌入、效率更高、工作集更小、吞吐量更大的方案；这样无论各种环境是用什么语言实现的、使用什么框架，都能更容易地集成进去。So as a like one of the main blockers for that runtime being able to be used everywhere I wanted to be usable was fundamentally what it was implemented in the frameworks it was using the runtimes it was using and so on and that that basically not moving to Rust but rather moving away from Typescript and Node to something that was more embeddible had better efficiency, lower working set, faster throughput, um better able to be consumed into any myriad of environments regardless of what language they're implemented in, regardless of what frameworks they're using.

嗯，这基本上就是促成这件事的契机。正如你刚才提到的，人们想要进行这类移植或迁移，原因有很多。这在整个行业里极为常见：无论是从一个已弃用的框架迁移到更新的版本，从一种语言迁移到另一种语言，还是把老旧代码库中那些笨重不堪、你知道、乱成一团的意大利面式代码，改造成更整洁、现代、焕然一新的代码。这类迁移和移植随处可见，但很多时候，人们之所以不着手做，只是因为成本太高、耗时太久。一两年前，我们也会面临这种情况。但有了现代模型和现代执行框架后，这些曾经难以负担的项目，现在只需很短的时间就能完成。Um that was basically the the catalyst for for this. And to your point earlier, there are any number of reasons folks might be looking to do these kinds of ports or migrations. They are incredibly common across the industry. Whether moving from one deprecated framework to a newer version or from one language to another or from a legacy codebase that has become an unwieldy, you know, mess of spaghetti to something cleaner and modern and fresh. These migrations and ports are everywhere and a lot of times people just don't take them on because they are too expensive. They take too long and we would have been in that situation a year or two ago as well. But with modern models and modern harnesses, these projects that were once unaffordable are now quite doable in a in a very small period of time.

他为什么选择 RustWhy he chose Rust

在你写的那篇帖子里，我看到有一行写着：“我一开始并不是打算转向 Rust。我一开始是想摆脱 Node.js 和 V8。”呃，这让我想知道，如果你并非必须转向 Rust，你有没有考虑过其他替代方案？为什么最后特别选择了 Rust？In the post that you wrote, I saw that there was a line. It said, "I didn't set out to move to Rust. I set out to move away from Node.js and V8 and uh that makes me wonder if you didn't need to go to Rust did you consider other alternatives and why did you end up going to Rust specifically?

所以我做的第一件事，基本上就是借助智能体快速搭建原型，因为现在你可以用不同的运行时，以非常快的速度创建原型，嗯，仍然使用 TypeScript，只是运行时不同，比如 Node、Bun 之类的。呃，我想我还用 Python、Go、C、Java 和 Rust 做了原型。嗯，我还在其中一个系统里搭建了一条非常精简的端到端链路，后端由一个本地的、嗯，聊天补全模拟服务器提供支持。每个原型大概有 5 到 1 万行代码。呃，然后我从端到端性能、嗯，工作集吞吐量等方面评估了它们，也看了看如果施加大量负载，系统能达到什么规模。So my the very first thing I did was basically create a prototype agentically because you can now create prototypes super fast in um still with TypeScript but with different runtimes. So, you know, node versus bun versus whatever. Uh, as well as I think I did Python, Go, C, Java, and Rust. Um, and created a very small end-to-end sort of steel thread through uh, one of these systems kind of backed by a local um, chat completion, you know, dummy server. um probably 5 to 10,000 lines for each of them. Uh and looked at them in terms of endto-end performance uh in terms of uh working set throughput uh what kind of scale that we could get if we threw a lot of load at it.

嗯，然后还有就是，它是否真的能满足需求，以及我们需要做些什么。我们目前有六种语言的 SDK，它们是构建在 Copilot 运行时之上的轻量绑定。要让所有这些 SDK 都能使用它，需要做些什么？尤其是现在，我们不想强迫——或者说，以前我们不想强迫——Node 进入使用方应用程序的进程里，因此 SDK 基本上会另外启动一个进程来承载会话的实际执行。不过，我们也很希望能提供进程内运行的选项，就像世界上几乎所有其他库一样。Uh and then also what kind of whether it could actually meet requirements and what it would take for someone for us with we have um currently have six language SDKs that are thin bindings on cross on top of the copilot runtime um what it would look like for uh this to be usable from all of those and in particular right now because we don't want to force or previously because we didn't want to force node uh into the consuming applications process the the SDK basically spawned another process to host the the actual execution of the sessions, but we'd really like as an option for it to be in proc as well, like pretty much every other library in the world.

嗯，所以基本上就是根据这些性能目标和约束来评估所有这些语言和运行时，再看看数据，了解哪些方案奏效，以及效果如何。Um, and so basically evaluating all of these languages and run times uh against these performance goals and constraints and just looking at the data to see what worked and then how well it worked.

总体而言，从性能和可嵌入性目标来看，TypeScript 和 Python 很快就被排除在外了。嗯，Go 的性能相当不错，而且代码库也很简洁。不过它并不是为可嵌入性而设计的。它的强项并不是互操作，对吧？你确实可以做到，但多少有点逆着它的设计来。Java 在这方面也不太理想。JavaScript 还可以。最终的两个主要候选方案是 C，尤其是采用原生 AOT 的 C#，这样就能提前编译成汇编代码；另一个是 Rust。从性能角度来看，两者在几乎所有指标上都不相上下。有些方面 Rust 稍好一些。And for the most part, TypeScript and Python were very quickly out uh from uh kind of all of the performance perspective and the the embedability goals. Um, Go had pretty good PF. Um, [snorts] and you know, had a pretty clean code base. Um, but it's not really designed for embedability. It's not really like interop is not its raet, right? It's it's uh you can do it, but you're kind of cutting against the grain. Um, and then and Java is also not not great for that. Um, JavaSc was okay. Uh, the two main contenders ended up being C, in particular C# with native AOT, um, so that it could be compiled ahead of time down to assembly and, uh, and Rust. And from a PF perspective, uh, they were both very much on par with each other, uh, across pretty much all the metrics. In some places, Rust was a little better.

在某些方面，C# 略胜一筹。嗯，回到你之前的问题，如果这个项目一开始就是用 C 写的，我们就不会讨论这个了。我们会继续用 C，也没有理由重写它。但事实并非如此，所以最后还是要看这些标准。从性能角度来看，两者表现都不错。嗯，从可嵌入性角度来看，两者也都不错。嗯，C# 的互操作能力也很强。最后主要还是要考虑一些比较软性的因素。比如，GitHub 上还有其他用 Rust 构建的项目，而用 C 实现的项目则少得多。GitHub 上懂 Rust 的人也比懂 C 的人多。In some places C# was a little better. Um, and so to your your you know question earlier, if the project was already in C, we wouldn't be having this conversation. We would have stayed with it. There would have been no reason to rewrite it. But it wasn't. And so that it came down to these criteria. So from a perf perspective, they were both they both look good. Um, from an embeddability perspective, they both look good. Um, C# has very strong interop characteristics as well. And then it came down to primarily softer factors. So uh there are other projects around GitHub in particular that are built in Rust and very few that are implemented in C. There are more people around GitHub that know Rust and fewer that know C.

显然，在微软内部情况并非如此，不过在 GitHub 内部是这样。嗯，考虑到这类标准，我们最终选择了 Rust。That's not the case within Microsoft obviously but uh within GitHub. Um and so factoring in those kinds of criteria we ended up going with Rust.

我知道 Anders 最近做了……我想这个项目启动得比你们的项目稍早一些。他和团队一起把 TypeScript 编译器从用 TypeScript 编写移植成了用 Go 编写。我很好奇你有没有和他交流过心得。I know Anders recently did I think the project started a bit before your project started, but he did a port with the team of the TypeScript compiler from type being written in Typescript to uh being written in Go. I was curious if you compared notes with him.

当然。是的。这是个很好的例子，说明不同项目可能得出不同结论，而且都没错。并没有要求必须把 TypeScript 编译器嵌入其他应用程序中。反过来说，编译 TypeScript 这项工作的性质，与向 LLM 发送请求、调整提示词、处理沙箱以及诸如此类工作的性质截然不同。这些项目的本质完全不同。Definitely. Yeah. And this is a great example where that different projects can have can reach different conclusions and and be right. there's no requirement to embed the TypeScript compiler into other applications. Um, and conversely, the shape of the um the shape of the work that is type compiling TypeScript looks very different from the shape of the work that is um sending requests to an LLM and manipulating prompts and dealing with sandboxes and and all that kind of stuff. um the nature of the projects was fundamentally different.

不过，有意思的是，过去几周里出现了好几个由智能体完成的移植项目。你知道，去 X（或者现在叫什么来着，Twitter）上搜一搜，就能找到有人把 TypeScript Go 编译器移植到 Rust 的例子。他们借助现代模型，移植得非常快——考虑到代码库的规模和整洁程度……Um although interestingly in the last few weeks there have been several uh uh agentic ports of uh which you can you know if you search X or Twitter or whatever it's called now um uh you know you can find people that are porting the TypeScript Go compiler into Rust um and doing so very fast with modern models uh given the size of the codebase and the cleanliness of the codebase and

讨论发布策略Discussing rollout strategies

呃，所以看到这个还挺有意思的，挺有意思的。Uh so it's interesting interesting to see.

所以，一旦你确定这个项目有其动机，而且要解决的问题也很明确之后，我想问问你们的迁移策略，因为我在不同项目中见过不同的策略。比如，你之前提到了 Bun。看起来他们基本上是从头搭建了一条庞大的新分支，整个分支完全用 Rust 编写，然后等到稳定版 Rust 发布时，一次性把所有东西都发了出去。而我读到你们的做法时，感觉你们的迁移要渐进得多，是一点一点完成的。你当时是如何权衡这些不同策略的？又会在什么情况下选择其中一种，而不是另一种？So once you decided that there was motivation for the project and it was clear problem to solve. I wanted to ask you a little bit about the migration strategy because I've seen different strategies in different projects. So for instance you mentioned bun earlier. Bun it looked like they kind of built up a giant basically a new branch that's just completely entirely in rust. and then they shipped all in all at once uh stable rust release and they did it all in one one go whereas when I was reading what you had done it looked like a lot more incremental and little by little. Uh how did you think through the the different strategies and uh when might you select one versus the other?

关于 Bun，最有意思的是，我很喜欢写那篇 Bun 博文的 Jared。我特别喜欢他写的那篇文章。那篇文章真的让人很兴奋，因为我之前不知道他们正在做这次移植。那篇文章是在我们的迁移进行到一半左右时发布的，看到另一个项目规模大致相当——代码量差不多是同一个数量级——却是用另一种语言实现的，感觉真不错。他们用的是 Zig，而不是 TypeScript。不过，他们也认为这样做是合理的，而且已经完成并合并了这项工作。文章里提到，他们大概花了 11 天，就让约 98% 的测试通过了，或者差不多是这个比例，这真的很酷。正如你所说，他们做这件事的方式有点像是大规模地重写一切，然后直接推进。The interesting thing about bun first I loved uh Jared who wrote the the bun post. I loved the post that he wrote. It was really exciting because that post I wasn't aware that they were doing that port and then that post came out about halfway through our migration and it was really cool to see that this other project of approximately you know order of magnitude same amount of code in implemented in a different language. they were in Zigg rather than TypeScript, but um that they had, you know, thought that this was a sensible thing to do and had landed it. And in the post, they talk about it taking, I think, 11 days for them to get to like 98% of their test passing or something like that, which is really cool. And to your point, like they did it sort of in a an onmass like rewrite the world and and go.

这很有意思。我没有参与这个项目，所以不知道幕后发生的所有细节，但有一点让我觉得很有意思：如果你看他们的 npm 页面，会发现他们几乎每天都在发布构建版本——我想，基本上一直到四月中旬，他们都在发布大量 Canary 构建，持续了很长一段时间，之后就停了。然后他们提到用了 11 天移植代码。从零开始，在 11 天内把一个 50 万行的代码库移植到 Rust，并且通过 98% 或 99% 的测试，这真是了不起的壮举。不过，之后至少据我所见，他们又过了两个半月才发布了任何形式的 npm 新版本。It is interesting. I I am not involved in the project so I don't know all the details of what's going on behind the scenes but it is interesting to me that um if you look at their npm page they were shipping near daily builds uh I think you know a lot of canary builds basically through midappril like for a long time daily builds and then midappril it stops uh and then they talk about doing the you know the 11 days of porting which is an impressive of feet to get from nothing to this, you know, a 500,000line codebase ported to Rust in 11 days and have a passing 98 or 99% of the tests. But then they didn't have another npm release of any kind, at least as far as I could see, for another two and a half months.

呃，当时有这么一段时间：没错，核心移植花的时间不长，但之后似乎还进行了一些稳定化之类的工作，所以两次发布之间隔了3个月。有意思的是，这差不多也正是 Copilot 所花的时间——我花了14周半来移植 Copilot。所以，造成两者不同的原因有很多，但从根本上说，感觉耗时差不多，只是我们选择了不同的做法。在那篇文章里，呃，Jared 提到，他不想采用 Copilot 那种渐进式的方法，因为那会带来一大堆临时的中断性杂活。Um so there was this window where yes the the core port took a small period of time but it seems then like there was some aspect of stabilization and things like that happening such that there was this 3mon gap between releases um which interestingly is also approximately the same amount of time that it took co-pilot you know the 14 and a half weeks that I spent uh porting co-pilot so you know there's lots of reasons why they differ but fundamentally mentally it seemed to be about the same amount of time and we just chose sort of different ways of going about it. In the post, uh Jared makes a point about um not wanting to do the incremental approach that Copilot did because it would incur lots of temporary interrupt junk.

他说得没错，确实如此。嗯，我们选择那种做法时，完全清楚自己会面对什么，因为当时我们受到各种限制。And he's right, it did. um we just chose that approach knowing full well what we were getting ourselves into because of the sort of the constraints that we were under.

所以，你知道，对于 JavaScript 运行时来说，修复 bug、推出新功能之类的事显然都很有价值。但你可以暂停三个月，世界也不会因此天塌下来，对吧？三个月前的 JavaScript 和今天的 JavaScript 并没有多大区别。你也可以采取某种“彻底重写一切”的做法：先把所有事情都暂停，等完成之后再继续发布。但在 2026 年夏天的 Agentic Harnesses 领域，如果你暂停三个月，那么等你重新发布时，所处的世界会和你开始时大不相同。所以，试图暂停很长一段时间——你知道，哪怕只是几天——这种做法我们都不愿考虑。So uh you know for a JavaScript runtime um obviously fixing bugs and new features like that's all valuable. Um but you can pause for 3 months and the world isn't going to fall over, right? JavaScript 3 months ago isn't all that different from JavaScript today. And you can do sort of a a rewrite the world approach. Uh pause everything and then continue shipping when you're done. Uh Agentic Harnesses in the summer of 2026, if you pause for 3 months, the world you ship into will be very different from the world that you started in. And so trying to pause for for a long peri you know a period of time even if it's a few days was not something that we were willing to entertain.

嗯，你知道，即便我们能在两周内完成，也意味着要么让整个团队停工两周，要么应付庞大代码库中不断涌入的变更，还得理清所有这些变更，并将它们作为移植内容合并进去。Um and you know even if we've been able to do it in two weeks that would have been two weeks worth of either blocking the whole team or uh dealing with a constant influx of of changes across a very large codebase and having to rationalize and you know um all of those changes in and merge all those changes in as ports.

嗯，我们没有那么做，而是选择了我认为对 Copilot 来说合理、对 Bun 来说却没那么合理的做法：在主分支的生产代码库中，把代码分成一块块，逐步从 TypeScript 迁移到 Rust，同时每天发布版本。一开始，代码 100% 是 TypeScript，0% 是 Rust；到最后则变成了 100% 是 Rust，0% 是 TypeScript。整个过程就是随着时间推移，逐步推进，大体上进展得相当顺利。不过，正如 Bun 那篇文章提到的，这确实意味着有很多临时性的代码被加入，然后又被移除，对吧？Um instead we opted for the approach which I think made sense for co-pilot and made less sense for bun uh where we just incrementally moved the codebase more and more from TypeScript to Rust in pieces in Maine in the shipping codebase shipping releases daily. And it's just that at the beginning it was 100% TypeScript and 0% Rust and at the end it was 100% Rust and 0% TypeScript and it was just you know more and more over time and for the most part it worked out quite well. It did mean that there was to the point of the post of the the bun post there was a lot of temporary code that came in and left, right?

我觉得当时我们呃用的是 napi-rs 来实现 TypeScript 和 Rust 之间的互操作，峰值时有好几千个中断签名——我想大概有两三千个，用于两者之间的调用。到最后，这个数字降到了零，因为已经不再需要中断了。I think at we um we were using uh nappy nappy rs for interop between typescript and rust and at the peak point we had several thousand interrupt signatures for I think two two or three thousand for calling between them and then by the end it had dropped to zero because there was no more interrupt required.

所以对这个项目来说，这样做非常合理。还有其他做法。你可以彻底重写整个系统。你也可以让所有人停下手头的工作，直接在主分支上重写。你还可以创建一个功能分支，让主分支继续推进，然后不断把改动移植过来。至于渐进式方案，你可以像我们做的那样，但不必采用我选的做法——也就是一次性用 Rust 替换 TypeScript；你可以采用某种 A/B 方案，同时保留 TypeScript 和 Rust 组件，并在发布时同时部署两者，再用功能标志在它们之间切换，对这些组件并行进行灰度发布和实验。So for for this project it made a lot of sense. There are other approaches. You could do a a rewrite the world. You could do, you know, um have everyone stop what they're doing and just rewrite in Maine. You could have a feature branch and allow Maine to keep going and then constantly be taking ports in. And for the incremental approach, you can do what we did, but rather than do the approach of uh what I chose, which was sort of atomically swap uh TypeScript with Rust, you could have sort of an AB thing going where you you have both the TypeScript and the Rust component and you ship with both and you have a feature flag that allows you to ship between them and do sort of uh flights uh and experiments of those components side by side.

你知道，选择哪一种可能有各种各样的原因。我觉得我们最后选的那个对我们来说效果不错，不过，嗯，其他选项也都可行，或者说对其他项目来说可能更好。There are, you know, various reasons you might choose which and I think what we chose ended up working out well for us, but um all the other options would be valid or probably better for other projects.

所以听起来，在那种 A/B 方案里，你们还是会有一些临时的测试框架来比较两边。总之，如果你有一个独立分支，而旧项目（或者说主分支）上又不断有新的提交进来，那就需要持续把这些提交移植过去。我想，用现在的模型来做这件事应该也相对容易，因为每次只移植一个提交，对模型来说大概都很简单。你们考虑过这么做吗？或者说，和承担互操作成本相比，这么做的成本收益怎么算？So it sounds like that that AB approach, you would also still have some uh temporary testing harness that would be comparing the two legs. Anyways, well in the case where you have a a separate branch and then new commits that come in on the old project or I guess main need to be continuously ported. Um I I imagine that would be relatively doable with modern models as well cuz each of those single commit ports is just uh probably trivial for a model. Did you consider doing that one or how did the math compare between doing that versus incurring the interop costs?

所以这里有几个问题。这不只是说单独建一个功能分支，然后把所有东西都合并过来。如果我们维护一个单独的功能分支，最后再进行切换，一次性部署整套新架构，那么我们就只能在真实环境中一次性测试所有东西，这样做风险相当大。嗯，而我们选择这种方案的一大好处是，实际上我们可以利用生产环境来完成最后一公里的测试，并且让每次部署都只是在上一次部署的基础上做少量改动。因此，在整个移植过程中，我想我们总共发布了大约 130 个版本，差不多吧；每个版本都只是在前一个版本的基础上逐步增加更多内容。So there there's a couple questions there. It it's not just about the having a separate feature branch and bringing everything over. If we had a separate feature branch that we were maintaining and at the end we did this swap where we deploy like the whole new stack, we would only be testing in the wild everything all at once and that's a pretty risky gambit. Um whereas if you uh one of the big benefits of the approach that we chose was we were effectively able to use production for the last mile of testing and have each deployment be a small diff on top of what the previous one was. So over the course of the port I think we shipped about 130 releases something like that each one of which contains just incrementally more.

因此，如果出现问题，即使完全不使用电脑，也能相当轻松、迅速地定位问题根因。And so if there were issues, it was very pretty straightforward and fast to root cause those issues even without using computer at all.

就像有人报告一个问题，你会说：“哦，是的，我完全知道这是怎么回事，对吧？”然后你就能很快地缩小排查范围。而要把整个世界都重写一遍，就很难进行这类验证了。嗯，这确实导致了一些回归问题。其中很大一部分是因为从 main 合并变更时出了问题，因为你知道，我当时仍然有功能分支，只是给它们设定的时间限制短得多。以前要花几周，现在移植一个组件只要一天；但就在这一天里，可能仍会有 10 或 20 个 PR 改动了这个组件。相当一部分回归问题，是因为代理在合并过程中漏掉了一些东西。嗯，而且移植和合并是同时进行的。Like someone reports an issue, you're like, "Oh yeah, I know exactly what that is, right?" And then you can very quickly narrow it down. Whereas the the whole rewrite the world, it's harder to do that kind of validation. Um, and there were absolutely regressions that came about from this. A good chunk of those were because of issues with merging in changes from main because, you know, I still had feature branches. They were just much more time boxed. Instead of being weeks, they were a day for, you know, porting a component, but within that day, there might still be 10 or 20 PRs that came in and touched that component. And a a non-trivial percentage of the regressions were places where the the agents just missed something as part of doing the the merge. Um and and the port and the merge at the same time.

嗯，原子式地把 TypeScript 替换成 Rust，而不是采用某种 A/B 方案，有个好处是：代理被要求在完成移植后立刻删除被替换的 TypeScript 代码。这样做有个好处，就是会导致从 main 合并时出现冲突，因为只要 main 中的任何代码改动了已经移植的 TypeScript，就会立刻产生冲突，这就像是亮起了红灯，提醒代理注意那段代码。反过来，如果我们没有删除它，main 中对那段 TypeScript 的更多改动就很容易悄悄进入分支中原封未动的 TypeScript 代码。Um, one of the the nice things about sort of the atomic uh replacing from Typescript to Rust versus having an AB thing going is um the agents were instructed to delete the TypeScript that they were replacing the moment that they ported it. And that in a good way created conflicts for those merges from Maine because it meant that if any code in Maine touched any of the TypeScript that had been ported, it instantly created a conflict which sort of raised red flags and pointed the agents attention at that code. Whereas if we weren't deleting it, it would have been very easy for more changes in main to that TypeScript to find their way into the existing untouched TypeScript in the branch.

然后，代理就得去分析它移植过来的所有代码，找出需要做哪些改动才能把那些改动带进来；而我们当时在做这个“沼泽”项目，所以它几乎是立刻就被指向了那些地方。And then the agent would have to reason about, you know, all of the code that it had ported looking for changes to that to bring that in versus it was sort of instantly uh pointed to those places because we were doing this swamp at the time.

他是如何发现回归问题的How he caught regressions

是的，我能想象很多人——嗯，很多团队或工程师——想要重写整个代码库时，最大的顾虑可能就是正确性问题：代码实在太多了，你根本没有条件让人逐行阅读代码来确认它是正确的。那么，你们一路上是怎么发现并修复回归问题的？嗯，最重要的是我们的端到端测试、集成测试，或者你想怎么称呼都行。嗯，我们的 Copilot 测试基本上有两种。我们有单元测试，它们大多是在代码库中测试一些微小功能，而且很多测试可以访问内部实现。因此，它们能够测试这里一个没有对外暴露的小函数，或者那里一段没有对外暴露的代码。Yeah, I can imagine a lot of people uh a lot of teams or engineers that want to rewrite their entire code base probably the biggest concern is the correctness concerns which is there's so much code that you don't have the luxury of a human even reading the code to confirm that it's correct. So um how did you discover and and fix regressions along the way? So the the most important thing was our um endto-end tests, our integration tests or whatever you want to call them. Um the we basically had two flavors of tests for copilot. We had unit tests which were largely sort of you know they're they're they're testing micro functionality throughout the codebase and a lot of them have internals access. So they're able to test this little function here that's not exposed or that piece of code there that's not exposed.

然后还有端到端测试，分别在运行时层、SDK 层，或者 CLI 层；CLI 和它们在同一个代码库中。因此，CLI 的端到端测试实际上也会测试运行时。虽然这并非百分之百准确，但大体上可以把 CLI 看作是通过面向人的界面，以尽可能底层的方式来展现运行时的能力。我们有所有这些端到端测试来验证行为。总体来说，虽然不是百分之百，但我们遇到的绝大多数回归问题，都是因为没有端到端测试覆盖相关功能。Um and then there are the endto-end tests either at the the runtime level, the SDK level or uh the CLI is in the same codebase. And so its endto-end tests also effectively exercise the runtime. And this isn't 100% true, but the CLI can largely be thought of as sort of a just um exposing the runtimes capabilities as sort of the lowest layer possible in a in a human interface. Um and so we had all of those endto-end tests to uh validate behaviors. And by and large, it's not 100% but the vast majority of regressions that we had were because we didn't have endto-end tests covering that functionality.

嗯，我们通常会有单元测试覆盖这些内容，但这种情况下单元测试的问题在于，它们也是用 TypeScript 实现的，用来测试 TypeScript，而且还会访问内部实现。因此在很多情况下，移植代码的同时也必须重写这些单元测试。Um, we generally had unit tests covering it, but the problem with unit tests in this situation is they were also implemented in Typescript testing TypeScript uh with internals access. And so those in many situations those unit tests had to be rewritten at the same time that the code was being ported.

现在，由于我们的实现方式，我们获得了一些好处。比如说，如果我把下面这小部分替换掉，那么 TypeScript 这部分现在就是 Rust 了，而上面的 TypeScript 是通过互操作与它交互的。这个部分有自己的单元测试，因此这些单元测试仍然可以间接验证受支持的 Rust 组件。我们确实从中获益了，但真正的保障是集成测试，或者说覆盖所有部分的端到端测试。我还为团队写了一份文档，说明这次移植的动机，并详细列出了我们希望如何推进这项工作、大致的时间安排、采用的方法和策略，以及会使用的技术等等。Now because of how we did it, we got the benefit of um if I replace this little piece down here and then the typescript so this is now Rust and the TypeScript sitting above it was interacting with this through interop. This guy had its own unit tests and so indirectly those unit tests are still validating the supported Rust component. So we got benefits from that but the safeguard was the the integration test or the endto-end tests across everything and I had written a document for the team sort of motivating this port and laying out exactly how we would want to go through it and approximate time frames and approach and strategies and technology we would use and so on.

有人指出，我们需要把加强端到端测试作为最先做的事，而我也确实这么做了。我认为我的一个失误，也是如果重来会做得不一样的地方，就是我本该做得更多。One of the things that called out was we needed to beef up our endto-end tests as the very first thing we did and I did. I think one of my failings and something I would do differently if I had to do it again was I would have done it more.

嗯，因为再说一次，这大多数情况下是不言而喻的：如果你为某个功能编写了测试，而且这些测试没有出现回归，那么只要测试覆盖了你关心的所有行为，这些行为就是正常的。所以，出现任何回归，都在某种程度上说明测试存在缺口。当然，这也不完全对。我们遇到过一些情况，最好笑、或者说最令人难过的情况是，我们忘了阻止智能体在修改被测行为的同时，也去重写端到端测试。你知道，智能体的目标就是让测试变绿。所以如果你放任它们，它们就会修改判定标准——也就是在改代码让测试通过的同时，也修改测试。还有一次，我们遇到一个情况——呃，这次倒是没让它蒙混过关。Um because again the vast majority it kind of goes without saying but if you have tests for something and those tests don't regress then like the behavior is good as long as you have tests covering all the behaviors you care about. And so the fact that we had any regressions sort of highlights that there were gaps. Now that's not 100% true. We had cases, the funniest or saddest cases were where we um neglected to prevent the agents from rewriting the endto-end tests at the same time that they were changing the behavior that was being tested. You know, the agents want to get to green like that's their goal. And so if you let them, they will change the oracle like they will change the tests at the same time as that the code to make it pass. Or we had one case where uh this one didn't slip through.

我注意到了这个问题，但当时智能体在做修改，结果改动了一些公共 API，CI 捕获到了这一点，并说：“不行，不行，不行，你不能这么做。”不过，CI 里有个应急通道：如果有人真的知道自己在做什么，就可以给 PR 加个标签，基本上就是在说：“不，这次特定的破坏是可以接受的。”智能体发现了这个通道，还加上了标签，好像是在说：“哦，这次破坏没问题。”尽管实际上并不是这样。嗯，还有些事情对你来说确实很难。你可以编写端到端测试，但它们往往会深入到一些很细的地方。比如说，你把数据发送到线路上，并确保在所有可能的情况下，你发送的每个字节在接收端都符合预期。I noticed this one, but like the the agent was making changes and it ended up changing some public APIs and uh CI caught this and said, "No, no, no, you can't do that." But we have an escape hatch in CI where if someone really knows what they're doing, they can put a label on the the the PR to basically say, "No, this particular break is okay." And the agent found that and applied the label to say like, "Oh, this this break is okay, even though it, you know, it wasn't." Um, and then there are other things that are just really hard for you. You could have endto-end tests, but they're sort of in the weeds. Uh, things like, you know, you're you're sending data out on the wire and making sure that every bite that you send in all possible situations is what's expected at the end of the end of the wire.

这需要一套相当全面的测试。如果有任何遗漏，就可能有问题漏网。还有，TypeScript 和 Rust 的线程模型、并发模型都非常不同。尽管很多人喜欢说，Rust 代码只要能编译就没问题，而且能防止所有这类 bug，但这话没错，不过它并不能防止所有类型的 bug。仍然有很多并发 bug 会漏过去。所以其中不少问题都与生命周期和并发有关，而这些问题 Rust 编译器并不会捕获。当你从一个基本上是串行的单线程执行模型，转向一个我们有意设计得更具可扩展性、希望让任务并发运行的模型时，情况就不同了。That requires an extensive suite. And if you have any gaps, there's the potential for things to slip through. And then there's also, you know, there's a TypeScript and Rust have very different threading models, um, and concurrency models. And, uh, as much as people like to say that, you know, if your Rust compiles, it's correct and it prevents all these kinds of bugs, that's true, but it doesn't prevent like uh, you know, all kinds of bugs. And there are plenty of concurrency bugs that slip through. Um, and so a bunch of the issues were lifetime issues, concurrency issues, things that the Rust compiler does not does not catch. Um, and when you're coming from a a largely serialized singlethreaded execution model and going to one where on purpose we wanted much better scalability, we wanted things to run concurrently.

嗯，里面会冒出不少问题。嗯，总之，回答你的问题，最重要的是确保你关心的所有行为都被编写成测试。然后让智能体在确保所有测试始终通过的前提下移植代码。嗯，还要设置好防护机制，确保它们不会同时改动你用来进行验证的东西——毕竟你需要这个验证本身是正确的。Um, there are plenty of of issues that that creep in there. Um so anyhow to to your question the the number one thing is to make sure that um all of the behaviors that you care about are encoded in tests. Um and then let the agents port the code while maintaining you know keeping everything passing. um and making sure that there are guard rails in place to make sure that they don't change the thing that you're using for the validation at the same time that you need that validation to be correct.

感觉回归问题还是不可避免的，因为测试覆盖率，呃，有点像是需要主动选择的——它依赖于有人指出哪些东西很重要，但你总可能会忘掉一些东西。嗯，所以我有点想知道，这类项目是不是注定会出现某些问题。It still feels like a regression is inevitable because the test coverage is uh it's like um it's optin kind of where it relies on a human saying these are the things that are important but you could always forget something. Um, so I I kind of wonder if this type of project, it's just inevitable that there will be some breakage.

我想是的。我的意思是，这真的很难想象。我是说，谁知道一年后会是什么样。变化太快了。但就目前而言，是的，百分之百会出现回归问题。如果没有，我会非常震惊。嗯，呃，除非我们说的是那种非常非常小的代码库，但你说的是要重写几十万甚至几百万行代码。这就是大数定律。如果智能体有 99.99% 的概率做对——这可能比大多数人高出一两个九——嗯，那仍然意味着有 0.01% 的时候它们会以这样或那样的方式出错。而如果你处理的是百万行代码，那 0.1% 就是一个很小但确实存在的数字。你可以多派一些智能体来处理。I think so. I mean, it's it's hard to imagine. I mean, who knows what things will look like in a year. Things are moving really fast. But right now, yes, 100% there will be regressions. If there aren't, I would be astounded. um uh unless we're talking about like you know a tiny tiny little codebase but you're talking about rewriting hundreds of thousands or millions of lines of code. It's just law of numbers. If if the agents get it right 99.99% of the time, which is probably uh a nine or two higher than most humans, um that still leaves.01% of the time where they're going to get it wrong in some way, shape, or form. And if you're dealing with million lines, that.1% is a very real small but real number. You can throw more agents at it.

我们确实做了这些，你知道，在合并之前进行大量的智能体审查，然后合并之后运行智能体，持续查找 bug、并发问题等等。它们确实能发现问题，但从来都不是——至少根据我的经验，面对如此庞大的代码量，它们从来都做不到百分之百完美。And we did you you know we just you know be before merge doing lots of agentic reviews and then after merge running agents to to continually look for bugs, look for concurrency issues, look for whatever and they find things but they never find it's never at least in my experience it's not 100% perfect for such a huge volume of of code.

我猜大部分内容都没有经过人工审核。你们是如何大规模进行代码审查的，还是说根本没有做代码审查？I imagine most of it wasn't human reviewed. How did you handle code review at scale or did you do any at all?

是的，所以肯定看了一些，嗯，但我没有逐行阅读。那是不可能的，而且即使我真这么做了，也是在浪费时间。嗯，我逐行读完并不能避免漏洞。事实上，让代理逐行阅读，效果要好得多——不只是速度更快，质量也比我自己逐行读更好。嗯，我当然看了我提交的每个 PR 的代码，但我更关注的是整体情况，对吧？嗯，这个方向对不对？架构对不对？组件划分得对不对？嗯，采用的方法和算法是否合适？Yeah, so definitely did some um but I did not read every line. That would be impossible and if I did it would also be a waste. Um there's no like me reading every line would not prevent bugs. In fact, agents reading every line is a lot more effective uh not just faster but more effective in quality than me doing so. Um, I I absolutely looked at the code for every PR that I put up, but I was focusing more on big picture stuff, right? Uh, is the direction of this correct? Is the architecture of this correct? Are is the component you componentization of this correct? Uh, is the approach are the algorithms that are being used appropriate?

呃，有没有什么可疑的地方？比如说，我很多时候都是从宏观层面快速扫一眼，看看这两个代码库，有没有什么地方不对劲，对吧？然后把逐行细节交给智能体去关注。在很多情况下，我会快速滚动浏览所有改动，看看有没有什么显眼的问题，可能会注意到一些零星的东西，但大多数时候，一个很出色的智能体审查者在逐行细节方面肯定比我强得多，我甚至觉得比大多数人都强。嗯，不过在宏观层面，有很多时候我会直接叫停智能体，说：“等等，这个方向不对，不能这么做。”嗯，反过来也有这种情况，智能体会停下来跟我说：“我们做完了。”Uh is there anything fishy about like you know if I a lot of it is like squinting at a 30,000 foot level and looking at the two code bases and does anything look off right and then relying on the agents to focus on the line by line details and in many situations I would sort of do a quick scroll through all the changes to just see if anything popped out and I might notice a few things here and there but for the most part a really good agentic reviewer is going to be way better at sort of the line level details than certainly me and I would argue most humans. Um, but at the macro level, there are lots of times when I basically stopped the agents and said, "Whoa, there like this is this is not the direction to be going." Um, or the inverse where the agents would stop and be like, "We're done."

我就会说：“不，你不是。呃，这、这不是我们……我们想要的。嗯，所以这涉及的是更大的整体图景，而不只是微观层面。”And I'd be like, "No, you're not. uh the this this is this is not what we're we're going for. Um and so it was a bigger picture uh than sort of the the micro level.

指导代理大小规律Guiding the agents

我在那篇博客文章里看到，你有一张很酷的柱状图，基本上把你发给智能体的每一条消息都分类了，还标明了你的意图。回顾一下这些内容，并反思你是如何引导智能体的，你的大部分时间都花在哪儿了？哪些时候你介入并做出调整，产生的影响最大？我做的事情还可以归纳成两三个更高层次的类别。嗯，其中有一两个类别，我觉得这恰恰反映了如今模型和运行框架在处理长周期任务方面的现状：有时是我对它们的进展不满意，不管具体是什么原因，于是它们停下来时，我会以某种方式让它们继续；或者它们朝着不太好的方向一路走偏时，我会叫它们停下来。I saw in the blog post that you you have this cool bar chart where you basically categorized every single message from you to the agents and what your intent was. Looking over that and reflecting on how you guided the agents, where was most of your time spent and what were the most impactful times where you would step in and change something? There were two or three even higher level categories of things that I was doing. Um, there was one or two categories where I think this just speaks to the the state of sort of long horizon tasks with models and harnesses today where it was me being frustrated with their progress in one way, shape or form and either telling them to keep going in some fashion when they had stopped or telling them to stop when they were going off down, you know, bad paths.

但大多数情况更像是……属于需要权衡取舍的那一类。我想确保合理的人或合理的智能体都可以选择不同的路径，同时也想确保我们走的是正确的路。嗯，很多时候，我会直接和主智能体聊一聊，大致评估一下我们目前的进展、需要做出的决定，并结合当时的具体情况，像对着橡皮鸭讲解那样，梳理这些选择的利弊。你之前问到的渐进式方案与……暂停一切、进行大爆炸式重写的方案，其中有些问题也属于这一类。But the the majority were more um were in a category that was more of um there were tradeoffs to be made and I wanted to make sure and reasonable people or reasonable agents could choose multiple paths and I wanted to make sure that we were going down the right one. Um, and a lot of that would be times where I would just have a conversation with the the main agent to basically evaluate where we were, the decisions that were needed to be made and look and kind of talking through rubber ducking through sort of the pros and cons of those choices given the current situation we were in. And some of that to the to your earlier questions about sort of the the incremental versus uh sort of pause the world or big bang rewrite approaches.

其中一些与其说是关注端到端流程，不如说是关注我们对我想为正在推出的这一部分做些什么的看法。Some of those were less about the end to end and more about where we like what I wanted to do for this one slice that we were putting up.

嗯，你知道，这些代理会在某个地方停下来，说：“这个组件我已经搞定了。”然后我看了看，会说：“考虑到它和其他系统之间的那些关联，这不是个适合就此收尾的地方。我们暂时可能还得多做一点，稍微越过这个架构边界，这样就能顺手理清不少东西。等我们把它合并进去时，代码库也就处于一个比较合理的状态了。”Um, so you know the the agents would pause at a particular place saying you know I'm done with with this component and I'd look at it and say this is not a good place to leave things in terms of the the the tendrils that this has into other systems you know temporarily or either maybe we need to do just a little bit more to kind of get beyond this architectural boundary and then that'll clean up a bunch of stuff so that when we merge this the codebase is in a reasonable place.

嗯，很多时候，它会把这一小部分工作宣布为成功。你眯着眼看，嗯，行吧；但如果它再多做一点点，或者有些情况下退回去、少做一点点，就能让代码库的状态更好。也有些情况是，它实际做的比我希望它做的更多。比如，我设定的一条规则——我在指示里写明了，但它并不总是遵守——就是我希望这些移植尽可能严格。Um, a lot of times it it would sort of declare success for the slice and if you squint, yeah, okay, but if it were do just a little bit more or in some case revert and do a little bit less, it would have left the codebase in in a better place or situations where it actually tried to do more than I wanted it to do. Like one of the one of the rules that I put in place with instructions that wasn't always abided by was I wanted these ports to be as strict ports as possible.

不删除功能，不添加功能，不进行任何重大更改或算法层面的改动，也不试图修复过程中碰到的任意 bug。我会让它把这些问题记下来，方便我之后单独处理。但我希望这些改动尽可能严格地保持功能一对一迁移。有时我会发现它违反了这些规则。嗯，我基本上会让它退回去，恢复到之前的状态；或者至少先和我仔细讨论它做了什么，确保我认可纳入这项改动。我们遇到的回归问题中，有相当一部分就是因为它违反了这些规则，而我没有注意到，结果它把一些本来不应该做的修复也加了进去。Not removing functionality, not adding functionality, not making any huge changes and algorithmic approaches, not trying to fix arbitrary bugs that it comes across along the way. I would have it note those for me so I could revisit those separately. But I wanted these to be very strictly as onetoone functionality ports as possible. And there were times when I would see it violating those rules. Um, and I would basically have it back off from that and go back to what it previously had or at least talk through what it did to make sure that I was comfortable with that particular change being included. um a non-trivial percentage of the regressions that we have had were from places where it violated those rules and I didn't notice and it included fixes uh for things that um shouldn't have actually been done.

事实上，只有一个。我觉得大多数情况下，我们已经看不到更多从移植阶段报告出来的回归问题了。我们一直都会看到回归问题。这就是每天向一个代码库合并数百个 PR 的必然结果。不过，你知道，这也是意料之中的事。我们通常会在发布前修复它们，而我们是在主分支上发现这些问题的，In fact, there was just one I for the most part we're not seeing any more uh regressions being reported from the porting phase. We see regressions all the time. That's the nature of merging hundreds of PRs a day into a repo. But um you know it's just to be expected. Um and we fix them generally before they ship and it's just where we see them in Maine,

对吧？right?

嗯，不过前天还是昨天，我们遇到了一个特别边缘的情况。我看了一下，心想这也太奇怪了。我不太明白，它说是我加了这个，但我不记得自己加过这个。呃，后来我回头看了看是怎么回事。我在一次移植工作中创建了一个 PR，而这个内容并不在最初的 PR 里。后来，一位同行或他所用的代理给了一些反馈，建议做点什么；而代理对此的回应，基本上就是通过收紧某些限制来处理这条安全反馈。嗯，但它收得比实际需要的更紧，结果就是一些在移植之前能正常工作的东西，移植之后反而不能用了。嗯，而这些事情真的很有意思，值得去弄清楚它们究竟是怎么发生的。Um but we had one yesterday or the day before where there was this really corner case thing and I looked at I was like that's really weird. like I don't it's saying that I added this and I don't remember adding this uh and looked back what it was. So I had I had created a PR as one of the porting efforts and it wasn't in the initial PR uh and then some feedback from uh a peer or a peer's agent came in recommending doing something and the uh agentic response to that was to basically address this security feedback by locking something down. Um, but it locked it down a little bit tighter than it needed to be locked down and that resulted in some things that had been working before the port, you know, no longer working after the port. Um, and uh, those are really interesting things to go and figure out how how they happened.

挺有意思的。那么，你的代理会自动应用更改并持续监听 PR，如果它觉得合理，就可能在没有你介入的情况下采取行动。是这样吗？That's interesting. So, your agent will auto apply and auto listen on the PR and if it seems reasonable may take action without your input. Is that right?

所以，呃，我大部分时候都会用 GitHub Copilot 应用，它有个代理合并功能，你基本上可以让它自动执行各种操作来处理 PR。比如，呃，如果有冲突，它可以自动 rebase，也可以自动合并。呃，它还能自动让 CI 变绿。所以如果测试失败了，它就会修复测试。它还能自动处理收到的评审反馈，并在一切都变绿后自动合并。对于这四项功能，你可以分别选择要启用哪些。大部分时候，我会勾选其中三项，除了最后一项。这样，呃，我就能在一切都变绿之后、点击合并按钮之前，抽出时间回头检查一下它为了让这个 PR 变绿都做了些什么。So, uh for the most part I use the um the GitHub copilot app and it's got this agent merge functionality where you can basically have it automatically do various things watching a PR. It can um automatically rebase if there are conflicts or merge if there are conflicts. Uh it can automatically drive CI to green. So if tests are failing, fix the tests. It can automatically address uh review feedback that comes in and it can automatically merge once everything is green. And you can for each of those four things you can choose which ones you want. For the most part, I leave all three of those checked except for the last one. So that um I can I have a time where after everything is green and before I click the merge button, I go back and I review to see everything that it did to drive this PR to green.

它修复了哪些测试？收到了什么反馈，又是如何处理这些反馈的？这样我就能从整体上了解它做了什么，而不是逐行审查。嗯，我记得我确实读过这一条，但没有完全理解它把限制收得有多紧、拦截了多少内容。因为如果我当时明白了，就会说：“其实，那些情况没问题，放行吧。”然后让它在这方面放宽一些。What tests did it fix? What feedback came in and how did it address it to make sure that I know again at a high level, I'm not reviewing every single line, what it did. Um, and this particular one I remember actually reading, but I didn't fully comprehend how how much it was locking things down and how much it was blocking. Uh, because if if I had, I would have said, "Actually, those cases are okay. Let those through." Um, uh, and and had it back off from that.

是啊，是啊。如果你把那些都勾选了，我、我，没人、没人会这么做，因为他们是你团队的人。不过，是啊，想象一下我在你们团队里。我大概可以写一条恶意的 PR 评论吧，我猜。你知道，就像：“嘿，听起来挺合理，但我确实要求你把几乎所有东西都删掉，或者……”Yeah. Yeah. If you had all of those checked off, I I no one no one would do this because they're on your team. But yeah, imagine I was on your team. I could probably craft a PR comment that is malicious, I guess. You know, it's like, "Hey, it sounds reasonable, but I did request you to delete like everything or

是啊，我确实让你格式化 C 盘了。你没格式化 C 盘吗？”Yeah, I did ask you to format your C drive. Did you not format your C drive?"

对，对，没错。把所有选项都关掉。是啊。而且你知道，当然，换成不同的项目，我勾选的选项也会不一样。比如说，如果这是个开源项目，呃，我唯一会考虑保持勾选的就是 CI 那项。你知道，自动处理——绝对不行。也许可以允许自动变基，嗯，但我绝不会允许它自动处理 PR 上收到的任意审查反馈，更不用说自动合并了。要是在一个私有的闭源项目里，我信任那些我知道正在参与项目的人，嗯，风险确实小一些。但即便如此，我还是觉得，那个复选框……你得非常非常信任它才行。Yeah. Yeah. Exactly. Just take everything down. Yeah. And you know, certainly different projects I would I would have different boxes of those checked. Like if this was an open source project, uh the only one I would consider leaving checked would be the CI one, you know, driving I there's no way uh and maybe rebasing um but there's no way that I would allow it to automatically address arbitrary review feedback coming in on a PR and certainly not automatically merge um on a closed on a private closed source thing where I trust uh you know the trust the people that I know to be working on the project. Um it's it's less risky, but even then I still, you know, the that less check box you have to be very very trusting.

嗯，是的，我也喜欢这个应用的一点：它确实有内置提示，作为 Agent Merge 的一部分，用来介绍这些不同功能；不过，这个应用也允许我针对这些不同能力添加自己的提示词。Um yeah, the one of the things I like about the app as well is it does you there there are built-in prompts as part of agent merge for what those different things are, but the app does permit me to add my own prompting for those different capabilities.

因此，对于每一项工作，我还会给代理补充一些限制和指导，告诉它应该怎么做。比如，其中一项要求始终使用 rebase，而不是 merge；或者针对代码审查反馈，我会告诉它哪些内容需要先仔细斟酌，不能盲目接受，哪些内容则可以直接着手处理。就是这类事情。OpenAI、Anthropic 和 Cursor 都在使用 WorkOS 来简化工作。WorkOS 是一个 API 平台，可以为你的应用添加 SSO、SCIM 和 RBAC 等企业级功能。这些功能是你需要向其他公司推销的，但并不是你开发产品时的主要关注点。And so I have additional constraints and guidance for the agent as part of doing each of those things about how it should do it. Like you know always rebase instead of merge for for one of them or for review feedback. Here are the kinds of things I want you to think critically about before you just blatantly accept it and these are the ones where it's okay just go ahead and address it. You know those sorts of things. OpenAI, Enthropic and Cursor all use this product to make their lives easier. Work OS is an API platform that adds enterprise features like SSO, skim, and arbback to your app. These are things that you need to sell to other companies, but aren't the main focus of what you're building.

现在他们还增加了对 AI 代理的支持。例如，面向代理的 O 会为每个代理提供独立身份，并使用短期作用域令牌，而不是长期 API 密钥。And now they've added more support for AI agents. For instance, O for agents gives each agent its own identity with short-lived scope tokens instead of a longived API key.

所以如果出了问题，你可以撤销一个会话，它下面的一切都会随之终止。他们还有一项名为 airlock 的功能，它是一个授权层。[哼笑] 智能体执行的每个操作都会根据你用通俗英语编写的规则进行检查。因此，它既不会对所有操作一概放行，也不会在无人监管的情况下运行。最后，他们还有 pipes，可以将你的智能体连接到 300 多种第三方工具，而且你完全不必管理令牌。So if something goes wrong, you can revoke one session and everything under it dies. Another feature they have is airlock, which is an authorization layer. [snorts] Every action agent takes is checked against rules that you can write in plain English. So it's not rubber stamping everything and it's not running unsupervised either. And lastly, they have pipes which connects your agents to over 300 third party tools without you ever needing to manage a token.

在上线前发现回归问题Catching regressions before prod

所以，如果你正在构建 SAS 或 AI 智能体并向企业销售，Work OS 只需几行代码就能填补所有这些空缺。去 work os.com 了解详情并开始使用吧。也感谢他们赞助我的节目。我们刚才聊了[笑了一声]一些回归问题，你提到其中很多似乎是在上线生产环境之前就被发现了。嗯，我想很多人听到“回归”这个词，可能会觉得，哦，你们把应用搞崩了，或者事情总是在失控，或者总是发生事故。你能不能讲讲，端到端测试之后还有哪些额外的防护措施，让这些回归不至于造成严重影响？我会说，我所说的这些回归里，大概三分之二是在合并到 main 后很快就被发现的，而且当时我们还没有发布任何预发布版本。So if you're building SAS or AI agents and selling to enterprises, work OS fills all these gaps in a few lines of code. Check them out at work os.com to learn more and get started. And I appreciate them for sponsoring my show. We talked [snorts] a bit about regressions and you mentioned that a lot of them were caught uh before they went out to prod it sounds like. Um and I think a lot of people might hear regressions and they think oh you're you're taking down the app or you know things are exploding all the time or their incidents. Could you explain maybe the the further protections after end to end testing that made it so these aren't bad regressions? I would say probably twothirds of these things that I'm calling regressions were caught very soon after merging to main and before we shipped any pre-release.

嗯，尽管到了这个阶段，我们每天大概会发布三个预发布版本。所以这些问题都很快就被发现了。嗯，还有一些问题进入了预发布版本。你知道，我们通过 CLI 进行的环形部署基本上分为预发布版和稳定版：在任意一天，预发布版可能会覆盖 5% 到 10% 的用户群，而 90% 或 95% 的用户只会使用不稳定版。因此，相当一部分这类问题——具体比例我不太确定，大概是 10% 到 20%——进入了预发布版。然后很快就会有人提交问题，或者有人说：“嘿，我发现行为有些不一样。”Um even though we're shipping at this point we're shipping I think three pre-releases a day. So they were caught like very quickly. Um then there was there were some that made it into a pre-release and um you know our form of ring deployments with the CLI is basically pre-release and stable where pre-release on any given day is probably hitting 5 to 10% of the um the audience um of the customer base and then you know 90 or 95% is only unstable and so a a a reasonable number of these issues uh I don't know exactly what percentage 10 20% um found their way into a pre-release. And then pretty quickly we'd get an issue opened or someone saying, "Hey, like I'm seeing this difference in behavior.

这是有意设计的，还是无意的？”所以，很多这样的情况都与众不同，但不一定是坏事，只是差异明显到有人注意到了，于是问我们：“这是故意的吗？Was this by design or not?" And so a lot of these were things that were different, not necessarily bad, but different enough that someone noticed and then asked us like, "Was this on purpose?"

嗯，其中一些情况是我没能阻止代理在移植过程中顺手修复 bug。所以其中一些实际上是我们希望看到的改动，但我们并没有打算这么做，你知道，这方面并不是有意为之。嗯，还有一些改动进入了稳定版。由于我们在主分支或预发布版本中都没有注意到它们，进入稳定版的那些改动往往会一直留在那里，而且很长一段时间都没人察觉。嗯，这基本上就是我们过去……你知道，最近已经不太常见了，但一个月前我们还会在几周内陆续看到的那些零星情况。Um, and some of those cases were ones where I failed to prevent the agent from fixing bugs as part of the port. And so it it some of them were actually desirable changes, but we hadn't intended to, you know, they they weren't purposeful in that regard. Um, and then there there were a a number that made it into stable releases. And the ones that made it into stable releases, because we hadn't noticed them in either main or pre-release, they often ended up staying in and being unnoticed in stable releases for quite a while. Um, and this is mostly the trickle that we have been seeing over the last, you know, well, you don't really see much anymore, but a month ago would have been the trickle that we were seeing, you know, over a few weeks.

嗯，有些特别极端的边缘情况只是表现得稍微不一样，然后我们就会收到有人提的 issue，说：“嘿，这个东西没按我预期的方式工作，你知道这是故意的吗？”很多时候都不是故意的，我们一听说就会修。另外，有些问题也会通过各种遥测系统发现，比如监控异常、崩溃之类的。显然，这些都不是有意为之，我们也会把它们修好。uh where some super corner case thing just behaved slightly differently and then we get an issue open like hey this this isn't working the way I expected you know is this on purpose or not and often times they weren't and we'd fix them as soon as we heard about them um there were also ones that we would see from various telemetry systems you know uh monitoring for exceptions crashes uh things like that and obviously those are unintentional and we fix those.

他使用了哪些模型Which models he used

在那篇帖子里，你分享了这张图，展示了整个项目过程中你使用过哪些模型。你有一张挺不错的柱状图，显示每周的 token 消耗中，Claude、GPT 各占多少，偶尔也会用到 Gemini。项目开始时几乎全是 Claude，到项目结束时则几乎全是 GPT。那么，是什么让你在项目进行到一半时改用 GPT 作为首选模型？In the post, you you shared this graphic of which models you used throughout the course of the project. So, you have this nice um I guess bar chart that's showing what portion of each week's I guess tokens are consumed by either Claude or or GPT and in rare cases Gemini. At the beginning of the project, it was almost entirely Claude and then by the end of the project, it was almost entirely GPT. So what made you shift your model of choice in the middle of the project to GPT?

嗯，大部分时候我一直在尝试不同的模型。你知道，那段时间两家实验室都在以非常快的速度发布新模型。所以我一直在试用，当我换到某个模型并发现它表现不错时，就会一直用下去，直到有些地方让我觉得不太满意，然后我就会试试别的模型。所以，这并不代表我有什么长期偏好。更多只是说，在那个时刻，哪个模型对我来说效果最好。嗯，刚开始的时候，我觉得我主要用的是 Opus 46 或 Opus 48，效果很棒。然后 GPT——是什么来着？5.5、5.6 之类的——在这些特定工作负载上的表现甚至更好。所以我就转用它了。后来 Opus 5 发布了，我当时就想，哦，这个挺不错。Um for the most part I was constantly trying different ones and uh cuz you know over that period like both labs were releasing new models at a very fast pace. And so I was experimenting and when when I would move to one and I found it to work well, I would stick with it until uh something bothered me and then I would try a different one and then you know so um it doesn't in any way reflect sort of a long-term preference. It was more of at that moment what was working best for me then. Um, and so at the beginning of it, I think I was on mostly Opus 46 or Opus 48, and it was working great. And then GPT uh, what was it? 5.5, 5.6, one of those, and that started working even better for the these particular workloads. So then I jumped to that. And then Opus 5 came out, and I was, oh, this is cool.

然后就切换到那个模型，对吧？最近我用 6.1 Sonnet 的效果一直很好。我会用 Opus 5.5 来审查我用 6.1 Sonnet 完成的代码。所以我会在它们之间分配工作。有时候，如果我对某件事特别恼火——在整个移植过程中，这两个模型，也就是 GPT 和 Claude，有时都会表现出一些让人很恼火的特性。比如在某个特定阶段，或者我用某个模型处理一段时间后，它可能会卡住；无论我怎么做，它都会一直盯着某个特定的问题，不肯跳出来从整体上看问题。And jump to that one, right? And then um recently I've been having a lot of luck with uh 61 Saul. Um and I'll use uh I'll use Opus 55 for code review uh for the work that I've done with 61 Saul. Um and so I'll like I'll sort of split the balance there. Um sometimes if I get really frustrated with something like both over the course of the port both models both GPT and Claude would exhibit some very frustrating traits at certain points like um a particular point or if I was working on it for some period of time one of the models might get stuck and kind of like no matter what I did it kept kind of refocusing on a particular thing and not stepping out and looking at the forest for the trees.

嗯，在那种情况下，我会采取各种做法。有时候我会保留代码更改，但用一个干净的会话重新开始，然后试着借此清空上下文窗口，你知道，让它重置自己。Um, and there are various things I would do in those situations. Sometimes I would start like keep the code changes but start over with a clear session and try and use that to, you know, clear context window and try and use that to get it to reset itself.

但它经常又会陷入老一套的模式。所以遇到这种情况时，我会切换模型，然后用新模型继续，直到它开始表现出一些令人恼火的行为，接着我就会，你懂的，换成别的模型。我也试过用 Gemini 和 Grock 作为主要模型。不过我通常会把它们用于多模型代码审查。But often it would find itself back into a rut. And so in those situations, I would switch models um and then keep going with the new model until it found itself exhibiting some frustrating behavior and then I would, you know, switch to something else. Um I did experiment using Gemini and Grock as my main drivers. Um but I would often leave those to be um as part of like a multimodel code review.

嗯，我做的大部分审查工作，主要都是用 GPT 或 Claude 来完成的。然后我会再用来自不同系列的三四种模型，逐行审查已经完成的移植工作。我会很频繁地这么做，至少每次 rebase、引入来自 main 分支的新功能时都会这样做，而且往往比这还频繁。Um so a lot of the the reviewing that I had that I was doing I would be using either GPT or Claude as or for the main work. um and then use three or four different models from different families to do a line by line uh you know review of the the porting that was done and I would do that very frequently um at at least as frequently as every time that I was rebasing to bring in additional functionality from uh you know changes that had been coming into Maine but often uh even more frequently than that.

是的，我确实看到你有一个提示词，我想是在 rebase 时自动应用的，或者类似这样；你要求以对抗性的方式来审查它，嗯，我想是让它们同时都来审查。你为什么会得出这个结论，而不是用同一个模型做代码审查，只是让每个线程关注不同的方面呢？是的，这基本上就是为了获得不同的、实际上是不同的视角。你知道，对一个非确定性系统做的每一点小改动，最终都会让它走上不同的路径，去关注不同的东西。而你能做的最大改变之一，就是彻底换掉模型。嗯，我发现，想要获得不同的意见，并让它们关注不同的方面，最好的办法就是使用不同的模型，尤其是和我最初用的那个不同的模型。Yeah, I did see that you had a prompt that was I guess auto applied on rebase or something like that where you requested it to be reviewed um I guess adversarially or you know all all of them at the same time. What made you come to that conclusion rather than I guess code review with the same model but I don't know different focuses for each thread? Yeah, it's it's basically to get different effectively different perspectives. You know, every every little change that you make to a non-deterministic system ends up sending it down, you know, different paths and looking at different things. And one of the biggest changes you can make is completely changing the model. Um and so the best way to get different opinions and focusing on different things I that I found is by having um different models especially than the one that I used initially.

最终收益与成本The end benefit and costs

那么，嗯，你知道，重写完成之后，具体带来了什么影响，成本又是多少？从商业角度来看，这个项目的收益和成本分别是什么？收益非常显著。我的意思是，收益在于，GitHub 和 Microsoft 的所有这些产品、服务和团队，都能依赖一个共享的工具框架。这个框架始终处于业内领先水平，而且每天都在不断改进，他们就不必投入自己的资源去做同样的事情。这样他们就能专注于自身的业务价值，并重复使用这个组件。这和依赖任何一个库是一样的。只不过这个库的规模特别大，对吧？So um you know after you finished the rewrite what was the concrete impact and the cost? So if you look at it from a business perspective what was the benefit and the cost of this project? So the benefits are stark. I mean the benefits are the ability for all these products and services and teams across GitHub and Microsoft to be able to rely on a shared harness that is constantly sort of, you know, best of breed and getting better and better every day and not have to spend their own resources trying to do the same thing. they can focus on their business value and in and reuse this component. It's the same as with any library that you want to rely on. And this library is a particularly large library, right?

你希望能够专注于自己的业务价值，并复用那些不断演进的工作成果，而这些成果由一个全身心投入其中的人持续完成。You want to be able to focus on your business value and reuse the constantly evolving work for someone that is spending all their time focusing on on that thing.

因此，从商业价值的角度来看，GitHub 和 Microsoft 能够利用这个共享的测试框架，带来的价值简直无法估量。嗯，在移植之前，其中很多事情都更难推销，也更难证明其合理性。嗯，部分原因是确实存在一些成本，比如性能成本。比如说，某项功能占用的工作集是原来的 10 倍——我只是打个比方——这就意味着密度降低到十分之一，也就需要 10 倍数量的服务器。诸如此类。嗯，如果你能提高同一台机器上能够同时运行的用户数量，就会影响你的利润底线。如果你能将吞吐量提高，并把延迟降低到原来的五分之一、六分之一，或者其他程度，那就会带来实实在在的好处。And so from a business value perspective, it's it's like incalculable across GitHub and Microsoft being able to leverage this shared harness. Um whereas before the port, a lot of that would have been a harder sell and harder to justify. And um part of that is because of very real uh costs uh like uh performance costs. So you know something that was taking 10 times as much working set waving my hands means 10 times less density which means 10 times as many servers. You know those sorts of things. Um and uh if you can you know make the um if you can scale the number of users that are able to be on the same machine running concurrently that has impacts on your your bottom line. If you can scale the you know the throughput and reduce the latency by five or 6x or whatever it is that has real benefits.

嗯，从性能角度来看，最大的影响之一是，很多这类系统，尤其是服务，都会这样运行：一个请求进来后，需要启动 agent loop 再执行一轮，而这一轮可能非常短，然后请求就处理完了。因此，对这类系统来说，启动成本最终会成为主要影响因素之一。Node 应用的启动与原生 Rust 或 C 应用的启动之间差异很大。我们说的是数百毫秒、甚至数百毫秒以上的差距，有些情况下甚至是几秒。对于每个请求来说，这种延迟都非常明显，也会对最终收益产生极大影响——前提是你甚至能采用这种方案，因为你可能根本做不到；那样的话，你就又回到了我之前提到的那种情况。Um one of the the biggest impacts from a Perf perspective um a a lot of these systems especially services will do um you know a request will come in and the the agent loop needs to be started to make one more turn which could be a very short thing and then the request is done. And um so for these things the startup cost ends up being one of the dominant factors and the startup of a node application versus the startup of a a a native Rust or C application is there's a big difference there. we're talking hundreds and hundreds of milliseconds uh difference or or in some cases seconds um on a per request basis that latency can be extremely measurable and extremely impactful for bottom line if you're even able to adopt it because you might not be able to and then you're back in the previous world that I was talking about.

所以从商业角度来看，做这个移植非常值得。至于实际成本，我保存了这项工作中所有（或者说 99%）的会话日志。根据这些日志，我得以推算出新 token、缓存写入、缓存读取以及输出 token 的数量。总计大约是 1360 亿个 token。按照当时这些模型的公开定价计算，费用约为 12 万美元。这还没有考虑公司可能享有的任何折扣，也没有考虑它们实际支付的 token 费用。So from a business perspective, it's worth a lot to to do this port. Um, in terms of what it actually cost us, um, it was about if I so I I stored all of my session logs or 99% of my session logs from this effort. And from those session logs, I was then able to sort of deduce the token counts for fresh tokens, cash rights, cash reads, uh output tokens. And all told, it was about 136 billion tokens broken up in such a way that if you were to use the public pricing um of those models at those times, it came out to approximately $120,000. Um this is not factoring any discounts or things that companies might have for what those tokens cost.

嗯，这基本上就是 OpenAI 和 Anthropic 网站上的内容，再加上我自己的时间——你知道，花了大约14到15周。不过我同时也在做其他事情。你知道，很多这类智能体项目……如果我看看现在屏幕上的 Copilot 应用，目前有大约35个会话，都是我正在处理的事情，其中有些属于同一个领域。Um and uh this is just sort of what's on the website of you know open AI and anthropic plus my time which you know the it took it was about 14 15 weeks um but I was doing other things at the same time you know a lot of these uh agentic projects if if I look at my my co-pilot app that I currently have on my screen I've got about 35 sessions right now for things that I'm working on some of them in the same area.

其中一些项目分布在不同的项目里。嗯，所以我不是……开发者不再一次只做一件事了。如果你看一下百分比，粗略估算一下我的时间：看看在同一时期我提交的 PR 中，有多少比例是我提交的，大约五分之一（上下浮动）是跟这个项目有关的，另外大约 80% 是其他事情。嗯，所以你算一下，大约 15 周的 20%，就相当于我花了大约三周时间，分散在三个月里。嗯，再加上 token 成本。嗯，而且成本还在降低。模型的价格正在以惊人的速度下降。我记得 OpenAI 发布时——具体的我记不清了——我想 61 Saul 的价格是 6 Saul 的一半，而 6 Saul 的价格又是 56 Saul 的一半。Some of them are uh across different projects. Um and so I'm not developers are no longer working on one thing at a time. So if you look at the percentage you just as a rough estimate for my time if you look at the percentage of PRs that I was submitting over that same period about a fifth of them give or take was to do with this this project and about 80% was for other things. Um and so if you figure you know about 20% of about 15 weeks you end up with about three weeks of my time spread out over three months. um plus the the token costs. Um and it's only getting cheaper. The the the models are becoming like at cheaper at an astronomical rate. I think like when OpenAI released I'm not going to remember the exact, but I think like 61 Saul is half the price of six Saul and six was half the price of 56 Saul.

比如说，它们只会越来越便宜、越来越便宜、越来越便宜。嗯，而且这些模型也在变得越来越好、越来越好、越来越好，这意味着我需要投入多少时间来引导模型把事情做对，也在不断减少。嗯，所以这些移植项目的成本、这些迁移的成本，随着时间推移只会越来越低。哇。所以，花三周开发者的时间，再加上 12 万美元，你们就能重写整个代码库。嗯，如果在 AI 出现之前要评估这件事，假设你想着手做这项工作，那会是什么样子？首先，这根本就不会发生。Like the they're just getting cheaper and cheaper and cheaper. Um, and the models are also getting better and better and better, which means that the amount of time that I need to put in to steer the models to do the right thing is also decreasing. Um, and so the costs of these ports, the cost of these migrations is only getting cheaper over time. Wow. So, three weeks of uh I guess of developers time plus $120,000, you're able to rewrite the entire codebase. Um uh if you were to scope this out preai, let's say you wanted to undertake this, what would that look like? First, it wouldn't have happened.

嗯，对于……而且我其实是，呃，对于这件具体的事情，这种情况本来就不会发生，因为等我们做完的时候，移植的就会是错误的东西。这个领域变化太快了，我们基本上永远都做不完。Um for and I'm actually ser like for this particular thing the it wouldn't have happened because by the time we were done, we would have ported the wrong thing. like the this space is moving so fast that we would have basically never finished.

事实上，你可以从博客文章里的图表中看到这一点：在移植工作的大部分时间里，Rust 代码的数量不断增加，而 TypeScript 代码的数量基本保持不变，只略有增加。这是因为移植过程中，代码库里一直在加入大量新功能，对吧？移植工作刚开始时，代码库的大部分是 TypeScript，因此新加入的代码大部分也是 TypeScript。到了项目后期，代码库的大部分已经是 Rust，新加入的代码也大部分是 Rust。但如果你看图表的前半部分，那时 Rust 通常占比不超过 50%，你会发现 TypeScript 代码的数量呈现出这样的走势，而 Rust 代码的数量则呈现出那样的走势。I think in fact you can kind of see that in the charts in the blog post in that for a good portion of the porting effort the amount of rust is increasing and the amount of TypeScript is largely remaining the same but increasing a little bit and that's because as the port was happening there was just a huge amount of new functionality coming into the repo right and at the beginning of the porting effort because the majority of the codebase was Typescript the majority of the new incoming code was also Typescript. By the end of the project where the majority of the codebase was Rust, the majority of the new incoming code was Rust. But if you look at like the first half of the graph where we're generally 50% or less Rust, the amount of TypeScript was doing this even as the amount of Rust was, you know, doing that.

你知道，即便不是那样，我们说的这个项目也有大约 45 万行 TypeScript 代码，光是生产环境的源代码而已，还没算移植过来的测试。我想 TypeScript 里还有几十万行单元测试，也需要移植到 Rust。嗯，想想一个开发者的工作效率，以及即便是顶尖开发者，一天之内合理地能完成多少工作。我们说的可是要一整个开发团队花上一两年才能做成这样的事，对吧？这确实是个……如果你能做到的话，那也会是一项巨大的投入。而且，我甚至很难想象自己去写那份提案，因为我知道那样看起来会很糟糕——我肯定会立刻被打发掉，毕竟我们当然不会做这种事。You know, even if that wasn't the case, this we're talking it was about 450,000 lines of Typescript, just production source, never mind tests that were ported. I think there was a few hundred,000 lines of unit tests there in Typescript as well that also needed to be ported to Rust. Um, just think about the productivity of an individual developer and how much they can reasonably do in a given day if they're top-notch. And we're talking a whole team of developers for a year or two to do something like this, right? It's it is a if you're able to do it at all, it is it would have been a monumental investment. And like I I struggle to think about even writing that proposal because I know that it would be sort of I would look bad like I would be dismissed immediately as of course we're not doing that.

嗯，而这只是一个提案：比如说，我打算在 3 个月里拿出 20% 的时间，到最后把整个东西都迁移过去，而且团队里基本不会有其他人受到影响。嗯，你知道，这并不完全属实，但我就是随口一说，声称几个月后我们就能获得这些业务收益。嗯，而且你知道，这就是账单金额。这样的提案要容易写得多。即便你真的要按那个价格付费——也就是未打折的公开标价，按 token 计费要 12 万美元——把它和一名高级开发人员一年的成本相比，你知道，再加上所有福利和奖金，即便如此也还是很划算。Um whereas this was a proposal like I'm going to take you know 20% of my time over 3 months and by the end this whole thing is going to be moved over and for the most part no one else on the team is going to be impacted. Um, and you know, that's not 100% true, but like waving my hands and we'll have all these business benefits in a few months. Um, and it, you know, this will be the bill. That's a much easier proposal to write. And even just the even if you were paying that actual, you know, uh non-disounted public-f facing uh token rates of $120,000, you compare that to the cost of a senior developer, you know, for a year, uh with all their benefits and their bonus, like even that is a good deal.

听起来，至少对于这类工作来说，我们想要的东西非常明确，而且已经体现在现有功能的某个地方。It sounds like at least for this type of work where it's super what we want is very obvious and encoded somewhere in the existing functionality.

所以，像这种工作至少便宜一个数量级，甚至可能更多。我的意思是，如果你有一支工程师团队，他们的薪酬可能每人每年二三十万到四十万美元左右，那这 12 万美元就只是九牛一毛。你在提出这个项目时，是否大致估算过在实际动手之前会消耗多少 token？So like this type of work is at least one order of magnitude cheaper maybe even more. I mean, if you had a team of engineers and they're, you know, engineers maybe two to 400K or something like that per this is a drop in the bucket, the 120K. When you were proposing the project, did you have a sense of how much this would cost in tokens before you went and did it?

是的。最后账单比我预期的要高。嗯，主要是因为代码行数最后比我预期的要多。嗯，我没有充分考虑到需要移植的代码量会以多快的速度增长。所以，这件事嘛，我觉得有几件事我做得挺对，也有几件事做得不太好，毕竟事后诸葛亮嘛。其中一件我已经提过了：我没有花足够的时间，真正把端到端测试打磨完善，确保它们尽可能全面。这方面我本可以做得更好。另一件我本可以做得更好的事，是认真估算需要移植的代码量的增长速度。我在撰写这份提案时查看了代码库，当时生产环境中的 TypeScript 代码大约有 15 万行。Yes. And the bill ended up being larger than I expected. Um mostly because the line count ended up being larger than I expected. Um I hadn't fully factored in the growth rate of how much incoming code there would be. So this was you know there were there were a few things I think I did a lot of things right. I did a few things wrong and you know 2020 hindsight and all that. One of them I already mentioned I didn't spend enough time uh really fleshing out the endto-end tests to make sure that they were as comprehensive as they could have been. I could have done that better. The other thing I could have done better was really factor in the growth rate of code that would need to be ported. So when I looked at the the code base as part of writing this proposal, it was somewhere on the order of 150,000 lines of production TypeScript.

三十左右吧。我不记得确切数字了。嗯，根据这个数字，再加上一些粗略估算，以及看看之前提到的那些原型花了多长时间才做出来——就是用不同语言创建原型，以便大致衡量性能之类的——我估计大概要六到八周才能完成。嗯，但当你看看我们实际上需要移植的内容时，我漏算了几件事。第一，有几十名开发者在用智能体，很快就为代码库增添了许多不同的功能。嗯，而且这个代码库虽然不是公开的开源项目，但实际上属于内部开源。30 something like that. I don't remember the exact number. Um and based on that and some back of the napkin calculations and looking at how long it took to create the prototypes that I'd alluded to earlier for you know in the different languages to sort of measure Perf and things like that, I think I'd estimated about um six to eight weeks uh to get it done. Um, but when you look at what we actually needed to port, I didn't factor in a few things. One, that there were, you know, tens of uh developers using agents uh very quickly augmenting the codebase with lots of different additional functionality. Um, and uh, you know, this codebase is one that's it's not public open source, but it's effectively internal open source.

所以会有很多人进来添加他们需要的东西，然后由核心团队协助审核这些内容。So there's a bunch of folks that come in and add the things that they need and then the core team is helping to review all that.

嗯，所以功能一下子增加了很多，这是我之前没有预料到的。我也没考虑到，由于代码库形成的方式，CLI、Copilot CLI 和运行时之间的耦合程度很高，远高于理想状态。Um and so there's this very significant influx of functionality that I hadn't accounted for. I also hadn't accounted for the fact that the because of how the code base came about. Um the CLI, the Copilot CLI and the runtime were kind of very intertwined more so than they ideally should have been.

所以，为了只移植运行时，嗯，你知道，如果你愿意，CLI 也可以单独移植。但要真正把它们拆分开，直到你不需要 CLI，也不必将 CLI 作为使用该运行时的应用程序的一部分来部署，就需要投入精力把它们拆开。同时，还需要把那些实际上不必放在 2e 中、而且可以由运行时共享的功能下沉到运行时。我没能预料到，目前被标记为 CLI 的代码中，有这么多其实适合下沉。And so in order to port just the runtime um and you know the CLI can be ported separately if you wanted to but to actually tease them apart to the point where you don't need the CLI you don't need to deploy the CLI as part of your application that's using the runtime there's this effort of teasing them apart but then there's also an effort of moving functionality that really doesn't need to be in the 2e and that could be shared in the runtime down and I failed to account for how much of the code that was currently marked as CLI would actually be beneficial to move down.

因此，在这项工作过程中，也不断有代码交付下来。到最后，我们移植了大约 40 万到 45 万行 TypeScript。我想，这大约是我预估的三倍，也影响了端到端所需的时间，以及 token 数量。And so over this effort, there was also code that was coming down. So by the end um we ported somewhere between four and I think 450,000 lines of Typescript. So about three times what I'd estimated um which impacted both the endto-end time that it took to do it and then also uh sort of token counts.

何时进行全面重写When to do a full rewrite

所以，经历了这个过程之后，如果有个软件工程师在某个团队里考虑进行某种重写，他们应该看哪些标准来判断这件事值不值得做？说到底，关键真的在于商业价值是什么吗？我的意思是，有些事你可以为了好玩去做，花的是自己的钱，值不值得玩由你自己决定。嗯，就像你知道的，我真的不喜欢去赌场，因为我真的不喜欢把我认为是输钱的赌博当回事，但我的一些朋友很喜欢，他们觉得那是在为一晚上的娱乐买单。他们不是在输钱，而是在花钱找乐子，对吧？所以，如果你有自己的个人开发项目，想花代币把它移植到别的技术上，那很好。So after having gone through this process, if there was a software engineer out there on some team considering doing some sort of rewrite, what criteria should they look for to decide if it's worth undertaking it or not? It it really comes down to what is the business value? I mean, there's there's the ones you can do for fun where it's your dime and you get to decide, you know, what fun is worth. Um, just like, you know, I I really don't like going to casinos because I really don't like what I consider to be losing money gambling, but friends of mine love it and they consider it paying for a night's worth of entertainment. They're not losing money, they're paying for fun, right? And so if you're if you've got your own personal development project and you want to spend tokens on porting that to something else, great.

玩得开心。这听起来很棒。但你需要了解它能带来什么好处。好处就是你能从中获得乐趣。对于企业来说，关键在于这件事能带来什么商业价值。然后，和其他事情一样，还要考虑成本是多少，以及投资回报是什么。这些都需要非常明确。世界各地有很多这样的迁移项目，企业早在智能体出现之前就已经在做了；他们评估了成本，并认定投资回报非常值得。我在很多 .NET 项目中都见过这种情况，比如人们从 .NET Framework 迁移到 .NET Core。Have fun. Like that sounds awesome. But but the benefit you need to understand what the benefits are. The benefits are your enjoyment. When it comes to a business, it's what is the business value that this is occurring to. And then like anything else, what is what is it going to cost? Uh and what is the return on that investment? And that needs to be very clear. And there are lots of these migration projects that are people all over the world and businesses all over the world are doing even before agents where they have done that cost assessment and they've decided that the return on investment is very much worthwhile. Um I saw this a lot with net with you know people moving from net framework to net core.

嗯，而且我们在让 .NET Core 变得极快这件事上投入了很多。因此，进行这种迁移会带来非常实实在在的性能提升。Um, and because we invested, you know, a lot in making.NET Core incredibly fast. And so there would be very real performance benefits that would come from doing that migration.

他们就能算出相关成本：嗯，我们现在每年要花 X 百万美元来运行这些服务。通过迁移，也就是进行这次移植，我们每年能省 300 万美元。这次移植要花一年时间，也会占用一名开发人员一年的时间。所以他们算了算，就知道这绝对值得。And they could figure out the cost associated with, well, we're spending X millions of dollars running our services today. And we would save $3 million a year by moving, you know, by doing this port. And this port is going to take a year and it's going to cost us a developer for that year. So they do the math and you know this is this is absolutely worthwhile.

嗯，即使是在更微观的层面，也不只是针对端口：你会投资开发哪些功能？比如，我曾经花很多时间做 .NET 正则表达式引擎相关的工作。我可以看看我们公司每年在正则表达式上花了多少钱，或者某项特定服务每年运行正则表达式要花多少钱。如果我投入两周时间来改进 .NET 正则表达式引擎，让他们使用的模式快四倍，那就……Um even at a a more micro level not even for ports like what features do you invest in? when I was um spending a lot of my time working for example on the net um reax engine uh you know I I could look to see how much are we spending we as a company you know or how much is this particular service spending on running on regular expressions every year if I invest two weeks in making these changes to the net reax engine and it makes the the patterns that they're using four times faster.

我投入两周时间的回报是什么？作为一项服务，它们在一年内能为我们节省多少成本，对吧？这类权衡无处不在。你只需要非常清楚自己为什么要做这件事、它会花费多少，以及你能从中获得什么。如果你能把这些都写下来，并且论证得通，那这就是一个值得做的项目。如果你无法论证，那就很值得怀疑。What is the return on investment for my two weeks and how much they save as a service over the course of the year, right? That these kinds of trade-offs are everywhere. And you just need to be very clear about why you're doing it, what it will cost, and what you get for it. And if you can write all that down and you can justify it, then it's a good project to do. If you can't justify it, it's very suspect.

迈向杰出级别的职业心得Career learnings on the way to Distinguished

我在为这次采访做资料搜集时，了解了一下你的职业经历。嗯，你的 LinkedIn 上没有相关信息，不过我找到了一次你和……做的采访。When I was doing research for this interview, I looked into your career. Well, on your LinkedIn there's no information, but I found an interview you did with

是啊，我非常……我，我基本上……我在微软已经工作 25 年了。我很喜欢这里。我在公司内部换过很多岗位，但我从没想过要去别的公司，所以也没怎么花心思更新 LinkedIn 个人资料。不过后来我做了一番了解，或者说听了你之前关于不同岗位的访谈。你成为杰出工程师的职业路径真的很有意思，因为你做过很多不同的岗位，并不是只在 A 团队做 Sui，然后又在 B 团队做 Sui。你担任过各种各样的职位，最开始几乎像是一名外派工程师、咨询工程师。对。Yeah, I'm very I'm I basically I I've been at Microsoft for 25 years. I love it. I've jumped around a lot internally, but I've had no interest in going to different companies and so I haven't really invested in keeping my LinkedIn profile up to date. But so when I when I did the digging though or like listened to an interview that you had done about the different roles, your path to distinguished engineer is really interesting because you did a bunch of different roles and it wasn't just sui on team A and then Sui on team B. You held a variety of ones starting as almost a like a forward deployed engineer, consultant engineer. Yeah.

后来你还做过一段时间的技术写作，然后当过顾问——一位副总裁的技术顾问。哦，你后来还转去做过 PM。在你尝试过的这些不同角色里，你觉得哪个最……就是说，如果有人想尝试一些不是纯 SU 岗位的工作，你会建议他们涉足哪个方向？And then you did technical writing at some point and then you were an adviser to a a technical adviser to a VP. Oh, and you also jumped to a PM at some point. From all those different roles that you tried, which of them would you say was the most um like you would recommend someone dabble in that direction if they wanted to try something that wasn't the pure SU roll?

是的，我做过的每个岗位都教会了我不同的东西。所以从这个角度来说，我非常尊重那些从大学毕业后想成为开发者、成为工程师，并且一直沿着这条路走下去的人；这对我来说是一条完全合理的道路。我觉得自己走了一条更曲折的路，因此学到了很多不同的技能。比如你提到的前沿部署工程师，当时这个岗位隶属于微软的咨询部门，所以我会在某个客户的现场待上三周到三个月，帮助他们解决问题，同时也会深入了解他们所在的行业。Yeah, every role that I did taught me something different. And so from that perspective like I have great respect for people that you know are come out of college and want to be developers and become an engineer and that's and that's what they do and they just stay on that path like that's a very valid path for me. I think I picked up a whole bunch of skills by having more of a ciruitous route. So like you you mentioned as a forward deployed engineer it was more at the time was Microsoft's consulting arm and so I would spend anywhere from three weeks to three months at a particular customer site helping them solve their problems and I would learn a whole lot about their industry.

我会学到很多，学到一大堆关于他们那类特定问题的知识。我会从外部了解微软提供的产品和服务，知道哪些方面做得很好，哪些方面不太理想。I would learn a whole about a bunch a whole lot about their particular breed of problem. I'd learn about Microsoft's offerings from the outside and places that it was great and places that it wasn't so great.

嗯，了解我们的客户喜欢什么、他们真正面临的困境和正在处理的问题，并能把这些了解带回来，之后再为他们构建解决方案，这非常有价值。嗯，后来我参与了《MSDN Magazine》的工作，做了很多技术写作和技术编辑，也琢磨自己想和大家分享什么、想怎样分享；此外还做了一些开发工作，构建一些东西，然后再围绕它们展开讲述。就这样，我逐渐掌握了一套技能，懂得如何思考什么才是真正有趣、真正相关的内容，如何清晰地表达，哪些内容不需要说，哪些必须说，以及如何讲述一个真正能引起共鸣的故事。um and learn how our customers like their their real struggles and and what they were dealing with and being able to bring that back to then later building solutions for them was incredibly valuable. Um, when uh I then worked on like MSDN magazine and was doing a bunch of technical writing and technical editing and figuring out what I wanted to share with people and how I wanted to share it and um some amount of development building things to then talk about like I I gathered a set of skills around how to think through what was really interesting, what was really relevant, uh how to communicate it in a clear fashion. uh what didn't needed to be said, what did need to be said, how to tell a story that would really resonate.

嗯，从产品经理的角度来说，问题就变成了：我们该如何弄清楚到底需要构建什么，以及要把哪些环节整合起来，才能真正交付这个东西。Um from a PM perspective, it became, you know, how do I how do we figure out what actually needs to be built and what are all the pieces that need to come together to really deliver this thing.

嗯，显然，从开发者的角度来看，最有意思的就是所有编码、设计、架构之类的工作。我觉得对我来说最有意思的角色——我非常感激自己做过，也很庆幸做过，但绝不会再做一次——是在微软担任一位副总裁的技术顾问。嗯，那是一次很棒的经历。我在那里待了几年，算是他的左右手，像个隐形的合作伙伴，跟着他参与所有工作，替他代笔；每次会议结束后，我们都会稍微复盘一下，梳理听到了什么、觉得该怎么应对，然后来回交流想法。Um obviously from a dev perspective all the coding and the design and the and the architectural and all that kind of stuff. The most interesting role for me I think which I am incredibly thankful and glad that I did and will never do again uh was being um sort of as technical advisor for a VP at Microsoft. Um it was an amazing experience. It was a couple years where I was there right-hand sort of like invisible partner um shadowing everything they were doing and ghost writing and after every meeting having a little you know review kind of going through what we heard and what we think we need to do about it and bouncing ideas back and forth.

嗯，这让我对事情实际是如何运作、决策实际是如何做出的有了非常深刻的了解。嗯，我还参与了一些原本不会有机会参与的事情，至少要等很多年以后才有机会。Um it was incredibly insightful into how things actually work, how decisions actually get made. um being a part of things that I wouldn't otherwise have been a part of, at least not for many, many years later.

嗯，那真是太棒了。同时也极其耗费精力，需要付出很多。呃，而且这让我消耗很大。嗯，所以我很珍惜那些经历，也很珍惜从中学到的东西。我非常鼓励任何有机会的人，都花一段时间去做这样的事。Um it was awesome. It was also incredibly draining, incredibly selfless. Uh and it it took a lot out of me. Um and so I cherish those experiences and what I learned from them. And I would highly encourage anyone who has the opportunity to do so to do that kind of thing for a period of time.

这个职位真的很有意思，因为听起来你基本上就像是一位技术领域的高层人士的幕僚长。所以你不是行政助理，而是会参与所有那些——我想应该算是——非常内部的谈话，并提供他们做决策所需要的信息。你提到自己学到了事情实际上是怎么运作的。嗯，如果有人担任类似的职位，你能举几个他们可能学到的例子吗？或者有什么经验对你特别有帮助？That role is really interesting cuz it sounds like you're uh you're basically a chief of staff to someone super high up but technical. So you're not like an executive assistant. you're like the you're pulled into all the really I guess privileged conversations and you're providing that that information that they need to make the calls. You mentioned that you learned how things are like really done. Um do you have some examples of stuff that someone might learn if they were kind of in that that role or something that was really helpful for you?

很多时候，如果你只是公司里一名普通的一线工程师，看到公司做出这些决定——比如发邮件宣布要走一条新路线、建立某种合作关系、进行一项投资、收购某家公司、逐步停止某款产品、投资开发一款新产品，或者组建一支新团队——这些事情看起来都像是凭空冒出来的。你知道，它们就像是直接被上面传达下来的，仿佛决定已经敲定、方案也早已成形。There are lots of times where if you're sort of rank and file engineer at a company and you see these decisions that get made, an email gets sent out about some new path we're following or some partnership we're forming or some investment that we're making, some acquisition that is being done, winding down a certain product, investing in a new product, spinning up a new team, like all of those things look like they just sort of come out of thin air and they're, you know, they're just sort of handed down as this is the finished decision and it's fully formed.

你其实无法真正了解一路走到那一步的所有情况，也不知道当时考虑过哪些备选方案、又否决了哪些，为什么否决，决策时权衡了哪些利弊，以及业内经验丰富的人是如何思考这些备选方案和问题的，还有他们选择把注意力放在哪些事情上、认为哪些事情无关紧要。嗯，这些都是我以前接触不到的。而能够像个隐形旁观者一样——呃，充分吸收这一切，我不是想混用比喻，不过你知道的——然后还能就这些内容展开交流。比如我们开完会出来，他基本上就会问：“那么，你听到了什么？你怎么看？”You don't really get insights into everything that led up to that point and all of the alternatives that were considered and dismissed and why and what the trade-offs were that were factored in and how experienced people in the industry are thinking through those alternatives and those problems and h which things they're choosing to focus on, which things they consider to be irrelevant. Um, all of that is stuff that I didn't have exposure to. And being able to be a fly on the wall, uh, just soaking in, not to mix metaphors, but, you know, soaking in all of that and then being able to have conversations about it to, you know, I we'd get out of a meeting and he'd basically say, "So, what did you hear? What do you think?

“就是，帮忙，你知道吧，帮着一起理清思路。”有时我会提出有价值的见解，有时他会说不。Like, help, you know, help reason through it." And sometimes I would have valuable insights and sometimes he would say no.

那并不是我们看待这些事情的方式。对我来说，这些经历每一次都是一次学习。That's that's not how how we think about these things. And and every one of those was a learning experience for me.

是的。我也听工程师这么说过：尝试做管理的价值在于，即使最后你发现自己讨厌管理、不想再做了，你也能……是啊。作为管理者，你能更多地看到事情背后的运作方式。你会看到管理整个组织时做出的那些决策，这对实际开展工作，或者说，对你的整个职业生涯都会有所帮助。Yeah. I've heard engineers say similarly about the value of trying management even if they ultimately decide they hate it and they don't want to do it again is you Yeah. You get to see how the sausage is made a little bit more as a manager. you see all those decisions uh in managing the org that can be helpful in operating uh just in your your career in general.

是啊。而且你知道，我之前没提过，我也做过这件事——我当时以为自己真正想做的是管理者。所以我基本上安排好了：在某个时候，我在微软内部换过很多岗位，后来转到了一个新职位，当时我们有个非正式的约定，六个月后我就能转到一个领导岗位，带一支团队。我把这一切都安排好了，最后也顺利实现了，我成了团队负责人。做了六个月负责人之后，我心想：“我不确定这是不是我原以为的那样。”又过了六个月，我就想：“不，肯定不是。这不适合我。”然后我又花了大约一年的时间，才稳妥地回到 IC 岗位，并一直做到现在。Yeah. And you know I also I I'm I didn't mention it before but I also did that like I I thought that what I really wanted to be was a manager and um so I basically set things up whereas a certain point I trade you I jumped around a lot internally at Microsoft and I changed to a role where I had sort of handshake agreement that six months later I would be able to sort of shift into a lead position and have a team of of people. And I arranged all that and it all worked out and I became a lead. And six months into being a lead, I was like, I'm not sure if this is actually what I thought it was. And six months after that, I was like, nope, it's definitely not. This is not for me. Um, and then it took me about a year after that further to sort of um, safely work my way back to uh, an IC role where I've been since.

嗯，再次回应你刚才说的，我真的非常感激这段经历，也非常感谢微软设立了个体贡献者职业发展通道，让人可以在整个职业生涯中不断成长、晋升之类的，同时又不必直接管理下属。对我来说，我后来意识到，我喜欢、或者说我以为自己想当经理的原因，是我觉得这样才能帮助他人，给别人一些职业发展指导，或者帮他们解决问题之类的。但这是我当时想错了。直到我成为经理之后才意识到，作为个体贡献者，我也能做到这一切。而我喜欢做经理的那些事情，其实并不需要成为经理才能做。Um, and again to your point like super grateful for the experience, incredibly grateful to Microsoft for having the for having an individual contributor career track that allows one to sort of progress their whole career and continue to be promoted and whatnot, but still not have to have direct reports. For me, what I realized was the thing that I loved about or that I thought the reason I wanted to be a manager was because I thought it was how I could help others and how I could kind of provide uh career guidance or help other people solve problems or whatever it was. And that was misguided on my part. And it took becoming a manager to realize that I could do all of that as an individual contributor. And the things that I that I loved about being a manager were things that I didn't have to be a manager to do.

最佳技术类书籍推荐Top technical book recommendation

有没有哪本最推荐的书对你产生过影响？Do you have any top book recommendation that maybe impacted you?

是啊，我的意思是，它到现在已经算是老书了，不过我读过好几遍。嗯，我真的特别喜欢它。Charles Petzold 写过一本叫《编码》的书。大概是 2003 年左右吧，我也不太确定，反正是那个时期的。但我很喜欢这本书，而且我特别看重的一点是，它能把复杂的概念解释得很清楚，让读者完全不需要了解你在讲什么。Yeah, I mean it's old at this point, but I've read it several times. Um, and I absolutely love it. There's a book by Charles Petzold called Code. It's probably from what 200 three maybe, I don't know, something in that time frame. But I love it and it is one of the things that I really value is being able to explain complicated concepts in a way that doesn't require the person to know anything about what you're talking about.

我认为这是一项很有价值的技能，即使你是在和懂你所说内容的人交谈也是如此。因为能够把复杂概念拆解成清晰、简洁的部分，清楚地表达出来，再从一个很宏观的层面进行讲解，是让所有人达成共识、阐明自己的观点并推动事情朝着你希望的方向发展的好方法。《Code》这本书里，查尔斯基本上就做了这样的事：他从最基础的原理讲起，介绍计算机是如何工作的，向那些对计算机一无所知、没有技术经验的人讲解二进制、逻辑门、电路、编程、计算机和互联网，一步步从零开始搭建知识，帮助读者合理地理解这一切是如何运作的。I think that is a valuable skill even if you're talking to people that do know what you're talking about because being able to break down complicated concepts into very clear concise pieces and communicate that clearly and then at a 30,000 ft level is a is a great way of getting everyone onto the same page and sort of arguing your case and pushing things in the direction that you want them to And code is a book where Charles basically does that where he talks about how computers work from the ground up teaching people that know nothing about computers that aren't technically experienced about binary about logic gates about circuits about programming about computers about the internet and kind of building things up from basic principles from nothing to a reasonable understanding of how everything works.

嗯，那本书让我大受震撼，从那以后我又重读过好几遍。尽管我已经熟知书中的所有内容，但我就是特别喜欢它传达这些内容的方式。所以，如果只能推荐一本书，那就是它。Um, and I was blown away by the book and I've since reread it on multiple occasions. Even though I know everything that's in there, I just love the way that it communicates things. So, if I had one recommendation that that's it.

给年轻工程师的建议Advice for young engineers

回顾你的所有经历，你已经在这个行业工作了一段时间。对于今天刚进入这个行业的年轻工程师，你有什么认为会有所帮助的职业建议吗？这是个很棒的问题，也是我一直在琢磨的问题。嗯，几年前，我可能会给出一个清晰得多的答案，但有了 AI 之后，我还是不知道什么才是正确答案。嗯，很多时候，我在和 AI 智能体一起工作时，会运用自己 25 年的经验——不管这是好是坏——迅速判断它们做得对不对，以及我该如何引导它们。或者智能体会说：“我百分之百确信，正确答案是 A。”而我会说：“好吧，不过你有没有也考虑过 B、C 和 D？因为这些权衡可能并不存在。”Looking back on all your experience, you've kind of been working in the industry for a while. What professional advice would you give a young engineer just entering the industry today that you believe could be helpful? That's a great question and it's one I've been struggling with. Um, a couple years ago I would have had a much more clear answer, but with AI, I still don't know what the right answer is. Um, a lot of the times that I spend working with AI agents, I'm bringing to bear 25 years of experience for better or worse in being able to very quickly sniff out things when they're doing the right thing or they're doing the wrong thing or how I want to guide them. Or an agent says, I'm 100% confident that the right answer is A. And I'll say, "Yeah, but have you also considered B and C and D because these trade-offs might not be there?"

对吧？而且，哦，当然。光是这些小事，要能引导他们走上正确的道路，就要求你自己经历过这一切。我担心的是，随着 AI——随着 AI 智能体的发展，他们正在剥夺人们亲身经历这一切的机会。我还不知道该如何应对。嗯，我非常坚信，真正学好基础知识、从核心原理出发逐步搭建起一切，是很重要的。所以，比如学习一门编程语言时，我仍然认为学习并从 C 这样的语言入门很有价值，因为这样你才能理解计算机实际上是如何工作的，理解内存管理意味着什么，也能了解各种算法之间的权衡，而不是停留在一个许多底层细节都被遮蔽起来的层面上。Right? And oh, of course. Like just those little things being able to guide them in the right way requires you to have gone through it. And I'm concerned that because of what AI how AI agents are progressing that they're taking away the opportunities to have gone through it. And I don't yet know what the answer then is. Um I am a very firm believer in um really learning kind of the basics and building everything up from core principles. And so like when you know learning a language, I still think it's valuable to learn and to start out with a language like C because you're understanding how computers actually work and you understand what memory management means and you learn about all the various trade-offs between this algorithm and that algorithm rather than working at a layer where a lot of that is obscured from you.

嗯，在这种情况下，语言和运行环境会为你提供大量服务，但你未必完全理解其中发生的一切。所以从这个角度来说，我仍然认为，专注于那些核心概念，并真正内化算法以及机器的实际工作原理，是有价值的；这样你就能据此进行推理和推演。我猜，无论你身处哪个领域，关于分布式系统或数据库的这类核心基础知识，都会帮助你在这样一个世界里取得成功：绝大多数代码实际上都是由智能体编写的，而你则负责引导它们。不过，我也不敢确定。uh where the language and environment are providing a ton of services to you but you don't necessarily understand everything that's happening and so from that perspective I still think that there's value in focusing on those core concepts and really internalizing algorithms and how machines actually work um and being able to reason and extrapolate from that and I suspect that that will in whatever domain you're in. That kind of core primitive knowledge about distributed systems or about databases like that foundational stuff will help to be successful in a world where the vast vast vast majority of the code is actually being written by agents and you're helping to guide them. But I don't know that for sure.

而这些智能体学习和进化的速度如此之快，我不知道两三年或四年后世界会变成什么样。And at the rate that these agents are learning and evolving, I don't know what the world's going to look like in two or three or four years.

嗯，我觉得它将会和今天大不相同。我想，接下来会是一段相当疯狂的旅程。Um, I suspect it'll look vastly different than it does today. I think we're in for a wild ride.

太棒了。嗯，Stephen，非常感谢你今天抽出时间。我真的很感激。Awesome. Well, yeah, thank you so much for your time today, Stephen. I really appreciate it.

很高兴认识你。这次谈话很愉快，谢谢。Pleasure. This has been a fun conversation. Thank you.

结尾Outro

嘿，感谢你收看这个播客。如果你喜欢这个节目，也希望它继续发展壮大，请评论或点赞支持我们。另外，如果你有什么想让我邀请来的嘉宾人选，也请留言推荐。像芭芭拉·利斯科夫、迈克·斯通布雷克、马克·布鲁克这样的嘉宾，都是因为有人留言推荐，我才邀请他们来的。说点别的，除了做播客，我还在打造一款我一直希望能问世的人体工学键盘。来看看原型吧。它是一款分体式键盘，由两部分组成。嗯，这是装在外壳里的样子。我们已经在 Kickstarter 上发起众筹，并在上线八小时内就达成了目标。Hey, thank you for watching this podcast. If you liked it and you want to see the show grow, please support with a comment or a like. Also, if you have any recommendations for people you want me to bring on, please drop a comment. Guests like Barbara Liskoff, Mike Stonereaker, Mark Brooker, these were all people that I brought on because someone left a comment. On another note, aside from the podcast, I'm working on building the ergonomic keyboard that I wish existed. Here's a glance at the prototype. It's a split keyboard, so there's two sides. Um, this is in the case, but yeah, we launched on Kickstarter and we hit our goal within eight hours of launching.

如果你是最早入手早期批次产品的人之一，我真的非常感谢。嗯，我们现在正着手开发生产所需的工具，这会是一个漫长的过程。如果你还想入手一台，我已经在 Kickstarter 上开放了后续认购，你可以去那里购买。I really appreciate it if you were one of the people who grabbed one of the early units. Um, we're now working on the long journey of building the tooling now and so if you still want to pick one up, I've left the late pledges open on Kickstarter, so you can grab one there.

我会把链接放在简介里。再次感谢你收看这期播客，我们下期再见。I'll put a link in the description. Thank you again for watching the podcast and I'll see you in the next