走进 Uber 的 AI 工程基础设施

简要描述Brief Description

从内部看 Uber 的 AI 基础设施Uber’s AI infra， from the inside

目录Table of Contents

• 简介
  Intro
• Uday 在 Uber 的早期岁月
  Uday’s early days at Uber
• Uber 向微服务架构的转型
  Uber’s move to microservices
• Uber 的工程团队：过去与现在
  Uber’s engineering team, then and now
• AI 之前的开发者工具
  Developer tooling before AI
• DevPods
  DevPods
• 为 AI 智能体构建项目
  Building for AI agents
• MCP 网关
  MCP gateway
• Agent Skills 仓库
  Agent Skills repo
• 上下文图谱与 Cortana
  The context graph and Cortana
• PRD 和 ERD 工具
  PRD and ERD tool
• 爪牙
  Minion
• 使用 Bazel、Slate 和 Capri 进行测试与代码审查
  Testing and code review with Bazel, Slate, and Capri
• 边缘网关、牧羊人和智慧
  Edge Gateway, Shepherd, and Wisdom
• 工程师角色的变化与生产力衡量
  The changing role of engineer and measuring productivity
• Uday 最喜欢的新产品
  Uday’s favorite new products

简介Intro

如今的开发是什么样的？你们使用什么类型的工具？What does development look like today? What kind of tools do you use?

主持人：我们提供一个兼容 OpenAI 或 Anthropic 的 API，所有供应商的所有模型——无论是开放权重还是闭源权重——都通过这个网关提供。Host: We expose one OpenAI compatible API or anthropic compatible API and all of the models across all the vendors open weight or closed weight. They're all behind this gateway.

主持人：什么是 AI guard？Host: What is AI guard?

嘉宾：你知道，我们想确保不会向用户返回任何敏感内容。大约在 2025 年 4 月，我们开始开发 MCP 网关。它类似于模型网关，是一个统一管理所有工具的网关。我们有 100 多种节点类型、130 多种边类型，这些都是大概 3 天前的实时数据。我们称它为 Slate。基本上，你可以创建自己的实例来部署你正在开发的分支。对于一些最复杂的平台，我们把这称为捕获回放——本质上，只需一个中间件，你就能捕获真实的生产流量。Guest: You know, we wanted to make sure we are not sending any sensitive content back to the user. Around April of 2025, we started working on MCP gateway. Similar to model gateway, this is one gateway for all of the tools. We have 100 plus node types, 130 plus edge types and this is live data as of say like 3 days ago. We call it slate. Essentially you can create your own instance that deploys the branch that you're working on. For some of our most complex platforms we call it capture replay like essentially you can capture real production traffic with just one middleware.

主持人：与 AI 出现之前相比，你对软件工程的认知发生了怎样的变化？Host: How is your mental model changing of software engineering compared to preAI?

嘉宾：在我整个工程职业生涯中，工作方式从未发生过如此大的变化。例如，Guest: In my total tenure in engineering there's never been this much change to how I work. For example,

主持人：Uber 的软件工程师会使用哪些内部工具来构建软件？今天，我们将通过 Uber 任职时间最长的工程师之一 Udai Kira Mendesi，独家深入了解这些工具，并看到大量相关画面。如今，Udai 负责 Uber 的许多智能体 AI 相关工作。在这次对话中，我们会深入探讨内部开发者系统。Uber 构建并使用过许多前 AI 时代的系统，例如 M3DB、Michelangelo、Cadence、Morpheus 等；也构建了 AI 基础系统，例如 MCP 网关、AI Guard 和 AIX。还有近乎所有工程师都在使用的近期系统，例如 Minion、Code Inbox、Slate、Shepard、Wisdom 等。如果你想了解，在 Uber 这样拥有数千名工程师的现代科技公司里，AI 原生开发是什么样的，以及为什么在经过实战考验的开发者基础设施之上改造、融入 AI 要容易得多，那么这一集就是为你准备的。Host: what kind of internal tools do software engineers at Uber use to build software? Today, we get an exclusive insider and visuals heavy look from one of Uber's longest tenure engineers, Udai Kira Mendesi. Today, Udai leads much of the work on Agentic AI at Uber. In this conversation, we get into internal developer systems. Uber has built and used PreAI like you own, M3DB, Michelangelo, Cadence, Morpheus, and others. AI foundational systems a company built like its MCP gateway, AI guard and AIX. Recent systems used by nearly all engineers like Minion, Code Inbox, Slate, Shepard, Wisdom, and others. If you want to understand what AI native development looks inside of a modern tech company with thousands of engineers like Uber, and why it's much easier to retrofit AI on top of battle tested developer infra, this episode is for you.

本期节目由 Turbopuffer、Vector 和基于对象存储构建的 Fulltext Search 呈现。它速度快、成本低，而且扩展性极强。本期节目将介绍 Uber 构建的近二十套内部系统，这些系统旨在帮助工程师借助 AI 代理交付代码。节目中的一个要点是，生产就绪的代码远比使用 AI 编码框架复杂得多。框架是简单的部分，真正困难的是围绕它们的一切。接下来要提一下本季赞助商 Work OS。代理面临的一个棘手问题始终是权限：你希望代理始终拥有访问它可能涉及的一切的权限吗？还是每个提示都要你逐一批准，直到你干脆不再看这些提示？Work OS 为此打造了 Airlock。Airlock 是一个授权层，位于代理和它们调用的工具之间，会根据代理的意图和你的规则审核每一个请求。This episode is presented by Turbopuffer, Vector, and Fulltech Search built on object storage. It's fast, cheap, and extremely scalable. Today's episode is about the close to two dozen internal systems that Uber has built to help engineers ship with AI agents. One of the takeaways will be that production ready code is far more complicated than just using an AI coding harness. The harness is the easy part. What's hard is all the things around them. This is where I need to mention our season sponsor, Work OS. One piece that is always a hard problem with agents is permissions. Do you want an agent to have standing access to everything it might touch? or do you just approve every single prompt until you just stop reading the prompts? Worker was built airlock for this. Airlock is an authorization layer that sits between your agents and the tools they call and it judges every single request against the agents intent and your rules.

你用通俗易懂的英语编写规则。智能体永远不会看到你的凭据，而且每项判定都会被记录。它适用于 Cloud Code 和 Codeex 等编程智能体。Works Airlock 让智能体无需获得过多权限也能完成实际工作。欢迎访问 work.com/airlock 了解详情。You write the rules in plain English. Agents never see your credentials and every verdict is logged. It works with coding agents like Cloud Code and Codeex. Works airlock is how agents do real work without over permissioning. Check it out at work.com/airlock.

Uday 在 Uber 的早期岁月Uday’s early days at Uber

Udai，欢迎来到播客节目。Udai, welcome to the podcast.

嘉宾：非常感谢邀请我来。我一直在关注你的工作。看到你产生的影响，以及你为所有工程师发布的精彩内容，真是令人惊叹。能来到这里，今天和你聊聊我们正在做的事情，我感到无比荣幸。Guest: Thank you so much for having me. I've been following your work. It's amazing to see the kind of impact you've had and uh all of the great content you publish uh for all of the engineers. So, I'm super honored to be here and talk to you today about what we doing.

主持人：嗯，我们早在《务实工程师》诞生之前，就都在 Uber 工作过。不是一起共事，不过我也了解你的工作。你是什么时候加入 Uber 的？没错。Host: Well, we we worked together well before the pragmatic engineer was born back at at Uber. Not not together, but I knew of your work as well. When did you join Uber? Exactly.

嘉宾：是的。我是2015年5月加入Uber的。事实上，推荐我来的那两位工程师现在还在这里。我加入的是实时 API 团队。当时我们正试图把单体架构拆分成微服务，我负责所有移动应用的 API 网关，也参与了移动端协议的改造。之后，我在许多不同的业务平台上工作过，包括履约、Uber for Business、创收者入驻、市场定价。过去这一年，我一直在做智能体 AI 相关的工作，涵盖我们的用户、客户支持、终端用户等。所以，过去这一年做这方面的工作真是像坐过山车一样。Guest: Yeah. So I joined Uber in uh May 2015. In fact like the the two engineers who referred me, they are still here. Um so I I joined in um our real time API team. Uh we it was during the period where we were trying to break out break our monoliths into microservices and I was working on our API gateway for all of our mobile apps and like I worked on changing our mobile protocol. Since then I worked on many different business platforms from our fulfillment, Uber for business on our earner onboarding, marketplace pricing and the last one year has been all agentic AI both for our users, customer support, end users and so on. So yeah, it's been like a roller coaster last one year on working on this.

主持人：是的。对于那些当时没有和我们一起在那里工作的人，我们能不能稍微回顾一下实时 API，也就是我们以前在内部称作 RT API 的东西，它是什么、为什么它很重要，以及为什么负载那么大？我刚加入时，Uber 当时只有两个服务：一个是用 NodeJS 编写的 Dispatch，另一个是用 Python 编写的 API。当时 Dispatch 负责处理乘客和司机的所有流量。那时的协议相当简单，就是我们称作“四秒 ping”的协议，它会告诉你整个世界的状态；行程结束后，我们就把请求交给 API，由它处理行程后的所有流程、付款、收据等等。RT API 就是在那个时候诞生的，目的是抽象掉所有内部服务和移动端，同时把协议从简单的四秒协议改成一种新的方式：既有 REST API，也支持服务器与客户端之间的通信。Host: Yeah. And for for those people who didn't work there with us, can can we recall a little bit what real time API or RT API as we used to call it inside what it was and why it was a big deal and why it had so much load. So when I joined I like um we had just two services at Uber. It was called dispatch written in uh NodeJS and API written in Python. Dispatch used to handle all of the rider and driver traffic at that point in time. And the protocol was pretty simple at that point. It was like a 4se secondond ping protocol we used to call that would give you the entire state of the world and when the trip is completed we would hand off to API. It would handle all of the post trip processing payments receipts and everything. RT API was born at that point in time to abstract all of the internal services and mobile and also changing the protocol from simple 4C protocol to back like where we have both uh rest APIs and we also have server to client communication.

我们以前把它叫作推送协议。因为我们的应用有很多内容都是动态的，几乎每隔几秒就会变化，所以我们需要一种高效的方式，在收到新更新、现实世界发生变化时，及时更新乘客端和司机端应用的状态。那时，它是 QPS 最高的服务之一。多年来，由于我们的技术栈全都是 NodeJS 和 Python，所以当时自然就选择了这些技术。但在随后的几年里，我们把它改成了 Go。如今，它仍然是 Uber QPS 最高的服务之一。我想，听到这里的朋友可能会很惊讶：“哦，2015 年前后，Uber 竟然只有两个服务。”毕竟，Uber 以微服务浪潮中的先行者而闻名。We used to call it push protocol. So we would uh because so much of our app is very dynamic everything keeps changing uh every few seconds. So we needed a way to efficiently update the state of the app across both riders and drivers as and when we get new updates as and when things change in physical world. Um so it used to be at that point like one of the highest QPS uh services and over the years we kind of uh because all of our stack was just NodeJS and Python. So that was the natural choice at that point. But in the subsequent years we have converted that into go and it's still one of our highest VPS services at Uber. And I think for people listening, they're going to be surprised when they hear like, "Oh, there was only two services in 2015 or around that time at at Uber because of course Uber was known for the microservices boom."

Tuan 也上过这档播客。他稍微谈过这件事，不过你介不介意……And Tuan was on this podcast. He talked a little bit about this, but would you mind just

Uber 向微服务架构的转型Uber’s move to microservices

回想一下这些微服务是怎么出现的，从你的角度来看，突然有这么多服务意味着什么？recalling how those microservices came to be and from your perspective, you know, like what it meant to suddenly have so many services?

嘉宾：是的，我是说，我觉得关于这件事有很多博客，也有很多人讨论。所以我觉得，我刚加入的时候，正如我说的，我们只有两个服务。回想当时，我们正处于高速增长阶段，不断拓展到新的国家，不断扩充出行服务的产品组合，后来又推出了 Uber Eats，它也在不断扩张。我们需要一种机制，让各个团队能够快速构建各种功能，以跟上业务增长的规模。因此，我们在各种服务基础设施上投入了很多，比如让团队更容易构建服务，以及指标、链路追踪、监控、可观测性、部署等等一整套相关工具。在当时那种规模和变化速度下，这些东西几乎没有多少现成软件可用，所以我们那时不得不在内部构建很多东西。这推动了一个繁荣周期，服务数量从两个增长到了数千个。后来，大约在 2018、2019 年，我们统一了所使用的编程语言。Guest: Yeah, I mean, I think uh there's like so many uh blogs and like so many people talk about that. So I think like when I joined right like we were um as I said like we had just two services and uh there and you also think going back in those days the kind of hyper growth phase where we were in we were constantly expanding to new countries we were constantly expanding our product portfolio across mobility and later we started Uber Eats and that also was expanding. We needed a mechanism for teams to really build all kinds of functionality very quickly um to match up to the scale at which the business was growing. So we have invested a lot in all kinds of service infrastructure making it easy for teams to build services metrics tracing monitoring observability deployment the whole bunch of stuff around that and there's not much off-the-shelf software um for these things at that point at the scale and rate of change that we had and there's a lot of stuff that we had to build in those days internally and that created a boom cycle of a lot of uh we went from like two services to order of thousands of services um Then um around 2018 2019 was when we standardized on our languages.

我们选择 Go 和 Java 作为主要编程语言，并对技术栈进行了大量现代化改造。我们花了很多时间思考架构的各个层次：如何实现平台化？事实上，多年来，在我负责其中一个履约平台时，我们还将一些服务精简成了少数几个服务。即使是现在，我们的服务数量也大致处于相同的水平。尽管我们的业务规模扩大了 10 倍，提供的产品也多了很多，但经过大量的平台化工作和多年的投入，我们仍在所有运营国家使用同一套全球技术栈。许多平台通过共享的基础构建模块，为所有出行垂直业务和配送垂直业务提供服务。我认为，多年来我们在很大程度上实现了平台化，这让我们能够构建更多出行垂直业务和配送垂直业务。We picked Go and Java as our primary languages. We modernized a lot of our text stack. We spent a lot of time in thinking about the layers of our architecture. How do we platformize? And over the years in fact when I worked on one of our fulfillment platform we actually reduced some of the services to few services. Even now we are kind of in the similar ballpark of number of services. Even though our business is 10 times larger, we are doing many many more products and uh this whole bunch of platformization and uh years of investment that has gone into it like we still running one single uh global tech stack across all the countries we operate in. There's many many platforms that serve all of the mobility verticles and all of the delivery verticles using one shared building block. And I think that kind of allows us to build more mobility verticles, more delivery verticles because we have platformized a lot over the years.

主持人：您能告诉我们您什么时候……吗？Host: Would you mind telling us when you

Uber 的工程团队：过去与现在Uber’s engineering team, then and now

我加入时，Uber 的工程团队有多大？这些年又是如何发展的？因为我觉得外界并不太了解这段增长有多么惊人。我加入时，大约是第 600 位工程师。事实上，当时整个 Marketplace 团队大概只有 50 到 60 名工程师。接下来的几年里，随着业务和流量规模不断扩大，我们的工程师人数也增长到了几千人左右。到了 2019、2020 年前后，我们也开始在不同地区扩展布局。现在，我们在全球各地共有 12 个技术中心，几乎遍布我们开展业务的每个大洲。这有助于我们为各个本地市场打造产品，其中一些产品随后也会推广到所有能够支持这类功能的市场。joined, how big was Uber engineering team and how did it grow over time cuz I don't think people from the outside appreciate just how big that growth was. So when I when I joined uh I was around the first 600 engineer um and in fact the whole of marketplace team at that point used to be like maybe 50 60 engineers and over the next few years as we've scaled uh our business as we scaled our traffic like the we we grew to some few thousands of engineers and around 2019 2020 we started also expanding across different sites now we have 12 global tech sites across the world in almost every continent uh we operate in um in fact and that helps us build like products for those hyperlocal markets and some of those products are then rolled out globally across all the markets where we could roll out that that kind of functionality.

所以我觉得，这让我们有很大的能力，比如说，能够在全球范围内发掘人才，同时打造更贴合本地需求的产品。这也有助于我们建立一套全球统一的技术栈。说到打造产品和技术栈，你之前——现在也还在——参与这个平台的工作……So I think that gives us a lot of ability to like you know tap into talent globally and also build uh nuanced local products um and that also helps us build like one global tech stack. And speaking of building products, tech sacks, you you worked you still work on on the platform

AI 之前的开发者工具Developer tooling before AI

团队，对吧？team, right?

嘉宾：是的。嗯，至少在最初的十年里，我一直在业务团队——也就是不同的业务平台团队——工作。嗯，过去一年，我花更多时间在开发者平台上，做面向工程师的智能体 AI，以及面向公司其他部门的智能体 AI。在这个过程中，我也参与了一些面向用户的 AI 项目。能不能带我们回到 2015 年，聊聊 Uber 的内部工具是如何变化的？因为我当时也在 Uber，看到我们有那么多定制工具，真的很惊讶。我们聊过 RT API，但还有 Morpheus edge gateway 之类的东西。能不能谈谈 2015、2016 年的时候，工程师加入 Uber 后会用哪些工具，这些工具后来又是如何演变的？我说的显然是那些主要工具，也就是当时大多数工程师，尤其是移动端工程师，在 Uber 都会接触到的那些工具。Guest: Yeah. So, um at least like until my first 10 years, I worked on business uh different business platform teams. Um and in the last one year, I'm more spending my time on uh on a developer platform on a agentic AI for engineers, agentic AI for rest of the company and in in the process I've also worked on some userfacing AI initiatives as well. C can you take us back 2015 on how Uber's internal tooling changed because again from I I I was there and I was really surprised to see just how much custom tooling we had. We talked about RT API but there was things like Morpheus edge gateway and other things. Would you mind talking through like back in 2015 2016 if an engineer joined Uber what kind of tools did they use and how did those evolve? I'm talking obviously you know the the main that the main ones that most engineers back under mobile engineers touched at Uber.

是的，当然。我觉得，嗯，在那个时期，考虑到我们的扩张方式，没多少现成产品可供我们使用。所以我想，我们谈到了 RT API 和边缘网关；我们还必须构建一些其他东西，比如一个叫做 You Own 的系统，它能让公司里的任何人识别出某项特定资产的负责人。Yeah, absolutely. I think um like in those days given the way in which we were scaling there's not many off-the-shelf products that we could tap into and like so I think we talked about RT API and edge gateway some of the other stuff that we had to build was a system called you own that will that will enable anyone in the company to identify who owns a specific asset

主持人：就像，像是一个服务之类的，对吧？然后如果值班期间发生故障，你可以查一下，弄清楚是哪个团队负责，谁在值班、谁不在值班之类的，对吧？Host: like like it was like a like services right and and you could look at the during on call if there's outage you could figure out which team owns it who's on off for that kind of stuff, right?

嘉宾：是的，当然。随着工程团队规模不断扩大，我们还必须搭建一套端到端的体系，比如把工程设计文档、产品设计文档都集中到一个地方。我们还得把所有的事故管理工作也集中到一个地方。我们非常重视如何处理事故，以及从中吸取经验教训，所以围绕这方面开发了很多很棒的工具。我觉得其中比较复杂的一项是指标系统。早在 2015、2016 年，我们每秒大概就会产生 5 亿个指标，而且基数很高。当时没有现成的指标系统可供我们使用，这促成了 M3DB 的诞生。后来它开源了，甚至还催生了一些围绕它成立的公司。这使我们能够处理那种规模的指标数据量。我们还必须构建自己的端到端机器学习平台，因为我们当时有数百个机器学习模型。如果把整个生命周期都考虑进来，从创建特征、构建模型，到监控和服务……Guest: Yeah, absolutely. And uh as the engineering size grew, we also had to build like end to end uh uh like our engineering design docs, product design docs, getting all of those into one single place. We had to work on getting all of our incident uh manager into one single place. We are like really obsessed about how we handle our our incidents, the learnings from that. So we we built a lot of really good tooling around that. I think the some of the more complex ones was our metrics like even back in 2015 2016 we used to probably have around like 500 million metrics emitted per second with with the cardality and there's not there's no offtheshelf uh metric system that we could use and that gave uh rise to M3DB which was open sourced later and like there's even companies formed around that um and um that enabled us to build like the kind of uh so like handle kind of metrics volume that we We also had to build our own end to end machine learning platform because we used to have like hundreds of ML models and if you take the whole life cycle from creating the features building the models monitoring serving

主持人：这、这是米开朗基罗，对吧？Host: this this is Michelangelo right

嘉宾：Michelangelo。就在最近，大概几个月前，我们开源了深度学习以及 Michelangelo 基础设施的一部分，所以我觉得那里有很多非常扎实的基础构件。嗯，另一个有意思的项目是 Cadence。如果你想想一些工作流，它们会跨越好几天，比如司机入职流程，或者某个 CRM 营销活动，你会发现很多工程师都在各自实现容错机制和状态管理。这就催生了 Cadence，一个持久化工作流引擎，后来它也开源了。Guest: Michelangelo and like we recently just few couple of months ago we open sourced our deep learning and like part of the Michelangelo infrastructure so I think there's a lot of like really good solid foundational building blocks there. Um the other interesting one is also cadence which is um if you think about some of the uh workflows right like they had they span across multiple days whether it's a driver onboarding whether it could be some CRM campaign you could see a lot of engineers doing their own resiliency stuff their own like state management and that gave rise to cadence which is a durable workflow engine and later even that got open sourced

主持人：我想，知道 Cadence 的人不多，但很多人都知道 Temporal，对吧？正如我刚才所说，它要么是受 Cadence 启发，要么很多理念都源自 Cadence。我甚至不确定它是不是 Cadence 的开源版本，但它有很多类似的东西，可以说是 Cadence 对 Uber 而言（或曾经而言）的某种外部版本。Host: I think not many people will know cadence but a lot of people will know temporal right which was as I said either inspired by Cadence or a lot of the ideas came from there. I'm not even sure it was the open source version of Cadence, but it had a lot of like it's kind of the external version of whatever Cadence is for Uber or used to be.

嘉宾：是的。我觉得那些人看到了，你知道，这类东西确实有实际需求，于是他们就能去构建它，并把它的规模扩展到 Uber 之外。但我想表达的模式是，当年为了满足我们当时的规模，有很多东西都得由我们自己发明。随着时间推移，行业也逐渐成熟，如今我们会打造一些产品；在某些领域，如果我们觉得有独特需求，就会选择购买。如果我们打造了什么东西，并且觉得它对行业里的其他人也有用，那么只要条件允许，我们就会尝试开源。Guest: Yeah. And I think those folks like they saw like you know there there's a real need for something like this and they were able to go and like you know build uh like scale it even further beyond Uber. uh but I think that the pattern that I was I'm trying to say is like there's a whole bunch of things that we had to like invent in those days for the scale that we had and over the years then we are like now as the industry also mature like now we have few few products that we build so some where we we think there's some unique need we buy and if we build anything if we think it's interesting for rest of the industry then we try to open source wherever possible

主持人：对，我觉得还有两个工具值得一提，我刚想起来。一个是 Morpheus，基本上是专门为 Uber 直接推出的，但它的集成范围要广得多，对吧？你可以进去运行实验，马上就能看到结果。它几乎就像是特性开关和实验平台的结合，而且集成到了移动端特性开关里。我记得在 Uber，所有这些集成都融入了各种工具，对吧？Host: yeah I think two more tools that I think is worth mentioning cuz I just remember them is one is Morpheus or it's basically launched directly for Uber but it was way more integrated into everywhere right like you could you could go in there and run experiments you could see the results immediately it was almost like feature flagging and experimentation but integrated into mobile feature flags and it was like I remember inside Uber like all of these integration it was just integrated into a bunch of tools all of them right

嘉宾：是的，而且因为我也……因为我们也在扩展到很多国家，所以你需要一个非常好的配置平台，无论是用来配置我们的后端服务，还是搭建功能开关系统、开展实验。Uber 是一家非常数据驱动的公司，几乎所有事情都要经过功能开关，也都要经过 XP。Guest: yeah and and since I also since we were also expanding to so many countries like you need a really good configuration platform platform whether it's for like configuring our backend services or also having feature flag system and also experimentation and Uber is a very datadriven um company and like everything goes through a feature flag everything goes through an XP

主机：实验Host: experiment

嘉宾：嗯，所以，另外，由于我们也是双边市场和三边市场，有些实验也不只是简单的 A/B 测试，我们还会做切换实验。我们必须构建非常深入的实验体系，并将其融入所有工具中，让它能够自动生成分析结果，比如 P 值是多少、应该使用哪些指标。这也促使我们在指标基础和数据基础方面投入了很多工作，以确保基于这些指标生成的分析结果值得信赖，这样我们才能据此做出业务决策。Guest: um so then and and since we have also it's not and since we are two-sided marketplace and three-sided marketplace some of the experiments are also not like really complex it's not just simple AB we also have switchbacks and like we have to build like really deep experimentation that's baked into are like all of our tools and it can automatically generate analysis and like what are the p values what metrics to use and that also led to like uh working a lot on our metrics foundations data foundations so that whatever analysis that we generate from these metrics are trustworthy and we can make business decisions based on that.

主持人：是啊。我记得我一直在纳闷，我们为什么要有自己的功能开关系统。后来我就在想，嗯，给我看看哪个供应商提供的功能开关方案支持城市。事实上，城市是一等公民，因为在 Uber 内部，我觉得城市非常重要，我们的很多工具都把城市作为一个维度。比如在 cardality 方面，每个城市都有自己的指标。然后你就会意识到，哦，几乎没有什么东西会把这些因素纳入考虑。Uber 有很多很特别的约束，对吧？在这个规模上，别的公司都没有遇到过这些约束。Host: Yeah. I remember I was always wondering why do we have our own feature flag system. But then I was thinking like hm like show me a feature flagging solution from a vendor that supports cities. In fact, cities are first class citizens because inside Uber like I I think cities were so important and like a lot of our tools had cities as for example for cardality like every city had its own metrics and and then you start to realize like oh very few things treat like Uber had a bunch of interesting constraints right like that no one else had at this scale.

嘉宾：是的，完全同意。是的，是的。我觉得按城市划分这个方案很有意思。它已经深深融入了我们的基础设施。比如，我们的一些关键平台服务也会按城市进行分片，这样我们还可以观察不同级别城市的情况。因为我们需要确保，即使是现在，在我们承载流量的所有区域中，每个城市都有一个指定的区域来承载其流量；我们也可以随时将该城市的流量故障切换到其他区域。另外，我们还有一种逻辑部署组：一组城市会被路由到一组容器。这只适用于少数服务，并非所有服务；我们会在需要这种分片级别和可靠性级别的服务中采用这种方式。Guest: Yeah absolutely. Yeah. Yeah. I think the the city one is an interesting one. It's like so deeply baked into like our infrastructure like for example some of our like critical platform services we also do sharding by city by city because then we can also like observe um across different tiers of cities um because like we we need to make sure like even now like um across all of the regions where we serve our traffic we shard a city to a specific region we can also at any point in time fail over that city to across regions and even and we also have like a you know logical deployment groups where a a group of cities are routed to a group of containers and this is for few services we don't do it for all the services we do it for some of the services that are where we need that level of uh that level of sharding and uh that level of reliability

主持人：我还想问问你另一个系统，我记得自己用过。当时它还在开发中，我记得大概是2017年，那就是 Death Pods。当时它感觉真的走在了行业前面。它是怎么发展起来的，后来又变成了什么？Host: one other system I wanted to ask you about that I remember using and and it was being developed in 2017 I believe which is death pods and it felt really ahead of the industry at the time. How did that come along and what did it turn into?

DevPodsDevPods

嘉宾：是的。嗯，我觉得 DevPod 的背景是，呃，因为我们之前已经谈过微服务的发展。最初我们用的是微型仓库，每个服务都有自己的仓库，那时候还没有标准的构建系统。每个服务都有自己的构建和测试系统——我们至少有两三套构建系统、两三套测试系统。甚至对中央团队来说，要升级库时，也很难逐一检查所有服务并进行升级。所以大概在 2017、2018 年，我们改变了策略，转向使用单体仓库，尽管这些服务仍然是微服务，但所有服务都从单体仓库部署。我们按语言各建了一个单体仓库。所以我们有 Go 的单体仓库、Java 的单体仓库，还有移动端 iOS、Android 的单体仓库，等等。后来，随着仓库规模变得足够大，运行构建变得相当困难；而且要在本地笔记本电脑上把所有这些系统都配置好也很麻烦，还得处理各种各样的配置问题。Guest: Yeah. Um, so I think the backstory for dev pod is uh u because we already talked about the growth of microservices. Initially we had micro repos where every service was its own repository and we didn't have a standard build system at that point. Each service had their own like we had at least two three build systems, two three testing systems and it was very hard for even for a central team when we wanted to do library upgrades go across all the services and do upgrades. So around 2017 18 um we we we then changed a strategy to getting to monor repos even though they're microservices but all of the services were deployed from the monor repos. We had one lang one monor repo per language. So we had a monor repo for go mono for Java, one for mobile iOS, one for Android and so on. Then once the scale of that repository was pretty big enough then like it was pretty hard to run the build and like getting all of those systems set up on your local laptop also then you have to like go through all kinds of setup issues.

我们开始采用一种叫 dev pod 的东西，它类似于云端开发环境，你可以拥有任意数量的 dev pod。我们为每个 monorepo 配备一个 dev pod。比如，如果我主要在 Java monorepo 上工作，就会创建一个 Java 版的 dev pod，其中包含所有构建系统缓存。这样我就能更快地构建。它运行在云端机器上，性能甚至比笔记本电脑还强。这样一来，我们就能为构建时间和测试时间设定很好的 SLA。我们也不用担心环境配置问题，因为可以预先安装好合适的工具。 有意思的是，进入 AI 时代后，我们发现智能体也需要类似的工作区和云端机器。于是我们就顺势利用了这一点，还创建了一个包含所有代码仓库的 mega dev pod，这样我的智能体就能跨多个代码仓库创建 PR。Um so we took on this thing of like dev pod which is like cloud uh developer boxes and you can have any number of them. uh we had one dev pod per monor repo. So if I'm working primarily in Java monor repo, I create the Java version of the flavor of the dev pod and it has all the build system cache. Um I can have faster builds. It's working on cloud machines as a beefier than even the laptops. So then you could have like we could have really good SLAs on your build time on your test time. Um and we don't have to worry about the setup issues because we can uh up we can install the right tools in place. And now the interesting part is as we got to the AI era like even the agents need the similar workspaces and cloud boxes and we are able to kind of latch on to that and we also created like now a mega dev pod with all of the repositories because then I can have my agents create PRs across the repositories.

它具备在任何代码仓库中工作的所有工具，可以查看任意仓库的上下文。这也帮助我们在仓库顶层文件夹创建了 AGENTS.md 和 CLAUDE.md，因为我们的构建和测试都已经标准化。设置好顶层文件后，大多数团队马上就能从中获得很大价值，不必再针对每个模块单独配置。嗯，而且在开发 Pod 中，所有 AI 工具默认都已安装。因此你可以创建一个新的开发 Pod，不必再专门安装 Claude Code、Codex 或 OpenCode，它们在开发 Pod 里都已经有了。这让公司里的非工程师也能很轻松地创建开发 Pod，马上开始试用 Claude Code 和 Codex。It has all the tools it has to work in any repository to look at context across any repository. This also helped us to create the agents MD cloud MD at the top repo at the top folder because we have standardized build standardized testing. Once we set up the top level files, most teams will get like really good value out of the bat. They don't have to like really go into per module setup. Um and uh like on the dev pod all of the AI tools are installed by default. So you can get a new dev pod. You don't have to go through even installing like cloud code, codex, open code, they're all there in the dev pod. So this made it really easy even for non-engineers uh in the company to create a dev board and start playing with cloud code, start playing with codecs out of the bat.

主持人：是啊。所以我想，尽早使用云开发环境确实帮了很大的忙。Host: Yeah. So I I guess like having cloud development environments just like really early helped a lot.

嘉宾：是的，是的。我觉得，嗯，这些年来我们在平台化上的投入也帮了很大的忙，嗯，因为随着我们不断推进智能体化进程，我们可以利用所有已经构建起来、能让工程师工作更轻松的东西，而这些东西也同样能让我们的智能体受益。Guest: Yeah. Yeah. Like I think uh all of those years of platformization bets also helped a lot uh as we expanded on our agentic journey because you could use all the things that we built making the life easier for engineers and it makes it easy for our agents as well.

主持人：是啊。在我们开始聊今天的内容之前，我想先稍微谈一谈，因为我觉得很多事情都是相互关联的，对吧？比如，过去有很多东西是为平台、为工具开发的，而且通常是为 Uber 定制的。不过这方面我们已经聊得够多了，那么今天的情况是什么样的？如今的开发是什么样的？你们会用什么工具？也许你可以给我们稍微讲讲……Host: Yeah. And I wanted to talk a little bit about be before we we we got into what's on today cuz I think a lot of things are connected right. So like there was so many things being built for platforms for for tools of often custom for Uber but h having talked enough about that what does it look like today? What does development look like today? What kind of tools do you use and may maybe you could give us a little bit of like a

为 AI 代理构建功能Building for AI agents

先介绍一下你们专门为 AI 打下的基础，以及你们构建了什么、为什么这么做。首先，尤其是在 AI 领域，我们很早就意识到，整个格局变化很快，工具链变化很快，模型也变化很快。我们希望投入构建一套合适的基础模块，使其不依赖于特定的工具链或模型。这样，无论这些东西如何变化，我们都能轻松迁移到不同的模型和工具链上。我们投入建设的几项内容中，有一项是模型网关。早在 ChatGPT 首次推出时，也就是 2013 年末、2014 年初，我们就开始着手开发模型网关了。当时，我们就有一个模型网关，可以提供兼容 OpenAI 的 API，或者兼容 Anthropic 的 API。primer of like the foundations that you put in place specifically for AI and you know what you built and why. First thing is especially in the AI world, one thing we realized um very early on was because the landscape is changing pretty quickly, harnesses are changing pretty quickly, models are changing pretty quickly. We wanted to invest in the right set of building blocks that are hardness and model agnostic so that no matter any of these things change, we can easily port across different models, different harnesses. So few things that we invested like so we started working on our model gateway from the initial introduction of charge GBT like late 2013 early 2014. So we had a model gateway back then which would expose you know OpenAI compatible API or anthropic compatible.

嘉宾：非常抱歉，呃，所以这是2023、24年，对吧？是的。抱歉。2023、2020、20年初，2024年，Chat GBD 刚推出、GBD 模型也刚推出的时候，我们很快就意识到，我们希望让公司里任何想利用这些模型构建智能体的团队都能轻松做到这一点。因此，我们搭建了一个模型网关，将所有复杂性封装起来，并在其中设置了所有安全和策略防护措施，也做好了可观测性。这样，你就可以在不同模型或不同模型供应商之间切换，只需要使用组织或项目的 ID，就能调用任何已接入的模型。Guest: So sorry so like this is 2023 24 right? Yeah. Sorry. 2023 2020 20 early 2024 when chat GBD was first launched and GBD models were first launched and we quickly noticed you know we we wanted to make it easy for any teams in the company who want to like uh leverage and build agents with these models. And so we had a model gateway that would abstract all of the complexity behind where we had all of the our uh security and policy guards in place. We had all of the observability in place. So you could switch across different models or different vendors for these models and you could just have your the the ID for the organization or the project and then you can you can use any of the models exposed.

主持人：所以，所以你就是把它暴露成一个类似模型选择器的东西，然后接入模型网关，由它把请求路由到合适的模型——呃，你知道，比如当时的 ChatGPT，或者当时的 Opus 之类的模型。Host: So so this was you just exposed it into like a model selector and it piped into the model gateway and it would it routed to the right uh you know may that be chatgpt at the time or opus at the time that kind of those models.

嘉宾：是的。嗯，而且它本身就是基于 Michelangelo 构建的。所以我们创建了一个 Michelangelo 生成式 AI。Guest: Yeah. Like um and and it was built on Michelangelo itself. So we create a Michelangelo genai.

主持人：哦，米开朗基罗。不过那个，那个，那个是 ML 平台，对吧？Host: Oh, Michelangelo. But that that's that's ML uh platform, right?

嘉宾：是的。然后现在 Michelangelo 平台上有一套生成式 AI 基础设施，所以你可以在 Michelangelo 上创建一个生成式 AI 项目或 AI 项目。接着你就可以设置项目，选择你想用的模型，然后你会获得一个唯一的项目 UID，之后可以用于你的智能体。这样一来，我们就能利用同一套 Michelangelo 基础设施，在相同的项目、相同的基础之上，开始构建面向 AI 的评估等内容。回过头来看，我觉得那是一项相当早期的投入，后来带来了很高的投资回报。Guest: Yeah. And then now there's a Gen AI set of foundations on Michelangelo platform like so then you go to create a Gen AI project or ai project on Michelangelo. Then you can set up the project. You can select uh the model that you want. You can you can uh then you'll get a unique project UID that then you can use for your agents. So then you have all of the same like we're able to use the same Michelangelo infrastructure then to start building on AI specific evals and like stuff on the same pro on the same foundations. Looking back I think that was pretty uh like um like pretty early investment that like paved that that paid that gave a lot of ROI later.

主持人：什么是模型网关？我可以快速介绍一下我们当时构建的东西。那是在 2023 年末，第一批大语言模型发布的时候。我们很快意识到，很多内部团队都想把它用于各种不同的场景，所以我们想让他们用起来非常方便。这个东西是构建在 Michelangelo 平台之上的，也是他们可以在平台里设置的另一个组件。简单回顾一下，Michelangelo 是一个机器学习平台，大约从 2015、2016 年就开始构建了。所以它早在 LMS 出现之前就已经建成很久了。Host: What is model gateway? I can show like a quick primer into like you know uh what we had built and this was uh late 2023 when uh the first set of uh uh LLMs were launched and we quickly realized that we had whole bunch of internal teams uh wanting to use it for different use cases and we wanted to make it really easy for them and this was built on top of the Michelangelo platform and one more component inside that they can set up there and Michelangelo just to recap that's the ML platform that was built from like 20 2015 2016. So it it was built a long time before before LMS.

嘉宾：没错。现在我们也在 Michelangelo 中加入了一套生成式 AI 或智能体 AI 组件。我们目前也在构建更多功能，其中一些还没有包含在开源版本中。我们正在积极为智能体 AI 领域开发很多东西。我的模型网关是最早推出的这类产品。最初版本没有这些中间件和组件；过去一两年里，我们陆续构建了其中一些。不过，公司里任何团队的使用体验都很简单：他们创建一个客户端，提供在 Michelangelo 中初始化的项目 ID，再提供调用方令牌，就可以了。我们提供一个与 OpenAI 兼容或与 Anthropic 兼容的 API，支持所有供应商的模型，无论是开放权重还是闭源权重模型。Guest: Exactly. And now we have like a genai or aentic AI set of components also inside Michelangelo. And we're also building a lot more uh right now that uh that we have that those are not in the open source version yet. Uh but we're actively building a lot of stuff for agent AI world. uh but my model gateway was the first such uh one and the initial version we didn't have all of these middlewarees and components and we had built some of these over the last one two years but what the experience for any team in the company is you know they they create a client they give their project ID which they initialized in Michelangelo and uh then the caller token and that's all and we expose one open compatible API or anthropic compatible API so and all of the models and across all the vendors uh open weight or closed weight.

它们全都在这个网关后面，呃，后面；而且在……上面，我们在那里添加的第一项功能是，呃，我们称之为 AI guard 的东西。They're all behind uh behind this gateway and over the and we the first thing we added there was uh what we call AI guard.

主持人：什么是 AI guard？Host: What is AI guard?

嘉宾：基本上，你知道，我们想确保不会把任何敏感内容发送回用户。这是我们可以添加中间件的一种方式，相当于一个统一的入口，让我们能够在所有项目中一致地执行这些规则。安全团队也因此能够不断迭代各种 AI 防护专用模型，无论是内容方面、安全策略方面，还是一系列通信策略方面；他们在那里构建了五种不同的微调模型。这些模型会在每个请求和每个响应上运行，同时受到严格的延迟限制。这在很大程度上帮助我们确保所有项目采用相同的基准标准。由于每个项目都会在这里进行跟踪，我们还可以按用户、按调用方进行归因。随着今年后续工作的推进，我们还能够为项目和用户设置不同的支出层级。Guest: Essentially like you know we wanted to make sure we are not um any sending any sensitive content back to the user. So this was one way where one uh one entry point where we could add this middleware that we can enforce consistently across all the projects. um and security team had ability to like you know then they could iterate on all of the AI guard specific models whether it's on content whether it's on safety policy whether it's on like whole bunch of coms policy where like five different set of fine-tuned models that they built uh there which would which runs on every request and every response uh with strict latency guardrails this helped a lot to make sure that all of our projects have same uh standard baselines and then because every project is tracked here we could have like per user, per caller attribution, we could have like then as as we progressed later this year, we were able to add spend tiers for projects, spend tiers for users.

嗯，由于所有请求都会经过同一个地方，我们还可以在这里添加审计日志、会话追踪，以及一整套相关的基础设施。Uber 的每个模型请求都会经过这里。嗯，也就是说，没有任何模型请求会绕过这里。这样一来，我们就为所有模型提供了一个统一的入口。我想我们也可以在这里处理计费，比如成本之类的。Um, and since all the requests go through one place, we could also add like audit logging, session traces and a whole bunch of infrastructure around this. Every model request at Uber goes through this. Um, so there's no model request that doesn't go through this. So that gives us like one single entry point for across all the models. And I'm assuming like you can just try billing as well like costs and all of those things.

主持人：就像你可以跟踪计费、缓存命中率以及各种指标一样。现在，随着我们不断深入，大家讨论了很多模型路由之类的话题。我觉得这可以成为一个很好的切入点，我们可以借此发挥，因为所有模型都会经过这里。我认为，我们也在朝着让工程师能够轻松切换不同模型的方向迈进。接下来要解决的是，如何让工程师能够轻松切换不同的执行框架？Host: Exactly like you can track billing cash it rate and all kinds of things. And now you know as we progress into like there's a lot of discussion of like you know model routing and all of that. I think this could be like a good entry point that we can leverage because all of our models go through this. I think on the same journey of like being able to make it easy for engineers to switch across models. The next thing was how do we make it easy for engineers to switch across hardnesses?

比如，如果我在用 Claude Code，后来切换到 Codex，就需要把所有设置、技能、工具和插件都带过去。我们在内部用一个名为 AIX AI Foundations 的封装层来封装所有这些开发环境，它预装在 Dev Pod 上，也可以安装在本地机器上。我们还有一个“角色”的概念，每种角色都附带一整套默认插件、代理和技能，这样你甚至不需要自己安装任何东西。每位工程师都能开箱即用地获得一整套默认技能。由于我们封装了所有内容，所有设置都由我们统一管理。因此，无论你是在本地还是在 Dev Pod 上使用这些开发环境，设置都会同步。这样，我们就能集中管理一套设置，并将其推送给所有用户；每位用户自己的设置也会在其所有环境之间同步。like if I'm using cloud code then if I switch to codeex then I need to like carry forward all of my settings all the skills all the tools uh all the plugins um so we internally wrap all of the harnesses with a wrapper called AIX AI foundations and this is pre-installed on dev ports or you can install on your local machine and uh we have a concept of personas so then that personas comes with whole bunch of default plugins begins and skills so that you don't even have to install like any engineer they get whole bunch of default skills that they can use out of the bat since uh we wrap everything all of the settings are managed so even if you're using uh any of the hardnesses on your local or on dev pod and or any of your dev pod the settings are synced so then centrally we can manage uh one set of settings that we can propagate across all the users and any user specific settings they are synced across all of their environments.

再说一次，这让事情变得非常简单，因为现在我们有一个统一入口，可以切换不同的运行框架和工具。我们还添加了一条状态行，可以在其中提供提示，并跟踪你在不同运行框架上的支出，以及该运行框架中当前会话的支出。所有这些额外功能，我们都可以在无需工程师进行任何明确设置的情况下添加。And again, this makes it really easy because now we have one entry point for switching the harnesses, switching the tools. And then we also made it like there's a status line where we can give tips where we can have tracking of your spend across harnesses within that harness in that session. So all of those additional things that we could we could add without any explicit setup from the engineers.

嘉宾：不错。说到跟踪支出，你们现在是怎么做的？比如说，作为 Uber 的工程师，如果我刚加入，成为某个团队的高级工程师之类的，我需要了解支出和成本吗？你们是怎么考虑这件事的？Guest: Nice. And then speaking of of tracking spend, how do you approach that right now? like as an engineer at Uber, if I just join as a you know like a senior engineer somewhere like do I need to be aware of spend of cost what's the thinking there

主持人：你看，我们想确保工程师能够使用所有可用的工具，而且我们有一个支出等级的概念。默认情况下，你会获得某个等级，然后我们还有很多个等级。当你快达到当前等级上限时，就可以申请升到下一个等级，而且只需要经理简单审批一下。这样你就可以和经理讨论这个项目的投资回报率，比如你打算如何支出；这其实就是围绕如何负责任地使用这些资源进行一次交流，对吧？但我们不想限制工程师。即使他们正在做某个项目，只要这些支出确实用于正确的项目，他们就应该可以按需支出，因为最终我们想要最大化从中获得的收益，同时也要确保资源得到负责任的使用。Host: see we want to make sure engineers have access to all like any of the uh tools that are available and we have this notion of spend tiers. So by default you get certain uh certain tier and then then we have many levels of tiers. Once you once you're close to that tier then um then you can raise a request to go to the to the next tier and uh you you just need a simple manager approval. So then you can discuss with manager the ROI for that project like how are you spending and it is just a conversation around responsible use of of this right? But we don't want to restrict engineers like even if they like if if they're working on like some uh if this is going to the right project use they should be able to spend uh whatever they need because ultimately we want to maxim we want to maximize the gain from this but you also want to make sure there's responsible use.

嗯，我们尽量减少跨越不同层级所需的文书工作。用户可以通过我们的任何工具查看自己在某个执行框架内以及所有执行框架上的支出。我们还为不同的托管代理设置了不同的层级。比如，对于所有交互式层级和交互式执行框架，我们都有一个支出层级；对于一些后台编码代理，则有单独、独立的层级。这样他们就能轻松管理所有这些内容。Uh so we try to do the minim minimal amount of like you know paperwork needed to like cross the tiers and they will see like the the the across any of our tools how much they're spending within that hardness and across all the hardnesses and we also have different tiers for different managed agents. Um so like like then for all of the interactive tiers interactive harnesses we have some spent tier for some of our background coding agents we have a separate independent tier. So we make sure like they can easily manage across all of these.

嘉宾：是的。不过我想，即使我做自己的项目，这也说得通。我会留意自己的花费，以及我设定的任何限额。我想，我理解得对吗？意思就是别把 token 浪费在那些蠢事上，或者那些毫无意义的事情上；也别一不小心就把 token 烧掉了，自己却不知道。所以就是要给出这样的反馈：这些东西不是免费的。Guest: Yeah. But but I guess it just makes sense even if I do my side projects. I will be aware of my own spend on my whatever limit I I have. And I guess I guess do I sense it correctly that the thinking is just don't use tokens on stupid stuff that is that makes no sense or or accidentally you know you might be just burning and you don't know. So like just just give that feedback like the these things are not free.

主持人：没错。我觉得，光是有这样的反馈就很有帮助。我们还有一个叫 Cost Check 的插件，它会分析你所有的会话，并根据你的使用情况提供个性化建议，比如告诉你：“你在这个项目和这次会话上花了这么多钱；以下是一些建议，可以帮助你进一步优化成本。”我觉得这一切都能促进管理者和工程师之间的良好沟通，也能让工程师清楚地了解自己的开销是否高效。我认为，这对我们帮助很大，比如在实时指导、上下文健康状况、模型规模选择等方面，也能让工程师意识到在这个新时代该如何工作。我们也不希望出现低效的支出。Host: Exactly. like I think just having that feedback and we also have like uh uh a plug-in called cost check that will run across all of your sessions and give you personalized tips based on your usage to say hey like this is the kind of thing this much you're spending on this project and this session um this is these are the tips that you can use to further uh improve your cost profile and I think all of this creates um a good conversation like for between man managers and engineers but also engineers will also see like good feedback into like how they are efficiently spending across I think and then that helped us a lot with like you know like all of the real-time coaching with context health model sizing um and uh like so that engineers are also aware of working in this new age like we also don't want like you know like inefficient spend

嘉宾：你们还建立了哪些其他基础？Guest: and what other foundations did you build

MCP 网关MCP gateway

与 AI MCP 让位有关related to AI MCP gave way

主持人：哦，对，我想另一个背景大概是在 2025 年初。嗯，我们一开始是随着 Cursor 逐渐兴起而开始的——我们启用了 Cursor，而 MCP 当时还是一项刚创建出来的规范。我们很快意识到，嘿，只要以一种简单的方式开放所有内部工具，就能大大增强 AI 的能力。这样一来，我们就有可能把它应用到编程之外的场景中，也可以用于各种软件工程任务。于是，2025 年 4 月左右，我们开始开发 MCP 网关——类似于模型网关，这是一个面向所有工具的统一网关。嗯，我们已经为所有服务建立了不错的 IDL 注册表，不管它们使用的是 Thrift 还是 Protocol Buffers（proto），因为我们有 IDL 规范，所以可以为这些服务自动生成 MCP 规范。Host: oh yeah I think the the other um u backstory was around like I think early 25. Um we started like as cursor kind of picked up like we enabled cursor and MCP was um was an was a spec that was like created around that time and we quickly realized that hey the power of we can really increase the uh power of AI by exposing all of our internal tools in an easy way acrossing. So then then we could potentially leverage this beyond just coding. we could leverage it for all kinds of software engineering tasks. So around April of 2025, we started working on like MCP gateway like you know similar to model gateway this is one gateway for all of the tools. Um so what we did was because we already had good ideal registry across all of our services uh whether they're using thrift or proto since we have the ideal specs we autogenerate the MCP specs for any of these services

嘉宾：另外补充一点，我想，对于不了解的人来说，像 Uber 这样的服务会输出 Thrift 或 Protocol Buffers，作为强约束契约，对吧？Guest: and just to add I guess for those who don't know like Uber services either emit thrift or product buffers as strong contracts right

主持人：是的，最初我们只用 Thrift，后来也增加了对 Protocol Buffers 的支持。嗯，过去几年里新增的服务全都用 Protocol Buffers。几年前创建的那些服务则全都用 Thrift。嗯，随着这些服务重写，团队通常会切换到 Proto。不过，无论它们用的是 Thrift 还是 Proto，由于我们把所有 IDL 都放在同一个注册中心里，因此可以利用这个注册中心为它们生成默认的 MCP 规范。这样，公司里的任何人都可以访问这个 MCP 网关工具，查看我们自动生成的所有 MCP 服务器；进入其中任意一个，就能看到根据现有 API 生成的所有工具。这些工具可能是从那些服务的 API 映射而来的 MCP 工具，而且我们会自动生成……Host: yeah and like we initially we are just thrift and later we added support for protobuff as well uh But all of the new services uh in the last few years they're all protobuff. Um the ones that were created like few years back they're all in thrift. Um and as they go through rewrites usually like team switch to proto but whether they're in thrift or proto like we we would have we since we have all of the ideals in the in one single registry we could use that registry to generate a default MCP spec for them. So they could anyone in the company they can go to this tool MCP gateway they can see all of the um MCP servers that we autogenerated and if you go to any of that you could see all of the tools that were generated from that existing API. So these might be the APIs from those services that are projected as MCP tools and we autogenerate

嘉宾：我们会自动生成请求包和响应包，之后再生成描述。我们还会根据 API 描述，为 API 和字段生成适合 LLM 的优化描述。比如，我们看到有些工具的描述长达 3,000 到 4,000 个 token，因此可以对其进行优化，方便模型使用。这只需要改一个配置和安全策略，然后就可以启用，不需要做任何代码更改。如果他们想要自定义 MCP，也可以使用我们的框架来创建，然后通过 MCP gateway 代理他们的自定义 MCP。我们做的另一件有意思的事是，所有供应商的工具——无论是 Google、Slack 还是其他工具——也都通过 MCP gateway 暴露出来。这大大减轻了配置管理的负担。我们之前提过 AFX，对吧？这样一来，无论是内部 MCP 工具还是外部 MCP 工具，我都可以用同样的方式安装。Guest: we autogenerate uh the request pack response pack the descriptions later we also based on the API descriptions we generate LLM optimized descriptions for the APIs and fields so that we can like we don't like there are we saw some tools with like 3,000 4,000 tokens so then we can optimize it for uh model consumption um and it it is just one single config uh and and security policy change uh and then they can enable it. They don't have to do any code changes. If they want any custom MCP, they can always lose we have like a framework for creating a custom MCP and and proxy their custom MCP also through MCP gateway. The other uh interesting thing we did was even all of our vendor uh tools whether it's Google, Slack, all of them are also exposed through MCP gateway. This reduced a lot of you know burden on configuration management since we already spoke about AFX right like so I could install any of the tools in a same manner whether it's internal MCP tools or external MCP tools.

过去几个月里，我们也把所有这些 MCP 都提供成了 CLI。这样，你就可以把任何 MCP 当作 CLI 来执行。如果某个智能体框架想把它当作 CLI 使用，就可以这么做；如果想使用传统的 MCP，也可以使用 MCP。In the last few months we also projected all of these MCPs as CLI. So you could execute any MCP as a CLI. So then if some agentic hardness wants to use it as CLI they can use it as CLI. If they want to use traditional MCP, they can use MCP.

主持人：那我理解得对吗？这一切之所以能实现，是因为一开始这些 API 就已经生成好了，或者说已经写好了——其中很多都是生成的。URI 已经有了一个 API 平台。所以加入 MCP 某种程度上，我是说，挺轻松的。当然它确实成功了，但你懂我的意思吧？现在生成 CLI 也相对轻松，因为在 AI 出现之前，你们就已经打下了基础，对吧？Host: And and do I understand that this all comes because the APIs to start with were already either generated or or written in like a lot of them were generated. The URI had a platform for the APIs. So adding MCP was somewhat I mean trivial. I of course it's worked but you know what I mean. And now generating CLI is also somewhat trivial because like you you had a foundation before AI, right?

嘉宾：是的。而且，然后，当你有了它……我开始在 Uber 身上看到的是，你们在 2022 年之前搭建了很多东西，之后就可以相当轻松地在上面添加 AIM 基础设施。Guest: Yeah. And and then so when you have it I that's what I'm starting to see with Uber that like you built a bunch of stuff before 2022 and you could just kind of add AIM infra pretty easily on it.

主持人：是的。而且，负责我们 API 网关的同一个团队也负责 MCP 网关，所以他们本来就在做 API 网关。他们懂得如何处理各种协议转换，也懂得如何构建各种 API 网关。因此，我们很快就把它搭建起来了。我觉得，尤其是在我们把 Google Workspace 和所有 Slack 工具也接入之后，这就催生了各种令人惊叹的智能体工作流。那时，人们几乎可以自动化任何事情，比如在 Slack、Google、Jira、Linear 之间操作；任何工具供应商提供的工具或任何内部服务，都只需一次 CLI 调用即可访问。我们开始思考这个领域时，奠定基础的另一组要素是——我认为这些都属于一类与供应商和框架无关的构建模块。无论接下来会发生什么，我们都得做这些，因为智能体总得有办法访问你的工具。</|final_end|Host: Yeah. And and the same team that works on our API gateway also worked on MCP gateway like so they're already in the business of building API gateways. So like they they they know how to do pro any kind of protocol transformations any kind of uh any kind of API gateways. So it was like we were able to pretty quickly build this. Um, so I think this was probably like uh especially once we got our Google Workspace and all of our Slack tools also into this, it kind of like created all kinds of like amazing agent workflows. People could automate anything right at that point like from Slack to Google, Jira, Linear like any any tool vendor tool or any internal service uh is we are at a CLI call away. And then the other set of foundations as we like um started thinking about the space was see I think all of these go in this camp of like vendor neutral harness neutral building blocks that we think like you know as like you would have to do this irrespective of what happens because you would need agents need some way to access your tools.

然后另一件事是关于技能方面的，你知道，我们……Then the other thing was around skills like you know we

Agent Skills 仓库Agent Skills repo

今年早些时候，我们就采用了这种做法。既然我们已经有 monorepo 这个概念，就创建了一个技能 monorepo。从某种意义上说，它包含中央平台团队构建的所有核心技能，以及各个团队可以自行构建的团队专属技能。jumped onto this uh earlier this year since we already have this notion of monor repos we created a skill monor repo in some sense that has all of the core skills that central platform teams build and all of the team specific skills individual teams can build

嘉宾：这些技能就是类似技能文件夹的东西吗？就像你可以用于 cloud codecs 等的插件。没错，就像这样。然后，插件和技能可以在所有 harness 中通用。接着，我们有一套可以在所有这些环境中运行的统一质量门禁，并且让它可以跨所有 harness 移植。所以，你可以使用我提到的同一套 AIFX 工具来安装；如果你执行 plugin add 添加那个技能，它就会更新所有 harness 的配置，无论你当时是否正在使用那个 harness。嗯。Guest: and are these skills these are just like the the skills folders like the plugins that you can use for for cloud codecs etc. Exactly like so then is plugins and skills that you can use across all the harnesses. Then we had like one single uh like quality gates that can run across all of them and we made it portable across all of the hardnesses. So you install with the same AIFX tooling that I mentioned and like if you do plug-in add that that skill it will update the configurations across all of the harnesses whether you are at that point using that harness or not. Mhm.

主持人：这样一来，即使你切换不同的运行框架，这些插件也会自动安装好。所以你不用担心重新安装、重新配置它们之类的事情。而且，由于所有技能都在使用 MCP 网关提供的工具，我们也有一套统一的机制，让所有这些技能都能访问任意工具。现在我们正在开发一些工具，让工程师可以选择自己的技能。工具会告诉他们各种提升技能的建议。接下来正在推进的一项工作是：如何收集各个运行框架的执行轨迹，并生成改进补丁来优化技能，这样我们就能根据经验不断改进技能。因为大家围绕技能质量、技能评测以及如何持续改进技能进行了很多讨论——刚构建出来的技能可能并不完美，因此你需要某种机制来持续改进它。我认为，MCP 和技能都给我们带来了很多好处。比如，过去几周里，每天大概有 3 万次技能执行。Host: So that even if you switch across harnesses, you have these plugins auto installed. So you don't have to worry about like reinstalling them and reconfiguring them and so on. And since all of the skills are using the tools from the MCP gateway, we also have one standard mechanism for all these skills to access any of the tools. Now we are working on like you know some tools where engineers can go select their skill. It will tell them all kinds of tips to improve their skills. The next thing that in in flight that you're working is how do we then collect the traces from the hardnesses and generate uh improvement patches for the skills so that we can continuously improve the skills based on learnings because there's a like there's lot of discussion around the skill quality skill evals and how do we improve that continuously like when when you first build a skill they might not be perfect and you need some mechanism to continuously improve that I think both the MCP and skills this this gave us like lot of uh like um benefits and like if I look at like in the last few weeks there's around probably 30,000 skill executions per day.

嘉宾：这对我来说挺有意思的，因为如果你跟别人说，比如“哦，我们在搭建一个技能仓库”，创业公司里大多数人会怎么想，对吧？我不是说 Uber，而是说“哦，你们就是弄个 Git 仓库，里面放一堆技能，也就是一堆技能文件夹”，差不多就这样。但很明显，事情要复杂得多，对吧？复杂是因为你们需要支持的工程师人数多吗？还是因为你们也在跑评测，因为你们一直在清理这些技能，或者追踪它们有没有被使用？诸如此类。Guest: This is kind of interesting for me to see cuz if if you told someone like oh we're building a skills repo like what most people would think inside a startup, right? I'm not talking about Uber, but like oh you just do a get just a repo and you have a bunch of scales there just a bunch of scale folders and and that's kind of it. But it's pretty clear that it's it's more complex, right? Is is it more complex because of how many people engineers you need to support? Is it because you're also running evals because you you keep cleansing these skills or like tracking if they're used or not used? Those kind of things.

主持人：是的，当然。我觉得，嗯，你看，现在有了这些最新模型中的任何一个，创建一个技能的粗糙版本都非常容易。但要让它在各种使用场景下都真正好用，就很难了。所以你确实需要确保，嗯，我们有一个统一的地方可以更新所有这些技能。事实上，当我们为所有 MCP 暴露 CLI 接口时，只有几位工程师借助 AI，就把这个仓库里的所有技能都更新成了 CLI 模式，因为这样更节省 token。如果每个技能都写着“我用 MCP ABC、MCP DEF”，那么我安装这两个插件后，这两者的六个 MCP 就会一直留在我的上下文中，造成很大的上下文负担，对吧？后来我们就把所有技能都替换成了 CLI 模式，这对我们的 token 效率也产生了很大影响。还有一点是，我们会为所有人自动安装这个技能市场，就这样。Host: Yeah, absolutely. I think um see it's like now in the now with any of these latest models, it's very easy to create like a vero version of the skill. It's very hard to make it really good across all kinds of use cases and that's why you want to really make sure like you know we have one one uh place we can like update all of these skills in fact when we ch exposed the CLI projection for all of our MCPS just couple of engineers using AI they were they updated all of the skills in this repository to use the CLI pattern because it's more token efficient because if every skill says I MCP ABC MCP DEF then if I install these two plugins I'm going to have like six MCPS across these two always in my context and there's so much context load right and like then we replaced all the skills with the CLI pattern so that that had like lot of impact on our like token efficiency as well and the other thing is because uh we auto install this skill marketplace um for everyone done.

它又减少了一步配置，所以这样一来就很容易发现。设置、配置和管理都很简单，我们还可以围绕这种可发现性构建一大批评估流水线和发现性流水线。我们仍然有一些团队拥有自己的代码仓库，但大多数技能都会通过这里。我最喜欢的是，呃，像 context graph 这样的东西。嗯It reduces one more step of configuration and so then like this the discoverability is easy. The setup and configuration and management is easy and we could build whole bunch of like eval pipelines around that discoverability pipelines around that. We still have teams that have like their own repositories but majority of the skills go through this. My favorite is uh like context graph. Um

嘉宾：什么是上下文图？事情的背景是Guest: what is context graph? The backstory is

上下文图谱与 CortanaThe context graph and Cortana

早在 12 月的时候，有位工程师开发了一个 AI 助手，你可以去找它——我们叫它 Cortana。你可以在网页上或者 Slack 里向它提问，询问任何代码库相关的问题。它会像智能体一样，在我们所有的代码仓库中进行搜索，然后回答问题。接着我们看到大家问各种各样的问题，比如做研究、询问怎么运行某个查询、某个数据源是什么、谁负责这个项目，或者能不能对此进行根因分析（RCA）——一大堆软件工程相关的任务。我们在查看其中一些执行轨迹时，很明显能看出，我们可以优化模型获取上下文所需的轮数，因为我们有这么多系统。过去，这些系统全都是彼此隔离的，对吧？工程师得去任务管理平台，还得去值班仪表板，也得去我们刚才提到的自有仪表板。back in December like one of the engineers built uh a AI assistant that you can go to like we call it Cortana like you can go to web or you can go to slack and ask any question from any codebase. So it would do agent it would like search across all of our repositories and answer questions and then we and then we saw people asking all kinds of questions like it could be research it could be like what is how to run this query like where what is the data source for this who owns this like can you do RCA for this whole bunch of software engineering tasks and when we're looking at some of these traces like it was clearly obvious that you know we could optimize the number of turns it takes for the models to get grounding because we have so many systems Like historically all of these are like siloed systems right like engineer has to go into like the the task management platform engineer has to go to our like uh on call dashboard has to go to like our own dashboard that we talked about

主持人：然后我们就想，好吧，怎么把所有这些整合到智能体的一张思维导图里？就在过去几个月里，他们搭建了一个很酷的可视化图谱，他们称之为 Uber Context Graph。我们有 100 多种节点类型、130 多种边类型。这些都是实时数据，比如说，数据截至大概 3 天前。你点击任意一个人物节点，就能知道他们管理过哪些资产、写过哪些设计文档，以及……Host: then we're like okay how do we plug all of that into one mental map for the agents then just in the last few months and like this is a cool visualization has built like they call it uber context graph like we have uh 100 plus node types like 130 plus edge types and this This is live data as of like say like like 3 days ago. If you click on any person node like you know know what assets they managed, what design docs they wrote, what

嘉宾：他们正在开发的功能、参与过的值班事件、配置变更、处理过的事故、Jira 事件；而且，你看任何一个节点，它都与所有其他节点存在关联。这让我们能够做到：现在，我之前提到的同一个助手，默认就以这个上下文图作为依据。因此，如果你提出任何问题，它会先利用这个图来导航，弄清楚这个人问的是什么。一旦确定了问题的背景——比如它涉及哪个领域、哪组服务——它就可以深入下去，阅读其他文档和资料。这样可以减少你的代理执行轨迹，也减少它给出好答案所需的轮数。我们已经看到，在很多类别的问题中，答案质量有所提升，成本有所下降，延迟也有所降低。而我们在充分发挥这项技术的全部潜力方面仍处于早期阶段。对于未来能够在此基础上构建的一切，我们都感到无比兴奋。Guest: features they're working on, what on call events they like worked on, what are the configuration changes, what incidents they managed, what Jira events and if you take any node, it has relationships to all other nodes. And what this allows us is um now the same assistant I talked about it has the context graph as a default grounding. So if you ask any question it will first use this to like navigate okay like what is this person asking about. Once you get grounded into oh like this is about this domain this set of services then it can go deep and like read other documents, read other things. So it it reduces your execute your uh your agent traces and the amount of turns it needs to give you good answer. And we already see like in a whole bunch of categories of questions the quality goes up, the cost goes down, the latency goes down and we have just in we still in early in harnessing the full potential of this and like we are super excited about like all all of the things that we can build on top of uh this going forward.

主持人：那么，这些节点是人还是工具？Host: And then the nodes are they people or are they tools?

嘉宾：这是架构图。公司里的每个人都会有一个节点，比如我的节点，以及每位员工各自的节点；然后这些节点会关联到所有实例。比如我创建的每个 PR 都会有一个对应的节点，并通过链接标明“Udai 创建了这个 PR”“Udai 管理这个资产”“我参与了这个产品功能”“我为这份设计文档做出了贡献”。Guest: This is the schema visualization. So every person in the company will have like know like there's a node for me node for like every employee and then it will have relationships to um all of the instances like every PR that I generate there will be a node for that and there's a link saying okay udai created this PR uh Udai manages this asset um um I work on this uh product feature I contributed to this design doc

主持人：所以，节点既是人，也是资产。对，没错。举个简单的例子，假设你问一个问题，比如全球有多少出行行程采用现金支付方式，你首先需要了解有哪些表、有哪些列、还有没有其他人写过这些查询，他们来自科学团队还是出行科学团队？现在所有这些数据都是互相关联的。你可以去查查，过去 30 天里有没有人做过相关查询，都有哪些表。再去探索这些列，因为这些内容都是相互关联的。这样你就能很快找到我们认可的查询，并迅速回答这类甚至很复杂的问题。否则，你就得翻很多很多文档，调用很多很多工具。Host: so so then the nodes are are both people but they're also assets Yeah, exactly. Like one simple example is uh let's say if you ask any question like how many um mobility trips around the world are with cash payment type like you first need to understand what tables exist, what columns exist, are there other people who wrote these queries, are they from science team, are they from mobility science? Like now all of this data is interconnected. Like you can go and check oh are there other queries that were done in the last 30 days around this and what kind of tables exist. Go and explore those columns because again all of this is uh interlin then you can quickly get to like the blessed queries that we have and um answer these kind of even complex questions pretty quickly. Um otherwise you would have to like go through many many dogs, many many like tool calls.

嗯，这样效率确实高多了。要我们手动编写所有这些 Cypher 查询可难多了，但智能体很擅长在图中导航，就像它们很擅长用 Bash 在文件系统中导航一样。Um, this makes it really efficient. Like it's much harder for us to manually write all of these cipher queries, but agents are really good at navigating the graph like how they're really good at navigating file system with bash.

嘉宾：这在我看来相当未来感十足，不过也挺酷的。我猜光是存储这些数据就得占用海量空间，毕竟 Uber 轻轻松松就有一万多名员工。所以……Guest: This seems very futuristic to me, but it's pretty cool and I assume it's a ton of data to store just because Uber has like 10,000 plus people working there easily. So

主持人：是的，好的一点是，你知道，和我们的 API 边缘网关团队参与 MCP 网关项目类似，负责管理图谱、也就是拥有数十亿节点的生产图谱的图谱平台团队，也帮忙构建了这个图谱。和我们的生产图谱相比，这个图谱小得多。所以对他们来说，这已经不再是规模上的挑战了。对，对。我们还谈到了 AI 助手 Cortana，它接入了部分甚至所有的 Context Craft 工具和技能，并且在 Slack、Web 和 CLI 中无处不在。公司里的任何人都可以使用它，比如身处某个国家的运营人员，他们可以询问任何问题，比如“这是怎么构建的？”“它位于哪里？”“谁负责这个？”“背后有什么历史？”它能够利用所有这些数据，也就是我们所有的 Google 文档，然后给出非常好的答案。这样就打破了各个组织之间的信息孤岛，对吧？大家可以自助使用，想问什么都行，也可以执行任何人构建的技能。Host: yeah, the good part is uh you know, similar to how our uh our API edge gateway team worked on MCP gateway effort, our graph platform team that manages graph or our production graph with billions of nodes, they also helped build uh this graph. So this is in compared to our production graph, this is much smaller. So this is like so uh it's not like a scale challenge uh for for them anymore. Yeah. Yeah. And we also tal I already talked about like the the AI assistant Cortana which um which kind of plugs uh uh some of all of the context craft tools skills uh and it's omniresent in slack web CLI. So anyone in the company like even someone in ops in some country they can ask any question about oh how is this built where is this located who who works on this what was the history behind this and like it is able to take all of this data all of our Google documents and then uh and then give them like a really good answer and it breaks down kind of the information silos across orgs right like they're able to like self-s serve ask anything execute any skill anyone has built

嘉宾：我想它必须能够访问数据源，至少要让合适的人能够访问。这样它就能查看日志之类的内容，还能进行这类监控。Guest: and I guess it must have like access to the data sources to for for at least for the right right people. So it can go into like things like logs and and yeah monitoring those kind of things.

主持人：是的，它有日志监控。你可以问，比如：“这个服务的健康状况怎么样？”它可以检查所有依赖项、使用的存储系统类型，运行所有查询，然后为你提供一份关于该资产的 360 度健康报告，而且我们会沿用所有现有的用户权限。所以，也就是说，我只能访问我原本有权限访问的内容。否则，那些数据就无法访问。Host: Yeah, it has logs monitoring. You can ask okay how what is the health of uh this service? It can go look at all of its dependencies, what kind of storage systems it uses, uh run all the queries and give you like a 360deree health report for that asset and we carry forward all of the existing user permissions. So like so then if I'll only able to I'm only able to access the stuff that I had access to. uh otherwise will that data is not accessible.

嘉宾：是的，是的。我记得在 Uber 内部，这种机制以前叫作“破窗”。如果你要为自己无权访问的系统添加权限，就得走审批流程，审批会交给经理、总监之类的人。但我知道，在我任职的时候，Uber 在这方面已经做得很成熟了。Guest: Yeah. Yeah. Because inside Uber I remember it used to be called break glass. When you added a permission for a system you didn't have access to you go through an approval flow which goes to either a manager or director or whatnot but I know Uber was was really big at at my time they were really big on this already.

主持人：是的，当然。我觉得这些算是一些基础。我们还在筹划更多内容，比如一些更具未来感的用例。不过，这些基础模块确实帮助我们推进了智能体化进程，也让我们把它从单纯用于编码，拓展到各种工程任务。Host: Yeah absolutely. So I think those were like some of the foundations like we have some more in in uh in in the workings like we think about some like more even more futuristic use cases but you know some of these building blocks really helped us in our like agentic journey and helping it take it from just pure um using it for just coding to all kinds of engineering tasks.

嘉宾：我是不是只要提一下，Uber 已经完成了 AI 基础工作，但接下来还有更多基础工作要做？我们的冠名赞助商 Turboper 对此深有体会。他们正在从第一性原理出发，重建存储架构，目的是让更多类型的存储在大规模应用时更快、更便宜、更可靠。名为 Turboper V3 的新系统尚未投入生产环境，但已经通过了 CI。不过，它的性能还远不及当前的生产版本 v2。因此，TUF 团队现在正全力优化查询计划，直到它们达到或超过 v2 的速度。他们公开记录这项工作，分享如何逐步提升性能，直至 V3 正式上线。例如，我们可以看到，团队通过批量读取，将全文搜索性能提升了 9 倍。目前，他们做的很多都是基础工作，但接下来几周，他们会分享一些非常酷的系统工程进展。Guest: Would I just mention how Uber wrapped up AI foundations work but already more foundations work is to come. Our presenting sponsor Turboper knows this all too well. They are rebuilding their storage architecture from first principles. They're doing this to make more types of storage faster, cheaper, and more reliable at scale. The new system called Turboper V3 is not yet in prod, but it already passes CI. Still, it's not nearly as performant as the current production version v2. And so, the TUF team is now perf maxing query plans until they are as fast or faster than v2. They're doing this in public documenting how they increased performance all the way up to the V3 production launch. For example, we can see how the team increased full text search performance 9x batching reads. A lot of this is basic foundation work right now, but they will be sharing some very cool systems work over the next few weeks.

不过等等，团队为什么要彻底重写？这是因为搜索正在发生变化，Turbuffer 也一样。TF 最初是一个向量数据库，但现在已经不再是向量数据库了。它已经发展成一个高性能搜索数据库。V3 弃用了向量主索引，所有索引都将共用同一套列式存储系统。每个查询都会由向量化查询引擎执行，这在搜索数据库中并不常见，但能带来极快的搜索速度，也让数据库整体运行得非常快。Turboper 团队正在记录他们的开发历程，你可以在 turboper.com/v3 跟进。看着它一点点构建起来，真是太有意思了。我还想提一下本季的赞助商 Linear，它是一个协调人类和智能体协作的平台。对我们软件工程师来说，最大的变化之一就是需要审查的代码量实在太大了。But hang on, why is the team doing a full rewrite? It's because search is changing and so is Turbuffer. TF started as a vector database, but it is no longer vector DB. It has evolved into a high performance database for search. The V3 discards vector primary index and all indexes will share the same colummer storage system. Every query will run on vectorized query engines which is something you don't usually see in search databases but which will result in both very fast search and just generally a very fast database. The Turboper team is keeping a worklog of their journey and you can follow along at turboper.com/v3. It's a blast to see how it's being built. I also want to mention our season sponsor linear, the place to coordinate work between humans as well as agents. One of the biggest changes for us software engineers is the sheer amount of code that we need to review.

Uber 开发了定制工具来处理这个问题，Linear 也一样。它叫 Linear Diffs，能让工程师直接在 Linear 里审查 PR。我特别喜欢 Linear Diffs 的一点是，差异内容会显示在项目中的某个 issue 旁边。因此，你可以轻松地在一个地方查看所有上下文，了解这项改动为何发生，而不必在不同工具之间来回切换。对于大型 PR，Guided Reviews 会通过说明改动内容，带你逐步了解核心变更。它还会将胶水代码和辅助编辑分别归类。这些细小却重要的改进，让你能一次性读懂大型 PR。Linear 也支持堆叠式 PR。如果你用过堆叠式 PR，就会知道它们是拆分复杂工作的好方法。如果你还没用过，试试看，很可能会喜欢上它们。Uber has built custom tools to deal with this, but so has linear. It's called linear diffs, and it lets engineers review PRs right inside of linear. One thing I really like about linear diffs is how a diff sits next to an issue inside a project. And so it's super simple to see all of the context in one place and know why the change has happened without needing to jump through different tools. For large PRs, guided reviews walk you through the core change with an explanation of what has changed. Guided reviews also group the glue code and the supporting edits separately. These small but important changes make a large PR readable in one sitting. And linear also supports stack PRs. If you use stack PRs, you'll know that they're a great way to break up complex work. And if you've not yet used them, you'll probably love them when you try them out.

随着 AI 智能体承担起更多工作，我们进行代码审查的方式也在不断变化。Linear 带来的这些巧妙改进，有助于我们工程师更深入地理解代码审查，也让我们期待 Linear 能不断改进 diff，以适应我们越来越多地与 AI 智能体协作的工作方式。想了解更多并领取 Linear 的 6 个月免费使用期，请访问 linear.app/pragmatic。 说到这里，我们回到 Uber，看看如今在这家公司，开发人员会使用哪些工具来构建功能。我想，PRD 和 ERD 就是其中的一些。How we do code reviews keeps changing as AI agents take on more of the work. These neat changes coming from Linear help us engineers understand code reviews more and expect Linear to evolve diffs to keep up with how we work more with AI agents. To learn more and to get 6 months off of Linear, head to linear.app/pragmatic. With this, let's get back to Uber and how building a feature looks like today in terms of tools used by devs at the company. PRD and erd I guess for those

PRD 和 ERD 工具PRD and ERD tool

对于不了解产品需求文档（PRD）和 ERD（工程需求文档）的人来说，它们就像是产品规格说明和工程计划，对吧？这是在 Uber 内部使用的术语。嗯，我想我们已经介绍了一些 AI 基础；现在，如果你想想一个功能的生命周期，我们怎样才能借助恰当的 AI 基础，真正提升生命周期中的每个环节，同时重新思考适用于 AI 时代的工具？有些方面我们已经走得比较远，有些还处于早期讨论阶段。我觉得，甚至像是让智能体和工程师能够共同迭代协作的一体化计划这个概念——我们有很多很酷的想法。不过，嗯，我们还有各种智能体技能，可以处理你们的各类设计文档。就像我之前说的，无论是产品设计文档、工程设计文档、上线评审、XP 评审，还是生产就绪评审，所有这些内容都集中在一个地方。who don't know product requirements documents and erd was engineering requirements documents right like it's like the product spec and engineering plan for those not familiar inside Uber yeah so um I think we went through some of the AI foundations and like now if you think about the life cycle of a feature then how can we truly uplevel each of those life cycle with the right set of AI foundations and also rethinking those tools for the AI era and Some we are more farther ahead, some we are early in the discussion. Like I think even the notion of a combined plan that agents and engineers can iteratively work to work together we like um there's like lot of cool ideas but this is kind of like you know um we also have like all kinds of agentic skills for working with any of your design documents and as I saying before like whether it's a product design doc engineering design doc launch reviews XP reviews production readiness reviews all of those things are in one place.

我在筛选一些非常旧的项目，这样我们就能查看它们；而且你可以准确知道谁在负责这些项目、它们的状态如何。这些数据也都会进入图谱，这样我们就能掌握完整历史：哪些是权威设计文档、审批状态如何、谁在负责它们。每个项目也都有对应的 Jira 或 Linear 链接，因此我们可以从项目追溯其执行过程是如何落地的。这就为人类和智能体提供了一个权威信息系统，让他们了解每个项目的规范。I'm like filtering some really old projects uh so that we can like uh look through them but uh like you'll know exactly like um who is working on them uh track the status of that um and all of this data also goes into the graph as well so that we know like full history of what are the source of truth design docs uh approval status who works on them and each of these also have like the Jira link or the linear link so then we can trace from from this project how was the execution translated to this gives like a source of truth system for both humans and agents to know the specs for every project.

主持人：是的。我想我想，规划仍然非常重要，尤其是对于那种并非只是很简单的、你知道的，一个提示词或一次性完成的项目。Host: Yeah. And I I guess I guess planning is still really important especially for like something that is not just a trivial you know like one prompt or oneshot project.

嘉宾：是的，没错。而且因为产品种类繁多，国家和地区之间也存在差异，我们真的需要多考虑各种边界情况，而不只是正常流程。你必须考虑各种边界情况，以及该如何应对；而且由于我们也在实体世界中运营，边界情况就更多了，对吧？你真的得想想，如果发生这种情况会怎样，如果发生那种情况又会怎样，我们该如何处理这类支持请求，我们该如何处理……Guest: Yeah. Exactly. And because we need to like there's so many with with the kind of product heterogenity and like country hetrogenity like we need to like really think about all kinds of corner cases more than like you know the the happy path. You have to think about all kind of corner cases and like how they should exist and since we also operate in physical world the number of corner cases are like even more right you to really think about oh what happens if this goes on what happens if this goes on how do we handle a support case for this how do we handle

主持人：是的，我想这点其实不用多说，但我还是补充一下，因为它可能没那么显而易见：Uber 在很多国家和城市运营，即使是两个城市之间，实际情况也可能不同，或者人们的预期不同、习惯的事情也不同，等等。是的，完全正确。我想这就是为什么一开始就制定合适的计划非常重要；这也能让工程师或智能体更容易拿着计划进行拆解并执行。比如说，有了计划之后，接下来要做的就是如何把它转化为具体任务、进行拆解并执行。现在我们有点……Host: yeah and I I guess just need needless to add but I'll just add it cuz it might not be as obvious but Uber operates in so many countries and cities and even between two cities the reality might be different or might be different expectations or people are used to different things and so on. Yeah, absolutely. I think and that's why it really becomes important to think about the right plan up front and this will also uh make it easy for both the engineers or agents to take that plan and decompose and execute on that. Like once we have the plan then the next thing is how do you convert that into decompose that and execute. Now we kind

爪牙Minion

进入我们内部称为 Minion 的系统。这是我们的后台编码代理。目前，大约 15% 的生产 PR 都会经过 Minion。现在有很多 PR：有些工程师希望交互式地完成，但我们也希望让这件事变得非常简单——如果某项任务的规范已经很完善，或者我只想尽量少地进行引导，就可以把任务交给 Minion。我们又进行了多轮迭代，这是最新一版，我们称之为 Minion v2。你可以在网页上使用它，也可以在 Slack 中使用；可以先进入计划模式，再让它在后台执行；也可以只输入一次提示词，让它在后台执行，适用于简单的提示词。它可以使用 Mega Dev Pod，因此能够访问我们所有的代码仓库和代码库，在多个仓库之间协作，并根据你的计划和规范，在各个仓库中生成一个或多个 PR。of going into uh what we call uh minion internally. Um so this is our background coding agent. Um so around 15% of all of the production PRs go through uh go through minion uh right now essentially um you know there's whole bunch of PRs that some PRs engineers want to do interactively but we also want to make it really easy once you have for the task that I have really good spec or I want to do minimal steering I can hand off that to uh to a minion and we had again we had we did multiple iterations this is the latest iteration we call it minion v2 but like you can use it from web you can use it from slack you can uh do a plan mode and then let it execute behind the scenes or you give one you can give the prompt once and let it execute behind the scenes if it's like a simple simple prompt and it has access to this is using the mega dev pod so then it has access to all of the repositories all of the codebase that we have it can work across repositories and uh and generate one PR or multiple PRs across repositories ries based on the plan and spec that you have.

最初开始时，主要是把维护类的任务交给 minion。随着我们在这段旅程中不断前进，并在 minion 环境中加入越来越多的工具，让 minion 能够在 Web、移动端或后端 API 中自行验证它的工作，我们的目标是随着时间推移，增加人们可以用 minion 完成的任务类型……嗯，逐步提高任务的占比。我想我们也可以看看，比如……它是如何……它…… uWhen we initially started mostly maintenance kind of tasks used to be offloaded to minion as we progress on this journey as we are adding more and more tools into the minion environment where minion can self validate its work in in web or mobile or or backend APIs. Our goal is we can we can increase the kind of task people can do with minion over time. uh increase the percentage of task uh over time. I think we can also like see like uh how it's how it u

嘉宾：它集成在 Slack 里。你只要提及……你是提及 Min，还是提及 Minion，还是怎么说？Guest: it's integrated in Slack. You can just mention do you mention min at minion or what do you say

主持人：你可以 @minion，然后它会显示状态。你也可以不必打开网页，直接在 Slack 里指挥它。比如，你可能和 EM、PM 有一个 Slack 讨论串；如果事情比较简单，我们可以在里面迭代设计、达成共识，然后说：“嘿，@minion，跟进这个讨论串，能不能帮我创建一个 PR？”它就会获取 Slack 讨论串的上下文，然后帮你创建 PR。所以，即使是在我们的支持频道里，它也非常有用。我会看到有人报告 bug，值班人员就问 minion：“嘿，你能看看问题是什么吗？”然后 minion 会回复：“哦，根本原因是这个。哦，这看起来挺简单的，我们直接在这里修掉吧。”于是我们就直接修复了，不用再把它加进待办列表，留到下一轮处理；可以当场就把它修好。随着 minion 变得越来越强大，我们看到这种情况也越来越多。我想我们刚才大致讲完了构建阶段，之前也聊过 DevPod。它后来改进了，对吧？Host: you can do at minion and then uh it will show you the status uh and you can also like you don't need need to go to the web you can steer it on slack itself um maybe you have like a slack thread with the with the em with the PM you can iterate on the design if it's a simple thing we are just iterating and get alignment there and then say hey at minion follow this thread and can you can you create a PR uh uh and then it will take the Slack thread context and then create a PR for you. So it's been really useful even in our support channels. I would see like someone reports a bug, the on call ask minion, hey like what can you see what is the issue and then minion will come back with oh this is the root cause oh this looks simple enough let's just fix it right there and then we would like fix it like there's no then loop of like adding it to backlog doing in the next next thing like you were able to fix it right there so we see like more and more cases of that as minion gets powerful as well so I think we like we we kind of uh went through like the build step we also before talked about devpot It's improved since, hasn't it?

嘉宾：是的，是的。所以，呃，这里显示的是我所有的开发端口。嗯，因为我在做 context graph，你看，你看到了。呃，不过，呃，就像你，如果你想创建一个新的开发 Pod，你会看到可以选择哪种规格。呃，对于 minion，有一种开发 Pod 规格包含所有代码仓库。呃，不过，嗯，呃，它针对 minion 的使用进行了优化。但对于交互式使用，它会创建一个开发板，短短几秒钟内，嗯，你就会有一个正在运行的环境，然后你就可以……Guest: Yeah. Yeah. So, uh, like this is showing all of my dev ports. Um, since I'm working on context graph, like you see you see that. Uh, but, uh, yeah, as you, if you if you want to create a new dev pod, you'll see like which flavor you want. Uh, for for minion, there is a dev pod flavor that has all of the repositories. Uh, but um, uh, and optimized for minion usage. But for interactive like it will create a dev board and within a matter of seconds um you will um you will you will have that running and you can

主持人：几秒钟就做到这点，真厉害。我相信你们会做一些巧妙的事情，比如让东西保持温热，或者提前构建好，这样就可以复制这类东西。Host: that's impressive to seconds. I'm sure you do some clever stuff like having things warm or or or pre-built so you can copy those kind of things.

嘉宾：对，就像，嗯，这一切之所以能运行，是因为这是开发者相关的东西。我们为此使用了好几个独立的云环境，里面有 Kubernetes 集群，而且所有东西都已经预热好了。所以你提出请求时，我们就能直接从预构建镜像克隆一份，里面已经有构建缓存、所有仓库，所有东西都已经克隆好了，随时可以开始构建。呃，因为我们所有仓库的构建系统都用 Bazel，所以我当时就能进入一个仓库，像是……Guest: Yeah, it's like um it's it's running on all of this is because this is developer stuff like we have like a separate multiple clouds that we are using for this and it's like Kubernetes cluster and like everything is warmed uh warmed up. So then when you ask it like we are able to just clone from the pre-built image and it has build caches everything repositories everything already cloned like it's ready for you to build uh so I'm since we use basil across all of our repositories for build system so I'm I was able to go into a repository like

使用 Bazel、Slate 和 Capri 进行测试与代码审查Testing and code review with Bazel, Slate, and Capri

为此运行 basil，然后就像我们刚才说的，我们按这个流程来过一遍：整个过程只要几秒钟。任何工程师想在这里改代码，都能在几秒钟内创建一个 dev pod。这是构建任何模块的统一方式，大家都知道这个命令。现在 agents 也知道同样的方式，对吧？在 cloud.md 和 agents.md 里，它们知道如何在任何代码库中构建和测试任何模块，也能使用完全相同的命令。对，所以我觉得这大致涵盖了代码生成要做的事情：你可以通过交互式 harness 来做，也可以通过 minion 来做。接下来，好，你有了 PR 之后该怎么做？然后，嗯……也许我们先讲讲自愈 CI。我们意识到的一点是，我们的 PR 数量逐年增加了。run basil for that and then just like we went we went through this in the order of this like you know seconds where any engineer like if they if I want to make a code change here like within order of seconds I create dev pod it's one single way to build any module and then anyone knows this command and now agents also knows the same way right in the cloud MD agents MD they know how to build test any module in any repository and they're able to use the exact same commands yeah so I think that covers kind of like you know what the kind of stuff that you have to do in code generation so you can do interactive hardness you can do like through minion the next thing is okay like you have the PR what do do you do after that? Then um um maybe we go through self-healing CI first. One thing that we realized is because our PR volume has gone up year over year.

主持人：哦，是啊。Host: Oh yeah.

客人：它肯定爆炸了。Guest: It must have exploded.

主持人：是的，是的。我们的 P50 周期时间和 P90 周期时间，都是针对内循环而言的。比如说，你已经有了 PR，想让它通过检查并创建远程 PR；或者你已经创建了 PR，接下来要获得人工评审，并让所有 CI 检查都通过。我认为评审会成为一个很大的瓶颈，嗯，和其他公司一样。我们也看到了逐年变化的差异，而且我们真的非常专注于拆解其中的每个环节，深入研究如何在 AI 时代对其进行优化。我们首先做的是，把一大批过去在创建 PR 后才会运行的简单 CI 任务移到了内循环，并把所有这些操作都放在一个命令后面：`FX validate`。这样，智能体创建 PR 后，就可以运行 linter、覆盖率规则、构建规则、模板规则等所有检查，从而提高 PR 处于良好状态的概率。Host: Yeah. Yeah. our P50 cycle time, P90 cycle time both for inner loop where like you know you are get you you have the PR and you want to get it to a green state and create the remote PR or once you have your PR created getting the human review getting all the CI things to pass I think reviews become a big bottleneck uh same as like any other company um and we see year over year the delta and we want to like really we're like really focused on like breaking down every aspect of is and like how can we go really deep and optimizing it for the AI era. So first thing that we did was we took a whole bunch of like simple CI jobs that used to run once you create the PR and we moved it to inner loop and we kept all of this behind one single command like a FX validate. So then when agent creates a PR it can run the llinter the coverage rules the build rules the template rules like all of these things so that the probability of this PR in a is in a good condition like goes up.

然后我们接入了视觉验证功能，这样我就可以启动模拟器并进行测试。Then we hooked on ability to do visual validation where I can spin up emulator and test it.

嘉宾：我可以快速搭建一个 Web 环境来测试。我也可以快速搭建一个预发布环境，在里面部署这项改动、运行 API 并进行测试。现在，Minion，甚至我的交互式测试框架，都知道这些智能体具备完成这类工作的技能。这样它们就能走完这个验证循环。假设你创建了 PR，但仍然有某些东西失败了，那么我们就有自愈式 CI，可以应用到所有 PR，也可以按 PR 单独应用。这是 Uber 专用的一行内容，我们会添加到所有 GitHub PR 中，你可以把它关联到自己正在处理的问题。然后你就能看到哪些检查没通过。你可以设置一大堆自动化功能，比如自动请求审查、自动合并、自动修复。基本上，我们能做的一切，都是为了缩短 P50、P90 周期时间。嗯，完成这一切之后，我们就有了一个叫作代码收件箱的东西。Guest: I can spin up like a web and test it. I can spin up like a staging environment where I can deploy this change run the APIs and test it. And now minion or even my interactive harnesses they know these agents know how to like have the skills for this. So then they can go through this uh verification loop and let's say you create the PR and still something uh fails um then we we have like a self-healing CI you can apply to all the PRs or per PR. So we this is like a Uber specific line that we add to all GitHub PRs that you can link to the issue that you're working on. and you can see which checks are failing. You can have whole bunch of automations, auto request review, auto merge, auto fix. Essentially everything that we can we can do to reduce the P50 uh P90 cycle times. Um and once you do all of this then we have this thing called code inbox.

这里展示的是一个模拟版本，嗯，任何工程师都可以进入这个页面，查看所有等待他们处理的 PR，以及他们创建的、正在等待评审的所有 PR。所以，如果有工程师把某个 PR 发起评审，我就会进入这个页面，看看我们目前为止做的所有事情。这样我就能很方便地看到：哦，AI 代码评审通过了。这些是它检查过的内容。这是健全性测试的截图。这样我就知道了，嗯，我明白了。This is showing a mock version of that uh where um it will any engineer can go they can see all the PRs that are waiting for them. all the PRs that they created that they're waiting for reviews. So if uh engineer send this for review, I go to this page, I I can see all the things that we did so far. So it makes it really easy for me to get, oh, the AI code review passed. These are the things it checked. This is the sanity test screenshot. So I know, okay, I know.

主持人：你可以快速看一下。你也有视频。Host: You can just quickly take a look. You have a video as well.

嘉宾：是的，我能看到构建检查之类的内容。我们还在添加风险评分、这个 PR 的影响范围等等。所以这些都会给工程师提供非常好的信号，帮助他们判断这个 PR 是否适合合并，以及应该重点关注 PR 的哪些部分。是的，我觉得我们仍在积极推进这一大堆想法，但我们也想确保，随着 PR 数量涌入，我们已经做好准备。实际上，由于 CI 任务的数量同比翻了一倍还多，我们不得不对 CI 系统进行内部代码审查。因此，我们还得加强这个领域的一系列基础设施，以便为新 PR 的涌入做好准备。Guest: Yeah, I can see the the build checks and all of that. And uh we're also adding risk score, blast radius of this PR and so on. So all of this will give engineers really good signals to figure out okay like is it good to land and which parts of the PR I should focus on and yeah I think we still uh like we are aggressively like working on this whole bunch of ideas that we have but we want to make sure like you know as we are ready for the influx of PRs we in fact had to like had internal code read for our CI system because year over year the number of CI jobs also doubled or more than doubled. So we have to like also like beef up whole bunch of foundations in this space so that we can inc we can be ready for the uh influx of new PRs.

主持人：嗯。还有，我想你们也发现了，工程师不可能像过去那样，以同样严格的标准审查所有 PR。所以我想，你们正在开发一些工具，用来提示哪些 PR 需要多加关注，哪些可能风险较低——比如影响较小、波及范围有限等等。你怎么看代码审查未来的发展？你觉得人工代码审查会一直存在吗？还是说，有些情况下它可能会消失？Host: Yeah. And and I guess like you're also seeing that there's no way engineers can review all PRs with the same rigor as we did in the past. So I guess you're you're building tools to signal when which ones to pay more attention to which ones might be low risk in so many ways maybe in impact maybe in blast radius. What is your take on where what will happen with code review over time? Do you think there will always be human code review or for some things it might go away?

嘉宾：是的，当然。我觉得首先，对我们来说，我们总是希望采取更稳妥的做法，因为对 Uber 而言，一次事故的代价非常高。因此，即使我们现在有了这些工具，仍然要求工程师审核每个提交的 PR。随着我们对风险评分越来越有信心，也能用多年的数据进行回测——我们拥有所有这些数据——而且随着我们对 AI 代码审查越来越有信心，也许可以对某些低风险类别的 PR 开放我们所说的“绿色通道”，也就是自动合并。但在此之前，我们希望对所有需要考虑的方面都非常、非常有把握。目前的首要任务是：如何让工程师能够轻松获取所有必要的信号，从而更快地完成审查。Guest: Yeah, absolutely. So see I think uh one is for us like we we want to air always on the safer side because the cost of an incident for Uber is like really high. Um so even with all of this tooling we like right now we still require an engineer to review every PR that that goes that goes in once the more confidence we get in our risk scoring and we can also back test it with years of data right uh we have all of this data and the more confidence we have in our AI code reviews then maybe for some class of lowrisk uh PRs we could enable uh we call it green lane like auto landing. Uh but we want to be really really sure about all of the dimensions that we want before we go there. But right now the main priority is how do we make it really easy for engineers to get all the signals they need to review faster

主持人：如果我们在这方面做得好，我相信，随着时间推移，我们可以提高自动化和 PR 的占比。不过，这算是 P1；P0 则是先把所有信号都放到正确的位置上。Host: and if if we do a good job there then I'm sure like you know we can increase the percentage of like auto and PRs over time. Uh but that's kind of P1 P 0 is like first getting all the signals in right place.

嘉宾：不错。是啊。不过代码收件箱挺好的，我喜欢。我们应该提一下这个事，比如，呃，我该怎么做健全性测试。Guest: Nice. Yeah. But code inbox is nice. I like it. This is like thing that we should touch upon is like maybe uh how I can do sanity testing

主持人：还有，健全性测试——给不了解内部术语的人解释一下，这就是手动测试，对吧？Host: and and sanity testing just for those who don't know the internal lingo that's the manual testing right

嘉宾：对，手动测试。对，我想做的是，比如说，我正在开发某个新功能，它涉及两三个服务，那我怎么知道这个功能端到端是否正常？所以，工程师的典型工作流程，在智能体出现之前，是我们称之为 slate 的东西。基本上，你可以创建自己的实例，部署你正在开发的分支。Guest: yeah manual testing yeah I want to do let's say I'm I'm working on some new feature it touches two two or three services how do I know if this feature is working end to end so then typical workflow for engineers is um like pre-agentic world is we call it slate uh essentially you And it can you can create your own uh uh instance that deploys the branch that you're working on

主持人：你可以将测试账户的 UID 路由到你创建的实例。Host: and you can route your test account UID uh to the instance that you created.

访客：所以，当你登录时，你执行的就是那个分支。Guest: So when you log in you are executing that branch.

主持人：没错。退出那个分支后，其余上游和下游的流量都会转到其他预发布环境。这样一来，你就可以进行端到端测试，而不必担心基础设施的其他部分，而且这种设置会沿着整个服务图继续生效。比如，你可以指定从移动应用发起测试，或者说“这是我想用于测试的账号”，请求就会经过其他服务；当它到达正在测试的服务时，就会转到你的 Slate 实例。如果你同时测试两三个服务，请求就会转到这两三个正在测试的服务。因此，你甚至可以用这种方式进行多服务测试。有了这套机制之后，我们只需要教会代理如何部署 Slate，并把所有过往的 API 都提供给它们，这样它们就可以调用这些服务。Host: Exactly. Exit that branch and rest of the upstream and downstream goes to the other uh staging setups. So that way you you can test end to end without uh without worrying about other parts of the infrastructure and this will carry forward across the service graph. So you can you can say from your mobile app or this is the test account I want to test it will go through other services and when it touches the service under test then it will go to your slate instance. If you're testing two or three then it will go to the service under test across all two or three. So you can do even multi-ervice tests uh with this. Now that we had this, we we had to just teach agents how to deploy Slate and we also give it all of the past APIs that um that you can use any of these services.

嘉宾：基本上，这件事是这样的：作为一名工程师，我会做合理性测试，也就是说，我想手动测试自己的改动。我有一种很酷的方法可以做到这一点，而现在我们可以把这件事交给智能体，让它们以安全的方式端到端地测试自己的改动。Guest: Basically this was a thing where I as an engineer well into sanity test meaning I want to manual test out my changes. I had a way to do this in a pretty cool way and now we can just give it to agents and they can actually test their own changes end to end in a safe way.

主持人：是的。所以我们这里展示的是它的一个网页版，你们实际上可以看到，嗯，这就是我们刚才讲过的 CLI 操作模式。它使用 slate MCP 创建一个新的 slate 环境，然后提示说，哦，这是环境名称；接着，一旦环境创建完成，它就会提示将服务部署到这个环境中，并填写服务名称。这是一个测试服务。嗯，现在它有了自己的环境。这就是 slate UI，你们也知道，如果看这里，可以将测试账户的 UID 固定到那个特定实例上。你们还会看到该环境中的所有服务。可以看到 git sha，而现在，智能体在内部可以使用所有这些技能和工具。嗯，所以，如果你们看它做了什么，它对那个网页进行了 A/B 测试。Host: Yeah. So here we are showing a web version of that and you can see in fact like um the this is like the CLI pattern we went through. It's using slate MCP create new slate environment and it's saying oh this is this is the environment name and then u once it creates the environment it's saying deploy service to this environment and giving the service name. This is a test service. Um, and now it had like it it has its own environment. This is like the slate UI and it has like you know and if you see here you can pin uh test account UIDs um to that particular instance. Uh and you'll see all the services that are in that particular environment. You can see the git sha and uh now like the the agent internally can like use all of these skills and tools. Um so it it if you see like what it did was it did AB testing of that web thing.

它在那个 Slate 环境中运行了 agent browser，比较 main 上的内容和我分支上的内容，然后生成了一份报告，供我查看所有变更。嗯，之后我们可以查看它生成的报告。比如，它生成了一份展示变更前后对比的报告，并显示没有回归问题。这对工程师来说很容易验证，我们还可以把一些截图和视频附到 PR 上。iOS、Android 和 Web 也都能用同样的方式测试。如果你想测试后端 API，也可以用它来测试后端 API。It ran agent browser in that slate environment with what is on main and what is on my branch and it created like a report for me to review the full set of changes that happened. Um and so then like you know after this we can go to the like a report that it generates like see it generated a report that shows before and after uh and it's showing me like all the there's no regressions and this is very easy for like you know engineers to validate and we can attach some of the screenshots and videos to to the PR and the same thing we have for uh iOS Android web and if you want to do backend APIs then you can also test backend APIs.

嘉宾：我只是想补充一点，你知道，如果你不是 Uber 的外部人士，可能不会意识到这一点：Slate 非常重要，因为直到今天，Uber 仍然有数千个微服务，或者也许不能算微服务……所以，如果你想端到端地运行一次变更，哪怕只针对一个服务，也很困难，对吧？这就是 Slate 最初花了好几年才建成的原因。Guest: I I just want to add like you know like one of the things that again uh you might not appreciate if you're not outside of Uber but Slate is so important because Uber literally it still today still has thousands of microservices or maybe not not micro but so to run a change on just one service when you want to do end to end that's just difficult right like this is why Slate took years to build to start with

主持人：是啊，而且你还得想想置顶账户的路由问题，对吧？Host: yeah and you also have to think about like the routing of the pinned accounts right like

Guest：而且，当你把账户固定下来时，比如说，这是我希望你路由到我那个分支的测试账户，而那个分支刚刚部署在 staging 环境里——这可一点都不简单。事实上，这件事的实现方式之一，是我们大概在 2017、2018 年左右构建的东西，叫作 Jaeger，它是一个分布式追踪基础设施，现在已经开源了。我们所有的服务基本上都有 Jaeger spans：无论调用从哪里发起，都会有一个父 span；每经过一跳，就会有一个子 span。而且你可以在追踪中添加任意 baggage，并让它沿着调用栈传递。所以，当你从这个 late web 上固定账户时，本质上就是在添加这样的信息：嘿，这些是正在测试的服务，这是我希望路由到的账户。Guest: and and when the pinned accounts like the this is the test account that I want you to route to my to my branch that is just deployed in a staging environment that's not trivial at all. So like in fact how this works is one of the things that we in that we built um like back in maybe 2017 18 like uh it was it's called Jerger which is a distributed tracing infrastructure. It's open source now. Um so and all of our services have Jagger spans essentially like whenever wherever the call originates there's a parent span and at every hop there's a child span and there is a you can add any baggage on that uh trace which can be propagated across the call stack. Um so when you pin uh from this late web we are essentially adding saying that hey like for these are the services under test and this is the account that I want to route.

所以，当它经过我们的服务网格时，服务网格会知道这些 baggage，然后就可以说：“哦，我想把这个请求路由到 Slate 环境，而不是这个环境。”因为我们已经构建了追踪系统，而且它现在能在整个技术栈中传播，所以你实际上就可以进行测试，我们也能可靠地把这类测试请求路由到正确的环境，这让测试变得非常容易。把这一套东西加进去花了好几年，确保所有现有服务都完成了改造，所有新服务在构建时也都考虑到了这一点。所以我开始感觉到一种趋势：比如说，Uber 在 AI 出现之前花了好几年构建基础设施，让人类能够轻松测试，或者让自动化测试变得容易。而现在有了 AI，感觉就像是“哦，你们去年才搭建了这套东西”，但实际上你们是站在多年打造的基础之上，对吧？So as it is going through our service mesh, service mesh knows this baggage and then it can say oh I want to route this request not to this environment but to the slate environment. So because we had tracing built out and because it's like propagated across the stack now you could actually test and we can reliably route these kind of test requests to the right environments so that it makes it really easy to test. This thing took like years to add all to make sure that all existing services were retrofitted with this and all new services were built with this in mind. So like I'm I'm starting to sense a trend where like Uber built stuff over years pre AI just infrastructure that is easy easy to test for like you know humans or or easy to automate and now with AI it seems like oh you just built this like the last like year but actually you built on top a foundation that was built for years right?

是的，我确定。我觉得，如果我们没有像 Slate 这样的系统，智能体就很难进行自我验证，那样你就无法自动化这件事。就像你说的，这类东西确实很有帮助。我还想再聊一件挺有意思的事，这件事非常 Uber 特有，而且……Yeah, I'm sure. I think if we did not have like system like slate, it would have been really hard for agents to self validate like then like you cannot automate uh this thing and like yeah as you said like this kind of stuff um uh really helps um and um I I'll also like talk about like one more uh like pretty interesting thing which is so Uber specific and

主持人：什么是锥度？Host: what is taper?

嘉宾：是的。而且这件事真的非常非常难做好。过去我们也做过很多个类似这样的版本，但我觉得这可能是我们做得最好的一个版本，而且我们真的可以把它推广到所有团队。基本上，这个想法就是，嗯，关于上下文，因为即使看看我们的核心骑手应用，对吧？我们也有数百个界面，我们有——Guest: Yeah. And this is like very very hard to get right. And like we had like even many versions of uh build like this in the past but I think this is probably our best version and we can like really scale it to all the teams. Essentially the idea is um like context like because we have even if you look at our core rider app right like we have hundreds of screens we have

主持人：而乘客端应用就是大多数人想到的 Uber 应用，对吧？那个应用……Host: and the rider app is what most people think as Uber app right the app

嘉宾：是的，比如我们有不同的业务、不同的产品线、不同的城市，还有一大堆其他维度。如果我想测试任何新东西，怎么才能知道这些维度的每一种组合都不会出现回归问题？如果要把所有排列组合都测一遍，就得在整个技术栈中维护成千上万个测试，而光这一点本身就已经是个非常棘手的问题了。Guest: yeah like we have different business different product lines different cities different like whole bunch of dimensions like if I want to test any new thing how do I know every combination of these has no regression and if you were to like create all permutations across is you would have to maintain thousands and thousands of tests across the stack and that alone is like again really hard problem in itself.

主持人：所以，对于一些最复杂的平台，比如我们称之为“捕获回放”的平台，基本上只需一个中间件，就能捕获真实的生产流量。因为现在所有服务都采用了标准化的语言框架，所以接入起来非常容易。它捕获的不只是服务层的入站和出站流量，还包括该服务调用的所有下游 API；这些请求和响应也都会被捕获。某种程度上，你可以捕获所有入站请求和响应，以及所有出站请求和响应。捕获到的所有这些请求会经过一条流水线，然后你就可以制定一些简单的规则，比如：如果请求中出现某种字段，那么它就是这个维度的取值。这样一来，你就能捕获平台团队关注的各种维度。Host: So for some of our most complex platforms like we we call it capture replay like essentially you can capture real production traffic with just one middleware uh because now we have standardized language frameworks across all the services. is very easy to hook this into and it will capture not just the inbound and outbound at that service layer but also any of the downstream APIs that service calls those requests and those responses are also captured. So in some sense you have like pull request response and all of your outbounds request responses captured. Now with all of these captured requests they go through some pipeline and then you can build simple rules that says oh if you see this kind of field in the request this is the value for that dimension. So then you can capture all kinds of dimensions that that platform team cares about.

一旦你掌握了所有请求和所有维度，就可以在某种程度上生成该平台当前支持的完整实时维度和实时组合集合。我们把它称为 matchguard，可以在 CI/CD 中的预部署阶段运行。这样，对于此次部署中提交的所有 PR，我都可以针对该平台支持的完整组合集合进行验证，并展示部署前后的差异。这样你就能获得很高的信心，确认自己是否会因为某个配置导致仅在巴西出现现金行程问题——这类情况让工程师或代理手动测试都相当困难。And once you have all the requests and all the dimensions, you can in some sense generate the full set of live dimensions and live combinations that that platform is currently supporting which we call it matchguard that I can run pre-eploy in our CI/CD that shows for all of the PRs that are going in this deploy let me validate against the full combinatorial set of like combinations this platform supports and show that delta before and after. So then you get really high confidence if am I breaking maybe cash trips only in Brazil with this configuration which would have been really hard for engineers to manually test or agents also to manually test.

嘉宾：所以，所以第一步是捕获你想要捕获的内容，然后让系统重新运行它，检查是否出现回归问题。Guest: So so step one is you capture what you want to capture and then you have the system that will rerun it check that there's regressions.

主持人：是的。然后我会进入启用了这个功能的某个域名。这样你就能看到它捕获的所有会话。我可以打开任意一个会话。然后你可以看到所有元数据和标签。也就是说，它会显示：哦，这个请求被标记为 marketplace 类型的 eats。它被标记为业务线 eats。它处于这种终端状态。它知道……捕获了所有请求。你还可以看到它使用的所有出站请求。点击其中一项后，它还会显示这些出站请求的请求和响应。如果你看顶部，会看到 slate workflow UID 或 search environment name。比如说，如果我正在处理一个 bug，而这个 bug 不知为何只影响旧金山的 Uber Eats 行程……Host: Yeah. So like then I'm I'm going to one of the uh domains that has this enabled. So you can see all the sessions that it captures. I can go to any of the sessions. Um then you can see all of the metadata and tags. So essentially it's saying oh this request is tagged as marketplace type eats. It's tagged as uh line of business eats. It is in this kind of terminal state. It knows uh captured all the request. You can also see all of the outbounds that it used. If we click on that it will also show you the um the request and response for the outbounds. And if you see at the top there is slate workflow UID or search environment name. Um so then let's say if someone if I'm working on a bug that's only affecting for some reason San Francisco Uber Eats trips

嘉宾：嗯，我修复那个 bug 后，就可以为它创建一个 slate 环境，嗯，用来测试我的代码更改。我可以到这里，然后说：“只在我的 slate 环境中重放这个请求，并在这里显示差异。”Guest: um I I fix that bug then I can create a slate environment for that bug um for for my code changes. I can go here and then say replay just this request on my slate environment and show me the delta here.

主持人：给我看看有什么不同，你是希望得到修复或做出某种改变吗？Host: Show me the difference and you're expecting a fix or some change.

嘉宾：是的，我在等这个修复。所以，因为我知道请求前后分别是什么情况，就可以针对某个特定维度来操作，或者像这样使用它。然后我们还会生成之前提到的 match guard，也就是该场景下所有可能维度的完整集合。这样，你就可以为团队创建一套自动化测试套件，也能更轻松地手动测试你所关注的某个特定组合。Guest: Yeah, I'm expecting the fix. So because I know the before request, after request what is what I'm and then I can do it for a specific dimension or I can use it like then the uh and then we also generate what we mentioned like match guard which is the full set of dimensions possible for that. So then you can create like a in some sense a testing automated testing suite for that team and also like more easy way to uh manually test one specific combination that you that you are focused on.

主持人：我在那里的时候，我们还没有这个，不过有个叫 Deputy 和 cberus 的东西，但我想那应该是非常早期的版本。Host: When I was there we didn't have this but we have something called deputy and cberus but those were I guess very early versions.

嘉宾：是的。所以，呃，事实上，像 Deputy 和 Cberus 提供的能力是：Deputy 允许将生产环境中的任意请求路由到你的测试实例，而 Cberus 允许将请求从你的本地测试实例路由回预发布服务器。不过现在我们有了 Slate，就不一定非得用 Deputy 和这些服务器了，因为你已经可以部署一个 Slate 环境，我们不需要用它们中的任何一个。我只是想强调这一点，因为我觉得从外部很难看出来，但这真的很难构建：再说一次，你们有成千上万个服务，所有服务都必须支持这些功能，比如请求路由、捕获生产环境流量以及回放，对吧？这工作量可不小，我猜。Guest: Yeah. So uh in fact like what uh deputy and Cberus allowed was ability to route any what deputy allowed was ability to route any request from production to your testing instance and Cberus allowed from your local testing instance back to the staging uh servers. But now that we have uh slate you don't we don't necessarily have to use deputy and servers because like it's already like you can deploy a slate environment and we don't need to use any of them. I just want to stress because I think it's hard to see from the outside but that's really hard to build this because every single again you have the thousands of services all of them have to support these things from routing for example to capturing production to replaying right like this one's a lot of work I assume

主持人：是的，现在我觉得，和我们把 Slate 开放给智能体的方式类似，我们也可以把 Capri 的所有功能开放给智能体。这样智能体就能检查：“哦，我正在尝试修复这个问题，让我获取所有这种类型的会话，然后看看请求是什么样的。我能不能在我的 Slate 实例上构造一个这种类型的请求，看看问题有没有修好？所以，问题修好了吗？如果没修好，就让我不断循环尝试，直到修好为止。”Host: yeah and now I think same like similar to how we exposed like slate to agents we can expose all of the capri functionality to agents so agents can check oh I'm trying to fix this let me get all of the sessions of type and then see what request looks like. Can I compose a request of that type on my Slate instance and see did it fix? So, did it not fix? Let me go in a loop until I fix this.

嘉宾：是的，是的。而且现在，尽管代理看不见，但它可以看到变化量，并判断出例如什么都没有改变。这样它就能形成反馈循环。Guest: Yeah. Yeah. And and and and now the agent can even though it cannot see but it can see the delta and it can tell that nothing has changed for example. So it it can get feedback loops.

主持人：没错，没错。我觉得这确实很有帮助。现在，类似地，同一个 agent 也可以在 Slate 上进行捕获回放。呃，对于这些中的任何一个，我们都……我们……Host: Exactly. Exactly. I think that that that that helps a lot. Now like the same it can agent can also do capture replay on slate. uh with any of these we al we

边缘网关、Shepherd 和 WisdomEdge Gateway, Shepherd, and Wisdom

已经谈到过了，比如边缘网关。already talked about like the edge gateway

嘉宾：Edge Gateway 就是 Uber 的那种内部 API，对吧？它以前叫 RT API，也就是实时 API，但现在叫 Edge。它是 Go 版本，对吗？如果我没记错的话，它是用 Go 重写的。Guest: edge gateway that's just the kind of Uber's internal API right like we just we it used to be RT API real-time API but now it's called edge it's a it's a go version is it it was rewritten in go if I remember right

主持人：是的，它是用 Go 编写的，现在支持很多种端点类型，因此外部 API 无论是 gRPC、webhook 还是 GraphQL，都可以通过同一个入口接入。这样我们就能围绕它构建各种工具，用于监控和告警。它甚至还具备各种可靠性功能。Edge 也有自己的捕获与回放功能，类似 Capri；它在幕后使用 Capri。如果你想了解任何组件的可靠性，我们可以查看该服务的整个服务图，并告诉你可能存在问题、需要重点关注的节点。通过 Jagger tracing，我们能准确了解服务依赖关系，还会自动标记哪些是必需依赖，哪些是可选依赖。Host: yeah it's in go and now it supports whole bunch of endpoint types so there's external API whether it's gRPC web hooks graphql so like again all APIs go through one single entry point um where we can build whole bunch of tooling around that on monitoring on alerting um um and like even there is like a um uh like where there's all the reliability features the edge also has its own capture replay similar to the capri it's using capri behind the scenes and uh like if you want to look at reliability of anything it can like we look at the entire service graph for that service and tell you what is the potential unhealthy node that we need to look at the with Jagger tracing we know exactly the service dependencies we've also automatically tagged which is a required dependency which is an optional dependency.

所以这些元数据也都在。我们查看告警、诊断告警时，这些丰富的上下文确实能帮助工程师更快地调试，也能帮助智能体更快地调试。对，我们已经谈过实验了。呃，最后也许要讲的是，我们迁移到 monorepo 之后，提到的 Sheepard 和 Fox。我们迁移到 monorepo，是因为升级库非常困难，对吧？So that metadata is also there. So as we are even when we look looking at alerts and diagnosing alerts all of this rich context really helps both engineers to debug faster and also agents to debug faster. Yeah we already talked about experiments. Uh the last thing uh maybe we'll cover is like um what we call sheepard and fox after we moved to monor repo like then we moved to monor repo because upgrading libraries was very hard right?

嘉宾：是的，是的，在所有不同的代码仓库上。是的。Guest: Yeah on on all of the different repos. Yeah

主持人：没错，然后你转到 monorepo，但现在你需要有人去把那个仓库里所有模块对这个库的用法都升级一遍。Host: exactly then you move to monor repo but then now you need to someone has to like go and upgrade this library usage across all of the modules in that repository.

嘉宾：那我们要怎么自动化呢？我们最初尝试使用 OpenRewrite 规则。比如说，如果只是简单地修改签名，你可以把它写进 OpenRewrite 规则里，然后 Shepherd 就知道不同团队负责哪些独立模块，并自动为每个模块创建 PR。后来 AI 加入之后，我们就可以引入基于 AI 的 Shepherd。比如说，如果我想在所有服务中修复上下文传播问题，让追踪效果更好，就可以创建一组提示词和技能，让代理在每个模块中运行，更快地修复每项服务里的所有问题。这就是我们针对模块进行大规模变更的方式：无论是确定性的变更，还是由代理执行的变更。多年来，我们通过这种方式完成了大量迁移。我们把每一项称为一个 campaign。比如查看从 JUnit 4 到 JUnit 5 的迁移，你会看到一个包含规则的 YAML 文件，还会看到为每个模块创建的一大批 PR。campaign 的负责人可以跟踪进度，并逐一查看这些 PR。Guest: So how do we automate that? So we first took a stab at like using open rewrite policies saying okay if it's a simple like you know just the signature change you can write that in open write policy and then shephard knows what are the unique modules owned by different teams and then it automatically creates PR for each of those modules when AI came into picture then we could add like a AI based shepherd where okay if I want to do fixed context propagation across all the services so that tracing works better. I can create set of prompts and skills that then agent can run across every module and fix all the issues in every service faster. So this is kind of our large scale uh changes whether you want deterministic changes or whether you want like agentic uh changes across the modules that gave us um like we were able to do a lot of migrations over the years uh through this and we call each one as a campaign and if you see JUnit 4 to JUnit 5 there will be a YAML file with the policy and you'll see whole bunch of PRs created for each module and you can a campaign owner can track it the status of that and and go through that.

这样就能非常轻松地管理这些迁移。几个月前，我们在同一个平台上又添加了一项功能。我们也希望让这件事变得简单。我们希望持续改进代码库，而不只是进行一次性迁移。假设我想要……So it makes it really easy to manage these migrations. Few months ago then we took the same platform and added like one more feature. We also want to make it easy. We want to continuously improve the code base notet one-time migrations. Let's say I want to

主持人：所以，就像代码维护一样。Host: so like code maintenance.

来宾：是的，代码维护。所以我们称它为 fox。本质上，我们想要创建能够自我改进的代码库，因为我们已经有一大堆智能体技能。基本上，Uber 的任何服务都可以到这里来，将自己的服务接入不同的技能，他们可以说，我想要Guest: Yeah code maintenance. So we call it fox. Essentially we want to create self-improving code bases since we already have whole bunch of agent skills. essentially any service at Uber. They can go here, enroll their service into different skills, they can say, I want

主持人：清理死代码。Host: dead code cleanup.

嘉宾：我想要清理死代码、做性能分析，或者处理噪声日志。这类你可以接入的技能有一大堆，呃。Guest: I want dead code cleanup or performance analysis or noisy logs. There's like whole bunch of uh these skills that you can onboard to.

主持人：哦，真聪明。Host: Oh, smart.

嘉宾：Fox 会做的是，每周一，或者基本上每个周日，在 CI 负载没那么高的时候，它会为这个服务生成一个新的 PR，用死代码清理之类的方式。比如，能不能清理一些代码？能不能提升它的性能？能不能修复一些嘈杂的日志？所以，这有点像是自动化的代码库改进，然后，嗯，就像这样向你展示你可以如何，嗯，就像……Guest: And what Fox does is every Monday or essentially every Sunday it will create it will uh when we don't have that much CI load, it will generate a new PR for this service using dead code cleanup like sex. Oh, can I clean up any code? Can I improve the performance of this? Can I fix any noisy logs? So, it's kind of automated then codebase uh improvement then um like this showing you how you can um like

主持人：我挺喜欢这个，因为我听说有人会做这样一件事：比如在 Claude Code 或 Codex 里设置一个 AI 循环，让它去寻找改进代码库的方法，然后每晚或每周执行一次。但听起来这个方案要……我不知道，优化得多得多。而且我特别喜欢你可以把代码库纳入进来，因为大家都知道，在 Uber，代码库是由某个工程师或团队负责的，对吧？Host: I I like it cuz I heard people do this thing where they would like set a AI loop let's say in cloud code or codec saying you know like look for ways to improve my codebase and they would execute it every every night or once a week. But sounds like this is just a lot more I don't know like uh optimized and I I I love that you can enroll your code base because everyone know like a code base is owned by an engineer or team at Uber, right?

嘉宾：对，没错。而且，呃，你也能看到，因为我们正在快速扩张，目前我们仍处于主动选择加入阶段。呃，等我们改进了一些技能后，就会把其中一些转到主动选择退出阶段。呃，不过，只要你为某项服务获得了一项技能，某种意义上说，每周都会有一个 PR 被创建出来。嗯，这就像一个托管式的智能体循环。这样你就不必……呃，从某种意义上说，你会持续改进它。我们减少的琐碎工作越多，工程师就能把更多时间用于真正的功能开发。所以我们会一直琢磨：怎样才能进一步减少工程师的琐碎工作，并把这些事情都自动化？我们也会持续审视各种维护类 PR：是否需要创建一项新技能？是否需要改进现有技能？Guest: Yeah, exactly. And and uh like you can see like I mean because you're rapidly scaling we have we have still in the opt-in phase. Uh once we improve some of the skills we'll move some of them to opt out phase. uh but when anytime you obtain a skill for a service then every week in some sense you are getting a PR created um it's like a managed agentic loop uh so then you don't have to like uh you are continuously improving this in some sense the more toil we reduce the more amount of time engineers can spend on the true feature work so like constantly thinking of okay how do how do I eliminate more toil from engineers and like and automate all of these things And um so we constantly look at all kinds of PRs with maintenance category PRs. Do we need to create a new skill? Do we need to improve that skill?

这样，那一类 PR 就可以，呃，纳入这个自动化管理流程。So that that class of PRs can be uh onboarded to this this automated manage loop.

主持人：这不错，我喜欢。最后，我听说过智慧。什么是智慧？Host: This is nice. I like it. And then finally I I heard about wisdom. What is wisdom?

嘉宾：哦，是的。嗯，我们使用 Wisdom，是因为我们的员工都会内部试用自己的产品。所以，如果你在使用我们的骑手应用或收入者应用时发现了任何 bug，就可以直接在应用里报告。我们现在有各种各样的 AI 功能，比如会根据问题内容自动将报告转给合适的团队。而且，因为 bug 报告来自员工使用的测试版应用，我们可以收集大量相关日志和遥测数据，这样负责该功能的团队就能很轻松地修复问题。比如这个案例，它是一个内部功能——基本上，如果你想在另一个国家测试某项功能，就可以在设备上模拟所在位置。有人报告了这个问题后，首先要考虑的是：现在该怎么复现？过去，复现问题曾是个大难题。Guest: Oh yeah. So um the we use wisdom for um because we we all of our employees dog food our products. So if you're using our rider app or earner app and if you see any bug then you can from the app report that bug. We now have all kinds of AI stuff where like it will automatically route it to the right team based on the stuff and then when you report a bug because it's from the employee from the employee like beta apps we can capture whole bunch of logs and telemetry around that that makes it really easy for the team owning that feature to fix it. So like in this case it's like an internal feature like it's essentially like if you want to test uh any feature in a different uh country you can mock the location on the device. So if someone reported um that that that issue the first thing is okay now how do you reproduce that before it is it used to be a huge problem to reproduce.

所以现在我们知道用户经过了哪些屏幕，因为这些信息都记录在日志里。然后我们可以把这些信息交给工程师，比如复现步骤、预期行为和观察到的行为，这样就能快速归纳出问题，以及我们从应用中收集到的所有遥测数据。你可以在一个地方看到所有日志。比如分析数据、网络请求、各种功能标志等等。这也提供了很多上下文。有了这些上下文之后，我们构建了一个名为 debug assist 的托管代理，它会查看所有这些信息和代码库，并尝试找出根本原因，以及与之相关的影响、证据和时间线。嗯，一旦我觉得这就是正确的根本原因，就可以把它交给我们之前提到的 minion。So now we know the the set of screens the user has gone through because we have that in the logs and then we can give the engineers like this is the reproduction steps and this is the expected behavior and this is the observed behavior and so that we can boil down to the issue quickly and all of the telemetry that we collected from the app. You can see all the logs uh in one place. So like whether it's analytics, network requests, uh all kinds of um uh feature flags. So that also gives a lot of context. And now now that we have all of this context, we built a managed agent called debug assist that looks all of this looks at the codebase and it will try to pinpoint what is the root cause and like what is the impact evidence timeline around that. Um now I once I think okay this is the right right root cause I can hand off this to minion that we talked about

主持人：嘿，编码代理，这个根因现在看起来不错。那你为什么不创建一个 PR 呢？Minion 会利用我们掌握的所有上下文来创建 PR，并把它关联到 Jira 任务上。Jira 任务里也有根因分析，所以任何参与处理的人都能了解根因分析。如果有人在评论里更新了根因，或者又创建了一个 PR，这些数据都能帮助我们弄清楚：该如何改进调试辅助功能、如何改进 Minion？我们没有得到正确的 RCA，或者没有根据 RCA 得到正确的 PR——这两方面我们都能持续改进。Host: hey coding agent this root cause looks good now like why don't you create a PR for this minion will take all of that context that we have it creates a PR it will attach that PR to the Jira task the Jira task also has the root cause analysis so anyone who's working they know the root cause analysis if someone updated the root cause on the comments or like create a new PR again all of that is data that we can then figure out okay what how do we improve the de debug assist how do we improve minion we didn't get the right RCA or we didn't get the right PR from the RCA like both we'll be able to uh continuously improve

嘉宾：那么，我理解得对吗？这样你就能更轻松地捕获并复现 bug，对吧？Guest: and then so do I understand correctly that you can capture and reproduce bugs so much easier right

主持人：是的，而且我们还增加了这项功能，因为对于员工报告的 bug，我们有所有日志。我可以拿上我的测试设备，要求它模拟那位员工的经历。然后，我只要提供 bug 的 UID，我们基本上就能向我的设备返回当初发送给提交 bug 的那个人的相同网络响应，这样我就能在我的测试设备上体验到同样的问题，也能在应用里体验。这样我就知道了：修复之后，我可以确认问题确实修好了，而且复现起来也容易多了。我特别喜欢对整个过程进行录制，因为这真的非常重要，对吧？即使是在 Web 上，这也够难的了。如果你想做网站，就得为自己开发一些定制工具。不过你们能做到这一点真是太棒了。Host: yeah and we also added ability to because we uh for the employee reported bugs we have all the logs I can taste take my test device and I ask I can ask it to simulate what that employee has experienced. So then um I once I give the uh bug UID we can essentially serve the same network uh responses that we served to the person who filed so that I can experience the same issue in my test device uh app 2. So that way I know okay did then once you fix it then I can see okay this actually got fixed and then it makes it really easy to reproduce. I I I I love the recording of of the thing because that's I mean so important, right? Like it's hard enough to do even on the web. You need to build like custom tools for yourself if you want to do a website. But that's awesome that you did it.

嘉宾：是的，我觉得这很有帮助，比如能提升复现性，也能更准确地找到根本原因。你搭建了所有这些系统，尤其是去年，似乎推出了很多全新的系统，它们能做很多事情，比如维护代码质量、让智能体自我验证。你怎么看？Guest: Yeah, I think this helps a lot like then in repro in uh reproducibility and like getting to the right root cause. So you you built all of these systems and especially last year it seems so many brand new systems that are just doing things like you know they're maintaining the the code quality agents are are validating themselves. How do you think

工程师角色的变化与生产力衡量方式The changing role of engineer and measuring productivity

Uber 内部软件工程师的工作正在发生变化，而你从所有这些新工具中学到了什么？感觉你是在自己摸索。你知道，在某种意义上，如果你身处智能体 AI 领域，就会不断尝试找出下一个瓶颈，然后设法消除它，并思考如何既利用 AI，也利用实际的系统工程来减少这个瓶颈，因为归根结底，我们的目标是尽可能多地在所有产品中发布功能。要做到这一点，我们就得审视整个生命周期：如何重新调整并为新时代做好准备，需要构建什么样的工具。我认为这会是一段持续的旅程，对吧？大家也在大量讨论一个叫作“能力悬置”的概念，因为我觉得即使是现在，我们也有很多事情可以借助最新模型来完成，但我们还没有足够的编排能力、脚手架或配套工具，无法在合适的地方以合适的方式利用这些模型。the job of software engineers inside of Uber is changing and what are you learning from all of these new tools? It seems like you're learning yourself. you know like in some sense like if you're in the agentic AI world like you're constantly trying to find the next bottleneck um and trying to like remove that and figuring out how can you can like both using AI and both using like both actual systems engineering like reduce that bottleneck because ultimately our goal is to maximize the number of features that we can ship uh across all of our products and that will require us to look at the entire life cycle how do we rewire prepare that for the new world and what kind of tools that we need to build and I think this is going to be like an ongoing journey right and there's a lot of discussion of you know this concept called capability overhang where feel like even right now like there's so many things that we could you leverage the latest models for but we don't have the sufficient orchestrating capabilities or like the harnesses or the tools around that to be to leverage those models in the right places in the right way.

我觉得，我们在这件事上做得越多，就越能释放……嗯，交付更多功能，提升更多生产力。我们也在努力衡量其中很多方面，并不断思考接下来需要构建哪些基础设施，才能让工程师更轻松地工作，也让智能体更容易开展工作。I feel like the more we work on this, the more we can unlock um the more features we can ship, the more productivity that we can unlock and like we are trying to measure whole bunch of this and like constantly think about like what is the next set of uh foundations that we need to build to make it easy for engineers to work and to make it easy for agents to work.

主持人：你提到了生产力。你怎么看待工程生产力？我在 Uber 工作时，也就是 2020 年左右之前，作为经理，我们确实会关注一些指标。虽然这不是最重要的事，但我们会看每位工程师提交的代码差异量之类的数据，只是为了了解情况。比如，如果有人两周都没做出贡献，我们就会想：“嗯，你在做什么呢？”或者，如果有人做了很多代码审查，这其实是我们希望注意到的。不过听起来，这些东西——你知道的，代码产出、代码工件——可能没那么重要了。你觉得现在在 Uber 或者你的团队里，怎样才算是一名高效的工程师？Host: You mentioned productivity. How do you think about engineering productivity? Because when I was at Uber, this was again in like up to 2020, us managers did look at things. It wasn't the main thing, but we did look at things like diffs per engineer just to show, you know, if someone didn't contribute for like 2 weeks, it was like, hm, what are you doing here? Or if someone did a lot of code reviews, that's actually what we wanted to notice. But it sounds like a lot of those things, the, you know, the code output, the code artifacts are less relevant potentially. You know, what what is a productive engineer in your mind right now inside of Uber or inside of your team?

嘉宾：是的。所以我觉得，在 AI 时代到来之前，我们衡量的都是一些常规指标，比如代码吞吐量之类的，对吧？这些指标比较容易衡量。然后我们还会每隔几个月做一次开发者体验调查，看看 NPS，也就是各个维度上的整体情绪；看看代码质量的各项指标。每个 def 的事故数是多少？代码覆盖率是多少？还有所有这些指标，对吧？Guest: Yeah. So, I think like before the AI world, like we were measuring, you know, like the standard stuff like code throughput and stuff like that, right? Those are the easy ones. And then we would we also have like every few months we do the developer experience surveys. We look at NPS you know like sentiment across all kinds of dimensions. We look at our code quality dimensions. What's our incident per def? What's our code coverage and all of these metrics right?

嗯，我觉得现在到了 AI 这个阶段，我们可以利用这批新能力，让讨论更细致一些。因为现在我们内部讨论的不一定只是 diff 速度，还有功能交付速度：所有这些额外吞吐量中，有多少用于解锁新功能？我们又能如何调整组织流程，缩短 P50 功能交付时间？因为你可以创建 100 个 diff，但如果它们并不全是功能相关的 diff，那可能并没有提升我们能为用户创造的价值，对吧？比如，我们做的一件事就是给所有 PR 打标签，按复杂度级别分类，也标明是功能 PR、bug 修复、维护、测试还是文档等类别。这样我们就能逐年观察这些类别的占比如何变化。好在至少目前，我们功能类 PR 的比例正在上升。Um I think now once you got to like AI I feel like we can uh leverage like the new set of capabilities to get more nuance into this discussion like because now we are internally talking about not necessarily just diff velocity but feature velocity like how much of all of this additional throughput is going towards unlocking new new features and how can we even change our organizational processes so that we can reduce the P50 feature time because like you can create 100 divs but if they're not all feature diffs then like maybe it's not necessarily increasing the value that we can provide to the users right so like one for for example one of the things we did is tagging all of our PRs to different categories like on complexity level and is it a feature PR bug fix maintenance testing documentation on different categories and we could see year-over-year how is that mix changing luckily like at least like right now I can our fe our percentage of feature uh PRs is going up.

维护类 PR 的占比正在下降，这很好地说明我们正朝着正确的方向前进。我们看到代码量在增加，功能类 PR 的占比也在上升，这意味着我们投入了很多。因为以前即使有 UpPlan 工具，有时也无法把产品设计文档、引擎设计文档和具体任务正确地关联起来。因此，要可靠地衡量所有功能端到端的周期时间，就会比较困难。 现在我们用 AI 自动对这些内容进行聚类，并标记缺失项，这样就能可靠地了解过去两三年里，每个半年周期中不同复杂度功能的 P50 交付速度。接下来我们就可以分析：瓶颈是在 PRD 阶段、发布阶段，还是 rollout 阶段？究竟出了什么问题？我们需要在哪些方面投入，才能加快每个半年周期内可交付的功能数量？Our percentage of maintenance PRs is going down. So that gives us good indication that okay we're in the right direction. Okay, we see the code volume go up and the percentage of PRs that for features going up then we are investing a lot because before even though we had the up plan tool sometimes you might not have the right tracing from this product design dog to this engine design dog to this particular task. So it becomes a bit harder to measure what is a cycle time end to end reliably across all the features. So now we're using AI to like automatically cluster these things and like tag any missing things um so that we can reliably get what is the P50 feature velocity for different complexity of features for every half over the last 2 years 3 years and then we can go into okay is the bottleneck in the PRD phase is the bottleneck in the launch phase is the bottleneck in the rollout phase what is going on there and what kind of things we need to invest to accelerate uh the number of features that we can ship per half.

主持人：那么，与 AI 出现前以及使用 PDS 工具时相比，你对软件工程的认知模式发生了怎样的变化？这还是同一份工作吗？情况变了吗？Host: And how is your mental model changing of software engineering compared to preAI and PDS tools? Is is it still the same job? Have things changed?

嘉宾：我是说，至少在我整个工程师生涯中，工作方式从未发生过这么大的变化。比如说，我们常说 Uber 的工程师都亲力亲为。像 Uber 的杰出工程师也都亲力亲为，会创建 PR。我以前觉得自己也是亲力亲为的工程师，但我对“亲力亲为”的定义，年复一年地彻底变了。就我个人而言，我现在提交的 PR 也比去年多得多。如今做这些事情都容易多了，对吧？我也能探索比过去多得多的想法。Guest: I mean, at least in my total tenure as in engineering, there's never been this much change to how I work. Um like for example like you know we say like all the engineers at Uber are hands-on. Like even distinguished engineers at Uber like are hands-on. and they like they create PRs and I used to think I was hands-on engineer but my hands-on definition year over year has completely changed. I'm like even personally I'm doing like way more PRs than what I was doing last year. Like it's more easy to do uh all of these things all of these things today right and I'm able to like explore more number of ideas than what I could possibly do in the past.

嗯，但工程工作的职责，在某种意义上，就是思考如何为智能体构建合适的上下文，让它们产出可靠的结果。你得非常擅长上下文工程。以前我会带着一些上下文去完成某项任务，所以我总是在想：如果我想把一部分工作交给智能体，它需要什么样的上下文？我该如何搭建环境，提供恰当的上下文，让它能够完成这项工作？然后，你还得非常擅长思考如何为智能体提供正确的规格和约束。除了构建合适的上下文之外，多任务处理也真的变得越来越重要了。嗯，以前创建一个 PR 要花很多时间，所以至少得全神贯注地处理它。uh but the job of the engineering is you know in some sense how do you uh build the right context for the agents to produce reliable output like in some sense like you need to get really good at context engineering before I was operating with some context to do some task so I'm always thinking about okay if I want to offload some of this work to an agent what kind of context it would need and how do I set up the environment with that right context so that it can produce that work um then you have to become really good at like you How do you give the right specs, right constraints for the agents apart from building the right context, then multitasking has become really like a thing? Um before if you're like it would take so much time to create a PR so like at least laser focused on that.

但现在你总是在这么多事情之间不断切换，所以你需要非常擅长在所有会话之间并行处理任务，弄清楚如何界定工作范围、如何跟踪进度，以及如何快速验证工作成果。因为如果你没法验证成果，比如没法从智能体生成的代码里挑出问题，那就很棘手了，最后可能会把一些回归问题带进生产环境，对吧？所以，像上下文工程、并行处理任务，以及审查并验证输出的能力，现在都变得重要得多了，因为你能并行处理的事情比以前多得多。But now you're constantly switching between so many things like so you need to become really good at uh multitasking across all the sessions figuring how do you scope the how do you scope the work how do you track the work how do you verify the work really quickly because if you're not able to verify the work and like pick any course from the agent then um then that's hard then you're shipping like some un like some regressions into production right so um like the context engineering ability to multitask ability to like review and like verify the outputs. These things have become like much more important uh now because you can because you're able to paralyze a lot more than before.

主持人：我注意到的一点是，也许一种新的原语是：过去我们构建软件时，会思考我该如何构建这款软件、又该如何验证它；而现在你说的问题往往是，我该如何设置一个智能体，让它能够构建这个东西并验证它。所以这有点像一种元层面的……Host: One thing I'm picking up is maybe a new primitive is we used to like build software thinking how do I build this software and how do I verify it whereas now the question that you said is often how do I set up an agent to be able to build this and to be able to verify it. So it's kind of like almost a meta like

嘉宾：没错。Guest: Exactly.

主持人：听起来还有更多类似的内容，对吧？Host: sounds like there's more of that, right?

嘉宾：是的。我给你举一个有意思的例子。我当时在和我们客户支持团队的一些工程师一起做事。我们正在探索如何让 AI 智能体处理部分客户支持政策。这个过程一开始是这样的：你有一份人工客服需要遵循的操作手册。接下来，你要经过一大堆步骤，把这份操作手册转化为 AI 智能体可以遵循的政策。然后你还得考虑其中有一些动态元素。接着要想清楚，我需要调用哪些 API 来获取这些数据，需要使用哪些属性。然后我得配置系统、修改一大堆配置。之后，在上线之前，我还得进行人工评估。上线之后，还得持续进行评估。最近几个月，我们已经能够把所有这些步骤都去掉。Guest: Yeah. I I'll give you an interesting example. I was working on like the with the with some of the engineers that work on our customer support uh team. Um like we are going through a journey of uh having AI agents handle some of our customer support policies. That process would go from oh you have some a runbook that human agents would follow. Um so then from that runbook you are going through a whole bunch of steps to create a policy that AI agent can follow. Then you have to think about okay there is some dynamic elements here. Then which APIs I need to call call to get that data which attributes I need to use. Then I need to configure my system change bunch of configurations. Then I need to do manual evals before launching. Then I need to do like continuous evals post launch. And then like in the last few months we are able to like take down all of the step.

好的。比如，我现在该如何把所有上下文以及我想用于政策转换的所有原则整合起来，并教会智能体完成这件事，从而得到一套非常出色、可以由智能体创建的 vzero 政策？然后，你可以自动完成所有配置设置、手动测试、寻找合适的 API、寻找合适的属性，这样我们就能把合适的动态元素接入其中。再说一次，以前这要花上好几周，现在任何一项政策都只需要几天左右。然后，不再进行手动评估，而是利用过去的所有上下文来生成合成评估，这样你就能加快进度、部署并快速验证。基本上，这就是我们正在讨论的，对吧？现在，你不再是……不再做那些工作。好吧，那该如何在此基础上构建合适的工具和支撑框架，从而实现工作流自动化？显然，你需要设置合适的防护措施，并在流程中加入人工验证环节，但你是在设计一个能够完成这些事情的系统，而不是在所有这些步骤中都依赖大量人工操作。Okay. Like how do I now take all of the context and all of the principles that I want to use for the policy conversion and teach agents to be able to do that so that I have really good vzero policy that that an agent can create. Then you automate the all of the configuration setup, manual testing, finding the right APIs, finding the right properties so that we can hook the right dynamic elements into this. Again, like that used to take weeks, now it's like order of uh days for any policy. Then instead of doing like manual evals then how can you like use all of the past context to create synthetic evals on top of that so that you can like you can accelerate you can deploy this and then verify this quickly like essentially what we're discussing right which is now you're not instead of doing that work okay how how can you build the right tools and hardnesses on top of this so that you can automate the workflow obviously you need the right guardrails and human verification steps in between but you're designing a system that can do these things instead of like relying on so much manual work across all of these steps.

主持人：是的。听起来你学到了很多，因为这些都是你现在开始使用的新概念。你正在了解这些东西。你正在尝试新的限制条件。我、我、我明白你的意思，比如更多学习、更多 PR。作为最后一个问题，有什么工具是你特别……Host: Yeah. And it sounds like you're learning a lot by cuz these are all new concepts which you're now using. You're learning about it. You're doing limitations new. I I I see what you mean by you know like more learning more PRs and as closing what is a tool that you really

Uday 最喜欢的新产品Uday’s favorite new products

比如在用什么？嗯，这可以是数字工具，也可以是非数字工具，任何你觉得有用的东西。是啊，我知道，最近几天我一直沉迷于一些个人智能体，比如 Instinct、Mio，还有 Grocks 之类的。我正在尝试用它们。它们帮我自动处理了很多我个人生活中的手动工作，而且在某种程度上，我可以按照同样的流程来做：给它完成任务所需的所有背景信息，提供我需要的所有输入。然后我可以加入人工审核环节，再完成这项任务。它确实帮了很大的忙，能把个人生活中各种手动工作自动化。like using? Um this this can be a digital tool this can be a non-digital tool something you find helpful. Yeah, I know the last few days I've been uh hooked to uh like some of the personal agents like instinct, Mio, Grocks of the world. I'm playing around with all of them. It's helping automate whole bunch of manual work that that I do in my personal life and I can in some sense this follow the same process. Give it all of the context it needs to do that task for me. Give me all of the inputs I need. Then I can have human in the loop verification step and do that do that task. But it's helping a lot to do all of the manual to automate all of the manual personal work.

嘉宾：太棒了。嗯，Udai，非常感谢你带我们了解 Uber 正在使用的所有内部实现细节。我觉得这特别有意思，尤其是我已经很久没见过这个了。Guest: Awesome. Well, Udai, thank you so much for walking through all of all of the internals that are being used at Uber. I I found this super interesting, especially I haven't seen this for quite a while.

主持人：是的，非常感谢你给我们这个机会。我觉得我们现在还处在相当早期的阶段，还有很多工作要做，还有很多东西要构建。我们非常期待踏上这段旅程，构建合适的东西，看看软件工程会如何变化，以及归根结底，如何通过专注于正确的事情，为公司和业务释放价值。Host: Yeah, you know, thank you so much for giving us the opportunity. You know, we are I feel like we're still pretty early in this uh there's so much to work on. And there's so much to uh so much to build and then we're super excited like to be on this journey to ex to build the right stuff and to see how you know the software engineering changes and how at the end of the day unlock the value for the company, value for our business by working on the right set of things.

嘉宾：希望你们和我一样，喜欢这次对 Uber 工具体系的了解。我离开 Uber 已经 6 年了，所以 Udai 给我介绍的很多工具对我来说也是新的。10 年前，Uber 以拥有大约 5,000 个服务、约 2,000 名工程师而闻名。但从那以后，公司开始将服务整合为规模合适的服务。如今，它的微服务数量比十年前少了。现在大约有 4,500 到 5,000 个。虽然这些服务没那么“微”，而是规模合适得多。我特别喜欢的一款软件是 Capri，这是一款事件捕获与回放软件。2016 年到 2020 年我在 Uber 工作时，我们有两个名为 Deputy 和 Cerrus 的工具，可以在一定程度上回放生产数据，但它们并不太容易使用，而且一开始就很难捕获真正基础的数据。Guest: I hope you enjoyed this look into Uber's tooling as much as I did. It's now been 6 years since I left Uber, so a lot of the tooling that Udai showed me was also new to me. Uber was known for having about 5,000 services with about 2,000 engineers 10 years ago. But since then, the company started to consolidate services into well-sized services. And today, it has fewer microservices than they did a decade ago. Today, it's in the ballpark of 4,500 to 5,000. Even though these services are not as micro, and they're much more wellsized instead. One piece of software that I really liked is Capri, the event capture and replay software. Back in my time at Uber between 2016 and 2020, we had two tools called Deputy and Cerrus that could do some level of production data replay, but it wasn't all that easy to use and it was just really hard to capture really base to start with.

有了 Capri，捕获会话要比重放会话简单得多。这对于复现 bug，以及验证某个 bug 修复是否生效，都非常有效。像 Uber 这样运行着近 4,500 或 5,000 个服务的公司，在 Capri 构建系统尤其复杂。另一个有意思的方面是，我们正在看到代码审查的未来走向。代码审查在各处都在慢慢消亡。但看看 Uber 围绕代码审查构建的工具，比如代码收件箱，每个 PR 仍然要由人类工程师决定可以合并后才会被合并。不过，Uber 正在构建大量自动化检查，其中许多都借助 AI，为变更的风险程度以及 AI 代理产出的代码质量提供更多信号。这样一来，工程师就更容易判断哪些 PR 值得花时间深入审查，哪些则可以放心地只留下一句简单的 LGTM（看起来不错）就批准，也就是说，你可能会决定不读代码。With Capri, it's far more trivial to capture sessions than replay them. And this works really well to reproduce bugs and also to test that a fix worked for a certain bug fix. Building a system at Capri is especially complex when you have closer to 4,500 or 5,000 services running like Uber has. Another thing that was interesting is how we're seeing where code review is headed. It's it's slowly dying everywhere. But looking at the tooling Uber built around code review with code inbox, every PR still gets merged after a human engineer decides that it's ready to do so. But Uber is building a lot of automated checks. Many of them are with AI to give more signal about the riskiness of the change and the quality that AI agents make of it. And so in the end, it's a lot easier for engineers to decide which PRs they should spend time looking at it in depth versus ones that are safe to approve with the symbol LGTM aka you might decide you're not going to read the code.

最后，我的感觉是，Uber 在开发者生产力上投入了 10 年，如今通过 AI 集成确实获得了非常非常丰厚的回报。比如，他们早在 2017 年就打造了名为 dev pods 的云开发环境。Uber 在构建名为 minions 的云端智能体时，几乎可以复用所有现有基础设施。用于验证软件并安全地逐步推出软件的现有系统也是如此。现在，智能体可以把这些系统用作工具或反馈循环。因此，我觉得，从 AI 中获益最多的一些公司，正是那些在 AI 出现之前就已经拥有开发者平台团队的公司。至少 Uber 就是如此。欢迎查看节目说明，了解更多关于 Uber 工程文化的务实工程深度解析。到目前为止，Uber 大概是我介绍得最详细的公司之一。Finally, I had the sense that Uber investing 10 years in developer productivity is really really paying off with their AI integrations. For example, they built cloud development environments called dev pods back in 2017. Uber got to reuse almost all of its infra in building out their cloud agents called minions. The same is true for existing systems to validate software to safely roll them out. And now agents can use these as tools or as feedback loops. So it feels like companies that are getting some of the most benefit from AI are the ones that have had developer platform teams pre-AI as well. At least in the case of Uber. Do check out the show notes for more of the pragmatic engineering deep dives on Uber's engineering culture. Uber is probably one of the companies I have covered in the most detail so far.

如果您喜欢这期播客，请为它评分，并告诉您的同事和朋友。If you've enjoyed this podcast, please do give it a rating and tell your colleagues and friends about it.