Amin Vahdat：AI 基础设施的物理与经济约束

简要描述Brief Description

谷歌人工智能基础设施负责人 Amin Vahdat 谈前沿人工智能的物理学与经济学Google's AI Infrastructure Chief， Amin Vahdat， on the Physics & Economics of Frontier AI

目录Table of Contents

• 引言
  Introduction
• 是什么让数据中心成为 AI 数据中心
  What makes a data center an AI data center
• 有效吞吐量，而非 FLOPS：每小时都有故障时，如何要求自己承担责任
  Goodput, not FLOPS: holding yourself accountable when something fails every hour
• 每六个月将 token 容量翻倍，以及这些提升究竟从何而来
  Doubling token capacity every six months, and where the gains actually come from
• TPU押注：从2013年的逆向判断到拆分为8i和8t
  The TPU bet: from a contrarian call in 2013 to splitting 8i and 8t
• 支持与反对协同设计的理由
  The case for and against co-design
• 与 DeepMind 并肩作战：拦截运输途中的芯片
  Shoulder to shoulder with DeepMind: intercepting chips mid-flight
• 长周期智能体正在改变数据中心的形态
  Long-horizon agents change the shape of the data center
• 光路交换与网络技术的现状
  Optical circuit switching and the state of networking
• 电力是硬性约束：公用事业、电力规模（吉瓦），以及数据中心的规模规划
  Power is the binding constraint: utilities, gigawatts, and sizing a data center
• 训练集群与服务集群、七年前的 TPU，以及开放标准
  Training vs. serving clusters, seven-year-old TPUs, and open standards
• 轨道数据中心与2036年的超级计算机
  Orbital data centers and the supercomputer of 2036

引言Introduction

再回到硬件方面，正如你所知，存在这样一种可能：针对某种特定工作负载的专用化程度越高，硬件的灵活性就越低，但速度会越快，能效也会越高。所以这是一门艺术，也需要预测你要针对什么进行设计，以及这种工作负载会持续多久。换句话说，如果它一个月、两个月，甚至三个月后就会消失，那么即使它在这三个月里规模很大，你能抓住它的窗口期也非常短。因此，它必须有一定的持久性。呃，而且你必须能够准确预测，针对它进行专用化究竟能带来什么收益。In hardware again as you know there is this opportunity where the more you specialize to a particular workload the less flexible it is the faster the more power efficient the hardware is going to be. So it is this art and it's this projection of what are you designing to and how persistent is that workload. In other words if it's going to go away after a month or two months or 3 months even if it's big for those three months you got a really narrow window to intercept it. So it has to be somewhat durable. Uh and you have to be able to project ex exactly what win you can get for specializing to it.

主持人：非常高兴欢迎 Amin Vad 来到节目。Amin，感谢你今天来参加我们的节目。 嘉宾：很高兴来到这里。我真的很期待。Host: Thrilled to welcome Amin Vad to the show. Amin thank you for joining us for today. It's really exciting to be here. I'm really looking forward to it.

嘉宾：我对今天的话题非常兴奋，因为我们正处于人类历史上规模最大的资本支出建设浪潮之中，而你正身处其中。预计仅谷歌今年的资本支出就将超过2000亿美元，其中大部分将用于建设数据中心。呃，你正处于这一切的核心。去年年底，你被任命为谷歌人工智能基础设施部门的负责人。嗯，你正在领衔推动人类历史上资本密集度最高的建设项目之一，所以我真的很期待今天和你深入聊聊。Guest: I am very excited for today's topic because we are in the middle of the biggest capex buildout in human history. You are in the middle of it. Google alone is expected to spend more than $200 billion on capex this year. Most of which is going into building data centers. Uh you're at the center of it all. You were named the head of Google's uh AI Infra at the end of last year. Um, so you are the man spearheading the efforts uh of one of the most capital intensive buildouts in in human history and so I am really excited to get into it with you today.

主持人：毫无疑问，今年对整个行业来说都是非常重要的一年，当然也包括谷歌。说真的，我觉得我们从未见过这样的情况，至少在谷歌肯定没有。不过，正如你所说，从基础设施建设的规模和转型的速度来看，这在人类历史上都堪称不可思议。在开始聊这个之前，也许我们先……Host: It's a huge year uh for sure of course across the industry including at Google uh really have I don't think we've ever seen anything like this honestly certainly not at Google but as you said I think in the history of humanity in terms of the buildout and the pace of transformation really unbelievable. Before we get into it, maybe just just

是什么让数据中心成为 AI 数据中心？What makes a data center an AI data center

给观众快速介绍一下：什么是 AI 数据中心？它与非 AI 数据中心有什么不同？crash course for the audience. What is an AI data center and how how is it different from a non-AI data center?

嘉宾：这是个很好的问题。实际上，AI 数据中心和非 AI 数据中心有很多相似之处，并没有根本性的不同。我的意思是，它由混凝土结构组成，对吧？有建筑围护结构、电气场区、机械设备场区和冷却系统；你会看到一排又一排的电力分配设施。嗯，里面还有大量的网络基础设施。换句话说，我们把大量计算资源连接在一起。里面也会配备相当多的存储基础设施。我认为，我们现在看到的 AI 基础设施最大的不同在于专业化。过去建设数据中心时，它确实是一项面向 20 年、25 年乃至 30 年的建筑投资。我们会考虑它在这 20 年、25 年乃至 30 年间将如何演进。Guest: It's a very good question uh in that AI data center and a non-AI data center have do have a lot of similarities actually and they're they're not radically different. I mean it consists of concrete right an enclosure it consists of uh electrical yards uh mechanical yards cooling uh you have row after row of uh power being distributed. Um there's a huge amount of network infrastructure there. In other words, we connect large amounts of computes to one another. Uh a fair amount of storage infrastructure goes in there. I think that the big difference that we're seeing with AI infrastructure is specialization. In the past, when we're building a data center, it really is a 20 25 30-year building investment. And we're thinking about how it's going to be evolving over that 20 25 30-year period.

你可能会把服务器放进去，也可能会有网络设备、存储设备，还可能会有一些加速器、GPU、TPU，或其他设备。但规划周期是25到30年，而硬件的生命周期可能只有六年。因此，我们必须为许多代硬件做好规划。AI数据中心通常会更具定制性。换句话说，我们往往会根据可能进驻其中的硬件来共同设计建筑。我们可能会说：“你知道吗，实际上我们不会在这栋建筑里放很多存储设备。”为什么？因为一个存储机架可能需要10、20、30或40千瓦的电力。把它放在一个TP机架或GP机架旁边，而后者如今轻轻松松就能达到数百千瓦，未来几年可能还会更高——人们正在讨论这一点。You might have servers go into it. You might have uh networking storage. You might have some accelerators, GPUs, TPUs, whatever they might be going into it, but it's a 25 30 year planning horizon. So we have and the lifetime of the hardware might be six years. So many generations that we have to plan for. An AI data center often times is going to be more purpose-built. So in other words, we are oftenimes co-designing the building with the hardware that might go into it. We might be saying, you know what, actually we're not going to put a lot of storage into this building. Why? Because a storage rack might have 10, 20, 30, 40 kilowatts of power. You put that next to a TP rack or a GP rack that easily is hitting hundreds of kilowatts today. Might hit me. People are talking about that in the next few years.

设计一栋建筑，可能要在一排里容纳 30 个存储机架，也可能只容纳一两个 AI 机架，这两种设计截然不同。光从规模上想想就知道了。再想想用电量，以及电力如何分配到整栋建筑里，等等。网络方面也一样。存储机架所需的网络规模很小，尤其是使用硬盘的存储机架，与 AI 机架相比更是如此。所以，如果你试图让它完全通用、可互换，建筑可能就会做得过大，配置得过于冗余。当然，在 30 年的时间跨度内，通用性还是很重要的。AI 数据中心很可能会更偏向专用设计，与硬件协同设计，甚至细致到冷却和电力分配等方面。因此，各方面的协同优化会多得多。你们给我的一家投资组合公司 Ineffable Intelligence 交付了一套大型 Ver Rubin 集群，我看到它运出时拍的照片了。Designing a building that might take 30 storage racks in a row versus one or two AI racks in that same row. Very very different design. Just think of it in terms of the size. Think of it in terms of the power and how that would be distributed across uh the building etc. Networking would be the same. The amount of networking that you would need for a storage rack tiny. I mean, if especially if it's hard drives compared to an AI rack. So, if you're trying to make it fully fungeible, you'll probably make it too big and too overbuilt. Fungeible over certainly a 30-year period. An AI data center likely is going to be much more purpose-built, co-designed with the hardware, even to the point of cooling and power distribution, uh, etc. So, a lot more co-optimization. You all delivered a big Ver Rubin cluster to one of my portfolio companies, Ineffable Intelligence, and I saw the photographs of it as it went out.

真是美不胜收。That is a thing of beauty.

主持人：是的。没错。Host: It is. It is.

嘉宾：这、这，我是说，这就是那种事情，你知道，类似于去看人类历史上最大的建筑项目。你看那座数据中心，就会觉得，哇，那真是一座纪念碑，彰显着人类能做到什么。不，我们拍了一张很棒的照片，而这只是几排机柜。你知道，它的布线、光纤配线，真的……我是说，美不美这种事，各人看法不同，但对我这样的人，还有你，以及观众中的其他一些人来说，这确实是一种美。嗯，我们把这张照片发到了社交媒体上，人们很喜欢……他们是……他们非常喜欢他们正在做的事情。一支很棒的团队，吸引了很多关注。好，很棒的机柜，适用于 Effable。不过，人们其实就是喜欢看那些光纤照片，以及光纤呈现出的那种分形结构。Guest: It's it's I mean, that's one of those things where, you know, similar to looking at the biggest construction projects in human history, you look at that data center, it's like, wow, that is a monu monument to what what mankind can do. No, we got a nice picture of it and it um and this is just you know a couple of racks. You know the the cabling the fiber distribution for it is is really be I mean you got beauty is not the beholder but for folks like me and probably yourself and others in the audience as well. Um it is a thing of beauty. We put this picture up on um social media and um people love the fact that ineffable by their huge fan of what what they're doing. Fantastic team that got a lot of attention. Okay, great rack for an effable. But actually, people just loved seeing the the pictures of the fiber and the sort of the fractal nature of the fiber.

这实际上是我们有史以来最受欢迎的帖子之一，嗯，主题就是这个。所以，嗯，是的，当然。我们对此真的非常非常兴奋。One of the most popular um posts that uh we've ever had actually uh was around that. So, so yeah, absolutely. We're really really excited about that.

主持人：是啊，我看到那个时都起鸡皮疙瘩了。Host: Yeah, I had goosebumps seeing the

关注有效吞吐量，而非 FLOPS：每小时都有故障发生时，如何做到自我问责Goodput, not FLOPS: holding yourself accountable when something fails every hour

图片。picture.

嘉宾：在这场大规模建设过程中，你们如何衡量自己的表现并对自己负责？嗯，节目开始前我们聊到，FLOPS 是个虚荣指标，而你们更喜欢另一种指标。能不能再多说一些？可以。我觉得有一点需要说明：无论是 FLOPS，还是你最喜欢的其他以芯片为中心的指标，理论上都是如此。换句话说，在某些条件下，对于你手头的任何一款芯片，这些指标代表的都是它所能达到的最大值。但归根结底，我们真正关心的是工作负载实际获得了怎样的性能。这才是我们关注的重点。性能很少由单颗芯片决定——虽然也有这种情况，但确实相对少见。FLOPS、HBM、SRAMM 容量等都非常重要，但关键也可能在于，如何将 2、4、8、16 颗，乃至 1,000 或 10,000 颗这样的芯片组合起来；不只是加速器——无论是 TPU 还是 GPU——还包括为它们提供数据的 CPU，以及连接所有这些部件的网络。Guest: How do you measure and hold yourselves accountable in this in the middle of this big buildout? Um we were talking right before the show about how flops is a vanity metric and you prefer an alternate metric. Can you can you say more? Yeah. So I think uh one thing to um note is that you know whether it's flops or pick your other favorite uh chip centric uh uh metric these are in theory. In other words under some conditions for whatever chip you have this is the maximum amount of flops that you can deliver. What we really care about in the end is what's the performance delivered by workload. That's that's what we're looking at. And rarely it it happens but actually relatively rarely is that performance determined by a single chip. It's flops, it's HBM, it's SRAMM uh capacity etc. Those are all super important but it might be how 2 4 8 16 a thousand 10,000 more of these chips composed together with not just the accelerators again whether TPUs or GPUs but then the CPUs that might feed them the data the network that connects them all together.

那么你运行的工作负载是什么？它的性能如何？一个值得关注的指标是 FLOP 利用率。比如，对于某个特定工作负载，如果理论上你的工作负载能够达到每秒万亿次或千万亿次浮点运算，那么你实际发挥出了其中的多少？So what is the workload that you're running and what is the performance of that workload? One measure that is interesting is what's your flop's utilization. So for example for a particular workload if you have in theory uh teraflop or a pedlop capable for your workload what fraction of that are you delivering?

主持人：这就是对有效吞吐量的衡量。Host: Now that's a measure of goodput.

主持人：什么是有效吞吐量？Host: What is goodput?

嘉宾：所以你可以把它理解为一个众所周知的术语——吞吐量。没错，Guest: So you can think of throughput which is a well-known term. Yep,

主持人：这就是可能实现的吞吐量。不过现在我们来考虑一些其他因素。其中一个是工作负载会减慢多少。这同样是工作负载本身固有的特性，但还有其他方面会严重影响我们的可靠性。说到我们如何对自己负责，如果有芯片在同步工作负载中发生故障——而许多这类工作负载，无论是训练、服务还是智能体工作负载，都是同步的——就会有非常非常多的组件同时协同工作。假设有一千、一万、十万个这样的组件同时工作，而且它们需要以微秒或毫秒为粒度进行协调。只要其中一个组件发生故障，就可能导致整个系统停摆，对吧？因为每个组件都指望其他组件各司其职，才能共同解答一个非常棘手的问题。Host: that's the throughput that is possible. But now let's consider some uh other considerations. One is what is the slowdown of the workload? Again, just inherent to the workload, but other aspects of it that really hits our reliability. So in terms of how we hold ourselves accountable, if we have a chip failing for a synchronous workload and many of these workloads, whether training or serving or agentic workloads, they're synchronous. There's many many components that are working together simultaneously. Now if you have let's say a thousand 10,000 100 thousand of these components working together simultaneously and they're really needing to coordinate uh at microscond or millisecond granularity. One of them fails it might actually bring the whole thing to a stop right because everyone is counting on everyone else to do their part of the job in order to come up with the answer to a really tough question.

其中一个停了。好，现在我们得弄清楚发生了什么，究竟是哪一个停了。我们有没有某个检查点，记录了计算在之前某个时刻的状态？我们该怎么恢复那个检查点？又该怎么重新启动？最糟糕的情况是，我们得从头开始。那就太糟了。不过在某些情况下确实可能发生，尤其是在推理这一侧。关键是，如果你现在不得不回头重做一大堆计算，或者不得不停下来等着弄清楚故障发生了什么，那些工作就只是白费功夫，并没有真正帮你得出答案，对吧？打个比方，假设你在纸上解一道题：第一步、第二步、第三步、第四步。如果你因为犯了错而不得不回到第一步，是的，你仍然是在做功。One of them stops. Okay now we have to figure out what happened which one stopped. What's a checkpoint that we have of the computation at some previous state in time? How do we restore that checkpoint? How do we restart? Worst case, we have to restart from the beginning. That would be really bad. But that that can happen in certain cases, especially on more on the inference side. The point here is that if you now then have to go back and redo a bunch of computation, if you then have to pause and wait to figure out what happened for the failure, all that work is work. It's not actually helping you get the answer. Right? the in terms of let's say that you're working out a problem on paper. Step one, step two, step three, step four. If you have to go back to step one because you you made a mistake, yes, you're still doing work.

这就是吞吐量。你确实在做工作，但交付答案的有效吞吐量是多少？基本上就是你解决问题总共花了多长时间。这才是重要的。如果出现故障、需要故障恢复，或者有任何事情打断了你的工作，这些都属于问题的一部分。那么，我们该如何让自己承担责任？看的是实际交付的有效吞吐量，而不是理论基准下的吞吐量或理论上的性能；而是要看在真实的工作负载和真实的故障条件下，实际表现如何？遗憾的是，在拥有 10 万个加速器的规模下，……That's throughput. You're doing work, but what is the goodput in delivering your answer? It's basically the total amount of time it took you to solve the problem. That's that's what matters. And if you have failures, if you have failure recovery, if you have whatever it is that's interrupting your work, that's that's part of the issue. So now, how do we hold ourselves accountable? It's delivered goodput, not theoretical benchmark throughput or goodness in theory, but actually for a workload for real failure conditions, what's happening? And the unfortunate reality is that at a 100,000 accelerator scale,

主持人：我本来想说，在那种规模下，总有什么东西一直在填充，对吧？Host: I was going to say at that scale, something's filling all the time, right?

嘉宾：一直都是。而且，而且你知道，这些芯片中的每一颗都是大自然的奇迹。我的意思是，它们处于制造工艺所能达到的最前沿，对吧？所以，再说一次，在我看来这令人惊叹。而且现在这些芯片实际上并不只是一颗芯片，而是封装体，很多情况下由两颗、四颗、八颗，甚至更多小芯片组成，对吧？然后当然还有旁边的 HPM、网络连接，也许还有共封装光学元件之类的东西。这不是批评。很多事情都可能出问题。现在，如果你有十万个这样的芯片，总会有东西出故障。你必须对此有所准备，在近乎实时的情况下发现问题，并在近乎实时的情况下恢复。要做到这一切，遥测问题确实非常棘手。就像要持续不断地在干草堆里找一根针——当然，时间尺度是几秒钟、几分钟。Guest: All the time. and and these each one of these as uh you're aware each one of these chips is a u wonder of nature. I mean that they're at the very bleeding edge of what's possible to manufacture, right? And so again to to me it's stunning. And now actually these chips aren't just one chip, they're packages made up of in many cases two, four, eight, maybe more chiplets, right, that are composing together. And then of course HPM off to the side, network connectivity, maybe things like co-ackaged optics. No criticism. Lots of things can fail. And now if you got 100,000 of them, something is going to fail. You have to be prepared for that. Detect that in near real time. Recover from that in near real time. Doing all that it's it's really like finding the the telemetry problem is massive. It's like finding a needle in the haststack continuously across certainly seconds and minutes.

但对于其中一些作业来说，运行时间可能长达数小时、数天，甚至数周，几乎是持续在线运行。我们该如何衡量自己的表现？关键在于我们为数据中心里真正重要的工作负载交付了多少有效吞吐量（goodput）。goodput 是 Google 的术语，还是行业术语？But for some of these jobs, hours and days and even weeks like just continuous and online. How we hold ourselves accountable is what's the goodut that we deliver for the workloads that actually matter in the data center. Is goodput a Google term or is it an industry term?

主持人：这是谷歌的一个术语，但我觉得整个行业里越来越多的人开始使用它了。Host: It's a Google term but I think more and more people across the industry are starting to pick it up.

嘉宾：然后只是想让我有个概念，10 万个加速器的规模，我们说的是每分钟会故障一次、每天会故障一次，还是怎样？到底是怎么样的？Guest: And then just to calibrate me 100,000 accelerator scale are we talking it's going to fail once a minute once a once a day like how how

主持人：10万个加速器。让我想想，我认为肯定会达到这样的规模：每天会发生多次故障；而且，具体取决于配置，可能每小时都会发生多次故障。那么，最常见的故障原因是什么？Host: 100,000 accelerators. So let me see I think that it is definitely going to be at the scale multiple times a day and perhaps depending on the exact configuration multiple times an hour uh something is going to fail. And what are the most common failure reasons?

嘉宾：问题就在这里。其实这是个非常好的问题。嗯，如果有一种常见的故障原因，我们早就找出来并解决了。这是一个不断探索、发现问题的漫长过程。你知道，每当我们推出一款新产品，总会遇到一些问题。坦白说，很多时候，毕竟这是最前沿的技术，问题可能与网络有关，也可能与我们如何以超高速将这些设备连接起来有关，还可能与硬件有关。这些问题我们都会逐一解决。不过，很多问题也可能出在软件上。这也是我们要求自己承担责任的另一个方面。芯片或许具备一定的浮点运算能力，但如果编译器有漏洞、运行时有问题、模型出了问题，或者操作系统有问题，那一切都无关紧要。Guest: This is the problem. Actually, it's a very good question. Uh if there were a common failure reason, we'd be we'd have figured it out and and fixed it. It is a long tale of constant discovery. You know, whenever we have a new product that's being introduced, there's going to be something that hits us. Um in many cases, frankly, because again, it's at the very bleeding edge, it might be network related. It might be how we connect these things together at super high speed. that might be something to do with the hardware. Those things we work through. Uh but then a lot of the issues could be software. So this is the other aspect of how we hold ourselves accountable. Again, the chip might be capable of a certain level of flops. But if you have a compiler bug, a runtime bug, a model issue, something else, operating system issue, it doesn't matter.

这会影响天线系统的性能。所以你可能拥有完美且完全可靠的硬件，但软件问题可能会拖累你。That that's going to impact the antenn system performance. So you you might have perfect hardware fully reliable but then you might have software issues uh that hurt you.

每六个月将 token 容量翻一番，以及这些提升究竟来自何处Doubling token capacity every six months, and where the gains actually come from

主持人：英伟达等加速器公司，或者你们这边的 TPU 团队，是否会提供一套标准的参考技术栈，告诉你们围绕加速器构建的最佳系统是什么，只要按照这套系统来搭建就没问题？还是说，除了半导体公司提供的方案之外，你们还需要自行设计多少数据中心架构？嗯，没错。确实有一套参考技术栈。我会说，英伟达是一家非常出色的全系统公司。我的意思是，他们显然是一家半导体公司，但又不只是一家半导体公司。他们提供了一套非常非常强大的参考技术栈。不过，我们发现，很多客户会采用这套参考技术栈，但也有不少客户会针对自己的需求进行专门优化。Host: Is there a standard reference stack from the accelerator companies from Nvidia or from the TPU team of you know here is the optimal system to build around our accelerators and as long as you build to that system you're you're good or how much of your own kind of data center design do you have to do above and beyond what the what the semiconductor companies give you? Uh right. So there's a there's a reference stack and I mean I would say that uh Nvidia is a incredible uh whole systems uh company that I mean they're obviously semiconductor company but they're not just a semiconductor company. They give you a really really strong reference stack but I would say that uh many we find that um the way I'd put it is most of our uh customers leverage that reference stack but many also specialize.

换句话说，他们可能会发现，对自己的特定用例来说，这样做很自然；他们有优化空间，或者需要做些不同的事情，于是就会这么做。TPU 方面也一样，我们有一套参考技术栈。大多数人会利用这套技术栈，不过也有很多人会对它进行专门定制。So in other words, they might find that as as natural for their particular use case, they have an optimization opportunity or something different that they need to do and they're going to do that. Similarly for the TPU side, we have a reference stack. But then and most people will leverage that. Many will specialize it as well though.

嘉宾：明白了。那么就整体产能而言，我想你们告诉团队，Google 必须大约每六个月将服务能力提升一倍左右。是这样吗？ 嗯，先说明一下，我们说的是实际可用产能，最终是从服务的角度来衡量的，也就是生成 token 的能力。所以，产能是软硬件结合的结果。硬件可能有某种所谓的固有算力。我这里的意思不一定是说每六个月就要把算力翻一倍——这只是其中一种途径。你们必须每六个月将硬件生成 token 的能力提升一倍。而这在很大程度上，甚至更多，得靠软件而非硬件来实现。Guest: Got it. And then I guess in terms of overall capacity, you've told your teams that Google has to roughly double the serving capacity every six months or so. Is that right? Well, so to be clear, uh this is in terms of effective available capacity from the way in the end that we look at it is from a serving perspective. It's token generation capability. So capacity and this is where I would say it's a combination of software and hardware. The hardware might have some quote unquote inherent level of flops. What I'm saying here is not necessarily you have to double the number of flops every six months. That's one one path. You have to double the capability of that hardware to generate tokens every six months. And as much or more of that is going to come from software as it is from uh hardware.

换句话说，可能是某项模型优化带来了这种效果，也可能是某项运行时优化带来的。实际上，很可能会有几十项甚至几百项单独的优化接连不断地推出，才让这一切成为可能。但没错，容量提升的速度实在惊人。So in other words, it could be a model optimization that delivers that. It could be some runtime optimization that uh uh delivers that. It's really probably going to be dozens or hundreds of individual optimizations that are just landing again and again and again to make it all possible. But yes, the rate the rate of capacity improvement is incredible.

主持人：我想，数据中心建设已经持续了几年，模型和软件也都取得了几年的进展。那么，从经验数据来看，容量提升中有多少是以每瓦智能来衡量的？容量的增长有多少来自硅片本身，有多少来自模型，又有多少来自其他软件以及其他重要因素？ 嘉宾：是个非常好的问题。嗯，我没有准确的细分数据，但根据我们的经验，每瓦智能方面的大部分收益都来自模型的改进。顺便说一句，每瓦智能是个非常好的指标。嗯，我们也会重点关注每瓦有效吞吐量；至于为什么要以瓦数作为分母，我们之后可以再谈。有效吞吐量完全可以用来衡量每瓦所交付的智能。Host: I I guess we've had a few years of data center buildout now and a few years of model progress and software progress. What has been the empirical breakdown of how much um capacity me as measured by intelligence per watt? How much capacity increase has come from the silicon itself uh versus the models uh versus other software and and any other kind of big components? Yeah, it's a really good question. Um, I don't have the exact breakdown, but I would say that in our experience, most of the benefits come from model side improvements in terms of intelligence per watts. And by the way, the intelligence per watt is a fantastic metric. Uh, really focusing we we also for us it's um goodput per watts and we can come back to why the watts need to be the denominator. Goodput could well be a measure of the intelligence delivered per watt.

换句话说，goodput 是一种特定于工作负载的指标。不过我想说，大部分性能提升往往相当程度上来自模型侧的软件系统。为什么？因为它们能够确保你拥有的硬件得到有效利用。而且硬件本身也相当惊人。换句话说，我们现在生活在这样一个时代：每年性能提升两倍甚至更多，完全是有可能的。所以硬件确实能支撑这种提升。你要愿意的话，可以把它看作一个免费的倍增器。当然也不是免费，但它上面的所有其他环节都可以指望它发挥倍增作用，让所有其他环节的性能逐年提升。In other words, goodput is a workload specific metric. But I would say most uh of the gains often come uh come from the model side software system um quite a bit. Why? Because they're able to actually make sure that the hardware that you have is being used effectively. And then the hardware, it it is pretty stunning. In other words, we're living in a world right now where 2x or more year-over-year performance improvements is absolutely possible. And so the hardware does does support it. And it it is a if you want it's a free multiplier. Not free but um everyone else above it can count on it in terms of that being a multiplier that lifts everyone else up uh year-over-year.

TPU 之赌：从 2013 年的逆势判断到将 8i 和 8t 拆分开来The TPU bet: from a contrarian call in 2013 to splitting 8i and 8t

主持人：我想聊聊 TPU 项目和协同设计。谷歌十多年前就开始打造定制芯片了。当时这是个逆势之举。嗯，TPU 项目这些年来是如何发展的？Host: I'd love to talk a bit about the TPU program and and codeesign. Uh so Google started building custom silicon more than a decade ago. It was contrarian call at the time. Um how has the TPU program evolved?

嘉宾：相当多。我的意思是，这个项目在 2013 年启动时，确实是个逆势判断。我觉得，当时要让自己回到 2013 年的那个时刻并不容易。那时的主流观点是，你知道，所有最聪明、最有见识的人都会说，你不会为单一工作负载打造定制加速器。为什么？因为摩尔定律摆在那里，性能每隔——不管是 18 个月还是 24 个月——就会翻一番；而且你可以利用标准的编程模型，所有那些 C++ 代码等等，还有 Java、Python，不管是什么，就像芯片领域的“苦涩教训”一样。Guest: Quite a lot. I mean when the program started 2013 it was really a contrarian call and I mean I think that at the time it's hard to put yourself back into the moment 2013 conventional wisdom you know all the smartest wisest people would say you don't build a customuilt accelerator for a single workload why because Moore's law is there doubling of performance every whatever it is 18 or 24 months is there uh you get to leverage standard programming models all your C++ code etc java python whatever it might be like the bitter lesson of chips.

主持人：对，没错。芯片领域的苦涩教训就像是说，嗯，专业化永远不会赢。但在这个特定案例中，我们有一个应用，或者说少数几个应用，能从中获益巨大，而要支持它们则需要难以想象数量的通用 CPU。所以说，2013 年这确实是一场豪赌。我想，公司内部也有不少人不确定这事能不能成。这确实是场赌注，结果证明这是一次极其成功的豪赌。第一款芯片完全是为推理而设计的。嗯，第二款芯片则是：嘿，实际上我们可以沿用同样的思路，打造一款训练芯片。从那以后，它被应用到了越来越多的场景中。大约在第二款芯片问世的时候，Transformer 诞生了。我的意思是，这是一个非常非常重要的时刻，而且它彻底改变了 TPU 项目的方向。Host: Yes, exactly. It's like the bitter lesson of chips is that um specialization never wins. But in in this particular case, it was we have this one application uh or or a few a small number of them that would benefit tremendously and that would require an unimaginable amount of general purpose CPU to support. So really in 2013 it it was a bet and there were quite a few people even within the company I would say who were um not sure that it would work out. So it was a bet. It turned out to be a massively successful bet. So the first chip was uh all all about inference. Um the second chip was hey actually we can take the same idea and build a training chip. From from there it got picked up for uh more and more use cases. Around the time the second chip came out transformers were invented. I mean this this was a major major moment and that actually completely shifted the TPU program.

呃，我们当然也了解到，推荐系统在 TPU 上能够运行得非常非常好，于是广告以及相关用例也随之出现。所以我想说，这种变化和演进体现为范围和影响力的不断扩大。换句话说，最初是几个用于推理服务的应用场景，影响力非常大，主要是语言翻译和语音识别；之后发展到训练，再到 Transformer、推荐系统，并且随着我想说的生成式 AI 浪潮席卷而来，继续推广到规模越来越大、可扩展性越来越强的系统。uh we of course learned that recommener systems could run really really well on the uh TPUs and so ads and related uh use cases came along. So I would say that change and evolution has been one of expansion in scope and impact. In other words, it started with a really impactful use case of a couple of applications for inference serving. This was language translation uh primarily and voice recognition to then training to then transformers recommener systems and continuing to generalize as I guess the um genai moment took a hold to ever larger ever more scalable systems as well.

嘉宾：你提到 Transformer 出现是一个重要时刻。我想进一步聊聊，TPU 是一种特定架构，但并不是专门针对 Transformer 的。你们如何把握这个微妙的平衡：芯片要针对工作负载做到多专用？Guest: You mentioned moment the transformer came through being a a major moment. I'm curious to explore the TPU was like a specific architecture but it wasn't you know specific to the transformer specific. Um how do you kind of straddle the fine line of you know how specific you want your chip to be for the workload?

主持人：这是个很好的问题，我认为归根结底要看芯片对特定工作负载的适用性。也就是说，嗯，我们一直在考虑每一代产品是否要进一步专门化。比如说，两年多前我们面临的问题是：到了 2026 年，我们应该推出两款芯片，还是一款？我们可以推出一款芯片，让它同时支持推理和训练，而且两方面都做得相当好；也可以推出两款芯片，一款进一步针对推理优化，另一款进一步针对训练优化。今年发布两款芯片，就是这番分析和工作的结果。Host: This is a great question and I think that it really comes down to the applicability. I mean I think that um uh of of the chip to a particular workload. So in other words in the end we've been thinking about this for um every generation. Would we further specialize? The question that we were facing let's say um two years ago or a little bit more was do in 2026 should we have two chips or one? We could have one chip that could do uh inference and training simultaneously and do both quite well or we could have two chips. One that was further specialized for inference and a second one that was further specialized for training. This analysis and this work led to the release of two chips this year.

8I 用于推理，8T 用于训练。最终我们意识到，几年前情况未必如此，但到 2026 年，我们看到推理和服务真正开始起飞。因此，相比通用芯片，我们认为如果有一款芯片在服务方面显著更快，而且在其生命周期内可能占据 30%、40%、50% 甚至 60% 的市场份额，那么对我们来说，开发这样的芯片就很有意义。反过来说，如果我们预计推理只占市场的 2% 或 5%，那么即使这种专用芯片快 2 倍，也未必划算，对吧？因为你实际上会选择那款没有完全优化、但用途更广的通用芯片。不过这样也没关系，因为你现在获得了统一性，等等。所以说到底，这确实是一个权衡问题：好，我们要不要进一步专门化？这类工作负载有多大？预计两三四年后，这类工作负载会有多大？8 I for inference 8 T for training and in the end what we realized is that it wasn't the case necessarily a couple years ago but by 26 we saw inference and serving really taking off and so having a chip that would be significantly faster for serving that we thought might be 30 40 50 60% of the market in its lifetime started making a lot of sense for us relative to if if inference if we projected inference to be 2% or 5% of the market even if that specialized chip is let's say 2x faster it might not make sense right because you'd actually go for the one general purpose chip that's not fully optimized but that that's okay because you now have the uniformity etc. So really uh it does come down to a calculus question of okay would you further specialize? How big is that workload? How big is that workload projected to be in a couple of three four years?

这是针对硬件中特定工作负载的持续增长吗？再说一次，你也知道，针对某种特定工作负载进行优化时，优化程度越高，灵活性就越低，但硬件的速度会更快，能效也会更高。所以这既是一门艺术，也需要预测你要针对什么进行设计，以及该工作负载能持续多久。换句话说，如果它一个月、两个月或三个月后就消失了，那么即使它在这三个月里规模很大，你能抓住它的窗口也非常短。因此，它必须有一定的持续性。你还必须能够准确预测，针对它进行优化究竟能带来什么收益。大致的权衡是：支持一个新项目需要承担一笔很大的固定成本。Is this a um sustained growth for a particular workload in hardware? Again as you know there is this opportunity where the more you specialize to a particular workload the less flexible it is the faster the more power efficient the hardware is going to be. So it is this art and it's this projection of what are you designing to and how persistent is that workload. In other words, if it's going to go away after a month or two months or three months, even if it's big for those three months, you got a really narrow window to intercept it. So it has to be somewhat durable. Uh and you have to be able to project exactly what win you can get for specializing to it. So the rough trade-off is there's a fix a large fixed cost to support a new program.

嘉宾：是的。Guest: Yes.

主持人：基本上，你必须认为这个特定项目会有足够的需求，才能证明这笔成本是合理的。Host: And you basically have to think that there's going to be enough demand for that specific program to justify the cost.

嘉宾：没错。而且在某种程度上，比如我们的 8i 和 8t 芯片，两种芯片都能处理另一种工作负载。这一点也很关键。如果 8i 只能做推理，完全不能做训练，也就是说，它在训练方面的性能为零；反过来也一样。如果它在训练方面表现出色，但在推理方面的性能为零，那也会是一个关键限制。为什么？因为在硬件六年的使用寿命里，我们就得提前准确预测每种芯片各需要多少。在这种情况下，好处是两种芯片在各自擅长的领域表现更好。但如果有需要，或者某一边还有剩余算力，两种芯片实际上都能承担另一种芯片的工作。不过，取决于你采用的专业化程度，你可能会让自己过度专门化，以至于失去灵活性和可互换性。Guest: Exactly. And uh to some to some extent, you know, for example, for our 8i and 8t chips, both chips can do the other workload. This is also key. If 8i could only do inference and it couldn't do training at all. In other words, it had zero performance for training. And then vice versa. If it was amazing at training and had zero performance for inference, that would have also been a key limitation. Why? Because we would have to predict ahead of time over a six-year period lifetime of the hardware exactly how much we need of each. In this case, it was nice because both chips are better at what they're specialized for. But if you needed to, if you had leftover capacity in one place or the other, both can actually do the other uh other's job. depending on the specialization you do, you might get so specialized that you actually limit yourself from being flexible, funible.

主持人：那么，粗略来说，另一端的权衡是不是在于程序的规模？在我看来，现代 AI 市场很大一部分似乎都基于 Transformer。所以我想提个有点挑衅性的问题：为什么不干脆把 Transformer 架构直接固化到芯片里？Host: And if the rough trade-off then is kind of size of program on the other end, like it seems to me that so much of the modern AI market is transformer-based. And so maybe just asked a provocative question, why not just burn the transformer architecture into the chip?

嘉宾：是的。所以我认为它是基于 Transformer 的。不过接下来要问的是，Transformer 归根结底涉及向量和矩阵乘法运算，以及 softmax。因此，这里面有一系列基本的线性代数运算。我们把这些运算以及其他一些运算大致固化到了硬件中。整个系统是基于 Transformer 的，但接下来就要看你的模型架构了：你有多少层？如何逐层处理？如何在各层之间传递？对于每个维度，你具体要使用什么形状的矩阵和向量？你还可以进一步专门针对的不只是 Transformer，而是你的模型。我认为这是下一个层次的专门化。有不少公司都在考虑这个方向，我觉得这也是一个非常非常有意思的发展方向。Guest: Yeah. So I think that uh it's transformer based but then uh the next level of question would be um transformers in the end are about vector and matrix multiply operations and you know a soft max. So there's a range of essentially linear algebra primitives. We have those and others do as well roughly baked into the hardware. It's all transformer based but then it's your model architecture is okay how many layers do you have? How do you go through the layers? How do you go across the layers? how for each dimension um what exact shape of matrices and vectors are you applying you you could go further and specialize to not just the transformer but your model that's the next level of specialization I think there are a number of companies out there that are thinking about that I think it's a very very interesting direction as well

主持人：好，那么目前你的客户是否认为 TPU 和 GPU 大致可以互换，还是说有些特定问题更适合其中一种？Host: okay so at this point in time do your customers view TPU and GPU as roughly fungeible equivalents or is is there a certain set of problems that is better suited for one or the other.

嘉宾：确实，有一类问题更适合其中一种方案。我是说，首先，GPU 的通用性比 TPU 更强，这一点很明确。谷歌和 Google Cloud 都有非常出色的产品。我们销售很多 GPU，内部也会使用 GPU。但我认为，最终还是要看具体问题。我想，我们的客户在这两种方案之间确实存在交集，但他们会评估自己的工作负载和可选方案。谷歌希望为客户提供选择。换句话说，我们希望为他们的需求找到合适的解决方案，当然，也会提供最能满足他们需求的方案。Guest: There's for sure a better set of problems that are better suited for one or the other. I mean uh GPUs are more general purpose than TPUs uh for for one. That that is clear. I there we uh there are incredible products at uh Google and Google Cloud. Uh we we sell a lot of GPUs. We use GPUs internally. But I think it really then comes down to the specifics of your problem. I so I think that our customers there there is overlap between the two but our customers then evaluate their workload and evaluate their options. What we like to do at uh Google is give our customers choice. In other words, we want to have the right solution for their needs and of course provide the solution that that best meets their needs for them.

支持和反对共同设计的理由The case for and against co-design

主持人：从芯片到网络再到软件，协同设计的价值是什么？反对协同设计的理由又是什么？是的，协同设计有巨大的优化空间。你可以想象，如果你有一套分层架构，并且希望能够自由选择任意组件——比如说，你要在多个云平台、许多种硬件和软件上运行——就可以设计一些抽象层，让系统能够在任何硬件、任何软件、任何网络拓扑上运行，不管网络资源有多少，都能轻松应对，而且系统很可能会完全自适应。这样一来，你就获得了强大的能力，可以迁移到任何地方；一旦有新的算力容量可用，一夜之间就能用上。Host: What is the case for code design from the chip to the network to the software and then what is the case against codeesign? Yeah. So there's huge optimization opportunities with codeesign. And so what you can imagine is that if you have a end layer stack and you want to be able to pick and choose whatever component you want, let let's say you're running across many clouds or you're running across many pieces of hardware, many pieces of software. You could design abstraction layers that would basically say I can run on any hardware. I can run on any software. I can run on any network topology um any amount of network that you give me no problem and my system is going to be fully adaptive very likely. So now you have a great um capability. You can move anywhere like new capacity becomes available overnight.

你之所以能启动并运行，是因为你把系统设计成了这样，确实能够利用任何东西。你没有把系统硬编码进去，也没有针对任何特定的基础设施做专门优化。缺点是，如果你变得完全通用，可能会白白损失很多性能；如果你要完全适配任何人的硬件、软件、网络、存储、计算等技术栈，情况也是如此。因此，进行协同设计的理由在于，如果你想实现完全的通用性，这些层与层之间会存在很大的阻抗失配。每一层都可能有 10%、20% 甚至 2 倍的优化空间。把这些优化机会一路相乘，最终就会形成巨大的端到端优化空间，无论你想用每瓦智能水平，还是每瓦有效吞吐量来衡量，都能从中受益，甚至还能一路优化到电力输送、电力可用性、软件优化等等。You're up and running because you've designed your system that way to actually be able to take advantage of anything. You have not hardcoded um or specialized at all to anyone's particular infrastructure. Downside is you probably leave a lot of performance on the table if you become fully funible. If you become fully flexible to anybody's hardware, software, network, storage, compute, etc. stack. So the case for code designing is um between each one of those layers um there's a big impedance mismatch if you try to get full generality. And so there might be 10% 20% 2x across each of these layers. You start multiplying those optimization opportunities through and all of a sudden you're left with a big endto-end opportunity in terms of whatever you want to say intelligence per watt or goodput per watt that you can leverage even again all the way down to power delivery uh and power availability software optimizations etc etc.

优点是，呃，你可以随时随地运行，不会被绑定。缺点是，呃，你会白白浪费相当可观的性能，而且很可能浪费的性能还不少。So pros are um you can run anywhere anytime you have no lock in. Cons are you're leaving um significant and in all likelihood significant amount of performance on the table.

嘉宾：所以我的理解是，如果你看 OpenAI 和 Anthropic，嗯，OpenAI 主要是在一个同构计算栈上构建，而 Anthropic 构建的计算栈则相对更加异构。你觉得这是不是他们的协同设计促使他们最终采用了据传截然不同的模型架构的部分原因？嗯，我不能……我不能推测。我不想推测 OpenAI 和 Anthropic 在做什么。这是一种可能性。不过我觉得，在不了解他们具体在做什么的情况下，Guest: So my understanding is that if you take an open AI and enthropic um OpenAI was kind of primarily building on a uh homogeneous comput stack and Enthropic was building on a roughly more heterogeneous comput stack. Do you think that's part of the reason that code design is part of the reason why they converged on what is rumored to be very different architectures for their models? Um I can't I can't speculate. I can't I don't want to speculate in terms of what open and anthropic are doing. It's is one one possibility. Uh but I think that without knowing the details of what they're doing,

主持人：我想，他们最终采用不同架构的原因可能有很多。Host: I would imagine that there could be many reasons for why they wind up with different architectures.

与 DeepMind 并肩作战：拦截空运中的芯片Shoulder to shoulder with DeepMind: intercepting chips mid-flight

嘉宾：那么，对于 Google 来说呢？我很好奇，你们团队和 DeepMind 之间的合作关系是什么样的。在模型开发的不同阶段，具体有哪些人参与其中？你们又是如何共同决定代码设计的？说实话，在 Google 工作，最有趣、也最令人欣慰的部分之一，就是有机会和 DeepMind 团队紧密合作，共同设计我们的硬件和模型。此外还有第三个方面：我们也能把这种合作延伸到消费服务和云服务上。我先暂且不谈这一点，之后可以再回来聊。不过，就 DeepMind 而言，这确实是一种深入的合作关系。Guest: Well, what about for Google then? I'm curious what the working relationship looks like between your team and DeepMind. Kind of who's in the room at what stage of model development and how are you making decisions together on code design? It's one of the most fun uh and frankly gratifying parts of uh being at uh uh Google is the opportunity to work really shoulder-to-shoulder uh with the deep mind team in terms of uh co-design of our hardware and models. Uh there's a third element to it in in terms of that we also get to extend that with the consumer services and cloud. I'll put that aside for a moment. I can come back to that. But with respect to deep mind, it it really is a deep partnership.

所以，我的意思是，回顾过去，我可以举些例子：他们提出了针对 Transformer 或某些特定运算的模型优化方案，而我们可能正好有一款芯片在研发中，还没有完全完成。然后他们会说：“天哪，如果硬件能支持这个，我们端到端的训练或推理服务可能会快很多、效率也会高很多。那我们现在要做些什么，才能修改我们可能要在其上运行的硬件定义来满足这个需求？”于是，我们的工程师和研究人员就会聚在同一个房间里，紧锣密鼓地讨论几天、一周，甚至两周，说：“好，我们能做到。更有可能的是，我们没法完全按你的想法来做，但可以换种方式实现。”然后，也许你可以朝另一个方向调整模型架构，这样就能达到我们预期效果的 90%——或者说 98%。So let me I mean from the past I can give you examples where uh they have come up with model optimizations um let's say to transformers or to particular math that they might want to do and we might have a chip in progress not quite done but then they say oh my gosh if we had hardware support for this our endto-end training or serving might get significantly faster more efficient what would it take to actually now change our hardware definition that we might be an execution on to accommodate. And this then leads to our engineers from and researchers to getting together intensely in the same room uh for a few days, a week, two weeks saying, "Okay, yes, we can do this. More likely, we couldn't quite do what you wanted, but we can do this other thing." And then maybe you can change your model architecture in this other direction that gives you 98% that gives us 90% of what we were looking for.

是的，然后我们就可以回过头来对硬件进行干预。或者我们可以说：“你知道吗，咱们把流片时间推迟一两周吧，因为哇，能带来这么大的收益，完全值得。”同样，在规划路线图、展望未来时，任何时候我们都有许多代芯片处于不同的研发阶段。我们有正在量产的芯片，这是一个阶段；有正在推进量产的芯片，它们已经从制造商那里返回，我们正在调试，让它们正常工作；有处于实现阶段、即将流片并送往制造商的芯片；有处于设计阶段的芯片；还有处于概念阶段的芯片。And yes, we can then go back and intercept the hardware. Andor we can say, you know what, we're going to delay the tape out by a week or two weeks because wow, for this level of benefit, it's it's totally worth it. You know, similarly, when we're um projecting our road map out and it's time for, you know, we we we have many generations of chips that are essentially in progress at any point in time. We have the chips that are in production. That's one. We have the chips that we're working on getting into production. They're already back from the manufacturer and we're debugging them, making them work. We have the chips that are in implementation that are about to tape out and go to the manufacturer. We have the chips that are in design phase and then we have the chips that are in concept phase.

所以这确实是一个五六个阶段、历时多年的流程，从投入生产一直到进入你的头脑，等等。与 DeepMind 在投入生产阶段合作意义重大，因为我们可以共同努力，最大限度地提高每瓦所交付的智能或有效吞吐量；我们清楚地知道模型中发生了什么，也清楚地知道硬件中发生了什么，以及两者之间的一切。我们还可以深入合作，参与那些即将流片的芯片。为什么？因为我们可以介入，并在芯片设计过程中对芯片做出改变，甚至改变芯片架构；如果我们是在公司之间合作，要做到这一点会很困难，甚至几乎不可能。倒不是不可能。对我们来说，要说“天哪，这颗芯片再过几周或几个月就要完成了，现在我们来挤进同一个房间，并肩坐在一起，看看是否应该打乱整个项目进度”，会困难得多，对吧？So it's really this five six stage many many year pipeline from in production to in your mind etc. The collaboration with deep mind for in production is significant because we get to work together in maximizing delivered intelligence or delivered goodput per watt and we know exactly what's happening in the model and we know exactly what's happening in the hardware and everything in between. We also get to collaborate deeply on though the chips that are actually just about to tape out. Why? Because we can intercept and we can make changes to the chip literally the chip architecture in flight which would be somewhere between hard and impossible to do if we were working across company boundaries. not not impossible. It would be much harder, right, for us to say, "Oh my gosh, we're a few weeks or a few months away from getting this chip done and now let's get in the same room shoulder-to-shoulder and figure out if we should disrupt the program."

这是有可能的，但我会说难度更大。当然，对于正在设计中的芯片，我们可以一起评估许多种架构。然后我们可以问问 DeepMind 的同事：你们觉得两三年后模型架构会朝什么方向发展？我们能做的事情有哪些？模型架构的发展方向又有哪些？实际上，我们拥有深厚而强大的仿真基础设施，可以预测工作负载将如何映射到不同的硬件架构上。我们能够深入、快速地迭代。团队并不是各自为政，而是常常在同一栋楼、同一个房间里工作。我们每天都有深入的交流。我每周都会和科里或 Demis 等人交流好几次。所以，这确实是这份工作中一个特别有趣的方面。It's possible, but but harder is what I would say. And then of course for the chips that are in design, we have many architectures we can evaluate together. We can then ask our colleagues in deep mind, where do you see model architectures going in 2 three years time? Here's the paro of things that we can do. Here's the paro of where model architecture is going. We have uh actually deep and significant simulation infrastructures that can predict how the workloads are going to map to uh different hardware architectures. Deep deep and fast iteration. It really isn't the teams are working separately. It's in in uh same building, same rooms many times. Deep daily interaction. I talk to whether that's Cororey or Demis multiple times a week uh etc. So it it really is a a super fun aspect of the work.

主持人：太棒了。最终，他们的模型还将帮助进行芯片设计。Host: That's awesome. And eventually their models will help with with chip design.

嘉宾：我们也在使用 Gemini 为未来的 Gemini 设计硬件。Guest: We're using Gemini to design uh hardware for future Geminis as well.

主持人：这真的很酷。稍后我想再聊聊合作本身。我想问的是，双方之间是否仍然存在某种不匹配？我想，和 DeepMind 的同事在软件方面相比，你们处理硬件时的迭代周期可能要慢得多；而且我猜，你们的规划周期也要提前得多。Host: That's really cool. I want to come back to that a little bit later on the collaboration itself. I guess is there an impedance mismatch still of just I figure the cycle times are probably just slower when you when you're dealing with hardware than than than your deep mind folks get to deal with on the software side and I figure your planning cycles are much further ahead

嘉宾：当然。Guest: for sure.

主持人：嗯，那你实际上有多大的调整空间？Host: Um and so how much room do you actually have to adjust?

嘉宾：是的，这是个非常好的问题。我们确实会提前两三四五年规划硬件。毫无疑问。换句话说，我们刚才谈到了 TP8 high，以及我们刚刚宣布的内容，但你可以想象，TP9、TP10，也许还有其他一些东西，正在朝着其他方向进行概念设计和开发。所以它们可能还要几年才能实现。如果你默认是围绕模型架构开展工作，就不会考虑几年之后的事。Guest: Yeah, it's it's a really good question. I we are planning hardware two three four five years in advance. No, no question. In other words, we talked about TP8 high and at uh just now that we announced, but you can imagine that uh 9 10 and maybe some others are are in in uh concept execution to to something else. And so they and they might be um years out if you're working with model architecture by default you're not going to be thinking years out.

主持人：是的。但我认为，这也是公司共同成长的一个很棒的方面。你知道，Google Research、DeepMind 和 Transformer 的发明，这些事情都发生在彼此交叠的圈子里。换句话说，有整整一代研究人员已经习惯了能够影响硬件设计，也知道硬件的迭代周期是按多年计算的。他们还明白，如果他们有个小改动，能让一颗即将流片的芯片提升 1%、0.5% 之类的，他们大概不会来找我们，因为他们很清楚，这不像软件——你不能只提交一个变更清单，然后指望它两周后就上线。Host: Yeah. But I think this is also the great thing about how the company has grown up together and you know there was words Google research and deep mind and transformers being invented all of this was happening in in these overlapping rooms as well. So in other words, there's a whole generation of researchers who've grown accustomed to being able to influence the hardware and knowing that the hardware is operating on multi-year cycles and they also know that look, if they have a small tweak that's going to deliver like 1% or.5% or something like that for a chip that's about to tape out, they're probably not going to come to us because they they they know enough to know that actually it's not like software where you can just do a change list and it's going to ship to production in two weeks time.

比如，真正完成一次流片是件大事，但他们也知道，如果遇上一个很大的、特别好的机会。是的，我们绝对会一起努力，看看能不能把它纳入其中。所以，嗯，正如我所说，收益的来源是乘数效应。硬件确实能带动各方面一起提升，对吧？DeepMind 团队中有相当一部分成员都在参与这件事，我们对此非常感激。这是一支了不起的团队，而其中相当一部分人都在思考如何影响路线图，因为这实际上是一条流水线。比如，我两年前提出的想法，现在已经投入生产，正在帮助 Google 的所有工作负载加速。这种感觉真好。like it actually stopping a tape out is a big deal, but they also know if they've got a big like a really good opportunity. Yes, we're absolutely going to work together to figure out if we can get it in there. So, um, as I said, it's multiplicative in terms of where the benefits come from. The hardware does lift all all the tides, right? And so, significant portions of the Deep Mind team and we're so grateful for it. It's an amazing team, but significant portions of it are thinking about how do I influence the road map because it's actually a pipeline. Like the idea I had two years ago, like it's in production now and it's helping all the workloads at Google go faster. Like that's a good feeling.

嘉宾：完全同意。不过，要预测五年后最常见的工作负载是什么、届时会出现哪些算法突破，听起来像是不可能完成的任务。呃，不过这似乎并不是不可能完成的任务。我明白你为什么会觉得这像是不可能完成的任务，但有件很棒的事是——事实上，我们正在非常非常详细地撰写这方面的内容，而且写起来很有趣。令人惊讶的是，从中等细节层面来看，而不是特别高的细节层面，TPU 架构自 TPU v1 以来基本上就没有变过。可以这样理解：CPU 的指令集架构，比如有加载、存储、加法、减法和分支指令。那么在这段时间里，运行在它之上的软件又发生了什么变化呢？Guest: Totally. It seems like an impossible task though to predict in five years what workloads will be most common and what algorithmic breakthroughs will have occurred. Uh doesn't seem like an impossible task. I uh hear you that it would seem like an impossible task, but here's the awesome thing and actually we're uh working on uh writing this in great great detail and it's a lot of fun to do it. The stunning thing is that um the TPU architecture at a medium level of detail, not at a super high level of detail, at a medium level of detail hasn't really changed since TPU v1. Like one way to look at it is the instruction set architecture for a CPU. like you've got loads and you've got stores and you've got ads and you've got subtracts and branches and like and yet what has the software on top of it done right over over that period of time.

TPU 也是一样。我们有一些基础指令和基础原语。当然，我们对它进行了扩展。并不是说指令集完全没有变化，但其中的基础原语包括专门用于数值计算的大型矩阵乘法单元、负责向量运算以及散布和收集操作等的稀疏核心。真正定义了这一切的有五六项功能：加载、远程加载、存储。实际上，那也是其中一项。我们可以通过 ICI 网络读写远程内存。这些基础功能一直都在，并且不断扩展，适用于多代模型，甚至多代深度神经网络算法和模型结构。Same thing with TPUs. We have some fundamental instructions and fundamental primitives. Of course, we've extended it. It's not like the instruction set hasn't gotten changed at all, but the primitives there in terms of specializing the numeric very large matrix multiply units um a uh sparse core that manages uh vector operations and scatter gather operations etc. There's five or six things that really define a load remote load store. That's another one actually. We can read and write uh remote memory through our ICI network. the fundamentals have been there and they've extended just to many generations of models, many generations of even deep neural network algorithms and model structures.

长时程智能体正在改变数据中心的形态Long-horizon agents change the shape of the data center

主持人：好，说到工作负载的转变，过去一年左右，工作负载最大的变化之一似乎是长时程智能体的兴起。我觉得这真正是从今年年初，也就是这个自然年的年初开始的。Host: Okay, speaking of shifting workloads, it seems like one of the biggest changes in workloads over the last year or so. I think it really started at the beginning of this year, this calendar year, was kind of the rise of the long horizon agent.

主持人：是的。我猜这和过去几年那种快速交互式的 LLM 对话相比，工作负载的形态非常不同。嗯，这对数据中心的需求意味着什么？Host: Yes. And I would guess that's a very different shape of workload than kind of the quick turn LLM conversations of of years past. Um what does that mean in terms of data center needs?

嘉宾：是的。所以我觉得，这件事有两个很重要的方面。其一，交互加速这件事不再受人类限制，嗯，你想怎么说都行。换句话说，当你在网页浏览器或手机上输入提示词时，当然，系统会根据你的提示词进行一系列处理；但等回复返回后，你还得读一读，想一想，然后也许再追问一句——这中间会花上好几秒钟。现在，在 Deep Horizon Agents 中，交互过程中没有人类参与，自然也就不会限制请求发送给模型的速度，对吧？所以，原本可能需要几秒甚至几十秒的交互时间，现在或许能缩短到毫秒级。也就是说，我一收到回复，就能解析它，稍微推理一下，然后确定下一条请求是什么。Guest: Yeah. So I think one there's two huge aspects to this. One is it's no longer human to um whatever you want to say accelerate interaction. In other words, when you are typing at uh prompt on a on a web browser or on your phone or whatever, uh, of course, in response to your prompt, there's going to be a bunch of work that happens, but then when the response comes back, you've got to read it. You've got to think about it, and then maybe you have a follow-up that's going to be multiple seconds of interaction time. Now, in Deep Horizon Agents, it's not there's no human in the loop that is going to naturally rate limits how quickly requests are going to go to the model, right? So this what went from you know seconds maybe tens of seconds in terms of inter interaction time is now going into perhaps milliseconds right as soon as I get a response back I can parse it I can reason about it perhaps a bit and I can figure out what my next request is going to be.

这就是第一个重大变化。第二个重大变化是，所有这些推理和解析很可能都会在 CPU 上进行。然后这个 CPU 可能还得考虑：好吧，在向模型发送下一条提示之前，我还需要收集哪些其他状态？换句话说，我从这个响应中学到了一些东西，接下来要再迈出一步，但实际上我需要先去获取一些上下文。也许是从本地的 DRAM 中获取，也许是从另一颗 CPU 上别人的 DRM 中获取，也许是从别处的 SSD 或 HDD 中获取。所以现在还必须进行大量的编排。因此，整个设计实际上正在发生相当大的变化：对加速计算的需求正在上升，而对 CPU、网络和存储——也就是传统数据中心里的 CPU 等资源——的需求也在急剧增长。So that's big change one. Big change two is all that reasoning and all that parsing is going to probably happen on a CPU. And that CPU is then going to have to probably think about, okay, what other state do I need to go gather before making my next prompt back into the model? Like in other words, I' I've learned something from this response. I'm going to take another step, but I actually need to go grab some context. maybe from uh DRAM local to me, maybe from someone else's DRM on another CPU, maybe on SSD or maybe on HDD somewhere else. So now uh huge amount of orchestration has to take place as well. So the design actually is changing pretty significantly where the demand for accelerated compute is going up but the demand for CPU and networking and storage the you know traditional data center CPU etc is also going through the through the roof.

呃Uh

主持人：这是否意味着你要在 GPU 机架旁边再放更多 CPU？所以，是的，这是 GPU/TPU 机架。这是个关键问题，回到优化和专用化这个问题上来。如果我们开始放置大量 CPU 机架，而问题在于，它们要放在 GPU 和 TPU 旁边，那就意味着，实际上我们无法完全针对 TPU 机架的密度和网络需求进行专门设计——相对于 CPU 机架而言。TPU 机架会比 CPU 机架更密集，可能也需要比 CPU 机架更多的网络连接。换句话说，我们的建筑设计现在也得改变。另一种选择是保持统一性：把 TPU 或 GPU 都放在一栋楼里，旁边那栋楼则可以放 CPU；而且，顺便说一句，硬盘可能还得放到另一栋楼里，因为它们还有另一套需求。Host: does that mean you're putting more CPUs alongside your G your GPU racks then? So this is yeah GPU TP racks. This is the key question is um going back to this question of optimization and uh specialization. If we start putting lots of CPU racks and it is a question uh next to the GPUs and TPUs that means that actually we can't fully specialize to the density and network requirements let's say of a TPU rack relative to a CPU rack. TP rack is going to be more dense than a CPU rack. It's probably going to need more networking than a CPU rack. So in other words, now our building design is going to change. Another option is maintain your uniformity. Put your TPUs or your GPUs all in one building and the building next door, but maybe put your CPUs and your maybe your um and maybe by the way the hard drives have to be in another building on the other side because they have yet another set of requirements.

现在，你需要在这些建筑之间实现相当大规模的网络互连。一旦网络连接跨出单栋建筑，从可靠性和成本的角度来看，网络复杂度就会显著提高。延迟也会增加，不过可能仍在可接受范围内；但由于各组件之间的排队，延迟可能达到数百微秒，甚至更高。因此，需要考虑的因素会以一些非常有意思的方式发生变化。Now you need networking between these buildings at a pretty significant level. So once you leave a building the networking complexity goes up significantly from a reliability perspective from a cost perspective latency goes up probably acceptably but uh still now you might go to hundreds of microsconds potentially more with queuing between the components. So the considerations do change in pretty interesting ways.

客人：你的工作很辛苦。Guest: Your job is hard.

主持人：真有趣。Host: That's fun.

嘉宾：挺有意思的。是啊。Guest: It's fun. Yeah.

光路交换与网络技术的现状Optical circuit switching and the state of networking

嗯，网络方面的情况怎么样？我听说 Google 一直走在最新网络技术的前沿，包括光网络。你能谈谈光网络目前的发展状况吗？大概在 15、16 年前，我们 Google 就是最早采用波分复用技术的公司之一。通过这项技术，可以在数据中心的一根光纤上承载多个信号，我们实际上也利用它来实现机架之间的所有通信。与此同时，我们还推出了一项与波分复用配套的技术，叫作光电路交换。与传统的电分组交换不同，光电路交换的本质是完全在光域中传输和移动数据。um what's happening on the networking side and uh I've heard that Google's always been the at the forefront of the newest in networking including optical. Uh can you say a word on what is like this the state of of optical networking? You know, we at um Google, this was probably 15 16 years ago, were among the first to bring um essentially uh what's called wave division multipplexing um where you could put u multiple signals on a single fiber within the data center and we actually leveraged that for all of our uh communication between racks. At the same time, we introduced a technology uh along with the wave division multiplexing called optical circuit switching. And essentially what optical circuit switching does is in contrast with traditional electrical packet switching, what it does is it transmits and moves data entirely in the optical domain.

它的妙处就在这里。接下来，我来介绍一下支撑它的技术——分组交换机。你会接收一个带有报头的分组，在电域中查看它，弄清楚报头里写着这个分组要去哪里。它可能包含一个 IP 地址，告诉你：“好，我该把它发到哪里？”你查一下，针对这个目的地，在一张表里查出应该从哪些端口转发出去。基本上，每秒都会有数十亿、数十亿乃至数万亿个分组以极高的速度通过，你要把它们转发出去。而光路交换的意思是：我不会在电域中处理这些比特。我要做的是确定，对于某个输入端口，应该把光发送到哪个输出端口。好，现在有多种方法可以实现这一点。The great thing about that is and so let me describe the technology that underpins it in a in a packet switch. You would take a packet that has a header. You would look at it um in the electrical domain. Figure out where the packet is headed in its header. It might have an IP address that says, "Okay, where do I head it?" You look up, okay, for that destination, you look up in a table, what ports do I forward it along? So basically, billions, billions and billions, perhaps trillions of packets coming through at super high speed per second. You're forwarding them along. Optical circuit switching says I'm not touching these bits in the electrical domain. What I'm going to do is I'm going to figure out for an input port which output port to send the light to. Okay. And now I have uh there's multiple ways to do this.

我们最初采用的这种设备叫作 MEMS 开关，也就是微机电系统开关，它们基本上是通过控制三维空间中的镜子来工作的。现在，我们可以通过编程来配置一个设备，比如说有 128 个端口、256 个端口，或者其他数量的端口。我们可以为每个接入设备的光纤输入端口进行配置，将其映射到一个输出端口，然后调整镜子的旋转角度，让光线照射到镜子上，再反射到正确的输出端口。 最初我们这么做有两个目的，其中之一是让不同机架组之间形成更直接的连接。假设我有一个计算集群和一个存储集群，它们都是为了支持——还是拿搜索服务来举例吧。我们知道这两个集群之间会有大量通信。因此，我们会配置这些镜子，在这两个机架集群之间纯粹通过光学方式建立捷径。The one that we started with is called MEMS uh switches, micro electrical motors that basically control mirrors in 3D. So we can now programmatically take a box that might have you pick 128 ports, 256 ports, some number like that. And we can configure every input port for a fiber that comes into it, map it to an output port, and then change the rotation of mirrors where the light literally shines down on these mirrors and gets reflected to the right output port. Initially, the reason we did this twofold was to create locality between groups of racks. Let's say I had a compute cluster and a storage cluster and both of them were in support of again making it up search. We knew that these two clusters would talk to each other a lot. So we would configure the mirrors to create shortcuts purely optically between those two clusters, clusters of racks.

这就是第一个原因。第二个原因是，我们希望能够扩展和收缩网络，而不必实际移动任何光纤。不展开细节的话，我可以在白板上画给你看。光路交换机可以让你重新配置网络的脊干，从而扩展或缩小网络，而无需人来做任何事情——只需要一个控制器来管理这一切。现在快进到 TPU。呃，TPU 采用 Taurus 拓扑，直接将所有 TPU 彼此连接起来。我之前谈到了吞吐量和有效吞吐量。我们能做的一件事是，如果一个 TPU 机架发生故障，就可以用另一个 TPU 机架替换它，而不必移动任何光纤。同样，我得打打手势，或者在白板上画一画。呃，但基本上，我们可以说我们始终有一个备用机架可用；当一个机架发生故障时，我们就把光信号重定向到那个新机架。That was reason number one. Reason number two was we wanted to be able to expand the network and contract the network without actually moving any fiber. And without working into the details, I could draw this on a board. The optical circuit switch would allow you to actually reconfigure the spine of the network to expand it or to shrink it without a human being um having to do anything other that' be literally a controller that would manage that. Now fast forward to TPUs. Uh a TPU has a Taurus topology that connects all the TPUs to one another directly. I I talked about throughput and goodput earlier. One of the things that we can do is if we have a TPU rack that fails, we can replace it with another TPU rack without moving any fiber. Again, I'd have to wave my hands or draw a whiteboard. Uh but essentially, we can say we have a spare rack available at all times and when a rack fails, we redirect the light to that new rack

主持人：这可以在几毫秒内完成。Host: and that can be done in milliseconds.

客人：那你们为什么还要光纤呢？Guest: Why do you have the fiber at all then

主持人：和完全自由空间相比呢？是的，这是个非常好的问题。嗯，衰减损耗会增加，带宽也会大幅下降。此外，要覆盖一栋非常大的建筑，如果没有光纤把所有东西连接起来，却想在三维空间中把信号传到各处，那会很有挑战，甚至可能根本做不到。嗯，我们谈过这个。其实我们确实谈过，也进行过一些非常有意思的讨论。嗯，不过是的，大部分连接还是通过光纤完成的；但当光信号到达光路交换机时，光就会直接照射到这些微小的芯片上。所以，嗯，这是网络发展的一个重要方向。网络领域的发展还有很多很多——说实话，我觉得数据中心网络的能力和需求都在飞速增长。Host: as opposed to fully free space? Yeah, it's a very good question. Um, attenuation loss and the bandwidth would drop uh dramatically. F furthermore, across the range of a very large uh building, aiming everything uh without f without the benefit of fiber to connect it all together uh in 3D would be would be challenging to possibly impossible. Um we've talked about it. We've talked about it actually and there have been some really interesting discussions. uh but but yes it's mostly in fiber but then when they hit the optical circuit switch uh essentially the light then literally shines down on these tiny chips. So um that's one big direction of uh networking. There's but there's there's a lot I mean I would say networking is exploding in in terms of its u uh capabilities and need frankly in the data center.

嘉宾：太酷了。我们可以就此展开一整场对话。Guest: So cool. We could have an entire conversation on that.

主持人：是的，这真的非常酷。Host: Yes, it is very very cool.

嘉宾：甚至，比如回到《难以言喻》里的那张图片，那里全是那些刺痛所有人眼睛的电缆。Guest: Even like the going back to the picture from ineffable it was all the the cables that that cut everybody's eyes.

主持人：没错，是的。那些线缆最终有一些会连接到我们数据中心里的光路。Host: Exactly. Yeah. And those cables eventually some of them wind up at optical circuits which in our data centers.

电力是硬性约束：公用事业、电吉瓦，以及数据中心规模测算Power is the binding constraint: utilities, gigawatts, and sizing a data center

嘉宾：有道理。好，我想转而谈谈功耗。你一直在谈每瓦有效吞吐量以及其他每瓦指标。这让我觉得，“每瓦”意味着功耗在某种程度上是限制因素、稀缺资源，或者成本高昂的约束。Guest: Makes sense. Okay. I want to flip to talking about power. You've been talking about goodput per watt and other units per watt. Makes me think per watt means power is kind of the the binding constraints or the scarce constraints or the expensive constraints in some way.

主持人：你知道，人们常问我这个问题：我们面临的最大制约是什么？现实情况是，我们面临的并不存在某个单一的最大制约。它们都是制约，而且都非常棘手。可能是这个，也可能是那个，而且它们还在不断变化。但如果要从根本上回答这个问题，我会说，电力是我们面临的最根本的单一制约。其他问题似乎都……我们知道该怎么解决，只不过需要花一段时间去解决。电力，没错，我觉得你说得特别好。这是一个长期的关键问题，而我们目前还没有……我是说，或许核能会带来充足的清洁能源，那肯定能解决很多问题。但这何时会实现，以及能否大规模实现，仍然是未知数。Host: You know, so I I uh I get asked this question, what is the biggest constraint that we face? And um the reality is there is no single biggest constraint that we face. They're all constraints. They're all super hard. It could be and then they shift continuously. But if I had to answer fundamentally uh I would say that power is the single most fundamental constraint that we face like everything else seems uh like we know how to solve them and it's a question of um solving them over some period of time. Power yeah I think the way that you put it is really nice. It's a it's a binding long-term issue that we do not have a I mean there you know nuclear you know perhaps nuclear is going to abundant clean energy which would would solve a lot of problems uh for for sure when that happens and at when it happens at scale uh still still unknown

嘉宾：那么，实际操作中这是怎么运作的？你知道，你正在建设一座新的数据中心，它需要一吉瓦的电力。我想你总不能直接打电话给 PG&，说“嘿，请给我送一吉瓦的电力”吧。那么，实际配置电力是什么样的？你们是否得一直垂直整合到自己造涡轮机这种程度？你们是怎么解决电力瓶颈的？Guest: and so how does it work in practice you you know you're standing up a new data center it needs a gigawatt of power I imagine you can't just call PG& and say hey please send a gig a lot of power. So, h what does provisioning power actually look like and are you having to actually vertically integrate all the way down to like, you know, doing your own turbines or how do you solve the power bottleneck?

主持人：是的，这又是一个很大的问题，一个重要的问题。我们，呃，我们在 Google 一直偏好的模式是，呃，与实用工具相连，保持良好连接。所以，这相当于调用你的，嗯，呃…Host: Yeah, it's again it's a it's a big question, important question. We do uh our our preferred model at Google is always to be uh utility connected to be good connected. So, it's the equivalent of calling your um uh

嘉宾：哦，所以你不能直接打电话给 PG&E。你得非常、非常礼貌地打电话给你最喜欢的那家公用事业公司——你的数据中心在哪儿，就联系当地的那家——当然，还要提前很多很多年通知他们。换句话说，如果我们说的是吉瓦级规模，那可不是你能说“嘿，我明天就需要一吉瓦的电力。你们什么时候能开始向我收费？”的事。这需要我们一起提前规划。Guest: Oh, so you can't just call PG&E. You exactly you call uh ve very politely your favorite uh utility wherever it is that your data center is and of course you give them many many years of notice. So in other words, if we're talking about a gigawatt scale, it's not something that you can say, "Hey, I need a gigawatt tomorrow. When when can you start billing me?" Uh it's something that we co-l plan together.

你知道，对我们来说，从确保与这些公用事业公司合作时，建设这类基础设施的成本由我们承担这一角度看，这也是我们非常认真对待的事情。我们也可以就这个话题谈很久，因为按照这种计费方式，理论上，公用事业公司为我们扩建容量，可能会导致其他人的费率上涨。我们要确保，像是必须新建或升级的输电线路、额外的公用事业变电站等等，这些费用也都由我们支付。 不过，这是一个漫长的规划过程。完全可能出现这种情况：比如我们需要在——我随便举个日期——2028年获得一吉瓦的电力，而公用事业公司要到2029年才能提供一吉瓦，但在2028年能提供七百兆瓦。这样一来，我们就得考虑如何补上这三百兆瓦。一个办法就是……另一个办法是想办法自行发电，满足其中一部分需求，比如用太阳能电池，或者用电池作为备用电源，等等。You know for uh for us it's something that we also take very seriously from the perspective of ensuring that when we work with these utilities the the costs of putting that infrastructure in place the we could get into a long conversations long conversation just on this topic as well that we cover those costs because with the way that billing works it actually could be that by the act of the utility building out capacity for us let's say other people's rates could go up in theory what we ensure is that actually let's say the transmission lines that have to be built upgraded ated uh additional utility based stations etc that we pay for those as well but so it is a long planning process uh it uh can absolutely be the case that in let's say that we need a gigawatt in I'll make up a date uh 2028 uh and the utility can get us a gigawatt in 2029 that can get us let's say 700 megawatts in 2028 so now we might be left with a question of how do we cover those 300 megawws well one answer is just Another answer is to say okay well would we figure out how we generate some subset of that power ourselves and maybe that would be with solar cells or with uh with batteries as a backup etc.

那会是其他能源来源吗？这样一来，我们又会与电力公司合作。这也可能是一种组合模式：我们可以保留一些本地发电能力，即使电力公司全面接入后——比如达到吉瓦级规模——我们也能把电力回送给电网。这样一来，他们有需要时也能用上，对吧？换句话说，一年中可能有两周——也许是最热的两周——居民用电需求会非常大。如果我们有一些本地发电能力，也可以把电力回送给电网。所以，对我们来说，这确实是一个与电力公司合作多年的过程。相比之下，你们为什么更倾向于这么做，而不是与电力公司合作，而是选择自己进行垂直整合呢？Would it be for other sources? So then we again work with the utility. So it might also be a uh combination where we might maintain some power uh generation local and have the capability even when the utility uh comes online uh fully at let's say the gigawatt scale where we could actually provide power back to the grid. So having that when they need it as well, right? So in other words, there might be the one way to look at it is there might be the two weeks of the year maybe it's the hottest two weeks of the year where there's huge amount of residential demand. If we have some local generation, we can then provide that back to the grid as well. So it really is working over multiple years with the utilities uh for us. Why is your preference to do that versus to kind of go like to work with the utilities as opposed to to kind of vertically integrate yourselves?

主持人：是的。主要原因是双方都有灵活性和收益。一种理解方式是统计复用，或者说大数定律。假设我们需要一吉瓦的电力，而且希望可靠性达到 99.99% 以上，那可能意味着我们必须建设两吉瓦的供电能力，对吧？在这种规模下，要达到 99.99% 或 99.999% 的可靠性，就必须采用 1+1 冗余。这会很贵。而且，要确保电力来自清洁能源，最好还就在我们的数据中心旁边，这也颇具挑战。如果我们再次与数据中心合作，也许可以自行提供一定量的电力；当电网的需求较低时，我们可以把电力供给电网，而在需要时则可以从电网取电。换句话说，通过在更大的范围内利用统计复用，实际上所有人都能受益：我们、电网、居民等等。Host: Yeah. So, um the main main reason is flexibility and uplift on both sides. So, one one way to look at it is um statistical multiplexing or if you want the law of uh large numbers. If we need to have a gigawatt of power, let's say we want to have that with 99.99% plus reliability, probably means we have to build two gigawatts of power, right? At that level at 99.99 or 99.999, you have to have oneplus 1 redundancy. That that gets expensive. And then making sure that that's going to be ideally a clean energy source probably right next to our data center, that can also be challenging. Now if we partner with the data center again maybe we have some some amount of that that we can bring ourselves that we can give to the grid when they needed less that we can take their power. So in other words by leveraging that statistical multiplexing over a much larger base actually everybody wins we win the the grid wins residences win etc.

我们更倾向于那样做。只有在极少数情况下，我们才会——嗯——比如说在电表后侧进行；但即便如此，我们也是抱着这样的前提：我们会与公用事业公司合作，可能一年后就希望接入电网。所以，再说一次，这对我们是个提升，对电网也是个提升。We prefer that and in rare rare cases we we will uh do it uh let's say behind the meter but even then we're doing it under the assumption that we're going to work with the utility where it might be a year later we we want to be connected to the grid. So again it's an uplift for us it's an uplift for the grid.

主持人：有道理。你们如何决定数据中心的规模？Host: Makes sense. How do you decide how big to make a data center?

嘉宾：是的，这确实是一门艺术，也是一个备受争议的话题。你知道，曾经有段时间，我们讨论过这个问题。我记得甚至在十多年前、十五年前，Google 就有过一场大辩论：我们是不是应该把所有东西都放进一个数据中心？当时设想的规模是一个吉瓦，也就是……Guest: Yeah that's that's an art and it's a a source of significant debate. You know, at some point, you know, we had a debate. I remember even 10 plus years ago, 15 years ago, big debate at Google. Should we just put everything into one data center? Like back then, it was going to be a gigawatt, which was

主持人：更简单的时光。Host: simpler times.

嘉宾：是的。时代不同了，比如说，105年前，我们就要建一个吉瓦级数据中心。所以，一个显而易见的担忧是单点故障，对吧？如果你从30年的角度来看，这确实是个很大的问题。另一方面，如果你说的是如今的训练工作负载，那规模越大越好。换句话说，从网络的角度来看，你确实希望把所有东西尽可能集中在一个小范围内。但话说回来，这又带来两个问题：单点故障，还有电力供应。吉瓦级的用电量在10年或15年前还很惊人，但至少可以想象；而现在，我们不可能在这个国家或世界上的任何一个地方，建成一处能满足谷歌全部需求的设施。这根本不可能。Guest: Yeah. Different times, like 105 years ago, we're going to have a gigawatt data center. So, one obvious concern there is single point of failure, right? If you're looking at things from a 30-year perspective, that's a a huge concern. You know on the other hand if you're talking about uh training workloads today bigger is better in other words having more because from a networking perspective actually you want to have things concentrated in as small a distance as possible but then again uh two issues single point of failure but then also now power availability right while a gigawatt was huge 10 or 15 years ago but imaginable um there's no way anywhere in the country or the world we're going to get the total requirements of Google built in in one place. it's just just not going to be possible.

那么现在最佳规模是多少？呃，这方面我们又有模型、模拟器等等。但这也取决于情况：你看，在某些地方，我们会位于网络边缘；也可能是在一个规模达到几十兆瓦的国家。实际上，我们会与一些互联网服务提供商合作，地点可能就在伊拉克，真的可能就是伊拉克。这是一个训练集群。好吧，这个规模可能会接近一吉瓦。嗯，其他一些站点可能有数百兆瓦，等等。So now what's the optimum size? Uh again we have models for this simulators uh etc. But this it it also depends on look in some places we're going to be at the edge of the network or we're going to be in country that might be tens of megawatts. We actually partner with ISPs that might be Iraq like literally it could be Iraq. This is a training cluster. Okay maybe that's going to be closer to a gigawatt. Um some other sites might be hundreds of megawatts uh etc.

训练集群与推理集群、七年前的 TPU，以及开放标准Training vs. serving clusters, seven-year-old TPUs, and open standards

主持人：有意思。我很想了解你们如何看待集群组合的生命周期管理。嗯，我猜想，最新、最大的集群会用于训练最新的前沿模型，然后你们会把较旧的设备循环利用起来，在上面运行推理。这个框架算合理吗？你们也会专门搭建用于推理的集群吗？这一切是怎么运作的？ 这是个非常合理的框架，你的想法很有道理。我想说的是，推理需求如此之大，以至于我们不能只依赖那些不再被充分用于训练的旧训练集群来承担推理任务。另外，如果你仔细想想，我们可能会在某一年再次集中部署训练集群，把它们放在少数几个大型站点中。让这些站点之间的网络距离尽可能短是有好处的。所以，不管它们位于世界上的什么地方，很可能都会在同一洲，甚至可能在某一年集中在同一洲的某个区域。Host: Interesting. I'd love to understand how you think about the portfolio life cycle management. Um so I guess my guess would be that you know the biggest newest clusters are used for training the latest frontier model and then you kind of recycle the older stuff and run inference on it. Is that is that like a fair framework? Are you are you also standing up inference specific clusters? How does that all work? very reasonable framework uh that that you're thinking about that uh and I think makes a lot of sense but I would say that the demand for inference is such that we can't just rely on um whatever older training clusters that are no longer being fully used just for training uh as the basis for inference and also if you think about it we might centralize again uh in a particular year into a small number of large sites our training clusters and there is benefit to keeping the network distance between them uh small so now let's wherever they're located in the world is they're probably going to be let's say on the same continent or they might be even in the same portion of a continent in a particular year.

所以现在你可能会发现，其他大洲等地的服务容量不够。那好吧，实际上我们就得在世界其他地方建设专用推理设施。所以，我觉得你的直觉完全正确，但也不尽然。换句话说，确实得这么做：这些是训练集群。是的，可能再过几年它们会用于提供服务。但与此同时，我们也得建设服务集群。So now you might be left with um not enough serving capacity on the other continents etc. So then okay actually we then have to go build the specialized inference uh elsewhere across the world. So uh I I think your intuition is uh spot on but not wholly. In other words, it really does have to be okay here are the training clusters. Yes, probably they're going to be used for serving in some number of years. Uh but then we're also having to build the serving clusters as well.

嘉宾：你们的服务集群和训练集群不一样吗？比如说，它们规模更小吗？每兆瓦的成本更低吗？呃，其实不一定更便宜，因为服务场景更需要把计算、网络和存储放在一起，这就导致我们难以进行专门化。因此，训练集群实际上可以采用高密度、统一部署等方式；而服务集群则需要混合配置存储、计算资源和加速器。此外，这方面还有个很有意思的地方，我们也可以进一步详细聊聊：在服务场景中，你其实不希望所有资源都集中在同一个地方。你需要从全球各地为工作负载提供服务。但现在，我们最终会有各自独立的模型端点，而这些端点实际上可能运行不同版本的模型。Guest: And your serving clusters are they different than the training clusters? Like are they smaller? Is there are they are they cheaper per per megawatt? uh not necessarily cheaper actually because uh for serving there is more need to colllocate uh uh compute networking and storage and so that then we get to the inability to specialize. So for training you actually have this big density uniform deployment etc. Uh whereas for serving you're going to have to have the mix of storage compute and accelerators. Furthermore, for um it's a very interesting aspect of this that we could go into more detail on as well, but for serving, you actually don't want to have too much in one place. You want to be serving your workloads from uh all over the the planet, but now uh we wind up having um individual model endpoints and that actually can be variations of models.

所以现在我们也必须考虑本地化需求，将这些模型部署到世界各地。因此它们会更小；在推理端，它们可能也不会那么垂直整合。So now we have to spread these models out across the world also accounting for locality. So they will be smaller. They'll actually be less vertically integrated probably on the infant side.

主持人：非常有意思。所以，如果你在 5 年前建造数据中心，当时最先进的加速器和现在相比，可就大不一样了。Host: Super interesting. So if you made a data center 5 years ago, you know, state-of-the-art accelerator 5 years ago was very different,

嘉宾：效率远低于如今制造的产品。Guest: vastly less efficient than the ones that are being made today.

主持人：是的。Host: Yes.

主持人：你们真的会回去把那些老数据中心里的芯片换掉吗？我知道这多少和一个一直存在的争论有关，我想，芯片的实际使用寿命到底有多长？Host: Do you actually go back and like swap out the the chips in those in those old data centers? And I know this kind of relates to there's been an ongoing debate, I think, of like what is the actual useful life of a chip?

嘉宾：是的，是的。所以，你知道，我之前，呃，呃，公开说过这件事。我说过的一件事引起了这么多关注，呃，这让我挺惊讶的。不过，你知道，我说过，我们那些已经用了七八年的 TPU 仍然保持着 100% 的利用率。Guest: Yeah. Yeah. So, you know, I've been uh uh on on record of saying this and one one of my more uh I was surprised by how how much pickup this had, but but you know, I said that our uh seven and 8 year old TPUs are still at 100% utilization.

主持人：哇。Host: Wow.

嘉宾：呃，然后，然后Guest: Uh and and

主持人：我想我见过这个。我想那是……Host: I think I saw this. I think that was

嘉宾：是的，我并不是想让它成为一个有意义的说法，但显然它确实是个有意义的说法。是的。所以，我们较旧的 TPU，呃，GPU，不过我们的旧款 TPU 目前的利用率相当高。最终我们确实会更换它们。问题是，好吧，它们的折旧年限大约是 6 年；所以一旦完全折旧，再考虑到新一代产品的能效等等，更换并升级确实是合理的。并不是说我会更换芯片，而是更换整个系统。换句话说，我们是以 Pod 为单位来考虑的。比如说，一个 TPU 8 Pod 可能有 9600 个芯片，可能由 140 多个——152 个左右的机架组成。然后我们会说，好吧，我们要把这个 Pod 拆下来，换成某种新 Pod，比如 TPU 12、13 或 14 Pod。Guest: Yeah, I didn't mean for it to be a a meaningful statement, but it was a meaningful statement apparent. Yeah. So, our um older TPUs uh GPUs, but our older TPUs are seeing significant utilization. We do uh replace them in the end. there's a question of okay have they uh uh depreciation lifetime is approximately 6 years uh so once they're fully depreciated uh and given the power efficiency of newer generations etc it does make sense to um replace them and upgrade them it's not that we I do replace the chips but it's really the the the systems so in other words we think in terms of pods so a TPU let's say um 8 pod might be 9600 chips and it might be uh 140 something 152 to racks etc. So then we would say, okay, we're going to pull out that pod and we're going to replace it with a whatever it is, TPU 12 or 13 or 14 pod.

可能不太匹配。换句话说，新 pod 的占地空间可能无法与旧 pod 腾出的空间完美契合。那我们就得把这一点考虑进去，想办法进行改造。我们没法提前规划，因为我们不知道那么多代之后的 TPU 会是什么样。呃，所以这又有点像一门艺术：要想出该怎么做；而且实际上，要想清楚如何尽快拆除旧设备，再换上新的 TPS，是件很费功夫的事。It might not be a perfect fit and then we have to, in other words, the new pod's footprint might not be a perfect fit for the old pod's vacancy. We we then just have to account for that. We have to figure out how we would retrofit. We can't plan that because we don't know what that many generations of TPUs are going to be. Uh so then it's again a bit of an art to figure out how we would and it's actually uh hard work figure out how we would decom and then replace as quickly as possible with with new TPS.

主持人：你曾公开撰文谈到开放标准。嗯，能谈谈这个吗？Host: You've written publicly about uh open standards. Um can you say a word on that?

嘉宾：是的。所以我想，这就是我们谈到的互操作性问题。对我们来说，虽然我们支持垂直整合，也让你能够按需发挥尽可能高的性能，但我们非常重视不强制用户被锁定，也不强制建立某种封闭花园式的生态，尤其是在系统端到端的运行方式上。Guest: Yeah. So I think that's the we we talked about this um interoperability question and so for us while we support vertical integration and we make it possible to you know extract just as much performance as you want for us it's really important to um not force lock in and not force a sort of walled garden in terms of how the system works end to end.

所以，你知道，对我来说，我举一个例子：在 Google，我们开发了一套模型开发框架，叫作 Jax。呃，我们很喜欢它，认为它真的非常好，而且在内部广泛使用。我们的许多客户喜欢 PyTorch，对吧？所以我们可以说：“嘿，如果你想在 TPU 上运行，就得用 Jax，它最好。”我这话多少有点戏谑，不过也许它确实最好，也许不是。我们认为它最好，所以它是唯一的选择，因为它实在太棒了。或者我们也可以说：“听着，如果你喜欢 Jax，我们也喜欢 Jax；你喜欢 Jax，就可以用它。但如果你喜欢 PyTorch，我们也有 torch TPU，你未经修改的模型等等都能在上面运行。”你知道，纵观历史，我们其实见过很多这样的例子。我用过的一个例子是 IP：为什么互联网协议会胜出？在 IP 出现时，其实还有许多相互竞争的协议。这都是上世纪 70 年代和 80 年代初的古老往事。So you know for mean I'll give one example at Google we've developed a um model development framework called Jax uh we like it a lot we think it's really really good and we use it extensively internally many of our customers like PyTorch right and so one one thing we could say is hey if you want to run on TPUs um you got to use Jax it's the best I'm being facicious a little bit but maybe it is maybe it isn't we think it's the best and that's only choice because it's it's so great or we could say look if you like Jax we like Jax if you like Jax you can use it but if you like PyTorch we have torch TPU where your unmodified models etc can can run we've and you know we've seen this uh throughout history uh actually with many many examples I've used the example of um IP why did the internet protocol win there were actually many competing protocols to IP this is ancient history in the 70s and early8s piece.

IP 之所以胜出，是因为它是一种开放标准，也是沙漏结构中的狭窄腰部：从软件角度看，任何东西都可以运行在它之上；从硬件角度看，任何东西都可以运行在它之下。因此，它是开放、可互操作的。任何采用 IP 的人都可以接入路由器端口，就这样成为互联网的一部分。这太棒了，也正是它让互联网得以迅猛发展，并扩展到世界各地。所以，我们确实相信这些开放标准能够成为接入点。现在，如果你想接入某种高度专业化的东西，你当然可以；如果你觉得有更好的东西可以接入我们的框架，完全可以。不过，我们真正想要支持的是开放标准，最好也支持围绕它的开源方案。Why did IP win was because it was uh open standard and it was this narrow waist to the hourglass where anything could run on top in terms of software and anything could run on underneath it in terms of hardware. So open standard interoperable anyone who brought IP could plug into a router port and become part of the internet like that. It was beautiful and that's what allowed the internet to explode in its uh growth and reach across the world. So we really do believe in those uh open standards in terms of plug-in points. Now if you want to plug in something highly specialized, you can right if if you think that you've got a better thing to plug into our our framework, you absolutely can. But we want to really support open standards ideally open source around it as well.

主持人：这项建设规模太大、也太重要了，不能由某一家供应商用封闭的专有技术栈来主导。是的，我们确实深信这一点，而且必须让大家有选择。这也是为什么，比如说，我们全面支持 TPU、GPU 和其他加速器等。Host: It's too big and too important of a buildout to be you know any one vendor's closed proprietary st. Yeah, we really we really believe that and it's got to be one of choice and that's also why for example we fully support and have uh TPUs, GPUs, other accelerators etc.

主持人：是的。人工智能改变了你们团队的日常工作和生活吗？它对你们职能的影响最大体现在哪里？Host: Yeah. How has day-to-day life for for your team changed with AI? Where is it changing your your function the most?

嘉宾：所以我想，不管你怎么说，最简单的答案都在软件工程这方面，我的意思是，这方面已经有不少公开报道了。我想，我们在谷歌，我的团队也在使用它，并且在软件开发能力方面取得了很好的效果；坦白说，它还帮助我们进行测试部署，甚至协助设计等。不过，硬件方面的变化同样显著，只是可能较少受到外界关注。换句话说，我的硬件工程师——我今天早些时候刚看过数据——也在使用人工智能。要是你愿意的话，可以看 token 数量；这不是最好的衡量指标，但仍然算一种指标。我们的硬件工程师使用人工智能的程度，和软件工程师一样高。而且生产力也提高了。Guest: So I think the the whatever you want to say the easiest answer is on the software engineering side where I mean it's been documented uh externally quite a bit. Uh so I think that we are at Google and my team using it to uh great effect in in terms of our software development uh uh capabilities but also uh frankly uh test rollouts even uh helping with design etc. But there have been uh on the hardware side which has maybe gotten less uh external coverage also significant change. In other words, my my hardware engineers, I was just looking at the numbers earlier today, in fact, are using AI using whatever the the token counts if if you want, not the best metric, but it's still a metric. Our hardware engineers are using AI as much as the software engineers are. And productivity has gone up.

从设计启动到流片的时间在缩短，你知道，bring-up 所需的时间也在缩短。因此，另一个生产力显著提升的领域是数据中心设计。与此同时，数据中心设计的方式也发生了显著变化。换句话说，我们如何评估——你提到过，是建一座吉瓦级的建筑，还是建设一个 100 兆瓦或 200 兆瓦的园区。过去，这个过程非常细致，主要依靠电子表格，也很大程度上由人来推动。现在在某种程度上仍然如此，但如今有大量 AI 参与其中，大大简化了规划和开发流程。Time to you know, time from design kickoff to tape out is shrinking. Time for bringup is shrinking. So another where's productivity is going up uh significantly but then other maybe less expected changes how we do data center design uh has uh changed significantly. In other words how we and how we evaluate you you mentioned hey do we do a gigawatt building or 100 megawatt um campus or 200 megawatt campus. In the past, this would be very detailed, very sort of um spreadsheet driven, human-driven processes, and it still is to some extent, but there's a lot of AI now involved that really streamlines the process in terms of planning and uh development as well.

主持人：有意思。那么，Host: Interesting. So,

嘉宾：比如推理模型或Guest: like reasoning models or

主持人：嗯，我还不会说它们已经算是推理模型了。它们并没有取代人的判断，而是让汇集必要信息变得容易得多，比如把信息集中到一个地方，然后基本上把正确的信息呈现在做决策的人面前。Host: um I wouldn't say quite yet reasoning models. It's not it's not replacing human judgment, but it's um making it much easier to bring the necessary information together like uh in in in one place and uh basically uh put the right information in front of the humans making the decisions.

轨道数据中心与2036年的超级计算机Orbital data centers and the supercomputer of 2036

嘉宾：好，我来用两个有点跳脱、挺有意思的问题为这段对话收个尾。Guest: Okay, I'm going to bring us home with two kind of out there kind of fun questions.

主持人：是的。Host: Yeah.

嘉宾：嗯，第一个问题，呃，关于轨道数据中心。我看到谷歌似乎正在非常认真地考虑这件事，你希望我对此发表看法。不过，我的意思是，人们开始认真测算在轨计算能力，这是否意味着地球上存在某种严重的、具有约束性的限制？你怎么看轨道数据中心？Guest: Um question number one, uh orbital data centers. Um I've seen that you know Google is thinking about this quite seriously and you want me to comment on that but I mean does the fact that people are seriously running the numbers on orbital computes mean that there are kind of serious binding constraints on Earth and and what do you think of of orbital data centers?

主持人：是的，这是一个令人兴奋的方向。我们确实正在推进，而且我们毫不戏谑地把它称为一项“登月计划”，是我们非常期待投入资金的一项重大工作。这又回到了谈话前面那部分：你提到，从根本限制因素来看，我认为你说得很对，能源和能源生产是一项关键挑战。所以归根结底，从基本原理来看，在太空中，由于大气层不会削弱阳光等原因，可获得的电力大约多出 40%。换句话说，太阳能发电能力更强，大概是 1.4 倍、4 倍？这是其中一个方面。不过，在太阳同步轨道上，太阳能电池能有 98% 到 100% 的时间受到阳光照射；而在地面上，大概只有 28%、30%，也许 35%，对吧？Host: Yeah, it's an exciting direction. We are actually pursuing it and we've uh referred to it uh with no tongue and cheek as a as a moonshot as one of our big um big efforts that uh we're excited about investing in. uh goes back to the earlier part of the conversation where you raised I think correctly that in terms of fundamental constraints um energy and energy production is a key key challenge and so really um bottom line is from a fundamentals perspective um in space um you have something like 40% more power available because of lack of attenuation in the atmosphere etc. In other words it's just more solar capacity what 1.4x 4x. So that's one part of it. But but in a um sun-synchronous orbit, you have 98 to 100% coverage of sunlight on your solar cells relative to 28 30 maybe 35%. Um on land, right?

换句话说，就是如此巨大的电力，显然来自太阳。你把1.4倍、4倍这些因素，再加上每天发电时长增加3到4倍考虑进去，基本上就可以把电池从等式中去掉。这样一来，你就有可能实现某种真正可行的方案——当然，它不产生碳排放，优势很多，挑战也很多。So in other words, just this enormous amount of power, obviously the sun, you you take the 1.4x 4x and you take the 3 to 4x uh in terms of number of hours per day, you remove batteries more or less from the equation and now you have the potential for something that can really and and of course carbon-f free lots of benefits, lots of challenges.

嘉宾：是的。Guest: Yep.

主持人：对。挑战非常多。换句话说，好吧，那冷却呢？呃，你知道，直觉上你可能会觉得太空中的冷却更容易。实际上更难。呃，可靠性。我们谈过这些东西有时会失效。所以维修，呃，在太空中会变得更难，并非不可能，但会更难。太空机器人和集群一起……Host: Right. Lots and lots of challenges. So in other words, okay, now what about cooling? Uh you know, naively you might think cooling in space is easier. It's actually harder. Uh reliability. We talked about how these things fail sometimes. So repairs uh is becomes harder not not impossible but it becomes harder in space robot up with the with the cluster

嘉宾：没错，所以也许这种模型会是一个很有希望的方向。冗余大概会是你的好帮手。你之前关于自由空间光学的问题，现在即将成为现实。我们可能不会在这些组件之间铺设光纤，而是实际采用自由空间传输，让激光器对准接收器，并进行实时校准。这些都不是难以逾越的障碍，没有什么根本性的障碍。Guest: and exactly so perhaps there'll be uh this model uh is a is a promising uh direction. Uh redundancy will probably be your friend. Uh your uh pre question on free space optics is now going to become reality. We're probably not going to be stringing fiber uh between these components. So it will actually be free space and we're going to have the lasers pointing at the um receivers and calibrating in in real time. These are there are no showstoppers here. No fundamental showstoppers.

主持人：最后一个问题。我脑海里一直有这样一幅画面：你们为他们建造的那台难以形容的……呃……大型超级计算机。Host: Last question. I've had this image of the ineffable uh big supercomputer that you built them

嘉宾：嗯，我一直都在心里想着这件事。所以我有个问题。Guest: um in my head this whole time. And so here's a question.

主持人：10年后，最前沿的超级计算机会是什么样子？Host: 10 years from now, what will the most frontier supercomputer look like?

嘉宾：哦，天哪。是啊。10 年正好处在那个临界点上，嗯，呃，我真心觉得，可能有些人已经在思考 2036 年的计算机会是什么样子了。也许有少数人会想得那么远，不过，哇，我，嗯，不确定性的范围实在是太大了。换句话说，如果你看看这些趋势，嗯，集成度将会非常高。所以，你知道，我们看到了为 ineffable 组装的那些漂亮的光纤和 VR200 机架。我猜，它会集成得更多，光纤更少。我不会说 2036 年就完全没有光纤了，但我觉得，从机架的角度来看，嗯，机架会显得集成得紧密得多，光纤的数量也许会少到只有一小束从机架里引出。然后，呃，换句话说，从模块化制造的角度来看，有一种看法是，2036 年的这些机架很可能会集中制造。Guest: Oh my gosh. Yeah. 10 years is right at that edge where it's um uh I would say in all sincerity probably some people are thinking about what the 2036 computer looks like. We might have a few thinking that far ahead but wow I um the cone of uncertainty there is is uh way way too wide. Like in other words if you look at the trends um the level of integration is going to be tremendous. So, you know, we saw the beautiful fiber and the um VR200 uh racks that we put together for ineffable. My my guess is it's going to be more integrated, less fiber to I won't say no fiber in 2036, but I think that probably from a rack perspective um the rack will look um much more tightly integrated and the amount of fiber there'll be maybe a small fiber bundle coming out of the rack. And then uh so in other words, one one view you might have from a modular manufacturing perspective is these racks likely in 2036 are going to be manufactured centrally.

所以，不管它们有 72、144、288，或者可能更多——比如 576、52，挑一个你喜欢的 GPU 数量倍数吧——因为我们说的是一个难以言喻的场景。不过，对于深度集成在机架中的 TPU 来说，情况也一样：一个机架可能会达到数兆瓦级。具体会怎样我们还不清楚，但到了 2036 年，我们可以设想，单个机架的功率可能达到数兆瓦。然后你接上水和电，再接上光纤，对吧？你把那个机架推过来，把这三样东西插上，就可以开始大展身手了。And so whether they have 72 or 144 or 288 or probably more you know 576 52 pick your favorite multiple of GPUs since we talking about the an ineffable case but then be the same for the TPUs deeply integrated into a rack would a rack be multiple megawatts you know we don't know exactly how but 2036 we get to we get to imagine things could be multiple megawatts in a single rack and now you bring water and you bring power and you ing fiber, right? You you you wheel that uh rack in, you plug those three things in and you're off uh off to the off to the races uh as well.

主持人：好吧。所以它就不会是那种巨大的外星球体了。Host: Okay. So, it's not going to be like a big alien orb then

嘉宾：到2036年进入太空。我不会说到2036年不可能直接把它发射到太空，然后也许由某个空间站上的机械臂抓住，再插入正确的模块。嗯，是的，也许到2036年能做到这一点。Guest: up in space by 2036. I wouldn't say that it's not possible for 2036 for it to be then directly launched into space uh and then maybe um taken by a robotic arm at some space station and then plugged into the right module. Yeah, maybe that for 2036.

主持人：有趣Host: Fun

嘉宾：想想就很有趣，很有趣，很有趣。Guest: fun fun fun to think about.

主持人：嗯，我真的很喜欢这段对话，在这段对话中……Host: Um I really enjoyed this conversation where this is

嘉宾：史上最激烈、规模最大的回怼倾泻而出，但我也觉得这是一种技术。Guest: the most intense biggest capback spilled out in history, but I also think it's it's a technology

主机：革命Host: revolution

嘉宾：革命是一件美好的事，我认为……Guest: revolution that is a thing of beauty and I think

主持人：你对技术之美、各种硬性限制，以及如何平衡这一切，都有着如此深刻的理解。嗯，谷歌交到你手上，我们很放心。感谢你抽出时间分享你们正在做的事情。Host: you have such a grasp over you know the beauty of the technology and and the binding constraints and and how to balance this this all. Uh Google is in good hands. Thank you for taking the time to share what you're doing.

嘉宾：非常有趣。我们能亲身经历。我们能亲身经历这一切，并由我们来定义它。所以，非常感谢。这是一次很棒的对话。Guest: Ton of fun. Ton of We get to live. We get to live through this and we get to define it. So So thank you very much. This is a great conversation.

主持人：谢谢。Host: Thank you.

嘉宾：是的。谢谢。Guest: Yeah. Thanks.