前 Intel CEO 谈硬件创业良机：AI 芯片、内存与能源瓶颈｜Pat Gelsinger

简要描述Brief Description

英特尔前 CEO：为什么现在是打造硬件的最佳时机Former Intel CEO: Why This is the Best Time to Build Hardware

目录Table of Contents

• 简介
  Intro
• 18岁从技校毕业后加入英特尔
  From tech school to Intel at 18
• 486 与现代 EDA 的诞生
  The 486 and the birth of modern EDA
• 人工智能如何改变芯片设计
  How AI changes chip design
• 为什么硅片仍需九个月才能制造出来
  Why silicon still takes nine months
• 100种AI芯片会收敛到少数几种吗？
  Will 100 AI chips converge to a few?
• 为什么 HBM 是一种糟糕的内存
  Why HBM is a hideous memory
• 薯片能堆多高？
  How tall can chips get?
• 能源能力等同于经济能力
  Energy capacity equals economic capacity
• 一个面向智能体的 VMware
  A VMware for agents

简介Intro

在人工智能数字时代，能源容量就等于经济容量。要是没有足够的电力，建新的数据中心、买上百万块 GPU 又有什么意义？你会看到越来越多的数据中心项目违约，因为根本没有足够的能源可用。In a AI digital age, energy capacity equals economic capacity. Why build a new data center and buy the million GPUs if I can't power them? You're going to see more and more defaults happening on many of those data center projects cuz the energy won't be there.

主持人：每当你拥有能让某件事变得简单的技术时，就意味着瓶颈会转移到别的地方。Host: Whenever you have the technology to make something easy, that means the bottleneck moves somewhere else.

主持人：现在已经没有什么是单独一块芯片了，全都是一个机架。我花了3个月设计它，但还得等9个月才能真正开始使用。没错。过去30年里，我们出现过多少种重大的新型存储器？Host: Nothing's a chip anymore. It's a rack. It took me 3 months to design it, but it's 9 months until I can actually start to use it. Exactly. How many major new memories have we had over the last 30 years?

客人：零。Guest: Zero.

主持人：零。Host: Zero.

嘉宾：30年来首次内存创新即将到来。大约25年前，我就宣告了铜的死期。终有一天，我会证明我们这些硬件人士全都看对了。这就像一场文艺复兴正在我们眼前展开。AI 推理加速芯片——我敢肯定，我甚至还没把它们全都了解清楚。你们为什么要给这么多这类项目提供资金？Guest: Memory innovation for the first time in 30 years is nigh upon us. I declared the death of copper about 25 years ago. Eventually, I'll be right for all of us hardware guys. This is like renaissance in front of us. AI inference accelerator chips. And I'm sure I don't even know them all. Why are you guys funding so many of those?

主持人：你也出了一份力。Host: You funded your fair share, too.

嘉宾：从历史上看，任何行业都从未有过100家相互竞争的处理器厂商。这只是暂时现象，最终会重新收敛到一种格局。Guest: Historically, there have not been a 100 competing processor vendors in any industry ever. This is a temporary thing. It'll converge back to a view.

主持人：我认为这是一个Host: I see it as a

18岁从技校毕业后进入英特尔工作From tech school to Intel at 18

嘉宾：今天我们请到了尊贵的嘉宾、亲爱的朋友兼前老板帕特·格尔辛格先生。欢迎你，帕特。帕特目前是 Playground Global 的普通合伙人。不过，当然，你在业界非常有名，曾领导英特尔，担任英特尔首席技术官，当然也领导过 VMware，还做过很多其他事情。Guest: I'm here with uh our esteemed guest and dear friend and former boss, Mr. Pat Gellinger. Welcome Pat. Pat is currently the general partner at uh at Playground Global, but uh as uh you're very very well known in the industry for leading Intel, for being the CTO of Intel, of course, leading VMware and many other things.

主持人：那么，欢迎。Host: So, welcome.

嘉宾：嘿，谢谢你，Rayu。很高兴和你还有Guido一起，对吧？对我来说，这感觉有点像回到了老地方，对吧？你知道，就像，呃，我们仿佛叠加了一年的变化，对吧？不过，呃，你们俩看起来还是老样子。我觉得，呃，精力充沛。那我们开始吧。Guest: Hey, thank you, Rayu. Great to be with you and Guido, right? And to me, this feels just a little bit like old home, right? You know, it's like, you know, you know, it was like uh we superimposed a year of change, right? But uh you both look the same. I feel uh energetic. So let's dive in.

主持人：你从来没有哪一天不精力充沛。所以这不算什么新鲜事。呃，不过是啊。不，我们还是从你的英特尔职业生涯谈起吧？我是说，最近安德烈·霍罗维茨创办了这所 Horowitz and Recreent Academy。Host: There has never been a day when you stopped being energetic. So there no news there. Uh but yeah. No, let's start actually from uh from your Intel career, right? I mean recently Andre Horovitz created this Horowitz and recreent academy

嘉宾：也就是发掘 16 到 22 岁之间的有才之人，然后让他们进入现代化的教育环境，为他们做好充分准备，让他们无论是自主创业还是加入当今的企业，都能应对自如。你也有过类似的经历，对吧？你上的是普通职业学校，之后就进入了英特尔。Guest: which was to find talented people between 16 and 22 and then put them into a a modern educational setting so they'd be well prepared for either doing things on their own or joining the companies of today. You had a similar experience like you went to a regular trade school and then jumped into Intel.

主持人：是啊，是啊。我们当时大概是18岁还是19岁，是……Host: Yeah. Yeah. When we were what 18 or 19 is

嘉宾：是啊。那真是一段有点神奇的时光。你知道，我当时才16岁，意外地拿到了一份奖学金。我去了技校，对吧？高中还剩一年半时，我就辍学了。等我18岁时，英特尔来招人。嗯，你知道，我从来没坐过飞机。那时我已经在技校爱上了计算机。嗯，面试官叫罗恩·史密斯。他在自己的记录上写道，那天我是他面试的第12个人。如果你连续面试了12个人，到最后连男女都分不清了，对吧？你知道，我排第12个。他说：“聪明、进取、傲慢。他会很适应这里。”于是我受邀去英特尔，然后，嗯，你知道，一切都变得特别美好。Guest: Yeah. Uh, you know, it was really a sort of magical period. You know, I'm 16 years old. I accidentally win a scholarship. I go to tech school, right? Skipped my last year and a half of high school and Intel comes recruiting when I'm 18 years old. And, uh, you know, I never been on an airplane. Uh, had already fallen in love with computers at tech school. And, uh, the interviewer, right, Ron Smith was his name. He writes on his page. And I was number 12 that he interviewed that day. And if you've interviewed 12 people in a row, you can't tell male from female, you know, by the end of it, right? You know, I'm number 12. He says, "Smart, aggressive, arrogant. He'll fit right in." So, I got invited to go, you know, come to uh Intel and, you know, just became uh, you know, really uh glorious.

嗯，你知道，我一开始是技术员，286 项目末期转入设计团队；在 386 项目中担任第四号工程师；到了 486 项目，则成为架构师和设计经理。与此同时，我还在攻读硕士、本科、硕士和博士学位，所以这简直是你能拥有的最好的职业生涯：白天学习，晚上把学到的东西付诸实践，然后……uh you know starting as a technician moved into the design team at the end of the 286 you know engineer number four on the 386 you know uh architect and design manager for the 486 all of that while doing my master my bachelor's my masters and PhD work so it was like the the the best career that you could possibly have you're learning by you know day and you're putting it to practice at night and

主持人：没错，没错，你们正在开创一个全新的芯片时代。我敢打赌，你们在那里学到的东西和打造出来的东西之间有很大差距，因为你们是在开辟全新的领域。Host: that's right that's right you're creating new completely new era of chips for I bet most of what you learned and what you built there was a big gap because you're breaking new ground.

嘉宾：是啊。从很多方面来说，嗯，你知道，那真是行业里一段相当不可思议的时期。我记得有一次在斯坦福上课时，有位教授提出了一种新的超前进位加法器设计。我当时就说：“嗯，我正在做 386 的超前进位加法器，那我就试试看。”两天后我回来找那位教授，说：“这东西不对劲。”然后我就和斯坦福的教授争论起来。接下来的两节课，基本上就是我和教授一直在争论他的设计为什么行不通。Guest: Yeah. In a lot of ways and uh you know really was a pretty incredible time in the industry and I remember you know one of the professor classes uh you know he uh postulated a new carry look ahead adder design right in one of my classes at Stanford. So I said well I'm working on the 386 carry look ahead adder so let me go try it. So I come back two days later to the professor and I said it doesn't work right and I started argument with my you know professor at Stanford and it was like you know it was like what you know the next two class periods were just me and the professor arguing over why his design didn't work.

主持人：他有一本书正在出版，讲的是那个设计是什么。所以他当时非常烦人。这件事对他来说时间非常紧迫。Host: He had a book that was being published what that design is. So he was he was very annoying. It was very time sensitive for him

嘉宾：你知道，我们开始做内建自测试了，对吧？你知道，我记得斯坦福的 Ed McCluskey 是我的教授，他可以说是这方面的鼻祖。我们围绕着为什么他的大多数想法都不靠谱、无法在真正的芯片上实用，展开过很精彩的争论。不过我当时正把内建自测试用在 36 上。你知道，John Hennessy 是我的论文导师。显然，他后来发展得不错。我常说，是我帮了他的事业一把，他后来当上了校长，从那以后也一直干得不错。是啊，是啊，是啊。Guest: and you know we started builtin self test right you know and I remember Ed McCcluskey at Stanford was my professor and he was sort of the father of that and we you know had wonderful arguments around why most of his ideas were bad they weren't practical for real chips but I was putting you know builtin self test to work on the 36 you know John Hennessy uh you know was my thesis advisor and obviously he did okay I say you know I helped his career he became president and he's done okay since then Yeah. Yeah. Yeah.

主持人：稍后我们会讨论 CIS 与风险之争，不过我想谈谈 486。抱歉，请继续。Host: Later on we'll talk about the CIS versus risk debate, but I want to talk about the 486. Sorry. Go ahead.

嘉宾：我在斯坦福教书时也有过同样的经历。你知道，你带着自认为是真理的东西走进课堂，然后有个在业界工作的学生会说：“嗯，其实不完全是这样。”所以，下一次上课时你就回来告诉大家：“好吧，呃，我查过了。你说得对。这个说法已经过时了。”Guest: I had the same experience when I was teaching at Stanford. You know, you you come with something that you think is the truth and then the student comes that works in industry is like, well, actually not quite. So, you know, the next lecture you come back with like, okay, uh, I looked this up. You're right. You know, this is this is outdated.

主持人：是的。我跟你说，你知道，那是学生群体的一部分。是的，那种互动，我觉得也是硅谷如此独特的原因之一。是的。Host: Yeah. And I tell you, you know, that's part of the set of students. Yeah. that interaction and I think that's part of what makes Silicon Valley so unique. Yeah.

嘉宾：没错。你知道，这里有很多公司，创业精神就像融入了这里的水土一样。大家在这里创办新事业、提出新想法、挑战教授。你在斯坦福这样的地方当教授，就得预料到自己会受到挑战，而且还得乐在其中。Guest: Right. You know is you have you know these so many companies and you know entrepreneurship it's like in the water here right you know starting things up and new ideas and challenging the professors and you can't be a professor at a place like Stanford and not expect to get challenged and enjoy it.

主持人：那座象牙塔进进出出的人流还真不少。Host: There's a good foot traffic in and out of that ivory tower.

486 与现代 EDA 的诞生The 486 and the birth of modern EDA

嘉宾：是的，我觉得挺好的。Guest: Yeah I think it's good.

主持人：是啊。现在，486 曾是你的重大……Host: Yeah. Now 486 was your big

嘉宾：是的。Guest: Yeah.

主持人：可以说是一项成就吧。显然，很多人都参与了这项工作。嗯，而且你们在芯片设计方面开创了一些新局面，对吧？Host: accomplishment if you will. Obviously lots of people worked on it. Um and you guys broke some new ground in chip design there, right?

嘉宾：嗯。Guest: Mhm.

主持人：嗯，还增加了芯片原本要执行的功能，等等诸如此类。Host: And um increased what functions the chip was supposed to do and so on and so forth.

嘉宾：是的。那也是一段颇为神奇的时期，因为当时还没有后来人们所说的 EDA 行业。Guest: Yeah. It was a pretty magic period uh as well because it was before what you would think of as the EDA industry.

主持人：哦，哇。是的，没错。嗯，你知道，386 后期的一小部分工作，但 486 才真正是第一款采用了现代 EDA 技术的芯片。是的，我们做了高层次设计，对吧？嗯，有设计描述和 RTL 描述，但那时还没有 Verilog。Host: Oh wow. Yes, that's right. and uh you know the the end of the a little bit of the 386 but the 486 was really the first chip to implement what what you would think of as modern EDA techniques. Yeah, we did a highle design, right? Uh description and RTL description, but there was no vera log.

嘉宾：所以，我们发明了 HDL，也就是英特尔硬件描述语言，对吧？你知道，我编写了自己的语言。那你得为这种语言构建一个编译器，对吧？所以，我们为这种语言创建了编译器。当时没有自动布局布线工具，你知道，所以我们不得不发明这类工具。我们还与伯克利的阿尔贝托·桑吉奥瓦尼-文森特利以及他的一些学生合作，完成了最早的布局、最早的布线，以及最早的自动时序管理……Guest: So, we invented HDL, right? The Intel hardware description language. So, you know, I wrote my own language. Well, you had to build a compiler for that language, right? You know, so we created the compiler for the language. There was no automatic place in route, you know. So, we had to invent that. And we worked with Alberto Sanji Yavanji Vincentelli at Berkeley and some of his students for the first you know placement the first routing the first automated timing you know management and

主持人：所以，他们在某种程度上创造了EDA，而且程度相当大。呃，你知道，这确实是一个重要标志：486不仅是一款兼容的流水线微处理器，我们还开创了后来成为现代EDA产业的许多基础。你知道，那真是这个行业一个相当神奇的时期。Host: so they created EDA to some degree. to a great degree that uh you know and that was really one of the hallmarks that not only was the 486 you know this compatible you know pipeline microprocessor but we ushered in you know many of the foundations of what became the modern EDA uh industry and you know it was really you know pretty magical period of the industry.

嘉宾：是的，是的。所以如果你看看今天……Guest: Yeah. Yeah. So if you look at today

AI 如何改变芯片设计How AI changes chip design

主持人：是啊。人们回顾 Blackwell 时，会不会说，你知道，这是 AI 接管微处理器设计流程之前完成的最后一代产品？我们现在是不是也处在类似的转型期？Host: Yeah. Will people look back as Blackwell and saying you know this was of the last generation that was was done before AI took over the design process for for micros are we in a similar transition right now

嘉宾：我觉得这其中确实有一些方面是这样的。你知道，我想，当你看看像今天的 Jalapeno 这样的东西时，你会说，这算是在芯片设计流程中从第一性原理出发应用 AI，对吧？现在你可以把很多那些东西抛开。不过，设计中仍有一些部分确实很难，尤其是模拟方面；电路的模拟部分大概就是最好的例子。它们仍然不太能用 AI 来做，你知道，要做到这一点，你需要大量的硅片数据。所以，要么你在模拟设计上采取非常保守的做法，以至于我会说你能够用 AI 来做；要么你就得用传统的困难方法来做。不过现在，很多逻辑功能确实可以通过 AI 工具和技术，以相当不可思议的方式实现，对吧？Guest: I think there are certainly aspects of that and uh you know I think you know when you look at you know things like jalapeno today right you sort of say you know that you know that's that's sort of a first principles use of AI in the chip design process right where you throw out a lot of those things now you know un You know there are pieces of design that are still really hard right uh in that sense and a lot of the analog right aspect certis is probably the best example of that you know are still not aiable right you just need lots of silicon data right to get those so you either get so conservative in your analog design you know that you're able to I'll say AI it uh or right you got to do those the hard way uh you know for but to now many of the logic functions can really be done with uh AI tools and techniques in pretty incredible ways, right?

你们的晶体管预算足够充裕，设计工具也越来越智能；你知道，只需要几位专家来指导这些工具，以及如何应用它们。我真的觉得，这方面会有点像 46，对吧？回头看时，你会说：“没错，那就是芯片设计新时代的开端。”Your transistor budgets are big enough, the design tools are getting smart enough, you know, a few experts guiding the tools and how to apply it. And I really think it will be somewhat like the 46 that way, right? Where you sort of look back and say, "Yep, that was the beginning of a new era of chip design."

主持人：每当你拥有让某件事变得简单的技术时，就意味着瓶颈会转移到别的地方。Host: Whenever you have the technology to make something easy, that means the bottleneck moves somewhere else.

主持人：是的。我们现在能不能猜测一下，未来的瓶颈会在哪里？Host: Yeah. Can we guess at this point where the bottleneck will be in the future?

嘉宾：有些事情看起来确实是 AI 特别擅长的，对吧？比如构建软件层、编写内核，嗯，还有很多……Guest: There's some things which it seems like AI is incredibly good at, right? Like creating the software layers, you know, writing kernels, um, you know, a lot of the

主持人：我觉得，能够在更高层次上设定目标，然后再把它转化成非常底层的东西，这方面有很多进展，对吧？那么新的前沿是什么？今后什么事情会变得困难？嗯，你知道，当你看的时候……Host: I think a lot of the sort of being able to specify the the the objectives at a higher level which then gets sort of translated into very log right and so what is the new frontier? What is the thing that that will be difficult going forward? Well, you know, when you look

为什么硅仍需要九个月时间Why silicon still takes nine months

今天，你知道，比如说，Guido 和 Pat，我们要一起打造一款很棒的新芯片。我们正在开发，at today, you know, so let's say, you know, Guido and Pat, we're off to do a great new chip together. We're building,

嘉宾：对吧？你知道吧？对吧？你知道，它会领先一步，因为我们对 8646 有所了解，Guest: right? You know, right? You know, and it's the one that's going to leap ahead because we have understanding of 8646,

主持人：你知道，我们了解 AI 工作负载。我们也了解如何将它们组合成合适的乘累加寄存器结构，嗯，寄存器结构，】【。Host: you know, we have understanding of the AI workloads. We have understanding then of how to compose that into the right, multiply, accumulate, you know, register structures,

嘉宾：同样也是面向未来工作负载的完美可视性模型，Guest: the perfect visibility model workloads of tomorrow as well,

主持人：你知道，诸如此类的事情。比如说，我们放开使用 AI 工具，三个月内就能做出很棒的设计，对吧？不过在所有模拟组件方面，我们一直比较谨慎。Host: you know, and all of that kind of stuff. And you know let's say you know we turn our AI you know tools loose we are great within three months we have an awesome design right you know you know for that we've been a little bit conservative on all the analog components of it

嘉宾：你知道，现在我们仍然有几个非常大的瓶颈。你知道，其中一个是，我们仍然需要九个月的硅片加工时间。Guest: you know now we still have like a couple of really major bottlenecks today you know one is right we still have nine months of silicon processing time

主持人：对，你知道有这样的人，对吧？所以我可以在三个月内把东西设计出来，但要过九个月才能真正大规模地把它做成硅片。好吧，这确实很糟，对吧？所以，如果我们想实现这种创新，就必须看到这个瓶颈确实有所改善，因为我们知道……Host: right you know someone like that right so I can design the thing in three months but I can't actually get it into real silicon at scale for 9 months. Okay, that sucks, right? So, we have to really see that bottleneck improve if we're going to have this innovation because we know that

嘉宾：关于验证和确认阶段，甚至更往后的阶段。嗯，你知道，首先，我没法在不到三个月的时间里从晶圆厂拿到任何东西，而且这还是在我拥有超强设计流程的情况下，对吧？所以我就在想，怎么才能把这个时间压缩下来？然后我还得转向先进封装。3D 封装越来越复杂了，这又要花三个月，对吧？接着我还得把它集成到机架级解决方案里，因为现在没有什么只是芯片了，而是整个机架，你知道。所以总共要九个月。我花三个月把它设计出来，但得等九个月才能真正开始使用它。我们该怎么缩短设计的这些环节所需的时间？我认为我们需要新的光刻技术来实现这一点。Guest: about the validation verification stages or even beyond that. Well, you know, first I can't get anything out of fab. Yeah. You know, in less than three months and that's if I have, you know, like supercharged design flows, right? So, I said, you know, you know, how do I compress that, right? Uh, you know, for it and then I got to get into advanced packages. 3D packages are getting complex. So, that's like another three months, right? You know, and then I have to put it into a rack scale solution because nothing's a chip anymore. It's a rack, you know. So we're 9 months, you know, it took me 3 months to design it, but it's nine months until I can actually start to use it, you know. So how do we start to compress, you know, those aspects of design? And I think we need new forms of lithography, you know, to enable that.

我们得设计一些流程，你知道，在我能把东西推进原型阶段之前，不需要先投入5000万美元的巨额成本。所以你知道，对我来说，怎么才能把这个周期压缩到一两个月呢？因为现在我做设计要三个月。怎么才能把这件事压缩到一两个月呢？因为如果要花一年半，我才能真正把规模做起来，并把软件部署上去，We need, you know, design flows that don't require, you know, $50 million of mass cost until I can get things into prototyping. So you know, to me, how do you compress that to a month or two, right? Because I can now do the design in three months. How can I have that done in a month or two? Because if it takes me a year and a half until I actually get scale and software on it,

主持人：好吧，我对 AI 工作负载的理解已经不再适用于我设计的这款芯片。Host: okay, my understanding of the AI workloads is no longer applicable to the chip that I designed.

嘉宾：没错。你知道，就是说，你知道，所有那些……Guest: Right. You know, so you know, all of those

主持人：你现在就能看到这种情况在人工智能领域实时上演。是的。Host: you see that play out live right now with AI. Yeah.

嘉宾：对。你说过，我们几乎可以从历史角度来看这件事。Graph Core 并不是一个糟糕的设计，但世界已经向前发展了。Guest: Right. You said, you know, it's almost, you know, we can look at it historically. It was like, you know, Graph Core wasn't a bad design, but the world moved on,

主持人：对吧？而且你知道，呃，在这方面……所以，所有这些方面在某种程度上都让“AI 已经让设计这一环节变得容易”这一事实变得无关紧要。对我来说，所有这些制造规模方面的问题都会成为巨大的瓶颈。对吧？当然，另一个瓶颈是内存。Host: right? and you know uh in that. So all of those aspects somewhat obviate the fact that AI has made the design piece easy. All of those manufacturing scale issues to me become a huge bottleneck. Right. Another bottleneck of course is memory.

嘉宾：是的。Guest: Yep.

主持人：没错。你知道内存带宽】【。Host: Right. You know memory bandwidth

嘉宾：我们注意到了。是的。Guest: we noticed. Yeah.

主持人：对。你知道，正如我所描述的，HBM 是一种糟糕透顶的内存。它就是我们……的最佳选择。Host: Right. And you know for that as I've described HBM is a hideous memory. It's just the best one that we

嘉宾：想问你一下那个。是的。Guest: ask you about that. Yes.

主持人：你知道，内存表现很糟糕。你知道，我的位密度很差。我受限于芯片边缘带宽，还有功耗方面的问题，对吧？呃，还有热问题。DRAM 不喜欢太热。Host: You know it's a terrible memory. You know I get terrible bit density. I'm limited to shoreline bandwidth the power right uh issues thermal issues DRAM doesn't like to be hot

嘉宾：把所有东西集中在一个地方Guest: concentrating everything in one spot

主持人：你知道，给予是很糟糕的。Host: you know it's terrible to give

嘉宾：是的，那种 AI 属于内存计算工作负载。Guest: that AI is a memory compute workload yes

主持人：对，所以，对，你知道，为了这个，我得把那个开发和具身化压缩到真实规模，你知道，我得修复记忆。Host: right so right you know for that so I you know I need to you know compress that development uh and embodiment into real scale you know I need to fix memory

嘉宾：对，呃，和……有关。所以在我看来，这是我们要攻克的另一个关键瓶颈。呃，还有功耗，对吧，你知道，功耗方面，你知道，热方面，呃，如今功耗仿真的大多数方面都很差。Guest: right uh associated with So to me that's another critical bottleneck for us to attack. Uh and uh power right you know power in you know thermal uh and most of the power simulation aspects today are bad

主持人：对，你知道，你没有好的 3D 建模工具来分析功耗热点之类的问题。你知道，我的电压，对吧？本质上，我几乎是在为整个功耗范围都留出保护裕量，对吗？Host: right you know you don't have good 3D modeling tools for power power dissipation hotspots uh etc. you know my voltage in right you know I'm essentially guard banding almost my full power envelope correct

嘉宾：对，所以本质上，我有40%的功耗都损耗在保护带上了，所以我得把功耗建模做得好得多；而且我的功耗耗散技术也不怎么样，对吧，所以……Guest: right so essentially I'm losing you know 40% of my power and guard banding so I got to get much better at power modeling you know and then my power dissipation techniques are bad right so you know

主持人：交付模式也必须改变。Host: delivery model has to change too

嘉宾：是的，所以你知道，在我看来，这些都是接下来的重大瓶颈，对吧？我该怎么改进其中的每一项？而现在，如果我已经做到了这一点，好吧，接下来就是一个令人兴奋的新时代了。感谢你们提供这么棒的 AI 工具，但即使这些工具都全面部署了，我仍然面临着巨大的挑战。Guest: yeah so all of these things you know to me those are the you know the next big bottlenecks right you know how do I you know improve each one of those and now right you know if I've done that Okay, now we're on to an exciting era for what's next. So, you know, thank you for those great AI tools, but you know, I got big challenges even when those are fully deployed.

主持人：所以，我们看到芯片数量、芯片架构等等方面都迎来了一种复兴。Host: And so, we've seen sort of a rebirth in the number of chips and chip architectures and so on and so forth.

客人：是的。Guest: Yeah.

主持人：感谢这些工作负载。Host: Thanks to these workloads.

嘉宾：是啊。感觉就像有 100 种 AI 推理加速器芯片，而且我肯定都认不全。Guest: Yeah. It's like there's a 100 AI inference accelerator chips and I'm sure I don't even know them all.

主持人：对，没错。对。所以，所以这里只精品Host: Yeah, exactly. Yeah. So, so

主持人：你们为什么要资助那么多这类项目？Host: why are you guys funding so many of those anyway?

嘉宾：你也承担了自己应付的那一份费用。Guest: You funded your fair share too.

主持人：是的。所以，嗯，那么我们是不是……你知道，他们都值得肯定，因为他们在你提到的组件中，或者……采用了不同的设计假设？Host: Yeah. So, um, so are we, you know, they're all to their credit making different design assumptions either in the components that you talked about or

嘉宾：在……Guest: in the

主持人：在他们瞄准的帕累托曲线那一部分。那么你觉得这些都有生存空间吗？还是说我们正处在这样一个时代，部分原因是 Guido 谈到的 AI 工具，让我们现在有能力打造更多不同类型的芯片。Host: in the part of the para curve that they're targeting. So do you think there's room for all of these or are we in a world where partly because of what Guido talked about AI tooling um we now have the ability to build more different kind of chips.

嘉宾：是的。Guest: Yeah.

主持人：无论你怎么衡量，这段时间都长得惊人。更夸张地说，从历史上看，任何行业都从未出现过100家相互竞争的处理器供应商，对吧？所以……Host: And the time is enormous whichever way you want to even more provocatively historically there have not been a 100 competing processor vendors in any industry ever right. So

嘉宾：未来会变得更加异构吗？我们会看到——这只是暂时现象吗？最终会重新收敛到一种观点吗？Guest: is this a more heterogeneous future? Will we see is this a temporary thing? It'll converge back to a view.

100 种 AI 芯片会收敛为少数几种吗？Will 100 AI chips converge to a few?

主持人：是的。而且你知道，我认为这只是暂时的，最终会趋于一致，这是我的预期。我觉得大概有三个不同的原因。你知道，其中一个原因是，你看到计算环境正在出现异构化：现在有专门用于预填充和解码的硬件。然后人们又说，我们确实需要一个专门用于中填充的层级，而不只是预填充，对吧？所以现在有了早期预填充，还有中填充，这两者是有区别的。Host: Yeah. And you know, I see it as a more temporary thing and it will converge is is what I expect. And there's probably three different reasons that I see that to be the case. You know, one of them is, you know, this emerging heterogeneity that you see in the compute uh environment where okay, now I have specialized prefill versus decode. Well, you know, and now people are saying, well, we really need a specialized tier for midfill, not just prefill, right? You know, so now I have early prefill and now I have midfill, which is, you know, differential.

主持人：也许我们可以听听 OpenAI 在这里怎么说，再推测并验证一下，对吧？Host: Maybe speculate and verify if we listen to OpenAI here, right?

嘉宾：是啊。你知道，突然之间，我的计算集群在不同工作负载上的划分变得越来越细粒度。我认为，历史上任何出现过这种情况的时候，它都无法持续。然后，突然间，你看到了这种趋势，现在人们又在说：“嗯，转向推理模型时，我想要更像 CPU 的东西。”所以，突然之间，我并不想要那种程度的细分——它没有成为我计算能力中的主要部分。我在这边需要更多，对吧？Guest: Yeah. You know, and you know, all of a sudden my compute fleet becomes more and more granular across the workload. And I think anytime that you've seen that in history is not sustainable. And then all of a sudden right when you know as you look at that and now people are saying well as I go to reasoning models I want something that looks more like a CPU again. So you know all of a sudden I didn't want that level you know that didn't become the predominant portion of my compute capacity. I needed more of it over here, right?

以及模型和模型工作流，诸如此类。所以我确实不太喜欢——我会说——大规模专业化这种说法，因为我认为工作负载会继续趋于温和，并且发生非常显著的迁移。其次，我认为如今的模型也即将经历一些进化式突破，对吧？我觉得LLM某种程度上已经快要触及极限了。现在人们开始思考如何真正对3D进行建模，对吧？扁平化的LLM在处理分子、化学物质和成像时效果并不好。所以我认为我们会看到这方面的局限，这会推动算法领域发生不同的转变。我还认为，一些最有意思的工作负载会出现在这样的场景中：你开始把类似HPC的东西重新引入AI领域，突然间，64位精度又变得重要了，对吧？所以我看到……and model and model workflows and you know so on you know so I really don't like I'll say at scale specialization right because I think the workloads are going to continue to moderate you know and migrate so significantly first you know second is I think today's models are about to go through some evolutionary breakthroughs as well right you know I think the LLM is sort of reaching the limits and now as people look to how do I really model 3D right you know where you a flattish LLM doesn't work well when you go to molecules and chemicals and you know imaging you know so I think we'll see limits of that that'll cause different shifts in the algorithmic domain I also think that uh some of the most interesting workloads become where you start you know I'll say bringing HPC like things back into AI like things where all of a sudden you know 64-bit provision uh precision counts again right so I see

主持人：那会是什么样的例子？Host: what would be an example of that

嘉宾：嗯，你知道，想象一下，我的 AI 模型现在正在为化学分析筛选出五个最有意思的领域，对吧？好，这些化学算法将以很高的精度运行，对吧？你知道，关键是要知道哪一部分会成为工作负载的主导部分，对吧？无论是确定我需要运行的五种算法，还是在不同的化学或生物系统上运行这五种算法。嗯，我确实认为，随着我们优化 AI 系统，它们会让我们重新关注一些更传统的高性能计算领域。我也看到了工作负载的这一面。总之，对我来说，计算架构中的这种极端专用化，我不太喜欢，因为我不认为两三年、四年后的工作负载会是这个样子。Guest: well you know imagine that my AI model is now inducing the five most interesting domains for a chemical analysis, right? Okay, those chemical algorithms are going to run with high precision, right? You know, when you know which is going to be the dominant piece of the workload, right? Getting to the five algorithms I need to run or running the five algorithms, right, on different chemical or biological systems. So you know I do think that uh you know or as we optimize our AI systems they will lead us back to some of the things that have been more traditional high performance computing. So I see that aspect of workload uh as well. So you know all of that said is you know this extreme specialization to me in the compute architect I I sort of don't like it because I don't think it's going to be what the workloads will look like 2 3 four years from now.

所以，这是我不认为这些 AI 芯片随着不断细分、越来越专注于计算工作负载中的某些部分，就一定是正确方向的一个原因。你知道，第二点是，现在这类芯片差不多有一百种了。So that's one reason I don't see all of these AI chips as they get more and more specialized for portions of the compute workload to necessarily be right you know second is right there's like a hundred of them now

主持人：对，你知道，有多种光学方案，也有概率型方案等等。它们不可能全都胜出，毕竟说到底，你必须实现规模化。Host: right you know there's multiple optical ones there's probabilistic ones and so on and you know they're not all going to win and they're not all going to win because at the end of the day you have to get scale

嘉宾：关于这些东西，规模化意味着你得赢，得获得资本，还得让工作负载运行在上面。所以我觉得，认为会有一百种这样的东西，这多少有点不合逻辑。因此我认为，它们会因为一些很简单的因素而重新洗牌，比如资本、市场份额等等。哪些团队会胜出，哪些设计和架构会胜出？我认为，工作负载会推动这一切；我也认为，行业自然会发生整合，而我还预计，赢家会挑选一些赢家。Guest: on these things and scale requires you know you have to win you have to get capital you have to get workloads onto it so I think it sort of defies logic that you're going to have a hundred of these things. So I see them you know coming back uh as a results of you know simple you know capital market share etc. you know which are the winning teams you know winning designs winning architecture so I see the workload driving that you know I see the natural you know industry consolidations you know that will occur and then I also expect that the winners will pick some winners

主持人：对，你知道，比如 OpenAI、英伟达和 Anthropic 会说，我喜欢这个，因为它不只是硬件，还关乎硬件和软件代码如何演进。是的，Host: right you know where you know an open AI an Nvidia anthropic will say I like this one because it isn't just hardware it's how do the hardware and software code evolve Yeah,

嘉宾：对。你知道，就这件事而言，我确实觉得会出现……嗯，你知道，这里有十个选项，我可以从中挑一个。我喜欢那个。而且，要利用这些平台，确实需要在软件上投入资金，并推动工作负载不断演进。所以，基于这三个原因，没错，我们会看到这个领域的范围在某种意义上不断收窄。Guest: right. You know, for it and I do expect that there's going to be h, you know, here's this 10 I could pick for this. I like that one. And it does require investment in the software and the workload evolution to take advantage of those platforms. So for those three reasons, yeah, we're going to see, you know, a narrowing of the field uh in that sense

主持人：还有部署的复杂性。抱歉，请继续。Host: and the deployment complexity as well. Sorry, go ahead.

嘉宾：我的意思是，这完全说得通，而且我想大多数观点我都同意。不过我还是想在这里稍微挑起一点争议，纯粹是为了挑起争论——你们已经改过它了，对吧？Guest: I mean, makes total sense and I think I agree in most point. Let me still try to stir a little bit controversy here just to to stirring controversy to you guys have changed it right

主持人：所以，很显然，100家公司不会整合，对吧？而且我完全同意你的观点，归根结底，大型消费者会在某种程度上选出赢家，对吧？我认为如今……Host: so so obviously 100 companies won't access consolidate right and and completely agree with your argument that at the end of the day the large consumers will pick winners to some some degree right today I think

嘉宾：在某种程度上，要成为一家一线芯片初创公司——不管你算不算——取决于你在那些真正的大牌厂商那里占有多大份额。话虽如此，嗯，也有人认为，在任何一种转型期，不同路线可能会在一段时间内大量涌现，最终局面才会逐渐明朗。Guest: to be a first tier chip startup to some degree whether you you're that or not is defined by how much fraction you have with with the with the really big Nice. That said, um, and there's of also an argument to say like, look, in any kind of transition period, you probably have a bit more of an explosion of different approaches over time that that'll shake out.

主持人：话虽如此，我觉得可以提出这样一种观点，大致是这么说的：你看，Host: Now, all that said, I think there's an argument you can make that basically says, look,

嘉宾：从历史上看，我们采用一两种架构作为标准，你知道，因为不同的指令集很难处理。我们不得不构建多个编译器，你知道，基本上就是这样，嗯……软件复杂性决定了我们只能有少数几种硬件平台。Guest: historically, we standardized on, you know, one or two architectures because, you know, different instruction sets were hard. we had to build multiple compilers you know like the this basically um this the the software complexity dictated that you can only have a certain small number of hardware platforms

主持人：这部分似乎正在随着 AI 发生变化，对吧？如今，写一个优化过的内核——我的意思是，我觉得这是个很好的例子：他们基本上是在说，听着，人类已经没法再给这个东西编程了，但一群智能体一夜之间就能做到。这就是我们需要的，对吧？所以，基本上，如果你能造出一款具备某种能力的芯片，那么在它之上构建能够充分发挥这种能力的软件层，现在就容易得多了。这样会不会带来更多一点异构性？毕竟，今后给这种异构系统编程会更容易。Host: that part seems to be changing with AI right today you know writing an optimized kernel for I mean I think was a great example where they basically said look humans can't really program this thing anymore but an agent swarm overnight can do it that's all we need right so where basically you can say like if you can build a chip that has a certain capability it's now much much easier to build the software layers on top that fully leverages that could a result of that be that we have a little bit more hetrogenity just because the you know it's it's easier to program that hetrogenity going forward

嘉宾：而且，而且我确实觉得，这会在某种程度上改变瓶颈所在的位置，对吧？在这方面会有变化。你知道，我首先会指出的瓶颈，就是我们已经谈过的那个：建造这些东西仍然要花一年半的时间。Guest: and and I I do think it changes some of you know you you know the you know where the bottleneck lands right changes in that regard you know the first bottleneck is the one I'd point to that we've already talked about still takes a year and a half to build these things

主持人：对，你知道，然后第二个瓶颈是，要把这些东西装进机架并扩展到足以发挥作用的规模，不仅需要一年半的时间。好吧，那我能借五百亿美元吗？我的意思是，这可不是小额资本支出。说的是数据中心方面非常庞大的资本支出，还有电力供应承诺等等。我得把这套东西大规模建起来，对吧？这也会带来很多异构性。显然，大玩家会设法掌控其中一部分异构性。我们看看英伟达对 Grock 做了什么就知道了，对吧？两年后，你都分辨不出英伟达里面用的哪部分是 Grock 了，对吧？Host: right you know and then the second bottleneck is not only does it take a year and a half to get these things into racks the scales and so on like that now to make them useful full. Okay. Well, could I borrow 50 billion, right? I mean, you're not talking about small capital expenditures. You're talking about very large capital expenditures and data centers, you know, power, you know, commitments and so on like that. Well, I got to build this thing at scale, right, associated with it. So, and that's going to defy a lot of heterogeneity uh as a result. Now, you know, clearly big players and we saw this with what, you know, Nvidia did with Grock, okay? you know, we're gonna sort of own some of that heterogeneity, right? And you know, you know, two years from now, you're not going to tell what Grock is inside of Nvidia, right?

这只会变成另一个类似的东西，对吧？你知道，就像那种东西。所以你会看到这些架构在某种程度上把异构性隐藏起来，对吧？嗯，在架构内部这么做，我觉得完全没问题，对吧？你知道，我的意思是，这就是分层和抽象一直以来所做的事情。但我认为这些力量也很强大，我会说，它们会暴露出很多异构性，对吧？我觉得其中一些异构性会在底层被抽象掉，但归根结底，要大规模构建这些东西仍然很难。It's just going to be like another, right? You know, someone like that. So, you'll see the architectures, you know, sort of hide the heterogeneity, right? Uh inside of it, which to me is just fine, right? You know I mean that's what layering that's what abstraction has always done associated with it you know but I think those forces are powerful ones to I'll say expose a lot of the heterogeneity no right I think it will get abstracted underneath you know some of these but at the end of the day it's still hard to build these things at scale

为什么 HBM 是一种糟糕透顶的内存Why HBM is a hideous memory

嘉宾：之前你提到 HPM 是丑陋的记忆。Guest: earlier you talked about HPM being hideous memory

主持人：这是我们今天迄今为止最好的一个。Host: by far the best one we have today.

客人：没错。这就像民主的问题。Guest: Exactly. It's like the thing about democracy.

主持人：嗯，我们记不清上一次内存创新是什么时候了。它可能已经实现了。我觉得你可能也参与其中。Host: Um, we can't recall when there was the last memory innovation. It might have been obtained. I think you probably had something to do with it.

嘉宾：对，那个死了。所以，Guest: Yeah, that one died. So,

主持人：那个死了。Host: that one died.

嘉宾：我们把它干掉了。Guest: We killed it.

主持人：是啊，我们表现太棒了。我们大获成功。Host: Yeah, we killed it. We killed it.

嘉宾：那么，你认为内存创新很快就会出现吗？或者说，要实现它还需要发生什么？显然，我们知道规模问题，因为它甚至更……Guest: So, do you think memory innovation is around the corner or what needs to happen for obviously we understand the scale problem because it's even more

主持人：是的。Host: Yeah.

嘉宾：那边更大。Guest: larger there.

主持人：是的。不过，即使从技术角度来看，先不考虑规模的话。Host: Yeah. But if you keep the scale aside even from a technology point of view.

嘉宾：是的。我们不妨看看这段历史。我个人大概参与过至少五种不同的新型存储架构。Optane 只是其中一种，对吧？你知道，它最终也没能面世。我大概熟悉过去 30 年来这个行业里出现过的近百种。那过去 30 年里，究竟出现过多少种重大的新型存储技术呢？Guest: Yeah. And let's you know uh you know look at that history. I've probably personally been associated with at least five different new memory architectures. Optane just being one of them right you know that did not see the light of day right uh as well. Uh I'm probably familiar with close to a hundred that have happened in the industry over the last 30 years. And exactly how many major new memories have you had over the last 30 years?

主持人：零。Host: Zero.

访客：零。Guest: Zero.

主持人：对。你知道，那个，你知道，好吧。DRAM、SR、RAM、闪存。好。还有什么？D RAM、SRAM、闪存。是啊。你知道，所以我，我确实认为……Host: Right. You know, which you know, okay. DRAM, SR, RAM, flash. Okay. What else is there? D RAM, SRAM, flash. Yeah. You know, so I I do think

访客：稍微闪一下堆叠控制台。Guest: flash stacking console a little bit.

主持人：几乎没有。是啊。对。你知道，不过总之。对。Host: Barely. Yeah. Right. You know, but anyway. Right.

嘉宾：所以，呃，你知道，在那方面，它它，你知道，一直是，对吧，你知道，一个、一个令人失望的……Guest: So, uh, you know, in that it it you know, it has been right you know, a a disappointing

主持人：我们还没有创造出新的物理机制。是的，能够实现低成本、高性能、可靠且可制造的存储器，对吧？所以我确实认为，而且现在，以及过去三四十年来，存储器一直是个糟糕的行业。Host: that we haven't created, you know, the next physics. Yeah. that enable you know lowcost high performance resilient manufacturable memory right you know for it so I do think and and now and and over the last 30 40 years you know memory was a terrible industry

嘉宾：对，因为你可能有一年赚钱，接下来四年都在亏钱，所以咱们去一个五年里只有一年赚钱的行业搞研发吧，对吧？你知道，这一直都特别难，因为整个行业始终受到剧烈商品化周期的驱动。Guest: right because you would make money one year and then you lose money for the next four right so let's go do R&D in an industry that makes money one out of five years right you know it's just always you know it's just been so terribly difficult because it's been such driven by such violent commoditization cycles.

主持人：不过，最近情况稍微有所改变。Host: That has changed a little bit though recently

嘉宾：在过去5年里。Guest: over the last 5 years.

主持人：开心的日子啊，兄弟。Host: Happy days, man.

嘉宾：所以，所以三大内存供应商都跻身全球最具价值的20家公司之列。Guest: So So all three big memory vendors are on the in the top 20 most valuable companies on the planet.

主持人：是啊，这变化真够惊人的，不是吗？Host: Yeah, that's a crazy change, isn't it?

嘉宾：哦，这实在是太对了。你知道，谁会想到呢？对吧？呃，和它相关，而且任何关注 AI 工作负载的人都会发现，情况总是这样：你知道，预测过去很容易，但预测未来就难一点，对吧？而且你看了之后会说，AI 是一种内存角色。Guest: Oh, it's just so right. you know who who would have thought right uh associated with it and anybody who looks at the AI workloads and it's always you know it's much harder to you know you know it's easy to predict the past but it's a little bit harder to predict the future right and you know in that you look at it and say AI is a memory role

主持人：没错Host: exactly

嘉宾：对，你知道，在它内部，所以我认为，资本和技术需求的结合，将为存储创新提供合理依据。现在有两个领域让我感到兴奋。一个是，你知道，我确实认为，HBM 的边缘带宽限制是一个根本性问题。Guest: right you know inside of it so I think there is the combination of capital and technology needs that will justify memory innovation now there's two domains things that I'm excited about uh you know for it you know one is you know this idea you know I do think that the shoreline bandwidth limitations of HBM is a fundamental issue

主持人：好，那么这个想法，你知道，显然我们有一家叫 Dmatrix 的公司正在做这件事，但我认为其他公司也会，你知道，不管是 Cabbrris 还是他们……Host: right so with that the idea you know and obviously you know we have a company Dmatrix that's doing this but I think others will be you know whether it's you know Cabbrris you know and their

薯片能堆多高？How tall can chips get?

技巧techniques

嘉宾：未来前景很可观，是啊。Guest: the future is stacked yeah

主持人：不知怎么，你得带上记忆和Host: somehow you got to bring memory and

客人：不止一种意义上Guest: in more ways than one

主持人：你知道，你得把内存和计算结合起来。Host: you know you got to bring memory and compute together

嘉宾：对，你知道，从根本层面上，直到……呃，新的结构。嗯，我确实认为，鉴于现在有如此多的资金投入到这个领域——你知道，人工智能领域——我认为我们实际上也能够突破其中一些物理学方面的挑战。Guest: right you know in a fundamental until uh new structures. Um I do think that uh given the workload now has such capital being deployed against it you know in the AI space that I do think we'll actually be able to break through some of those physics challenges as well uh as well.

所以，我确实这么认为。你知道，我最近也创办了一家新的存储器公司，目前还处于隐身模式。不过，我对新型存储器领域即将出现的一些创新相当兴奋。你会看到新材料的出现。存储器领域的很多人都在研究铁电材料等方向，也有人在寻找非电容式高密度存储器，探索如何实现兼具高性能、高密度且可堆叠的存储器。当然，也有人在研究低成本闪存。我个人对这个想法并不是特别热衷。我认为，无论是闪存还是 MRAM 架构，其底层存储单元的速度和性能都存在一些问题。不过，我确实认为，30 年来首次出现的存储器创新即将到来，这会是一个令人兴奋的领域。我们在那里应该会有几家公司，希望我们能一起做成几个项目。你怎么看？So I do believe and you know I've just substited a new memory company as well that's still a stealth but uh you know you know I'm quite excited about some of the innovations that will occur in new memories and you're going to see new materials uh you know for memory people you know a lot of people looking at things like ferro electrics uh you know finding non-capacitive you know uh highdensity memories finding memories that can you know be high performance high density and stackable right but you know thing you know people are looking at uh lowcost uh flash you know for that I'm not particularly enthusiastic about that idea myself I think you know speed and performance of the underlying cell are somewhat problematic with a flash or an MRAM structure you know but I do think there's memory innovations are coming for the first time in 30 years and I do think that's going to be an exciting space and you know we'll have a couple of companies there I'm sure hopefully we'll do a few of them together What do you think?

我们要不要……好吧。你知道，不过我们会在这里找些事情做。但我觉得，30年来首次出现的内存领域创新已经近在眼前了。Should we do Okay. You know, but we'll we'll find some things to do here. But I think memory innovation for the first time in 30 years is nigh upon us.

主持人：嗯。嗯。Host: Yeah. Yeah.

嘉宾：这、这太惊人了，几个100亿美元的市值。我都不知道。你说的是一万亿。好吧，抱歉。Guest: It's it's amazing what a couple of$10 billion of market cap. I don't even know. You say a trillion. Okay. Sorry.

主持人：你知道，我认为内存行业的市值如今已经增长了2.5万亿美元。太棒了。你知道，在过去四年里， 񟿿Host: You know, I think the memory industry has now increased by market cap by $2 and a half trillion dollars. That's great. You know, over the last four years,

嘉宾：我觉得差不多。没错，是的，Guest: I think that's about right. That is Yeah,

主持人：我们本来可以只在那里投资，而不做任何对外的事情。Host: we could have just invested there and not done any forential stuff.

嘉宾：对，没错。用自己的钱。嗯，你觉得这会持续多久？抱歉，你先说。Guest: Yeah, exactly. Use your money. Um, how long what's your view? Sorry, go ahead.

主持人：是的。稍微展开说说堆叠这部分，如果要预测一下，我们的芯片会以多快的速度变得更高？会不会是先做到两层，然后在那里停留一段时间？还是说，我们很快就能看到那种 16 层的“夹心结构”，把逻辑和存储器愉快地混合在一起？Host: Yes. To to expand a little bit on the stacking part if you have to predict h how quickly will our chips get get taller? Is this going to be like, you know, we're going to go to two and then sit there for a while or will we have, you know, the 16 layer sandwich, you know, happily mixing logic, memory,

嘉宾：把这个 HPM 拉下来，英伟达这边正在走下坡路，不过你怎么看它接下来会如何发展？是啊，我觉得这个问题在于堆叠，对吧？现在，本质上说，堆叠的维度越高，堆叠的价值就越大。这意味着，在堆叠过程中，每一层的良率都必须持续提升；本质上，它不能线性增长。没错，对吧？它必须呈指数级增长，制造工艺和缺陷特性也得以指数级的方式改善。你知道，堆到八层或十六层时，算一算相关的数学问题，就会变得非常非常困难。你知道，制造流程必须接近完美。Guest: pulled down this HPM is going down at Nvidia, but how how do you see this evolve? Yeah, you know, I think there's um um you know, the problem with stacking, right, is now essentially the value of the stack, right, becomes greater as you go to higher dimensionality of the stack, which means the yield of any individual layer in the stacking process has to keep coming, you know, essentially it can't go up linearly. Yep. Right. you know, it has to go up exponentially and improving the manufacturing and defect characteristics in an exponential way. You know, when you get to eight or 16 high stacks gets to be so so hard when you run the math on that uh associated with it. You know, your manufacturer flows have to be so close to perfect.

主持人：尽管你可能会看到一些设计会考虑特定的模型成本，就像我们在二维领域早已看到的那样。Host: Although you could see designs that take into account a certain model cost like we've seen it for like a long time for two dimensions.

嘉宾：是的，当然。不过要记住，这本来就是 DRAM 的特性。它们一直都有坏块替换、备用行之类的机制。所以我绝不是在淡化这一点，但即便你做了这些，良率还是得非常高。你知道，如果裸片裂了，我才不管你在里面做了多少冗余设计。它就是一颗坏裸片，对吧？所以，嗯，也正因为如此，我不是那种疯狂追求 16 层、32 层堆叠的人。我觉得最后会有一些不错的三层、四层、五层堆叠，这大概就是最合适的方案。Guest: Yeah, of course. But remember that was the nature of DRAM. they always have bad block replacement and you know spare lines and so on like that. So I'm not in any way, right, you know, minimizing that, but assuming you're doing that, you still need the yields to be so good. You know, if I have a cracked die, I don't care how much resilience you've built into it. You can't, you know, it's a bad die, right? So, you know, those kind, you know, uh, so as a result of that, I'm not, you know, a crazy 16 32 stack, you know, kind of guy. I think there's going to be, you know, nice three, four, five stacks, you know, that just end up sort of being the sweet spot.

主持人：中等规模的。对。对。没错。呃，你知道，就我而言，我预计这部分制造工艺及其复杂性会影响到与之相关的良率最佳区间。所以我认为大多数东西都会落在那个范围里。但即便如此，对吧？你知道，假设现在我有一块很棒的计算芯粒，对吧？那这块芯粒需要做多大？我是说，现在一切都受光罩尺寸限制了，对吧？嗯，我能不能采用更多具有更好良率特性的芯粒，散热性能也更好，你知道，然后你会说，我还想在上面放一叠内存。你知道，是两层堆叠内存还是四层堆叠内存？我不觉得会变成八层堆叠内存。对我来说，两层或四层比较合理，对吧？但随后还需要把供电集成进去，这又成了另一个要素。而且我希望它沿着 Z 轴方向集成进去，因为你知道，我也希望电源轨能接入其中。Host: The mid-rises. Yeah. Yeah. Right. Uh you know for and I and I expect for you know part of its manufacturing part of its complexity uh associated with it you know where's those yield sweet spots associated with. So that's sort of where I think most things land. But even there right you know so let's say I now have you know I now have a great compute tile right you know how big does that tile need to be? I mean everything's become reticle limited now right? you know hm can I go to more chiplets that have better yield characteristics you better thermal you know and then you're going to say I want a memory stack on that you know is it a two stack memory a four stack memory you know I don't see it becoming an eight stack memory you know so to me two or four sort of right you know but then power delivery needs to get integrated into that so that becomes sort of another element and I want that in the Z dimension right associated with it because you know I want my power rails coming into this as well

嘉宾：对，然后我需要相当智能的 RDL 层，来把信号分布到整个平台上。所以某种程度上，你知道，把所有相关部分加起来，我的标准方案已经有大约八层厚了。我觉得这差不多就是我们能做到的极限了，对吧？至少按照我们目前对制造工艺的理解是这样。当然，我们也预计光学技术会加入进来。所以你会看到更多 3D 材料结构，它们会更接近这个复杂结构，因为我希望我的 I/O 也能以深度集成的方式融入其中。这样一来，我的硅基部分就需要由其中一些其他 3D 材料来补充。那么，如何把光互连集成到其中？又如何把电源轨集成进去？Guest: right and Then I need, you know, fairly intelligent RDL layers to distribute signals across that platform. So to some degree, you know, my canonical thing is already about eight layer thick when you add up all of those pieces associated with it. And I think that's about all you can do, right? At least as we understand manufacturing today. And then of course you we expect optics to come in. So you're going to have more 35 material structures, you know, that are going to be closer to this complex because I want, you know, my IO getting, you know, essentially, you know, deeply integrated into that construct as well, you know. So my siliconentric pieces need to get complemented by some of these other 35 materials and how do I get optical connectivity built into this? How do I get power rails, you know, built into that?

你知道，这确实是一项工程壮举，也是一场制造噩梦，对吧？要把所有这些部件都组合起来。不过，随着时间推移，物理规律正把我们推向这个方向。所以，正因为如此，要堆叠 16 层，对我来说简直太疯狂了。你知道，我觉得两层或四层的内存堆叠，大概才是速度上的最佳平衡点。You know, that is a, you know, I'll say that is an engineering feat. and a manufacturing nightmare, right, to bring all of those pieces. But that's where the physics is taking us uh over time. So because of that, you know, throwing a 16 high stack and to me, you know, it's like that's crazy. You know, a two or four high memory stack, you know, I think that's probably where the speed sweet spot will be.

主持人：而且你没有谈到热量散逸。Host: And you didn't talk about the thermal dissipation

嘉宾：以及为此所需的新材料。Guest: and new materials needed for that.

主持人：是的。显然，首先会有很多……你知道，很多事情。你不能再像现在这样出现一些物理热点了。所以，你得更聪明地利用和分散热量。不过接下来你会说，好吧，我该怎样既向这些环境输入电力，又把电力耗散出去？顺便说一句，如果我能以我想要的电压更高效地输入电力，同时获得更好的功率响应和电压调节，那么我产生的热烟囱就会少一些。Host: Yeah. And obviously there's going to be a lot of, you know, you know, first thing, right? You just can't have some of the, you know, the physics hotspots that you have today. So you have to be more intelligent about how you use the the heat and spread the heat. But you know then you're going to be saying okay how do I both get power in and dissipate power out uh of those environments. And by the way if I can you know de you know get much more efficient power in at the voltages I want with much better you know power responsiveness and my voltage regulation you know then I don't create as much thermal chimneys.

嘉宾：是啊。不过我确实得想想，怎么给这些东西降温。你知道，人们提出过新材料、金刚石以及其他东西，认为它们能帮助我们更好地散热。那我接下来又该如何采用更好的液冷技术呢？还有其他方法，你知道，直接装在设备上，具备良好的热特性。所以，没错，你知道，我们工程师都快成水管工了，对吧？Guest: Yeah. But I do have to think okay how do I cool these things and you know what role do new materials diamonds and other things have been postulated as ways to you know do a better job of the thermal cooling how do I then have better you know liquid cooling techniques and others you know sitting right on it you know with good thermal characteristics so yeah you know uh you know we we engineers are becoming plumbers right

主持人：没错，确实，我们得关注 MEC，也就是芯片级的。Host: exactly yeah we got to worry about MEC the chip scale

主持人：我想问一个关于光互连的现象。有些人说，咱们把内存从封装上移开，让计算单元和内存之间完全采用光互连，这样就能有更多的灵活性，也有更多种构建方式。你怎么看这种做法？与堆叠相比，这可是另一个极端。Host: let me ask about an operate phenomenon which is some people have said hey let's get the memory off of the package and just go all optical between the compute and the memory and that'll give you more degrees of freedom and more ways of building things what's your view on that as opposed to stacking that's other extreme

嘉宾：是啊，你知道，我觉得 OEO 接口越多，也就是光电转换、再电光转换的接口越多，你知道，在核心计算集群中需要的这类接口越多，情况就越难处理。在这个意义上，你总会因此产生损耗。你知道，人们正努力想出更巧妙的技术，来降低成本和功耗。比如说，我在经过 DAX 和各种转换时，到底浪费了多少功耗？所以我不太喜欢这些方案，因为我觉得它能让你访问大量内存，但为了做到这一点，我要消耗很多功耗。Guest: yeah you know I think the more OEO right uh interfaces that you have you know it's you know you know optical to electrical to optical to you know the more of those that you demand in your core compute complex you enmy is just hard right uh in that sense and you're always going to have losses associated with it you know people are you know working hard to come up with more clever techniques to decrease you know both the cost and the power right you know how how much of the power am I wasting as I go through my you know my DAX and uh you know conversions so I'm not a fan of those you know because I think it gives you access to large pools of memory right but I I burn a lot of power getting there.

主持人：是的。Host: Yeah.

嘉宾：对。记住，我的——你知道，本质上，每次计算的能耗现在其实低得惊人。对。每比特通信的能耗相比之下要高出一千倍，对吧？而且如果我得花那么多能耗去访问大型存储池什么的，我自己是不喜欢这些方法的。你知道，我觉得它们在某种程度上违背了解决方案所依赖的基本物理规律。我是说，还有另一种思路：我们能不能真正把计算和内存直接集成起来，对吧？把一部分计算转移到内存里……Guest: Right. And remember my you know the essentially you know the phentojoules per compute is now actually pretty amazingly good. Right. The phentojlesw per bit of communication is comparatively a thousand times worse right and if I'm spending that much to go get to large pools of so I don't like those approaches you know myself. I think they somewhat defy the underlying physics of the solution. I mean and then there's sort of the other approach to say like look can we actually integrate compute and memory directly right shift some of the compute into the memory it's

主持人：一些 PIM 方法Host: some of the pim approaches

嘉宾：对，没错，那又怎样？Guest: yeah exactly right so what

主持人：你对此有什么看法？Host: what are your thoughts around that

嘉宾：我也不太喜欢那些方案，对吧？因为我觉得工作负载方面，你知道，现在我得把我的工作负载拆开，然后把这些小块计算任务推到内存里，对吧？这样一来，我就不需要那么大的内存带宽把数据送到真正的计算单元里，对吧？这会给工作负载带来限制。当然，这方面我也可能说错了，尤其是考虑到我们现在能够通过设计层面带来的某些设计灵活性和智能。不过，大多数 PIM（存内计算）构想，你知道，已经存在了 25 到 30 年。我觉得再过 25 到 30 年，它们大概还会继续存在，对吧？从这个意义上说，我希望能有出色的内存带宽，同时实现高密度、高性能，并让内存靠近真正的计算引擎。Guest: I'm not a fan of those either right um because I think workloads you know because now I have to take my workload and I have to like push these little pieces of compute into the memory right you know so that you know I don't need as much bandwidth of memory coming into my real compute Right? And that puts constraints on workloads. And again, you know, I could be wrong on this, particularly some of the design flexibility and the intelligence that we can now bring, you know, through the the design aspects. But most PIM ideas, you know, have been around for 2530 years. And I think 2530 years from now, they'll still be hanging around, right? In that sense, you know, give me great memory bandwidth, you know, that I can have, you know, high density, uh, high performance, and get it close to the real compute engines.

我认为，这将是一个更具普适性的解决方案，能够适应随着时间推移不断变化的各种工作负载。I expect that's going to be a more generalizable solution across many and evolving workloads over time.

主持人：嗯，所以我们仍然受困于越来越高密度的芯片、越来越强的算力、越来越多的逻辑单元，以及越来越大的功耗，Host: Well, so we're still stuck with denser and denser chips, more compute, more logic, more power,

嘉宾：而且 IO 也好得多。Guest: and much better IO.

主持人：现在，我是，我是光学技术的忠实拥护者，你知道，就是把所有 I/O 功能都转向光学技术。没错。对。你知道，大约 25 年前我就宣告铜缆时代终结了。总有一天，我会证明自己是对的。Host: Now, I am a I am a huge fan, right, of opt, you know, moving to optics for all IO function. Yeah. Right. You know, I declared the death of copper about 25 years ago. Eventually, I'll be right.

嘉宾：你还是对的。总有一天，很快就会有那么一天，呃，我会是对的。但我觉得，是这样吧？你知道，在我看来，所有 I/O 都应该走光学，对吧？你知道，但核心计算和内存这个复杂部分不是这样，对吧？Guest: You're still right. one day one day soon uh I'll be right but I think is that a you know to me all IO should go to optical right you know but that core compute memory complex no right

主持人：那么，要实现全光化，第一步是进一步纵向扩展，而不是横向扩展吗？嗯，纵向扩展要比横向扩展做得更多？Host: so when do you the first step for it going um all optical is to scale up even more so than scale out

嘉宾：你对此有什么看法？Guest: what's your views on that

主持人：是啊，你知道，我觉得我们现在……Host: yeah you know and I think we're now

嘉宾：那会是什么样的应用，或者说工作负载驱动因素呢？嗯，我认为工作负载驱动因素就是我们眼前这个。Guest: what would be the like the application or the workload driver if you will Well, I think the workload driver is the one in front of us.

主持人：好的。Host: Okay.

嘉宾：对。我的意思是，你知道，如今在我们的规模扩展环境中，我们铺设了大量铜缆，对吧？本质上，铜缆正在变成越来越短的波导，对吧？突然间，要让铜缆在呃、你知道，5 米的距离上正常工作，成本比让光纤在 100 米的距离上正常工作还要高，对吧？所以，你知道，我认为如今是物理规律在推动我们走向这种转变。Guest: Right. I mean, you know, today, you know, our scaleup environments, we're putting tons of copper, right? You know, and essentially copper is becoming a wave guide that's getting shorter and shorter, right? And all of a sudden, you know, to make my copper work at uh, you know, 5 m is more expensive than making optical work, right, at 100 meters, right? You know, so you know, I think the physics is driving us there today,

主持人：你知道。而且，如果是这样的话，Pat，我们为什么还没转向那里呢？是啊，这是条复杂的供应链，你知道，它从未经过大规模验证。我想现在大家都在考虑某种形式的 NPO、CPO 等方案，时间大概在 2028、2029 年。因为你知道，如果不做出这种转变，我就没法在我的纵向扩展环境中扩展大型 radix 计算集群。所以，显然，英伟达已经表明了这一点。我想，业内其他人也都认为那会是转折点。实际上，我根本就不该造 NVL72。你知道，它是工程上的奇迹，却是制造上的噩梦，对吧？你知道，整个行业花了一年半的额外时间才消化掉这个庞然大物，对吧？而我们竟然能把它扩展到如此规模，真的很了不起。Host: you know. And you know so if that's the case Pat why haven't we moved there already? Yeah it's a complex supply chain that you know has never been tested at scale and I think now you know I think everybody is looking at you know some form of NPO CPO etc you know in the 28 29 time frame because you know I can't scale up to large radics you know compute uh clusters for my scale up environment without making that move. So, you know, clearly Nvidia has uh indicated that. I think everybody else in the industry expects, you know, that's the conversion point. You know, in reality, I should have never built an NVL72. You know, it's an engineering marvel and it's a manufacturing nightmare, right? You know, it took the industry, you know, one extra year and a half to digest that beast, right? And it really is amazing that, you know, we're able to get it to scale, you know, so on.

但你知道，这确实证明了，如果当时我们拥有更成熟的光学供应链，今天就会先进得多。我们今天就能构建更好的规模化环境，你知道，更好的功耗表现、更低的成本、更高的能效。但当时我们的光学供应链还不够成熟，无法支持这一切，而这些问题并不是轻而易举就能解决的。还有很多工作要做，不过我认为 28、29 年会是关键年份。But you know that really proved that had we had a more mature optical supply chain, we'd be much more advanced today. We'd be able to build better scale up environments today, you know, better, you know, power, better cost, better energy. But we didn't have the maturity of the optical supply chains to allow that and those things are not trivial to get resolved. There's a lot of work to happen, but I think 28 29 is the year

嘉宾：容量。很好。用于 CSS。Guest: capacity. Good. for CSS.

主持人：我们——我们可能正在积累很多关于网络中的封装内光学技术如何运作的经验，对吧？而且我认为其中很多经验都能很好地转化到……转化到……Host: We we're probably gaining a lot of experience how in package optics work in networking, right? And and I think many of that will translate nicely to to

嘉宾：GPU。我确实这么认为，但突然之间，你知道，这就变成了：好吧，这些东西得能在一千万颗 GPU 的规模下运作。哇。没错。你知道，它们得装进封装里，而且还有很多其他问题。许多形式的光学技术都受不了糟糕的热条件。可你猜怎么着？对它来说，糟糕的热条件恰恰是常态，你知道。所以我们得处理热问题，还有封装集成问题。我们还面临激光产能不足的问题，整个行业都缺。所以问题确实不少，但我不觉得其中有什么根本性的障碍。现阶段就是得下很多苦功，把所有这些环节整合起来。Guest: GPU. I do think but all of a sudden, you know, it's like, okay, these things have to work, you know, at 10 million GPU scale. Wow. Right. You know, it has to fit in packages and, you know, there's just a lot of things there, you know, and many forms of optical don't like bad thermals. Well, guess what? Bad thermals is the nature of the game for it, you know. So we have thermal problems to deal with, you know, package integration problems. We didn't have enough, you know, laser capacity in the industry. So a bunch of things, but I don't see anything there fundamentals. It's just a lot of hard work at this point to pull all of those pieces together.

主持人：从网络架构的角度来看，这是否也意味着网络会转向更多采用电路交换网络？毕竟，你知道，这些速度极快的光网络不可能无限扩展。嗯，你知道，我喜欢……嗯，你知道，呃，Nick Mune，你知道，也是这里斯坦福大学的另一位教授，是我的好朋友。你知道，我是说，你也稍微认识他，对吧，Guido，你知道……Host: In terms of the network architecture, will this also mean a switch towards more circuit switch networks just because, you know, you you can't scale these these very fast optical networks indefinitely? Well, you know, uh, you know, I like, uh, you know, uh, Nick Mune, you know, another Stanford professor here, good friend of mine, you know, as well, you know, I mean, you know him a little bit also, uh, Guido, you know,

主持人：他是顾问吗？Host: was he adviser?

客人：他不是顾问。Guest: He was not the advisor.

主持人：是的，是的。不过，你知道，在我看来，他说得很到位。我们构建网络，是为了能够处理发往任何地方的任何数据包，而无需知道它可能会去哪里。你想想人工智能，情况几乎恰恰相反。没错。我们处理的是规模很大、可预测的数据流。Host: Yeah. Yeah. So, but, you know, to me, he describes it well, you know, we built, you know, uh, networks for to be able to handle any packet going anywhere with no knowledge of where it might go. When you think about AI, you know, it's almost exactly the opposite. Yes. Right. You know, we're doing flows that are large and predictable.

嘉宾：所以，从这个意义上说，仅仅是这个描述就表明，嗯，它看起来更像是一个开关，而不是一个数据包。Guest: So, in that sense, just the very description says, hm, that looks more like a switch than like a packet.

主持人：所以，我确实认为，转向光技术的趋势正在汇聚，对吧？而且，你知道，工作负载要可预测得多、易于管理，你知道，规模大且流量……你知道，对我来说，我确实看到了这一点。是的，我们会一路推进，把光技术引入计算集群；也会一路推进到 OCS，或者某种形式的光交换——你知道，流量更可预测。我的意思是，这才是正确的架构。然后你就会问，这是纵向扩展还是横向扩展，在某种程度上……Host: So, I do think this confluence of moving to optical, right? And, you know, having a workload that is far more predictable, manageable, you know, large and flow, you know, to me, you know, I do see that. Yeah, we will go all the way to bringing optical into the compute complex and we will go all the way to OCS or some form of optical uh switching you know with more predictable flows you know I mean that is the right you know architecture and then you sort of say is that scale up or scale out and to some degree

嘉宾：就说它在那里某种程度上消融了边界。Guest: say it sort of dissolves the boundary there

主持人：没错，如果我的纵向扩展规模足够大，那确实如此。你知道，我只是在说，好吧，我的纵向扩展环境的基数是多少？我什么时候想把它连接到其他规模很大的集群？对我来说，这最终会成为一个相当漂亮的未来答案。呃，你知道，不过我现在请教的是一位在这方面比我专业得多的人。所以……Host: yeah right if my scale up is big enough then yeah you know just saying okay what's the radics of my scale up environment and when do I want to hook it up to other clusters that are really big and to me that ends up being a pretty beautiful answer uh you know to the future but I'm looking at somebody who's way more expert on this than myself. So

嘉宾：不，我，我，我是这么看的，我一直觉得，scale up 和某种程度上的 scale up 之间的区别是人为的。你明白这种区别是怎么来的，对吧？我们使用的协议不太一样，一个是前端，一个是后端网络，等等。所以现在从实际角度来说，这确实是一个很大的区别。Guest: no I I I look I always found scale up versus scale up to some degree an artificial distinction. You understand where it comes from right is we're kind of using different protocols one is front end one is backend network and so on. So right now practically speaking it is a it is a big difference.

主持人：是的。Host: Yeah.

嘉宾：如果你退一步，从系统架构的角度来看，你会发现它实现的是同一个目标，只是形式和协议略有不同。我认为，随着时间推移，它们会逐渐趋同。Guest: If you take a step back and look at it from a system architecture perspective it achieves the same goal you know with slightly different flavors slightly different protocols that over time I think will converge.

主持人：是的。Host: Yeah.

嘉宾：我的意思是，尤其是针对 AI 训练工作负载。Guest: I mean especially for the AI training workload.

主持人：是的。不过，我是说，训练和推理实际上看起来相当相似。Host: Yeah. Although I mean training and inferences actually look fairly similar.

嘉宾：推理公司不那么……Guest: The inference corp is less of

主持人：是的。现在我接着说，你知道吧。你的大脑会停止学习吗？Host: Yeah. Now I'll go on a you know right. Does your brain stop learning?

嘉宾：是的。Guest: Yeah.

主持人：对。你知道，你知道，你懂的。顺便说一下，作为工作负载演进的另一个特点，我确实认为，训练和推理的区分今后会变得不那么明显，而不是更加明显，对吧？这也是反对采用越来越多异构架构的一点理由。我认为，持续学习会成为某些算法领域的特点；也就是说，算法本身会是这样：我确实希望推理环境能够更新我的模型权重，因为这样能让我不断改进、不断演进，适应那些转向不那么正式的训练环境的工作负载。这些环境中的数据集也会更小。所以，我不确定这是不是……Host: Right. you know you know when you know by the way as another characteristic of workload evolution I do think the you know the separation of training and inferencing is going to become you know less so going forward not more so right that's a little bit of the argument against you know you know more and more heterogeneous architectures is I think continuous learning you know I think there'll be algorithmic domains that will say yes continuous learning you know becomes the nature of the algorithm where yeah I do want my inference environment to be updating my model weights because you know that's giving me more and more refinement you know evolution across you know uh workloads moving into environments that are less formalized training uh as well we'll have smaller data sets in those environments so you know I'm not sure that's a

嘉宾：但是，要达到实用程度所需的节点数量，对于推理来说，其直径比训练时更小——这就是剩下的部分。Guest: but the number of nodes you need in order to be useful is a smaller diameter for inference than than for training that's what remains

主持人：今天确实如此。Host: today that's true

嘉宾：嗯嗯Guest: yeah yeah

主持人：我是说，我仍然觉得，换个角度来看，异构性可能也有其作用。不同类型的 AI 工作负载，对底层硬件的要求可能大不相同：有些更受计算能力限制，有些则更依赖内存带宽。比如说，如果你有一个小型扩散模型工作负载，那么适合它的最佳显卡，看起来就会和顶级的、拥有 15 万亿——抱歉，15 万亿个权重的 LLM 截然不同；后者需要在极大规模下运行。所以我觉得，如果有什么因素会推动这种异构性的话，也许就是这个。Host: I mean I I still think there might be a role for hetrogenity coming from a different angle which is Different types of AI workloads require very different underlying like some are more comput limited some are more more memory bandwidth if you have a small diffusion workload or so right the optimal card for that looks very different from you know a top-of-the-line you know 15 trillion 15 sorry 15 trillion weight LLM right that needs to operate at massive scale right so it's I I think that that might driveity if anything

嘉宾：是的，我觉得这方面还是有一定空间的。不过我也想说，好吧，现在我说的是要在我的数据中心里投入一百万个 GPU，对吧？你会说，嗯，对于那个特定的扩散工作负载，我确实应该配置一些专门针对这种方式优化的东西，对吧？所以，我现在要开始部署一千个这样的设备，对吧？而针对那个工作负载，我还需要部署两千个这样的设备，嗯，还有一些其他需求。突然之间，你就失去了某些规模扩展上的灵活性，对吧？而且，你知道，管理、升级、连接这些设备要花多少钱？故障域又该怎么处理？还有诸如此类的各种问题。Guest: yeah I I think there's some room for that you know but I'd also say okay now I'm talking about putting a million GPUs to work in my data center, right? And you say, well, you know, for that particular diffusion workload, you know, I should really put something optimized this way, right? So, I'm now going to start, you know, putting, you know, a thousand of those in, right? And for that workload, I need, you know, 2,000 of these in and, you know, for some, you know, and all of a sudden, you know, you don't get some of the, you know, the flexibility of scale, right? that you would say and you know how much does it cost me to manage and upgrade and you know connect and you know fault domain you know all of those types of things

主持人：代理将完成所有升级，这样就不会再有问题了。Host: the agents will do up all the upgrades so there's no longer a problem

嘉宾：是的是的，谢谢，一切都变得很简单，外面真好。Guest: yeah yeah yeah thank you everything gets easy so good out there

主持人：是的，舰队管理已经完成了。嗯，我们刚才讨论了计算、内存、网络，接下来是什么？Host: yeah so fleet management is done um so we talked about what compute memory networking power is the next one

能源能力等同于经济能力Energy capacity equals economic capacity

嘉宾：对，对，对。Guest: yeah yeah yeah

主持人：你也提到了芯片内部的垂直传输，以及芯片上的芯片。Host: and you touched upon it with vertical delivery inside the chip on the chip.

嘉宾：但我的意思是，有 800 伏直流电，而且电力会输送到数据中心。Guest: But I mean there's 800 volt DC, there's power delivery to the data center.

主持人：嗯，整个生态系统涵盖了发电以及相关的社会政治层面，等等。Host: Um there's an entire ecosystem all the way to power generation and and the sociopolitical dimensions of that and so on and so forth.

嘉宾：嗯。嗯。Guest: Yeah. Yeah.

主持人：那么，创新在哪些方面发挥作用，哪些又只是纯粹的执行？嗯，你知道，如果我们从第一性原理出发，你知道，我们国家在能源容量这件事上做得一塌糊涂。你知道，基本上有 10 到 15 年的时间，我淘汰煤电的速度，基本上和新增可再生能源的速度一样，所以从整体能源容量来看，我们国家基本上停滞不前。这可不是什么好事，因为在 AI 数字时代，能源容量就等于经济容量，对吧？所以基本上，如果你接受这个论点的话，我国的经济容量有 15 年都没有增长。我觉得过去 5 年的数据很能证明这一点。你知道，我们看到大量资金涌入，大家都在说：“给我更多容量。”不过，你知道，我觉得即便如此，我们的全国能源容量可能也只是在以每年 4% 的速度增长。Host: So where does innovation come in and where does like its pure execution? Well, you the you know and if we start at you know uh first principles here you know our our nation has done a terrible job with this energy capacity right you know and essentially we went for 10 to 15 years where essentially I was taking coal offline at the rate I was adding renewables and essentially the nation was flatlined in terms of overall energy capacity. Not good because in a AI digital age energy capacity equals economic capacity right so essentially my economic capacity as a nation flat for 15 years if you buy that thesis right which I think is very you know provable over the last 5 years I you know we've seen this enormous influx into just give me more capacity uh but you know I think uh you know even with that we've gone to maybe increasing our national energy capac capacity 4% per year

嘉宾：类似这样的东西Guest: something like that

主持人：从基本上是从 0 到 1，对吧？哇，是 4 倍，还是嗯，4%？对吧？所以在我看来，这种情况不太好，我们只是需要更多的能源容量。Host: going from you know essentially 0 to one right wow 4x or hm 4% right so to me you know that you know it's a bad situation so we just need more energy capacity

嘉宾：呃，第一，显然，这个领域里有一些公司。你知道，我们谈过核能运营商，比如 Alva。嗯，我觉得核能作为基荷能源非常棒，我们希望建设更多核能设施。遗憾的是，我们所有的可再生能源都严重依赖中国，这对我们来说很不幸。你知道，燃气轮机的交货周期现在竟然要八年，对吧？所以，我们要迅速扩大规模，面临的环境确实很艰难。但首先，我们需要更多产能。是的。从根本上说，能源产能在某种程度上限制了我们对 AI 能有多兴奋，对吧？因为你想，建一个新的数据中心，再买一百万个 GPU，可要是没法给它们供电，又有什么意义呢？Guest: uh for it so number one and obviously we have some companies in that space you know we talked about nuclear operating as one of those with Alva um and uh you know I think nuclear right you know as a base load is you know fabulous thing that we want to go build more of. You know, unfortunately, all of our renewables have, you know, deep dependencies in China, which is very unfortunate uh you know, for us. You know, we're, you know, uh, you know, gas turbine lead times are only eight years at this point, right? So, you know, you know, I mean, we really have a difficult environment to scale up rapidly, but one, we just need more capacity. Yeah. And fundamentally energy capacity somewhat dampens how euph fork we can get on AI, right? Because you know why build a new data center and buy, you know, the million GPUs if I can't power them, right?

我认为，你会看到越来越多的数据中心项目违约，因为届时将没有足够的能源。And I think you're going to see more and more defaults happening on many of those data center projects because the energy won't be there.

主持人：所以你认为这会构成显著的阻力？Host: So you think it'll be a significant headwind?

嘉宾：我认为这会形成阻力，而且我觉得你已经开始看到一些最初的迹象了。Oracle，你知道，呃，Comet，你知道，刚刚……我认为这只是你接下来会看到的许多例子中的第一个。当然，我们不会因此放慢脚步，但我觉得人们会开始问：我真的有精力去承担我正在做出的资本承诺吗？比如，为数据中心机架和 GPU 采购水泥之类的投入。所以我确实认为，这会成为一个上限。Guest: I think it will be a headwind and I think you're already starting to see some of the first indicators of that. the Oracle, you know, uh, comet, you know, was just I and I think that's the first of what you're going to see many. Now, it's not like we're going to slow down as a result, but I think people will start saying, do I really have the energy, you know, to underwrite the capital commitments that I'm making, you know, for, you know, cement for data center racks and, uh, GPU purchases. So I do think it's you know going to be a you know an upper bound

主持人：但我们需要更大的能源容量，也需要在这方面推动更多创新，还需要把更多这类供应链布局在美国。你知道，核能是我最喜欢的领域之一。你知道，美国上一次有核反应堆投入运行是什么时候吗？对，你知道，已经是20年前了。Host: but we need more energy capacity and we need to get more innovation into that and we need more of those supply chains in the US you know nuclear one of my favorites you know when was the last nuclear reactor that came online in the US right you know it's 20 years ago

嘉宾：对，你知道，我们刚刚叫停了一个非常有能力的行业，所以这是我们需要做的一件事。我们需要更高效地建设电力网络，并推动数据中心转向 800 伏直流供电。没错。你知道，我们还可以因此省去沿途许多转换环节，这些环节之所以存在，是出于历史原因和标准化方面的考虑。好吧，我们得解决这个问题。800 伏直流，爱迪生的复仇来了。Guest: right you know we just stopped you know a very capable uh industry so that's one we need more uh you know we need to get much more efficient at delivering you know the power networks and as you you know, moving 800 volt DC data centers. Absolutely. You know, right, you know, and being able to eliminate so many conversion steps along the way because of uh of uh uh historical uh you know, reasons and you know, standardization reasons. Okay, we got to fix this. You know, an 800 volt DC, the revenge of Edison is upon us.

主持人：是的，我正想说，Host: Yes, I was just going to say,

嘉宾：是的，我想这么说。你知道，我确实认为，最终我们需要更好的电源转换技术。我们有几家公司，其中一家做垂直结构的氮化镓技术。我认为这会是一项非常出色的技术，能够通过一次转换，将 800 伏降到 48 伏、12 伏，甚至 5 伏，对吧？这样能为你带来更多能量。你知道，我们之前也提到过……Guest: yeah, I like to say so, you know, I do think that ends up being we need better power conversion. You know, we have a couple of companies, one of those vertical, you know, vertical GAN, you know, I think will be, you know, a killer technology, you know, to be able to do 800 to 48 or 800 to 12 or even 800 to 5 in one conversion step, right? Which just gives you a lot more uh energy, you know, we touched on

主持人：固态变压器，你知道的。Host: solid state transformers, you know.

嘉宾：是啊。你知道，所有这些，比如固态变压器、开关等等，都是在重建供电网络。我觉得这会带来很多创新，有点像是，电力又变酷了，对吧？你知道，对它来说是这样。而且冷却系统里也会需要所有这些东西，对吧？所以会涌现出一大批创新，你知道，涡轮机、制冷剂什么的，突然之间又成了令人兴奋的新技术。所以，对我们这些搞硬件的人来说，眼前就是一场复兴。这只会是……Guest: Yeah. You know, all of those are, you know, solid state transformers, switches, etc., you know, rebuilding, you know, the power delivery network. And I think that will, you know, have a lot of, you know, innovation, you know, it's sort of like, okay, power's cool again, right? You know, you know, for it. And then you're going to need all of those in the cooling systems as well, right? You know, and so there'll be a bunch of innovations and that, you know, so you know, you know, turbines and refrigerants and, you know, all of a sudden they get to be, you know, exciting new technologies again. So, you know, for all of us hardware guys, this, you know, this, you know, is like renaissance in front of us. It's just going to be

主持人：25年后。我不得不回去读读关于汽车的汽车的内容。Host: after 25 years. I had to go back and read about car's car

主持人：卡诺效率。哦，是啊。它算是好东西吗？热力学第一定律又是怎么回事来着？Host: carinal efficiencies. Oh yeah. Is it like good stuff? How does that first law of thermodynamics work again?

客人：这里毕竟还是有点什么，Guest: Something after all here,

主持人：你知道。所以，这会是一个令人兴奋的时期，你知道。不过，我们也常说，你知道，我们需要改变物理原理，对吧？因为成本一直没有改善。你知道，过去五代英伟达芯片，每太拉浮点运算的基本功耗都停滞不前，这不行，对吧？这也是我们涉足某些领域的原因。你知道，我们有一家超导公司 Snowcap，它让我们有机会从根本上改变物理原理，实现功耗性能提升一千倍。所以，我确实认为，未来不久我们就会迎来其中一些创新。Host: you know. So, so it's going to be you know exciting time that way you know but you know you know we also say you know we need to change the physics right because cos hasn't improved you know the essential power per terror you know power per teraflop for the last five uh generations of Nvidia chips you know flatline not okay right and that's where some of the things you know we have a superconducting company snowcap right which affords the opportunity to fundamentally change the physics you know a thousand times better power performance. So, I do think some of those innovations are upon us in the near future as well.

嘉宾：过去这一小时里，我的时间管理糟透了。还有好多话题没聊到。我们没能一一谈完，但离开前至少得谈一个。Guest: I've been a terrible time manager here for the past hour. So, so many topics yet to cover. We didn't cover, but we got to cover at least one before we leave.

主持人：好的。Host: Okay.

一个为智能体打造的 VMwareA VMware for agents

嘉宾：因为现在Guest: Because now

主持人：由你负责。Host: you're in charge.

嘉宾：是的。所以，我想，嗯，现在虚拟机又回来了。Guest: Yes. So, I thought um now VMs are back.

主持人：作为 VMware 的前首席执行官，你怎么看这件事？Host: How do you think about that as a former CEO of VMware?

嘉宾：嗯，你知道，我，我呃，我显然觉得自己被排除在这场对话之外了，如果我们继续这样的话。Guest: Well, you know, I I uh I feel excluded from this conversation obviously if we go on

主持人：其中一部分是，我本来也打算谈谈人脉拓展，不过我觉得我们没时间了。Host: part of it was I was going to talk about networking as well so but I don't think we have the time

嘉宾：你知道，嘿，你知道，我觉得每一次创新总会带来下一个抽象，对吧？我认为这里面有很多这样的想法。而且你知道，我总喜欢说：你知道，今天我们这个世界里有什么新东西？数据结构、算法、抽象。你知道，我们把它们追溯到那些基础概念。嘿，你知道，虚拟机抽象，无论是在基础设施层面还是应用层面实现，我认为它仍然是一种基础性的抽象模型，理应始终在计算层级中占有一席之地。所以，你觉得我们是不是该回去再跑一次 VMware？Guest: you know hey you know I I think the you know every innovation has always led to the next abstraction right and I think many of these ideas and you right you know I always like to say you know you know what's what's new in our world today you know data structures algorithms you know an abstraction you know you know we bring it back to those you know foundationals and hey you know I you know the virtual machine abstraction you know whether it's done from the infrastructure level or from the application level you know I think it's still a foundational abstraction model you know that deserves to always have a place uh in the compute hierarchy so you know so what do you think one of us go back and run VMware again

主持人：是啊，我就是这么说的。我想我说过，是时候回去再运行一次 VMware 了。Host: yeah that's what I tell I think I told it's time to go back and run VMware again

嘉宾：是啊，是啊，还有，呃，你知道，我确实觉得有些这样的抽象层，不过你也可以想想，比如我们在 VMware 做过的一件事：我们基本上管理了整个计算层级结构的方方面面。Guest: yeah yeah and uh you know and I do think some of these abstractions you know but you also think about it and like one of the things we did at VMware where we you know essentially you know managed every aspect of the computing hierarchy.

主持人：是的。Host: Yeah.

嘉宾：没错。你知道，你要管理工作负载、管理网络、管理存储系统，还要考虑如何将它们抽象化。我认为在 AI 领域，对吧？我们会想到这些智能体集群。那么，谁来管理这些智能体？谁来为所有智能体创建安全配置？谁来管理智能体的性能？你知道，虚拟化和管理的每一个基础要素，基本上都需要在这一层新的计算体系中重新构建，而且会有很多优秀的公司在这些方面脱颖而出，推动人们实现这些目标。Guest: Right. You know and you manage the workload, you manage the network, you manage the storage you know system and how do you abstract those? And I think in the AI context, right? We think about you know these agent swarms. Well, who's going to manage the agents? Who's going to create the security profiles around all of the agents? Who's going to be manage the performance of the agents? You know, essentially every fundamental element of virtualization and management needs to get recreated in this next computing hierarchy and there's going to be lots of wonderful companies that you'll sort of break through for how those things will be, you know, done.

主持人：好。你想以人脉交流作为结尾吗？Host: Okay. Do you want to end it on networking?

嘉宾：其实，我想再聊一会儿 VM。对我来说，最有意思的是，我们现在构建 VM，是为了让智能体能用，而不是为了让人类能用。我们在其他一些公司看到的是，这会很大程度上改变产品形态，也会很大程度上改变它们的营销方式，对吧？你希望智能体替你选中你的 VM 产品。Guest: Actually, I'd love to stay with the the VMs for a second. And the most interesting thing for me is that we're now building VMs so that they're usable for agents as opposed to usable for humans. What we've seen in sort of other companies is that that changes a lot the form factor that changes a lot how you market them, right? You want you want the agent to make the pick for for for your VM offering.

主持人：嗯，它会很大程度上改变你能处理的复杂度，也会改变启动应该有多快。人类比智能体有耐心得多。Host: Um it it changes a lot how much complexity you can have. It changes how how quickly the startup should be. Humans are a lot more patient than than agents.

嘉宾：呃Guest: Um

主持人：那么，如果你今天必须重新构建 VMware，它会有什么不同？Host: so it's if you have to rebuild VMware today, how would it look different?

嘉宾：是的。而且我确实认为……Guest: Yeah. And and I do think

主持人：就像智能体领域的 VMware。Host: like the VMware for agents.

嘉宾：是的。我确实觉得，你知道，是否需要一个面向人类的智能体 VMware，对吧？你知道，Guest: Yeah. And I and I do think there's, you know, does there need to be a VMware for agents for humans, right? You know,

主持人：我明白你的意思了。是的。Host: I see what you mean. Yes.

嘉宾：是的。你知道，从某种意义上说，我需要人类。Guest: Right. You know, in the sense that, you know, I need humans.

主持人：我们还是聊聊智能体吧。它们可比眼下的话题有意思多了。Host: Let's stick to the agents. They're way more interesting than right now.

嘉宾：是的。不过你知道，我觉得这是个很重要的观点，因为我确实认为需要有一些东西——我就用 Dario 的宪法这个说法吧，或者说护栏，对吧？也可以是制定政策的能力。我认为这样一来，你就可以说，好吧，虚拟机的方方面面，就像我们如今所熟知和喜爱的那样，都是在为智能体提供服务。Guest: Yeah. But, but, you know, I think it's an important point because I do think that there needs to be, you know, and I'll use, you know, Dario's constitution, right, or guard rails, right? Or being able to set policies. And I think then you you really say okay every aspect of the virtual machine as we know we'll love it today it is servicing agents

主持人：没错，相比于为硬件和人类提供服务，对吧？所以我想，这就成了你们这边的根本设计约束：怎样才能让智能体变得出色？Host: right as opposed to servicing hardware and humans right so I think you know that becomes your fundamental design constraint on this side you know how do I make agents great

嘉宾：对，怎么让它们安全？怎么让它们性能更好？你知道，怎么对它们进行抽象？怎么迁移它们？诸如此类。你知道，智能体会有新的具象形态——你知道，V motion——每一种都会有适合它们的新具象形态。但我还需要考虑，人类如何制定策略、描述章程、获取仪表盘，等等。我觉得有意思的地方就在这里，因为这两方面都得兼顾，对吧？不能只做其中一方面。但从根本上说，我会把我们在虚拟机上做过的那些艰难的设计工作归结为：以高性能的方式，为应用程序或操作系统抽象硬件。而在这里，你知道，是为智能体抽象硬件和操作。Guest: right how can I make them secure how can I make them performant you know you know uh you know how do I abstract them how can I migrate them all of those things you know have the new embodiment you V motion for agents right it's like every one of these will have the new embodiment you know for them but then I also need to think about how can humans be setting the policies describing the constitutions you know getting the you know dashboards uh etc and I think that's sort of you know where the interesting thing is because you need to do both of those right not just one of those but the fundamental you know I'll say the hard design stuff like we did for you know virtual machines was associate abstracting the hardware in a performant way for applications or operating systems here. It's, you know, abstracting the hardware and the operations for agents.

这是你所服务的一个根本要素。它It's a fundamental thing that you're serving. It

主持人：很高兴聊了这些，非常感谢你来参加，Pat。我们下次再聊。Host: was great discussing this and thank you so much, Pat, for coming in and we'll do it again.

嘉宾：嗯，我最喜欢的两个人。随时奉陪。Guest: Well, two of my favorite people. Anytime.

主持人：好的。谢谢。Host: All right. Thanks.

客人：太棒了。谢谢。Guest: That was awesome. Thank you.

主持人：谢谢。Host: Thank you.