Walter Goodwin：内存带宽如何影响模型架构

简要描述Brief Description

与 Fractile 创始人兼首席执行官 Walter Goodwin 共话前沿模型架构的未来The Future of Frontier Model Architectures with Walter Goodwin， Founder & CEO of Fractile

目录Table of Contents

• 内容
  Content

内容Content

现在我们正尝试打造一颗芯片。如果你拆开一台 Nvidia 系统，会发现里面有大约六到九颗定制芯片，都是 Nvidia 自己设计制造的，它们协同工作，组成一个性能极其强大的系统。所以我们已经存在差距了。你知道，如果我们能提高效率，达到足以开始以类似方式打造自己的方案，那就太好了。这个想法是，在任何时候都能持续推进一系列你希望高度契合的发展方向，并随时准备启动相关项目的规模化，这会带来巨大的优势——前提是你能打造出那套机器和引擎。对于芯片领域来说，这就相当于前沿模型：如果你能找到一种方法，在结构上抢出三到六个月的领先优势，那么所有那些部署你都会胜出。Right now we're trying to build a single chip. If you crack open an Nvidia system, it has anywhere between kind of six and nine custom chips all built by Nvidia to come together to build something that is really, really potent. So there's already a gap there. You know, it'd be great to be productive enough that we could start to build our own responses in that same kind of way. This idea of being able to have at any given moment a kind of rolling frontier of bets that you hope are deeply aligned in which you're ready to trigger the ramp of this is a huge advantage if you can kind of build that machine and build that engine. It's the equivalent of the frontier model for the chip space is if you can just find a way to structurally carve out a 3 to 6 months of the vantage you will be winning all of those deployments.

听众朋友们好，欢迎回到《No Priors》。今天我请到了 Fractile 的创始人兼首席执行官 Walter Goodwin。Fractile 是一家全栈式 AI 芯片公司。我们聊了什么才算是一家全栈公司，他们押注的技术方向，他为什么如此关注内存带宽，他对未来前沿模型架构的预测，以及如今和未来由 Nvidia、AMD、内部研发团队和这一类新型加速器共同构成的前沿芯片市场格局。欢迎你，Walter。Walter，非常感谢你来参加节目。Hi listeners, welcome back to No Priors. Today I'm here with Walter Goodwin, founder and CEO of full stack AI chip company, Fractile. We talk about what it means to be full stack as a company, the technical bets they're making, why he's so focused on memory bandwidth, his predictions for Frontier model architectures in the future, and the structure of the chip market at the frontier today and tomorrow amongst Nvidia, AMD, internal efforts, and this new class of accelerators. Welcome Walter. Walter, thanks so much for doing this.

主持人：很高兴来到这里。莎拉，Host: Very good to be here. Sarah,

主持人：你创办了 Fractal 这家非常有意思的公司。能给我们概述一下你们是做什么的吗？Host: you started this incredibly interesting company called Fractal. Can you give us a overview of what you guys do?

嘉宾：Fractile 是一家芯片公司。嗯，我们为全球最大规模的模型打造速度极快的推理芯片。从一开始，我们就押注于速度。嗯，公司成立于 2022 年夏天。我想当时我们开始看到两件事。你知道，一件显然是基于互联网数据训练、能够泛化到各个领域的基础模型的出现。另一件是，有几位颇有远见的人提出，我们需要找到一种方法，在测试时为这些模型投入更多算力。我们需要找到一种方法，把我们在 AlphaGo 上采用的做法——先拥有一个能力很强的神经网络，再通过大规模展开让它变得超越人类——带到语言建模之类的领域。嗯，因此过去四年来，我们一直在努力寻找一种方法，制造出既能运行这些庞大模型、又能让它们比当今芯片快得多的芯片。Guest: Fractile is a chip company. Uh we build very very fast inference chips for the world's largest models. This sort of bet on speed is something which we've had right from the outset. Um started the company in summer of 2022. And I think at the time we were starting to see two things. You know, one was obviously the arrival of foundation models trained on the internet and generalizing across everything. And the other was, you know, a few kind of wise people saying, you know, we need to find a way to pour more compute into these models at test time. We need to find a way to take what we had for kind of alpho where you have a capable neural network, but then it becomes super human as you roll it out at scale and bring that to kind of language modeling, for example. Um, and so our big pursuit over the past 4 years has been to find a way to build chips that will allow for kind of simultaneously taking these huge models and running them much much much faster than the chips of today.

嗯，但要以一种能够扩展的方式来实现。也就是说，能够适配如今前沿模型之外的模型，也能够支持超长上下文。嗯，这就是我们一直以来的使命。自你创办公司以来的四年里，芯片领域的格局变得有趣得多了。你会如何看待自己在整个推理 GPU 和加速器领域中的位置？Um, but do that in a way that kind of scales. So scales to models beyond the frontier today, scales to extraordinarily long contexts. Um, and so that's the the mission that we've been on. it's become a much more interesting chip landscape in the uh four years since you started the company. Uh where would you place yourself in the uh overall GPU and accelerator landscape for inference?

主持人：是的，现在可选方案多得令人眼花缭乱。嗯，如果纵观整个 AI ASIC 领域，我觉得有一点特别显眼：市面上有很多相对同质化的芯片。嗯，这是一种结构性的产业特征。尤其是看看各大超大规模云服务商的布局——你知道，谷歌在十多年前就通过 TPU 开启了这一模式——你会发现，尽管有不少专有的内部芯片，比如谷歌的 TPU、Meta 的 MTIA、微软的 Maia，以及 OpenAI 现在的 Jalapeno，但这些芯片最终都是与数量相对有限的几家 ASIC 厂商合作开发并交付的，我们可以把它们称为 ASIC 设计服务商，也就是这类后端交付商。Host: Yeah, it's a uh there's a kind of extraordinary zoo now of options available. Um and if you look across this kind of entire space of kind of AI as6 um one of the things that I think is very striking is there's a lot of relatively identicate chips out there. Um and this is kind of a structural it's a kind of structural industry uh property where if you look especially across kind of hyperscaler efforts and you know Google kind of kick this off with the TPU more than 10 years ago um you see this paradigm where although there are a number of kind of proprietary internal chips you know Google with the TPU Meta with MTIA Microsoft with Maya OpenAI now with Jalapeno um these are chips that are ultimately developed and delivered in partnership with kind of a relatively small number of what we might call ASIC houses, these kind of back-end delivery houses.

博通是最大的——你知道，一家市值达 2 万亿美元的公司，帮助人们实现这些设计。所以，当你深入观察这看似令人眼花缭乱的丰富选择，以及如今存在的各种方案时，你会发现这些平台之间其实有很多相似之处。你知道，所有这些芯片都采用 HBM，也就是这种 DRAM 内存，它常见于英伟达 GPU、AMD GPU 以及其他所有这些 AS6 中。它们都押注于同类张量核心来执行矩阵乘法，也都采用台积电的先进封装。因此，我认为我们在这个领域看到的一个现象是，真正贯穿整个硅片技术栈、试图打造根本性新能力的努力仍然相对匮乏。Broadcom is is the largest at, you know, $2 trillion company um that help people kind of realize these designs. And so when you kind of zoom into this this sort of seeming uh embarrassment of riches and the spectrum of what exists today, uh you see that actually there's a lot of similarity across these platforms. um you know uh all of these chips. You have HBM uh this type of DRAM memory that is you know common to Nvidia GPUs, AMD GPUs and all of these other AS6. You have the same kinds of bets on tensor cores to do your matrix multiplications, the same advanced packaging with TSMC. Um, and so I think one of the things that we see in the landscape is there's a relative dir still of kind of efforts that go all the way up and down the silicon stack and try and build kind of fundamentally new capabilities.

嗯，这在某种程度上是个结构性问题。决定要从你知道的、可以称为架构层、前端设计层的环节开始构建的团队并不多；这些环节看起来大多就是编写代码，但还要一路深入到我们所说的物理设计、工艺技术、与晶圆代工厂的互动，以及那些基本上让你每周都得飞去台湾或韩国的工作。嗯，这类事情实际上仍然往往只由相对少数的公司承担。对于不在芯片行业工作的人来说，架构方面的公司所做的事情，与像博通这样的公司之间，通常通过什么样的抽象层或交接方式来衔接？Um, and this is kind of a structural thing. There's just not that many teams that have decided we're going to build from, you know, what you might call the kind of, you know, the architecture layer, the front-end design layer, which mostly looks like kind of writing code, but also all the way down to what we call kind of physical design, the process technology, the kind of foundry interactions, the stuff that essentially has you flying back and forth to Taiwan or Korea every week. Um, that's something which actually tends to still be left to a relatively small number of companies. for people who don't work in the chip industry, what is the common abstraction or handoff between what they're what you know architecture focused players are doing and a player like Broadcom.

嘉宾：是的。所以我想，如果你是 Google，比如说你在设计 TPU，那么你的团队里会有不少人非常了解他们试图加速的工作负载。回溯到十多年前，这是一个让你开始构想张量核心这类东西的领域：它是架构师构想出来的一种专用电路，特别擅长矩阵乘法，因为你意识到，在你运行的这些模型中，矩阵乘法占了绝大多数浮点运算量。我想，这大概就是架构师的职责所在：既要真正了解芯片的运行方式，理想情况下，也要非常了解目标工作负载的类型。Guest: Yeah. So I think the if you're let's say you're Google and you're designing the TPU um you have a number of people in your team who have a really strong understanding of the workloads that they're trying to accelerate. Um and so you know going back over 10 years this is uh a world that then sees you conceive of things like the tensor core which is a designated circuit dreamt up by an architect uh that is exceptionally good at matrix multiplications because you have this insight that matrix multiplications are the overwhelming majority of the kind of flop count in these models that you're running. um that's kind of the preserve of the architect I guess is somebody who really understands the way that chips behave but also ideally has a very strong understanding of the kind of target workloads.

接下来，你会在这些组织内部看到一群非常聪明的人，他们会把设计进一步细化，形成一种电路级描述，说明这款芯片应该如何运作。这类工作大多就是我们所说的前端设计。从机制上说，它看起来仍然是在电脑上编写代码，而这些代码之后仍会交给 Broadcom 这样的厂商。前端设计描述了芯片的底层意图和逻辑，最终会被转换成一种东西——最后交付给台积电的，实际上就是一种位图。Um what you'll then see inside those same organizations is a set of sort of you know very smart people that kind of lower that down into I guess you know essentially a kind of circuit level description of how this chip should behave. And so a lot of this is what we call front-end design. um you know mechanistically it still looks like like writing code on a computer and it's still code that is then handed off to uh a player like Broadcom. Um and so you know that kind of front-end design that describes the kind of underlying intent of the chip, the underlying logic of the chip is then transformed ultimately into something that you know what gets shipped to TSMC in the end is literally a kind of bit map.

呃，你知道，我们有一种叫 GDS2 的文件类型，里面具体写着金属层该放在哪里、每个单独的晶体管该放在哪里。所以它是一份完整的版图。你要经过一个综合流程，把 RTL 转换成一组电路，最终再完成布局。这里面很多复杂之处在于，嗯，比如在博通的案例中，他们拥有用于芯片间互连的模拟 IP；而这些 IP 的物理布局又是针对台积电某个特定制程节点的。你知道，我们会说 3 纳米、5 纳米之类的。嗯，所以直到今天，对于所有这些项目来说，这部分通常仍然是外包的工作。uh you know we have this file type you call GDS2 and it's literally you know where do I put the metal layers where do I put each individual transistor and so it's a it's a full layout and so you go through this process of synthesis of that RTL uh into a kind of set of circuits that then ultimately get laid out and a lot of the complexity there is around um you know in Broadcom's case you know it will be ownership of analog IP that is used for chipto-chip connectivity and it is this kind of physical placement that is specific to a given process node at TSMC. So you know we talk about like 3 nanometers, 5 nanometers etc. Um so that piece is still today you know generally it is a kind of outsourced activity for all of these projects.

主持人：你描述的听起来像是一个规模非常庞大的项目，要端到端地打造一款全新的芯片。嗯，从工作负载的角度来看，你们正与几家前沿领域的参与者合作，对工作负载进行特征分析，并确保你们能够满足他们的需求。你们是怎么考虑 Fractile 团队如今的构成的？作为一家初创公司，你们如何做到这一点？ 是的，我觉得有一点挺有意思：很多行业做事时都会采用一种瀑布式的方法，然后突然之间，一种新的思路出现了。借用管理学的说法，这种思路会更加敏捷，对吧？我认为，作为一家全栈公司，我们的团队涵盖了对工作负载的深入理解。实际上，我们会努力在很多方面走在前面，思考我们应该如何调整模型架构，让它更好地契合我们的押注。Host: You are describing what feels like a very large project to do endtoend um new chip creation. uh from a workload perspective you're working with several frontier players to sort of characterize that and make sure that you can serve them. How did you think about like what does the fractile team look like today? How can you do that as a startup? Yeah, I think there is a you know it's funny there's a lot of industries where there will be a kind of waterfall style approach to doing things and then suddenly you know along comes a new way of thinking and it's kind of more agile to borrow from like management philosophy right and I think for us as a very full stack company so we have a team that does span you know very deep workload understanding I think we actually try to you know run ahead uh in many places and look at you know where would we change a model architecture to be more exceptional aligned with our bet.

我们认为，针对我们所做的具体押注，规模定律的走向大概会是什么样？这样我们就能据此向客户和合作伙伴提出建议，也能帮助我们为一款芯片制定一系列押注——这款芯片可能要一两年后才会批量问世。这就成了我们必须培养的一项非常重要的能力。 不过，正如我们所看到的，我们有能力快速迭代，这意味着在 Fractile 内部，我们有前端设计师、自己的物理设计团队、自己的后端实现团队；比如说，我们也会自己做先进封装方面的工作。这些团队的人数并不多，对吧？我的意思是，Fractile 目前大约有 150 人，在这些领域里我们的人手都相当精简。但这让我们能够形成一个更敏捷的闭环。What do we think is like kind of the scaling law vector for the particular bets that we are taking and then we can advocate to our customers and our partners on that and it also helps us inform a set of bets for a chip that you know isn't going to necessarily exist in volume for you know one or two years. Um that becomes a really important sort of part of the muscle that we have to build. But then as we look across this kind of you know ability to spin a very kind of agile loop it means that inside fractile we do have front-end designers we have our own physical design team we have our own kind of backend implementation team you know we do our own work on advanced packaging for example um and that's not an enormous headcount right I mean Fractile today is about 150 people uh we're sort of skinny in in every single one of those uh sectors but what it allows us to do is have this kind of much more agile closed loop.

嗯，我认为，考虑到这个行业所要求的节奏，这一点正变得越来越重要。你知道，在 AI 芯片领域，工作负载的变化总是让人疲于追赶；与以往任何其他类型的芯片业务相比，你必须非常准确地押对方向。呃，所以你需要很多专业能力，也需要很多运气，而且还得迅速出手。因此，这种我们把整个业务链条都掌握在自己手中的架构，与我想此前那种做法非常不同。此前几乎总有一个交接点：做到一定程度后，就把工作交给另一家合作伙伴；而在某种程度上，之后你就得看这家合作伙伴如何行事，多少会受制于他们。Um, and I think that's becoming increasingly essential as you look at the sort of cadence needs that are set by this industry. Um, you know, the chasing, you know, perennially of the sort of chasing of the tail of the workloads is, you know, you have to in the AI chip space more than any other kind of chip play before, you have to really place your bets correctly. uh so you have to have a lot of you know skill and a lot of luck and you also then need to strike very fast and so this kind of structural setup where we have the entire story in house uh it's very very different to this kind of I guess prior approach where there's almost like a handoff point and so you get to a certain level and then you hand off to another partner and at some to some extent you're kind of at the mercy of of how that other partner then behaves.

你认为这家公司做出的最重要的技术押注是什么？what do you think are the single most important technical bets that the company has made?

嘉宾：所以对我们来说，我觉得你知道，其中一点是发展方向。四年半前，你显然正处于推理领域的核心，但我仍然觉得，之后大约两年里，我们一直在向全世界普及推理的含义。如今的一些推理芯片，包括最近一家上市公司的芯片，大约18个月前还是训练芯片。你知道，当时人们确实在回避这样一种想法：这种较低的、最终的边际成本——我觉得“边际成本”有两种含义，对吧？它听起来像是成本很低，但实际上指的是每次部署这些模型时你都要支付的成本。所以我觉得，首先，关键在于我们大胆押注：行业会转向部署时代；其次是押注速度。这些判断确实深刻影响了我们后来在架构层面采取的许多应对措施。在架构方面，我觉得公司成立后的头两年左右，我们经历了一段探索历程。Guest: So for us I think you know one is the direction you know four and a half years ago um uh you were obviously uh right in the heart of an inference story but I still felt that we were educating the world about what inference meant um for two years or so following that. Some of the inference chips today including one that recently went public was a training chip until about 18 months ago. you know there was a real avoidance of this idea that this sort of small you know final marginal cost you know marginal cost I think has two senses right it either sounds like it's a small cost what it in fact means is this is the cost that you pay every single time you deploy these models um so I think firstly just the the sheer bet that we were going to slip into a kind of deployment era um and the bet on speed that was something that kind of really informed a lot of what we then did as a kind of architectural response on the architectural side you We've been through a journey I think for the first kind of 2 years or so of the company's life.

嗯，我们当时在做一种有点像 Groq 或 Cerebras 的、基于 SRAMM 的芯片。我们注意到，SRAMM 是一种带宽极高的内存，它和逻辑电路位于同一块硅片上。因此，在运行模型进行计算时，计算单元与模型权重（或者模型逐步生成内容时使用的 KV 缓存）之间就能实现极高的带宽。这就能让这些语言模型达到每秒数千个 token 的生成速度。不过，我想我们在 2023 年底开始担心这种方案的可扩展性，而到了 2024 年，这个问题就更令人担忧了。我认为，当前有两样东西在随着 AI 的发展而增长。Um we were working on a little like grock or cereabbras uh working on an SRAMM based chip. So we'd sort of made the observation that um uh SRAMM is this super high bandwidth memory. It lives on the same piece of silicon as your logic. uh and so you have this extremely high bandwidth between your compute where you're running the maths as you run these models and let's say the weights of your model or the KV cache of this model as you're rolling it out. Um and that's something which then then can drive you to thousands of tokens per second on these language models. Um but I think one of the things that we started to worry about in kind of uh towards the end of 2023 and certainly in 2024 was the scalability of this approach. Um and you know I think there are two things that grow with AI today.

一个显然是模型的参数。嗯，另一个因素——也正是它让我们对那种架构方案感到有些紧张——就是你知道，逐渐增长的上下文长度，这在我们设想这些模型如何推出的过程中变得越来越重要。嗯，过去几年，这促使我们转而押注一种我们认为更令人兴奋的方向，我想在某些方面是这样：与存储器供应商以及我们的逻辑代工合作伙伴更加紧密地合作，设法以极高带宽访问更大容量的存储器。因此，在过去几年里，我们一直参与这类秘密研发项目，探索如何摆脱 SRAMM，例如研究怎样大幅提升对 DRAM 存储器的带宽。One is obviously the parameters of the model. Um but the other and this was the one that kind of really got us nervous about that architectural approach was um uh you know this kind of growing context length that was becoming more and more part of the story for for how we saw these models rolling out. Um, and it's that that has taken us over the past couple of years to a kind of what we see as a more exciting bet, I guess, in some ways, which is working more closely with memory vendors as well as our sort of logic foundry partners to find ways to get extremely high bandwidth access to higher capacity memories. So for the past couple of years we've been involved in these kind of skunk works projects to move away from SRAMM you know looking at how we can get for instance much much much higher bandwidth to DRAM memories.

嗯，这对我们来说一直是一个非常令人兴奋的战略押注，因为它让我们现在能够搭建一个平台，并将在明年下半年逐步扩大规模。嗯，这个平台可以说是把这类更高容量、成本更低的 DRAM 内存（GPU 或 TPU 上用的那种）的可扩展性，与 Grock 芯片或 Cerebrus 芯片带来的所有速度优势结合起来。嗯，我认为我们觉得这项技术尤为重要的地方在于，如果你看看如今速度真正能带来显著影响的领域，就会发现其中有一种应用方式，就是让聊天机器人响应更快一些。但我觉得这就像是亨利·福特曾经提出的那个反问：如果问人们想要什么，他们大概会说想要更快的马，而不是汽车。嗯，更灵敏的聊天机器人，某种程度上就是快速推理领域里的“更快的马”。Um and this has been a very exciting bet for us because it's allowed us now to put together a platform which will be ramping in the second half of next year. um which uh kind of unites I guess the scalability of these kind of higher capacity lower cost DRAM memories that you have on a GPU on a TPU uh with all of the speed advantages that you get from a Grock chip or a Cerebrus chip. Um, and I think where we see this as especially vital is if you look at where it is that speed really moves the dial today, there is a kind of there's a version of this which is like a snappier chatbot. But I think this is like you know when um uh Henry Ford sort of asked the rhetorical question of what people would have said they wanted and you know faster horses uh rather than a car. Uh the snappier chatbot is kind of the faster horses of kind of fast inference.

在我们看来，能够将一个数万亿参数的模型以每秒数千个 token 的速度流畅运行，是提升 AI 能力的一种根本性方式：让这类长时间运行的智能体快上许多。嗯，所以如今这里存在一种耐人寻味的不匹配：我们现有的快速推理芯片拥有超高带宽的内存，但容量却极低。看看它们如今实际使用和部署的技术细节就会发现，它们并不运行长上下文注意力计算；要做这件事，还是得切回 GPU。因此，这里存在一种耐人寻味的不匹配：恰恰在这个领域，我们几乎已经能够让这些任务快上许多。the place where the ability to take a multi- trillion parameter model and run it comfortably at many thousands of tokens per second in our view is a kind of fundamental elevator on capability for AI is in taking these kind of very longunning agents and making them radically faster. Um and so that's kind of a it's a tantalizing mismatch today between the properties of the fast inference chips that we have which have super high bandwidth memory but incredibly low capacity. And if you look at the sort of technical detail of how they actually get employed and deployed today, they're not running long context attention. You still flip back to a GPU to do that. And so there's this sort of tantalizing mismatch where precisely the area where we are almost able to go and take these things and make them much faster.

我们也不具备那种技术能力。嗯，所以从某种意义上说，这就像许多技术挑战一样，归根结底是一个稍显平淡的技术观察：我们需要具备某种难以言喻的特殊属性的芯片，也就是内存带宽极高，这样我们才能加载权重、加载状态，而且每秒要加载成千上万次；但同时还得使用经济实惠的内存。因为当你考虑如何在数据中心规模上为数千名用户运行推理时，成本实际上最终归结为所用内存的每 GB 成本。嗯，因此，对我们来说，另一个关键重点就是解锁这种全新的基础构件：找到一条途径，让这类成本更低的内存——也就是全球成本较低的 DRAM 内存——能够提供极高的带宽。We also don't have that technical capability. Um and so in a sense this is you know like many technical challenges it boils down to a slightly mundane technical observation which is we need chips that have this kind of particular ineffable property which is incredibly high bandwidth to memory so we can load the weight so we can load this state you know many thousands of times per second uh but also have like economical memory because when you look at what it looks like to run inference at data center scale for thousands of users the economics actually collap collapse down to uh essentially a kind of cost per gigabyte of the memory that you're employing. Um and so that's been the sort of other key focus for us is unlocking this kind of fundamentally new building block which is uh finding a path to get aggressively high bandwidth from this kind of uh you know from the world's lower cost memory uh DRM memory.

主持人：你提到了一些我认为不太符合传统观点的事情。第一，你们自己押注工作负载将走向何方；第二，你知道，传统芯片行业的人可能会认为，芯片是按代交付的……Host: You've said a few things that I I think are um not the conventional wisdom. one in in terms of making a bet yourselves on where the workloads are going and then two the idea that you can um you know traditional chip people might think of delivery of chips in a generation

嘉宾：你知道，每年，或者说比每年稍微快一点。嗯，你也知道，大型架构变革的节奏要慢得多，嗯，慢于这种节拍。不过，你们在 Fractile 一直有个愿景：你认为架构变革的速度，以及你们采用的技术方案变革的速度，都可以快得多。能不能谈谈，哪些因素可以促成这一点？因为我觉得，如今人们比过去更了解供应链的物理限制。那么，这个行业还能有多大的灵活性？Guest: you know every year or a little faster than that um and you know big architectural changes come slower than um that that sort of tick rate uh but you you you've aspirationally said within fractile that you think the speed of like change in architecture um and change in you know the the technical plays you're making can be much faster than that. Can you talk a little bit about like what would enable that to happen because the you know I I I think people have a better understanding of the physical limits of the supply chain today than they used to. So what is the flexibility that the industry could have?

主持人：是的。所以我觉得这里面有一点微妙之处。对于 AI 芯片来说，在任何一个时刻，你想要的都是一款能够精准适配你所关注的工作负载、而且今天就能批量拿到手的芯片。但即使你有了这样一款芯片，你知道，六个月后你又会想要别的东西，因为这些工作负载变化得非常快。Host: Yeah. So, I think there's a nuance on uh on this, but what what you want at any given moment in time for an AI chip is you wish that you had a chip that was precisely targeting the workload that you care about uh and in your hand today in volume. But if you had that chip uh you know, you're going to want something else again in 6 months time like these workloads move on very fast.

嘉宾：嗯，你知道，我们现在大约每两周就会推出一个模型。所以……Guest: Well, today you know we have a model coming out about every uh two weeks. So

主持人：没错。所以，合理的判断来自横向观察这些模型，然后问：好吧，它们有什么共同点？幸好，它们的共同点还不少。你知道，新型 LLM 总是迫切需要高出几个数量级的内存带宽，以便运行得更快。如今情况依然如此：这些 LLM 往往是自回归的，生成文本时批量很小；而这里始终存在一种根本性的权衡：吞吐效率和成本，与模型服务速度之间的权衡。因此，这些模型都有一些共同的迫切需求，而内存带宽就是其中之一。Host: that's right. And so so sanity comes from looking across those models and saying well okay what's what's common to all of these and like thankfully there's enough that is common. So you know it's always the case that a new LLM uh desperately wants to have orders of magnitude more bandwidth to memory in order to run faster. Uh it remains the case today that you know these LLM tend to be autogressive very low batch as you generate text uh with this sort of fundamental trade-off again between kind of throughput efficiency and cost and the speed that you can serve these models at. Uh so there are certain things that these models are unified in kind of crying out for and and memory bandwidth is one of them.

嗯，不过正如你所说，这些技术在不断演进。尤其是如果你关注开源中文模型的前沿，就会发现，注意力机制的具体形态每隔几周就会发生变化，前沿技术也在不断更迭，比如稀疏性——注意力机制本身的稀疏程度。嗯，我们可能还会面临更大的冲击，也可能出现更根本性的转变。所以我认为，这里存在一种双重性：一方面是物理世界和金融世界的种种限制，另一方面则是工作负载的需求——它们基本上以软件演进的速度不断变化。尽管大家显然希望能够更快、更积极地推出新平台，但我认为也存在一些根本性的制约。因此，在 Fractile，我们对在芯片设计中全面采用 AI 感到非常兴奋。我认为，我们有能力掌握整个端到端的问题流程，这确实让我们能够从根本上重新思考做事的方式。Um but then you know as you say there are these evolutions and especially if you look at the kind of frontier of open- source Chinese models you know the exact nature of an attention mechanism will churn every couple of weeks in terms of what's kind of at the frontier the sparity the sparity level of the you have the sparity on the attention itself um and you know we may be in for like bigger shocks as well um there may be kind of more fundamental shifts and so I think there's this duality uh around sort of the limits of the physical and the kind financial world I guess and then these demands of you know workloads that churn at the pace of essentially software where although there is this sort of clear desire to be able to kind of ship new platforms you know aggressively faster I think there are also some sort of fundamental breaks on this so so at Fractile you know we're extremely excited about being AI forward in how we do chip design I think we've seen this ability to own the entire end toend gamut of the problem is really enabling uh a kind of fundamental rethink of how we do things.

你知道，如果你想想经典计算机科学中的阿姆达尔定律，对吧？就是所有我能并行化的部分都会变得非常非常快，而不能并行化的部分则不会。我觉得，身处一个很长的组织链条中时，情况也差不多。比如说，作为一家专注于前端设计的芯片初创公司，你积极采用 AI，把前端设计周期从可能需要 12 个月缩短到其中很小的一部分，但如果接下来仍然会遇到传统的瓶颈，后续环节也仍然按传统速度推进，那么彻底改变流程未必有压倒性的必要，或者说未必能带来相应的回报。我觉得，从某种程度上说，这类瓶颈总会存在。你知道，我们和晶圆代工合作伙伴面临的晶圆厂生产周期都差不多。You know, if you think about um classic CS law as Amdal's law, right? Like everything I can paralyze becomes very very fast and the part that I can't paralyze doesn't. And I think it's similar when you're in a like long chain of organizations um that you know there isn't necessarily an overwhelming need to uh or return on radically changing your processes as a front-end focused chip design startup adopting AI in aggressive ways to shorten your timelines from being you know maybe 12 months of front-end design work to a tiny fraction of that if there's then going to be kind of the conventional bottlenecks and the conventional pace on the next piece. Um and I think that's sort of you know at some extent there will always be those bottlenecks. You know we all have the same kind of fab cycle times with our foundry partners.

所以，从你把芯片送给他们，到拿回芯片，大概要三到五个月；即使是在生产批次非常紧急的情况下也是如此。我想，这就是这个行业固有的延迟。拿回芯片后，我认为至关重要的是要看看这些芯片实际上如何才能变得具有财务可行性：它们确实需要一段回本期。要让这颗芯片成为合理的财务决策，折旧周期确实得有三到五年左右。嗯，所以这里有两件彼此存在张力的事，而我想我相信这两件略有冲突的事情可以同时成立。第一，能够缩短整个芯片设计周期，价值巨大，而且缩得越短越好。另一方面，我认为我们也不能忽视在架构上做出深思熟虑、细致入微的选择，因为最终做出来的芯片仍然需要有三年以上的实用寿命。So so from the moment you send the chip to them to getting it back is you know 3 to 5 months even in a kind of super hot lot scenario. And so these are the kind of innate latencies I guess in the industry. And then when you get that chip back, I think it is vital to look at the uh the way that these chips actually become financializable, which is that they do have a kind of payoff period. You have to have really a kind of 3 to 5 year amatization window for that chip to make it a sensible financial decision. Um and so there's these sort of two things that are intention and which I I guess I believe two slightly conflicting things at the same time. one is there is a huge value in being able to take the overall chip design cycle and compress that and compress that. Uh there is also I think what we don't lose is the need to place really thoughtful and subtle bets architecturally because the chip you actually end up making does still need to have a a threeyear plus you know useful lifespan.

嗯，所以我觉得对我来说，你知道，随着时间推移，能够压缩这种芯片设计周期，本质上意味着你会有更多尝试的机会。你希望自己始终做好准备，随时可以拿出一个旗舰平台并说：“没错，就是它。它才是真正的旗舰产品，我们要把它投入量产；接下来还会有一个为期 12 到 18 个月的产量爬坡期。”而且你希望它能长久发挥作用，也希望它能持续为客户创造价值。所以我觉得这里微妙之处在于，你知道，我不认为我们最终会达到这样一种境地：仅仅因为缩短了周期，我们就每隔几周推出一款根本上全新的芯片。我这么想是因为，我们身处一个物理世界。Um, and so I think for for me, you know, what it means to be able to over time compress that kind of chip design cycle is essentially that you get to have more shots on goal. You want to be forever ready to take a given flagship platform and say, "Right, that's it. That is actually the flagship and this is the one we're going to ramp, but then it's a, you know, 12 month 18month volume ramp." And you're expecting it to have longevity. You're expecting it to actually continue to drive value to your customers. And so I think that the the sort of subtlety here is you know I'm not a believer in the idea that we will ultimately get to a place where we are shipping a fundamentally new chip uh you know every few weeks just because we've shortened that down because I think this is a this is a physical world.

你知道，数据中心的电力供应会对你构成一定限制。安装这些设备需要时间，而且你得为底层芯片本身提供资金。所以这需要有相应的回报周期。但我认为，你可以设想这样一个世界：你能把这个延迟缩短得越多——也就是从观察到某个机会，到大规模兑现这项押注之间的间隔——就越能从中获取巨大的价值。嗯，我认为这也是 Fractile 所涉足的一个领域……There is a certain amount of constraint on you know data center power. There are latencies to installing these things and you need to finance the actual underlying silicon. So it needs to have that payoff period. But I think what you can get to is a world where the shorter you can make that latency, that gap between an observation and realizing that bet in volume, that is where there's an, you know, an enormous amount of value to be captured. Um, and I think it's also an area where, you know, fractile is a

嘉宾：只要是因为你对该加大投入什么所做的决定是正确的。Guest: if only because the decisions that you make about what to ramp are the correct ones.

主持人：没错。我想你知道，你想成为……也许你也能同时推进更多事情。所以，你知道，我觉得人工智能能力不断提升有一件特别令人兴奋的事：随着我们做事的效率越来越高，嗯，你知道，整个经济通常会出现两种反应。一种是：“哦不，我们要做的事情没那么多了。可能会有失业。”另一种是，我们可以做更多事情。嗯，我想你知道，站在 Fractile 的立场上，我们现在正努力打造一款芯片。我们知道，竞争对手的实力不容小觑——比如你拆开一套英伟达系统，就会发现里面有六到九块定制芯片，都是英伟达自己制造的，它们协同工作，组成一套真正非常强大的系统。Host: That's right. I think you you know, you want to be and maybe you get to have more irons in the fire as well. So, you know, one of the things that I think is so exciting about AI capability rising is um as we do become more productive in doing things, uh you know, there's obviously these kind of two responses, you know, generally across the economy. One is, oh no, we're not going to have as much to do. We may maybe there's jobs loss. The other is uh we get to do more things. Um and I think, you know, standing here from from Fractile, you know, right now we're trying to build a single chip. We know that we're up against competitors that have uh you know if you crack open an Nvidia system it has anywhere between kind of six and nine custom chips all built by Nvidia to come together to build something that is really really potent.

嗯，所以那里已经存在差距了。你知道，如果我们能高效到足以用类似的方式开始构建自己的应对方案，那就太好了。嗯，所以我认为，能够在任何时刻都有一批不断推进的前沿押注，而且你希望它们能高度契合，并且随时准备启动扩张，这个想法有着巨大的优势——前提是你能打造出这样的机器和引擎，从而让这种做法持续为你带来六个月的领先优势，领先于你的竞争对手。呃，这就是那种能让你切入市场的楔子；我们在前沿实验室也看到了这一点，对吧？在芯片领域，这相当于前沿模型：只要你能找到一种方法，从结构上创造出三到六个月的优势，呃，你就能赢下所有那些部署。Um so there's already a gap there. You know it'd be great to be productive enough that we could start to build our own responses in that same kind of way. Um, and so I think this idea of being able to have at any given moment a kind of rolling frontier of bets that you hope are deeply aligned in which you're ready to trigger the ramp of this is a huge advantage if you can kind of build that machine and build that engine and the six-month gap that that might then perennially give you against your comp competition. Uh, that is the kind of, you know, wedge that allows you to drive in and we see this in in Frontier Labs, right? It's the equivalent of the frontier model for the chip space is if you can just find a way to structurally carve out a 3 to 6 months advantage. Uh you will be winning all of those deployments.

嘉宾：这里面肯定能编出个梗：首席执行官说：“哇，这 AI 真棒，我们还把组织架构设计成能用上它的样子。我们不搞一个程序，而是安排六个人，大家还都欢呼雀跃。”Guest: There's a meme in here somewhere of CEO saying, "Wow, this AI stuff is great and we structure the organization in a way that can consume it. Instead of one program, we're going to have six guys and everybody cheers."

主持人：是的，我觉得这话有些道理。呃，我之前和一家排名前三的半导体公司的 CEO 聊过，他不让我把这个预测——你知道——真的播出去。不过我问他，大概还要多久，我们才能把首席架构师的意图直接转化成一个完全可用的 GDS2 文件？他说，10 年。Host: Yeah, I think there's some truth in that. what uh I was talking to um one of the top three semi CEOs and he wouldn't let me put this prediction uh you know actually on air but I I asked him like how soon until we can have essentially like intent from chief architect into a fully usable GDS2 file and he's like 10 years

嘉宾：这不会像现在这样发生。你对此有什么看法？或者你认为它会首先对你们组织的哪个方面产生影响？Guest: it's not going to happen like now what is your view on that or where do you see the impact first in your organization

主持人：你知道，过去几年里一直很有效的一种启发式方法，就是始终质疑自己的那种逻辑假设，然后把时间尺度缩短到四分之一。所以你知道，有一种可能是，十年看起来是个非常合理的时间框架，但我会把它缩短到四分之一，而且可能还会再减一点。你知道，我认为未来几年里，我们会看到一种能够端到端完成原型设计的空间。我觉得芯片设计有一点非常有意思：其中仍然存在一些环节，采用的还是解决 NP 困难问题的传统方案。所以你知道，如果你今天真的想输出那个 GDS2 文件，仍然要经过一大堆布局工具，用传统的算法方法进行布局和布线，而这些操作会连续运行好几天。Host: you know a a heristic that has been successful over the past couple of years is to just always question your uh your sort of logical assumption and then divide it by four on time scale. So you know there is a world in which 10 years seems like a very sensible time frame for that but I would divide it by four and I probably subtract a bit from that. You know I think that we will have uh a space where there is a kind of prototyping that goes end to end um uh you know in the next few years. I think one of the things about chip design that is very interesting is there are still these loops in the middle of it that are kind of conventional solutions to MP hard problems. And so you have these sort of you know if you really want to output that GDS2 file today you are still running through a bunch of layout tools that are doing place and route with conventional kind of uh uh algorithmic approaches which will run for days on end.

我觉得这是一个非常有意思的现象，而且也能在一定程度上联系到我们同样感兴趣的那类工作负载。我认为，世界上很多难题都多少有点像这样：如今有大量思考工作可以自动化，有很多脑力劳动；但另一方面，你做的事情本身也存在一定的固有延迟。我认为，我们在当今用 AI 指导 AI 模型开发时也能看到这一点。关于 RSI 这项工作，没错，我们现在可以拥有超强智能来指导实验，但如今我们只是受限于实验本身，也就是说受限于计算时间。因此，我觉得芯片设计也有点类似。也许从架构师的意图一路走到 GDS2 这个问题，和 RSI 问题有些相似：阻碍我们以极快速度实现目标的，是这个流程中确实有若干环节在更传统的意义上计算开销很大。You know, I think this is a a really interesting thing that we see and actually to bring it back somewhat to the sort of workloads that we're excited about as well. Um, a lot of hard problems in the world I think look a bit like this where there is a lot of thinking that can now be automated. There's a lot of cerebral work. Um, but then there is also uh, you know, this kind of intrinsic latency to something that you do. Um so I think we see this in uh you know AI for guiding uh AI model development today. So the RSI work um you know yes we can now have extreme intelligence that is guiding our experiments but now we're just experiment bottlenecked you know we're compute time bottlenecked and so I think with with chip design it's a little bit like this maybe to take the kind of you know uh architect's intent through to GDS2 question it's a little bit like the RSI question in that the things that will prevent us getting there extremely fast is the fact that actually in that flow there are a number of things that are genuinely kind of computationally expensive in a more conventional way.

我认为，对于所有这类问题，我们通常都会看到……I think what we will generally see for all of these sorts of problems and

嘉宾：而且它们不会被代理模型取代。Guest: and they won't be replaced by surrogate models.

主持人：嗯，我想你知道，确实有这类模拟模型，我认为它们推动了近似计算。你知道，如今我们看到的所有模型，比如有限元分析、热估算等等。我认为，正是因为这种阿姆达尔定律，这类工作才会大幅回归：随着我们加速智能的发展，也加快这些实验、这些试验之间的周期……Host: Well, I think you know there's definitely these sort of uh simulation models I think driving approximations uh you know uh all of the models that we see today for kind of finite element analysis and and sort of thermal estimation and so on. I think there is a huge return to that kind of work precisely because of this kind of Andals law that as we accelerate the intelligence as we accelerate the period between these experiments between these trials

嘉宾：他们成为了 BGuest: they become the B

主持人：因此，加快这些试验也变得重要得多。所以你知道，我们正在内部研究：这些算法的底层究竟是什么？有没有一种粗略的近似方法，可以完成其中一部分工作？我认为，在相当长一段时间内，芯片设计在最终签核环节不会发生变化。Cadence 和 Synopsys 数十年来一直与台积电和其他晶圆代工厂合作，制定出那种最终的检查标准——确认这个东西符合我们所说的 DRC 和 LVS 清洁标准，这一点极其宝贵。它符合那家代工厂制定的规则。嗯，这类工作确实非常有价值。不过，在此期间，能不能有某种模糊布图算法，让你更快地几乎完成最终设计，帮助你更快地迭代？Host: it becomes much more important to accelerate those trials as well and so you know we are looking internally at you know what is actually what are in the guts of some of those algorithms and is there a crude approximate way that you could do some of this you know the I think chip design is not going to change in the final sign off for quite some time. It's incredibly valuable to have Cadence and Synopsis who've been working with TSMC and the other foundaries for decades to build the sort of you know the final check mark that says this thing is uh what we call DRC and LVS clean. It conforms to the rules that that foundry has set. Um that is actually where you know that sort of work is immensely valuable. But could you have some kind of fuzzy placement algorithm in the meantime that gets you almost all the way to your final designate faster, helps you iterate faster.

你知道，我觉得这恰恰是人们应该投入更多工作的地方，否则它就会成为瓶颈。嗯，另一方面，我觉得这对 AI 实验中的 RSI 也同样适用：随着我们的智能水平不断提高，驱动这些实验的思考也越来越深入，而实验因此成为瓶颈时，我认为我们就会投入更多思考。这其实就是 Fractile 对工作负载的整体押注，也是我们如此热衷于攻克难题的原因之一；某种程度上，这也是我们认为 Fractile 这款芯片能够推动我们加速解决各类难题的原因。我觉得，在你启动每一项实验之前，认真多想一步，简直成了一种……几乎是道德上的责任。You know, I think that is exactly where people should be should be doing more work because otherwise that is the bottleneck. Um the other part there and I think this is true of you know RSI for AI experiments as well is as we do become you know as our intelligence levels on the the thinking that then drives that experiment gets higher and higher um and as that experiment therefore becomes the bottleneck I think we do more thinking and this is just generally Fractile's workload bet it's the reason we're so excited about hard problems and it's kind of the reason that we see Fractile as the chip that takes us toward accelerating the solution solution to hard problems in general is I think it becomes absolutely it's like almost uh you know moral compunction to think harder before every single experiment that you fire off.

嗯，我想，呃，最近可能是 Baron Millage 在 Dwarsh 上说过，你知道，也许你应该先思考一百年，再启动一项实验，然后再用这些模型思考相当于人类一百年的时间，来分析这项 AI 实验的结果，因为中间的实验相当昂贵，而且需要一定的实际运行时间。所以我觉得，在芯片设计这类领域也会遇到同样的问题：对于更传统的算法来说，存在这类根本性的实际运行时间瓶颈。我们在去放置某个电路之前，会生成大量推理 token。Um I think uh uh it was maybe Baron Millage on Dwarsh recently who said you know um that perhaps you should think for a h 100red years you know before you fire off an experiment and then think for another hundred years of human equivalent with these models uh about the results of that AI experiment because the experiment in the middle is pretty expensive and takes takes a certain amount of wall clock time and so I think you get there also with things like chip design where there are these kind of fundamental wall clock time bottlenecks for the more conventional algorithms. We're going to be generating a lot of reasoning tokens um before we go and place a given circuit.

主持人：你们在押注工作负载，并与设计合作伙伴密切协作，嗯，提前预判模型架构的变化，希望能稍微领先一些，这样就能做好规划。你们有什么发现？Host: You are making workload bets and then collaborating closely with design partners uh on the model architectural you know shifts hopefully a little ahead so you can actually do some planning. What are you seeing?

嘉宾：我们正在做的最重要的事情，当然就是这种对速度的追求，也就是尽可能提升内存带宽，以便让这些模型运行得更快。我认为，作为一家芯片公司，你必须兼顾两方面：打造一种芯片，既能出色地应对即将涌现的各种工作负载——考虑到所谓的“硬件彩票”，这些模型将会在基于 HBM 的 GPU 或 XPU 上训练；同时，这种芯片也要非常擅长处理当今的自回归 Transformer，让它们运行得更快、更高效。不过我认为，打造具备全新基础能力的芯片令人兴奋之处在于，我们还能探索这些特性是否会产生某种引力效应，让我们得以亲自牵引新的模型格局朝着某个方向发展。Guest: The biggest thing that we're doing of course is this kind of speed chasing right maximizing memory bandwidth uh in order to be able to run these models faster. Um, and one of the dualities I think you have to span as a chip company is to build something that is both exceptionally good at the kind of gamut of workloads that are coming down the pipe under this idea of the kind of hardware lottery where these will be trained on HBM based GPUs or XPUs. Um, and so it's great that this is then a chip that's exceptional at taking kind of today's autogressive transformers and running them much faster and more efficiently. Um, but I think what's exciting about building chips with kind of fundamentally new capabilities is you then also get to explore whether there are kind of gravitational pulls that you yourself get to pull the new model landscape in because of those properties.

所以对 Fractile 来说，你知道，每颗芯片的带宽比 HPM 架构芯片高出大约 25 倍，这让我们能够探索一些想法，比如带宽的缩放定律。基本上，传统上我们所说的缩放定律，通常是指 FLOP 缩放定律。你知道，在训练和测试时，我们向这些模型投入的 FLOP 越多，性能就会越来越好。嗯，如果只看当下已知的各种思路，我们已经能看到一些带宽方面的缩放定律。比如混合专家模型，众所周知，理想情况下，我们会让它们越来越稀疏。所以，如果要在相同智能水平下运行模型，从每 6 个中激活 1 个，变成每 128 个中激活 1 个，甚至每 256 个中激活 1 个，就能节省大量 FLOP。And so for Fractile, you know, this idea of having like 25 times more bandwidth per chip than a than an HPMbased chip, we get to explore, for instance, internally these kind of ideas of like scaling laws for bandwidth. Basically, you know, the scaling laws traditionally we think of them as like flop scaling laws. You know, we uh we get better and better performance the more flops we pour into these models at train time and and test time. Um and if you look at just like the known landscape of ideas today, we already see some scaling laws for bandwidth. So one is uh these mixture of expert models, it's pretty well known that actually ideally we would make them sparer and sparer and sparser. So for like ISO intelligence, you will save a ton of flops if you go from being like 1 in6 sparse on your to 1 in 128, one in 256.

嗯，但要做到这一点，其中一个挑战、一个阻力，实际上是：在如今基于 HPM 的 GPU 和 XPU 上，高效服务这些模型的成本会高得令人难以承受。你往往会受到带宽瓶颈的限制，最终 MFU（浮点运算利用率）会非常非常低。嗯，所以即使你是为当下的应用场景构建系统，也有一些领域可以让你审视如何释放更强的能力。你知道，对于如今的 LM 来说，注意力机制其实也很类似。有些形式的注意力机制对带宽的需求较低；要达到一定的智能水平，它们往往会消耗更多浮点运算。所以我认为我们很兴奋的一点是，可以提升另一项特性——内存带宽这一属性，而近来芯片上的内存带宽并没有得到太多扩展。Um, but one of the challenges, one of the headwinds to doing that is actually that it becomes incredibly prohibitive on today's HPM based GPUs, XPUs to serve those models efficiently. You end up often bandwidth bottlenecked. you end up uh serving at really really low uh MFU low flops utilization. Um and so there are these areas where even as you build for today's world you get to kind of look at where you can unlock greater capability. And you know with LM today it's very similar actually with attention. There are forms of attention that are less bandwidth hungry. They tend to be more flops hungry for a certain level of intelligence. And so one of the things I think we're excited about is as we elevate this other quality, this property of memory bandwidth which hasn't really been scaled so much on chips recently.

过去 20 年里，我们的浮点运算量提升了大约一百万倍。同一时期，内存带宽提升了大约 40 倍。嗯，随着你不断拓展这条前沿，就能节省更多另一种资源。为了让这些模型达到一定的智能水平，我们可以减少所需的浮点运算量。嗯，所以这确实是影响这些模型全球吞吐量的一个乘数因素。We've scaled flops like a millionfold in the last 20 years. Memory bandwidth has gone up about 40x in the same time frame. Uh so as you scale that frontier, you get to conserve more of the other thing. We get to reduce the number of flops that we're using on these models to get to a certain level of intelligence. Um, and so that's a kind of really a multipplay playing factor on global throughput for these models.

主持人：我现在先忍住，不问你这会不会改变人们对自己该如何训练的看法。Host: I will resist asking you if that changes people's ideas about how they should train for now.

嘉宾：最后一个问题，想问问市场格局。嗯，我不太清楚，五年后大型 AI 玩家或超大规模云服务商会采购什么，或者说他们会使用哪些芯片——在英伟达、AMD 这些厂商和新一代加速器之间做选择。而且，至少有四家这样的公司在内部研发自己的芯片。你怎么看这个问题？Guest: One last question for you just on on market structure. uh here it's very unclear to me you know 5 years from now what a uh large AI player um or a hyperscaler buys in terms of or what what chips they consume between the Nvidia and AMDs of the world um the new accelerator class and you know four at least four of these players have their own internal efforts like how do you make sense of this

主持人：所以我觉得，今天我们看到的情况是，你知道，所有大规模部署的玩家似乎都在努力部署尽可能多的独立平台。嗯，我认为有一点仍会成立：随着计算能力对这些玩家变得如此生死攸关，市场确实需要多元化的供应来源。现在有个说法，算是个玩笑吧：这些厂商自研项目的主要目的，就是压低人们付给英伟达的价格。这话可能也有几分道理，因为这些项目在架构上其实相当相似，对吧？所以，这并不是在押注一种其他芯片无法实现的根本性能力。于是我觉得，这其实是一种博弈：你知道，我们能不能在这边自己造一些，或者我能不能从 AMD 买一些，这样或许就能从英伟达那里拿到更好的价格，诸如此类。Host: so I think that today we see uh you know behavior where really everybody who is deploying at scale is kind of trying to deploy as many separate platforms as they can. Um I think one thing that will remain the case is that there will be a real need for kind of a diversity of supply as something like compute becomes so existential to these players. There's a bit of a joke today that the sort of first party efforts their primary purpose is to reduce the price that people pay Nvidia. And there may be some truth to that as well because those efforts are kind of architecturally quite similar, right? So it's a it's a bet that is not on enabling a fundamental capability that those other chips cannot enable. And so there I think it is this kind of gamesmanship of you know can we build some of this over here or can I buy something from AMD so that maybe I then get a better price from Nvidia and so on.

嗯，这也是一种提升总体算力并增加一些控制力的办法。我认为，随着你们看到这种趋势转向能够带来全新能力的芯片，你知道，对于我们正在做的事情，我突然意识到，任何想在前沿领域部署 AI 的人都确实需要这种能力。他们必须找到某种方案，让这些模型的运行速度快上几个数量级。嗯，你知道，对于前沿实验室来说，拥有这种优越能力、优越智能的窗口期，就是他们存在的全部意义。你知道，否则我们大家早就一直在用 Kimi 模型了。嗯，所以，当开源模型从后方步步紧逼时，你知道，对于那些想在前沿领域部署 AI 的人来说，全面具备这方面的能力显然就变得更加重要了。Um and and a play also for kind of aggregate capacity and a bit of control. I think as you then see this move into chips that actually give new capabilities, you know, that's a place where, you know, I suddenly see for what we're doing a real need for everybody that wants to deploy AI at the frontier. They have to have some solution to running these models orders of magnitude faster. Um, you know, for the frontier labs, there's this window of kind of premium capability, premium intelligence that is their entire razon datch. You know, otherwise we'd all be using Kimmy models all the time. Um and so as you do have this pressure from behind from open source, you know, it does become clearly even more important for folks that want to deploy at the frontier to have every aspect of what that means.

因此，既要有最好的模型权重，也要能最快部署，这样你就能在最短时间内完成最多的推理。所以我认为，芯片领域中这种高端部分——也就是不惜一切代价优化速度的部分——会成为这个世界里非常重要的一环。我认为，芯片供应商和前沿实验室之间的这种关系，有一点值得注意。比如说 Fractile，我们努力让自己尽可能贴合前沿领域的需求，也努力让自己看起来有点像那些团队；我们也有一些人，他们会深入思考这些工作负载等等。所以我认为，有时候我们需要论证的一点是，为什么我们相信，未来几十年里，世界仍会容纳某种前沿的第三方芯片厂商？为什么这些厂商不会最终都被这些实验室收入囊中？So the best model weights, but also then the fastest deployment so you can do the most reasoning in the in the shortest amount of time. And so I think this kind of premium end of of of the chip space where you're optimizing for speed over everything else becomes a kind of very important part of this uh part of this world. I think one thing that is notable about you know the dynamic between a chip vendor and a frontier lab you know if you look at fractile we try to be as aligned as possible with the needs of the frontier we try to look a bit like some of those teams as well we have people that really think deeply about these workloads and so on so one of the cases that I think we sometimes need to make is explaining why we believe that over you know multiple decades the world sustains some kind of frontier third party chip players you know why does this not just roll up into these into these labs?

嗯，我认为这归根结底是所有这些实验室和前沿领域参与者都不得不玩的一种非对称博弈。如果他们孤注一掷押注某种硬件，就会面临巨大的风险。比方说，我是实验室 1，已经把全部筹码押在某种专有硅芯片上；随后实验室 2 发现了一项新的计算突破，能在同等智能水平下大幅提升计算效率，但这项突破只适用于他们决定部署的那款芯片。等到我部署了足够多的这种芯片、也获得了 5 倍的计算效率提升时，可能 9 个月都过去了，而在此之前我就已经撑不下去了。所以，这些公司确实迫切需要部署相同的芯片、采用相同的平台。他们玩的游戏各不相同，对吧？Um, and I think that comes down to the kind of asymmetric game that all of these labs and frontier players have to play. Uh, where they're exposed to an enormous risk if they go all in on a hardware bet. Um suppose I'm you know I'm lab one uh and I've gone all in on some proprietary silicon and then lab 2 discovers some new computational breakthrough that delivers far better computational efficiencies for the same level of intelligence but it only works on the chip that they've decided to deploy. I could die in the 9 months before I get to deploy enough of that chip that I've also now gained that kind of 5x in computational efficiency. And so there is a really urgent need for these chips to actually deploy the same these companies to deploy the same platforms as one another. They're playing different games, right?

他们现在也在试图进军中间件领域。嗯，我认为，对于任何身处前沿领域的参与者来说，在芯片厂商这一层面押注高度差异化的方向，并在这些差异化押注上全力以赴，是一种不理性的做法，也非常危险。They're trying to now compete, you know, in the middleware. Um I think having deeply differentiated bets and going all in on those differentiated bets at the chip player is uh sort of an irrational and a very dangerous move for anybody that is playing at the frontier.

嘉宾：嗯，我觉得在这里结束正合适。非常感谢你，沃尔特。Guest: Well, I think that's a great place to end. Thanks so much, Walter.

主持人：非常感谢你，Sarah。欢迎在 Twitter 上关注我们，账号是 no prior pod。如果想看看我们的样子，可以订阅我们的 YouTube 频道。请在 Apple Podcasts、Spotify 或你收听节目的平台上关注本节目。这样你每周都能收听到新的一期。你也可以在 no-briers.com 注册接收邮件，或查阅每期节目的文字稿。Host: Thanks very much, Sarah. Find us on Twitter at no prior pod. Subscribe to our YouTube channel if you want to see our faces. Follow the show on Apple Podcasts, Spotify, or wherever you listen. That way, you get a new episode every week. And sign up for emails or find transcripts for every episode at no-briers.com.