为什么 AI 算力以矩形合约出售——Modal CEO Erik Bernhardsson

热烈欢迎 Basil Chatha 加入我们，成为来自 AI 工程领域前沿部署阵营的最新联合主持人！我们将和他一起筹备一档新节目，也不能错过趁 Modal 完成重磅 D 轮融资之际采访其 CEO Eric 的机会！Warm welcome to Basil Chatha, our newest cohost from the Forward Deployed side of the AI Engineering world! We’ll be ramping up a new show with him and couldn’t pass up an interview with Modal’s CEO Eric on the occasion of their monster Series D!

两年前，我们曾与 Erik Bernhardsson 谈论他对真正无服务器 AI 基础设施的愿景：开发者应该能够运行代码，而无需配置集群、预测负载或为闲置资源付费。随后，首席技术官 Akshat 加入我们，介绍 Modal 的基础设施原语，以及用于提供这些原语和智能体体验的技术栈。Two years ago, we spoke with Erik Bernhardsson about his vision for truly serverless AI infrastructure: developers should be able to run code without provisioning clusters, predicting load, or paying for idle resources. CTO Akshat then joined us to talk about Modal’s infra primitives and stack to offer those primitives and agent experience.

过去 5 个月发生了很多变化。自完成 50 亿美元的 C 轮融资以来，Modal 现在表示，其年化营收已突破 10 亿美元，并以 160 亿美元的估值完成了 7.5 亿美元的 D 轮融资，同时已启动近 30 亿个沙箱。在这次对话中，Erik 与 Vibhu 一起阐述了推动这一增长的基础设施和经济因素，以及为什么计算资源的需求往往与购买计算资源所采用的合约形式大相径庭。A lot has changed in the past 5 months. Since their $5B series C, Modal now reports it crossed $1 billion in annualized run rate, raised a $750 million Series D at a $16B valuation, and has spun up nearly three billion sandboxes. In this conversation, Erik joins Vibhu to explain the infrastructure and economics behind that growth—and why demand for compute rarely looks like the contracts used to buy it.

在 YouTube 上观看完整对话 →Watch the full conversation on YouTube →

GPU 需求并非矩形形状GPU demand isn’t rectangular

典型的容量合同会在固定期限内为客户提供固定数量的 GPU。将容量与时间绘制成图，就会得到一个矩形。实际工作负载的表现则不同：它们会在一天中不断增长、激增和回落。若按峰值需求购买足量资源，就要为闲置硬件付费；若购买不足，则可能无法满足需求。A typical capacity contract gives a customer a fixed number of GPUs for a fixed period. Plot capacity against time and you get a rectangle. Real workloads behave differently: they grow, spike, and fall throughout the day. Buy enough for the peak and you pay for idle hardware; buy less and you risk missing demand.

Erik 将 Modal 的押注描述为一种期限转换：汇集数千家客户的需求，在底层确保容量，再让每位客户在需要时使用所需的资源。总体需求越平稳，这个资源池就越有用。这也为我们与 SF Compute 的 Evan Conrad 的对话提供了一个很好的补充：Evan 专注于让容量可以交易；Erik 则专注于让人们能够以更小、更具弹性的份额使用容量。Erik describes Modal’s bet as a kind of duration transformation: pool demand across thousands of customers, secure capacity underneath, and let each customer use what they need when they need it. The smoother the aggregate demand, the more useful that pool can be. It’s a useful companion to our conversation with SF Compute’s Evan Conrad: Evan focuses on making capacity tradeable; Erik focuses on making it usable in smaller, elastic slices.

另一方面也存在财务风险。如果服务商按当前价格锁定算力，而 GPU 价格之后下跌，就可能被昂贵的算力供应套牢。Erik 表示，他不认为价格会很快暴跌，但认为在构建业务时必须考虑这种可能性。跳转至 GPU 价格讨论 →There is a financial risk on the other side. If a provider locks in capacity at today’s prices and GPU prices later fall, it can be left holding expensive supply. Erik says he doesn’t expect an imminent collapse, but thinks the possibility matters when building the business. Jump to the GPU price discussion →

这个差点没能建成的沙盒The sandbox that almost didn’t make it

Erik 和联合创始人 Akshat Bubna 注意到，语言模型能够生成代码，而这些代码需要一个运行的地方，于是 Modal 推出了沙箱。早期采用情况并不理想，团队甚至讨论过是否要关闭这款产品。后来，编程智能体、后台智能体和强化学习带来了新的需求。Erik 表示，沙箱业务如今约占 Modal 营收的三分之一，这个结果是他在产品上线时没有预料到的。Modal launched sandboxes after Erik and cofounder Akshat Bubna noticed that language models could generate code—and that someone would need a place to run it. Early adoption was modest enough that the team discussed shutting the product down. Then coding agents, background agents, and reinforcement learning created new demand. Erik says sandboxes have grown to roughly a third of Modal’s revenue, a result he didn’t predict at launch.

这种增长也改变了基础设施需要承担的职责。智能体可能会创建环境、运行代码、检查结果，然后反复重试。启动和恢复这些环境所需的时间，也成了体验的一部分。在最近对 Akshat 的采访中，我们探讨了从开发者体验转向智能体体验的变化；在这里，Erik 则补充了产品和商业层面的故事。跳转至沙箱讨论 →That growth also changes what infrastructure has to do. An agent may create an environment, run code, inspect the result, and try again many times. The latency of starting and restoring those environments becomes part of the experience. In our recent interview with Akshat, we explored this shift from developer experience to agent experience; here Erik adds the product and business story. Jump to the sandbox discussion →

快速启动是一项基础设施功能Fast startup is an infrastructure feature

只有当容量能够足够快地到位、及时发挥作用时，弹性才有意义。Modal 关于无服务器 GPU 的介绍描述了背后的机制：一组可用 GPU、自定义文件系统、进程检查点和 GPU 内存快照。这些环节减少了启动另一个模型副本所需的工作量。这是一个具体例子，说明基础设施决策如何影响应用的响应速度和成本。Elasticity only works if capacity can arrive quickly enough to matter. Modal’s serverless GPU write-up describes the machinery: a buffer of available GPUs, a custom filesystem, process checkpointing, and GPU memory snapshots. Those pieces reduce the work needed to bring another model replica online. It is a concrete example of how infrastructure decisions show up in an application’s responsiveness and cost.

我们还将讨论 Erik 从 Spotify 推荐系统到 Modal 的职业历程、公司如何采购算力、NVIDIA 的替代方案、前置部署工程师，以及为什么在 Modal 代码审查是可选的。跳转到工程文化 →We also discuss Erik’s path from Spotify’s recommendation system to Modal, how the company buys compute, alternatives to NVIDIA, forward-deployed engineers, and why code review is optional at Modal. Jump to engineering culture →

节目说明Show notes

• Modal——Erik 创立的 AI 基础设施平台。

  Modal — the AI infrastructure platform Erik founded.

• Erik 的写作——他关于基础设施、市场和创办公司的文章。

  Erik’s writing — his essays on infrastructure, markets, and building companies.

• Modal 的 D 轮融资公告——公司最新公布的增长和沙盒里程碑。

  Modal’s Series D announcement — the company’s latest reported growth and sandbox milestones.

• Modal 如何实现真正的无服务器 GPU——弹性推理背后的启动与扩缩容技术。

  How Modal achieved truly serverless GPUs — the startup and scaling techniques behind elastic inference.

• Modal 的资源求解器——它如何在价格和可用性不断变化的情况下选择容量。

  Modal’s resource solver — how it selects capacity across changing prices and availability.

• 扩展至一百万个并发沙箱——沙箱规模化的工程实践记录。

  Scaling to one million concurrent sandboxes — an engineering account of sandbox scaling.

• 我们与 Erik 的首次访谈——关于无服务器基础设施的最初论点。

  Our first interview with Erik — the original serverless infrastructure thesis.

• 我们对 Akshat Bubna 的采访——关于智能体的基础设施与开发者体验。

  Our interview with Akshat Bubna — infrastructure and developer experience for agents.

• 可编程数据中心——Evan Conrad 谈 GPU 交易与实物结算。

  The Programmable Data Center — Evan Conrad on GPU trading and physical settlement.

时间戳Timestamps

• 0:00 Modal 的功能

  0:00 What Modal does

• 1:25 From Spotify to building Modal

• 4:19 在生成式 AI 热潮兴起之前创立

  4:19 Founding before the generative AI boom

• 7:43 首批客户与融资

  7:43 First customers and fundraising

• 10:00 构建推理与自动扩缩容能力

  10:00 Building for inference and autoscaling

• 13:21 Modal 从哪里获得 GPU 容量

  13:21 Where Modal gets GPU capacity

• 15:39 租用还是自有基础设施

  15:39 Renting vs. owning infrastructure

• 19:45 GPU价格下跌的风险

  19:45 The risk of GPU prices falling

• 24:18 AI需求是无限的吗？

  24:18 Is AI demand infinite?

• 26:12 NVIDIA 的替代方案

  26:12 Alternatives to NVIDIA

• 28:25 Modal 可以从新云厂商那里学到什么

  28:25 What Modal can learn from neoclouds

• 30:26 GPU 作为一种资产类别

  30:26 GPUs as an asset class

• 31:55 Modal 的产品：函数与沙箱

  31:55 Modal’s products: functions and sandboxes

• 36:03 采用更小训练工作负载的理由

  36:03 The case for smaller training workloads

• 41:21 智能体沙箱如何演变

  41:21 How agent sandboxes are changing

• 46:48 Anthropic and sandbox security

• 49:29 使用智能体进行安全测试

  49:29 Using agents to test security

• 52:51 智能体时代的开发者体验

  52:51 Developer experience in the age of agents

• 57:45 Modal 的前沿部署工程师负责什么工作

  57:45 What Modal’s forward-deployed engineers do

• 59:55 Modal 的产品管理方法

  59:55 Modal’s approach to product management

• 1:03:29 可选的代码审查与工程文化

  1:03:29 Optional code reviews and engineering culture

• 1:05:56 云端、本地部署和自带云模式

  1:05:56 Cloud, on-prem and bring-your-own-cloud

• 1:09:25 能力、募资与增长

  1:09:25 Capacity, fundraising and growth

• 1:13:07 扩大团队规模

  1:13:07 Scaling the team

• 1:17:28 Erik 的推文：临时沙箱

  1:17:28 Erik’s tweets: ephemeral sandboxes

• 1:18:31 矩形 GPU 合约的问题

  1:18:31 The problem with rectangular GPU contracts

• 1:21:20 推理服务提供商 vs. 模型构建者

  1:21:20 Inference providers vs. model builders

• 1:23:25 模型接下来将如何改进

  1:23:25 Where models improve next

文字稿Transcript

根据修订后的编辑内容自动转录；姓名和措辞可能有误。Automatically transcribed from the revised edit; names and wording may contain errors.

0:00 感谢你来做客，Eric Fern-Hartson，来自 Modal。能不能介绍一下你们的业务，以及你是怎么走到今天这一步的？可以。简单来说，我们打造的是 AI 基础设施。我们提供计算平台和存储平台，让 Dacagon、Suno、Runway、Cognition 等许多客户能够构建 AI 应用。实际来说，这意味着大量推理任务、大量沙盒环境，有时还包括训练和微调、批处理作业，以及其他任务。我们基本上打造了一套软件 SDK，让客户能够快速行动并构建这些应用。我们负责管理底层的所有算力资源，让部署、扩展和在生产环境中运行这些应用变得更容易。两年前你上过 Light and Space，从那以后发生了很多变化。当时你们规模有多大？可以分享哪些数据？和现在相比，现在有多大？我其实不记得当时的情况了，但规模可能至少增长了 50 倍吧，不管是营收还是用户数量。是啊。我自己有个副业项目，确实在用你们的产品。不错。是啊，我用它把 2D 图像转换成 3D 模型。对，我在这里托管了几个模型。你觉得怎么样？有什么反馈吗？我很喜欢。不错。很好。我用它就是因为比其他竞品什么的都好。很好。你创办 Modal 之前，在 Spotify 工作过，也在 better.com 工作过，对吧？0:00 Thanks for coming on, Eric Fern-Hartson from Modal. Could you explain the business a little bit and maybe how you got here? Yeah. So we build, the easiest way to put it is we build AI infrastructure. So we offer a compute platform and a storage platform that enables a lot of customers like Dacagon, and Suno, and Runway, and Cognition, and many others to essentially build AI applications. So what that means in practice is a lot of inference, a lot of sandboxes, sometimes training and fine-tuning, sometimes batch jobs and other things. So we have basically built a software SDK that enables customers to move really fast and build this. We manage all the capacity underneath, make it easier to deploy and scale and run these things in production. So you were on Light and Space like two years ago, a lot has happened since then. How big were you back then, like whatever numbers you can share and comparatively, how big are you now? I actually don’t remember where we were, but it’s probably like at least 50X, there’s revenue and users and users. Yeah. I use it actually for a side project that I have. Nice. Yeah. It’s like, yeah, turn 2D images into 3D models. Yeah. So yeah, just hosting a couple of models here. What do you think? Any feedback? I mean, I love it. Nice. Good. I mean, yeah, that’s why I use it over any competitors or anything. Good. So before you started Modal, you were at Spotify and also like better.com, I think.

1:30 没错。是啊，是啊。你在 Spotify 做什么？其实我在 Spotify 做过很多不同的事情，但我最主要的贡献是构建了一套音乐推荐系统。我大概是在 2008 年或 2009 年毕业后加入的，在 Spotify 待了七年。那段时间里，我基本上从零开始构建音乐推荐系统，组建了一支小团队，还推出了很多功能，比如“相似艺人”“每周发现”等等。所以做了很多矩阵分解，也做了很多计算密集型的工作。可以说，后来逐渐发展成 Modal 的那些想法，很多都源于我当时构建 AI 应用时意识到基础设施真他妈糟糕。肯定有更好的办法。所以，Modal 背后的很多理念，说到底就是想做出一款我在 Spotify 工作期间一直想用的工具。对。所以 Modal 更像是你在 Spotify 学到的东西的延伸，和 Better.com 关系不大。Better.com 算是一次小小的绕道。那段经历就像坐过山车，说来话长。我们之前单独聊过。不过我在 Better.com 做了五年 CTO，把技术团队从大概两三个人发展到三四百人。我学到了很多管理、组建团队和招聘方面的知识，也学到了很多关于提升开发者效率的东西。我想了很多工具之类的事情，也做了更多全栈开发。不过后来我还是回到了自己最在意的领域，也就是基础设施、AI 和机器学习。Better.com 当时是做什么的来着？房贷。明白。你是怎么进入那个行业的？1:30 That’s right. Yeah. Yeah. What were you doing at Spotify? I actually did a lot of different things at Spotify, but my main contribution was I built a music recommendation system. The first version I joined after school in 2008, 2009 maybe, and spent seven years at Spotify. And most of that time was building basically from scratch and music recommendation system and built up a small team and we launched a bunch of features like related artists and Discover Weekly and a bunch of other stuff. So a lot of, a lot of matrix factorization, a lot of compute intensive stuff. And arguably like a lot of what sort of later became the genesis of Modal was just building AI applications and realizing like the infrastructure fucking sucks. Like, there’s got to be a better way to do that. So a lot of sort of, you know, the rationale for Modal was like kind of building a tool I always wanted to have throughout my years at Spotify. Yeah. So Modal was more of an offshoot of the stuff you learn at Spotify, not much from Better.com. Better.com was a little bit of a side tour. It was kind of a roller coaster, a long story about the company. We talked about it separately. But I was a CTO for five years of better and grew to a tech team from, I think, you know, two to three, four hundred. So I learned a lot about management and building teams and hiring and also learned a lot about developer productivity. I thought a lot about tools and things like that. Did a lot more like full stack stuff. But yeah, I kind of came back to my, was close to my heart, which is infrastructure and AI and machine learning after that. What was Better.com doing again? Mortgages. Okay. Yeah. How did you get into that?

3:00 我的意思是，我们没必要在这上面花太多时间。嗯，不，没事。就像这个人，扎克·弗兰克尔，我当时并不认识他，后来他把我招去了 Better 担任 CTO，那时公司只有 10 个人。你知道，出于某种原因，Vishal 的创始人是个很有意思的人，非常非常聪明，也有点疯狂，不过大多是积极意义上的疯狂。我们相处得特别好。所以，嗯，我们在 2015 年开始做这家公司，我花了五年时间发展它。后来你知道，最终发现做房贷业务挺难的，而且疫情之后利率迅速上升，给公司造成了很多问题。不过那时候我已经离开了。 那是你离开的时间，大概是 2020、2021 年？对，差不多。嗯。所以，你是在想到 Model 这个点子之前离开的，还是离开之后才开始创业的？ 这很难说。我离开 Better 时有很多零零散散的想法，而且我其实不确定自己想不想创业。疫情期间，我基本上算是休了个长假，家里有两个孩子。我的确还是有孩子，不过我是说，当时他们应该分别两岁和四岁。所以我决定多陪陪他们，休息一阵子，同时也做些兼职，捣鼓一些东西。我当时不太确定，也许该做些咨询之类的。后来我开始做东西，突然就对这个后来发展成 Model 的想法特别兴奋。不过实际上，Model 大概是我做的第 30 个项目吧——在它之前，我还尝试过几个，最后都没做成。3:00 I mean, we don’t have to spend too much time on that. Yeah, no, it’s fine. Like this guy, Zach Frankel, who I didn’t know at that time, ended up recruiting me to Better as a CTO and the company was 10 people. And, you know, for whatever reason, you know, the founder of Vishal, very interesting guy, very, very smart, a little crazy in mostly positive ways. We got very well along. So we, yeah, we started building it in 2015 and I spent five years growing that company and then, you know, eventually turned out that doing mortgages is pretty hard and, you know, interest rates went up very quickly after the pandemic and that cost a lot of issues at the company. But by then I had already left. That was like 2020, 2021 when you left? Yeah, something like that. Yeah. So, OK, you left before you had the idea for a model or you left and then you started? It’s hard to say. I had a lot of random ideas when I left Better and I, I actually wasn’t sure if I wanted to start a company. I was kind of just, you know, during the pandemic, I sort of almost took like a sabbatical at two kids. I mean, I still have kids, but I mean, at that time there were there were I think two and four and so I decided to hang out and took some time off and was kind of working like part-time, like hacking on certain things and wasn’t sure maybe I should do some consulting and I started building stuff. And I just got like super pumped about this idea that eventually became model. But actually both, you know, model was like, I think the 30 I built that I got, you know, that so there were like a few attempts before then that went nowhere.

4:30 但我的想法是，第一个点子让我觉得挺酷的，觉得自己应该做这个。这可能会成为一个很酷的平台。其他点子也和开发者工具类似吗？最初那个是某种网页抓取工具，最后不了了之。不过说实话，现在回头看，也许它本来是有机会做起来的。这是在 ChatGPT 发布之前吗？对，对，2021 年。哦，明白了。那 ChatGPT 是 2022 年底推出的，对吧？后来我就开始着手做了。所以，或许还是稍微倒回去补充一点背景：我在 Spotify 工作时，发布过一个名为 Kaluigi 的开源框架，用于工作流编排。算是同一类东西，比如 Airflow 和 Prefect 之类的，只不过这些类别在某种程度上已经有点过时了，人们不怎么提起。但有一阵子它确实受到了很多关注。总之，我从没把它变成一门生意。但几年后，也就是 2021 年，当我在考虑商业点子或想做的项目时，我又想到了工作流编排。我开始试着琢磨各种点子，比如如何在云端运行代码，基本上就是编排云端的代码执行。后来我意识到，说到底，核心难题不只是编排计算机，更是执行计算机上的任务。我觉得这其实是个更大的问题。所以某种程度上，我把它先搁到一边，转而去做另一件事，想着之后再回头做工作流调度。但总之，长话短说，我最终没有回头继续做它，而是做出了 Modal，它的重点更偏向于：我们就想办法让人们能轻松地在云端运行代码。最初的定位相当通用。4:30 But my model was like the first idea was like, this is kind of cool. Like I should do this. This is like, this could be a really cool platform. Were the other ideas like also in a similar vein like developer tools? The first thing was like some sort of web scraping thing, which went nowhere. But actually, you know, in hindsight, maybe could have gone somewhere. And was this before the chat GBT launch? Yeah. Yeah. 2021. Oh, OK. So chat GBT was late 2022, right? Then I started building, so maybe actually to rewind a little bit context, like when I worked at Spotify, I released an open source framework, Kaluigi, which is like workflow orchestration. So sort of the same categories, airflow and pre-effect index or sort of dated categories to some extent, like people don’t talk about as much. But for a while, it was like, you know, got a lot of attention. Anyway, I never turned it into business. But a couple of years later, when in 2021, when I was thinking about business ideas or things to work on, I kind of came back to workflow orchestration. And I started hacking on a bunch of ideas of, you know, running code in the, you know, like basic orchestrating code running in the cloud. And what I realized was like, you know, at the end of the day, like core puzzle pieces, not just like orchestrating the computer, it’s like executing the computer. And I was like, that’s kind of a bigger problem actually to solve. So in a way, I actually like, you know, left it on the shelf and then said, like, actually, I’m going to like go to this other thing over here and I’ll go back. I’ll get back to the workflow scheduling a little bit. But, but like, you know, I never got back to long story short and ended up building modal, which is like the focus is more like, let’s just figure out a way to make it easy to run code in the computer, in the cloud. And the focus was initially like fairly general purpose.

6:00 实际上，某种程度上我们一直都是这样，最近又有点兜了一圈回到原点。不过我们之前关注过很多不同的应用。我们的想法大概就是：让大家能非常轻松地在云端运行代码。可以是 GPU 代码、AI、机器学习，也可以是数据相关的东西。我们花了很多精力思考如何扩展规模，如何打造出色的开发者体验。我们很早就意识到，要做到这一点，就得深入技术栈底层，于是我们构建了文件系统和容器运行时，也解决了很多棘手的工程难题。然后你提到了 ChatGPT，它其实还不算真正的杀手级应用。不过，如果你还记得的话，那年秋天前后，差不多同一时间还出现了另一款东西：Stable Diffusion。结果 Stable Diffusion 成了我们当时正在打造的产品的杀手级应用。那时我们只有四五个人。后来我们发现很多人来找我们，想运行 Stable Diffusion 模型。于是我们意识到，生成式 AI 推理可能真的是一个非常适合我们的应用场景。你是怎么发现的？我就是个发推狂魔。以前比现在更狂，因为现在忙着创业，注意力很容易被分散，但我一直发很多推文，也写很多博客。我想自己当时积累了一些关注者。所以在早期，推动增长的很大一部分原因，就是我不停地发推，聊模型、聊我正在构建的东西，让大家兴奋起来，还写很长的博客文章，讨论云的发展方向以及诸如此类的话题。我把这称作一种创始人主导的营销。我总觉得，你得充分发挥自己的长处。6:00 And it actually in a way like it’s always, we’ve kind of gone full circle more recently, but, but we focused on so many different applications. It’s kind of the idea was like, let’s just make it super easy to run code in the, in the cloud. Could be GPU code, AI, machine learning. It could also be like data stuff. And focused a lot on, you know, how do we scale it up? How do we make the developer experience awesome? Realized super early that like in order to do that, we’d have to go very deep in a stack and built on file system, built on container runtime and did a lot of, you know, solved a lot of hard engineering challenges. And then you mentioned chat, CPT, that was actually not quite the killer app. But like, if you remember, like that fall, another thing came out around the same time, which was stable diffusion and stable diffusion turned out to be a killer app for our, for what we’re building at that time. We were only four or five people at that time. So we started seeing a lot of people coming to us and wanting to run a stable diffusion model. And then we were real, that, that, that made us realize like maybe gen AI inference is actually a really good use case for us. And how are you finding it? I’m just like compulsive tweeter. I was even more like now, like I’m too distracted by building a company, but I always like tweeted a lot. I was blogged a lot. So I think I had a little bit of a following. And so, you know, in the early days, I like a lot of what drove the growth was just like me just compulsively just like tweeting about model and like what I was building and getting people excited and writing long blog posts about where the cloud is going and all these things. And I call it a bit like founder led marketing. I always like feel like you kind of have to lean into like what you’re good at

7:30 我擅长的，我想大概就是发些垃圾帖，还有整天泡在网上。所以你知道，工作上也是这样。我觉得我们的很多增长，都只是因为我们挺擅长社交媒体，而且经常在线。不错。你们花了多久才有了第一个客户？花了挺长时间。还有，你们早期就融资了吗？我们过了挺久才融资。其实有段时间都是我自己出钱，甚至一直到最初几位员工入职之后也是这样。某种程度上这还挺好的，因为这让我对钱特别节省，也很看重投资回报率。我很长一段时间都没给自己发工资。后来有一阵子这成了问题，因为我需要钱，但我们很久都没有融资。直到2022年初，我们才融了种子轮。那时候我已经为这个项目忙活了一年左右了。大约五年前，也就是差不多现在这个时候——可能是2021年8月——Akshah 加入，担任 CTO 和联合创始人。但直到2022年下半年，我们才终于有了像样的收入。我想这就是我为你感到骄傲的原因吧。是啊。对我们来说，Stable Diffusion 就是那个关键因素。我们当时的月收入还是100美元左右，我想那是2022年11月。接下来一个月，月收入就到了大约2000美元。然后到了2023年1月，突然就做到了大约3万美元。那时我终于觉得，压在心头的巨大焦虑一下子消散了。我想，也许我们真的找对方向了，感觉我们真的要做成了。接着，2023年我们增长得非常快。7:30 and like what I’m good at is like, I guess like just shit posting and being terminally online. And so, you know, like work, it’s still like, you know, I think a lot of our growth is we’re just kind of good at social media and just like being, you know, online a lot. Nice. How long until you had your first customer? It took a while. Also, did you raise like early on? We didn’t raise for a while. I actually self funded it for a while, even with up until the first few employees, which in a way was like kind of nice because it may be really like frugal about money and like wanting to see like ROI. I wasn’t taking out a salary for a while for a long time. Actually, at some point it became a problem because I needed the money, but we didn’t raise for a while. And until early 2022, we raised the seed rounds. So at that point I’d already been working on it for about a year. Akshah joins the CTO and co-founder in about five years ago, about now August, maybe 2021. But then it took until like basically like late 2022 for us to see any meaningful revenue. I guess that’s why I’m proud of you. Yeah. And stable diffusion then in our case was the main thing that’s sort of, so we went from like, you know, $100 a month in like, I think like November, 2022. And the next month we did like $2,000 a month. And then like January, 2023, we did like 30,000 all of a sudden. And, you know, so finally I felt this like massive, just like anxiety thing off my chest. I’m like, maybe this is actually like we’re onto something. It feels like we’re onto something. And then 2023 we grew very fast.

9:00 我想从那以后，我们基本上一直在快速增长。月收入达到 3 万美元的那个阶段，是因为几个大客户，还是客户群很多元？就一个。我不确定我以前有没有说过那家公司是谁，也不知道他们是否知道这件事，不过我觉得他们应该猜到了。那是一家在纽约的公司，叫 Mage。他们做的是 Stable Diffusion 相关业务。所以在 2023 年第一季度，我想他们可能占了我们收入的 80% 到 90%。但之后，收入来源就开始变得更多元了一些。2023 年底，我们很快又迎来了一家大客户 Suno。他们有一段时间也占了我们收入的很大一部分。不过随着时间推移，我们现在的收入来源实际上已经非常多元了。 说到 Stable Diffusion，我记得当时用它来跑 ComfyUI。我猜很多人都在用 ComfyUI，比如在上面运行 ComfyUI 工作流。确实有一些人这么做，但我觉得用 Modal 的那些人，往往更偏底层一些。他们会做些复杂的事情。他们喜欢 Modal 的一点是，它一直都很好折腾。他们会部署自己的自定义工作流，使用适配器和自定义推理，还会做很多优化。而我们为了让开发者体验变得非常好所做的一切，结果也特别适合运行生产工作负载。因为自动扩缩容其实是个非常相似的问题：你希望容器能够非常快速地扩缩。所以我们早期恰好很适合承接这类工作负载。那时候没有自动扩缩容的工具吗？为什么你们得自己开发？9:00 And I guess we’ve been growing very fast since then basically. That 30 K a month moment, was that just like a couple of big customers or was it like very diverse? It’s just one. And I don’t know if I ever talked about like who that was and if they even know it, but I think they suspect it. It was a company in New York called Mage. They’re doing like a stable diffusion company. So at some point, you know, in the first, I think first quarter of 2023, I think there were like 80, 90% of our revenue. But then after that, you know, it started diversifying a little bit. And late 2023, we got soon as a big customer. They were a big chunk of our revenue for a while too. But then, you know, it’s over time, like we actually have really good revenue diversification now. Yeah. For the stable diffusion stuff, I remember using it for like comfy UI. I’m assuming a lot of people were using comfy UI, like running comfy UI workflows on it. Some people were, but I think the people that were using model, they tended to be more like low level stuff. Like they did like complicated stuff. What they like with models, like it’s always been pretty hackable. So they deployed their own custom workflows with adapters and custom inference and a lot of optimizations and all the stuff we had done in order to get the developer experience to be really good, turn out to be really good for also running production workloads. Cause auto scaling turns out to be actually a very similar problem. Like you want to scale up containers up and down very fast. So we just happened to be like very well positioned to capture a lot of those workloads in the early days. Did tools for auto scaling like not exist? Like why did you have to build those yourself?

10:30 总的来说，我的核心观点是，每一个新的技术时代都会遇到工具的局限。你回顾一下，无论是 Web、云、移动设备、PC，还是 AI，都是如此。所以我认为，工具往往是存在的，但通常并不是特别好用。为了从中获益，真正加速发展，并开始为这个新世界构建东西，我们往往需要重新思考很多工具。云计算时代就是这样，我们不得不发明 Kubernetes、Docker 之类的东西。类似地，对于生成式 AI，你可以基于上一代工具来构建这些东西。比如，你可以在 Kubernetes 和 Docker 之上构建，但这些技术是为截然不同类型的工作负载打造的。它们是为非常稳定的工作、稳定的后端工作负载打造的，处理的请求数量也比较稳定，而且运行时的利用率相当低，因为 CPU 通常很便宜。所以人们并不太担心成本。但现在我们进入了一个新的世界：你有非常昂贵的 GPU，需要管理容量和利用率；你有非常大的模型，需要通过网络将它们加载到 CPU、再加载到 GPU。结果发现，很多这类工具根本不是为此打造的。所以我们觉得，必须重建其中很多东西。就像我提到的文件系统，我们过去几年还花了很多时间，打造了一些围绕 CPU 的很酷的东西。10:30 In general, like my thesis is like, you know, every new sort of era of like, you know, technology kind of runs up against limitations of tools. You know, whether that’s you look back, you know, like the web or the cloud or, you know, mobile or PCs or, you know, AI or whatever. So I think, you know, it’s often the case that tools exist, but, but they’re often not like particularly good. And often you kind of need to rethink a lot of those tools in order to reap the benefits of, you know, and really turbo charge and start building for this new world. And you saw this with cloud, you know, we have to invent things like Kubernetes and Docker and things like that. And in a similar way, you know, like with, with gen AI, you could build these things on the sort of prior generation of tools. Like you can, you can build these things on top of Kubernetes and Docker and in these things, but, but, but those technologies are built for a very different type of workload. They’re built for a very sort of steady work, steady back end workloads. They have an even, you know, steady number of requests, like running a pretty low utilization because CPUs are normally pretty cheap. And so people don’t really worry so much about cost. And, but now we’re entering this new world and you have, you know, very expensive GPUs you want to manage capacity and utilization. You have very large models that you need to load over network into CPU into GPU. And, and it turns out like a lot of these tools, like, you know, like they were just not built for that. So we felt that, you know, we had to rebuild a lot of that, you know, building our own, I mentioned the file system, but we also spent a lot of time over the last few years, like building, you know, some really cool stuff around like CPU

12:00 的快照、GPU 快照和审计，我们都基于自动扩缩容器、调度器，以及各种各样的容量管理工具来构建。我们觉得这么做是必须的，因为你没法基于旧一代的东西，为新一代软件构建……你没法在旧系统上再套一层，来实现新功能。某种程度上，你得从头开始重新做一些东西。 说到自动扩缩容，你们有没有在平台上遇到什么奇怪的行为？你们扩容时有没有什么东西出过问题？我的意思是，容量一直以来时不时都会是个挑战。你要怎么弄到几百、几千，甚至几万块 GPU？这始终都很有挑战性。我们很早就必须跨多个区域部署。所以现在回头看，我觉得这是个影响深远的决定：我们当时下定决心，咬咬牙把它建起来。我们一直把容量管理看成一个大池子，就像一个覆盖全球的大池子。我们不会把它看作各个区域里分别运行的独立集群。所以，早期我们做出的一个非常重要的决定，就是从这个角度来思考问题。 所以我不确定有没有什么特别具体的问题。我是说，我们显然遇到过 bug，也发生过宕机。虽然我当然希望模型能做到 100% 正常运行，但我想最近我们的可用性基本达到了四个或五个九。不过纵观 Modal 的发展历程，我们当然也遇到过一些扩容方面的问题。 是啊，GPU 市场——不过在说这个之前。你们底层用的是 AWS、GCP 之类的云服务提供商，对吧？12:00 snapshotting, GPU snapshotting and audit, we built on autoscaler and scheduler and, and, and in all kinds of capacity management tools. And we kind of feel like you have to do that because like you can’t build, you know, for this new generation of software on top of, you can’t build like a wrapper on top of the old stuff in order to do the new stuff. You kind of need to start over a little bit from scratch. So I guess like the auto scaling, like did you, did you see any like weird behaviors on the platform? Like was there anything that like broke while you guys were scaling? I mean, capacity is always from time to time been, been a challenge. Like how do you get like, you know, hundreds and thousands and tens of thousands of GPUs? So that’s, that’s always challenging. We had to go multi region very early. So I think that was like kind of a fateful decision in hindsight, like doing that and kind of just biting the bullet and building it. Like we always thought of like our capacity management as like one big pull. Like it’s like one big pull over the entire world. Like we don’t think of it as like separate clusters running in each region. And so that was like one of the like very consequential decisions we made early to kind of think about it that way. So I don’t know if there’s like been like, you know, specific issues. I mean, we’ve obviously had bugs and, you know, outages and I hate, you know, as much as I, you know, would love for a model to be a hundred percent uptime, you know, and I think recently we’ve basically hit four or five nines, but throughout the history of model, of course, we’ve had, you know, some run up against scaling issues. Yeah. The GPU market and well, actually before that. So you guys use AWS like GCP, like all these cloud providers underneath.

13:30 比如说，你们也会购买基础设施吗？不，我们稍微聊聊云端算力吧。我觉得这方面其实变化很大。最初几年，我们几乎所有算力都是从大型云服务商那里按需获取的，主要是竞价型算力，部署在各个不同区域。最开始用的是 AWS，后来也用了 GCP 和 Azure。我们很早就开始从 Oracle 租用一些算力，因为我们想要 A10 和 A100，而他们给出的价格非常优惠。后来我们又接入了几家云服务商，也开始做一些预留；但直到去年年底，我们的大部分算力仍然是竞价型算力或一年期预留算力。竞价型算力就是你需要时直接租用。没错。对，而且这种算力还可能被收回，我们得想办法应对。不过，这种模式在汇集算力时还挺好用的，因为你可以从许多区域——基本上是几百个区域——汇集算力。所以，这种模式一直运作得很好，直到我们的业务再次开始快速扩张。我的意思是，首先，Modal 的增长非常快，我们需要更多算力；但更根本的原因是，云市场也发生了很大变化。所以在过去一年里，为了获得所需算力，我们不得不与更多云服务商合作。目前我们合作的云服务商将近 25 家。而且很多情况下，为了拿到想要的算力，我们得签三年以上的合约，还得预付一大笔钱。你们会锁定三年期的价格，对吧？13:30 Like do you also buy any infrastructure? No, let’s talk a bit about like cloud capacity. Because I think it’s actually changed a lot. Like in the first few years we did, we got almost all the capacity pretty much from, from the hyperscalers spot and on demand capacity, mostly spot capacity, running in all the different regions, you know, using initially AWS, but also GCP and overtime measure. We started renting some capacity from Oracle pretty early because we wanted the eight tens and eight 100s and we got really good rates on those. And over time we added a couple more clouds, like doing reservations, but, but like still up until like actually end of last year, like most of our capacity was either spot or one year reservations. Spot means you’re just like renting it right there and then. Exactly. Yeah. And it’s also preemptible, which we had to deal with. But, but it sort of works in Agria because you can sort of just, you know, aggregate capacity from many, you know, hundreds of regions basically. So that, that, that model worked really well until we started running up again. I mean, first of all, like modal started growing like very fast as we needed more capacity, but fundamental also like the cloud market started changing a lot. So in the last year, basically now in order to get the capacity we need, we had to work with a much larger set of clouds. We actually work with almost 25 clouds at this point. And in many cases, in order to get the capacity we want, like we, we have to go into, you know, three plus year deals and you have to pay a meaningful amount of money upfront. And you’re locking in pricing on a three year term, right?

15:00 所以，你知道，呃，我们过去六到九个月基本上一直在重新思考该怎么获取算力资源。我们得从头搭建一个采购部门，差不多有四个人全职负责这件事，他们基本上就是采购算力资源、联系 Neoclouds、谈交易、审核算力资源。我们还得组建一整个工程团队，基本上有十个人，专门管理来自世界各地的裸金属算力资源，处理网络问题、故障硬盘、GPU 从总线上掉线之类的各种问题。我们不拥有算力资源，迄今为止这些算力全是租来的，通常签的是三年期合约。这是个重大决定：我们该不该买？还是继续租？其实我并不是特别固执地坚持某一种做法。我觉得租赁不错，因为它有一定的资本效率。要是买的话，你知道，首先，这里面会有运营上的复杂性。你得和供应商打交道，还得处理电力、电缆之类的各种事情。但这也涉及财务上的复杂性。比如，资金该怎么筹？要不要融资？你可以筹十亿美元，然后用这笔钱去买硬件，但这会导致很大的股权稀释。你也可以举债，但你知道，通常得有锁定的长期收入，才能支付利息。这也不是件简单的事。不过，我们会看成本效益，也会看运营方面的……15:00 So, you know, like, and, and, and so we have to spend the entire, you know, last six to nine months, like, you know, kind of rethinking how we get capacity. We have to build a whole procurement like desk, like we have like four guys working full time basically, all they’re doing is like procuring capacity, calling Neoclouds, you know, doing deals, vetting the capacity. We’ve had to build a whole engineering team, basically 10 people just like managing like bare metal capacity from all over the world and dealing with network issues and bad drives and GPUs falling off the bus and, and all these things. We don’t own capacity. We rent all the capacity so far, um, you know, on, on, on typically this three year deals. And that’s like a big decision. Like, should we buy, should, should we, you know, or should we keep renting? I’m actually like not like mega dogmatic about it. I think renting is nice because there’s a certain amount of capital efficiency to it. If you’re buying, you know, like, I mean, first of all, there’s not this like operational complexity of it. Like you have to, you know, deal with vendors and suppliers and, and, you know, power and, and, and cables and all kinds of stuff. But there’s also, you know, financial complexity to it. Like, how do you fund it? Like, should you raise equity, you know, and, and you can raise a billion dollars and go buy hardware for it. That’s a very dilutive way to do it. Um, you can raise debt, but you know, if you raise that, you know, you typically want to have like locked in long-term revenue to, to, to pay the interest rates. That’s also not trivial. But you know, like we look at the economics, we look at, uh, the operational

16:30 复杂性方面，我们会看风险。呃，这是一个复杂的权衡，而且我们也愿意购买。我们会……还有其他一些架构。比如，我们需要为 2027 年 확보所需的产能。要锁定这些产能，需要做很多工作，也就是要通过很多不同的途径来争取。所以，你们得雇很多财务人员来处理这些事吗？真的？你知道，我在这件事上花了很多时间，我们也花了很多时间。我确实认为，随着时间推移，毕竟这些硬件资产非常昂贵。你知道，我觉得这真是令人震惊。你开始看到，AI 投资占美国 GDP 的比例已经达到好几个百分点；而且过去几周，美国利率也开始飙升。我的意思是，在这个阶段，你去看看那些新云厂商，就会发现它们是非常非常精于金融工程的公司。所以，是的，我们会研究这些情况，也会考虑所有选项；我们自己也可能不得不做类似的事。我不太愿意这样——我宁愿专注于做一家非常精简的软件公司，做好我们擅长的事。我认为，一般来说，软件公司拥有硬资产并不是好事。就像你该不该拥有自己的办公室一样——我不认为这对大多数公司来说一定是正确的选择。但当然，为了获得你想要的产能，什么办法都可以考虑。是啊，我想供应在一段时间内都会受到限制。16:30 complexity, we look at the risks. Uh, it’s a complex trade-off and, and we’re open to, to, to buying. We’ll, we’ll sort of, there’s other constructs too. Like we’re, it’s to get the capacity we need for, for 2027. It’s going to take a lot of work on, on, you know, going a lot of different avenues to, to lock it up. So do you have to hire a lot of like finance people to do all this stuff or not? Really? You know, I’ve had to spend a lot of time on this and we’ve had to, to, to spend a lot of time on this. And I do think over time, I mean, there, there is a lot of, you know, these are very expensive assets, uh, the hardware. And, you know, I think, I mean, it’s kind of staggering. You starting to see like AI investments starting to become like several percentage points of, of, of GDP of the U S and, you know, starting to see like interest rates in, in America, shooting up in the last few weeks. And I mean, I like, at this stage, like, you know, and you go and look at the Neoclouds, like these are very, very sophisticated financial engineering companies. And so yeah, we, we look at that and, you know, we look at all the options and, and it’s possible we might have to do similar things ourselves. I wouldn’t want to like, I would rather focus on being, you know, like a very lean software company, like doing the things we’re good at. I don’t think it’s good in general for software companies to own hard assets. You know, in the same way as like, should you own your office? Like I don’t think that’s necessarily the right solution for most companies. But of course, like in order to get the capacity you want, like, you know, everything is on the table. Yeah. And I guess the supply is going to be constrained for a while.

18:00 所以你多少得这么做。你同意这种说法吗？还是觉得这只是某个阶段的现象？我不太认同。我是说，英伟达每个月都在交付几十万块 Blackwell GPU，对吧？我不是说市场完全有效，但一般来说，只要你愿意出高价，就能买到 GPU 算力，对吧？如果你愿意比其他人出更高的价，当然，Anthropic 和 OpenAI 也在尽可能多地采购。但从根本上说，市场上一直有新的供应。所以我不太认为这是一种资源被彻底垄断的情况。价格确实在上涨，我觉得值得反思一下。硬件通常的状态就像受重力作用一样——硬件价格最终总会下降。所以我们现在有点处在一个奇怪的时期，硬件价格反而在上涨。很难说这种情况会持续多久，可能还要六个月，也可能还要三十六个月。真的很难判断会持续多久。但我从根本上说，不确定“你必须买下算力”这种说法是否成立。对我来说，这有点像疫情期间那场史诗级的卫生纸短缺，或者类似的情况，对吧？至少有点像挤兑银行那种局面。不过话虽如此，算力在一段时间内还是会很紧张，价格也在上涨。幸运的是，我觉得这对我们来说甚至未必是坏事。因为价格上涨意味着稀缺，而稀缺也意味着我们可以……18:00 So you kind of have to do that. Like, do you, do you agree with that? Or do you think this is just like a moment in time? I don’t necessarily agree with that. I mean, like Nvidia is delivering, you know, several hundred thousand black walls, you know, a month, right? And, and I don’t want to say like the market is like 100% efficient, but like, you know, in general, like if you’re willing to pay a high price, you can get GPU capacity, right? Like if you’re willing to pay more than everyone else, and, and of course, Anthropic and OpenAI is buying up as much as they can. And, but fundamentally, like there’s constantly supply coming on the market. And so I don’t necessarily think it’s like a fully cornered resource. Prices are going up. And I do think it’s worth kind of reflecting on that. It’s like the normal state of hardware is like, it’s like out of like gravity. Like eventually, hardware prices always go down. So we’re a little bit in like a weird sort of period of time where like hardware prices are going up. And, you know, it’s hard to say, you know, how long this is going to last, you know, could be another six months, could be another 36 months. Like it’s really hard to know how long. But I fundamentally don’t know if like it’s true to say, like, you know, you need to buy capacity in order to me, that’s a little bit to sort of almost like the great toilet paper shortage of the pandemic or whatever. Right. Like, you know, at least this like, you know, run on the bank type situations. And, but that being said, like, you know, capacity is going to be tight for a while. And, you know, prices are going up. And, you know, luckily, I think, you know, it’s not even necessarily bad for us. If prices go up, because, because, you know, scarcity also means like we can

19:30 向我们的客户收取更高的费用。而且，你知道，我们有很多预订已经锁定了价格，这意味着在某种程度上，我们实际上已经相当于做多了 GPU 价格。所以我不一定太担心价格上涨。我更担心的是价格真的崩盘。我不认为这种情况近期会发生。但我觉得那对市场上的很多参与者来说会是件相当糟糕的事。为什么会糟糕？我觉得，如果有人锁定了大量供应，那对他们来说就会很糟糕。是啊，因为他们付的钱实在太多了，你知道，所以存在高位买入的风险，对吧？再说一次，我并不是特别担心这个。我不觉得这会在近期发生。但正如我所说，从非常长远的角度看，硬件价格往往会下跌。所以，值得想想这一点，对吧？对。就像现在，我觉得你的产能受到了限制，而这就导致了某种程度上的——用经济学术语来说，就是稀缺租金，对吧？谁攫取了稀缺租金？我觉得主要流向了两个地方：大部分流向英伟达，当然，在一定程度上也流向了 Neoclouds。总体来说，任何拥有产能的人都能获得更好的经济效益。在产能受限的今天，这一点毋庸置疑。从长远来看，你知道，从非常长远的角度看，我觉得这未必完全成立，对吧？拥有 GPU 总能让你获得一点溢价，但这值得冒险吗？你知道，持有这些非常昂贵的……19:30 charge more from our customers. And, you know, and the fact that we have a lot of reservations locked in at a rate means like, to some extent, we’re effectively like long GPU prices already. So I don’t necessarily worry too much about prices going up. I do worry more about actually prices collapsing. I don’t think that’s going to happen anytime soon. But I think that would be a quite bad thing for a lot of the players in the market. Why would that be bad? I think it would be bad for if, for anyone who’s locked in a lot of supply. Yeah. Because they paid way too much and they paid too much, you know, so there’s a risk of buying at the top, right? And again, like, I don’t worry about that much. Like, I don’t think it’s going to be like, you know, it’s going to happen anytime soon. But like, you know, like I said, like hardware prices in the very long run, they tend to go down. And so it’s, it’s worth, you know, reflecting on that, right? Right. Like right now, I think, I think your, your capacity is constrained and, and that leads to sort of, you know, in economic terms, there’s a scarcity rent, right? Like who captured the scarcity rent? I think the primarily goes to two places, mostly Nvidia, but of course, but also to some extent, Neoclouds, like Neoclouds and in general, like, you know, anyone owning capacity has better economics. Like that’s undeniably true today with capacity being constrained. Long term, you know, in the very long run, I don’t think that’s necessarily entirely true, right? Like there’s always going to be a little bit of a premium you can get from owning GPUs, but is it worth the risk? You know, is it worth the risk of like holding this like very expensive

21:00，你们资产负债表上的资产？我不太确定，不过你知道，这事还得过几年才能尘埃落定。长远来看，我们可能会拥有一部分，也可能租用一部分，或许两者兼有，也可能做些别的疯狂尝试，比如合资企业，谁知道呢？也许会设立一些复杂的特殊目的实体之类的。嗯，一切皆有可能。 你们怎么和 OpenAI、Anthropic 竞争，抢购 GPU 算力？我猜他们肯定想把市面上能买到的东西全部买下来。 他们确实在这么做。我觉得你看到的情况是，嗯，很多东西的价格都在上涨。这是不是反映了这一点？话虽如此，NVIDIA 也在出货大量产品。我还觉得，在某种程度上，NVIDIA 其实更倾向于稍微分散一下客户，不希望大部分产品都卖给少数几个大买家。就像你看到的那样，超大规模云服务商拿到的配额也受到了一定限制。嗯，我觉得 NVIDIA 也更希望看到很多新兴云服务商和其他云服务商获得更多配额。我认为市场上还有一点情况——这只是推测——那就是对我们来说，和一些规模较小、合作风险稍高的云服务商合作，机会可能更多。比如说，我觉得 Anthropic 和 OpenAI 可能更倾向于从那些经过长期验证的供应商那里采购，部分原因是他们必须进行大量训练，所以需要非常大规模的算力集群。不过，市面上还有很多算力，你知道，在亚美尼亚就能找到 32 个节点。21:00 assets on your balance sheet? I’m not so sure, but you know, we’re going to see, like it’s going to take a few years to get through this. And, and, and in the long run, you know, we might own some, we might rent some, maybe a combination, maybe do some other crazy stuff, joint mentors, who knows? Maybe some complex SPDs, whatever. Um, everything is on the table. How do you compete with open AI and enthropping and buying GPUs capacity? I’m assuming they’d want to like buy up literally everything that’s available. They are. And, and I think what you’re seeing is, um, you know, a lot of that, the, the price is going up. Is it a reflection of that? That being said, you know, NVIDIA is shipping a lot of stuff. I think also like to some extent, like NVIDIA is like kind of prefer a little bit of a diversification. Like they don’t want like, you know, small amount of large buyers in much, in the same way as you see, you know, hyperscalers getting a little bit of a constrained allocation. Uh, I think, you know, also NVIDIA prefers to see, you know, a lot of the neoclouds and a lot of alternative clouds, like getting more allocation. And I think a little bit, what we see in the market is also this is more speculation, but I think we see, you know, smaller clouds that are a little bit more sort of risky to work with being more of an opportunity for us. Like where, you know, maybe NVIDIA, sorry, enthropping and opening AI, my feeling is like they prefer to buy from like the, the, the try and through, uh, part of it is also because they had to do a lot of training runs. So they need to have like very large blocks, but there’s plenty of capacity out there, you know, you can get 32 nodes in Armenia.

22:30 你在印度尼西亚能拿到 128 个节点。你知道，我们就喜欢这种东西。这样我们就能去世界各地，拿下许多容量较小的资源，而且价格相当不错。 是不是基本上只要有人能提供一定数量的 GPU，你就只关心这个？你不在乎它靠不靠谱，也不在乎这些设备是不是放在某个破破烂烂的仓库里吗？ 嗯，有时候确实是在挺破烂的仓库里。感觉上是这样。不过，我们确实有一份规格清单。跟供应商沟通时，我们做的第一件事就是把规格清单发给他们。我们会说，我们需要达到多少 IOPS；CPU 至少要达到某个标准，比如通过某项基准测试，或者其他指标。我们还要看最低的南北向带宽，以及出入站带宽。所以，我们确实有一份需求清单。 我们通常会尽量安排一段验收测试期，先拿一个节点运行并测试一段时间。不过，在某种程度上，这也需要一点信任。有时我们会签下非常大的合作承诺，金额高达数千万甚至上亿美元，然后开始和一家我们还不完全了解的供应商合作。之后我们再慢慢观察。我觉得，某种程度上，我们内部也有自己的一套评估标准。你知道，SemiAnalysis 有 ClusterMAX，我感觉我们内部差不多也有自己的 ModalMAX 评分。我知道哪些供应商比其他供应商更合我意，等等。22:30 You can get 128 nodes in Indonesia. Like, you know, like we love that kind of shit. Like, you know, so we can go out there and like get a lot of, you know, smaller capacity all over the world. Pretty good prices. Is it pretty much just like if someone offers you like a certain number of GPUs, like that’s all you care about. Is there nothing like, oh, you care about like how reliable it is, you know, is it just some like janky warehouse that someone has these in? I mean, sometimes it’s kind of a janky warehouse. It feels like, you know, but, but like, we do have like a spec sheet, basically. Like when we talk to vendors, the first thing we do is we send them a spec sheet. We say, we need this number of IOPS. We need, you know, like the CPU needs to be at least, at least, you know, have, have this many, you know, whatever the pass mark scores or whatever it is. We need to see like, you know, minimum, you know, North, North, South, bandwidth, egress and ingress. So we do have a list of requirements. Often we try to do like an acceptance testing period where we get like one node and, and runs and tests over time, you know, like to some extent, it also involves a little bit of a leap of faith. You know, sometimes we sign very big commitments, you know, 10s or 100s millions of dollars and we start working with the vendor that we still don’t fully know. And, you know, over time, like, you know, we’ll see, you know, and, and, and I think over, like, in a little bit of, like, there’s a little bit of sort of just, you know, our own sort of, like, you know, how like semi analysis has like a cluster max, I kind of feel like we almost like have our own, like modal max score internally. Like, I know which vendors I like more than others, you know, like, and, and,

24:00 而且，我们会更倾向于和那些我们接触过、可靠性非常好的供应商做更多交易。当然，不只是可靠性，还要看当我们遇到支持问题时，他们的响应速度如何；我们要求修复的问题，他们能多快处理好。如果出了故障，他们能多快修好？诸如此类。你是否同意，对智能的需求基本上是无限的？如果同意，那为什么不尽可能多地买下容量呢？如果买多了，你也可以转卖给别人。我觉得资源稀缺的时候，很容易就会说：能买多少就买多少。但我觉得，首先，如果我们今天买下这些容量，我们没法全部用上。我们确实能用上很多，对吧？但你可以把多余的卖给其他想用的人，对吧？我觉得难点在于，如果市场上发生了什么情况，很多事情会同时出现关联。如果价格下跌、利用率下降，转售容量的市场也差不多同时消失，那就麻烦了。我现在每天都在非常积极地买容量，也在筹钱来买。所以总体来说，我非常看好这个市场，但我确实想指出，还是存在一些尾部风险。而这些尾部风险一旦发生，情况可能会变得非常糟糕。部分原因可能是我年纪大了，过去见过这种情况。我第一份工作其实是在1999年，所以我经历过互联网泡沫危机，也经历过全球金融危机之类的事情。我想打造一家能持续经营20年的公司……24:00 and we sort of prefer to do additional deals with the vendors that we’ve seen, you know, very good reliability, not just that also, like it’s not just reliability, it’s also, are they very responsive when, when we have, you know, support issues, do the fixed issues we ask for to turn around time. If things go down, you know, how quickly do they fix it? Like things like that. Do you agree that the demand for intelligence is basically infinite? And if so, like, why wouldn’t you just buy up as much as you, as much capacity as possible? And then, like, if you over buy, you can just like sell it to somebody else. I think it’s very easy to say when there’s like a scarce resource, like, let’s just buy as much as you can. But I think, you know, first of all, like, if we bought it today, we couldn’t utilize all of it. Like we could use utilize a lot of it, right? But like, But you can sell it off to somebody else who wants to use it, right? I think the challenge is like, if there was like something in the market that happens, there are a lot of correlations that happen at the same time. If, you know, prices going down and utilization going down and the market for reselling capacity goes, vanishes at the kind of the same time. I’m sitting here and like, I’m extremely aggressive, like buying capacity every day and like raising money to buy, you know, so overall, I’m, I’m very bullish, but like, I do want to point out and like, you know, like there are tail risks here. And when those tail risks happen, like it could be get very bad. Part of it is like, maybe I’m, you know, I’m just old and I’ve seen this in the past, like I actually had my first job in 1999. So I saw the dot com crisis and I’ve seen the great financial crisis and things like that. And, you know, like, I want to build a company that, that is around for like 20

25:30 到 30 岁，还能再有 50 年，对吧？所以，你知道，我觉得我们当然可以现在就出去买下惊人的大量算力，试图垄断市场。但你知道亨特兄弟吗？这是交易界一个传奇故事。他们试图垄断白银市场，结果惨败。你知道，所以这就是垄断市场。我觉得总的来说，这可能不是个好主意。不过，我们总体上确实非常、非常激进，但我们正在加快扩张，采取激进的计划，基本上跟上需求增长，并达到我们希望在明年年底实现的算力储备或运行速率。你们只买 NVIDIA 的东西吗？还是也会考虑从其他类型的公司获取算力，比如训练它们？我们正在关注这方面。我觉得替代型加速器开始变得非常有意思了。讽刺的是，过去五年，甚至可能十年来，每年我都在说：“明年就会是非 NVIDIA 加速器的天下。”结果我总是说错。不过，我确实觉得 2027 年可能终于会成为非 NVIDIA 加速器的天下。到时候看看今年我会不会又说错。但我觉得这些产品在某些方面已经开始取得非常好的成绩了。我想，TPU 显然已经在 Google 内部得到了广泛应用，而且似乎效果很好。我对训练方面了解不多，感觉没怎么看到类似的情况，不过你知道，他们的团队很出色。我知道他们正在研发很多软件方面的东西。25:30 to 30, 50 more years, right? Like, and so, you know, I think we can certainly go out and like buying an insane amount of capacity right now and trying to just corner the market. But, you know, do you know about the Hunt Brothers? It’s like a legendary story in trading. They try to corner the silver market. It fails spectacularly. You know, so it’s a cornering the market. I think in general, like it can be not, not a good idea. And, but, but we are in general very, very aggressive, but we’re ramping up, you know, like at an aggressive plan that basically matches our demand growth and matches where we want to try to get to at the end of next year in terms of run runway or run rate. Are you only buying up NVIDIA stuff or also like, you know, maybe a capacity from other types of companies like training them and we’re looking at it. I think the alternative accelerators are starting to get very interesting. It’s, you know, ironically, like every, every year in the last five, maybe even 10 years I’ve been saying, you know, like next year, it’s going to be the year of like non NVIDIA accelerators. And then I’m always wrong. But, but I do think 2027 might be the year of non NVIDIA accelerators. Finally, so we’ll see if I’m wrong this year too. But so I, you know, I am these are starting to get like really good scores on certain things. I think, you know, TP use are obviously used very, you know, widely internally at Google. I seem to work really well. I don’t know much about training. I don’t feel like I see quite as, but, you know, the team is very good. I know they’re working on a lot of the software stuff.

27:00 你知道，硬件在某种意义上确实很难。软件就相对容易一些。这让我觉得，如果我们能把这些加速器的软件层面解决好，我认为它们会有很大的需求。而且从根本上说，NVIDIA 现在是个很大的瓶颈，所以我们可能会看到更多替代加速器。当然，产业链下游也有瓶颈。我的意思是，还有台积电和 ASML 这样的公司，它们或许本身就是整个产业更深层次的瓶颈。不过从长远来看，我认为垄断并不会永远持续下去。因此，我觉得市场上有更多替代选择会更健康。要说明的是，虽然我很喜欢 NVIDIA，但他们显然是一家很了不起的公司，产品也很出色。我确实希望看到市场上出现更多元的选择。还有像 Cerebras、Grock 这些公司。你怎么看这些供应商？我觉得它们相当令人振奋。你也能看到它们在某些工作负载上表现得非常出色。还有一些很有意思的做法，比如混合式、解耦的推理：预填充阶段用 NVIDIA，解码阶段用 Cerebras，诸如此类。哦，我还真没听说过这种做法。是啊，还有 Grock 也是。因为这些工作负载里，有些受限于计算能力，有些受限于内存，所以你可以把不同的东西搭配起来。我觉得这相当令人振奋。不过，实际部署时会变得非常复杂。但我确实认为，这类更复杂的硬件配置有很多机会。你从一些新兴云服务商那里，比如 Nebius 之类的，能学到什么吗？比如说，他们的做法和……相比呢？27:00 You know, hardware is hard in a certain way. Like software is like easier. And that makes me feel like, you know, if we just fix the sort of software layer for these accelerators, like, I think there’s going to be a lot of demand for them. And I think fundamentally, you know, with NVIDIA being such a bottleneck, like, we’ll probably see, you know, more alternative accelerators. That being said, you know, there’s bottlenecks further down the pipe. I mean, you know, there’s like TSMC and ASML, maybe like a deeper bottleneck to all of this anyway. But, you know, long term, I think monopolies don’t always last. And so I think you’d be healthy for the market to have more alternative. As much as I love NVIDIA, like, to make that clear, like, I mean, they’re obviously an amazing company and within the amazing product. I do expect to see a little bit more diversity in the market as well. Also, things, you know, like Cerebras and Grock and these things. Yeah. What are your thoughts on like some of those providers? I think they’re pretty exciting. And, you know, and you see them doing really well on certain workloads. And, you know, there’s some interesting, you know, people doing these like hybrid, you know, disaggregated inference where they run, you know, NVIDIA for the pre-fill and Cerebras for the decode and things like that. Oh, I haven’t heard of that actually. Yeah. And, you know, Grock too, like, because a lot of these things, you know, certain things are compute bound, certain things are memory bound. So you can sort of mix and match. And I think that’s quite exciting. It starts to get very complex when you’re deploying these things in practice. But I do think there’s a lot of, there’s a lot of, you know, opportunity and these sort of, you know, more complex hardware setups. Are there any learnings that you get from some of these Neoclouds, like, I don’t know, like Nebius or whatever, and like the way that they’re doing things versus

28:30 喜欢你们现在做的事情吗？我觉得，新云厂商有一件事特别有意思：他们怎么能扩张得这么快？我觉得最夸张的例子其实是 Nscale，它几乎是从零起步，一下子就有了价值数千亿美元的项目储备，尽管手头几乎没有任何算力。新云厂商怎么能发展得这么快？其实答案是这样的，可能显而易见：他们会去找 Anthropic、OpenAI 或者某个大型买家，问：“嘿，你们想买五年的芯片吗？签这份合同，我们给你们一个好价格。”然后他们拿着这份合同去华尔街，说：“看，我们有个大买家，已经承诺购买价值 100 亿美元的芯片。我们现在只需要把它变现。”接着他们就去融资，然后找 Super Micro、戴尔或者英伟达之类的公司采购硬件，再把设备安装好。然后，他们就把算力交付给客户了，对吧？所以这是一种非常巧妙的金融结构：你基本上可以预售那些自己还没买下来的东西。这就意味着你可以极快地扩大规模。这就是我们在这些新云厂商身上看到的情况——他们确实能非常、非常有效地扩张。我觉得 Modal 因为一直更侧重 PLG，也不像它们那样能够签下长期合约，所以从财务角度看，要那么快地扩张就更有挑战性一些，尽管我觉得从长远来看，我们会……28:30 like what you’re doing? I think one of the things that, you know, is very interesting with Neoclouds is like, how can they expand so quickly? I think actually the most crazy example is like N scale, which went from like nowhere to like, you know, having like, you know, whatever, hundreds of billions of dollars in the pipeline, despite barely having any capacity. How can Neoclouds grow so fast? I mean, there’s like an, you know, and basically the answer is like, you know, and this might be obvious, but like what they do is like, they go out to like, you know, enthropic or open the eyes and some big buyer or whatever. And basically say like, Hey, do you want to buy like five years of chips? You know, sign this contract, we give you a good rate. And then they take that contract and then they go to, you know, Wall Street and say, look, we got this like, you know, big buyer, you know, they committed to $10 billion worth of chips. We just need to monitor it right now. And then they go on to raise the money. And then they like, you know, go to like Super Micro and DAL or whatever in the video and like get the hardware and then they install it. And then, you know, now they have like, you know, delivered to the customer. Right. So you have this like very nice sort of financial construct where you can basically sort of pre-sell things that you haven’t even bought. And that means, you know, you can scale up extremely fast. And so that’s what we’ve seen with these Neoclouds is, is, is they can scale very, very well. I think modal because we’ve always had a little bit more of a PLG focus and, and not the sort of, you know, ability to lock in long-term contracts in the same way, it’s a little bit more challenging from a financial perspective to scale that fast, even though I think, you know, in the long run, like we’ll get to the

30:00 同样的观点。不过我觉得，看到这些新云厂商，尤其是 CoreWeave 这样的大型厂商，表现得如此精明，确实挺有意思的。你知道，他们很懂得如何融资，也很懂得如何把 GPU 作为贷款抵押品来考虑，诸如此类。比如，我忘了是谁了，有家金融公司说，哦，我们要把 GPU 变成一种资产类别。哦，我想起来了，应该是英伟达。他们说，我们要把 GPU 变成一种资产类别。这么做只是因为他们需要融资，正在想办法从哪里筹钱吗？这件事的意义是什么？我的意思是，有一堆初创公司。我不知道你是不是也在想这个。有 Silicon Data、Computer Exchange、Orn，还有其他几家公司，基本上都在试图建立价格指标和指数、交易衍生品之类的。我觉得这挺有意思的。我觉得，就计算资源而言，把它作为一种资产类别确实面临不少挑战。它显然不像其他资产类别那样同质化。你会遇到不同的网络、不同的位置，还有不同的磁盘、CPU、各种各样的东西，以及可靠性问题。话虽如此，石油也是如此，但我们还是把石油变成了相对标准化、可交易的资产。结果如何还有待观察。我确实觉得，以我们如今 AI 数据中心和 GPU 部署的规模，我们需要在风险对冲、风险如何分担等方面做得更聪明一些，你知道，…30:00 same point. But I think that’s been kind of a fascinating thing to see with these Neoclouds and especially like CoreWeave and those big ones. They’re very, very savvy with, you know, how they raise money and how to, you know, think about GPU as collateral for loans and all these things. What’s the relevance of, like, I forget who it was, but like some like financial company was like, oh, we’re going to make, oh, I think it was Nvidia. Actually, they were like, we are going to turn GPUs into an asset class. Is that just because like they need to raise money and they’re like trying to figure out where to get the money from? Like, what’s the relevance of that? I mean, there’s a bunch of startups. I don’t know if you were thinking about that. There’s like Silicon Data and computer exchange and like Orn and, and, and a few other ones that are like, basically like trying to get to like, you know, creating price indices and indexes and trading derivatives and things like that. I think it’s interesting. Like, I think with compute, like it does, there are a number of challenges with compute as an asset class. Like it’s definitely not as homogenous as other asset classes. You have different network and different locations and, and, and different disks and CPUs and all kinds of stuff and reliability. I mean, that being said, like you can say the same thing about oil and like we kind of created like, you know, relatively commoditized, you know, tradeable assets out of oil. It remains to be seen. Like I do think, you know, at the scale where we are with the sort of AI, like data center and GPU deployments, like we do need to be a little bit smarter with like risk hedging and, and, and how we share risks and, and, and, you know,

31:30 有更多类似保险之类的东西。所以我非常高兴大家都在认真思考这个问题。话虽如此，我不太确定我们会不会看到 GPU 变成某种类似亨利枢纽天然气、布伦特原油那样，完全商品化、可交易的资产。好。我想把话题转到 Modal 沙箱上。嗯，在那之前，我想先问一下，Modal 有哪些不同的产品？我想主要产品显然是 GPU，然后还有 Modal 沙箱。还有别的吗？我觉得我们有两大产品：第一是 Modal Functions，第二是 Modal Sandboxes。Modal Functions 是我们在 2021 年推出的，理念是提供易于部署、易于扩展的无服务器函数，容量由我们来管理。事实证明，它非常适合推理，也很适合管理 GPU，以及运行类似这样的任务。但其实我们从来没有专门做过推理产品，或者说 GPU 产品。我们从没把 Modal 想成一家新型云服务商，我觉得有一种风险，就是被拉向那个方向，最终变成一家新型云服务商——只卖我认为像是矩形的东西，比如“给你三年时间，你能拿到这么多 GPU”。Modal 一直很擅长让你部署某个东西，然后按需扩展。你的工作负载可能有很强的周期性，或者会突然激增。容量管理都由我们来做。好处是，我们有成千上万的客户，……31:30 having more like sort of insurance and, and things like that. So I, I, I’m very happy that people are thinking a lot about this. That being said, like I’m, I’m not entirely sure if we’re going to see, you know, GPUs becoming sort of, you know, whatever it is, sort of Henry hub, like, you know, Brent crude type, like, you know, super commoditized, like tradeable asset. Yeah. I want to shift the conversation to modal sandboxes. Yeah. I guess like before we do that, what are the different products that modal has? It’s, I guess like the main product obviously is like the, the GPUs. Then you have modal sandboxes. Is there anything else? I think of the like the two big products that we have as basically number one is modal functions and number two is modal sandboxes. So, so with modal functions, we launched that, you know, back in 2021. It was the idea of like serverless functions that are very easy to deploy and very easy to scale and we managed all the capacity. That turned out to be really, really good for inference and it turned out to be really good for managing GPUs and running on things like that. But we actually never had like, you know, dedicated like inference product or like a GPU product per se. Like we never thought of modal as like a neocloud and I think there’s like a risk of like getting sucked in that direction and becoming a neocloud, like just selling, you know, sort of what I think was like rectangular, like here’s like a three year. You get this many GPUs. Models has always been very good at like you deploy something you can kind of scale it up. You have this like very cyclical or like bursty stuff. Like we manage all the capacity. And the beautiful thing is like, because we have many, many thousands of customers,

33:00 我们可以把这种容量汇总起来，从而获得非常平稳的工作负载，也就是非常平稳的总需求，这样我们就能非常非常高效地进行管理。所以，Modal Functions 是核心产品。我们在 2021 年推出了它。就像我提到的，我们在 2023 年开始看到产品与市场的契合。现在，它大约占我们收入的三分之二，也为很多……实际上主要是推理场景提供支持，不过你也可以用它来做各种其他事情。 Modal Sandboxes 是我们在 2023 年推出的。实际上，和 Functions 类似，我们花了一段时间才看到它与市场的契合。我们为什么要推出它？我们发现，和我们交流的很多人都在用 LLM 生成代码。这就像有一天我和 Akshar 边吃午饭边聊天，我们想，也许他们需要一个地方来运行这些代码。然后我们意识到，其实我们已经有很多基础设施，可以直接接收代码并运行它。因为我们已经构建了管理容器和隔离所需的基础原语。我们已经有 gVisors、隔离原语之类的东西了。所以我们对这个想法挺兴奋的。Akshar 写了个原型，基本上一两天之内就把它发布了。结果增长几乎为零。倒也不是说它一点增长都没有，因为我们确实看到了一些增长。就是有那么一些……当时是 2023 年。对，是 2023 年。有些人开始使用它，我们也有一些早期用户觉得它在某些场景下挺好用的。但总体来说，直到 2023 年夏天，我们从它获得的收入都微不足道。所以两年后，我们甚至还讨论过：这个产品我们真的有必要做吗？33:00 we can sort of aggregate that capacity and get very smooth workloads, you know, very smooth aggregate demand, which we can manage very, very efficiently. So modal functions was the core. We launched that in 2021. Like I mentioned, we started seeing product market fit in 2023. And now, you know, it’s about two thirds of our avenue and empowering a lot of, you know, in practice, it’s mostly inference, but you can use it for all kinds of other stuff. Modal sandboxes, we launched that in 2023. And it’s actually kind of, you know, took a while to see product market fit similar to functions. Why did we launch it? We realized a lot of people we talked to were using LLM to output code. This is like me and Akshar talking over lunch one day. We’re like, maybe they need a place to run that code. And we realized like, actually, we kind of have like a lot of the infrastructure already to just take code and run it. Because we’ve already built the primitives for like managing containers and isolation. We already had, you know, G visors and isolation primitive and things like that. So we get kind of excited about that idea. And Akshar wrote a prototype and launched it in basically one day or two days or something like that. Zero. Like we saw like very little like growth. I don’t want to like trivialize it. Because we did see some growth. Like there were like some sort of, you know, And it was 2023. So yeah, it was 2023. And like, you know, like some people picked it up. Like we had some early users who kind of enjoyed it for certain things. But like overall, like it was like a pretty insignificant source of revenue into going into the summer of 2023. So two years later, we even discussed like, do we even need this product?

34:30 我们要不要把它关掉？然后它就开始爆发式增长了。所以是 2023 年。我觉得 2023 年真正发生的事是，大语言模型生成代码的能力变得非常强。是在 2023 年。2023 年。我的意思是，大概就是那段时间。我感觉那时它们从能够重构函数、编写代码片段，发展到了可以生成完整应用。对吧？于是我们开始看到，很多增长来自氛围编程，你知道，像 Lovable 这样的公司，还有其他一些公司。然后，像 Ramp 这样的公司开始用模型来运行后台智能体。接着，我们看到不少 Neolabs 在模型沙盒中进行强化学习等工作，由此带来了巨大的增长。所以，沙盒现在大约占我们营收的三分之一。你知道，我们基本上把它看作第二款产品。对我来说，这真的很令人兴奋，因为早在 Modal 刚起步的时候，我就一直知道，Modal 要做的是打造一整套产品工具箱。我们一直希望它能像云服务一样，你有存储和计算能力，还有各种不同的使用方式。我们一直想成为一家多产品供应商，但直到最近，这在实际中才真正实现。现在我觉得情况已经很明朗了：我们有 Functions，也有 Sandboxes。我们还在做很多其他事情，比如训练、批处理，以及其他各种工作。所以，对我来说，发展成为一家多产品公司，这种演进实在令人振奋，因为这始终是我们愿景的核心。甚至在最早期的演示文稿里，我就放了很多关于这方面的幻灯片。但直到最近，这一点才真正为外界所知。34:30 Should we shut it down? Then it just started exploding. So 2023. And I think what really happened 2023 was like, LMS got really good at generating code. In 2023. 2023. I mean, it was like around that time. Like I feel like that was like when they went from like being able to like refactor functions and like write snippets, so like generating like full apps. Right. And so we started seeing a lot of growth coming from like vibe coding, you know, companies like Loveable and other ones. And then, you know, companies like ramps started using model for background agents. And then we started seeing tremendous growth coming from a number of Neolabs doing things like reinforcement learning on model sandboxes. So sandboxes is now about a third of our revenue. You know, we considered it like basically like our second product. And to me, it’s like really exciting because like even back in the early days of model, I always knew that model, you know, is about like creating a whole toolkit of products. Like we always wanted to be almost like a cloud, you know, you have the storage and the compute and a bunch of different ways to work with it. We always wanted to be a multi-product vendor, but that wasn’t necessarily true in practice until like now. We’re like, I think now it’s like kind of clear. We have functions, we have sandboxes. We’re working a bunch of other stuff too. We’re working on training, working on batch, we’re working a bunch of other stuff. So to me, that sort of evolution into becoming like a multi-product company, I think it’s been amazing because that was always so core to the vision. Even in the very early decks, I had a bunch of slides on this. But it wasn’t necessarily true to the external world until recently.

36:00 你们有什么正在开发的、你觉得会成为下一个热门产品的东西吗？对市场有什么预测吗？类似的都可以。我非常看好训练。我觉得训练是一个某种程度上服务不足的市场。显然，大家都非常关注超大规模的训练任务。但我们实际上也看到，对小型和中型训练这一领域的需求很大。所以这周晚些时候我会稍微聊聊这个。我们正在筹办一场会议。我觉得批处理是一个非常有意思的市场。我觉得我们看到很多需求，来自那些需要……你知道，前几天有位客户需要转录长达 5000 年的音频。他们有数以百万计的音频文件。我们正好有很多算力，于是就让他们处理了这些文件，花了几天时间。我觉得这里面有一些非常有意思的应用场景。我也非常看好低延迟领域。我觉得，一旦我们把音频、语音转语音模型和语音模型做好，就会出现很大的机会。我预计这会成为一个非常有意思的市场。当然，还有所有这些智能体相关的东西，对吧？某种程度上，我对“智能体”这个词有些复杂的感受，因为我觉得从某些角度来说，所有软件最终都会变成智能体。所以对我来说，“智能体”到底意味着什么，几乎成了一个问题。我觉得从某种意义上说，随着时间推移，所有软件都会由 AI 驱动。不过显然，我们已经有了一套构建模块，可以用来打造更多由 LLM 驱动的软件，而这些模块似乎在各种不同的应用场景中都很常见。36:00 Do you have anything in the works that you think is going to be like that next product? Any predictions maybe about the market? Anything like that? I mean, I’m very bullish on training. I think training is like kind of an underserved market. Obviously, there’s like a lot of focus on like very large training runs. But we actually see a lot of demand, the sort of smaller and mid-sized training segment. So I’ll be talking a little bit about that later this week. We’re organizing a conference. Batch is I think like a really interesting market. I think we see a lot of demand for sort of, you know, customers that need a... You know, we had a customer the other day that needed to transcribe, you know, 5,000 years of audio. They had like, you know, millions and millions of audio files. And we had a lot of capacity. So we let them do that, you know, and took a few days. I think there’s some really interesting use cases there. Very bullish on like low latency stuff. I think there’s going to be a lot of, you know, once we sort of get the audio, like the speech-to-speech models working, like the voice models. I sort of expect that to be very interesting market. Then of course, like all this like agente stuff, right? And in a way, like I have like mixed feelings about like the word agente, because I feel like in certain ways, like, I think all software will become agente. So it’s almost like to me, like, what does it mean to be agente? Like I think all software in a certain way, like will become like AI powered over time. But like clearly there’s like a set of like building blocks of like, you know, doing more sort of LLM powered software that seems to be sort of, you know, common across all the different use cases.

37:30 我们是这么想的，你知道，我看待模型的方式是：我们做的是构建基础模块。我们想给你提供一些很棒、可泛化、经过实战检验、通用且可组合的基础模块。我觉得还有一些很有意思的东西，比如智能体用例，这就挺有意思的。你提到了训练，因为我记得在另一个播客里，可能是一年前，也可能是两年前，你说你们当时没有特别专注于训练。市场上有什么变化吗？我的意思是，我们断断续续地开发训练产品已经好几年了，所以我不会说我们完全没关注过这件事。但这就像是——我的意思是，我们现在有大约150名工程师，不过一两年前，我们的工程师人数要少得多。所以，只是因为团队规模更小，我们就不得不把精力更集中一些。于是推理就成了我们的唯一重点，而且多年来一直如此。但现在我们规模更大了。我们确实也有机会在其他市场上下注。我也很感兴趣，比如继续聊聊其他方面：后训练、强化学习。我觉得这方面有些东西特别令人振奋。我觉得还有很多其他领域，比如存储、构建数据仓库和特征存储，以及实时流处理管道，还有各种各样的东西。我也说不太准。对 Modal 的长期愿景基本上就是：如果你想做点很酷的 AI 玩意儿，我们能帮上忙；我们有所有这些基础模块，也会帮你把它们组合起来。37:30 And we’re like, you know, the way I think about models, like we’re in the business of building building blocks, you know, we want to give you like amazing, generalizable, battle-tested, you know, general purpose, composable building blocks. And I think there’s some really interesting stuff also, like, you know, with the agent use cases. That’s interesting. You bring up training because I think in another podcast, maybe like a year ago, maybe like two years ago, you said training you were not focused on like specifically, like what changed in the market? I mean, we’ve been building a training product for multiple years on and off. So I wouldn’t say we’re like, we’ve never been focused on it. But it’s like, it’s like one of those things we’re like, I mean, now we’re like 150 engineers, but, you know, a year or two ago, we were a much smaller number of engineers. And so, but just virtue of, you know, being smaller, we had to focus a little bit more. And so inference became like our sole focus and was our sole focus for many years. But we’re bigger now. Like we, we, we do have opportunities to also place bets in other markets. I’m also really interested in, you know, kind of continuing the list of things. Post training, reinforcement learning. I think there’s some really exciting stuff around that. I think there’s also many other like stuff around like storage and building data warehousing and feature stores and real time streaming pipelines and all kinds of like, I don’t know. I think the longterm view of, of modal is like, basically like, you want to build some cool AI shit, like, you know, we got you covered, like, you know, we got all the building blocks, like comp tasks and like, we’ll help you, you know, take all these building blocks and compose them.

39:00 软件的妙处就在于，只要有合适的构建模块，你几乎什么都能做出来。所以，这很大程度上就是我们想打造的那类公司和产品。你说的训练，是指更专注于那些进行大规模训练的巨型企业吗？不，不是。他们会去做那些大型超大规模云服务商，或者说新兴云服务商的巨额交易，拿到 10 万块 GPU 之类的。但我觉得，有一个服务还不够完善的细分市场，比如研究人员反复迭代，或者训练较小模型的人。我们看到，文本、语音和文本模型、计算生物技术等领域都有不少需求。我们在这些领域有很多客户，他们训练的模型规模相对适中。比如，你可以用 64 块 GPU 来训练推荐系统，或者做很多传统机器学习。很多事情都不需要 10 万块 GPU。我觉得，我们暂时不会把重点放在那个市场上。但我认为，稍小一些的市场里有很多需求尚未得到充分满足。搭建训练集群非常困难。是啊。你有没有注意到，从 GLM 5.2 或者，也许是 Qwen 2.5 之类的模型出来之后，很多人都在用它们并对它们进行微调？要说的话，我看到的其实正好相反。我觉得几年前微调的价值主张是：是啊，开源模型没那么好，但你可以拿一个小模型，用你自己的特定……39:00 And that’s the beauty of software is like you, you can, you know, build almost anything if you have the right building blocks. So that, that’s very much the sort of company and the product that want to build. When you say training, are you focused on more of like the bigger enterprises doing huge training runs? No, no, they’re going to go and like, you know, do these like, you know, big, hyper-scaler or rather Neo, Neo cloud mega deals and get, you know, 100,000 GPUs or whatever. But, but, but there’s like a little bit of an underserved market of like, in my opinion, you know, like researchers iterating or like people training smaller models, there’s, you know, Texas beach and speeches, text models, computational biotech is actually a market that we see a lot of demand from. We have a lot of customers in that space training relatively, you know, modest models. You can train on 64 GPUs or whatever recommendation system, a lot of sort of traditional machine learning. There’s a lot of things like you don’t need a, you know, 100,000 GPUs. That market, I think is like, we’re not going to like focus on that for a while. But, but I think there’s a lot of stuff in the, you know, slightly smaller markets that that are relatively underserved. It’s very hard to set up training clusters. Yeah. Have you noticed like since, I don’t know, like GLM 5.2 or like, maybe like, can we 2.5 that, you know, a lot of people are using those and like fine tuning them? If anything, I’ve actually seen the other direction. I feel like fine tuning the value prop a couple of years ago was like, yeah, you know, open source models, they’re not quite as good. But, you know, if you take, you know, a small model and you fine tune it on your specific

在 40:30 的领域里，你实际上可以在质量和成本方面达到前沿水平。如今，我会说，开源模型已经非常出色了，基本上可以直接替换掉前沿模型来运行，比如用 Kimi K3、DeepSeq 4.1 或 GLM 5.3。它们的表现基本上几乎能媲美前沿模型，但成本低得多。所以，我们仍然看到对后训练的巨大需求。我也确实认为这是一个很有意思的领域，我们正在开发一些相关产品，涵盖 SFT，也包括强化学习之类的东西。我还想说，微调已经不再是你非做不可的事情了，完全可以按需选择。开源模型直接拿来用，效果也很好。 说回沙箱。我刚才忘了问你们：你们为提供的沙箱做了哪些改进，是现有沙箱无法支持的？ 并没有太多。我觉得我们一直非常擅长的事情——毕竟我们是基础设施狂热爱好者，对吧？我们非常擅长扩展规模、保障可靠性，以及处理各种事务。我认为这在很大程度上解释了为什么我们在这个领域的市场份额远远最大——这是我的看法，我没有具体数据。我们就是很擅长运行几十万，甚至几百万个沙箱。是的，我想说，为了应对这种规模，我们不得不多次重写系统。40:30 domain, you can actually reach the frontier in terms of, you know, quality and cost. Today, I would say like the open source models are so good that like you can basically just like swap them out and just run, you know, instead of, you know, the frontier ones, you can run, you know, Kimi K3 or whatever, DeepSeq 4.1 or GLM 5.3. And they’re like basically almost as good as like the frontier model at a much lower cost. So we still see a lot of demand of for post-training. And I do think it’s a really interesting space and we’re working on some products in that space, you know, both sort of SFT and but also reinforcement learning and things like that. I would also say, you know, fine tuning is no longer the thing that you kind of have to do. Like it’s very optional. Like you can open source models like work really well just off the shelf too. Back to sandboxes actually. I forgot to ask you. So like what did you have to update about the sandboxes that you guys were offering that like existing sandboxes did not did not allow for? Not a whole lot. I think the thing we’ve always been very, very good at, I mean, we’re infrageeks, right? Like we’re very good at like, you know, scaling and reliability and doing a lot of stuff. And I think that’s why that explains to a large extent like why we have, you know, in my opinion, I don’t have numbers, but like I think we have by far the largest market share in this space. Like we’re just very good at running, you know, hundreds of thousands or even millions of sandboxes. Yeah. I will say we’ll have to, we’ve had to rewrite our systems, you know, a bunch of times in order to handle that scale.

42:00 不过我们一直都很擅长这个。随着时间推移，还有其他一些方面。我的意思是，整个沙箱领域，说实话，并不是一个差异化程度很高的领域，但我们正在做一些很酷的事情。比如，随着时间推移，我们不得不增加对虚拟机（VM）的支持。我们之前的沙箱产品是基于 GVisor 构建的，但现在也提供完全虚拟化的机器了。事实证明，在很多特定用例中，你需要完整的 Linux 兼容性。还有一些事情是我们不得不做的，比如构建更好的基础组件，以支持更具交互性的功能，例如在沙箱之间来回传输数据、与文件系统交互。我们还花了很多时间做快照。我觉得，长远来看，真正能形成差异化的，可能会是一些很酷的功能，比如沙箱分叉、创建快照、从快照恢复，以及实时迁移。总的来说，我认为沙箱的独特之处一直在于它的短暂性：你可以启动一个沙箱，只让它运行几秒钟。这是我想进一步挖掘的方向。我觉得这里有很多很酷的事情可以做，比如让智能体和运行框架与沙箱解耦，让智能体控制多个沙箱、分叉沙箱、对东西做快照；又比如运行时间非常短的任务，可能每条命令都单独启动一个沙箱，诸如此类。所以我觉得这里还有很多很酷的可能性。我们已经在这方面花了很多时间，比如做快照。不过我觉得这里还有很多值得开发的酷东西。你刚才提到快照好几次了。42:00 But we were always like very good at it. There are other things like over time. I mean, the sandbox space in general, like, you know, like it’s like not the most like differentiated space, but you know, we’re working on some really cool stuff. Like over time, like we’ve had to add support for VMs, like virtual machines. So we previously bought our sandbox product on GVisor, but now we offer full virtualized machines as well. Turns out a lot of for certain use cases, you kind of need full Linux compatibility. There are other things we’ve had to build, you know, better primitives for like more interactive, you know, piping data back and forth between sandboxes, working with file system. We’ve had to spend a lot of time on snapshotting. I think a lot of the things that over in the long run will differentiate will be like, you know, really cool things like, you know, forking sandboxes, snapshotting, resuming snaps, like live migrations. In general, like I think sandboxes, the novelty of sandboxes is always, you know, this sort of ephemerality, like you can spin off sandboxes for a few seconds. And that’s something I like, that’s a string I want to pull more in. I think there’s a lot of really cool stuff you can do, you know, kind of decoupling the agent and the harness decoupling it from sandboxes, having agents control many sandboxes, forking sandboxes, you know, snapshotting things like, you know, like running basically like very short lived, maybe one per command, like things like that. So I think there’s a lot of cool stuff there. We’ve already spent a lot of time doing that, you know, around snapshots. But I think there’s a lot of cool stuff to be built there as well. You brought up snapshotting a couple of times.

43:30 比如说，你觉得快照的应用场景是什么？它能给我们带来哪些目前还没有的东西？对我来说，最重要的一点是，如今很多沙箱工作负载都属于原样迁移。大家往往会把沙箱想成这样：我有一台开发机，现在要把它搬到云端，让一个智能体来控制它，也许还会在沙箱里运行测试框架。我不确定从长远来看，这是不是思考智能体计算的正确方式。我觉得更好的模式是运行这种持续时间很长的智能体；对我来说，它们有点像持久化执行工作流。它们会全天候运行，时不时执行一些命令。如果能在 100 毫秒内启动一个沙箱，运行几条命令，然后关闭并创建快照，那就能实现更快的唤醒、休眠循环。这个循环越快，就越能创建只运行很短时间的沙箱，从而大幅提高计算效率。这样也能实现一些很酷的功能，比如分叉、迁移等等。再想象一下，像 Claude Code 这样的工具，不再采用非常线性的顺序流程，而是可以分支运行，同时启动数百个智能体。我觉得 CI 也是一个很能说明问题的例子。在 CI 的发展历程中，大部分时间里，人们谈论的都是 GitHub Actions 之类的东西。43:30 Like what do you see as like the use case for snapshotting? Like what would that give us that we don’t get right now? I think the big thing for me is like today a lot of sandbox workloads are kind of like lift and shift. Like people think a lot about sandboxes almost like, you know, I have my dev box, now I’m going to shove it in the cloud, I’m going to have an agent control it, maybe I’ll run the harness inside the sandbox. I don’t know if like long term that’s like the right way to think about, you know, on like compute for agents. I think a much better model would be you’re going to have this like very long running agents, which almost like to me are like kind of durable execution workflows. Like they’re going to run, you know, 24 seven, you know, every once in a while they’re going to run commands and, and you know, having the ability to like spin up a sandbox in, you know, 100 millisecond, run a few commands and then like shut it down and like snapshot it. I think the faster you can get that like, you know, wake up, you know, go to sleep cycle, the more you can sort of create sandboxes that only run for like a very brief period and that will make compute much more efficient. And it will also enable you to do some really cool things, you know, like forking and moving things around and, and, and, you know, having like imagine, you know, like Claude code instead of, you know, having sort of a very linear flow, like sequential flow, like kind of branching and, you know, running hundreds of agents at the same time. I think a lot of like CI is like another like illustrative example, like for most of the, the history of CI, you know, talk about like GitHub actions and things like that.

45:00 就像你的流程非常按顺序来，比如你知道，慢慢安装一堆东西，然后运行测试套件，接着生成一些报告之类的。你可以把它拆分成多个分片，但这有点麻烦。我特别想做的是，把整个测试套件拿过来，让每一项测试都在自己的沙箱里运行，对吧？然后让它在大约一千个沙箱里疯狂并行。为什么做不到呢？问题是，启动和停止沙箱的开销太大了。所以我觉得，启动和停止越是顺畅无阻，你就越能进入这样一个世界：可以以极高的突发性、大规模地并行运行你想运行的任意数量的任务。所以你知道，我是个基础设施迷，就像我说的，这些东西让我特别兴奋。一旦我们搞清楚这些原语和合适的构建模块，我觉得就能解锁很多以前用软件做不到的酷东西。 为什么以前没人做出来呢？我本来以为，以前大家也会想尽可能多地并行化 CI。你做不到，是因为开销，对吧？就像阿姆达尔定律什么的。现在并行化的开销非常高，因为比如说，现在你可以用 GitHub Actions；举个例子，我们在 Modal 的测试套件里做了并行化，对吧？但这基本上意味着，如果你把测试套件拆成两份，那么这两份都得各自完成所有安装和测试准备工作，而这要花很长时间。可能需要好几分钟。但假设你可以只做一次测试准备工作……45:00 Like you had a very sequential flow, like, you know, you slowly install a bunch of stuff, then you run the test suite and then, you know, generates some reports or whatever. And it’s like, you can break it up into shards, but it’s kind of annoying. Like what I would love to do is like take your entire test suite and just run every single test in its own sandbox, right? Like, and just paralyze the shit out of that around like a thousand sandboxes. Why can’t that be done? I mean, the problem is like there’s too much, there’s too much overhead around like, you know, starting and stopping sandboxes. So I think the more like you create like, you know, frictionless, like starting and stopping, the more you can enter a world where like you can do this like extremely bursty, like large scale, like run as much as you think you want in parallel. So, you know, I’m an infrageek, like I said, like I get very excited about these things. You know, once we figure out all these primitives and the right building blocks, I think it’s going to unlock a lot of really cool stuff that, you know, we previously could not do with software. Why hasn’t that already been built? I would assume even previously people would want to parallelize as much of their CI as possible. You can’t, but the overhead, it’s like Amdahl’s law or whatever. Right? Like the overhead of parallelization right now is very high because like right now, like you can get up actions, for instance, like we parallelize in the modal test suite, right? But like what that basically means is like now if you take your test suite and you split it up in two, like now each of those shards both need to do all the installs and the test setup. And that takes a long time. Like it can take minutes, but let’s say you can do the test set up once

46:30 然后你给容器创建快照，再克隆它，这样就可以独立运行测试套件了。我觉得，只有把摩擦和开销，也就是交易成本降下来，很多好处才能真正实现。我觉得你们和 Anthropic 有合作，他们好像在某些地方使用沙箱，比如 Modal 沙箱。是的，我是说 Anthropic 的托管代理，这算是一种自带沙箱集成。所以你可以用 Modal 来提供沙箱，是吧？这项合作是怎么达成的？具体是怎么开始的？我觉得 Anthropic 想要打造一个蓬勃发展的生态和社区，而我认为他们靠推理服务赚钱，对吧？好吧。所以我觉得，他们很有动力去推动其他方面尽可能发展出一个庞大的生态系统；只要你使用 Anthropic 的推理服务，他们就希望覆盖面尽可能广。所以我觉得，他们和我们这个沙箱提供商合作，是非常合理的选择。另外，我觉得我们得聊聊 Hugging Face 那次事件。你愿意谈谈吗？愿意，当然，相关信息都已经公开了。好，那你能不能简单说说当时发生了什么？我记得他们用的是 Modal 沙箱，但问题并不是出在你们的沙箱上。没错。我记不清这次漏洞利用的所有细节了，不过我想，你指的是 Modal 也牵涉其中这件事。是的。所以，关于这件事我想说的是，基本上，里面运行的代理……46:30 and then you snapshot the container and now you clone it and now you can run the test suites independently. I think a lot of the benefits cannot be, you can’t materialize those benefits until you get the friction and the overhead down, the transaction cost. I think you guys have a partnership with Anthropic and they’re using sandboxes for something, like modal sandboxes. Yeah, I mean the managed agents with Anthropic, this is sort of bring your own sandbox integration. So you can use modal for those, yeah. How did that come out? Like come to be? I think Anthropic is like trying to create like a thriving ecosystem and community and so I think they make money from inference, right? Okay. I think it’s very much in their interest to promote, for everything else, as much of an ecosystem as possible and they want to have the largest possible surface area as long as you use Anthropic for inference. So I think it makes like very rational sense for them to partner with us as a sandbox provider. Also, I feel like we have to bring up the hugging face incident. Are you okay talking about that? Yeah, yeah, I mean it’s all there. Yeah, yeah. So yeah, could you just briefly describe like what happened and I think they were using the modal sandbox but it wasn’t your, you know, it wasn’t like a fault of the sandbox. Yeah, exactly. I forget all the details of the exploit but I mean, you know, I think what you’re referring to is the fact that like modal was involved. Yeah. And so, you know, just to comment on that, like basically the agents running inside,

48:00 你知道，那个针对 Hugging Face 的攻击，他们在一个部署在 Modal 上的应用中发现了一个漏洞，并利用它作为某种跳板来获取访问权限，据我了解，最终获得了完整的互联网访问权限。我不记得全部细节了。需要说明的是，这是一个运行在 Modal 上的应用遭到了入侵。被黑的不是 Modal 本身。就像 AWS 一样，很多人会在 AWS 上运行东西，偶尔这些应用会遭到入侵。这是否意味着 AWS 被黑了？当然不是，对吧？所以有很多人在 Modal 上运行应用，你知道，其中一个应用遭到了入侵。我认为，我们确实能从中吸取一些教训。我觉得我们一直在反思的一件事是：我们能不能默认采用更合理的安全限制？借用 AWS 这个例子，再打个比方，S3 过去很容易因为误操作而被设为公开。AWS 意识到，这未必会直接给 AWS 本身带来安全问题，但很多客户会不小心泄露数据。因此，现在要把 S3 存储桶设为公开就难多了。类似地，我们也在探索，能不能默认把安全限制设得更严格一些，同时又不至于破坏让 Modal 好用的那些特性。我觉得还有一件事——算是另一个话题——就是智能体在网络安全方面越来越厉害了。所以我们其实也，作为一种……48:00 you know, that exploited hugging face, they discovered a vulnerability in an app that was deployed on modal and they used that to sort of, it’s almost like a springboard for getting access, getting full internet access is my understanding. I don’t remember the full details. To be clear, this is an app running a modal that was hacked. Like it was not modal that was hacked. So it’s like AWS, a lot of people run stuff on AWS, occasionally those apps get hacked. Does that mean AWS got hacked? Like no, of course not, right? And so there’s a lot of people running apps on modal and, you know, one of those apps got hacked. And I think, but like there are some lessons we can learn and I think one of the things we’ve been reflecting on is like, could we have more sensible, you know, security restrictions by default, I think to use the AWS analogy and just as another analogy, you know, S3 used to be this, like, you know, it used to be very easy to accidentally make an S3 bucket public and AWS realized like, you know, it doesn’t necessarily cause it, you know, poses security problem for AWS, per se. But a lot of their customers accidentally would, you know, like leak data. And so S3, it’s much harder now to make S3 buckets public. And so in a similar way, it’s like, we’re exploring like, you know, can we make security a little bit more restricted by default without necessarily, you know, destroying the magic that makes modal good. And I think there’s also, you know, it’s kind of a separate sidetrack is like, agents are getting really good at cybersecurity. So we’ve actually also, as a sort of, you know,

49:30 从这次事件中学到的另一个教训是，我们开始在内部大量使用智能体来寻找漏洞。到目前为止还没发现什么，但它们非常擅长深入系统的各个部分进行探测、利用漏洞，或者尝试利用漏洞、提升权限并逃出沙箱。所以，这确实是我们目前正在积极探索的方向。 你们具体是怎么让智能体遍历代码库的？就直接跟它们说：“嘿，找找安全漏洞”？ 不是，我们只是把它们放进沙箱里，然后告诉它们：“你能试着解释一下吗？试试看怎么逃出沙箱。”这就像密室逃脱，只不过参与者是智能体。 挺酷的。你们什么时候开始这么做的？ 几个月前。还挺让人兴奋的。大概是 Fable 出来的时候吧。 对，差不多同一时间。 好吧。我对网络安全的看法是，我确实觉得它们很擅长发现漏洞，但我也认为我们可以很好地用它们来防御。所以我觉得，像 Modal 这样的公司，以及现在的许多其他公司——我敢肯定，你去问问几乎所有系统方面的公司，他们都在考虑同一件事：我们怎样才能利用智能体进行防御，修复所有漏洞？所以从长远来看，实际上换个角度说，我倒没那么担心。我觉得不管怎么说，智能体和网络安全领域的所有突破，都会让世界变得更安全。 你觉得那些大型网络安全公司是不是都完蛋了？ 我不确定是不是真的，因为你问的是一个还很遥远的问题。49:30 separate sort of learning from this incident, like we started using agents internally quite a lot to try to find vulnerabilities so far hasn’t found anything, but like, they’re very good at like, probing into like very deep parts of the system and exploiting, trying to exploit things and trying to escalate and get out of sandboxes. So, you know, it’s, it’s, it’s something we’re exploring very actively right now. How exactly do you get the agents to like run through the code base? You’re just like, hey, like find security vulnerabilities? Well, no, you just like, you just put them in a sandbox and tell them like, can you try to explain? Just run through the escape from the sandbox. It’s like, escape the room, but for agents. Cool. When did you start doing that? We started doing it a few months ago. And it’s pretty exciting. Like when Fable, I guess, came out. Yeah, roughly around the same time. Okay. I mean, my view on like cybersecurity is that, you know, I actually think they’re very good at finding vulnerabilities, but I think you can also use them defensively very well. So I, I, I think, you know, companies like modal and many other companies right now, like, I think, you know, if you go to talk to like pretty much every system, I’m sure they’re all thinking about the same thing. Like how do we use agents defensively to, to, to, you know, fix all our vulnerabilities? So over time, actually, in a way, like I’m not that nervous. I think if anything, like agents and cybersecurity, all the breakthroughs in cybersecurity will, will make the world safer, if anything. Do you think that like all of the cybersecurity, the big cybersecurity companies are just cooked? I don’t know if that’s true, because I mean, you’re asking something that’s like very far

51:00 这超出了我的核心专业领域。不过我想到网络安全时，你们卖的很多东西实际上都是恐惧感。这几乎既是一种品牌塑造和营销策略，当然也包括核心产品。但我觉得，在 AI 时代，品牌之类的东西实际上可能更有价值。所以我不一定会直接断定他们要完蛋了。我觉得他们和企业客户有着深厚的关系，而且他们有庞大的销售团队，非常擅长讲一些吓人的故事之类的。我觉得他们会继续很擅长这一套。我问这个只是因为，嗯，你们是自己做，而不是找 Palo Alto Networks 这类公司。不过我猜在意这些的也只有大型企业。而且出了问题，他们还能找人背锅。确实。对。很多时候，你就是想找一家老牌供应商，这样出了问题就能怪他们。我是说，在某种程度上，这确实会稍微减少我们请人工渗透测试人员的必要性，而大多数初创公司都会定期做这类测试。所以以前我们可能会做——我其实不知道之前的周期是什么，但我觉得以前大概每六个月做一次。现在也许一年做一次就够了吧。我不知道。不过我觉得这些东西也能让人类更高效地工作。所以，嗯，我不知道，我也有点相信，现在人类加上 AI 会更加强大。所以也许我们可以请渗透测试人员……51:00 outside of my like core expertise. But when I think about like cybersecurity, a lot of what you’re selling is effectively fear. And that’s almost like a branding and like marketing strategy, as much as, you know, obviously the core products. But I think, you know, brand and, and things like that are actually like arguably like more valuable in the air of AI. So I wouldn’t necessarily jump to the conclusion that they’re cooked. I think they have, you know, deep relationships with the enterprise customers and they have, you know, big sales teams are very good at telling, you know, spooky stories and, and things like that. And I think they’re going to continue to be very good at that. I just ask, because like, yeah, you’re doing it yourself, you’re not going to one of like the Palo Alto networks or whatever. But I guess like it’s only like giant enterprises that care about that. Also, they can blame somebody if something goes wrong. That is true. Yeah. Yeah. You kind of want to hire in many cases, just like an established vendor in order to blame them. I mean, to some extent, it does reduce a little bit the need for us to do, to bring in pen testers like human pen testers, which most startups like do on a regular cycle. So maybe previously, you know, we would do it, I actually don’t know what the cycle was, but, but I think we used to do it like once every six months, roughly. Maybe now we only need to do it once every year. I don’t know. But, but like, I think with all of these things, you know, humans can also use them to get more efficiently. So like, you know, like, I don’t know, I’m kind of a believer also that like humans with AI is even more powerful now. So like, maybe there’s an argument that like we can bring in pen testers

52:30 真正擅长使用智能体、能更快发现漏洞的人。我不知道。我不是那种人，你知道，我是个乐观主义者。我创办了一家公司。所以你知道，我总体上对一切都挺乐观的。但我当然不觉得人类完蛋了，也不觉得公司完蛋了。我觉得在 AI 时代，很多公司会想办法弄清楚如何利用 AI 提高效率。我想稍微转向聊聊开发者体验，因为早期——其实直到现在——你们都很重视这一点，也谈了很多。开发者体验现在还重要吗？ 我觉得还是重要的。我想你说的背景大概是，你要谈的重点是：现在有了智能体，对吧？开发者体验，比如说文档，其实早就在那里了。智能体会去看文档，诸如此类。不过就这一点而言，我们在实践中发现，几乎所有能改善开发者体验的东西，也都能改善智能体体验，比如完善的文档、清晰的错误提示，以及快速的反馈循环，对吧？我确实觉得有些事情变了。前几天我在用 Cloudflare 配置一些东西，我觉得 Cloudflare 有时出了名地难用。我就让智能体来处理，它能帮我配置一大堆东西。你知道，所以我想说，智能体确实让过去那些比较难处理的事情变得容易了一点。你知道，处理 CLI 所带来的人力负担，可以说是稍微减轻了一些，因为你只要……52:30 who are really good at using agents and, you know, find vulnerabilities even faster. I don’t know. I’m not that, you know, I’m an optimist. I started a company. So like, you know, I’m generally like optimistic about everything. But like, I certainly don’t think humans are cooked or companies are cooked. Like I think in the era of AI, a lot of companies are going to figure out, you know, how to use AI to be even more efficient. I want to transition a little bit to like developer experience, because like early on, I mean, even now, like you guys focus a lot on that. I mean, you’ve talked a lot about that. Does developer experience even matter anymore? I think it still matters. And I think you’re sort of, you know, like the context, I think you’re official, like the thing you’re talking about is like, now with agents, right? Like the, the developer experience, like the docs are already there. The agents are going to look at the docs, like all that stuff. To that point, though, like almost all the things that make developer experience good, also makes the agent experience good is kind of what we’ve seen in practice, which is like things like having good docs, having descriptive error messages, having fast feedback loops, right? Like, I do think there’s certain things have changed. The other day, I was like setting something up with Cloudflare, and it’s like, I find Cloudflare like notoriously annoying to work with sometimes. And, and I threw an agent at it, and it was able to like just configure a bunch of like, you know, things and pollute me for me. And, you know, and so I will say like agents does, they do make it a little bit easier to work with things that are like harder to work with in the past, you know, like the sort of human toll of dealing with the AML as arguably gone down a little bit, because you can just like,

54:00 你知道，让你的 agent 去处理它。但大多数情况下，我觉得你知道，让开发者体验良好的那些事情，也会让 agent 体验良好。我还觉得，从某种程度上说，随着时间推移，基础设施也会变得更加程序化。我想这一直都是沙盒的理念：你知道，以前必须用 YAML 配置的东西，应该直接做成 SDK。应该更像命令式的操作，比如启动一个沙盒、用它做点事情，再把它销毁，对吧？而不是用某种 DSL 进行声明式配置。这也会推动一种可能以前并不存在的、稍有不同的接口。我觉得无论如何，SDK 依然和过去一样重要，甚至更重要了。你们有没有因为大多数人都在上面运行 agent，就对产品做过什么调整？我们有。我是说，我们也确实会进行很多内部测试：让 agent 构建一个移动应用并运行它。我是说，有一件非常具体、也挺有意思的事是，我们发现，由于 agent 的训练数据，现在要弃用东西变得非常困难。我们发现 agent 经常特别喜欢用旧版本的 Modal 语法，也更难引导它们改用新版本。所以过去大约一年里，我们在弃用和移除功能时不得不更加谨慎。我们必须非常小心地更新所有已发布的示例……54:00 you know, tell your agent to deal with it. But, but for the most part, I think, you know, the things that make developer experience good also make agent experience good. I think to some extent, also like infrastructure over time, anyway, becomes, it needs to be more programmatic. I think that’s always been the thesis about sandboxes, is that, you know, like things that you previously had to configure using YAML, that should just be an SDK. That should be more like imperative, like launch a sandbox, like do stuff with it, tear the sandbox down, right? Like, rather than sort of declarative using, using, you know, some sort of DSL, it also pushes like a little bit of a different interface that maybe previously didn’t exist. I think in either way, like SDKs are as relevant as they were, as they always were, if not more. Is there anything you changed in your product? Just because like, you know, most people are just running agents on it? We do. I mean, we also, we actually do run a lot of internal tests where we have agents, we tell agents to like, build a mobile app and run it. And I mean, a very concrete thing that is kind of funny is that we have actually seen that like, because of agent training data, it’s very hard to deprecate things now. We see that like, agents often like, love using, you know, old versions of modal syntax and it’s kind of harder to nudge them away from that. So we’ve actually had to be a little bit more conservative going forward, like, you know, over the last year or so with, with how we deprecate and remove features, it takes, we have to be very careful to update every single example that’s out

55:30 那些东西在网上。是啊。而且，而且，而且，你知道，技能，对吧？是啊，还有类似这样的东西。但是，但是，网上现在也有海量的 Modal 代码，会被纳入训练数据。我觉得这算是一个挺有意思的例子。还有一件事，顺便说一下，显然有点类似搜索引擎优化，也有人称之为智能体引擎优化，或者别的什么，智能体优化。我们经常会跑内部基准测试，比如问一个智能体：“嘿，你能帮我们部署一个 Stable Diffusion 吗？用什么工具都行。”然后看看它会不会推荐 Modal。这些事情对我们追踪长期变化很有帮助。或者问：“嘿，启动几个沙箱”，或者“嘿，把所有沙箱供应商排个名。”所以我们花了很多时间做这些事，以确保……你们怎么应对这个问题？是啊，你们会说：“嘿，搭建这个 Stable Diffusion 的东西，它会不会推荐 Modal？”那你们怎么应对这个问题？是啊，我觉得这更难。我觉得这就像搜索引擎优化，主要靠内容——你就是得发布大量内容。在一定程度上，你也可以了解大家的看法。我觉得可以多问问智能体，比如：“嗯，按可靠性给所有沙箱供应商排名”，或者“按价格给所有沙箱供应商排名”，然后看看结果。我感觉智能体似乎更容易误解定价，还有诸如此类的事情。所以我们也花了很多时间，让内容变得更清晰。55:30 there online. Yeah. And, and, and, you know, skills, right? Yeah, there’s like stuff like that. But, but like, there’s also an insane amount of like modal code that’s now like online too that goes into the training data. I’ve seen like that to be like kind of a, you know, a little bit of like an interesting example where like, another thing by the way too is like, there’s obviously like to sort of a similar to like search engine optimization, there’s like agent engine, you know, whatever people call it, agent optimization. We often like, run internal benchmarks, we’re like, ask an agent, like, hey, like, can you deploy a stable diffusion for us? Use whatever tool you want. And we see if it recommends modal or not. And those things are very useful for us to sort of track over time. Or like, hey, spin up some sandboxes or like, hey, like rank all the sandbox vendors. So we spent a lot of time like doing those things to make sure. What do you do about that? Yeah, you say, hey, like, you know, build this stable diffusion thing, does it recommend modal or not? Like, yeah, what do you do about that? Yeah, I think that’s harder. I think, you know, it’s like search engine optimization, I think it’s primarily through content is like, you just have to put out a lot of content. You can also to some extent, like sort of see the perception, I think, a little bit more like you can ask an agent like, you know, how does, you know, rank all the sandbox vendors by reliability or rank all the sandbox vendors by price and sort of see, and it does feel like agents are a little bit more prone to like, misunderstanding pricing and, and, you know, things like that. So we also spend a lot of time like just making content more clear

57:00 而且更容易被智能体理解，因为我们知道它们会索引所有内容，也就是网上所有关于 Modal 的内容。你有没有想过，干脆建一大堆 GitHub 仓库，让智能体往里面写一大堆 Modal 代码？试过吗？简直就是垃圾内容大炮。我觉得说到底，这事大概是能做的。我猜现在的语言模型训练数据，对什么是 AI 生成的内容可能相当不敏感，所以你大概可以生成很多垃圾内容。我们也考虑过这事，但我不太喜欢这个主意。不过理论上，这就像早期的 SEO 一样——直到现在某种程度上也是如此——自动生成内容是一种非常有效的策略。我觉得这确实可行。你知道，你也可以这么做。对。我想聊聊 FDE。听说你们会聘用前线部署工程师。你们是做什么的？我们进入了一个构建应用的新阶段，能够做出一些几年前还做不出来的东西。我们发现有很多公司需要一点帮助，对吧？所以几年前我们开始投入资源做 FDE，不过真正大规模扩展是在今年。年初我们大概只有 4 名 FDE，现在我想有 24 名左右了。所以规模扩大了不少。他们做的事呢，不完全是 Palantir 那种模式，但他们会和客户深入合作，投入很多精力。他们会和很多客户一起做性能调优，尤其是针对语言模型。我们发现，坐下来……能带来很大的好处。57:00 and more like, you know, understandable for agents, because we know they’re all indexing, like everything that’s out there and that’s talking about modal. Have you thought about just like, making a bunch of GitHub repos and having the agents like write a bunch of modal code in there? Have you tried that? Slop cannon. I think the truth is like, that is probably something you can do. My guess is like LM training data today is probably quite insensitive to like, what’s like AI output in the first place and you can probably like create a lot of slop. Like we thought about it, like, I don’t love that idea. But like in theory, like much like, you know, SEO in the early days and still to some extent, like auto-generated content is like a very effective strategy. I think it’s possible. Like, you know, you could, you can do those things too. Yeah. I want to switch to FDEs. Apparently, you guys hire like forward-appoid engineers. Like, what do you do? We’re in a new era of like, building applications and, and, and, you know, things that we couldn’t build, you know, just a few years ago. And we see a lot of companies that, you know, they need a little bit of help, right? Like, you know, and so we started investing in FDEs a couple years ago, but it’s really been this year, it was kind of scaled it up a lot. I think we started this year having like four FDEs now. I think we have like 24 or something like that. So we’ve grown it a lot. And they do it, you know, it’s more, it’s not exactly the pound term model, but they do a lot of like heavy, you know, sort of customer engagement. Like they work with a lot of customers on performance tuning, especially with LMS. We see that there’s so much benefit from like, you know, sitting down

58:30，和客户合作、训练 draft 模型并调整参数，还有 DLM 和 SG Lang。他们经常会修 bug——或者不修 bug——但比如客户遇到问题时，他们通常会和客户一起想办法，看看怎么绕过这些问题。我们让 FDE 做的另一件事是主动发现问题：比如看到某些客户在模型资源上配置过量，例如申请了过多 CPU 或内存，这样成本会很高。我们会尽量发现这类情况，并和客户坐下来聊：“嘿，你们现在在模型上花了不少钱，其实我们可以帮你们省钱。”当然，在正在推进的交易中，我们也会帮客户把代码从其他平台迁移过来，诸如此类。我觉得我们从来没打算把公司做成那种服务占比很高的业务。我认为 FDE 的目标是：A，参与销售流程并促成交易；B，为客户提供服务，做一些客户支持。但我觉得最重要的是 C：弥合产品中的差距，并最终把这些需求变成真正的功能。所以在很大程度上，我总会问 FDE 做的任何事情：从长远来看，我们要怎么把它变成产品，这样就不需要 FDE 来做了？对了，你们会招聘 PM 吗？58:30 with customers and training draft models and tweaking, you know, parameters and DLM and SG Lang. They often, you know, fix bugs or not fix bugs, but like, you know, when customers run into issues, like they often work with customers trying to figure out like, how do we work around that? Or another thing we’ve used FDEs a lot for is like, you know, kind of proactively, you know, if we see certain customers, you know, over provisioning with model, like customers requesting too much CPU or too much memory, for instance, like that’s kind of expensive. And like, we try to catch those things and we try to sit down with those customers like, yo, like you’re spending a lot of money on model right now, we can actually help you save money. And then of course, like with sort of active deals, you know, we helped them migrate code from other platforms, things like that. I think, you know, we never have the intention of becoming like service heavy business. I think that the goal for FDEs is like, you know, A is to be a part of, you know, the sort of sales process and like closing deals. B is to serve customers and do some like, you know, customer support. But like, I actually think like the most important thing is like, C is like bridging gaps in the product that we later want to turn into like real features. So I think to a large extent, you know, anything an FDE does, you know, my question is always like long term, how do we turn this into product so we don’t need an FDE for this? Yeah. Do you guys hire PMs?

1:00:00 产品经理？嗯，不是。你会说这些 FDE 有点像准产品经理吗？这是稍微有点不同的职能。我觉得，当然，在更偏客户-facing的那一面，他们确实是；但在更偏内部的那一面，这算是产品管理中一个相对独立的部分，更多是在思考 Modal 接下来应该投入哪些基础构件。然后还有介于两者之间的工作，比如看着可能有 10、20 个客户，他们各自提出的需求都有些不同，但其中有没有什么共性？我们该怎么把这些需求转化为产品？我们确实有人和我们一起做产品，而这个角色也逐渐发展起来了。你知道，有些人之前做 DevRel 或者是工程师，后来基本上成了产品经理，因为他们在战略性地思考我们应该构建什么这方面展现出了很强的能力。所以我其实并不反对产品管理。我确实认为，过去很多公司太急于招聘产品经理了。而且我觉得有个风险：你多少能在很多产品中看到这种情况——有些产品显然是由一群急于求成的产品经理想出了一些奇奇怪怪的东西，然后就直接上线了。我觉得，要在一家基础设施公司把产品管理做好非常、非常难，因为这需要非常深入的战略思考，要考虑哪些是基础构件，我们希望一到两年后发展到什么程度。因为事实是，构建基础设施非常困难。我们该怎么确保最后不会弄出 35 个功能，而它们……1:00:00 Product managers? Yeah. No. Would you say these FDEs are kind of like pseudo ish PMs? It’s a little bit of a different function. Like I think certainly on the more like, you know, customer facing side they are, but there’s like, you know, on the more like inward, you know, internal side, like this is sort of separate part of like product management, which is more like, what are the next building blocks Modal should invest in and like, you know, and then there’s like something in between like, you know, looking at like maybe 10, 20 different customers that are all asking for like slightly different things, but maybe what are the patterns? Like how do we turn this into product? We do have people doing product with us and it’s sort of evolved, you know, we’ve had people basically like become product managers after previously being either DevRel or engineers and showing strong aptitude on doing thinking very strategically about what we have to build. So I’m actually not like anti product management. I do think that a lot of companies in the past hired product management product managers too eagerly. And I think there’s a risk that you kind of, I can sort of see it in a lot of products like certain products like you can tell like, there’s just like eager product managers that just thought of like, kind of like, wigity things and just ship them. I think doing product management at an infra company really well is very, very hard because I think it takes like a very deep, you know, level of strategic thinking thinking about like, what are the foundational building blocks? Where do we want to be in one to two years? Because the truth is building infrastructure is very hard. And how do we make sure we don’t end up with like 35 features that are like,

1:01:30 看起来有点相似，但做的事情都稍微有些不同。这种层次的思考，你知道，是非常罕见的。也正因如此，我们还没有急着组建产品管理团队。是的，这很有道理。你觉得你们的团队自从生成式 AI 出现以来，发生了哪些变化？我想，你之前显然在 Spotify 工作过，当时有特定的团队结构；现在有了编程智能体，你们的团队结构是不是发生了很大变化？有趣的是，没错，我确实认为团队结构变化很大，但我其实不确定这是由生成式 AI 推动的。我尤其注意到一件事：大家似乎重新开始重视做个人贡献者（IC），我觉得这对整个领域来说非常、非常积极。你知道，如果回顾这个领域十年、二十年前的情况，大家都想做管理，因为那是获得更高薪酬、更多认可和更高声望的途径。我认为这种转变甚至在生成式 AI 出现之前就已经开始了，但我确实觉得 AI 大大加速了这一趋势。比如说，我们会非常明确地告诉 Modal 的每个人，经理和个人贡献者（IC）在声望上没有区别：大家都是技术团队的一员。归根结底，重要的是你做出了什么贡献。我们看到很多前任经理转为个人贡献者，因为他们想回去写代码、做开发。所以，你知道，大家也逐渐不再那么倾向于非技术型经理了，1:01:30 kind of similar looking, but all do like a little bit slightly different things. That sort of level of thinking is something that, you know, is very rare to find. And for that reason, we haven’t aggressively tried to build up a product management team yet. Yeah, that makes sense. How would you say your teams have been evolving maybe like since Genialite? Because I would assume, yeah, I mean, you obviously worked at like Spotify, you had like a certain team structure, have your team structures like evolved a lot now that, you know, we have coding agents. In a funny way, like, yes, I do think team structures have changed a lot, but I’m actually not convinced it was like Genialite driving that. I think in particular, like one thing I’ve seen a lot is that there’s been sort of a focus, like a swing back towards being in IC that I actually think is very, very positive for the space as a whole. I think, you know, if you look back at the space like 10 years ago, 20 years ago, everyone wanted to be in management because that was the way you would get more money and more recognition and more prestige. I think that shift actually started even before Gen AI, but I do think it’s like heavily accelerated with AI. Like, you know, we make it very clear to everyone at Modal that like manager or RC, it’s like there’s no difference in prestige. Like everyone is just a member of technical staff. You know, it’s about ultimately your contributions and we see a lot of X managers, you know, becoming ICs and because they want to go back and build code. And so, and you know, and there’s been sort of shift away from like, you know, non-technical managers,

1:03:00 大家，包括我在内，现在在 Modal 写很多代码，Akshath 也是，技术团队里几乎每个人都在写代码，许多非技术团队的人也在写。所以在我看来，这是一个比较新的现象。不过我其实不确定这是否一定是生成式 AI 带来的，但我觉得生成式 AI 大大加速了这一趋势，因为它进一步降低了写代码的成本，让写代码变得更容易。如果非技术人员也在提交代码，那该怎么审查呢？你们是怎么做代码审查的？ 我们会做代码审查，但我们认为它是可选的。我的看法是，在很大程度上，代码审查往往是人们试图逃避将东西合并到生产环境所带来的责任的一种方式。如果你们有这种文化，那在我看来，这就说明你们需要在 CI/CD 等方面投入更多，尤其是 CI，但 CD 也很重要——容器集成，因为这关系到部署。 我们会做代码审查。比如我写代码时，偶尔会问代码的作者：“这符合你对事情的看法吗？”我希望自己没有从根本上改变架构。不过我们也会在内部明确说明，代码审查是相当可选的。如果你只是在改文档，或者只是在做一些基础的事情，就别等着人来审查，尤其是在 AI 时代，为什么每一段代码都需要人来过目？其实没必要。但这确实要求你们有非常完善的控制措施，你知道的，1:03:00 everyone, including me, I still write a lot of code at Modal and Akshath too and pretty much everyone in the tech team and a lot of people who are not in the tech team still write code. So that to me is like, you know, a recent phenomenon, but I actually don’t know if it’s like necessarily cost by Gen AI, but I think Gen AI has like greatly accelerated it because it made it even lower the cost of like writing code is even easier. So if you have like non-technical people like committing code, like how do you, how do you review that? Like how do you guys do code reviews? We do code reviews, but it’s, we think of it as optional. I mean, for most, my view is like code reviews in large part is like people, in many cases, trying to just escape the accountability of like, you know, merging things into production. And if you have that like, you know, if you have that culture, like then to me, that’s a sign that you need to invest way more in, you know, CD and CI and things like that is CI in particular, but also CD, containers integration, because it’s deployment. We do code reviews. Like, I mean, like when I write code, like occasionally, like I’m asking, you know, authors, like, is this in line with how you think about things? I hope I’m not changing the architecture fundamentally, but we also make it clear internally that like code reviews are quite optional. And, you know, if you’re just changing docs, if you’re just doing, you know, some basic stuff, like don’t wait for a human to look at it, especially in the era of AI, why do you need a human to look at every piece of code? You don’t have to. But that does require you to have, you know, very good controls, you know,

1:04:30 内部会围绕测试之类的事情。对，没错。比如基础设施公司，我们的正常运行时间要比一般初创公司多一个 9，对吧？但这些都是你必须做的投入。你必须构建非常有韧性的系统，比如到处都要设置速率限制，系统之间要解耦，确保没有任何单点故障，还要有极高的测试覆盖率。如今智能体其实很擅长编写测试。几年前，我还觉得这是智能体的一大短板。现在它们其实已经相当擅长了，因为我认为编写好的测试需要稍微跳出当前正在做的事情，换个角度，甚至有点像唱反调一样审视问题。不过现在它们也很擅长写测试了。所以我觉得这些都有帮助。你们写代码时会用前沿模型吗？还是也试过开源模型？我们什么都试。不过，是的，我的意思是，Initiative 很多时候用 Cursor，当然也用 Claude 和 Codex。现在很多人也在用 Night Shift，以及各种开源工具和其他技术栈。所以我没有特别明确的偏好；我希望大家去尝试不同的东西。我们会用前沿模型，也会用开源模型，而且现在我们正在内部更多地进行“自家试用”，因为我们实际上已经在大规模运行许多开源模型的推理服务。我觉得在很多智能体开发中使用某些内部端点也很有帮助，所以我们也在努力推广这一点。我记得大概一年前，也可能是两年前，读到过一些东西，……1:04:30 internally around, you know, testing things. Yeah, and exactly. Like, you know, infra companies, like we need one more nine than the average startup, right? In terms of uptime. But like, you know, these are investments you just have to make. Like, you have to build systems that are very resilient to like, you know, you have to have rate limits everywhere. You have to decouple systems. You have to make sure that like, nothing is a single point of failure. They have to have extremely good test coverage. Agents are actually really good at writing tests these days, which was like, you know, a few years ago, I thought like, one of the major things missing in agents, now they’re actually quite good at like, because I think it takes a little bit of like, sort of kind of like, need to step out of what you’re trying to do for a second and be almost a little bit like devil’s advocate to write good tests. But now they’re actually quite good at writing tests too. So I think, I think all these things help. Do you guys use like frontier models for coding or do you guys have you guys experimented with like, open source ones? We experiment with everything. But yeah, I mean, you know, initiative is a lot of cursor, then, you know, of course, Claude and Codex and now a lot of people are using, you know, night shift and all these open source stuff or other stacks. So I don’t necessarily have like a strong point of view, like I want people to experiment and try different things like we, you know, use frontier models, use open source models, we’re trying to do a little bit more dog floating internally. Now that we’re actually running inference for a lot of the open source models, like a very large scale. I think it’s also helpful to like use certain internal endpoints for a lot of agentic development. So we try to promote that a little bit. Something I read maybe like, I don’t know, a year ago, maybe like two years ago,

1:06:00 你刚才说，你觉得你们不会有很大的本地部署业务。你们有没有可能改变了看法？没有，我没改变看法。对我来说，我是 Claude 的极端拥护者。或许我们可以退一步想想：2008 年、2009 年左右云计算兴起时，我的第一反应是：这太疯狂了。我为什么要把代码放到别人的电脑上运行？这简直荒谬。结果大概六个月后，我就这么做了。我想，我花了六个月才改变想法，而企业大概又花了 15 到 20 年。但你看看 Snowflake 这样的供应商，或者 AWS，还有 Datadog 等等，云显然是赢家，对吧？多租户系统显然也在胜出。对我来说，本地部署越来越像是一种过时的商业模式。事实上我觉得，BYOC，也就是 VPC 对等连接之类的做法，在我看来都反映了一种过时的安全模型：我们需要保护自己网络里的所有东西，对吧？把网络当作护城河。但我认为现在已经不是这样了，对吧？我们已经转向了更健全的安全范式，比如零信任之类的。因此对我来说，BYOC 这些做法看起来越来越……1:06:00 you were saying that you didn’t think that there was going to be like a big on-prem business for you guys. Have you maybe like changed your opinion on that? No, I have not changed my opinion. I think, moral to me, I’m a Claude maximalist and maybe to take a step back, like when the cloud came in 2008, 2009 or whatever, my first reaction was like, that’s crazy. Like why would I want to take my code and run it on someone else’s computer? That’s like insane. And then like six months later, I was doing it. And I think, you know, like took me six months to change my mind. It took probably enterprises like another 15 to 20 years. But like you look at, you know, vendors like Snowflake or I mean AWS and like, I think there’s many examples of Datadog, et cetera. Like the cloud is clearly winning, right? Like and the multi-tenant systems are clearly winning. And you know, to me, on-prem looks increasingly like sort of a legacy business model. And I actually think, you know, BYOC, you know, sort of VPC peering, like all of these things, you know, like to me, they reflect sort of an outdated security model of like, we need to protect everything inside our own network, right? Like using the network as a mode. And that I don’t think is true anything anymore, right? Like we’ve sort of moved on to more like actually robust security paradigms, like, you know, zero trust and things like that. And so to me, BYOC and those things, like they look increasingly

1:07:30，就像是过时的，至少从安全角度来看是这样。从我们的角度来说，我们能够完全掌控整个基础设施栈这一点也很重要：所有硬件都是我们租用的，所有基础设施都由我们部署和管理，我们负责部署、管理运行状况，并监控所有指标和主机。这意味着我们可以非常快速地迭代，打造更好的产品。我认为这一点非常有价值。相比之下，以前有些客户使用企业版，需要把它部署到客户自己的环境中；他们在发布管理方面必须非常谨慎。而我们不需要操心这些。比如说，如果我们想引入一个新的数据库，我们就可以引入，因为我们能够掌控它。我们不需要考虑客户能使用什么。所以在我看来，多租户确实有很多不同的好处。 话虽如此，如果将来我们要和某家非常大型的《财富》500 强企业客户合作，而他们要求 BYOC，我们以前也做过这种事，到时候会给他们报价。这个报价会非常高，目前还没人接受过。但谁知道呢？如果价格合适，我还是会把它做出来。我还觉得，可能另外有一点也挺有意思：理论上，BYOC 也可能成为一种合作方式，让我们以某些方式获得算力。我们有时会和 Neolabs 合作，而这些 Neolabs 已经预留了非常大的算力。问题是，既然他们已经有了算力，我们该怎么让他们运行 Modal？BYOC 或许能解决这个问题。我们仍然可以把控制平面留在自己的环境中。1:07:30 as like outdated, at least from a security perspective. From our perspective also, like the fact that we have full control over our entire infrastructure stack, we rent all the hardware, we deploy, we manage all the infrastructure, we deploy and like manage health and observe, you know, all the metrics and all the hosts. It means we can iterate very fast and build a greater product. And I think that’s very, very valuable versus, you know, customers in the past who had enterprise editions that they had to deploy on customers environment, they had to be very, you know, they have to think very carefully about release management and having the like, we don’t care. Like if we want to introduce a new database, like we can introduce a new database because we have control over it. Like we don’t need to think about what customers have access to. So to me, multi-tenancy is, is, yeah, has so many different benefits. I mean, that being said, like, you know, if we were to work with like, you know, some very large Fortune 500 customer in the future and, you know, they ask for BYOC, you know, we’ve done this in the past, we’ll give them a number. Like, you know, that number is very high and no one said yes to it so far. But like, who knows? Like, you know, at the right price, like I would still build it. And, and I think there’s like maybe separately, like another thing which I think is interesting, which is like, in theory, BYOC might also be a way to work with and get capacity in certain ways. Like we sometimes work with Neolabs. And those Neolabs have already have very large capacity reservations. And, you know, the question is like, you know, how do we, how do we get them to run modal when they already have capacity? And BYOC might be a solution. We can still keep the control plane in our environment.

1:09:00 另一方面，我们甚至也可以考虑利用这部分容量来运行其他人的业务。所以理论上有些挺有意思的做法，比如把所有这些容量汇集到一个大池子里。我们对其中一些想法感到很兴奋。不过总体来说，我觉得本地部署这件事，我们可能永远都不会做。但 BYOC，也就是自带云服务，也许可以。我可以说 BYOC。我应该解释一下。假设你很快就要再融资——我的意思是，筹更多的钱。你会把这笔钱用在哪儿？我的意思是，就像我刚才提到的，在容量方面，需要投入很多资金，也就是现金流，或者说预付款，对吧？所以在我看来，这就是现在最值得花钱的地方。我们正在非常积极地扩充容量。也许未来我们会转向其他类型的部署模式，比如自有资产之类的。但我觉得这始终需要投入一些现金，比如支付某种首付款、预付款之类的。所以在我看来，这才是眼下资金最重要的用途。当然还有其他开支，比如薪资、办公室场地之类的。但放在我们整体的财务状况里看，这些基本上只是四舍五入时会忽略不计的小数，对吧？是啊，有意思。你觉得这种情况会改变吗？不会，我觉得我们这项业务的运营杠杆简直惊人。你看，我们有 200 名员工，雇他们要花不少钱，但是……1:09:00 And the other flip side of that is like, we could maybe even run like other people’s stuff on that capacity. So there’s some like, kind of fun stuff you could do in theory where like, you can sort of, you know, like kind of aggregate all that capacity into one big pool. So we’re excited about some of those ideas. But in general, like I feel like on-prem, I don’t know if we’re ever going to do. But BYOC, like maybe bring your own cloud. I can say BYOC. I should explain. Let’s say hypothetically, you are going to raise, I should tend more money like soon. What would you use that money for? I mean, like I mentioned, like in this capacity situation, the, a lot of the money, a lot of the cash flow, I should say, that’s required is capacity, prepayments, right? And so that to me represents the best use of cash today. We’re ramping up capacity very aggressively. Maybe we move into, you know, other types of deployment models in the future, ownership or whatever. But I think there’s always going to be a cash component of that, like some sort of down payment or prepayment or some sort of that. So that to me is the greatest use of cash right now. There’s of course other things like, you know, salaries and office spaces and stuff like that. But that actually in the grand scheme of like our financials, like it’s basically a rounding error, right? Yeah, interesting. Do you think that’s going to change? No, I think, you know, the sort of operating leverage that we have in this business is kind of insane. Like, you know, we have 200 employees, you know, they cost a lot of money, but, you know, the

1:10:30 GPU 贵得多。是啊。如果要你说说，现在你最关注的三件事，或者说公司最关注的三件事是什么，你会怎么说？产能，以及各种财务层面的复杂问题。还有，整体来说，我们该如何看待这个领域？该如何考虑定价和产品组合？该如何看待自有与租用之间的矛盾？我们应该朝着新云的方向发展吗？还是应该反其道而行？这种反向立场，可能是我最常思考的事情。第二件我常想的事，大概是产品战略。我经常会想，下一批基础构件是什么？比如说，如果我们要更多地做智能体相关的东西，需要哪些东西？是否需要某些存储原语或计算原语？或者说，未来几年，世界会突然意识到自己需要什么？理想情况下，到那时我们已经准备好了。总体来说，我们一直很擅长预判需求，并提前构建产品，比如无服务器函数和沙箱。那么下一件会是什么？第三件我常想的事，其实更多是文化、公司建设之类的。我是说，我们最近刚超过 200 人。公司发展得非常快。到年底，我们很可能会达到 250 人。公司快速发展时，有 250 个人都在努力思考如何为 Modal 创造最大价值；我们得确保他们掌握正确的背景信息，有恰当的动力，也乐于每天来上班，因为他们有机会打造真正酷炫的东西。1:10:30 GPUs cost a lot more money. Yeah. If you had to say, like, what are the top, like, I don’t know, like three things that are your biggest focus or the business’s biggest focus right now, like, what would you say those are? Capacity and the sort of financial complexity. And, you know, in general, like, how do we, how do we think about that space? How do we think about pricing and packaging? How do we think about the sort of tension between ownership and renting? Should we be going a more neocloud direction? Should we go in the opposite direction? The sort of counter position, like, that’s probably the biggest thing that I think a lot about. The second biggest thing I think a lot about is probably around, like, more like product strategy. Like, I think a lot about, like, what is the next building blocks? Like, if we, you know, do more like, agentic stuff, like, what are the things we need, you know, are there certain storage primitives or compute primitives? Or in general, like, what is the thing that the world is going to wake up and realize they need over the next few years? And ideally, we have that at that point. We’ve been pretty good in general at, like, you know, anticipating demand and building products in advance of it with the serverless functions in the sandboxes. So what’s the next thing? Then I would say, like, actually the third thing I think a lot about is, like, more on, like, culture and company building and stuff like that. I mean, we recently crossed 200 people. We’re growing extremely fast. We’ll probably hit 250 by the end of the year. Like, when you’re growing very fast, you have 250 people that are all trying to figure out how they can have the most value to modal and, you know, making sure that they have the right context, that they have the right motivation, that they’re excited to come into work every day because they get to build something incredibly cool

1:12:00 而且他们知道，你知道，是什么让 Modal 成功，他们能做些什么，如何让激励机制保持一致，并确保我们——我们这些人——拥有一种非常积极的文化，让大家在默认彼此信任的基础上能够做很多事，同时也有责任感，比如说，交付最好的成果，并且，如果没做到，也要承担相应责任。对我来说，这些都是公司发展到这个规模时至关重要的事情。比如说，只有 10、20、30 个人的时候，大家可以一起坐在一个房间里，很多事情都是不言自明的，大家也能凭感觉理解。人们大致能读懂我的情绪，比如我稍微有点不高兴时，他们看得出来；我高兴时，他们也看得出来。但现在我们有 250 个人了，大家不一定都能察觉到这些，而且，当然，也有很多新员工，可能有点紧张，不太敢跟 CEO 说话之类的。所以，你知道，我该怎么保持平易近人？还有，这个邓巴数，我觉得它更多是被夸大了，至少围绕它的那个说法是如此。但它其实还是有点道理的：大约在 150 到 200 人这个规模，打好文化基础非常、非常重要。你们明年打算招聘多少人？过去 12 个月里，我们按年化计算，规模大约增长了 4 倍。我觉得相对而言，这种增长速度是不可持续的。我见过团队每年翻一倍，这并不算特别难做到；但增长 4 倍时，你开始会看到文化出现偏移，也会开始看到不同工作方式形成各自的小圈子。1:12:00 and they know, you know, what makes modal successful, what they can do, aligning the incentives and making sure that, you know, we, we, people have like a very positive culture where they have the sort of implicit trust to do a lot of stuff, but also the accountability to like, you know, deliver the best things and, you know, and be held accountable to like, if they’re not doing it, to me, those are like, you know, super critical things around this scale. Like, when you’re like 10, 20, 30 people, you can all just like, kind of sit in a room and like, it’s all kind of implicit and you kind of just like, feel it. People can sort of read, read me like, they can tell me when I get a little bit annoyed. They can tell when I’m happy, but like, now we’re 250 people and like, people don’t necessarily see it and, you know, of course, there’s like a lot of new people that are like, maybe a little bit nervous to like, talk to CEO and like all this stuff. So like, you know, how do I remain approachable and how do I, like, it’s, it’s, there’s like the Dunbar number, which I think is like more sort of, you know, faked than, than anything, like at least like the story behind it. But like, it is actually kind of real still, like there’s something around like 150, 200, we’re like, laying the foundation of the culture is very, very important. How many people are you guys trying to hire over the next year? We’ve grown about four acts on an annualized basis in the last 12 months. I think that’s like an unsustainable growth rate on a relative basis. I’ve seen that, you know, you can double the team every year and that’s not super hard to do, but four acts, what you start to see is, you know, cultural drift, you start to see islands of different working styles.

1:13:30 你会开始看到有人做一些非常投机的事情。我一直都会留意工程团队里这种情况。你去找工程师聊聊，问他们：“你在做什么？”结果发现他们在做一些挺边缘的东西，你就会想：“这真的是你应该花时间做的事吗？”所以，增长得非常快是有代价的，你知道，培养和带动新员工也要付出成本。所以我也说不好。我觉得年底时可能会有250人；到明年年底，可能会有……不过你知道，我们会继续增长，大概增长到现在的2.5到3倍。所以很可能明年我们会达到700人左右。大部分应该都是工程师吧。大部分是工程师。我们也在大力扩充销售团队。好。都需要哪些类型的工程师？主要是系统工程师。我的意思是，我们一直都很专注于 Linux、文件系统、可扩展系统、网络和云计算这类领域。所以我们主要想找在这些方面很出色的人。我们也在大量招聘其他类型的人才，因为我们的技术栈往上走了一些，我们在 LMA 推理领域做了很多工作。所以现在我们还要处理另一系列问题，比如 CUDA、GPU、MFU，也就是尽可能提高每块 GPU 能处理的 token 数量，以及很多类似的问题。而且，虽然我认为那个领域的很多工作归根结底还是由系统层面驱动的，因为事实证明，KV 缓存和路由其实仍然是非常典型的系统问题。但对，稍微有一些。1:13:30 You start to see people doing like very speculative things like that. That’s something I always watch out for in the engineering team. You go and talk to the engine and engineer and I’m like, what are you working on? And it turns out they’re like building something like kind of like fringe and like, is this really what you should spend time on? So growing very fast has that cost and you know, it has the cost of ramping people up. So I don’t know. I think by the end of the year, maybe 250, about the end of the next year, you know, but like, you know, we’re going to keep growing like maybe two and a half, three acts. So it’s very likely we’ll end up like next year, like 700 people, something like that. I’m going to assume most of those are engineers. Most engineers. We’re wrapping up sales a lot too. Okay. What kinds of engineers? It’s primarily systems engineers. I mean, we’ve always been like very deep in the sort of, you know, Linux and file systems and scalable systems and networking and cloud and stuff like that. So that’s the core of what we’re looking for. Like people are really good at those things. We’re also looking a lot at like, you know, more like, because we moved up in the stack a little bit, like we’re doing a lot of work in the LMA in front of space. So now we’re dealing with a separate set of concerns around, you know, CUDA and like GPUs and and and MFU, you know, like maxing out the tokens that you can get per GPU and a lot of that things. And, you know, even though I actually like will say a lot of the work in that space, in my opinion, turns out is like more systems driven still, because it turns out like KV caching and routing is actually like, is still like very like systems like like these are very like systems have problems. But yeah, a little bit

1:15:00 更像是 GPU 性能之类的东西。我们正开始组建一支研究团队。我们非常期待进军一些领域，比如后训练、蒸馏，以及为客户或公司内部寻找更好的量化方法。两方面都会做。我觉得这有点像 FDE 的情况。起初，我们会和客户一起做很多工作，优化他们的模型。但随着时间推移，我们得想办法把这些工作转化成产品。可以说，如果每次客户需要优化模型，我们都得去和他们合作，那我们在某种程度上就算失败了。长远来看，这些工作必须转化为产品。因此，无论是 FDE 团队还是研究团队，起初都需要和客户一起做很多工作，我们也确实是这么做的。但随着时间推移，我们也在开发大量内部工具，并逐步打造许多完全自助式的内部产品。我们目前在后训练领域做了很多工作，也在 GPU 性能领域做了很多工作。所以有很多很酷的东西，真的有很多。回到你刚才谈到的产品战略。对，对。你怎么看？未来一两年我们需要什么？我觉得这实在很难说。如果我说“我完全知道我们需要打造什么”，那我觉得自己可能是在自欺欺人。到目前为止，我们看到的情况，以及我更笼统地相信的一点是：我们需要更好的构建模块。这是一个全新的软件时代，我们正在解锁一整类全新的应用，而这些应用几年前还无法构建。1:15:00 more like sort of GPU performance, that kind of stuff. We’re starting to build up a research team. We’re very excited about, you know, moving into stuff around like, you know, post training and distillation and better methods for quantization for your customers or for internally. Both. I think it’s like a little bit like with the FDEs. Like I think we initially will do a lot of stuff with our customers to optimize their models. But over time, figure out how do we turn those into products? I think we’ll sort of arguably fail in a certain way. If every time a customer needs to optimize a model, we have to go and work with them. And, you know, long term, it has to be turned into products. And so with the FDEs and with the research team, initially, you’re going to have to do a lot of work with the customers and we are that. But over time, we’re also building a lot of internal tooling and a lot of over time internal products that are fully self-serve. So we’re working a lot of stuff right now in post training space, but also in the GPU performance space. So a lot of cool stuff. A lot of cool stuff. Back to the product strategy stuff that you were talking about. Yeah. Yeah. Like what are your thoughts? Like what do we need over the next year, two years? I think it’s incredibly hard to say. And if I said like, I know exactly what we need to build, I think I would be like delusional. I think what we’ve seen so far, and what I kind of believe in on a more general level is like, we need better building blocks. Like this is a new era of software and we’re unlocking a whole new type of applications that could not be built a few years ago.

1:16:30 做所有 GNI 相关的事情，把代码写出来、把 LMS 写出来；这些输出的代码需要以隔离的方式运行。要和 GPU、LMS 协同工作，需要许多新的基础原语。到目前为止，我们已经构建了一些基础模块，并因此解锁了过去几年里世界上看到的许多应用。当然，不只是我们做的这些。但我觉得未来几年还会出现更多很酷的东西。我刚才提到了一些方向。比如，我非常看好训练，非常看好快速作业，也非常看好智能体、低延迟之类的某些方向。但说到底，我们认为自己是一家基础设施公司，对吧？我们并不打算构建那些非常狭窄的东西。我们想构建通用性很强的基础模块，并将它们提供给客户。然后客户会把这些基础模块组合起来，构建出一些我们完全无法预料的惊人应用。我想聊聊你发过的一些推文。这是最后一部分。好。我们可以逐条聊聊，也许有些内容你想详细展开。现在有很多长时间运行、持久化的沙箱。这感觉还是一种迁移现有工作负载的做法。我猜沙箱会从开发环境演变成云端的某种东西，变得更短暂；每条命令对应一个环境。这样我就能在两者之间大声喊话，让你暂停、恢复、克隆。是的。我是说，我们聊过很多这个。对，我们也聊到过一些，对吧？我觉得，至少我认为，如今 Modal 里的沙箱中位运行时长大概是 8 秒左右。那真是挺不可思议的。1:16:30 Doing all the GNI stuff, code that, LMS that output code that needs to run like in an isolated way. Like there’s so many like new primitives that are needed to work with GPUs and work with LMS. And we built some building blocks so far and that’s unlocked a bunch of applications that the world has seen over the last few years. Not just stuff with us, of course. But I think over the next few years, there’s going to be so many more cool things. And so I mentioned some stuff. Like I’m very bullish on training. I’m very bullish on fast jobs. I’m very bullish on certain things around like agents and low latency and things like that. But like in the end, we consider ourselves like an infrastructure shop, right? Like we’re not trying to build, you know, very narrow things. Like we try to build like very general purpose building blocks and put them in front of customers. And then those customers will combine those building blocks and build some incredible applications that we could have never anticipated. I want to go through some of your tweets. This will be the last part of it. Okay. And we can just like, maybe you want to double click on anything. A lot of long running persistent sandbox stuff right now. Today feels lift and shift. I suspect sandboxes will evolve from dev box into cloud to something more ephemeral, one environment for every command. And now I’m shouting in between so you can pause, resume, clone. Yeah. I mean, we talked about this a lot. Yeah, we talked a bit about it, right? Like I’m fully, I think, I think the median sound box in modal today is like eight seconds long or something like that. Like, which is kind of incredible

1:18:00，仔细想想确实如此。而且随着时间推移，我觉得它可能还会变得更短。我们也想支持长时间运行的沙盒。我觉得它们很棒，长时间运行的沙盒确实有很多合理的使用场景。但我的愿景是，你知道，世界也需要……我们得想办法大规模并行处理各种任务，同时降低启动和停止沙盒的开销。我不知道我是不是已经运行了一个 IP，启动后就投入了与竞争对手的 catfax 军备竞赛；要是能协调一下，让这场军备竞赛的节奏稍微慢一点就好了。不予置评。GPU 紧缺最大的缺点，不是 GPU 难以获取，而是几乎所有容量都以三年期的方式售出——也就是说，买的是三年的整块容量，而且要到三到六个月后才开始提供。这不符合大多数用户的需求。我觉得我们之前也稍微谈到过这一点。是的。你知道，也许现在可以再深入一点。我觉得 Modal 一直特别擅长的一件事，就是提供完全按需的容量，对吧？比如客户找我们要 100 块 GPU，我们通常可以马上给他们，甚至都不需要跟销售人员沟通。而现实中的很多工作负载并不像这种平坦的、矩形的工作负载；它们会在 24 小时周期内扩缩容，可能会随着时间推移而增长，也可能会有特定的突发时段。在我看来，这是如今很多客户面临的一个挑战。而且我确实认为，这对整个领域来说是个实实在在的问题：获取容量的主要方式就是买下这些矩形容量块。这不太理想。我认为整个世界需要……而从根本上说，我是有偏见的，但这就是为什么我认为 Modal 能在这类事情中发挥很大作用，不管你想怎么……1:18:00 when you think about it. And over time, I think it might get even shorter. We want to support long running sandboxes too. Like I think they’re awesome, like very valid use cases around long running sandboxes. But like my vision is like, you know, the world needs also, we need to figure out this like how can we just massively parallelize things and reducing the overhead of starting and stopping sandboxes? I don’t know if I ran an IP already start up engaged in catfax arms race for the competitor, it would be nice to coordinate and paste arms race a bit. No comment. The biggest downside of GPU scarcity is not that it’s hard to get GPUs, but that almost all capacity is sold as three years as a three year rectangle is starting three to six months in the future. That’s not what demand looks like for most users. I think we talked about that a little bit as well. Yeah. And you know, maybe just go a little bit deeper now. Like one thing I think modal has always been very, very good at is fully on demand capacity, right? Like if a customer comes to us and need 100 GPUs, like we can typically give them that like immediately, you don’t even have to talk to salesperson. And a lot of workloads in real life, like they don’t look like these like flat, you know, rectangular workloads, like they scale up and down throughout the 24 cycle, they maybe grow over time, they maybe have certain burst periods. And to me, that is a challenge with a lot of customers today. Like the, and I do think it’s a legit problem for, for the space is like the only, the prime way to get capacities to, to buy these rectangular blocks. It’s not great. I think the world needs to, and that’s fundamentally, I mean, I’m very biased, but like that’s why I think modal can play a very large role in that sort of, you know, whatever you want

1:19:30 可以说，这是一种时长转换，提供更灵活的容量。我们之所以能做到这一点，是因为我们可以聚合成千上万的客户，需求就会变得非常平稳、非常聚合。所以这正是 Modal 很大一部分价值主张。你觉得这是一个差异化优势吗？比如说，哦，真的吗？好吧。是的，这是一个巨大的差异化优势。事实上，我觉得，如果你想把 Modal 的价值主张非常简化地拆解，可以分成两件事。一是开发者体验，也就是能够不断迭代、构建东西，而不用考虑基础设施。另一件事就是容量方面的东西。我越是思考 Modal，以及我们长远的发展方向，就越觉得我们正在构建、聚合全世界的计算容量。我们非常擅长把它切分得非常精细，因为我们围绕着极快地扩容和缩容做了大量工作，还有各种 GPU 快照之类的技术，以及一些能让这一切快得多的方法。而且这种模式在非常大的规模下运行得特别好，因为你可以聚合很多客户。我每个月都会说，沙箱业务的收入可能适用于这个说法，但它每个月都在增长 30%。这真是有点疯狂。我不明白，你知道吗？每个月我都会想，嗯，沙箱业务增长了很多，但显然已经快到平台期了吧，结果它还在继续增长，甚至增长得更多。所以我，我，我也不知道。很多风投都会问，你怎么知道沙箱会变得这么重要？说实话，我并不知道。我确实觉得它在 2023 年会比实际情况更大一些。但如今沙箱业务占了我们收入的三分之一，这简直不可思议。当然，我们对此非常高兴，而且我们有一支规模很大的团队在投入……1:19:30 to call it, sort of duration transformation of offering more flexible capacity. And we can do that because we can aggregate many, many thousands of customers and the demand gets very smooth and aggregate. So that’s a lot of the value prop of modal is exactly that. Would you say that’s a differentiator? Like the, oh really? Okay. Yeah, massive differentiator. Actually, in fact, like, I think, modals, like if you want to be like very sort of reductive, like you can break down modal’s value prop into two things. What is a developer experience, the ability to like iterate and like build things and not have to think about infrastructure. The other thing is the capacity stuff. The more and more I think about modal and like where we’re going in the long run is like we’re building, like we’re aggregating the world’s compute capacity. And we’re very good at slicing it out like very granularly, because we’ve built all this work around, you know, scaling up and down extremely fast, you know, all this stuff around GPU, snapshotting, working some, some ways to make it much, much faster. And that model works at very large scale, like really, really well, because you can aggregate lots of customers. Every month I keep saying or saying box revenue might apply to it, but it keeps going up 30% every month. It’s kind of nuts. Like I don’t understand, you know, like every month I’m like, well, you know, sandboxes grew a lot, but like clearly the kind of plateau and then they keep growing even more. So I, I, I don’t know. And a lot of VCs, like they’re like, how did you know sandboxes was going to be big? Like, and the truth is like, I didn’t like, like, I thought it was going to be bigger than it was in 2023 for sure. But like the fact that sandboxes is a third of our revenue today, it’s, it’s insane. And, you know, we’re obviously very happy about it. And we have like a very large team investing

1:21:00，关于沙盒。你知道，我不想把这件事说得无足轻重。显然，我确实想好好肯定一下我们自己——早在 2023 年，我们就多少看到了沙盒如今的样子。但我觉得，你知道，我大概从来没敢赌过，沙盒能带来这么高的收入，市场需求也会这么大。模型推理服务商应该独立于模型构建者而存在，这背后有一个有意思的结构性原因。最好的模型一直在变化，而推理服务商可以轻松地在不同模型之间重新分配资源，因为整体需求相当平稳。所以，我的意思是，如果你看看所有中国实验室，以及未来希望能有更多美国实验室，每当一家新实验室发布最新、最强的开源模型时，你都会看到对这些模型的需求大幅转移。我认为，如果这些实验室都各自持有算力资源，那么它们面临的挑战是：推出新模型时，需求会大幅上升；接着另一家实验室推出了更好的模型，需求又会下降。不过，当然，它们也在幕后开发下一个版本。所以我觉得，正如我所说，这存在一种结构性原因，这也解释了为什么你会开始看到需求转向最新、最强的中国实验室。当然，并不是说一切都会在一夜之间转变，因为很多模型、提示词之类的东西都有一定的黏性。但我确实认为，有一个理由：模型的部署很容易调整。比如，今天我们为某个模型运行了 X 个、总计数千块的 GPU；明天我们就需要把其中 50% 转给另一个模型。1:21:00 in sandboxes. You know, I don’t want to like trivialize it. Like we obviously, I do want to give ourselves like a lot of credit for, for seeing sandboxes for what it is today a little bit back in even 2023. But I think, you know, if like, I probably would have never, you know, taken a bet that revenue would be this large from sandboxes and the market demand would be so much. There’s an interesting structural reason why inference providers should exist separately from model builders. The best model keep changing a lot inference providers keep can easily reallocate resources between models since overall demand is quite smooth. So I mean, if you think about all the Chinese labs and, you know, in the future, hopefully more US labs too, every time a new lab releases, you know, the latest and greatest open source models, like you see big shifts in demand, you know, for these models. And so I think a challenge with, you know, if all these labs independently like held capacity, they would see when they launched a new model demand would go up a lot. And then another lab launches a better model and then the demand goes down. You know, but, but of course, like in the background, they’re working on the next version. So like, I think, you know, there’s like a structural reason, like I said, like why you’re going to start, you’re seeing demand kind of shifting to like the latest and greatest Chinese lab. I mean, not like, it’s not like, you know, this is like, everything just moves overnight because, because a lot of models and prompts and stuff like, like a lot of things that get sort of there’s like a certain stickiness to it. But I do think, you know, there’s an argument that it’s very easy for model to like, you know, right, today we’re running, you know, X number of 1000 GPUs for one model. Tomorrow we need to shift like, you know, 50% of that to another model.

1:22:30 没错，就是这样，这正是这个系统擅长处理的事。根本不需要有人告诉我们这么做，它就会神奇地做到。所以你同意，基本上大语言模型已经成了大宗商品？如果你同意，那你觉得大型实验室的生意会走向何方？我不确定这一定是真的，对吧？比如说，泰勒·斯威夫特算是大宗商品吗？她发行一张大热专辑，然后去巡演，赚很多钱。接着，你知道，她会有一段空档期，去录制些什么，我不知道，这可能不是个最好的类比。然后她又发行一张新专辑，你知道，所以我不知道，她仍然有很强的品牌影响力，也拥有很大的号召力，而且能赚取很高的利润。所以，只要这些实验室能不断推出新模型，而且这些模型的利润足以让它们逐步收回训练模型的成本，我觉得这仍然可能是一门不错的生意。所以你不觉得智能会停滞不前吗？不，我不这么认为。我是说，这还有待观察，对吧？如今的模型非常擅长做那些能通过强化学习高效训练的事情。我们一开始在编程和编写简单代码片段方面就看到了这一点；随着时间推移，它们也变得非常擅长编写可用于生产环境的代码。它们在数学方面也很厉害，在网络安全方面也很厉害。这些都是相当容易验证的事情。我仍然觉得还有待观察：我们的模型能不能在做饭之类的事情上表现得非常出色，或者我不知道，那些我更……1:22:30 Sure, that’s just like, that’s what the system is built to handle. No one even needs to tell us to do that. It just magically does that. So you agree that basically the LLMs are commodities? And if you agree, then where do you think the business for the big labs is going to be? I’m not sure if that’s necessarily true, right? Like, is Taylor Swift the commodities? Like, she releases a big album and goes on a tour and makes a lot of money. And then, you know, then there’s like a down the air when she goes and like records, I don’t know, this is best analogy. And then she releases a new album, you know, so I don’t know, like she still has like a lot of brand and there’s a lot of, you know, power and like makes a lot of margin. So like, as long as the labs like repeatedly can launch new models, and as long as the margin that they have on those models are such that they can sort of pay off amortized like that the cost of training those models, it could still be like a decent business, I think. So you don’t think that the intelligence is like plateauing? No, I don’t think so. I mean, I think it remains to be seen, right? Like, models are today are like very good at like doing things that can be trained very efficiently, you know, using reinforcement learning. So we’ve seen that initially with, you know, coding and writing simple snippets over time, it’s gotten really good at also writing production like level code. It’s gotten really good at math. It’s gotten really good at cybersecurity. Like these are all like very quite verifiable things. I still think it remains to be seen, you know, our model is going to be really good at whatever, cooking food or I don’t know, like things that I have like more

1:24:00 像目标比较模糊、反馈周期可能更长之类的领域，我觉得人类在相当长一段时间内可能会表现得相当不错，也能和模型竞争。不过你知道，当然，AI 模型确实越来越厉害了，我相信大家都认同。是啊，显然，模型在编程方面越来越强了，它们现在已经很擅长编程了。你觉得还有哪些目前可能还没那么热门的应用，会在未来几年变得非常重要？是指大语言模型吗？我不确定，不过我觉得任何具有某种可验证计算流程的领域都可能是这样，比如计算生物学，我觉得你刚才提到了。对，计算生物学。我觉得这些领域并不是完全可验证，但确实有一些方法可以进行模拟：比如你可以用大语言模型生成预测，然后用计算方法来模拟分子动力学，获得某种反馈。我觉得材料科学也可能是另一个领域，你能在其中形成一定的反馈闭环：比如合成制造某些材料，测量它们的性质，再让智能体参与其中，开展一些工作。所以，我有物理学背景，对这些领域很感兴趣。药物、材料，还有什么？我一时想不起来了，但肯定还有很多其他领域。很好。我想问的问题都问完了。还有什么是你想聊、但我没问到的吗？没有了。这样很好，聊得很开心。太棒了，谢谢。1:24:00 like diffuse targets, maybe longer feedback loops or whatever, I think humans probably like my feeling is like will be quite good and quite competitive with models for a long time in those domains. But you know, certainly, you know, AI models, they’re getting very good as I’m sure everyone can agree with. Yeah, so obviously, like the models are getting really good at coding, they’re already really good at coding. What other applications do you think maybe that aren’t as big right now are going to become huge over the next few years? For the LLMs? I don’t know, but I think anything that has like kind of a verifiable computational bio, I think you mentioned. Yeah, computational bio, I think those are like, not exactly verifiable, but there’s like certain ways you can sort of run simulate, like you can use an LLM to output predictions, then you can do like computational like, like methods to like simulate molecular dynamics and get like kind of feedback loops. I mean, like I think like material science is like maybe another area where like you have a little bit of that loop where you can sort of, you know, synthetically manufacture certain materials and measure the properties and use kind of put an agent in the loop and like, you know, do things. So, you know, I have a physics background, I’m pretty excited about that. So drugs, materials, what else? I’m drawing a blank, but I’m sure there’s many other things. Cool. I mean, those are all the questions that I had. Is there anything you wanted to talk about that I haven’t? I’m good. This is great. This is a lot of fun. Awesome. Thank you.