读见

Satya Nadella:用零信任约束前沿模型,让企业掌握自己的智能

原题Satya Nadella: Whether to Trust Frontier Models | Why Companies Need to Own Their Own Intelligence

约 19 分钟原文英文已翻译

视频由 YouTube 提供

看前速览

5 个要点

Satya Nadella 主张用模型之外的零信任系统约束能力增强却未充分对齐的模型,并让企业保有自身业务积累的智能。

  1. 在对齐尚未解决时约束智能体

    身份、受保护的日志、明确策略与行为监控在模型之外形成安全边界,这与模型本身实现对齐是不同问题。

  2. 推理透明有用,但不能保证可靠

    Nadella 承认思维链可能不忠实,主张获得记录并借助多个模型检查,而非将可见推理视为可靠性保证。

  3. 私有评测帮助企业保留积累的知识

    上下文、反馈和执行轨迹可充实企业自己的基准,再据此比较不同模型是否满足自身目标。

  4. 模型选择与实施成为业务决策

    按任务路由和使用低价模型可以降低成本,而重构工作流程同时需要技术与业务理解。

  5. 基础设施投入需要经济需求支撑

    电力和芯片产能建设周期长,Nadella 将可持续训练投入与收入及生产率联系起来,没有假定支出可以无限增长。

完整摘要

Satya Nadella 将 AI 对齐与当下部署智能体所需的安全系统分开讨论。他不再认为扩大模型规模就会自然带来对齐,主张通过智能体身份、不可随意篡改的活动日志、策略和行为监控建立外部约束。思维链透明度有助于审计,但他也承认推理记录可能并不忠实。在模型市场上,他预计前沿能力仍有价值,而大量常规调用会面临开放与封闭模型之间的价格竞争。他的核心商业判断是,企业应保留自身上下文、反馈、执行轨迹和评测积累的知识价值,而非全部让渡给模型提供方。私有评测基准可能成为核心知识产权,实施工作也会结合业务重构与技术咨询。他还将巨额训练预算与收入、电力、芯片供给及全球经济增长联系起来。这些属于战略判断和企业案例,不能视为对齐、安全或宏观收益已经得到解决的证明。

章节

文字稿

查看原文

简要说明Brief Description

萨提亚·纳德拉探讨了模型对齐、面向智能体的零信任遏制机制、开放权重与许可权重、企业的私有评估与情报,以及人工智能发展所需的基础设施和经济条件。Satya Nadella discusses model alignment, zero-trust containment for agents, open and licensed weights, firms’ private evaluations and intelligence, and the infrastructure and economic conditions needed for AI growth.

目录Table of Contents

  • 简介
    Intro
  • 为什么萨提亚说我们应该假设 AI 模型已遭入侵
    Why Satya Says We Should Assume AI Models Are Compromised
  • 为什么 AI 需要“零信任”架构
    Why AI Needs a “Zero Trust” Architecture
  • 为什么思维链并不可靠
    Why Chain of Thought Is Not Reliable
  • 开放权重模型会因为更加透明而胜出吗?
    Will Open-Weight Models Win Because They’re More Transparent?
  • 开放模型与闭源模型:AI 流量将流向何处?
    Open vs Closed Models: Where Will AI Traffic Go?
  • 为什么每家公司都需要拥有自己的智能能力
    Why Every Company Needs to Own Its Own Intelligence
  • 为什么私有评测可能成为公司最有价值的知识产权
    Why Private Evals Could Become a Company’s Most Valuable IP
  • 为什么 AI 将催生新的实施经济
    Why AI Will Create a New Implementation Economy
  • AI 是否需要全栈垂直整合?
    Does AI Require Full-Stack Vertical Integration?
  • 为什么推理需求远超所有人的预期
    Why Inference Demand Is Bigger Than Anyone Expected
  • 全球能否支撑耗资 1000 亿美元的 AI 训练任务?
    Can the World Support $100BN AI Training Runs?
  • 人工智能能否显著加速全球 GDP 增长?
    Can AI Meaningfully Accelerate Global GDP Growth?
  • 哪些行业将从人工智能中获得最大的生产力提升?
    Which Industries Will See the Biggest AI Productivity Gains?
  • 为什么萨提亚不再相信“对齐”会随着规模扩大而“自然而然地涌现”
    Why Satya No Longer Believes Alignment Will “Just Fall Out” of Scaling
  • 为什么每家公司都可能成为 AI 基准测试 ҡар
    Why Every Company Could Become an AI Benchmark

简介Intro

  • 萨提亚对未来十年最期待的事;Daario、Ilia、Sam、Greg 和 Elon 当时我想都像一个幸福的大家庭。他们都是真正相信规模定律的人,这也是我在 2018、2019 年时所相信的。我想,我现在最大的变化是:我过去总觉得,对齐问题会自然而然地得到解决。
    What Satya Is Most Excited About for the Next Decade and Daario and Ilia and Sam and Greg and Elon were all I guess one happy family. They were really the believers in scaling laws and that's what I believed in back in 2018 2019. I think the biggest change now I have is I used to always think that alignment will just fall out.

我已经不再这么认为了。说实话,现在这档节目对我来说非常触动情感。我在伦敦自己的卧室里创办了 20 VC。I no longer believe that. Now this show honestly is quite an emotional one for me. I started 20 VC from my bedroom in London.

我当时18岁,身无分文,通讯录里也没有任何联系人。I was 18 years old. I had no money. I had no contacts in my phone book.

今天,我有机会与微软首席执行官萨提亚·纳德拉坐下来交谈。微软是过去三四十年来最伟大的公司之一。关于萨提亚,我想说的是,他能在这么短的时间里为这次访谈注入如此温暖的氛围,这几乎是我前所未见的。他的论述和想法清晰明了,而且能够迅速深入探讨问题。And today I have the chance to sit down with Satcha Nadella, CEO of Microsoft, one of the greatest companies of the last three or four decades. And I will just say on Satcha, the warmth that he was able to imbue in this interview in such a short space of time was almost unlike anything I've seen before. The clarity of his arguments and his ideas, his ability to go very deep, very quickly.

这几乎与我以前见过的一切都截然不同。那是一场我会永远铭记的演出,我真心希望你也喜欢。准备出发。It was almost unlike anything I've ever seen before. It was a show I'll always remember, and I really hope you enjoy it. Ready to go.

Satcha,非常感谢你和我们一起参与这次采访。我本来想说,感谢你来到演播室,不过实际上是你在这里接待我们。所以,真的很感谢你抽出时间。Satcha, thank you so much for doing this with us. I would say, you know, thank you for coming to the studio, but you're actually hosting us here. So, really appreciate you taking the time.

哦,真的非常感谢你邀请我上节目,哈里。这件事我们期待很久了,我很高兴我们终于能成行。Oh, like thank you so much for having me on the show, Harry. It's been a long time coming, but I'm finally glad we were able to do it.

为什么萨提亚说我们应该假设 AI 模型已遭入侵Why Satya Says We Should Assume AI Models Are Compromised

你大约十天前写了一篇具有开创性的文章,提到我们应该假设模型已经遭到入侵,我想就此进一步展开。那是否意味着我们不能或不应该依赖模型提供商公司的披露信息?Now you wrote a seminal piece 10 days ago or so and you said that we should assume a model is compromised and I just wanted to expand on that. Does that mean that we can't or shouldn't rely on the disclosures of the model provider companies?

不,我觉得我现在看到的挑战是,模型正变得越来越强大,对吧?缩放定律确实有效,但我曾有一个假设:随着规模扩大,对齐问题会自然而然地得到解决;但现在看来并非如此。而且还有更多科学挑战需要应对,对吧?机制层面的理解也仍然遥不可及,所以……No, I think the challenge the way I see it right now is the models are becoming more and more capable, right? scaling laws are working except one of the assumptions I had was that you'll have alignment just fall out of scaling and it doesn't seem to be and there's more science challenges there that need to be tackled right and mechanistic understanding is also elusive and so

为什么 AI 需要“零信任”架构Why AI Needs a “Zero Trust” Architecture

鉴于此,我认为,如果你想继续使用这些复杂模型,最好的做法就是把这些模型纳入一种信任架构中。毕竟,零信任并不是什么新概念,对吧?我们早就有零信任架构方法了。所以,至少我提出的主张是:让我们创建一个经过工程化设计的零信任环境,在任何出现故障的智能体造成损害之前将其隔离。对吧?given that I think the best posture to have if you want to continue to use these sophisticated models is to treat these models and a trust architecture I mean after all zero trust is not something new right we've had zero trust architectural approach approaches. So the claim at least I'm making is let's create an engineered zero trust environment that can contain any malfunctioning agent before it does any harm. Right?

所以这就是关键所在:如果你真的想拥有能力更强、但对齐程度更低的智能体,就需要有一个值得信赖的信任体系。So that's the point because if you really want to have agents that are more capable and yet less aligned then you need to have a trust system that you can rely on.

我能问一下,这个信任系统是什么样的吗?它的架构由谁设计?谁来担任仲裁者?Can I ask what does that trust system look like? Who's the architecture of it? Who's the arbiter of it?

这是个很好的观点。所以我看待这件事时会说,先把 AI 安全放在一边,对吧?比如说对齐,至少在我看来,主要还是要体现在训练过程中,对吧?It's a great point. And so the that's why I kind of look at it and say you leave AI safety outside of this, right? Because that alignment for example, at least my take is that it's it's really mostly has to show up in training, right?

事实上,也许即使在预训练阶段,我也不认为你能单靠强化学习实现对齐。你必须让对齐真正从一开始就嵌入权重之中。所以这关乎模型本身。希望它在某个时候能得到一条界线。In fact, maybe even in pre-training, I don't think you can even RL your way to alignment. You have to truly get alignment to be something that truly gets embedded in the weights right at the very beginning of it. And so that's the model itself. hopefully at some point will get a line.

所以,如果把 AI 安全排除在外,信任架构就会变成那种——你知道的,像微软这样的公司或所有网络安全从业者所做的那一套。So if you leave AI safety then the trust architecture is going to be just you know what a company like Microsoft does or all the cyber folks do.

比如说,我们做的就是构建了这两个东西,对吧?Agent 365 和 Perception。它们有什么作用?它们会为每个智能体分配一个身份,然后记录它的所有活动、所有工具调用,实际上还有每一次模型调用,以及其中的代码。事实上,我也要求所有代码都最好用英文编写,而且不要放到新版本里;然后你可以用多个模型进行检查。它必须是防篡改的,对吧?不能让智能体修改自己的日志。然后,在此基础上制定策略,再对智能体的活动进行行为监控,确认它们符合策略。这就是我们的架构。我们现在还没有这样的架构。So what we have done for example is we built these two things right agent 365 and perception right what does it do it gives every agent an identity it then logs all of its activity all its tool calls in fact every model call and the coot in it in fact that's where I'm also asking that all the coot better be in English and not in new release and then you can check using multiple models it better be tamperproof right you can't have the agent go modify its own u logs and Then on that you set policy and then you behaviorally monitor the activity of the agents to the confirm to the policy. So that's the architecture. We don't have that architecture today.

这只是象征着我们目前所处的阶段吗?Is that just symbolic of where we are in the process?

这种架构确实存在。所以我认为,人们会把两件事混为一谈:我姑且称之为一种经过设计的安全零信任架构体系,用于遏制风险;以及 AI 安全。对吧?因为我觉得人们会说:“AI 安全很难、难以捉摸,而且至今尚未实现,所以我们在安全方面肯定什么也做不了。”我并不是说这两件事必须同时到位。事实上,我只想说,我们必须应对眼下的情况,所以不妨在模型之外构建一个信任体系,然后积极主动地制定策略,并监控是否出现偏离策略的情况。That architecture exists. So there there are two I think people conflate what I'll call an engineered security zero trust architectural system for containment and AI safety right because I think people say well AI safety is hard and elusive and not yet achieved so therefore there must be nothing we can do on security and I'm not saying that these two things are all got to come together at the same time in fact I would just say hey we have to deal with the situation as it exists so let's build a trust system outside in fact of the model and then just be super aggressive in setting policy and monitoring for policy deviation

为什么思维链并不可靠Why Chain of Thought Is Not Reliable

在监控方面。on the monitoring side.

如果我们考虑思维链,它有多可靠?If we think about chain of thought, how reliable is that?

这不对。我们知道,思维链本身可能并不忠实,而且,而且事实上,当模型公司说“嗯,它并不忠实,我要把它隐藏起来”时,情况可能会变得更糟。It's not right. We know that the chain of thought itself can be unfaithful and and in fact it can be made worse when the model company says, "Well, it's unfaithful and I'm going to hide it."

所以这就变得更加棘手了。我认为,这正是推动标准化的一部分原因;至少应该让可信合作伙伴能够直接获取代码,这样就可以检查、审计,也可以用多个模型进行验证,等等。坦白说,这实际上可能会推动更多开放权重模型的发展,因为如果你不能信任它们——说来也奇怪,闭源模型的能力更强——但如果它们没有意识到需要提高透明度,尤其是在可信合作伙伴使用它们时,那我们可能就有麻烦了。and so it becomes even more problematic and that's where I think a little bit of the push for standardization and the coot at least for trusted partners being available as is so that it's inspectable it's auditable and I can check using multiple models and what have you right so that's going to be one of the in fact that's one of the things that may even drive more open weights quite frankly because if you can't tr it's funny close rate models are more powerful But if they kind of don't recognize the need to be more transparent especially as their trusted partners use it then we may have a problem.

开放权重模型会因为更透明而胜出吗?Will Open-Weight Models Win Because They’re More Transparent?

我想问一下,谈到开源的兴起时,我完全同意你的看法:闭源模型更强大,而开源模型主要是中国模型。那么,考虑到安全性是在预训练阶段融入的,再考虑到我们刚才提到的信息披露问题,而这又与开源模型大多来自中国这一点形成对照,这会不会令人担忧?是的,我的意思是,我认为这里有两方面的问题。我确实认为,退一步来看,AI 安全缺位对中国来说并不是件好事。Can I ask you when we think about that rise of open source I completely agree with you in terms of clothes being more powerful open source is predominantly Chinese models is that a concern then when we think about safety being baked into pre-training when we think about disclosures as we said there and then juxtaposed with the majority of open source being Chinese. Yeah, I mean I think there two sets of things. I do believe if you sort of step back somehow AI safety not being there is not a good thing for China.

这对美国来说不是好事,对英国来说也不是好事。所以我认为,每个人心里都有一部分是这么想的,因为这并不是说某个未对齐的中国开放权重模型就一定会攻击中国。所以你不能放心地认为它只会被用作攻击载体。It's not a good thing for the United States is not a good thing for the UK. So I think that there's a part of everyone because this is not something that somehow an unaligned openweight Chinese model will probably attack even China. So it's not like you can rest think that this will only be used as an attack vector.

暂且把这点放在一边,我对一些进展还是挺满意的,甚至西方的开放权重模型也在不断进步。而且,我知道,Jensen 显然也在为此努力,其他人也在做这件事。So setting that aside, I'm pretty pleased with some of the progress even the open weight models in the west are making and I'm you know obviously you know Jensen's working on it, the other folks are working on it.

我们甚至会有不开放权重的 MAI 模型。我想我们会更多地把它称为“授权权重”。可以这么理解。we will have even the MAI models that will be not open weight. We'll talk I think of it more as licensed weights. Think of it that way.

我想把这做成一家简单直接的软件公司,而不是一家依赖网络效应的智能业务公司。I want to turn this into just a simple straightforward software business as opposed to a network effects intelligence business.

开放模型与封闭模型:AI 流量将流向何处?Open vs Closed Models: Where Will AI Traffic Go?

所以我认为会出现不同的商业模式,而最终,决定哪些模式会被采用的,将是我对你运用和掌控这些模式的能力的信任。So I think there will be different business models and I ultimately trust in your ability to use those models and control those models is what's going to determine what gets used.

作为一名 Vantra 投资者,我必须预测今天的流量会流向开放还是封闭的平台。考虑流量和收入这两个不同维度,如果展望未来 24 个月,你认为有多少比例的流量会通过开放平台,而有多少会通过封闭平台?As a vantra investor I have to predict where traffic will go today between closed open. When you think about the two different dimensions which is like traffic and dollars if we look forward say 24 months what percent of traffic goes through open versus closed do you think?

这是个很好的问题。我觉得吧,这取决于前沿模型发展到了什么程度,对吧?我觉得大多数东西似乎最终都会趋于饱和;一旦饱和,开放权重模型,甚至只是更便宜的模型——哪怕是闭源模型——似乎也能胜任。尤其是,如果你能用自己的执行轨迹来训练,那么只要问题本身没有变化,你确实不需要最新的模型,也能解决同一个问题。That's a great question. I you know I think it depends on what the frontier looks like, right? Which is I think most things seem to get saturated and when it gets saturated open weights or or even just cheaper models of even closed weights seems to work because especially one of the things is if you can train on your own traces then you really don't need the latest model to be able to solve that same problem if the problem has not changed.

对吧?所以,如果你知道自己有一个想持续推进的评测,那么你可能愿意付费。不过顺便说一句,我对这个 hydroofusion 的东西非常兴奋,我们现在在 GitHub 上有了它。你知道,我刚把这个卖掉了,是的,因为它很酷的一点是,它现在是一个学习模型……总之,我们现在正进入这些智能体蜂群和多个子智能体的阶段,所以我可以调用很多子智能体,其中甚至有些是本地模型。因此我猜,前沿模型市场会一直存在,也总会有一些新东西。科学或许会是下一个设定前沿的领域。但我认为,绝大多数 token 使用都会是商品化的 token 使用。Right? So if you know if you have an eval that you want to continuously keep moving forward then you may want to pay but even there by the way I'm very thrilled about this hydroofusion stuff we now have in GitHub where you know I just sold this yeah because the cool thing there is it's a learn model it now it's anyway we are now going into these agent swarms and multi- multiple sub aents so I can call a lot of sub aents that even happen to be local models and so I think my suspicion is that there will always be a market for frontier. there will always be some new things. Science may be the next thing that you know frontier is being set at. but I think predominantly token usage will be commodity token usage.

这是否意味着它主要是开源的?Does that mean it's open source predominantly?

它可能是开放权重的,或者像我说的,它……它也可以是价格低廉的闭源方案,对吧?因为会有很多价格竞争。我这么说吧。对吧?It may be it may be open weights or as I said it does it has it can also be cheap closed rates right because there will be lots of price competition. Let me put it that way. Right?

因为如果你看看我自己在 copilot 中的使用情况,我们会大量使用价格更低的闭源模型,以及开放权重模型。Because if you look at my own usage inside a copilot, we use a lot of close- source cheaper models along with open rate models.

你提到了拥有自己的智能对你的一些优秀客户来说有多重要。你当时谈到了一些英国的客户。You said about kind of the importance of owning your own intelligence for some of your amazing customers that you have. You were talking about some in the UK.

为什么每家公司都需要拥有自己的智能能力Why Every Company Needs to Own Its Own Intelligence

这就是我们未来将拥有的模型环境吗——为拥有自身数据的公司提供专业化智能?Is that the model environment that we have moving forward specialized intelligence for companies with their own data?

是的。所以我觉得,对我来说,这确实是最重要的问题之一:在智能时代,一家公司的未来会是什么?因为即使回到哈耶克,他对经济的根本描述也是:局部智能,以及默会知识,分布在整个经济体中。没有这一点,就不可能只有一家公司,也不可能只有一种智能。所以这可以说是基础。事实上,资本主义之所以能够运作,价格发现之所以能够实现,定价机制之所以能够发挥作用,都是因为这一点。因此,我认为,重要的是每家公司都能借助 AI,以越来越大的规模积累知识,而不是让知识泄漏出去。Yeah. So it's I think this is to me one of the most important things right which is what's the future of a firm in a era of intelligence right because if you go back even to Hayek his fundamental description of a economy is local intelligence tacet knowledge is distributed throughout the economy right without that you know there can't be only one firm and it can't be one intelligence and so that's sort of the foundational thing in fact capitalism works because of that price discovery works because of that or pricing mechanism works because of that. So therefore I think it'll be important for every firm to be able to accumulate knowledge at increasing scales using AI versus leak.

这就是为什么我把它称为反向信息悖论,对吧?你知道,肯尼思·阿罗大致描述过这样一个信息悖论:如果你是卖家,在没有专利、商业秘密保护或版权保护的情况下披露秘密,就会泄露它的价值。That's why I called it the reverse information paradox, right? Which is you know the you know Kenneth Arrow kind of described this paradox of information where if you're a seller and you disclose the secret without a patent or a trade secret protection or copyright protection you leak the value.

现在,买方第一次需要这种保护了,因为买方——尽管大多数人会把它看作“哦,这是我的数据”,但它甚至是我使用它时产生的数据尾气,对吧?点赞、点踩这样的信号,比如谁喜欢什么——你是在为模型公司创造价值,还是把这种价值留给自己?所以我认为,至少我认同的一点是,要让这种关系保持稳定,我们需要让更多由智能应用所创造的剩余价值留在……Now for the first time the buyer needs the protection because the buyer even it's most people think of it as oh my data it's even my exhaust of using it right thumbs up thumbs down signal like who like who you're creating value for the model company versus capturing that value for yourself so I think one of the things that at least I ascribe is that we need to have for this to be stable more of that surplus that gets created by use of intelligence to remain at the

私有评测为何可能成为公司最有价值的知识产权Why Private Evals Could Become a Company’s Most Valuable IP

公司。而且顺便说一句,这不一定只能通过嵌入你的知识和权重来实现,也可以放在上下文中。firm. And by the way, it doesn't need to be only by embedding your knowledge and weights. It can be in context.

它可以放在你的评测中,对吧?你可能会有一个基准测试。事实上,我非常看好的一件事是:最终每家公司都会拥有一个私有评测基准,它将成为公司的核心知识产权。每个模型都会接入进来,尝试在这个评测上不断优化。It can be in your evals, right? You may have a benchmark. In fact, one of the things that I'm very gung-ho about is we all become every firm will have a private eval benchmark which is their core IP. and every model comes in and tries to sort of hill climb on that eval.

在我看来,这就是一家公司的运作方式。我很喜欢这种为每家公司建立私有评测基准的做法。不过,这似乎只适用于一类非常特殊的公司,比如 Harvey、Lora 这类公司,或者那些极具前瞻性的 AI 公司。And that to me is how a firm works. I love this private evals benchmarks for each company. This is for a rarified era of companies though it would seem your Harveys or your Loras or the kind of very forwardthinking AI companies.

这会普及到全球每一家公司吗?你提到的那个——是的,这种事总是会发生,对吧?你知道,我记得在80年代末、90年代初,有那么一段时间,只有少数公司部署了个人电脑,并用它们做预测,这简直让人觉得不可思议。他们可以把数据放进 Excel 电子表格,发邮件,而不用传真机和公司内部备忘录,当时感觉真是太棒了。那时人们会想,这种情况什么时候才会普及?大概花了10年吧,才慢慢推广开来。现在也差不多,人们会说“哦,我是一家 AI 原生公司,我会这么做”。我觉得,5年后这大概就会显得过时了。我想,这就是技术周期一贯的发展方式,对吧?云计算也是如此。我记得云计算刚兴起时,大家最关心的是“谁是云原生,谁不是”。你知道,现在人人都是云原生了,就连劳合社——它可是有500年历史了——也采用了云计算。所以这种事就是会发生。我们会进入……Will that disperse to every company in the world? You mentioned your Yeah, it happens har always right which is you know they were there was a day when I remember in the late 80s and the early 90s that the few companies that had deployed PCs and were doing for forecasting which felt magical. They were able to put it in Excel spreadsheets, send it in email and avoid the fax machine and inter office memos and it felt like oh wow when will this happen and it took 10 years maybe and it spread out throughout this time around what is considered oh I'm an AI native company and I do this I think you know 5 years from now it'll be pretty pass I would imagine and then that's kind of what I mean these tech cycles always start like that right which is even cloud I remember in the beginning of cloud it was like the big deal was oh who's cloud native and who's not and you know if you're not cloud native and now everybody is right even Lloyd's you know which is what 500 years old is cloud native and so it happens do we move into

为什么人工智能将催生一个全新的落地经济Why AI Will Create a New Implementation Economy

那么,接下来会出现一个实施服务经济吗?因为当我们想到这些客户时,正如你所说,你们的一些超大型客户需要大量全职员工支持,远远超过过去十年传统做法所需的支持。那么,我们是否会迎来一个规模庞大的实施服务经济,帮助现有客户完成转型? 是的,这个问题很有意思,对吧?系统集成商一直都存在。你知道,在许多其他技术扩散过程中,我们都依赖实施服务生态。所以我认为,这一次的情况有所不同,因为它结合了技术和业务。如果非要说它有点像什么,那它既不是传统的管理咨询,也不是传统的系统集成,更不是传统意义上的技术支持。它有点像……举个例子,你看,当我回顾自己的 FD 项目时,就拿我今天在主题演讲中提到的 UniDeliver 为例吧。这很有意思。他们构建了一个智能体系统,用来实现订单到收款流程的自动化。an implementation economy then because when we think about these customers like as you said some of your very big customers they will need heavy FTE support much more so than they have done in traditional years in the last 10 years will we have a very heavy implementation economy to transition existing customers into this yeah this it's it's interesting right the implementation economy systems integrators have always existed right you know we've depended on implementation economy for lots of other technology diffusions so this one I think has different contours right because it's the combination of tech and business if I could say it's kind but it's not classic management consulting it's not classic systems integration and it's not classic sort of technology support it's sort of a like for example think about this right you want to start like when I look at my FD engagements right even take the uni deliver example I talked about at the keynote today. It's fascinating. They've built an agentic system to say, "Hey, we're going to really have order to cache automated using an agentic system.

另外,他们会处理理赔。顺便说一下,他们一开始使用前沿模型,然后再换成——实际上,我认为他们现在所有的理赔都在使用 MAI 模型,因为它成本较低,而且他们还能控制它,对吧?因为他们可以用自己的轨迹数据来训练它。所以我们有一些 FTEEs,但部署出去的有很多,而现在他们自己的 IT 团队,你知道,已经在管理这一切了。不过一开始要考虑的是:业务是什么?评估方式是什么?该如何构想这种智能体架构?所以这算是业务咨询与技术咨询的结合。我想,是的,你说得对,生态系统中会逐渐形成一套新的技能。如今我想我们所有人都可以说各自有自己的 FTEEs,但 FTEES 会变得像传统的系统集成一样,成为新型的 FTE。Plus, they trade claims." and by the way, they start at the using a frontier model and then they substitute for in fact, I think they're using currently an MAI model for all their trade claims because it's sort of a lower cost and it's by the way they control it, right? Because they're able to train on their traces. and so we had some FTEEs but a lot of who got deployed and now their IT team itself is you know managing it all. but it started with what's the business what's the eval what's the way to conceptualize the agentic architecture so it's sort of business consulting combined with technical consulting and I think yeah you're right in saying that there will be a new set of skills that'll get built in the ecosystem today I think all of us have quote unquote our own FTEEs but FTEES will become like you know the traditional systems integration will become new FTE

AI 是否需要全栈垂直整合?Does AI Require Full-Stack Vertical Integration?

你显然提到了 MIA,MIA,天哪。不过,当我们观察许多不同的参与者时,会意识到你需要具备从芯片、模型到应用的全栈能力。you've mentioned obviously MIA a MIA gosh. but when we look at a lot of different players, it's kind of this realization that you need to be full stack from chips to models to application.

那就是未来吗?你必须拥有垂直整合的所有权。这是什么意思?Is that the future? You have to have vertical ownership. And what does that mean?

是的,这完全取决于行业结构,对吧?我觉得,今天可能会有五六个开放权重模型和五六个闭源模型在竞争,谁知道呢。所以,从某种程度上说,如果你在构建一个像智能体系统这样的应用,甚至可以采用我刚才说的那种方案:在路由器中内置一个决策引擎,然后针对特定任务路由到最合适的模型。这是一种可能性。当然,始终存在这样的风险:NEMIC 可能会成为一个垂直整合的模型和应用层,并一路延伸到芯片和系统。我觉得这种可能性不大,对吧?Yeah, it all depends on the industry structure, right? you know I think that I would say today you know there would be who knows five six openw weight you know closed models all competing maybe five six you know openweight models all competing and so therefore at some level if you're building an application like aentic system you could even have something like what I said which is a router with even a decision engine built into it and then you just route to the best model for a given task. So that's a possibility. of course there's always the risk that NEMIC could be one vertically integrated model and app app tier and the and the all the way to the silicon and system. It's unlikely to me, right?

我曾一度觉得,可能会出现一家这样的公司。但事实恰恰相反:每天醒来,你都会发现,各个技术栈层面上的竞争都更加激烈了,无论是在芯片领域,还是在管理芯片的系统领域。I thought at some point that it might be the case that there will be one. It's if anything it's going the other way, which is every day you wake up and you say, "Wow, there's more competition in every layer of the stack, whether it's in silicon, whether it's in systems to manage the silicon."

在模型层,在应用层。所以我觉得这是一个不错的发展方向,不过谁也说不准,对吧?因为这类事情完全取决于哪种设计最终成为主流。但我希望它能发展成一种横向的专业化格局,让每一层都有多个赢家,同时也让人们能够自由组合。in the model layer, in the app layer. And so I think that's a healthy sort of direction to travel, but you never know, right? Because this is one of those things where it all depends on what becomes the dominant design. but my hope is that this becomes a little bit of a horizontal specialization with multiple winners in each layer and then the ability for people to compose.

为什么推理需求远超所有人的预期Why Inference Demand Is Bigger Than Anyone Expected

我当时没看清的是每一层的深度。比如我现在才明白、真希望12个月前就知道的一点,就是推理需求是无限的。对,这个观点很棒。The thing I didn't see is the depth of each layer. Like what I know now that I wish I'd known 12 months ago is just like the infinite demand for inference. Yeah, it's a great point.

世界能支撑规模达 1000 亿美元的 AI 训练任务吗?Can the World Support $100BN AI Training Runs?

是的。Yeah.

而且我真希望自己12个月前就知道这一点。那样我就会采取一些截然不同的投资策略。说到这个,我们听穆斯塔法·苏莱曼说,规模达1000亿美元的训练轮次即将到来,而只有少数几家实验室负担得起。And I wish I'd known it 12 months ago. I would have done some very different investing. Speaking of that, we've had like Mustafa Sleimman saying training rounds of 100 billion are coming and only a handful of labs can pay for that.

千亿级的训练运行。我们是否有足够的能源、基础设施和电力,来支撑整个技术栈达到这种规模?尤其是当我们……是啊,我的意思是,我的看法是,最终这必须建立在全球 GDP 开始加速增长的基础上,然后这才能让某些人把一定比例的资金投入研发,对吧?因为有人说到训练运行时,它其实只是研发支出的一部分,对吧?像那些研发投入很大的公司,研发支出大概占多少?15%、20%吧。所以,是的,到了某个阶段,如果你的收入足够多,就能在训练上投入足够多;当然,如果你对全球经济增长有所贡献,你的收入也会足够多,这样你就能分到应得的份额,可以这么说。Hundred billion training runs. Do we have the energy, infrastructure and power to enable this scale across the stack, but especially when we Yeah. I mean the way I come at it is ultimately it has to come from the world GDP inflecting and then that allowing someone to then have as a percentage of R&D right because when somebody says training run it's just a percentage of R&D right sort of companies that are heavy R&D spend what 15 20% you know on R&D and so therefore yeah at some point if you have enough revenue you can spend enough on training and then of course you'll have enough revenue if you're contributing to the world economy growing so that you can get your fair share so to speak.

然后你也提到了另一面,也就是供应端:我们面临电力短缺、硅短缺,今天还有各种各样的问题。它们都是交付周期很长的项目,对吧?你不可能打个响指就增加发电能力,也不可能马上建成一座新的晶圆厂。And then you also bring up the other side which is the supply side which is we have power shortages, we have silicon shortages, we have you know all kinds of things today. They will all they're long lead items, right? You just can't you know snap your finger and have power generation or you can't have a new fab.

所以现在有意思的是,鉴于需求如此旺盛,大家都在建设。那么问题是,这些东西什么时候能投入使用?供需情况会怎样?说实话,没人知道。今天我们任何一个人坐在这里,都只能是在猜测。但我的感觉是,如果你从最终形态开始思考,它应该就和今天一样,对吧?公司的损益表不会突然变得不一样:无非是有一些收入、一些营收成本,还得有一些利润。如果这一点成立,那么问题就是,全球 GDP 增长率会是多少?and so now interestingly enough given the demand everyone's building and so the question is when will all of this come online what will be the demand supply no one knows right quite frankly any one of us if we're sitting here today will just be speculating but my feeling is that if you start by thinking of ultimate state would be just like today right income statements are suddenly not going to be different right for a company it's going to be hey I have some revenue I have some cost of revenue and I have to have some profit and if that is going to be true then the question is what is the world GDP growth rate?

人工智能能否显著加快全球 GDP 增长?Can AI Meaningfully Accelerate Global GDP Growth?

对我来说,关于 GDP 增长率转折的问题,更重要的是这一点。长期以来,全球经济增长率一直稳定在每年 2% 左右。AI 能否在短时间内显著改变这一趋势?That's the bigger thing for me in terms of the inflection of GDP growth rate. The world has kind of stabilized at 2% a year for a very large amount of time. Does AI have the ability to seriously inlect that in a short amount of time?

我想是的,而且我多少也希望如此。因为关键就在于,你看,哪怕把我们医疗保健支出的金额占各自 GDP 的比例算进去,对吧?所以我看到 NHS 部署 Copilot 时也会想,哇,如果它能改善患者的治疗效果、降低医疗保健成本,那从定义上说,你知道,这就是在助力英国的 GDP。所以我认为,经济中有一些领域非常适合采用 AI;事实上,这些领域受到的冲击最小,带来的益处却最大。我想,我们必须在这些领域找到最合适的切入点。软件开发也是如此,对吧?我们看到软件开发的需求弹性有多大,这真是令人着迷。也就是说,IT 积压的工作量巨大,而我们谁都没意识到。我当时……I think so. and I somewhat hope so. because that is the point right when I look let's take even as a percentage of any of our GDPs the amount of money we spend in our healthcare right so that's why I look at even the deployment of co-pilot in NHS and say wow if it can improve patient outcomes and reduce cost in the health care that is by definition you know helping the GDP of the United Kingdom and so I think that there are these pockets of the economy which are ripe for adoption of AI and really in fact there's the least amount of displacement, the most amount of benefit. and so that I think is where I think we'll have to find the best fit. software development is another one right which is we it's fascinating to see how much the demand is elastic for software development right which is that means the IT backlogs were huge and none of us even recognized it I was

哪些行业将从人工智能中获得最大的生产力提升?Which Industries Will See the Biggest AI Productivity Gains?

人们过去认为,在风险投资领域,你必须在一个市场中排名第一或第二才能胜出。但我们现在看到的是,你知道,Factory 排名第五,却依然在第五的位置上赢得了竞争。这打破了风险投资的所有规则。我还有个问题,稍微有点不按常理出牌,也不在议程上,但我就是想问:在财务、销售、营销等其他职能领域,我们会看到像编程领域那样的能力跃升吗?毕竟编程是最大的金矿。我觉得这是个很好的问题。你知道,人们经常会谈到,你是否拥有可验证的奖励机制。之所以这些领域容易实现,是因为其中很多魔法都来自在这些预训练模型之上进行强化学习;这正是帮助编程领域取得进展的关键。我们能否在知识工作领域也做到这一点,似乎要难得多。不过,话说回来,我们谈到的所有例子——比如联络中心、供应链系统、营销和现金到订单系统——我觉得这都是有可能的,而且我们已经看到了这样的趋势。它的需求是否像编程领域那样具有弹性?我觉得科学领域或许也有这么大的弹性,对吧?或者说,工程领域整体上都如此。顺便说一下,你知道,如果我看看正在发生的事情,不只是软件开发领域,就拿芯片开发来说,那也是正在经历剧变的另一个领域,对吧?你们能更快地完成流片。taught that you had to be the top one or two in a market to win in venture we're in you know factory which is fifth and it's winning fifth this breaks all rules of venture a question that I have again slightly naughty not on schedule but it's just like will we see the tipping in terms of capability advancement in alternative functions finance sales marketing that we've seen in coding because coding's been the mother load I think that's a good point it you know people talk a lot about you know do you have a verifiable re reward and you know it's easy to do these because a lot of the magic has been RL on top of these pre-trained models that is what you know is helped coding can we crack that in knowledge work seems much harder but again all the examples talked about sort of you know in contact centers in these supply chain systems in these marketing/ cash to order systems I think it's it's possible and we're seeing it is the demand as elastic as coding I think it maybe science may be as elastic right or engineering in general like by the way you know if I look at even what's happening inside not just in software development let's take silicon development that's another thing that's going through radical shift right your ability to tape out much faster.

在硅基环境中模拟一切。你的第一次尝试就可能完全成功。不过,就像你说的,即便只是优化维护报告——采用某种我绝对想不到的选择——也太不可思议了。我的意思是,这件事他们一直以来都在处理,就是安排维护,而我之前竟然没意识到,哇,这其实是一个很大的优化问题,现在你可以着手解决了。所以,我看过你的很多很多期节目,我们来快速问答一轮吧。有些问题别人还没问过,但我一直很想知道,我觉得大家也都想知道,不过,确实还没人问过。Simulate everything in silic. Your first spin can completely come out. but even just like optimization of maintenance reports like you said with choice which I would never it's unbelievable like I mean it's a thing that they've been dealing with forever just schedule maintenance and I not realized that wow that's a big optimization problem that you can go now run. So, I've I've watched you many many of your shows and we're going to do a quick fire around and people haven't asked some of these questions which I've always wanted to know and I think everyone wants to know but yeah people haven't asked them.

考虑到你的旅行安排,你的锻炼计划是什么?嗯,我是说,你的身体状况不错。嗯,我尽量保持吧。我不知道是不是都藏得挺好,不过你知道,有一件事我已经养成习惯了,可能持续了有25年吧,那就是每天早上起床后,你知道,我以前还能跑步。What's the exercise routine given the travel schedule that you Well, I mean you're in good shape. Well, I try to be. I don't know whether it's all well hidden, but you know, the one thing I've gotten into the habit, you know, this has been maybe 25 years even on, which is every morning I get up and, you know, I used to be able to run.

不幸的是,我的膝盖已经不太行了。所以现在至少我会在跑步机上调高坡度走。就连今天早上,我也锻炼了半小时。所以无论我身处何地,即使有时差,我每天都会去健身房。Unfortunately, my knees have gone way. And so now at least I walk with incline. even this morning I was able to get a half hour in. And so that's what it in spite of any jet lag wherever I am, I get hit the gym every day.

太棒了。我膝盖也不好,可能也是年纪大了的缘故。过去12个月里,你有什么想法改变了吗?Love it. I too have bad knees. is my age as well. what have you changed your mind on in the last 12 months?

为什么萨蒂亚不再相信,只要扩大规模,AI 对齐就会“自然而然地实现”Why Satya No Longer Believes Alignment Will “Just Fall Out” of Scaling

最后,这可能是因为毕竟我算是最早的一批人之一。是啊,显然,当达里奥、伊利亚、萨姆、格雷格和埃隆——我想那时大家就像一个幸福的大家庭——他们确实是规模定律的信奉者,而这也是我在2018、2019年时所相信的。嗯,我想我现在最大的变化是,我过去总觉得你知道,对齐问题会自然而然地解决。我现在不再这么认为了。所以我觉得,对齐可能是一个更难、也不同的问题。嗯,这就是我改变想法的一点。Last it might be because after all I was one of the first Yeah, obviously the when when Daario and Ilia and Sam and Greg and Elon were all I guess one happy family, they were really the believers in scaling laws and that's what I believed in back in 2018 2019. and u I think the biggest change now I have is I used to always think that you know alignment will just fall out. I no longer believe that. So I think this alignment thing may be a harder different problem. and so yeah that's the one thing I've changed my mind on.

为什么每家公司都可能成为 AI 基准测试的对象Why Every Company Could Become an AI Benchmark

今天看起来很疯狂,但五年后会变得极为普遍并被人们接受的事物。What looks crazy today but will be incredibly common and accepted in 5 years.

你知道,可能就是我们刚才谈到的那些评测,对吧?什么是 firm?我觉得 firm 应该是一种……一种 AI 基准测试。You know it might be that thing that we just talked about those evals right which what is a firm? I think a firm would be an an AI benchmark.

最后一个问题问你。未来10年有很多值得期待的事情。具体到你来说,就像我一样,我是因为我母亲患有多发性硬化症。Final one for you. There are many things to be excited about over the next 10 years. For you specifically, like for me, it's my mother's got MS.

我对可能具备这种能力的药物研发感到非常兴奋。I'm so excited for the drug development that could be capable.

萨提亚对未来十年最期待的事What Satya Is Most Excited About for the Next Decade

你最期待什么?what are you most excited for?

我是说,这真是个很好的例子。我想,我一直期待的事情之一,可以说,就是出现一种像这样强大的通用技术。人们会说:“哦,这不只是又一种通用技术。”无论这是什么,我都希望它能以史无前例的速度在全球传播、扩散,并像以往任何事物都不曾做到的那样,惠及经济的各个领域。I mean, that's a great one. I think one of the things that I've always been, I would say, been looking forward to is a general purpose technology even as powerful as this. And people say, "Oh, this is not just another general purpose technology." Whatever this is, I hope this spreads, diffuses throughout the world faster than anything else and throughout all sectors of the economy unlike anything else.

而且说真的,这一次全球增长的前沿确实发生了真正的转变:这不再只是数谁是赢家、谁是输家,而是有点像所有人都能被带动起来。这一直是我的梦想。事实上,我父亲是一位发展经济学家,你知道,我们谈的总是趋同增长:全球南方什么时候才能迎来某种能让发展突然提速的东西?我希望这就是了。And truly for once there is a real shift in the frontier of that global growth that happens where it's not about just counting winners and losers but it's kind of like everyone gets lifted like that's been my dream in fact my father was a development economist and you know it was always about convergence growth when will the global south get something that just suddenly boosts that I hope this is it.

做这件事真的很开心。非常感谢你抽出时间。没问题。This has been such a joy to do. Thank you so much for taking the time. Absolutely.

很高兴认识你,哈利。It's a pleasure, Harry.

值得记下

让我们建立一个经过工程设计的零信任环境,在任何失控智能体造成伤害之前将其限制住。
每家企业都会拥有私有评测基准,而这将成为其核心知识产权。
我不再相信这一点。