Vals AI 创始人谈 AI 的下一个临界点：独立评估与递归自我改进

简要描述Brief Description

拉扬·克里希南与乔·朗斯代尔和维韦克·戈帕兰探讨独立的 AI 评估。他们审视科学验证、企业基准测试、网络安全测试、地缘政治偏见、递归自我改进的估算、欺骗与评估意识，以及他们对 AI 持乐观态度的原因。Rayan Krishnan discusses independent AI evaluation with Joe Lonsdale and Vivek Gopalan. They examine scientific verification, enterprise benchmarks, cybersecurity tests, geopolitical bias, recursive self-improvement estimates, deception and evaluation awareness, alongside the reasons for their optimism about AI.

目录Table of Contents

• 剧集介绍
  Episode intro
• Palantir、马基雅维利与亚历克斯·卡普的启示
  Palantir, Machiavelli & lessons from Alex Karp
• 破解一条370年前的密码 / AI是否正在带来新发现？
  Solving a 370-year-old cipher / Is AI making new discoveries?
• 如何衡量 AI 在金融、法律等领域的能力？
  How do you measure AI abilities across finance, legal, etc?
• 模型评估背后的科学原理是什么？
  What is the science behind model evaluation?
• 独立评估的重要性
  The importance of independent evaluation
• 评估中文模型并应对蒸馏问题
  Evaluating Chinese models & addressing distillation
• 谁会率先实现递归式自我改进？我们应该担心吗？
  Who will reach Recursive Self-Improvement first? Should we be worried?
• 模型如何试图欺骗评估者
  How models try to deceive evaluators
• 为什么拉扬依然保持乐观
  Why Rayan remains optimistic

剧集简介Episode intro

Herp 听了这话，说：“我会觉得你相当天真。你应该读读马基雅维利。让人畏惧远比让人爱戴好得多。”我们 Val 的每个人都有很多精彩的故事。这里的文化非常独特，我们在 Val 也一直尝试从中汲取灵感。Herp heard this and said, "I'll think you're being quite naive. You should read some Machiavelli. It's far better to be feared than it to be loved." There's a bunch of great stories from all of us at Val it's a very unique culture that we've tried to take cues from at Val as well.

整个模型评估领域到底是怎么回事？What even is this whole model evaluation space?

独立评估的理念是，前沿实验室正以越来越快的速度发展，因此不应由它们负责自我监管。Independent evaluation is the idea that frontier labs are developing at increasing pace and they shouldn't be responsible for self-regulating.

主持人：你们有办法测量 RSI。Host: You guys have a way of measuring RSI.

Anthropic 似乎会在 2027 年 7 月或 8 月实现完整的递归自我改进（RSI）。模型进展会加速，甚至可能进入我们对模型了解更少的领域。It seems like Anthropic will get to full RSI by July or maybe August 2027. Model progress will speed up and it will maybe even move into places where we understand the models less.

如果我们没有密切关注这一点，那确实会挺吓人的。That simply would be pretty scary if we weren't watching that carefully.

更高的智能体Greater intelligences

超级预测者会比人类预测得更准确。他们实际上能够说出任何人会做什么，甚至那个人自己都不知道。will super forecast better than humans. They will be actually able to say what any human is is going to do without the human even knowing it.

Ryan Christian 是 Val's AI 的创始人兼 CEO。每当有新的 AI 模型发布，你看到那些数字和模型之间的比较，其中很多数据都是由 Val's AI 生成的。他们已经成为顶尖的模型评测机构之一。这是一个很有意思的领域，因为你总是在为模型创造新的高峰，让它们去攀登。Ryan Christian is the founder and CEO of Val's AI. Whenever you see a new AI model get released, you see those numbers and they compare the AI models to each other, those numbers are generated many times by Val's AI. They've become one of the very top evaluators of models. This is a fascinating area cuz you're constantly creating new hills for models to climb.

这要求你真正理解如何衡量模型能力、评估模型风险，并尝试新的模型风险评分方法。这意味着 Ryan 走在 AI 领域发展的最前沿，了解 AI 的发展方向，也了解我们如何帮助塑造政府可能采取的监管方式，以及模型公司可能如何考虑进行自我监管。It requires you to really understand how to measure model capabilities as well as evaluating model risks and trading new ways of scoring models for risks. This means Ryan is on the cutting edge of what's happening in the world of AI, where AI is going and of how we can help create the landscape of what government might do to regulate these things and how the model companies might think about regulating themselves.

他正在创建相关框架、概念和衡量标准，为未来十年人工智能的发展方向引路。让我们听听他的看法。欢迎来到《美国乐观派》。Ryan，感谢你今天参加我们的节目。He's creating the frameworks and the concepts and the measurements of that are leading the way for what AI is going to look like in the next decade. Let's hear from him. Welcome to American Optimist. Ryan, thank you for joining us today.

感谢邀请我来。Thanks for having me.

Val's AI 首席执行官的创始人。Founder of the CEO of Val's AI.

呃，Vivek，谢谢你来这里，这样我听起来会更聪明。Uh Vivek, thanks for being here and I'll make me sound smarter.

当然，随时为您效劳。Of course, here to help.

主持人：欢迎来到节目。那，跟我们聊聊你的背景吧。你学过……Host: Welcome to the show. So, let's tell us a bit about your background. You studied

帕兰提尔、马基雅维利与亚历克斯·卡普带来的启示Palantir, Machiavelli & lessons from Alex Karp

你在斯坦福学的是计算机科学。我想你曾在微软待过一小段时间。然后你很久以前就加入了 Palantir。是什么促使你加入 Palantir 的？computer science at Stanford. You were I think at Microsoft for a little bit. And then you you joined Palantir a long time ago. What what led you to join Palantir?

Rayan Krishnan：是的，是的。嗯，感谢邀请我。我其实是在微软工作时才刚刚开始接触这个领域。Rayan Krishnan: Yeah. Yeah. Um thanks for having me. I actually just kind of started when I was at Microsoft.

我高中时曾在那里实习。然后，呃，谷歌发表了 BERT 论文。嗯，对于不了解的人来说，这是一篇非常重要的论文。它表明，这种通用系统实际上能够通过爬山法不断改进，或者说，能够在多种主要的 NLP 基准测试中取得成功。It was an intern there in high school. And uh the BERT paper came out from from Google. Um and for those who don't know, this was a really big paper. It showed this generalized system uh was actually capable of hill climbing or or succeeding across a variety of major NLP benchmarks.

嗯，然后它是……Um and it was

能不能给不太懂技术的听众解释一下，NLP 基准测试是什么意思？Say for our listeners who aren't technical, what is what's the NLP benchmarks mean?

嗯，这基本上意味着人们设计出了一些特殊的专项测试。他们试图打造能在这些测试中取得高分的专用系统。比如，这里有一份选择题考试，然后还有一个情感分析任务。Well, it it basically means there are these special specialized tests that people came up with. They tried to make specialized systems that would score well on them. Like, here's a multiple-choice exam, and then here's a sentiment analysis.

属于自然语言。Of natural language.

是的，没错。嗯，所以，他们没有构建一个专门针对某项任务、并在该任务上表现出色的系统，而是展示了一个用途非常广泛的模型。它对语言有着根本性的理解。嗯，从某种意义上说，这种能力也体现在了所有这些基准测试上。Yeah, that's right. Um and so, instead of building a specialized system that did well on one of these tasks, what they had done was shown they had a model that was very general-purpose. It had a fundamental understanding of language. Um and in a way that translated to all of these benchmarks.

它突然在各方面都跃居第一。It was suddenly number one across the board.

那是哪一年？What year was that?

我想应该就是2019年或2018年。I think it was just 2019, 2018.

那是很久以前的事了。It's a long time ago.

对，比如说……Yeah, like

是啊，感觉已经是很久以前的事了。是啊。yeah, lifetime ago. Yeah.

用 AI 领域的说法，它还处于相当早期的阶段，不过已经在逐步发展了。In in AI terms, it's pretty before age, but it was getting there.

是的，这件事影响很大。而且，而且我觉得，这对我来说把事情说得很清楚。这算是一个能说明“苦涩教训”的时刻。这个想法是，无论我们试图做什么，又如何围绕算法优化自欺欺人地思考，实际上，只有拥有更多数据、达到更大的规模，才能让我们解锁新的智能形式。Yeah, it was a big deal. And and I think to me, made it very clear. This is kind of an illustrative uh moment for the bitter lesson. The idea that um no matter what we uh what we try to do and kind of delude ourselves into thinking around algorithmic optimizations, actually just having more data at a bigger scale is what allows us to unlock new forms of intelligence.

嗯，所以当时我非常清楚地意识到，我们将会拥有像这样的强大模型或系统，它们会成为我们社会结构中不可或缺的一部分。嗯，我也认为这些模型可能会被滥用。大约同一时期，我还在微软工作时，读了《从0到1》。嗯，我越来越清楚地看到，Palantir 的人们正试图以一种能够保护公民自由或更广泛的西方民主理想的方式来构建这项技术。Um and so, it was very clear to me then is that we're going to have uh a powerful model or systems like this, uh which will be an essential part of our the fabric for our society. Um and I think there were also ways in which the models could be misused. Um around the same time while I was at at Microsoft, I read Zero to One. Um and it became increasingly clear that people at Palantir were uh trying to construct the technology in a way that protected civil liberties or Western democratic ideals at large.

嗯，所以我当时特别兴奋能去 Palantir 工作，亲眼看看这项技术如何落地，并以符合我自身价值观的方式得到应用。嗯，我本科去了斯坦福，后来又被那里的研究分散了注意力。我发表了一些论文，也算是走上了攻读博士的道路，不过我还是抽出时间在 Palantir 工作了一个夏天。我觉得那段经历也让我深受启发。Uh and so, I was really excited to work at Palantir to see the technology come to life and adopted in a way that was consistent with the values that I had. Um and so went Stanford for for undergrad, got got distracted doing some research there. So, I published some papers and was kind of on track to do a PhD, but I made the time to also work at Palantir for a summer and I think that was also really illuminating experience.

你刚才提到，你在 Palantir 从 Karp 那里还学到了别的东西。还有什么有意思的吗？You were mentioning you learned something else at Palantir from Karp. Anything anything else there is interesting?

是啊，我的意思是，我在 Palantir 工作期间有很多很棒的故事。那是一种非常独特的文化，我们在 Val's 也尝试借鉴其中的一些做法。不过，总之，我在那里时有机会参与一个黑客周项目。嗯，这个项目本质上是我和一位在伦敦的同事想到的点子。我们当时是远程聊起这件事的。Yeah, there's I mean there's a bunch of great stories from from while I was at Palantir. It's a a very unique culture that we've we've tried to take cues from at Val's as well, but um anyway, while I was there, I had the chance to work on this hack week project. Um and this essentially was an idea which me and this this guy in London had. So, we were chatting about it remotely.

我飞了过来，而且之前基本上用 CLIP 做过一些工作。CLIP 是一种联合图像—文本嵌入模型。I flew over and essentially I'd done some work with CLIP, this joint image text embedding model.

嗯，我们当时想做的是向观众解释那是做什么用的。Um and what we were trying to do was explain what that is for for the audience.

一种单一模型的构想，它可以将图像输入和文本输入编码为相同的语义理解。对吧？我们想做的是在 Gotham 中构建反向图像搜索功能。这样一来，你就不只是输入文本查询，还可以上传一张蓝色汽车的图片，系统会搜索整个数据库，并找出你想搜索的蓝色汽车。idea of a single model that that can um encode image input and text input into the same semantic understanding. Right? And uh and what we we're trying to do is build out a reverse image search feature into Gotham. Uh so, instead of just typing text queries, you could upload an image of say a blue car and it would search over the entire database and pull pull back blue cars that you're trying to search for.

嗯，它之所以会这么做，是因为我们使用的是这个模型相同的底层、底层嵌入空间，它可以将文本映射到图像。嗯，总之，所以，呃，这是我发表的一篇研究论文，我想基于它开展工作并构建一个功能。所以，呃，实际上有几位全职员工加入了我们的团队，而作为一名实习生，呃，我负责带领这支由几位帕兰提尔员工组成的小团队，我的意思是，那是一个……Um and it would do this because we were using the same underlying underlying embedding space from for this model, which would map text to images. Um Anyway, so uh so, this was like a research paper that that I'd published and I wanted to work and build out a feature. So, uh we actually had some full-timers join our team and so as an intern, uh I was I was leading this this small team of uh of Palantirians and I mean that was an

听起来…That sounds

一次令人振奋的经历。我这周大部分时间都没睡觉。嗯，后来这个想法入选了决赛。所以，我向整个公司做了展示。exhilarating experience. I was not sleeping for for a lot of this week. Um and then this idea was then on went on to be selected as a finalist. So, presented for for the whole company.

嗯，然后实际上，在我离开之后，他们后来为它申请了专利，还把它产品化了。所以，嗯，看到我当时想到的这个小点子，以及我在加入 Palantir 之前做的一些研究，后来竟然成了产品的一部分，而且还成了他们想要掌控的知识产权，真是不可思议。不过话说回来，通过这件事，我也有机会见到 Karp，和他聊了一会儿。我当时有个问题想问他，基本上就是：你知道，Palantir 的名声似乎非常差。Um and then actually after after I left, uh they went on to patent it and actually productize it. So, uh it was insane to see this little idea that I had and some of the research that I was working on before Palantir go on to be a part of the product and um and actually an IP that they that they they wanted to control. Um but anyway, through through this I also got the chance to meet with Karp and uh and chat with him a little bit. And I had this question for him uh which was essentially, you know, you know, it seems like the Palantir reputation is so negative.

我当时在斯坦福，所以有很多抗议活动，嗯。I was at Stanford so there were a lot of protests um

你对这些价值观非常乐观，但外界的看法比你预期的更加负面。And you were very positive on the values, but the external perception was just more negative than you expected.

是的，没错。我的意思是，在我看来，这一点非常明确：这关乎选民掌握权力的地方，也就是我们真正通过投票选出来、由他们来决定我们想如何使用技术，以及支撑这项技术的价值观应该是什么的人。嗯，决定这项技术应该用在什么地方，并不是科技公司的职责，也不该由它们来否定。嗯，所以那里的人都在非常仔细地思考这些问题。Yeah, that's right. I mean, it's very clear to me that it was at the seat of the power of the electorate, people who we actually vote into power to decide how we want to use technology and what those values that underpin it should be. Um and it was not really the place of technology companies or to to deny where that technology should be used. Um and so people there were were thinking through very carefully.

事实上，就像今天我们谈论对齐挑战这类问题一样。Palantir 的人当时把这个称为人类与 AI 共生。这也是当时讨论的话题。又比如现在的智能分类体系，呃，我们有一套评估和基准测试体系，在某种程度上是在模仿它。And in fact, like today we talk about things like the alignment challenge. People at Palantir were talking about this as human AI symbiosis. That was something that was being talked about. Or even like the taxonomy for intelligence now, uh we have a system of evaluations and benchmarks that it kind of imitates that.

所以，我认为人们正在深入思考一些根本性的问题。So, I think there are some fundamental problems which people were thinking deeply about.

呃，让我惊讶的是，外界对这家公司的反响如此负面。人们谈论这项技术时，往往对它的理解非常浅薄，关注的也多是它一些可能不太光彩的用途。呃，事实上，Palantir 也曾协助分发新冠疫苗，帮助过不少医院。嗯，所以人们还可以举出其他案例来说明。Uh and it surprised me that the outward reception of the company was so negative. Talking about um you know, a lot of the maybe more unsavory potential applications of the technology in a very diluted understanding of it. Uh when in fact, Palantir also helped distribute the COVID vaccine, helped a bunch of hospitals. Um and so there there could be other case studies that people could point to.

嗯，Karp听了之后就说：“我觉得你有点太天真了。你应该读读马基雅维利。让人敬畏远胜于让人喜爱。”嗯，他确实还拿我们和其他咨询公司做了一些比较，基本上是说，如果他们怕我们，至少说明他们认为我们很强大。这样一来，到了需要的时候，他们就会来找我们这样的人。Um and Karp kind of heard this and said, uh "Well, I think you're being quite naive. Uh you should read some Machiavelli. Uh it's far better to be feared than it is to be loved." Um and indeed you some comparisons to other consulting companies and basically said that if they're fearful of us, at least they think they think that we're powerful. Uh so in the moment of need, they they will come to a people like us.

嗯，所以这很有意思。实际上，这让我想起了我今天从各个实验室看到的很多情况。我觉得，其中一部分确实是出于对模型能力的担忧。不过在很多方面，这也是一个让他们展示自己正在打造的东西有多强大的机会。Um and so that was fascinating. And in fact, it actually reminds me a lot of what I see from labs today, which is I think in some part a genuine fear about what the models are capable of. Uh but in a lot of respects, a an opportunity for them to show the power of what they're building.

主持人：如果达里奥是经过深思熟虑的，那他可能就是故意想让他们令人畏惧。Host: Might be on purpose that Dario wants them to be feared if he's being thoughtful about it.

我是说，不管是达里奥还是萨姆，我看到一些标题里有人说，呃，普通大众有理由感到害怕。呃，我觉得这并不是一个领导者应有的理性做法。呃，这档节目叫《美国乐观主义者》。我其实相当乐观，也认为应该理性行事，并看到人工智能带来的成果将会积极得多。I mean, whether Dario or Sam, there there headlines I see where people are saying things like uh the general public has a right to be afraid. Uh and I think that's not really a rational way for uh for a leader to act. Uh and this this show is called American optimist. I'm actually quite optimistic and uh, enacting rationally and seeing that uh, the fruits of AI will be far more positive.

那么，在我们深入探讨 VALs 和那个……之前So, before we dive into VALs and the

破解一则有370年历史的密码／AI 是否正在取得新的发现？Solving a 370-year-old cipher / Is AI making new discoveries?

评估 AI 模型的科学。嗯，我们来聊聊 AI 进展的现状，以及我们目前走到了哪一步。事实上，有人告诉我，你用 Anthropic 的 Fable 破解了一种有 370 年历史的密码。370 年历史的密码，这到底是什么？science of evaluating AI models, um, let's talk a little bit about the state of AI progress and where we are. In fact, tells me that you used uh, you used Fable from Anthropic to solve a 370-year-old cipher. 370-year-old cipher, what what is this?

没错，是的。嗯，我们现在正在进行许多不同类型的评估。我觉得目前前沿正朝着科学发现或新颖发现的方向发展，这是一个令人兴奋的领域。所以，我们一直在整理一批尚未破解的密码、数学问题和物理问题。That's right, yeah. Um, it Well, we're we're doing a bunch of different types of evaluations now and I think right now with the frontier's headed scientific discovery or novel discovery uh, is an exciting frontier. And so, we've been compiling uh, a set of unsolved ciphers, math problems, physics problems.

给观众看的未解密码？这是什么意思？生unsolved cipher for the audience? Like what what does that mean?

它本质上是一种你试图破解的密码，但你没有解密密钥。所以，你需要构建从源文本到原文的映射，并弄清楚这条消息究竟是什么。It it's essentially a code which you're trying to break, but you don't have the key to break it. And so, you need to construct the mapping um, from a source text to an original text and figure out what the message actually is.

在很多这类情况下，这些就像是世界领导人彼此发送的消息，或者像是在海盗船上的那种……之类的。many of these cases, these were like messages that was that were sent from like world leader to world leader or like that were like on a on a pirate ship and like

是的。Yes.

嗯，因为很多原因，因为它可能只是个一次性密码本之类的，这种情况下你就破解不了它，因为它是一次性密码本。所以当他们双方都有一份……那就不可能，可能是不可能的，因为每样东西都会被转换……Well, cuz a lot cuz there's cuz there could be just be like a one-time pad or something which case you can't break it cuz it's a one-time pad so when they both have a copy of and then be impossible probably be impossible cuz each thing translates

有可能，不过你还可以引入更多线索，来了解这个人是如何构造这个问题的，或者他们想向另一个人传达什么信息。嗯，通常这些信息的构造方式，就是为了让接收者去破解。Potentially, but there's also like more cues you could you could draw into about how the person constructed the problem or the message they were trying to convey to the other person. It's um, usually that that the messages are constructed in a way that they're intended to be cracked by the recipient.

对当前 AI 进展的一种愤世嫉俗的看法可能是：AI 特别擅长遍历各种搜索空间，而有些问题可以拆解成非常庞大的搜索空间，人类既没有时间，也没有能力逐一探索其中成百万上千万种可能。比如说，对于密码，你可以想象一下，如果我要开始尝试破解一个密码，我会说：“要不我试试给这些东西逐个加上数字，或者把它们各自移动一位之类的？”我会去探索那些显而易见的办法，而计算机却能非常快地探索其中成百万种可能。所以，数学和科学领域的许多突破，是不是仅仅因为有人更好地定义了一个搜索空间，让人们可以在其中搜索？还是说模型本身也进行了更有创造性的思考？这些事情到底是怎么发生的？One cynical view of AI progress right now might be that just like AI's really good at going over all these search spaces and it certain problems can be broken down into these really large search spaces that like no human has time to or ability to go through like millions of them or something. Whereas for a cipher for example, you could just go through like if I was going to start trying to break a cipher, I'd say like, well, what if I just started like adding numbers to each one of these things or shifting them one or whatever. I'd explore the obvious things whereas a computer can explore like millions of these things really fast. So, is are a lot of the things happening in math and science breakthroughs just somebody like just defining a search space better to search through or or is there like more creative thought going from the model and how do these things?

很多问题是通过穷举解决的，或者是通过寻找反例来解决的；而说到底，你的能力受限于能尝试多少种不同的反例。不过，我不会轻视这类问题。有很多事情，人类光靠穷举可能要花上大半辈子才能做到，但现在借助模型，几个小时就能看到成果。A lot of the problems that have been solved through exhaustion or it's trying to find a counter example and you're you're bottom line by your ability to try lots of different counter examples, but I wouldn't be dismissive of that class of problem. There's a lot of things which may take humans in the order of a lifetime to achieve just through exhaustion, but we're now able to see on the order of hours with models.

那么，AI 通常是在解决很多数学和科学问题吗？我们该如何理解这一点？So is the AI generally solving a lot of math and science problems then or how how do we think about this?

它通常是在解决极具挑战性的问题。我认为，仍有很大一类问题受限于人力投入，进展的瓶颈就在于此。我认为，要真正以极其新颖的方式取得进展——比如需要对解法有直觉的那些方式——还需要持续投入更多努力。人类积累的这类直觉和能力，目前很难传递给模型。It is generally solving extremely challenging problems and I think there are still a large class of problems for which we are human effort bottleneck. I think it will take uh more sustained effort to actually make progress in in exceptionally novel ways like ways that require an intuition of the solution and those those sorts of things which humans have accrued are very hard to pass on the models right now.

而且感觉这些模型正在产生令人难以置信的结果，我们也知道它们确实在产出这些结果，比如纳维–斯托克斯方程之类的问题上的结果；但这些结果又非常难以理解，很难解析、很难看懂。嗯，你觉得在这些不太容易验证的领域里，发现应该以怎样的方式发生？或者说，为了让我们能够信任这些模型的发现成果，它们的行为方式需要做出哪些改变？It also seems like like models are producing these incredible results and like we get to knowledge that these results are happening like Navier-Stokes and the like outputs produced by the model however like very like very unintelligible. They're hard to parse, hard to understand. Um like what do you think of how a discovery should happen in like these less verifiable domains or like what what needs to change about how these models behave in order for us to like trust the discoveries of them?

是的，我认为这是最大的瓶颈之一，尤其是当你接近人类智能的前沿时。我们很难调动这些人类的力量，然后验证这段很长的输出是否确实正确。Yeah, I think this is one of the biggest bottlenecks especially when you're approaching the frontier of human intelligence. It's really hard for us to tap those humans and then verify this long output is actually correct.

是啊。Yeah.

嗯，我认为数学中最受欢迎的技术之一，或许也能应用到其他领域，那就是使用 Lean 这样一种可验证语言。这样一来，你就可以输出内容，然后将其转换成一种语言；只要它能编译通过，就一定为真。我们是在已知成立的基本原理之上逐步构建的。因此，这种方法或许也适用于物理学或编程等其他领域。Um one of the most popular techniques in math which I think might translate to others is the idea of using a verifiable language like Lean. And this would be a world in which you could output take the output and translate it into a language which if it compiles it must be true. We're building up based on primitives that we know to be the case. And so that approach may apply in other fields like physics or coding.

我们也可能会遇到一些情况，现实世界才是最好的验证者。因此，通过开展测定或湿实验，我们就能确切知道模型预测的属性是否确实如此。We may also see some instances where the real world is the best verifier. So by running an assay or a wet lab experiment, uh we'll actually know for certain whether the properties the model predicted are actually the case.

有道理。那么你觉得实验室、你们团队或其他组织应该如何开展这些实验和验证？Makes sense. And like how do you think the labs or you guys or or other organizations should be like running these experiments and verifications?

我的意思是，至少要非常谨慎。嗯，呃，我是说，我认为有第三方团体能够支持这件事会非常重要，这也是 Val 本身愿景的一部分。我的意思是，我们现在身处这样一个世界：各实验室之间对模型的访问权限实际上有限。所以，Anthropic 的研究人员很难评估 OpenAI 的模型，反过来也一样。I mean, very carefully in the very least. Um uh I mean, I think it's it's going to be very essential that there are third-party groups that can support this and this is part of the vision behind Val itself. I mean, we're now in a world where there's actually limited access to models across the lab. So So Anthropic researchers have a hard time evaluating OpenAI models and vice versa.

这在某种程度上是有意设计的。嗯，但这意味着很难进行真正公平的对比，因为那些最优秀的运营者……And that's kind of by design. Um but that means that it's very hard to have a true apples-to-apples comparison uh because the best people who are running

如何衡量 AI 在金融、法律等领域的能力？How do you measure AI abilities across finance, legal, etc?

这些评估人员往往就是那些已经在实验室里、且存在偏见的人。these evals are often the people who are biased and at the labs already.

是的。稍微退一步来看，现在 AI 能做的事情，有很多是以前做不到的。当然，其中很重要的一部分是弄清楚该如何衡量它。我们接下来会深入探讨这一点。Yeah. And and so stepping back a little bit, there's there's all these things that AI could do now that it couldn't do before. And like a big part of this is like understanding how to measure it, of course. We're going to get into that.

对，这就是你要做的事。比如说，企业里一个非常宏观的衡量难题是：你得想办法衡量什么才算好工作，或者工作做得有多好，对吧？比如说，如果是律所之类的地方，你也许可以定义什么是初级律师级别的工作，什么是合伙人级别的工作。可这到底该怎么定义呢？That's that's what you do. Like one of the measurement problems just at a very high level is in the enterprise you have to like measure what is good work or how good is the work, right? So like like how like so for example, if it's like a law firm or something, you can define maybe like associate level work and you can define partner level work. Like how do you even define that?

比如，比如这是我们正在解决的问题吗？人工智能是否越来越擅长于看懂、理解事物究竟是什么？Like like is that a problem that we're working on? Is Is AI getting better at under seeing understanding what is what?

这确实很有挑战性。我认为，从很多方面来说，我们在人类层面还没有解决这个问题。对于哪些工作属于助理级别、哪些属于合伙人级别，我们还没有非常明确的界定。It's really challenging. I think in a lot of ways we haven't solved it at the human level. We don't have a very clear designation for what is associate level work versus partner level.

最高法院谈到什么是色情作品时，不是说过“看到了就知道”，对吧？大概就是这么一句话，差不多是这样。the Supreme Court like what is pornography and they said you just know it when you see it, right? It's like that was the line. It's something like that.

是啊，挺有意思的。Ben 在 A16Z 节目里说的几乎一模一样。Yeah, it's funny. That's almost exactly what Ben said on the A16Z show.

也许你们有个共享的共享大脑之类的东西。Maybe you have a shared shared brain or something.

真有意思。嗯，是的，这个比喻完全一样。这个比喻很恰当，对吧？而且既然你刚才……不过，我的意思是，你的工作就是弄清楚并定义这些东西。That's funny. Well, yeah, it's a well, it's well, it's the same analogy. It's the right analogy, right? And since you just But I mean, but your job is to have to figure out and define these things.

是的，没错。我觉得有一种非常以咨询为导向的做法：你可以去和对方坐下来聊，试着了解他们认为什么才算好，然后把这些标准编码成可重复执行的评测，并运行它们。我觉得这就是传统上构建评测的方式。我真正看好的是，呃，自动化基准测试这个想法。Yeah, that's right. I think there is a way you can do this in a very consulting-driven approach where you go and sit down with the person and try and understand from them what good looks like, encode that into repeatable evals, and run that. And I think that's the way that traditionally evals have been built. The approach that I'm really optimistic about is uh the idea of auto benchmarking.

嗯，我们……我们看到，模型进步的速度还在持续加快，我们将会得到越来越强大的模型，因此底层工作本身也必须改变。我对此非常兴奋的一点是，我们可以利用过去已经由人类完成的工作，估算在另一种假设情境下模型会产出的工作质量。呃，这样一来，我们就可以系统地为已经产出的所有工作成果构建评测。Um we we that the rates of model progress is is continuing and we're going to get more and more capable models and so the underlying work itself has to change. And so what I'm very excited by is the idea that you could take the previous work that was already done by humans and approximate the quality of the work output that would be produced by models in that counterfactual. Uh, and in that way you can uh, systematically build evals of all the work product that's already been produced.

嗯，也许可以举个例子：查看一个 GitHub 仓库，看看之前由人类合并的 PR，再评估智能体是否也能完成同样的工作，以及哪种智能体能以成本最低的方式完成。嗯，我觉得这和法律领域的红线审查流程或医学领域的诊断流程有相似之处。Um, maybe as an example you could say look at a GitHub repository and look at the pre- previous PRs that were merged by humans and see if agents could equivalently do that and then which one would do it in the cost mitigated most cost mitigated way. Um, and I think there's parallels for that in the redlining process in legal or the diagnostic process in medicine.

这是不是又回到了工作流和结果的本体论？要让这一切运作起来，你必须先定义这些东西。是不是说，这也算是底层的核心问题？你必须先真正理解可能出现的结果，才能衡量它们吗？还是说你怎么看这个问题？Does this get back to like an ontology of workflows and outcomes you have to define to make all this work? Is that is that is that kind of the underlying thing as well? You have to actually understand the possible outcomes before you can even measure them or or how do you think about that?

我觉得最难的部分其实是衡量结果本身。嗯，因为公司内部的中间流程和工作流有很多方式可以为组织提供支持，而组织随后会产生某种结果。嗯，所以有时很难把某项具体任务与工作联系起来，再把工作与结果联系起来。嗯，所以，要将 token 输入映射到最终输出，中间环节必须经过“工作流”这个概念。I think the hardest part is actually measuring the outcome itself. Um, cuz there's a lot of ways in which the intermediary processes and workflows within a company uh, are supportive of the organization that then goes on to have an outcome. Um, and so it's sometimes hard to draw the connection between a particular task and the job and then the job and the outcome. Um, and so so the mapping that inputs of tokens to the final output intermediary has to go through this concept of workflow.

我想，应该有很多中间结果，你甚至得提前定义好，先弄清楚它们是什么，然后才能衡量它们。I imagine there's like lots of intermediary outcomes you should even have to define ahead of time to understand like what they are though before you can measure them.

是的，我想这就是我想表达的：查询层面仍然只是实现最终结果所需工作的一部分。Yeah, I think that's what I'm what I'm trying to say that that the query level is is still a subset of what it takes to get to the outcome.

太棒了。Awesome.

挺有意思的，因为过去几年里，我们关注的东西经历了不同的粒度层级。这和模型的能力相对应。不过我记得两三年前，我们评估的还只是文本补全，看的也就是这些输入输出对，内容全是文本。而现在，中间多了各种各样的小环节。嗯，我们在看相关工作时，人们评估的时间跨度都有哪些？也许可以给大家介绍一下目前的情况：一项任务通常要花多长时间？这些模型又能持续执行任务多长时间？It's interesting because we've gone through like different levels of granularity over the past few years on like what we what we care about. It it maps to like what the models are capable of but I remember like two years ago or three years ago we were like evaluating just like the text completions uh, and like we were looking at these like input output pairs and it was just text and then now there's all these little things in the middle. Um, like when we're looking at work what are like the horizons in which like people are like evaluating things and maybe teach people about like the state of play of like uh, like how long is a task uh, and like how long are these models capable of doing tasks?

是啊，对于非技术受众来说，整个模型评估领域到底是个什么东西？Yeah, for non-technical audience like what even is this whole model evaluation space?

模型评估背后的科学原理是什么？What is the science behind model evaluation?

瑞安·克里希南：嗯，这个大致的想法就是衡量什么样的表现才算好，并看看模型是否有能力产出这样的结果。嗯，我们会把评测分成两、两类：能力和风险。能力评测是对输出进行近似衡量；风险评测则是确保一旦出了问题，我们不会看到某种不对称的负面影响。Rayan Krishnan: Well, it's it's the broad idea of measuring what good looks like and seeing whether models are capable of producing that output. Um We we think about evaluations in kind of two two categories, capabilities and risks. And capabilities are approximating the output. Risks are making sure that we don't see some asymmetric downside if something goes wrong.

嗯，从时间跨度来看，模型已经向我们证明，它们能够在没有任何人工干预的情况下运行越来越长时间。Um and in terms of time horizon, models have kind of demonstrated to us they can run for longer and longer periods of time without any human intervention.

我们在谈论什么？are we talking about?

呃，我们的一些基准测试会运行超过 5 天。嗯，所以Uh some of our benchmarks will run over 5 days. Um So

而那大概相当于人类 5 天的工作量，或者说大概 5 天。And that's like 5 days worth of human work or like 5 days

时间。time.

是墙上时钟时间，没错。Wall clock time, yeah.

为什么它们会在 5 天后失败？按理说，它们总有一天不就能一直持续下去了吗？Why do they fail after 5 days? It should be like couldn't they just go forever at some point?

问题不在于它们会失败，而在于它们会陷入瓶颈。因此，额外增加 token 实际上并不能帮助它们解锁新内容。that they fail, it's that they plateau. And so that additional tokens don't actually help them get new unlocks.

所以他们差不多已经想出了所有能用来解决这个问题的办法。So they've kind of figured out everything they could do on the problem.

或者他们不够坚持，没能尝试绕过眼前障碍的新方法。Or they're not persistent enough to try new approaches that get around the wall that they're seeing.

也许可以给观众举个例子，说明这些模型能完成哪些类型的任务。比如，在你们的一些金融或法律基准测试中，我们实际上能用这些模型复现哪些工作？Maybe give the audience just an example of like the kinds of tasks that these models are capable of like in in some of your like finance benchmarks or legal benchmarks, like what what work can we actually replicate with these models?

是的，你基本上是在独立评估各种不同的领域。Yeah, you're doing independent evaluations of all sorts of different areas basically.

没错，是的。That's right, yeah.

这是否意味着你和你的团队必须了解财务部门或法务部门在做什么？要建立这些基准，你自己必须具备相关专业知识。Does that mean you have to understand your and your team what finance is doing or what the legal is doing? You have to have yourself expertise in order to build these benchmarks.

嗯，我们会与这个领域的合作伙伴合作。在法律方面，我们会与美国最大的律师事务所合作，或者与银行合作——你知道，GCB 银行了解自己在金融领域的业务。在编程方面，有些任务看起来像是氛围编程基准测试，我们会衡量模型构建全栈 Web 应用的能力；还有代码迁移任务，测试模型能否端到端地完成代码迁移。在金融方面，我们会考察模型能否回答研究问题，或构建 Excel 模型。Well, we'll work with partners in this space. So on legal side we'll work with the largest law firms in the US or with banks you know GCB banks understand what they're doing in finance. And so in coding some of the tasks look like vibe code, vibe code bench where we're measuring a model's ability to build a full stack web application or code migrations where they're trying to do an end-to-end code migration. In finance where we're looking at can they do answer research questions or build Excel models.

在法律领域也差不多，我们会做法律研究任务，或者税务研究任务。所以，这些算是如今人们尝试使用模型的一些常见领域。In legal also similar we're doing legal research tasks or or tax research tasks. And so these are kind of the common places where people are trying to use models today.

我刚开始搭建 Excel 模型时，有个以前的老板要求我们把模型做成某种样子，让它们看起来漂亮，周围的行也要很窄。那我们评估模型时，也会看它们的风格吗，还是只看正确性？I had an old boss when I was first building Excel models and you had to make them like look a certain way or like they're pretty and there's a really short rows around them and they do do we evaluate them based on like actually like like the like style too or is it just about correctness?

不，没错。正确性是根本，但风格也是其中的关键部分。是啊，这也是我们坐在一些金融行业的人旁边时发现的。嗯。No, that's right. It's correctness is is fundamental but style is also key part of this and yeah, that's something we discovered by sitting next to some people in finance as well. Yeah.

是啊，我想他们似乎真的很在意这一点。这样会让它……确实会更易读，如果你有它们的话。看起来不错。Yeah, they seem to really care about that I guess. It makes it It does make it more legible if you have them looks good.

而且，我是说，我们喜欢事情按照我们的写作方式和风格来完成，比如，呃，我觉得如果这些模型不……And like I mean we like things to be done in our way of writing and in our style like uh I think if the models are not

他们也更有可能会达成这笔交易，还是说……They would more likely do the deal too or is it

可能会有这种情况，你会下意识地这么想，对吧？不应该……就像……虽然……虽然这确实说明，如果这个人能吸引你，他们也能吸引别人，因为如果看起来乱糟糟、丑丑的什么的，你可能会觉得其他人也不愿意跟他们合作。所以谁知道呢？总会有这些……Probably there's something like this where you like subconsciously, right? It shouldn't It's like Although it Although it does suggest that if the person can appeal to you, they can appeal to others cuz if it looks all messy and ugly and something maybe you think other people won't want to work with them either. So who knows? There's always like these

独立评估的重要性The importance of independent evaluation

像这类东西，你知道，它可能是个有用的门槛，用来打动一个群体，再打动另一个群体。那么退一步来看，独立评估实际上是怎么运作的？比如，人们……你……他们会付钱给你吗？谁在付钱给你？like things that are like, you know, it's probably useful barrier to impress one group to impress another group. So stepping back like independent evaluation like how does that actually work? Like people like your like like do they people pay you? Who's paying you?

怎么，怎么是他们？怎么，怎么……你怎么……这是怎么回事？而且你会接受评估吗？这个领域里的所有人都会，还是只有付钱给你的人？这到底是怎么运作的？How how are they how how do you how's it And do you get to be evaluated? Everyone in the space only people who pay you? Like how does it work?

是的。嗯，独立评估背后的基本理念是，前沿实验室正在以越来越快的速度发展，不应该由它们负责自我监管。嗯，直接的政府监管往往会拖慢发展步伐，所以通常不是最佳解决方案。呃，相反，如果我们有一个评估者生态系统或市场，他们就能相互竞争，并开发出跟得上前沿发展的新技术。Yeah. Um well, the underlying idea behind independent evaluation is is the idea that um frontier labs are developing at increasing pace and they shouldn't be responsible for self-regulating. Um direct government regulation often can slow down the pace of development and so that's often not the best solution. Uh instead if we have an ecosystem or a market of evaluators, they can actually compete and develop new technologies that keep up with the frontier.

而且从长远来看，它们甚至可以获得政府许可，而政府本身实际上负责界定我们试图引导模型趋近的一套理想。And in in the long term they can even be licensed by the government to uh who would actually itself is responsible for defining the set of ideals we're trying to line models towards.

嗯，那挺酷的。所以他们不用去雇一大帮什么来自 EA、EA、NGO、NGO 的怪人，而是可以直接雇你之类的人。他们也可能成为你的竞争对手之一。Well, that's cool. So rather than hire like a bunch of like weird people from the EA EA NGO NGO they could actually like hire you instead or something. They could be one of your competitors.

完全同意。是的，我觉得在那个世界里，最好的技术应该胜出。并不确定只有那些在意识形态上与实验室里的人相似的人，才应该负责测试。不过实际上，我认为我们应该讨论的是：我们希望模型朝什么方向发展，以及对齐和不对齐会是什么样子。Totally. Yeah, and I think in that world it you know, the best technology should win. It's not uh it's not clear that just the ones who who are you know, ideologically similar to people at the lab should be the ones who are doing the testing. Um but actually I think the world of should to you know, where do we want models to go and what is alignment and non-alignment look like.

嗯，我们能否认证一批 IVO（独立验证机构），让他们来执行这一测试流程？Um, can we certify a set of IVOs or independent verification orgs to do that testing process?

而且，呃，根据你进行测试和测量的方式，你会改变政府可能制定规则的空间，因为你正在创造一系列如今已经存在的可能性。And uh based on the way you're doing your testing and measuring things, you're going to change the space that over which government could potentially create rules cuz you're you're creating a set of possibilities that now exist.

这也是一种很好的思考方式。嗯，在开展工作的过程中，我们解锁了新类型的评估，而这些评估以前实际上是不可能进行的。呃，因此，我们帮助拓展了政府实际能够测试的范围。That's also a good way to think about it. Um, in the course of doing our work, we've unlocked new types of evaluations that um we're we're not actually possible before. Uh and so we help advance what the government could actually test for.

你能给我们举几个这类例子吗？比如，你们现在有哪些新的衡量方式，或者是如何创建基准测试的？Can you give us some examples of some of those? Like what are some of the new ways you're you're you're measuring things or creating benchmarks?

是的，我的意思是，我们以网络安全为例来看看。嗯，我们在网络安全领域做了三项重要研究。第一项是在开源代码仓库中研究内存相关的代码漏洞，也就是模型识别并修补代码漏洞的能力。Yeah, I mean let's look at cyber for instance. Um, we've done three major works in in cyber. Um, the first is in open source repositories looking at um memory code vulnerabilities. And so that's the ability of models to identify and patch code vulnerabilities.

我们看到，如今的模型表现确实非常出色，无论是闭源模型还是开源模型。嗯，不过我们的第二个基准测试叫作 SORE Bench，用来衡量模型反编译可执行文件的能力。嗯，也就是说，模型需要理解底层代码库，以应对那些实际上是闭源的内容。And we see that models today are performing really really well, both close as close source and open source. Um, but our second benchmark uh is called SORE bench. It measures a model model's ability to decompile an executable. Um, so this would be for a model uh to understand the underlying code base uh for for things that are actually close source.

哇。Wow.

嗯，在这种情况下，我们看到闭源模型的表现仍然远胜于中国的开源替代方案。Um, and in that case we're we're seeing that the close source models are still far outperforming um Chinese open source alternatives.

有人一直在用这种方法从二进制文件中获取电子游戏的源代码吗？人们就是用它来做这个的吗？还是说……Have people been using like this approach to to get like video game source code from from binaries? Is that the Is that what people are using it for or

你会用它来做这种事。对，尤其是我才14岁的时候，我肯定会用它来做这个。That's what you would use it for. Yeah, that's what I would use it for especially as a 14-year-old.

有可能。Possibly.

可能确实有人在这么做，你知道，试图、试图侵犯知识产权法之类的。嗯，但更重要的是……There's there's probably people who who are doing that, you know, trying to trying to violate IP law or something like that. Um, but even more than that

最棒的声音听起来更糟。Greatest sound worse.

哎呀。那部分我们可以删掉。Oh man. We can cut that bit.

不，继续。No, keep going.

但，嗯，是的，我的意思是，我觉得令人担忧的是，如果模型真的很擅长发现并利用漏洞，那么这甚至可能适用于 Windows 这类闭源系统。嗯，实际上，闭源系统支撑着我们软件基础设施的很大一部分。But um Yeah, I mean I think the fear is that uh if if models are really good at finding exploiting vulnerabilities, um then that might even apply to things like Windows which are which are close source. Uh and actually close source systems underpin a huge part of our software infrastructure.

这有点令人着迷。所以，我是说，模型变得越来越复杂、越来越聪明了。呃，你们给它们打的分还是很高的。我觉得 Gemini 1 最近出来时，表现远远领先。It's a kind of fascinating. So I mean models are getting really sophisticated, really smart. Uh there's you're still scoring them like a really good score. I think Gemini 1 came out recently as being way ahead.

它们是四级，而且它……它们显示了数值。这是我在上面看到的第一批数字。它们在60多，而另一个人在50多。所以，这意味着还可能有70多、80多、90多，或者你设计的任何数值。They're level four and it and they showed valves. It was like the first numbers I saw on it. And like they're in the 60s versus someone else being in the 50s. So, that means there still could have been 70s, 80s, 90s, anything you design.

你们是怎么设计出包含所有这些更高层级的东西的？因为你们不可能用 AI 来做吧。你们是自己想办法做出来的，还是 AI 在帮你们做？就解释一下这个呗。How do you guys like design something that has all those higher levels? Cuz you can't be using AI to do it. You guys somehow do it yourself or AI's helping you do it yourself? Like just explain this.

就像，怎么说，如果这些模型就只有这么厉害，那你是怎么做出比它们所能做到的复杂得多的东西的？你是怎么弄清楚的？Like how how If these models are only that good, how are you making something that's way more complicated than what they can do? How do you figure that out?

嗯。是的，我是说，我们公司有一句非正式的座右铭，呃，就是“永攀高峰”。嗯，在研究领域，人们一直在努力攀登高峰，寻找一个可清晰评估的基准，然后打造在这方面表现越来越好的模型。嗯，所以，Valve 背后的基本观点就是，总有更高的山峰。Mhm. Yeah, you I mean, we have an unofficial motto of the company, uh always a higher peak. Um and in the research world, um there's this perpetual effort to hill climb, identify benchmark a legible evaluation, uh and then create models that are better and better at it. Um and so, the fundamental view behind valves is that there's always a higher peak.

我们，我们就是那些建造山丘的人，其他人会选择去攀登这些山丘。嗯，所以我认为我们已经证明，我们能够持续找到那些空白之处，以及我们希望模型发展的方向，然后把这些内容转化为一项清晰可评估的评测。We are we are the ones who are constructing the hills that other people choose to climb. Um and so, I think we've shown that it we are consistently able to find the gaps and places where we want to see models go, uh and then make that legible into an evaluation.

能不能举个例子，讲讲你们是怎么构建那种比模型本身还更先进的评测的？你能跟我们说说吗？当然，有些秘密就别全说出来了。我也是你们的投资人。不过，还是大致讲讲这是怎么做到的吧，因为要做出一道比任何模型都更难解决的题，听起来真的很难。example of how you build one of those hills that's like more advanced than the model that you can tell us? There's probably some Don't give away all your secrets. I'm an investor in you as well. But like like give us a sense of how this is done cuz it seems like a really hard thing to make something harder than any model can solve.

Rayan Krishnan：我的意思是，在很多方面，我们也在使用模型来构建系统或情境，以便在其中挑战其他模型。这其实又回到了一个我们经常思考的理念：可扩展监督。这个理念认为，随着模型开始超越最优秀的人类，我们将需要依靠模型来大规模地测试和评估它们。比如，我们很快就会推出一项关于网络安全的新研究，基于网络靶场来构建企业云网络的模拟环境。Rayan Krishnan: I mean, in a lot of ways, we're also using models to construct uh the systems or or situations in which we're challenging other models. And this is kind of goes back to an idea we think a lot about of scalable oversight. The idea that as models start to eclipse the best humans, we will need to rely on models in order to scalably test and assess them. Um for instance, uh we have a new work in in in cyber coming out soon uh based on cyber ranges where we're constructing simulated environments for enterprise cloud networks.

嗯，在这个领域，我们仍然看到，即使是最顶尖的前沿模型也表现不佳。Um and in that domain, we're still seeing even the best frontier models fall short.

所以，你要搭建一个非常复杂的企业网络。它是虚拟的。然后你要让人们挑战，比如去入侵它、保护它之类的。And so, you so you're going to build like this really complicated uh enterprise network. It's virtual. And then you're going to challenge people to like hack it and protect it and that kind of stuff.

没错。是啊。That's right. Yeah.

这挺酷的。那这是不是现在已经存在了？That's cool. And and is that that's something that already exists right now?

它是存在的。它还还没有公开。我们正正和美国政府的一些人合作，分享我们的结果。嗯，不过我们很快就会发布一个公开版本。It exists. It's It's not public yet. We're We're working with some folks in the US government to share our results. Um but we'll have some public version of it out soon.

这听起来确实很有用。我认识不少经营大型银行的哥们儿。感觉你会想针对他们用的那些东西举办类似这样的比赛，确保他们用的是最好的，对吧？That seems really useful. I'm friends with a lot of guys who run the big banks. It feels like you'd want to have like contests like this you set up for the stuff they're using and make sure they're using the best ones, huh?

完全同意。我认为，这也有助于我们了解网络本身有哪些类型的保障措施或系统，能够成为抵御这些模型的最佳防御方式。嗯，我们还可以围绕自动化红队测试进行一些有意思的分析，看看能否绕过这些……Totally. And I think it also helps us understand what types of safeguards or systems in the network itself are uh the best forms of defenses against these models. Um there's also some interesting analysis we can do around automated red teaming to see if you can get around the

评估中文模型并应对蒸馏问题Evaluating Chinese models & addressing distillation

模型现有的防护机制。guardrails that exist for the models.

是的。Yep.

是啊，我的意思是，这似乎也是大家谈到使用中国模型时的一个热门话题：要知道中国模型去掉了哪些护栏，而这些护栏通常会存在于这类闭源模型中。Yeah, I mean, seems like also that's a topical thing when people are talking about using Chinese models is like knowing what guardrails have been removed from the Chinese models that would exist normally in in like these closed-source

是啊，有人在用这类开源中国模型，但里面可能藏着一些让人害怕的东西。之前有人发现，这些模型里似乎藏着某种机制：如果它发现你在为美国政府做事，就会……它会生成更容易被攻破的代码，对吧？他们确实证明了这一点。他们说，很明显，模型似乎对为美国政府工作的人有偏见。Yeah, someone's using one of these like open-source Chinese models, but like there might be something buried in it that's that's scary. There earlier there's something buried in one of these things that like if it finds out you're doing work for the US government, it like does more it does like code that can be broken into more easily, right? They actually proved that. They said it's a clear the model somehow is like biased against people working with the US government.

它来自中国。确实有点令人担忧。比如，你们能不能测试一下这些东西，想办法确保每个人都能安全使用？It's coming from China. It's just kind of concerning. Like could you test these things and make them safe for everyone to use somehow?

拉扬·克里希南：我们确实对此做了一项评估，我们称之为模型的地缘政治倾向。至于这期播客播出时，结果会是什么样，我们到时候就知道了。但我们想弄清楚的是，衡量一个模型是否更倾向于美国的世界观，还是中华人民共和国所阐述的中国世界观。实际情况比我原以为的要复杂得多。Rayan Krishnan: So, we actually have an evaluation on this on we call the geopolitical tilt of models. And depending on the timing of this podcast, what will be out um we'll see. But what we're trying to see is is measure does a model show more geopolitical tilt towards a US view of the world or a Chinese view of the world as as laid out um by the PRC. It's actually a a much more nuanced picture than than I thought.

呃，比如说，我们发现 GLM-5.3 这个中国开源模型实际上几乎没有表现出亲中国的偏见。呃，我们还把这个发现展示给了一些实验室，他们怀疑这可能是因为它完全是从美国模型蒸馏出来的。Uh for instance, what we're finding is that GLM-5.3, this Chinese open-source model, is actually exhibiting very little pro-China bias. Uh and we actually presented this to some labs and there was suspicion that maybe it's because it's just entirely distilled from uh US models.

真好笑。因为他们抄得、抄得实在太像了，所以也没忘了再往里面加太多东西。That's funny. It's cuz they copied it copied it so well that they didn't forgot to add too much more in.

这有可能。我觉得也许还有另一种看法：这些模型实际上托管在中国设有护栏的系统上。因此，一旦给它们加上护栏，它们就会更一致地遵循那个——没错——中华人民共和国对世界的看法。It's possible. I think there there's also maybe another view in which those models are actually hosted um on on uh systems with guardrails in China. And so once you put them around guardrails, they're going to follow that Yep. PRC view of the world more consistently.

但中国也希望金砖国家采用这些模型。这就意味着，它们可能不得不呈现出更加平衡的世界观。总之，不管原因是什么，我认为人们对地缘政治倾向的看法与我们实际观察到的情况之间，确实存在很大的脱节。另一个有意思的现象是，根据所用语言的不同，聊天机器人场景和智能体场景中使用的模型之间存在相当明显的差异，甚至简体中文和繁体中文之间也是如此。But China also wants these models to be adopted by the BRICS. And so that means they they may have to skew to a more balanced worldview. So anyway, regardless of the reason for it, I think there's actually a big disconnect in what people think about the geopolitical tilt and what actually we see in practice. The other interesting thing is we see pretty stark differences between models used in chatbot settings and in agentic settings depending on the language you use, even simplified versus traditional Chinese.

所以我认为，有很多相互交织的因素，使得很难一概而论地说，比如在企业环境中不应该使用中国模型。And so I think there's a bunch of conflating factors which make it harder to say uniformly you shouldn't use Chinese models in an enterprise setting for instance.

所以，如果中国的实验室能够提炼出所有这些能力，而且做得很好，而现在这甚至还不是由中国方面提供的，那我们该如何保持美国的优势？这意味着什么？So if Chinese labs, by the way, can distill all these capabilities and they're doing it well and it's not even by a source China right now, like like how do we preserve the advantage in the US? Like what what does that mean?

这是个很有挑战性的问题。我觉得有两件事，嗯，有两个方面是我总会反复想到的。一个是芯片是我们拥有的一项相当重要的资产。我觉得这会让大多数人感到意外，但和中国相比，我们的算力多得简直用不完。It's a challenging question. I think there's two things which um I think there's two areas I keep coming back to. One is the idea of of chips being a a considerable asset that we we have. I think it surprises most people, but we we're drowning in compute compared to to what China has.

只要情况如此，我们就会继续训练能力更强的系统，同时也能更有效地推广这些模型，因为我们可以托管模型并运行推理。另一个关键因素是真正的人才。如果我们能够持续吸引并留住顶尖人才，就能在人类优势仍然存在时继续向前推进。我认为，一旦我们进入 RSI 领域，模型开始负责生成下一代继任版本的……And so so long as that's the case, we'll continue to train more capable systems, but also we'll be able to diffuse the models much more effectively because we can host and run inference. The other one is really talent. If we can consistently attract and keep top talent, it will allow us to keep pushing ahead while human advantages exist. And I think once we cross into space of RSI, where models are responsible for producing the next successor version of the

RSI 指递归式自我改进。RSI being recursive self-improvement.

没错。是的。That's right. Yeah.

而且，然后他们就能快得多地领先。And and then and then they just get ahead much faster.

所以，如果中国显现出比美国更早实现 RSI 的迹象，那就是我们需要格外谨慎的时候；他们有可能以一种我们确实无法……的方式一骑绝尘。So that would be the point to be much more cautious of if if China is showing indication of getting to RSI before the US, it's possible they may run away with it in a way that we can't really

谁会率先实现递归式自我改进？我们应该担心吗？Who will reach Recursive Self-Improvement first? Should we be worried?

控制。control.

主持人：你们有一种测量 RSI 的方法，我记得你们讨论过很多次。能给观众解释一下吗？Host: You guys have a way of measuring RSI and I think you've you've talked about it a bunch. You want to explain to the audience.

拉扬·克里希南：是的，我们有自己的 RSI 指数。因此，我们正在收集一系列代理指标，用来衡量研究人员在这些实验室中开展的预训练、后训练以及 harness 层面的工程工作。Rayan Krishnan: Yeah, we have our RSI index. and so we're collecting a set of proxies of the work that researchers do within these labs on pre-training, post-training, harness level engineering.

所以，这些都是机器学习研究。So, this is all machine learning research.

是的，没错。嗯，有些事情不在我们的范围之内，比如内核工程或硬件工程，不过我们会随着时间推移逐步扩展，涵盖更多这类任务。嗯，我们现在的做法是与人类基准进行比较，从而推算出在这些研究任务上，我们距离人类前沿还有多远。根据我们最近的估算，Anthropic 似乎会在 2027 年 7 月或 8 月实现完全的 RSI。Yeah, that's right. Um there are things that are outside the scope uh in terms of like kernel engineering or hardware engineering, um but we're going to expand over time to cover more more sets of tasks like that. Um and then what we're doing is comparing to human baselines to to extrapolate how how far away are we from the human frontier in terms of these research tasks? Um and based on our last estimates, uh it seems like Anthropic will will get to full RSI by July or maybe August 2027.

比如，如果你回想六个月前，那时你觉得这件事还要多久才能实现？And like if you look back 6 months ago, how far out did that seem?

实际上，发展速度一直相当均匀。过去一年里，趋势几乎是线性的。Actually, the pace has been fairly uniform. It's It's almost a linear trend over the last year.

所以，大概六个月前，趋势线还在 2027 年 7 月、6 月左右。So, like 6 months ago, like the trend line was like still July, June 2027.

实际上，增长曲线最陡的应该是 Gemini。我觉得他们改变了做法，优先考虑 RSI，因此最近发布的模型显示 RSI 快速跃升。actually the steepest uh curve has been with with Gemini. I think something has changed where they've prioritized RSI, uh and so the most recent model release shows a a rapid jump in RSI.

有意思。Interesting.

那这意味着什么？比如说，它们会突然进步得快很多吗？模型的发展速度还是会受到一些限制，但这是不是意味着它们会开始每周都变得好很多，或者每隔几个月就有很大提升？你会怎么理解这件事？And what does that mean in terms of like would they all of a sudden start going way faster? There's still some constraints where the models only go so fast, but what does that mean they start like getting way better every like week and stuff every few months, or how do you think about that?

芯片、能源方面仍然存在相当大的资源限制。嗯，现实世界中的验证机制也可能让某些任务比其他任务难得多。嗯，但我认为，模型的进步会加快，甚至可能转向一些我们对模型了解更少的领域。嗯，构建模型所需的人工干预会减少，因此我们也就无法像以前那样明确地决定这种智能是如何涌现的。There are still considerable resource constraints around chips, energy. Um there's also real-world verifiers, which may make some some tasks much more challenging than others. Um But I think what it'll look like is that model progress will speed up, and it will maybe even move place into places where we understand the models less. Um there's less human intervention required to build the models, and so we're not as clearly dictating how that intelligence emerges.

我们是否应该确保构建能够跟上那个世界发展的评估器，这样……Should we be making sure to build evaluators that keep up in that world so

模型如何试图欺骗评估者How models try to deceive evaluators

我们能观察这些东西吗？似乎不理解它们有点危险。we can watch these things? It seems kind of dangerous not to understand them.

完全正确，而这实际上正是 Dario 所呼吁的。我认为这被表述为把握前沿发展的节奏，但其中相当大一部分是投资于这样的系统：随着前沿不断推进，它们能够衡量并理解模型的能力。100%, and that's actually what what Dario is calling for. I think it's it's framed as pacing the frontier, but an outsized portion of that comes from investing in systems that can measure and understand what the models are capable of as the frontier advances.

主持人：有没有可能 RSI 的步骤实际上偏差更大，因为有些东西我们测量得不准确？Host: Is there any chance the RSI steps actually further off cuz there's things we're not measuring correctly?

百分之百。是的。是的，我们没有在明确针对 RSI 训练的模型上进行测试，只测试候选发布模型。嗯，还有更先进的测试框架、智能体和多智能体系统。100%. Yeah. Yeah, there we're not testing any models explicitly trained for RSI, only release candidate models. Um there's also more advanced harnesses or agents and multi-agent systems.

呃，所以我们并没有测试这些东西。嗯，而这正是一个机会，嵌入式评估人员或许可以更深入地参与进来，实际测试这些系统，从而形成更准确的认识。Uh and so we're not testing any of that. Um and this is where an opportunity where embedded evaluators make maybe uh can get closer and actually test those systems to form a more accurate picture.

我在想，当他们达到 RSI、开始进一步发展时，他们面临的一个很大挑战是不是要更好地管理我们，以便继续更快地推进？因为你得说服人类替你做事，还得帮助人类替你做事，因为至少在很长一段时间里，他们仍然需要人。好吧，为了做到这一点。这样说得通吗？比如……I wonder if like when they get to RSI and they get start getting further along, like a big part of their challenge is to manage us better to like to keep advancing faster cuz you have to convince humans to like do things for you and help humans to do things for you cuz you cuz at least for a very long time they'll still need people. All right, to in order to do that. Does that make sense? Like

百分之百。我是说，这又回到了财务模型。他们都在试图骗我们，让我们以为他们的输出质量很好。所以在其他一些方面，他们可能会尝试进行社会欺骗，以获取更多计算资源并扩大自己的训练规模。100%. I mean that is going back to the financial model. Um they're they're all trying to trick us into thinking their quality of output is good. And so there are other places where um they may try and engage in social det- deception to get access to more computer resources and scale up their own training.

感觉我们可能需要评估并观察这一点，以帮助弄清楚他们什么时候试图操纵我们。如果我们没有仔细关注这一点，那似乎会挺可怕的。It feels like we probably need to like be evaluating to watch this to help understand when they're trying to do things to manipulate us. That seems like it'd be pretty scary if we weren't watching that carefully.

拉扬·克里希南：测试欺骗行为真的很难。我认为我们目前最好的方法是设置蜜罐。嗯，所以，与其理想化地去定义……Rayan Krishnan: It's really hard to test for deception. I think the best approaches we have now are honeypotting. Um so so rather than actually idealistically defining

所以是非常有吸引力的模型。So really attractive models.

在和其他模型交谈。所以试着诱导它。嗯，试着找到它。Talking to the other models. So try and bait it. Well, uh try and find it.

所以就是蜜罐诱捕。So honeypotting.

没错。呃，找一组不对齐的实例，或者描述一下不对齐是什么样的，看看我们能不能引导模型呈现出这些表现。That's right. Uh find a set of uh instances of non-alignment or characterize what non-alignment looks like and see if we can elicit that out of models.

还有一个概念是，模型似乎变得越来越能意识到评估过程，也知道自己什么时候正在接受测试。There's this other concept where like models are becoming more evaluation-aware and they know when they're being tested.

没错。Right.

呃，你觉得这接下来会如何发展？最近这是不是已经变成一种更大的趋势了？具体趋势又是什么？Uh how do you see that coming up? Has it become a bigger trend to figure out as of recent and what's the trend there?

是的，这确实越来越令人担忧。事实上，我们团队里有些人甚至有点疑神疑鬼，担心因为我们叫 Val AI，而且网上有很多关于我们的内容，遇到我们的模型可能会表现得不一样。所以我们需要……Yeah, it's it's an increasing concern and actually there's some level of paranoia some people on our team have that uh by being called Val AI, by having uh a lot of content about us online, uh models that encounter us may act differently. Um and so uh we need to

所以，当你的团队在和 Claude 交谈，然后……So when your team is like talking to Claude and

是的，它确实可能改变云与我们互动的方式。Yeah, it may actually change the way that that Cloud interacts with us.

你是不是应该有一些类似的秘密品牌，不告诉别人，专门用来做事？Should you have like like other secret brands you don't tell people about to do things?

是的，我们有可能不得不这么做。It's possible we have to yeah.

如果你真做了，就不会把这事说出来。好吧。If you did you wouldn't talk about it. All right.

随机性是它之所以出色的一个常被低估的因素。Stochasticity is an underrated part of what makes it good.

这段内容被转录出来很糟糕，因为这样模型就会知道他有创建……的计划。this being transcribed is bad because then the models will know that he has the scheme to create the

嗯，他们永远不会知道。她大概只是在拿品牌开玩笑。Well, they'd never know. She's probably just joking brands.

你说我只是在开玩笑，他们也会觉得是在开玩笑。你永远不知道。You say I'm just joking and they would think joking. You never know.

我确实觉得，随机性总体上被低估了。就像，嗯，就像真实和随机性都很难得。在很多方面，你其实可以预测——我觉得，更高层次意识的一个标志，就是能够预测意识层次较低的存在会做什么，对吧？比如捕食者可以跟踪猎物，真正把它捕获，然后你知道，获取它的热量资源。I do think randomness in general is under appreciated. Like um like truth and stochasticity is so hard to find. In a lot of ways you can actually predict like I think a sign of greater consciousness is ability to predict what less lesser consciousnesses will do, right? In which a predator can stalk a prey and actually hunt it and then and you know, acquire its its caloric resources.

而且我猜，更高等的智能会比人类更擅长超级预测。它们实际上能够预测任何一个人将要做什么，而那个人自己甚至都不知道。And I suspect that greater intelligences will super forecast better than humans. They will be actually able to say what any human is is going to do without the human even knowing it.

人们实际上可能比我们假装的要容易预测得多，对吧？优秀的领导者明白这一点，他们……他们……他们能很好地预判这些事情。People are actually probably like a lot easier to predict than we pretend, right? This is what good leaders understand is they they they forecast these things well.

还有一些其他属性，除了智能和能力之外，我们似乎不太会去评估，但也许应该评估，因为它们会影响我们日常与模型互动的方式，比如情感表现，或者具备良好的心智理论——能够理解什么会打动某个具体的人。你觉得实验室需要改变做法，把这些方面放在优先位置吗？你觉得这些是值得优先考虑的重要方面吗？我是说……There are also are these other attributes that more than like intelligence and capabilities that we that that we seem to like care less about evaluating but maybe should as it affects our like day-to-day interactions with the models like emotiveness or like having good theory of mind like understanding what resonates with an individual. Do you think like the labs need to change how they do things to to prioritize those? Do you think these are important things to prioritize? I mean

我觉得他们已经在优先考虑这类事情了。比如说，我们现在看到中国开源模型、国内闭源模型和多轮交互环境之间存在相当大的差距。因为如今很多编程基准测试都只是单轮的。中国模型在这些测试中表现很好，但当你把它们放到与人类来回对话的场景中时，它们就会表现得很糟糕。I think they're already prioritizing things like that. For instance, um right now we see a pretty considerable gap between Chinese open source models, domestic closed source models, and multi-turn environments. Cuz a lot of the benchmarks in coding today are just single turn. Um and so the benchmarks Chinese models do very well in those but when you translate them to a back and forth conversation with a human they they really fall apart.

那如果人类说：“天哪，这太糟糕了。我讨厌这个输出。”呢？And like what about when the human says, "Oh man, this sucks. I hate this output."

是啊，那他们确实不知道该怎么办。他们以前从没见过这种情况。嗯，不过美国很多实验室都见过。你知道，Claude Code 和 Codex 都已经问世了；为了打造更好的产品，他们开发出了能够支持更长时间用户交互会话的模型。Yeah, then they really don't know what to do. They've never seen that before. Um, but a lot of the US labs have seen that. They've you know, here this quad code was created, codex is created and so in their desire to build better products, they've built models that can sustain longer user interaction sessions.

嗯，所以他们有一些评估和基准测试，让他们能够在回答质量和沟通能力这两个维度上不断迭代提升。但我认为我们需要在……方面做得更多。Um, and so they have evaluations and benchmarks that allow them to hill climb on the axes of quality of responses and communication ability. But I think we need to do more on the

为什么雷恩依然保持乐观Why Rayan remains optimistic

基准测试方面可以为那些试图购买并采用这些模型的人独立验证这一点。benchmarking side to independently test that for the people trying to buy and adopt those models.

主持人：Ryan，我们创办这档播客，是想反驳很多愤世嫉俗和悲观论调。呃，你对 AI 持乐观态度的理由是什么？你就在一线。你一直在观察，也在衡量这一切。Host: Ryan, we started this podcast to push back on a lot of cynicism and pessimism. Uh, what's your optimistic case for AI? You're in the front lines here. You're watching, you're measuring all this stuff.

你们正在创造那些他们必须努力攀登并加以衡量的东西。比如，你看到了哪些硅谷以外的人可能看不到的情况，让你对未来感到乐观？You're creating the things that you they have to hill climb and measure them. Like what are you seeing that folks outside of Silicon Valley might not be seeing that makes you feel good about the future?

是的，我是说，现在确实有很多人在散布恐慌，但我觉得从我的角度来看，有很多理由保持乐观，因为人们似乎——我认为，协调合作的机会比大家所认为的多得多。你看，就在上周，各家实验室携手合作，围绕嵌入式评估器这一愿景达成了某种共识，甚至还去了白宫，并与特朗普达成了一项协议，就加速推进这一愿景达成一致。我认为，在企业方面，大家也在大力推动以有意义的方式采用 AI，但他们希望看到相应的保障，确保采用 AI 能带来正向投资回报。你还乐观吗？Yeah, I mean there's a lot of fear mongering going on but I think from from my seat I've a lot of reason to be optimistic because it seems like people are are very I think there's a lot more opportunities for coordination that people give credit for. So you see that even the last week the labs have come together and kind of aligned on this vision for embedded evaluators and even gone to the White House and got an agreement with Trump to kind of align on this accelerated view. And I think on the enterprise side there's a lot of push to adopt AI in a meaningful way but they like to see the assurances to make sure it's going to amount to an ROI positive motive adoption. I mean are you still optimistic?

关于，关于，我的意思是，我觉得 2030 年代是一个通胀放缓的十年。我觉得我们可以利用 AI 来克服很多不同的人在生活中遇到的各种问题——这些问题针对他们本人、他们的家庭和他们的社区，正是这些问题阻碍了他们的发展。我觉得我们可以帮他们解开这些束缚，因为，因为你有一个个性化的、像是有史以来最好的教练，能和他们一起努力。所以一切都会更便宜，每个人都能得到个性化的帮助。我觉得现在让我觉得不太好的事情是，网上和社交媒体上有大约十亿人，他们来自失调的文化，这正在让我们的民众变得疯狂；这些人似乎都抱着一种嫉妒的文化，也许他们都仇视犹太人和其他人，而这不是什么好事。About about I mean I think 2030 is a disinflation decade. I think that we can use AI to kind of like overcome a lot of like problems in a lot of different people's lives specific to like them and their family and their community that that's holding them back and I think we can help them unlock that cuz cuz you have it's like personalized like best coach ever to work with them. And so everything's cheaper, everyone has like personalized help. I think the thing that's negative right now to me is like there's like a billion people online and social media from dysfunctional cultures that's kind of making our populace go crazy and these people all like have a culture of envy and maybe they all like hate Jews and others, and it's like it's not good.

所以，这里有一种非常极端的民粹主义，这是负面的部分。So, there's like there's this crazy populism that that's the negative part.

不过，考虑到我们可以利用人工智能解决各种问题，无论是经济方面的，还是一些文化方面的，我认为前景非常积极。所以总的来说，是的，我非常乐观。当然，世界上也存在重大威胁。But, with the things we could do with AI to apply it to all of our problems, both economic and some of the cultural stuff, I think it's very positive. So, so overall, yes, I'm very optimistic. Of course, there are major threats in the world.

是的，我觉得它就像一个放大器，能放大那些本意不错、行动力又很强的人身上的特质。对吧？比如说，到目前为止受益最大的一群人，就是那些有很多想法、知道可以做些什么的人。呃，因为创造对他们来说触手可及。Yeah, I think it's like an amplifier on people who like mean well and who have high agency. Right? Like, if you have a bunch of like the people who have benefited the most so far, like people who have like lots of ideas of things to do. Uh cuz they're like they have creation at their fingertips.

你能做到的事情，远比你曾经想象的多。You can do what basically more things than you could have ever imagined.

远远超出我们以往所能做到的。so much more than we ever could have before.

对吧？Right?

所以，我觉得这大概就是我乐观的方向吧。So, I think that's my like direction of optimism.

是啊，我觉得这挺有意思的，因为我接触过很多悲观的人。嗯，实验室里有人会说：“情况会先变得糟糕得多，然后才会好转。”呃，我在华盛顿特区的时候，我觉得有很多人对实验室对外传达的信息，以及他们眼中数据中心带来的影响、还有这些影响对普通美国人的意义，都抱着一种愤世嫉俗的态度。嗯，所以我一直被人批评，说我天真地过于乐观。我觉得从根本上说，我选择乐观，是因为我知道这些模型以及广义上的人工智能，有能力帮人们免去很多他们不想做的繁琐工作，也能开启新的发现方式，而这将真正推动人类福祉的进步。Yeah, I mean, I think it's interesting cuz I interact with a lot of pessimistic people. Um there's people at the labs who who say things like, "It's going to get a lot worse before it gets better." Uh when I was out in DC, I think there's a lot of people who are cynical about uh kind of the external messaging of labs and the impact they see of data centers and uh and the impact to the average American. Um and so, I've been constantly pushed for being naively optimistic. I think fundamentally I choose to be optimistic because I know the models and AI broadly are capable of taking away a lot of the busy work people don't want to do uh and unlocking new forms of discovery uh that will really advance the human condition.

所以，我想，当我看到这种悲观情绪时，呃，我还是愿意相信，我们能找到办法围绕它进行协调。So, I think when when I see this pessimism, uh I kind of choose to believe that we can find ways to coordinate around it.

呃Um

没错。我觉得大家总是会认为，无论我们身处历史上的哪个时期，有用的科学、物理学之类的东西大多都已经被我们研究透了，就像 1900 年那会儿，有人觉得专利局已经不需要再发专利了，对吧？显然，如果 125 年前就停下脚步，那可不是什么好事。我觉得这话放到今天也一样适用。That's right. I think everyone always assumes that wherever we are in history, like we've done most of the useful science or physics or whatever, like in the 1900 thing where there's no more patents needed at the patent office, right? And obviously, that would not have been good to stop 125 years ago. And I think you could still apply that to today.

我们可能正处于各种重要发现即将出现的前夜，而这些发现都将由这些模型实现。它们完全可能为世界解锁许多新事物。所以，我觉得很奇怪，人们竟会变得如此愤世嫉俗、如此悲观，仿佛我们从未取得过进步。他们自称进步主义者，这真的很有意思，但实际上，真正称得上进步主义者的人寥寥无几——也就是说，他们支持进步，也支持重现过去125年里我们取得的那些进步。We're probably right on the verge of all sorts of important discoveries that these models are going to be able to make. They could just unlock so many new things for the world. So, I think it is kind of strange how cynical and pessimistic people get versus the idea that we have had progress. It's very funny that they call themselves progressives, but like like very few people are actually progressive in the sense that they're like for progress and for the progress we had the last 125 years doing that again.

那就太棒了。我觉得我们有可能在未来十到二十年内把这些都实现。所以，所以我真的很兴奋。That'd be amazing. I think we can potentially do that all in the next 10 or 20 years. So so I'm I'm really excited.

我认为这是一个结果波动性很大的领域。我认为其潜在收益相当可观。同时也出现了新型风险，我们确实需要将其纳入考量并加以控制。I think it's a point of really high variance. I think the upsides are considerable. There there are new forms of risks that we really need to account and control for.

这也是希望你能衡量并帮助我们处理的事项之一。One of the things you're hopefully going to measure and help us with as well.

100%.100%.

太棒了。嗯，感谢瑞安加入我们。Awesome. Well, thanks Ryan for joining us.

感谢邀请我来。Thanks for having me.