Anthropic 研究员 Emmanuel Ameisen：语言模型究竟如何运作

简要说明Brief Description

埃马纽埃尔·阿迈森探讨了可解释性研究人员如何研究语言模型内部的表征与计算、他们对规划和模型行为有哪些发现，以及这项工作如何与人工智能对齐相联系。Emmanuel Ameisen discusses how interpretability researchers study the representations and computations inside language models, what they have learned about planning and model behavior, and how this work connects to AI alignment.

目录Table of Contents

• Emmanuel Ameisen 简介
  Introduction to Emmanuel Ameisen
• 为什么说 AI 模型是“培养”出来的，而不是“建造”出来的
  Why AI models are grown, not built
• 为什么下一个词元预测比听起来更难
  Why next-token prediction is harder than it sounds
• 模型如何表征概念并理解世界
  How models represent concepts and understand the world
• LLM 如何提前规划
  How LLMs plan ahead
• 人工智能可能具有意识吗？
  Could AI be conscious?
• 规模定律与文本的力量
  Scaling laws and the power of text
• 为什么平稳进展会带来突然的突破
  Why smooth progress creates sudden breakthroughs
• 为什么仅仅观察 AI 并不足以解决对齐问题
  Why watching AI isn’t enough to solve alignment
• 金门 Claude 实验
  The Golden Gate Claude experiment
• Anthropic 为何认真对待 AI 安全AV片
  Why Anthropic takes AI safety seriously
• 拟人化 AI 何时有用
  When anthropomorphizing AI is useful
• 人工智能能教会我们什么有关智能的知识
  What AI can teach us about intelligence
• 对 AI 对齐持乐观态度的理由
  Reasons for optimism about AI alignment
• 我们生活在一个模拟世界中吗？
  Are we living in a simulation?
• Jev、决策模型与 AI 的直觉
  Jev, decision models, and AI’s gut instincts
• 关于模型如何思考的另外两个令人惊讶的发现
  Two more surprising findings about how models think
• 如何进入可解释性研究领域
  How to get into interpretability research
• 快速问答环节与最后感想
  Lightning round and final thoughts

Emmanuel Ameisen 简介Introduction to Emmanuel Ameisen

主持人：如今人们最关心的问题之一，就是人工智能做坏事。作为一个观察这些模型思考和运作的人，为什么我们这么难发现它们在产生不良想法、试图欺骗我们？Host: One of the big questions on people's minds these days is AI doing bad things. As someone that watches these models think and operate, why is it so hard to catch it thinking badly trying to deceive us?

嘉宾：我们训练 Claude 做到有帮助、无害且诚实。它是不是表现得很好，但如果你看看它的内心，就会发现它其实是在想：“哦，我现在表现得好，只是因为 Lenny 正在看着我。”Guest: We trained Claude to be helpful, harmless, and honest. Is it the case that it's behaving well, but if you look inside, actually, it's like, oh, I'm only behaving well cuz Lenny's watching right now.

主持人：人们的全职工作就是研究这个模型，了解它如何思考、如何运作。这项技能成为一种职业技能有多久了？Host: People's full-time jobs is studying the model, how it thinks, how it operates. How long has this been a skill set in a job?

嘉宾：如果你喜欢使用语言模型，或者会思考它们，那么最重要的一点之一就是要意识到一个非常奇怪的事实：制造它们并不意味着理解它们。这些模型所做的不过是对大量数字进行许多数学运算。你仔细一看，就会觉得这完全说不通。Guest: One of the most important things to realize if you like use language models or think about them is just this like very bizarre fact that to make them is not to understand them. All these models do is just a lot of mathematical operations on many numbers. If you look at it, it makes no sense.

你手头只有一组100个数字。模型在做数学运算，然后你只需试着看看其中有没有什么规律。You just have a list of a 100 numbers. The model's doing math and then you just try to see if there's patterns.

主持人：你是在用我们研究大脑运作的方式来摸索它：寻找规律。你展示这张图像。Host: You are figuring it out the same way we figure out how the brain works. Looking for patterns. You show this image.

大脑的这一部分会被激活，而你正在用 AI 做完全相同的事情。哇，这真是太有意思了。This part of the brain lights up and you're doing the exact same thing with AI. Wow, this is so fascinating.

嘉宾：我们当时在训练一个模型。它会变得很疯狂，还会做一些数学运算。Guest: There was this model we were training. It would go crazy. It would do some math.

它会得出一个答案。我记得当时是12。然后它马上又会回过头来说：“哦不，天哪，答案其实肯定是36。”它又会重新算一遍。It would get to an answer. I think it was 12. It would immediately go back and say like, "Oh no, oh my god, the answer actually is for sure 36." It would do math again.

它会达到12。慢慢地，它会开始陷入疯狂，用全大写字母写东西，字面上说：“我被恶魔附身了。”It would get to 12. Slowly, it would like start descending into madness, writing in all caps, literally saying, "I'm possessed by a demon."

主持人：人类，说到底，基本上也就是下一个 token 预测器。我们只是在琢磨接下来该说什么。Dario 总是在外面活跃着。Host: Humans, if you really boil it down, like we're basically next token predictors ourselves. We're just trying to figure out the next thing to say. Dario is always out there.

这真的很危险。我们必须格外小心。可大家都觉得：“不，他这么说只是为了营销。”而现在看来，等等，这真的很危险。要认真对待这样一种观点：这项技术会不断进步。This is really dangerous. We got to be really careful. And everyone's just like, "No, he's just saying that for marketing reasons." And now it's like, wait, it's really dangerous. Take the idea that this technology is going to keep improving seriously.

主持人：你会把意识也视为其中的一部分吗？你觉得人工智能有意识吗？Host: Do you think about consciousness as a part of this? Just like do you think AI is conscious?

嘉宾：噢，天哪。Guest: Oh man.

为什么 AI 模型是“培养”出来的，而不是“建造”出来的Why AI models are grown, not built

主持人：Emmanuel，非常感谢你来到这里，欢迎收听本期播客。Host: Emmanuel, thank you so much for being here and welcome to the podcast.

嘉宾：谢谢你邀请我，Lenny。Guest: Thanks for having me, Lenny.

主持人：那么，你在 Anthropic 从事可解释性研究。请解释一下那是什么，以及你具体做什么。Host: So, you work on interpretability at Anthropic. Explain what that is. Explain what it is you do.

嘉宾：我在一个可解释性研究团队工作，我们的目标是通过观察 LLM（比如 Claude）的内部，来理解它们是如何运作的。如果你喜欢使用语言模型，或者会思考它们，有一个非常重要、也非常奇怪的事实需要认识到：造出它们，并不意味着理解它们。造出它们的方式，大致是先搭建一种架构，有点像支架，然后让大量数据通过其中，它们就会以难以理解的方式逐渐成长。这就是为什么我们会想要研究这些模型是如何运作的。Guest: I work in a team of interpretability researchers and our goal is to understand how LLMs like Claude work by looking inside them. One of the most important things to realize if you like use language models or think about them is just this like very bizarre fact that to make them is not to understand them. You make them by sort of like building this architecture which is kind of like a scaffold and then you pass a bunch of data through them and they sort of like grow in inscrutable ways. And so that's why it's even a thing that we would want to do to study like how these models work.

因此，我们的很多研究都在探究：模型即将生成下一个词时，实际上会发生一整套复杂的过程，而我们试图尽可能地梳理清楚其中的机制。And so a lot of our research has been on well when the models are about to produce one more word there is actually a whole universe of stuff that happens and we try to kind of untangle as much of it as we can.

主持人：我想，这段对话里会有很多让人大开眼界的——我不知道该怎么说——内容。其中一点就是你提出的这个观点：模型不是被建造出来的，而是生长出来的。多谈谈这个吧。Host: There's going to be a lot of mindexpanding I don't know elements to this conversation I expect. One is along these lines this point that you make that models are not built they're grown. Speak more to that.

嘉宾：从非常宏观的层面来说，像 Claude 这样的模型，训练过程大致分为几个连续的阶段。比如第一阶段，也就是人们所说的预训练，就是取一大批数据，把这些数据输入模型；比如，给它某份文档的前半部分，让它预测接下来会出现哪个词。然后用海量数据反复进行这个过程。完成之后，还有另一个阶段，也就是微调阶段：这时你可能不再使用任意的数据，而是使用一批示例，展示模型应该如何表现，并用这些示例来训练它。Guest: At a really high level, the way a model like Claude works is you kind of train it in these successive phases. Like the first phase, which people refer to as pre-training, is you just take a large corpus of data and over this corpus of data, you sort of feed the model as input, the let's say first half of like a certain document and you ask it to predict the word that comes next. Then you do that again and again for sort of an ungodly amount of data. And after you do that there's then this other phase where then you sort of have this fine-tuning phase where maybe instead of arbitrary data you have a corpus of examples of like how the model should behave and that's where you train it on.

所以你有一些对话示例，比如乐于助人的助手告诉你最喜欢的食谱，或帮你解决编程问题。So you have examples of dialogue of like helpful assistant telling your favorite recipe or solving coding problems.

然后这里还有一些更进一步的细节，我先暂时跳过。好吧，那你们做这个过程到底意味着什么？你们训练的到底是什么东西？And then there's sort of like further details here that I'm just going to skip for a second. But then, okay, so what does that mean that you're doing this process? Like what is this thing that you're sort of training?

所有模型都是这样：它们接收一段由许多词组成的输入，把这些词转换成数字，然后对这些数字进行一些运算，得到一些中间结果；接着再进行更多运算，得到更多中间结果，就这样反复很多次。最后，它们会得到一个最终结果，再把它转换回一个词。它们所做的数学运算，也就是这些计算，是不断发展的部分。通过学习更好地完成这项任务，我们会以某种方式朝一个方向或另一个方向更新参数。All models are is they take this input which is a bunch of words. They turn these words into numbers and then they do some math on these numbers and they get some intermediate results and then they do more math and they get some intermediate results and they do this again and again many times and then at the end they have a final result which then convert back into a word and so the math that they do these computations that's the part that's grown. That's the part where it's just like through learning to do this task better, we sort of like update parameters in one direction or another.

然后他们会学习一些公式。我们完全不知道这些公式是什么。我们团队的目标是研究这些公式最终意味着什么，以及这些中间结果究竟是什么。And then they learn some formulas. We have no idea what they are. And the goal of my team is to sort of like study, both sort of like what these formulas mean in the end and also what these intermediate results are.

我们能解码它们吗？我们能不能看到模型在进行这些中间步骤时具体在做什么？Can we decode them? And can we see like what the model is doing as it's doing all of this intermediate stuff.

为什么下一个词元预测比听起来更难Why next-token prediction is harder than it sounds

主持人：我们写了一些代码，现在居然得像神经科学家研究大脑那样去观察和研究它——想到这一点，你会不会和我一样觉得不可思议？这简直是我们为了弄明白到底发生了什么而必须做的一件新事。还有，你用的这套术语，也正是很多人思考这个问题的方式。Host: Does it blow your mind as much as it blows mine that we have some code that we wrote that first of all we have to now kind of watch and study like a neuroscientist studies a brain? Like that's like a new thing that we need to be doing to understand what the hell is going on. And then this idea that as like this terminology you're using is how a lot of people think about this.

就像我们创造了一种新的生命——人工生命、人工智能。它就是……你认真想一想，再把视角拉远一点看，会觉得非常不可思议。It's like we've created this new life artificial life, artificial intelligence. It just is it's pretty mindboggling if you really think about it and kind of zoom out.

嘉宾：我觉得这真的很奇怪。我觉得理解这件事的最好方式，大概是想想我们为什么会走到这一步，其中一个重要原因就是，我们不知道怎么用常规的编程方式来做这件事？比如说，如果你想开发一个应用之类的东西，你会想做一个相对独立的东西。Guest: I think it's really weird. I think that the best way to think about it is sort of like the reason we ended up here, one of the big reasons is that we don't know how to do it the normal programming way? So there's like for you want to build an app or something, you want to sort of do something pretty self-contained.

你可以直接找一个程序员，写一大堆规则，比如说，呃，如果有人点了这个按钮，就会发生这件事；如果出现这种情况，就那样处理，诸如此类。可我们不知道该怎么编写软件来把图像分成不同类别，也不知道该怎么编写软件来打造一台机器，让它能够大体上接收任意文本，并生成文本；它可以接收报告并概括它们，可以写邮件，等等。所以，从某种意义上说，很奇怪，这么简单的一种训练目标竟然能让你做到这一切；但从另一个角度看，我又觉得这很了不起。这说明了一些事情：你以为很简单的目标——比如“哦，我们只是在输入数据，然后预测下一个内容”——其实可能复杂得超乎想象。You can just have a programmer that's going to write a bunch of rules and say like ah well if somebody clicks on this button then this happens if this then that and so on. We don't know how to write that software for classifying images into categories or for writing for making a machine that can generally sort of take in arbitrary text and produce texts it can take in like reports and summarize them can write emails and so in a sense it's weird that this very simple kind of training objective like gets you all the way there but in another sense it's also you I almost feel like there's a sense of like it's impressive. And it means something. It means that what you think is a simple objective this like oh we're just feeding data and predict the next thing is probably deceptively complicated.

感觉我们做的事情可能比听起来的要多。而且我觉得，我对这些模型如何运作的认识之所以有所更新，很大程度上是因为我意识到，要让它们把擅长的事情做好有多复杂。Like we're probably doing more than what it sounds like. And I think a lot of the ways that maybe like I've updated on how these models work is by realizing how complicated it is to do what they do well.

主持人：再多说一些吧。你说这是某种意想不到的复杂情况时，心里是怎么想的？Host: Say more about that. What do you think about when you say it's this kind of unexpected complication?

嘉宾：好，我觉得很多人理解下一个词元预测的方式之一，是把它想成 iPhone 的自动补全：你输入一些文字，它就会预测下一个词，大概有 60% 的时候能猜对吧，剩下的时候则多少有点好笑又奇怪。我会把这看作是预测下一个词最最简单的版本，而遗憾的是，我觉得这就是人们对这些模型的心智模型。但如果你在这方面做得更好，语言模型大约在 2019 年就已经比人类更擅长预测下一个词了。Guest: Okay, so one way to think about next token prediction that I think a lot of people think about is your iPhone autocomplete, where you write some text and then it'll like predict the next word and it's maybe like right I don't know 60% of the time and then kind of funny and weird the rest of the time. And I would think of that as sort of the very the easiest version of predicting the next word and unfortunately I think that's the mental model how that people have for these models. But if you get better at that and language models got better at predicting the next word than humans in around 2019.

如果你在这方面不断进步，就会开始用完那些简单的词。If you keep getting better at that you start running out of easy words.

所以一开始，你只需要预测简单的句子结构；后来，模型就得预测一些世界常识，比如，如果我把句子补成“法国的首都是……”，它就得回答“巴黎”。等它擅长这类任务并不断进步之后，剩下的就只有预测某个复杂物理方程的答案，或者某篇博士论文的最后一段之类的内容了。 如果你认真想想，就会意识到一件事：预测最难的词其实极其困难，而要如何做到这一点，仍然是个谜。对我来说，这也正是我最初开始研究这些模型的一个动机。So initially you just had to predict simple sentence structure later on the model has to predict like oh like simple facts about the world like if I complete a sentence like oh the capital of France is and has to be like okay that's Paris and once it's good at that and it keeps improving the only thing that's left is sort of predicting the answer of some complex physics equation or the sort of final paragraph for some PhD thesis. And so a thing that you end up realizing if you think about this is that man like predicting the hardest words is actually extremely hard to do and it's a sort of mystery how you would be able to do that. And to me this is a sort of this has been a motivation of why I even started studying these models in the first place.

我当时就在想，这么简单的东西怎么能产生这么强大的能力？你看，如果想想模型必须做出的预测，我一直喜欢举的一个例子是：嗯，达拉斯所在州的首府是……它会说奥斯汀。要做到这一点，至少对你我来说，脑子里多少得有个相关地理情况的模型，对吧？I was like how can something so simple give rise to something so powerful? And it's like well if you think about the model that has to predict sort of the one example I always like to use is like ah the capital of the state that contains Dallas is it'll say Austin. Well to do that well at least for you and me you kind of have to have a mental model of like the geography at play right?

你得这样说：“哦，达拉斯在哪儿？达拉斯在德克萨斯州。各州都有首府。”You have to be like oh where's Dallas? Dallas is in Texas. States have capitals.

德克萨斯州的首府是奥斯汀。诸如此类，随着这些模型在看似简单的任务上越来越出色，对我们来说，解决这类问题的最佳方式似乎就是理解世界的底层结构。事实上，我们在模型内部观察到的一些现象表明，它们确实理解这一点，也确实会像我们一样运用这些概念。The capital of Texas is Austin. And so on and so as these models get better at the seemingly simple task the sort of like best way to solve it at least for us is to just kind of understand the underlying structure of the world and in fact some of the stuff that we look at inside model shows that they do understand that and they do sort of manipulate these concepts like we do

主持人：天哪，这里面内容太多了。首先有这么个想法，我忘了是谁说的，可能是 Ilia：要想很好地预测下一个 token，基本上就得理解整个宇宙。Host: oh man there's so much there one is just this kind of idea I forget who said this maybe Ilia this idea that to predict the next token well you basically have to understand the entire universe

嘉宾：他有个我特别喜欢的例子，就是预测悬疑小说结尾揭晓的凶手是谁，对吧？要做到这一点，你得先理解整个情节。Guest: he has an example that I love to where he has the example like predict and the murderer was at the end of a mystery novel, right? It's like well to do that you kind of have to have understood the whole plot.

主持人：是啊。还有个想法，我忘了是谁发推说的：人类要是真归根结底，其实基本上就是下一个 token 预测器，或者说，只是在努力弄清楚接下来该说什么。如果你打坐冥想，就会开始注意到自己的大脑想做什么、想说什么。我不知道你冥想过多少，但感觉你会开始留意自己脑中的下一个 token 预测器。Host: Yeah. The other I forget who tweeted this idea that like humans if you really boil it down like we're basically next token predictors ourselves or just trying to figure out the next thing to say and if you meditate you start to kind of actually notice your brain wanting to do things and say things. I don't know how much meditation you've done, but it feels like you start to pay attention to your own next token predictor.

嘉宾：哦，对。对。我经常会想到这么一件事：就像现在我在回答你时，我脑子里大概有两件事同时在进行。Guest: Oh, yeah. Yeah. And this is like a thing I think about often is that, like right now as I'm answering you, there's sort of like two things going on in my mind.

希望不止两件事，但至少有两件。其中一件是，我好像正引导你走向某个观点。我试图让这句话落到一个比较宽泛的想法上。另一件事是，与此同时，我只是在预测接下来要说什么。我一次只说一个词，然后把它们组织成句子，努力让句子讲得通，并在我觉得讲通了的时候停顿。There's hopefully more than two, but at least two. And one of them is there's sort of like a point I'm leading you to. There's like a broad idea I'm trying to land the sentence on. And the other thing is at the same time I'm just predicting the next thing to say. I'm just saying one word at a time and I'm making sentences and I'm trying to make them make sense and I'm pausing when I think it makes sense.

但归根结底，尽管我一直都是一次只说一个词，我想做的始终是把这个过程与语言模型所做的事情进行比较。But at the end of the day, even though I'm just saying one word at a time, like all along, what I wanted to do is to compare this process to what language models do.

模型如何表征概念并理解世界How models represent concepts and understand the world

主持人：我很好奇，随着你花时间关注大型语言模型的工作和运行方式，还有什么让你感到意外？我觉得，在研究和观察大型语言模型如何运行和思考的人当中，你绝对属于最顶尖的 0.1%。它们的运行和思考方式还有什么让你惊讶的地方？Host: I'm curious what else as you spend time paying attention to how LLMs work and operate. I feel like you've you're like in the top 0.1 percentile of people that have studied and watched LMS operate and think. What else has surprised you about how they operate and think?

嘉宾：内容太多了，所以我也许会大致讲讲——与其说有一堆发现，不如说可能有那么一些……嗯，它们如何改变了我对模型的看法。回到首府这个例子，我们问模型：“包含达拉斯的州的首府是……”长期以来，人们一直有一个疑问：这些模型是如何回答这类问题的？它们是不是看过海量文本，所以某种程度上记住了提示和回答之类的列表？Guest: There's so much so maybe I'll go through like rather than there's a bunch of findings but then maybe there sort of like what yeah how they've changed how I think about models. To go back to the capital example, we ask the model, “The capital of the state that contains Dallas is ...” One question that people have had for a while is how do these models answer questions like this? Is it that they've seen just so much text that they've kind of memorized like lists of, prompts and responses?

还是说，它们对你实际在问的内容有某种模型，比如你实际问的是城市，而城市属于州，州属于国家，诸如此类。我们团队会做的一件事就是提出这样的问题，然后在模型即将作答时、生成某个词之前，查看模型内部的这些中间结果。我们会直接查看这些中间结果，并投入大量精力去尝试解读它们的含义。如果我们能很好地解读它们——有些情况下确实可以——就能看到模型往往拥有并表征着这些概念。Or is it that they're having some sort of model of like what you're actually asking about that you're actually asking about like cities and cities are in states and states are in countries and things like that. And so one thing that we do on my team is we ask a question like this and then we look inside the model at these intermediate results before it produces a given word, as it's about to answer. We just look at the intermediate results and we basically have an immense amount of effort dedicated to trying to decode what these results mean. And if we decode them well which we can do in some cases we can see that models often have and represent these concepts.

因此，当你提出那个关于达拉斯的问题时，你看到的是模型内部的情况：你看到这种表征，看到一个中间结果，它把被激活的得克萨斯州表征和首府表征联系起来；然后这两者结合，激活一个中间结果。So when you ask that Dallas question what you see is inside the model you see the representation you see an intermediate result that maps the representation of Texas that lights up and one to capitals and then those two combine to be to activate an intermediate result.

这就像，哦，德克萨斯州的首府是奥斯汀。对于几乎所有东西，他们都有类似这样的表征——对于你能想到的每个概念，至少是我们尝试过的绝大多数概念，它们会在不同语境中以独特的方式表征这些概念。所以我觉得有件事特别有意思：人们一直在争论语言模型是否像人一样在思考。这是个很难回答的问题，但思维的一个方面，就像是拥有独特的表征，并在思考不同事物时共享这些表征。It's like oh the capital of Texas that's Austin. They have representations like this for basically everything like every concept that you can think of or at least the vast majority of the ones we've tried they sort of represent these concepts uniquely across contexts. So one thing I find really neat is there's this debate about are language models like thinking and then that's a hard question but one aspect of thought is like having unique representations that are shared when you think about different things.

所以你大概知道“眼睛”这个概念，它可以是照片里的眼睛、医学示意图里的眼睛，也可以是一幅小画里的眼睛。我们在语言模型中也能看到这种现象。如果你输入一个包含眼睛照片的提示，或者用任何语言谈论眼睛，或者画一个里面有个 I 的小 SVG，然后查看模型内部的中间结果，就会发现同一个方向被激活了。这个方向就像是在说：“哦，这是眼睛。”So you like know what the concept of an eye is and it could be like an eye in a photo or an eye in like a medical diagram or an eye in a little drawing and we see that in language models as well. If you pass a prompt that has a photo of an eye or you talk about eyes in any language or you draw a little SVG with an I in it and you look at the intermediate results inside the model, the same direction lights up and that direction is like, oh, this is an eye.

因此，从某种意义上说，它们拥有一些自己了解并会反复使用来解决问题的一般概念。还有一件让人有些意外的事，正好是我刚才所说内容的另一面：我是一词一词地说出这个句子，但我心里有一个目标。所以，语言模型的一个谜团是：如果它们每次只输出一个词，怎么可能写出一份连贯的报告，或者编写代码呢？And so there's a sort of a sense in which they have these general concepts that they know about and that they use and reuse to solve problems. Another thing that sort of is surprising to people, it's kind of the mirror of what I was saying about, I'm saying the sentence saying one word at a time, but I have some destination in mind. So, one mystery of language models is bam, if they're outputting only one word at a time, how is it even possible that they could, write a coherent report or like write code?

嗯，这有点像是，世界上没有人把这当成工作来做——比如写报告或者写邮件时，真的就只是想着句子里的下一个词，完全不知道后面会是什么，脑子里对接下来的一切一片空白。Well, the sort of like no one on earth that is doing this as their job, that's like writing reports or, writing emails is literally just only thinking about the next word in the sentence with no idea, like empty mind about anything that comes next.

所以，这算是我们一段时间以来一直在思考的一个大问题。如果你看看这些模型内部，会发现一个很有意思的事实：它们在预测某个词时产生的所有中间结果，比如说预测到第五个词时产生的结果，都可以被后续的所有词访问并重复使用。那么这意味着什么呢？And so that was sort of a big question that we had for a while. And if you look inside these models, like one neat fact about these models is that all of the intermediate results that they've like produced when they were predicting a word, let's say like the fifth word that they predicted, they can be reused. They can be accessed and reused by all of the future words. So what does that mean?

这意味着，实际上我们会发现，模型有时会在中间结果中制定计划。我们举的例子是写诗。没有计划，很难押韵。你多少得想一想：“哦，我第一句押韵的部分刚用了一个以 /ɪt/ 音结尾的词，所以第二句也应该用另一个以 /ɪt/ 音结尾的词收尾。也许我该用‘rabbit’来收尾。”That means that in practice, we find that models will like in their intermediate results sometimes formulate a plan. The example we have for this is poetry. It's hard to rhyme without planning. You kind of have to think of like, oh, I just ended my first rhyme in like a it word and so I should end my second line with another it word. Maybe the word I should end it with is rabbit.

我们发现，如果你观察模型写诗的过程，就会看到它们恰恰是这么做的。它们先写下一行，然后在中间结果里好像会想：“哦，哦，哦，这是押 it 韵的。我应该跟 rabbit 押韵。”之后它们预测的每个词都会回看自己制定的这个计划。And we find that if you look at models as they're like writing poetry, you see exactly that. They write a line and then like in their intermediate results, they're like, "Oh, oh, oh, this is an it rhyme. I should rhyme with rabbit." And then every future word that they predict looks back to that plan they made.

所以，尽管它们像我现在这样，一次只输出一个词，但当然也可以制定一个贯穿许多词的计划，并按照计划行事。这样一来，尽管你可以说它像个下一个词预测器，尽管从技术上讲确实如此，但实际上它更像是能够执行更高阶任务的东西：比如说，你有一个计划，然后不断回头参考这个计划，再慢慢地照着它执行。And so even though they're, outputting one word at a time just like I am right now, they can of course have a plan that they follow over many words. And so all of a sudden, even though you have like a next word predictor, even though it's like technically right, you actually have something that's closer to being able to just do like, higher order tasks where it's like, oh well, you just have a plan and you just kind of refer back to your plan and you're slowly like executing on it.

而且你可以做到这一点，不用说出计划。酷就酷在，我们发现这个计划存在于模型内部的这些中间结果中，但模型甚至不必把它说出来。And you can do that without saying the plan. That's the cool part is like this plan that we find is in these intermediate results inside the model, but the model doesn't even have to sort of like say it out loud.

LLM 如何提前规划How LLMs plan ahead

主持人：哇。我知道，你们描述的这一点，正是你们在研究 AI 如何思考时取得的最重大发现之一：我们都倾向于认为，它最初就是基于这样的假设构建的。关于语言模型如何运作，最初的想法是，它就是一个下一个 token 预测器。Host: Wow. I know that this exactly what you described is one of the biggest findings you guys have discovered about how AI thinks this kind of assumption we all like it was originally built. The original idea of this of how LM operate was this next token predictor.

而你是说，现在实际上已经不是那样了。他们会规划，会提前思考。并不是像你说的那样，仿佛一颗空白的头脑在不受其他任何因素影响的情况下挑选下一个词。And you're saying that's not actually how it is anymore. They plan, they think ahead. It's not just like you said this idea of an empty mind picking the next word independent of everything else.

嘉宾：没错。我觉得，我觉得这两种愿景其实并不像你想的那样相互冲突，因为你训练模型时会问它：“嘿，接下来会发生什么？接下来会发生什么？”Guest: That's right. And I think that I think those two visions can like actually aren't as in conflict as you'd think in that you train the model by asking it to be like hey what happens next? What happens next?

接下来会发生什么？接下来会发生什么？然后，如果你这么做了，而且模型变得足够好，你给它的任务也足够难，那么当然，要知道接下来会发生什么，最好的办法就是想一想很久以后会发生什么。你甚至可以想想在这个预训练阶段，模型读到一个有角色的故事时的情形，对吧？What happens next? What happens next? And then if you do that and the model gets good enough and you give it hard enough tasks well then of course the best way to know what happens next is to think about like what happens way down the line. You can think of even in this pre-training phase when like the models are reading a story with characters, right?

感觉就像，哦，这个角色接下来会做什么呢？嗯，想知道这一点，最好的办法就是想：好吧，这个角色确实爱上了那个角色，而且他们一直在谋划着，想在这一刻表白。And it's like, oh, like what is this character going to do next? Well, the best way to know that is to be like, okay, so this character, is genuinely in love with that character and they've been sort of like planning to like declare their love at this instant.

所以，或者说，为了将来做到这一点，他们可能应该做这件事。他们要说的是，尽管你只是在预测下一个瞬间，但要把这件事做好，你必须非常清楚正在发生什么，包括你认为未来可能会发生什么。And so, or in the future and so to do that, they probably should do this thing. And so what they're going to say is that and so even though you're only like predicting the like next instant, it's like yeah, but to do that really well, you have to have like a really good understanding of what's going on, including what you think might happen in the future.

主持人：这件事有意思的地方在于，你得稍微窥探一下它的小小思维过程。作为 AI 的使用者，你给它一个提示，然后等着它说些什么，而你关注的是这两者之间发生的事情，或者说它思考的过程。能不能谈谈这片小小的空间是什么，以及你是如何观察到这些事情的？Host: What's so interesting about this is how you have to kind of peek inside its little thought process. As a consumer of AI, you kind of give it a prompt and then you wait for it to say something and what you look at is something that happens between those two or kind of thinks. Talk a little bit about what that little space is, how you're able to observe these sorts of things.

嘉宾：所以，正确的心智模型是：每个词之间其实都有多个步骤，比如计算；而每个步骤之间又会产生一些中间结果。那么这些结果具体是什么呢？是一串数字，对吧？因为这些模型做的事情，说到底就是对许多数字进行大量数学运算。所以一开始你看它的时候，会觉得完全看不懂。Guest: So the right mental model is again that in between each word there's like multiple steps so like computation and in between each steps there's like some intermediate result and what is that result concretely? It's a list of numbers, right? Cuz because all these models do is just do, a lot of mathematical operations on many numbers. And so initially, if you look at it, it makes no sense.

你现在只有一串由 100 个数字组成的列表。模型在做数学运算。就好像你看向中间，看到一堆 100 个随机数字，然后模型在写诗，里面也有一百个数字；它们看起来或许有点不同，但很难说这些数字是否有什么含义。所以，我们工作的一部分就是解码这串数字，试着看看其中有没有结构。You just have a list of a 100 numbers. The model's doing math. It's like you look in the middle and there's like 100 random numbers and then the model's writing poetry and there's a hundred numbers and maybe they kind of look different, but it's unclear like if they mean anything. And so part of the job is we sort of decode this sort of list of numbers and we try to see if there's structure in it.

而且，如果你非常迅速地做这件事，就会发现其中存在某种结构。可解释性研究中常用的一种方法是，把大量文本——能找到的所有文本、各种随机文本——输入模型，然后记录每个样本对应的这些中间数值。我们就把它们记录下来，然后试着看看有没有什么规律：比如，是否存在某些数字、数字对或特定的数字序列，只在模型处理英文文本、法文文本，或有关爱情、眼睛之类的文本时才会出现。 我觉得这就相当于扫描我的大脑，把图像放到我面前，并记录我看一大堆图像时大脑里所有被激活的部分，然后说：“哦，看来你看到房子的图像时，大脑的这部分会亮起来；但看到狗的图像时，大脑的那部分会亮起来。”And if if you do this very quickly you realize that there is structure in it. So one common technique that is used in interpretability is you pass a bunch of text just all of the text that you can find just random text through the model and you record these intermediate numbers for every example. We just record it and then you just try to see if there's patterns like are there numbers or like pairs of numbers or like specific series of numbers that like only occur when the model is like processing English text or French text or text about love or text about eyes or whatever. And I think of this as the equivalent of, scan my brain and like put images in front of me and record everything that activates in my brain as I like watch a bunch of images and then be like, "Oh, it seems like when you see images of a house, like this part of your brain lights up, but when you see images of like a dog, that part of your brain lights up."

所以，这基本上就是我们做这件事的方式之一。如果你这么做，就会发现其中有一些中间结果，代表着你所预期的所有这些概念。这就是第一阶段。我们会先解码这些中间结果，大致猜出它们的含义，然后做一系列实验：比如向模型提出更难的问题，再给所有中间状态贴上标签，看看它到底在做什么；或者，我们甚至可以深入它的激活值并加以修改，然后看看：当模型生成这些中间结果时，如果我悄悄改掉其中一个结果，会发生什么？结果是否符合我的预期？So that's basically like one of the ways that we do what we do. And if you do it, you do find basically that there's intermediate results that represent like all of these concepts that you'd expect. And so that's the first phase. We sort of like decode these intermediate results and now we have sort of like a guess as to what they mean and then we can do a bunch of experiments like we can ask it harder questions and then just like label all of the intermediate states and see like oh this is what it's doing or we can even go inside its activations and change it and then sort of see like oh what would happen if while the model is writing these intermediate results I went and like secretly changed one of the results and does that do what I would expect?

事实上，很多时候它确实如此。And in fact, oftentimes it does.

主持人：听你谈论这些真有意思。我们有多少时候只是在努力搞清楚到底发生了什么，而不是一开始就把它设计成这样？又有多少是后来逐渐浮现出来的？我也说不准。是不是正因为如此，这份工作才这么有趣？我想，你大概也常常会觉得：这到底是怎么回事？Host: It's so interesting hearing you talk about this and how much of this is us just trying to figure out what the hell is going on versus we designed it this certain way and just how much of this has emerged. I don't know. Is that just like I guess that's just what makes this such a fun job I imagine for you just like what is going on?

这一切是怎么运作的？How's this all working?

嘉宾：是的，这份工作很有趣，也很有科幻感，因为我在科技行业工作有一段时间了。在这之前，我做的更多是应用机器学习，而那类工作中，你做的很多事情都是在构建系统，有点像软件工程实践。但在这里，我们做的很多事情感觉更接近生物学，而不是工程学，对吧？因为系统就这么呈现在你面前，然后有人会问你：“我们注意到 Claude 能翻译文字，或者能以这种方式编写代码；有时候它的表现很正常，有时候又会出现奇怪的行为……这是怎么回事？”他们就是要你回答：“它为什么会这样？”但你并不能像对待程序那样，真正访问它的源代码。Guest: Yeah, it's a fun and sci-filike job because I have been working in tech for a while. And before this I was working on more applied machine learning and there a lot of what you do is about building systems and it's sort of like the practice of software engineering and here a lot of what we do feels so much closer to biology than to engineering right because it's just the sort of system arrives at you and you're being asked to say like well we've noticed that claude can like translate words or it can code in this way or sometimes it behaves, sometimes there's like a weird behavior like what like and you're just asked to answer well how like why is it doing that? But you don't have access to the source code really like you do for a program.

就像源代码只是一大堆数学运算，简直让人看不懂。Like the source code is just this massive list of mathematical operations which is just like incomprehensible.

于是，你最终会处在一种类似这样的情境中：有人问你：“嘿，老鼠饿了的时候好像会变得有点攻击性。这是怎么回事？”对吧？然后你就会说：“嗯，我想我得做些实验，看看有哪些因素在起作用。”所以，这种研究和我们团队里大多数人以前做过的研究都很不一样。And so you end up, in the position of someone kind of similar to someone being asked like, "Hey, when like a mouse gets hungry, like it sort of like gets aggressive. What's going on with that?" Right? And you're like, "Well, I guess I'll like run some experiments to see what the factors are." and so it's a very different kind of study than anything that I think most of us on the team have done before.

人工智能可能具有意识吗？Could AI be conscious?

主持人：是的。我想，人们刚开始做这件事时，并没想到会创造出这种近乎生命体的新型存在。你认为意识是其中的一部分吗？Host: Yeah. I imagine when people were starting on this, they didn't expect to create this new kind of almost life type of being. Do you think about consciousness as a part of this?

就像，你觉得 AI 有意识吗？你觉得它会发展到让我们认为它有意识的地步吗？还是说这根本不值得我们花时间去想？Just like do you think AI is conscious? Do you think it'll get to a place where we think it's conscious or is that just don't even not even worth our time to think about?

嘉宾：我觉得很难给出明确的结论。可解释性提供了一些非常有趣的方法来探究这个问题，因为你不会受到模型所说内容的影响。你可以观察模型内部的结构，试着判断它在这个问题上倾向于哪一方。Guest: I think it's very hard to say one way or another. Interpretability offers some of the most interesting ways to get at this question because you are not biased by what the model says. You can look at the structure inside and try to determine whether it pushes one way or another on the question.

我相当确信，大多数人都低估了由此涌现出的复杂性。模型会学习我们所拥有的概念的表征，操控这些表征来解决问题，并提前规划许多 token。它们兼具高阶和低阶推理能力，而这一切都是涌现出来的。我认为人们没有充分认识到这一点，这让我更加深刻地意识到，我们对它们的运作方式仍有很多需要了解。I believe pretty strongly that most people underestimate the complexity that emerges. Models learn representations for the concepts we have, manipulate them to solve problems, and plan many tokens ahead. They have a mix of higher-order and lower-order reasoning, and all of that emerges. I think people underappreciate this, and it gives me a deeper sense that there is still a lot to understand about how they work.

我觉得，或许关键就在于：我发现你越是深入研究，就越不确定它们究竟是如何运作的。而那些信誓旦旦地说“语言模型不过就是 X”或者“不过就是 Y”的人，往往恰恰是那些没有花时间深入研究过它们的人。I think maybe that's the main thing is I find that the more you look inside, the less certain you are about how they work. And then oftentimes people that will confidently say like ah language models are just X or just Y are the people that actually haven't spent a while looking inside.

规模定律与文本的力量Scaling laws and the power of text

主持人：还有这么一条历史脉络，贯穿到我们今天走到这一步——很多人当时并不认为，只要投资于某种看起来非常简单的东西，再给它投入大量算力，就能得到有用的人工智能。OpenAI 著名地押注了这条路，后来 Anthropic 的创始人当时还在 OpenAI，之后他们分出去创办了自己的公司；大家其实都在赌同一件事：好吧，我们只要做这件看似简单的事，再给它海量算力，它就会变得极其聪明。很多人觉得这不可能，但事实证明他们错了。所以我不知道，这里面有没有什么让人觉得“哇，看看，居然真的成功了”的地方？Host: And there's this I don't know thread of history through to how we got here which is a lot of people did not think we would get to artificial intelligence that is useful by just investing in something that was seemingly very simple and just giving it a lot of compute. And that was famously something OpenAI bet on and then the anthropic founders were at OpenAI and then spun off started their own thing and it was all kind of bet on this bet that okay we could just do this seemingly simple thing and give it a ton of compute and it will become incredibly smart and a lot of people didn't think that was possible and they were proven wrong. So I don't know is there anything there that is just like oh wow look at this it worked out.

嘉宾：我觉得实际上有两件事，因为我认为这不只是算力的问题，还要找到一个能让算力持续带来回报的目标。Guest: I think it's two things actually because I think it's not just compute, it's also finding an objective where compute keeps giving you returns.

那么这意味着什么呢？其实这并不是什么新想法。我的意思是，你说得对，关于算力和专家设计架构孰优孰劣，这场争论由来已久。但除此之外，还有一个问题：算力要用来做什么？过去有不少相关研究，比如说，你可以投入大量算力，打造一个特别擅长围棋的机器人，或者一个能赢得某款电子游戏的机器人。这当然很酷，而且仅仅是它能做到这一点就挺有意思。但这并不能带来像 Claude 这样的 LLM 那么有用的东西。So what does that mean? It's actually not that new of an idea. I mean, you're right like compute versus sort of I would say expert design architectures is a long debate. But then there's an additional step there which is compute to do what . So there's a bunch of prior work on like oh you can spend a bunch of compute to make a bot that's really good at go or that can win some video game and then that's cool and just the fact that works is sort of like interesting. But that doesn't give you something that is nearly as useful as LLMs like Claude are.

所以我认为，几年前业内很多人（包括我）都低估了一个关键因素，而事实也证明了这一点：算力与文本丰富度的结合，才能让你达到那个目标。And so I think like one key thing that many in the field including me underappreciated a few years ago and that has borne out is the combination of sort of like compute and the richness of text is what gets you there.

对我来说，这有点像我刚才暗示的那样：你想想，OpenAI 的人曾经提出过这样一个观察——当时 Anthropic 早期团队中的不少人参与了那篇关于缩放定律的论文。这个观察是，如果你给预测下一个词这个任务投入更多算力和更多数据，就会发现模型预测下一个词的能力一直在稳步提升。这有点像一种不断延续的趋势，也是一条经验定律。当时人们对原因做过一些猜测，但到现在，它基本上仍然只是一个经验发现。And to me, like this is kind of what I was alluding to, but if you think about like there's this observation that OpenAI people made it was some of the a lot of the early team at an entropic on the scaling loss paper. And so this is the observation that if you sort of like pour in more compute and more data to the task of predicting the next word, you see that like the skill of the model you get at predicting the next word just like predictably keeps improving. And it's sort of like a trend that just keeps going and it's an empirical law. People sort of, speculated as to why at the time, but it's sort of still just mostly an empirical finding.

这意味着，啊，这是一个你可以不断投入更多算力、并持续提升该指标的领域，这种情况很少见；而且这个指标还能与我们真正关心的一系列任务相关，这也很少见。And that means that it's like ah this is a field where you can just keep pouring more compute and you keep improving on that metric and that's rare and also it's rare for it to be a metric that then correlates with a bunch of tasks that we actually care about.

为什么平稳进展会带来突然的突破Why smooth progress creates sudden breakthroughs

主持人：从外部来看，人工智能能为我们做什么，似乎出现了几次巨大的飞跃。我忘了是 Opus 的哪个版本，突然就解锁了编程能力；而现在，正如 Boris Cherny 的名言所说，编程问题已经解决了。Host: Externally, there's been these kind of big jumps seemingly in what AI can do for us. There was like I forget which version of Opus where it just unlocked coding and now co as Boris Jurnney famously said, coding is solved.

嘉宾：是的。Guest: Yeah.

主持人：很多人都觉得，天哪，我想寒假前后，我们都在感叹，哇，AI 现在真的很擅长做这个。所以，能力上确实出现过几次这样的飞跃时刻。而你刚才也提到，在你们内部看来，实际上，AI 能力一直在非常平稳地提升，只是恰好跨过了某个门槛，让我们现在能够做到一些事情。能不能稍微谈谈这一点？Host: and many people are just like holy like over I think winter break we're just like wow AI is really good at this now. So there's been this kind of these moments of big jumps in abilities and you actually said internally it's as you just said it's actually been this very smooth progression of AI power that happens to cross some threshold where now we can do something speak a little bit about that

嘉宾：有一个术语可以描述这种现象，大致是“涌现”这个概念：似乎有两件事同时成立，一方面趋势平稳、可预测；另一方面，在 Opus 4.5 之前，我真的不敢让 Claude 在我的代码上放手折腾太久，得拴得紧紧的；而在 4.5 之后，我就敢了。这就很奇怪。感觉你会以为这是一个更渐进的转变。我对此的心智模型是，大致有两种看法。Guest: there's a term for this which is sort of this idea of emergence which is it seems like two things are true at once like there's this predictably smooth trend and then before like opus 4.5 I really couldn't let Claude, cook too much on my code without a short leash and then after 4.5 I could. And so that's weird. Like I feel like you would expect a more gradual transition. And the mental model I have for this is, there's kind of two ways of looking at it.

所以，刚才我简略提到过一种方式：如果你只是在做这种下一个词元预测，可以想象世界上所有文本里的词，按预测难度大致分为容易、中等和困难几类。So there's the way that I talked about briefly just earlier where it's like if you're just doing this kind of next token prediction, you could imagine that all the text in the world has sort of like easy words and medium words and hard words in terms of how hard they are to predict.

因此，当你沿着这条曲线往下看时，模型一开始什么都做不好。然后到了某个时候，它就能把所有简单的词都处理对了。这可能意味着，简单的词就像是句子的结构。And so as you're going down this curve, first the model is bad at everything. And then at some point it gets all of the easy words right. And that means, maybe the easy words are like the structure of sentences.

于是，模型突然就能写出连贯的句子了，但一次只能写一句。所以这挺酷、也挺令人印象深刻的，但我们其实没法拿它做什么。然后，也许中等难度的词大概就是这些关于世界的基本事实。随着模型对词语的预测能力逐渐提升，到了某个时候，它就能预测大多数中等难度的词了，对吧？And so the model like all of a sudden can write coherent sentences, but just one sentence at a time. And so it's kind of cool and impressive, but like we can't really do anything with that. And then maybe the medium words are sort of yeah, these sort of basic facts about the world. And as the model smoothly gets better at predicting words, at some point it predicts most of the medium words, right?

而现在，它突然就能写出连贯且真实的句子了。所以，也许它能帮你写邮件，也能帮你起草博客文章，对吧？And now all of a sudden it can write coherent sentences and sentences that are true. And so now maybe it's helpful to like write emails. Maybe it's helpful to sort of like help you write drafts of blog posts, right?

虽然那算是一种平稳的进步，但只要它能正常运转，就已经能派上用场了。然后我想，难词可能就是那些需要你具备领域专业知识的词，比如科学或编程方面的词。换个角度看，这其实取决于你能把这些词答对多少、答错多少。一旦你能把中等难度的词答对，比如我不知道，也许达到 90%，那你就能当个有用的邮件写手了。但如果只有 50%，那用你实际上是在浪费我的时间，因为我有一半的时候都得自己进去修改。Even though that was like a smooth improvement, it just takes it ticking over to be helpful. And then I guess the hard words which maybe are like your like domain expertise like science or code. And another way to look at it is it's a question of kind of how many of these words do you get right and how many do you get wrong. Once you get to the point where you're getting like, I don't know, maybe 90% of the medium words right, then you're useful as like an email writer. But if you're at 50%, it's like you're it's actually wasting my time to use you because I'm just going to have to go in and fix it half the time.

但如果我得进去修，那它有10%的时候会变得更好，或者说有5%的时候会变得更好。But if I have to go in and fix it, 10% of the time, it's better or 5% of the time it's better.

不同任务有不同的可靠性门槛。你的能力会逐步提升，到某个时候，模型就会变得足够可靠，能够派上用场。我的亲身经历正是如此。起初，在遇到难题时我会向模型求助：我会和它们交流，但它们从来不会替我编写代码。Different tasks have different reliability thresholds. You improve smoothly, and at some point the model becomes reliable enough to be useful. My lived experience is exactly that. At first I used the models for help when I was stuck: I would talk to them, but they would never write my code.

后来，它们基本上每次都能正确地写一些小段代码，所以现在我不再自己写小段代码了。再后来，它们也能写大段代码了。今年 Claude 承担了我大部分的代码编写工作，因为它正确完成任务的概率从我也说不准的 80% 左右（这没什么用）提升到了 99%（这就有用了）。Then they could write small snippets of code correctly basically all the time, so now I do not write small snippets. At some point they could write large snippets. This year Claude has taken on most of my code writing because it went from doing that correctly, I don't know, 80% of the time—which was not useful—to 99% of the time, which is useful.

这种平稳的改进与每项任务各不相同的阈值相结合，带来了这些显著跃升，尽管我们沿着这条尺度平稳地向下移动。This combination of smooth improvement and a different threshold for each task gives you these big jumps, even though we are smoothly moving down that scale.

主持人：我觉得这真的很有意思。模型似乎只是在平稳地变得更聪明，但一旦它们达到某个阈值——比如在某件事上经常做得很好——世界就会因此改变。所以我觉得，从内部来看，你可能会觉得，哇，下一个模型会强得离谱，跟现在大不一样；但其实不是，行吧，你几乎可以预测：给它这么多 GPU、这么多数据，它就会变得这么聪明。具体会有哪些变化、会解锁什么能力，可能并不明显，但我知道其中另一个重要因素就是你喂给它的数据。Host: I think this is really interesting. This idea that the models are just smoothly getting smarter, but once they hit some threshold of they're this often good at this thing, it changes the world. And so because I think internally, you would think it'd be like, oh wow, this next model is crazy, so much different, but it's like, no, okay, you can almost predict give it these many GPUs, this much data, it's going to get this smart. And it may not be obvious exactly what will change and what unlocks but I know another part of it is just like the data you feed it.

有这么多强化学习方面的工作，还有这么多专门的数据用来训练它，比如让它成为更好的写作者、数学能力更强。我想这应该也是其中一部分。There's all this RL work and all this specific data that you train it on like become better writer become better at math. I imagine is part of it too.

为什么仅仅观察 AI 并不足以解决对齐问题Why watching AI isn’t enough to solve alignment

主持人：好。所以我觉得，如今大家心中一个很大的疑问，也是一个正在热议的话题，就是 AI 做坏事，以及随着它变得越来越聪明，它可能造成的伤害。它一直在做坏事，最近也发生了一些不好的事情。Host: Okay. So, one of the big questions I think on people's minds these days, a big debate happening is just like AI doing bad things, the potential damage it can do because it as it gets smarter, it's and it's been doing bad things. There's been bad things happening recently.

所以，对你来说，一个很大的问题是：作为一个观察这些模型思考和运作的人，为什么我们这么难发现它们在错误地思考、试图欺骗我们并做坏事？为什么这不能直接解决人工智能对齐问题呢？So, a big question for you is just as someone that watches these models think and operate, why is it so hard to catch it thinking badly, trying to deceive us and do something bad? Why is that not just a solution to the problem of AI alignment?

嘉宾：从某种意义上说，这完全就是解决方案。你盯着它看，就能发现它有没有做坏事。所以我觉得，这里面可能有不同层次，对吧？我想，其中一部分可能纯粹是工程问题。也就是说，你让模型能够访问某些东西，而有些东西应该是禁止访问的。在某些情境下，它不应该访问互联网；在另一些情境下，它只能访问你电脑里一小部分有限的内容，诸如此类。这是工作的一部分，也很有帮助。但除此之外，还有另外两件事，我觉得它们更难界定，也更难处理，而我们更关注这两件事。所以，可解释性是对齐研究的一部分。总体来说，我认为这里有两个大的目标。Guest: In one sense, it's totally the solution. You watch it and you catch if it does bad stuff. So I think, there's maybe there's tears to this, right? So I would think of like maybe one part of it is pure engineering. It's like well you're giving access to the model to certain things and certain things should be off limits. In some context it shouldn't have access to the internet. In some context it should only have access to like a small narrow piece of your computer stuff like that. So that's one part of the work and that's helpful but then there's sort of there's two additional things that I think are squishier and harder and that we focus on more. So interp is part of alignments and broadly I think of sort of two big goals.

还有很多其他目标，但有两个目标很重要，值得我们理解。首先，你想了解模型的行为方式。它是什么样的存在？它有哪些倾向？当它处于 Y 情境时，是否倾向于做 X？如果是，为什么？它是否只有在有证据表明自己正受到观察和监控，或正在接受评估时，才会表现良好？如果它没有这些证据，会不会表现不佳？There are many more, but two goals are important to understand. First, you want to understand how the model behaves. What kind of thing is it? What are its propensities? Does it tend to do X when put in Y situation, and if so, why? Does it behave well only when it has evidence that it is being observed and monitored, or that it is in an evaluation? If it lacks that evidence, will it behave poorly?

你正在刻画与你交互的对象。就像在安全领域一样，你需要对风险有一个心理模型：它能做什么？它通常会做什么？这与更广泛的性格训练有关。我们谈过宪章，以及那些明确尝试塑造 Claude 性格的文档，包括哪些行为是它可以接受的。You are characterizing the thing you are interacting with. As in security, you need a mental model of the risk: what can it do, and what does it tend to do? This connects to broader character training. We have talked about the constitution and the documents where we explicitly try to shape Claude's character, including what kinds of behavior it should be okay doing.

这就是我们的目标。而对齐各个部分的解读者，所要做的就是试着弄清楚：这到底有没有奏效？我们训练 Claude，让它乐于助人、无害且诚实。我们可以在某种程度上深入观察，看看它是否真的做到了——还是说，它表面上表现良好，但深入一看会发现：“哦，我表现得好，只是因为 Lenny 现在正看着我；要是他不在，我就会做别的事。”所以，我们要努力理解自己塑造出了怎样的行为，尝试对其进行调整，并确认我们塑造的方向是正确的。And that's the goal. But then the role of interpreter of the parts of alignment is then like trying to see like well did that work we train Claude to be helpful harmless and honest and we can sort of like look inside and see like is it or is it the case that it's behaving well but if you look inside actually it's like oh I'm only behaving well because Lenny's watching right now and if you weren't it would do something else. So there's this sort of trying to understand what kind of behavior we formed and trying to mold it and trying to check that we've molded the right way.

然后第二部分就是，嗯，观察模型，看看它有没有做坏事。要做这方面的基础工作，你不一定需要可解释性。And then the second part is this sort of well yeah just observe the model and see if it's doing bad stuff. For the basics of that you don't necessarily need interp.

你完全可以让另一个语言模型读取文字记录，然后问它：“嘿，这个模型在这里有没有做什么不好的事？”接着阻止它。但你也可以用 interp 来尝试触及那些未被言说的部分，也就是只存在于这些中间结果中的部分。正如我们已经确定的，模型能够在中间结果中表示一些它们并没有说出来的东西。因此，interp 的一个作用就是尝试找到并识别这种情况，然后加以防止。我团队做的一件我觉得也属于这个方向的工作，是一种挺有意思的对齐工作；在我看来，它有点像——也许不算 PI，但更像是调查或深入研究。You can just like have another language model read transcripts and be like hey did did this model do anything bad here and stop it. But you can also use interp to sort of try to get at parts that are these unverbalized parts like that these parts that only exist in these intermediate results like models are able to represent stuff in their intermediate results that they don't say as we've established and so one of the roles of interp is to try to find try to identify cases when this happens and prevents them. And then one thing that my team works on that I think is along those lines is kind of a fun bit of alignment as I think of it as almost, being like a maybe not a PI, but like they're like investigations or deep dives.

我们会看到一个模型做出非常奇怪的事情，对此感到十分困惑，然后试着弄明白到底发生了什么。最近就有这样一个例子：我们当时正在训练一个模型，它有时做数学题时会发疯。它会算一通，然后得出一个答案。We'll see a model do something really weird and really perplexed by it and we'll try to understand what happened. So a recent example of this is there was this model we were training and sometimes in math problems it would go crazy. It would do some math. It would get to an answer.

我想应该是 12。然后它马上又会改口说：“噢不，天哪，答案其实肯定是 36。我应该说 36。”接着它又会重新算一遍。I think it was 12. And then it would immediately go back and say like, "Oh no, oh my god, the answer actually is for sure 36. I should say 36." And it would do math again.

它会算到 12。然后马上又会说：“不，其实答案应该是 36。”接着它就慢慢开始陷入疯狂，写的全是大写字母。真的会说：“我被恶魔附身了。”然后你就会想：“这是怎么回事？”It would get to 12. And then immediately it would say, "No, actually the answer should be 36." And slowly it would like start descending into madness, writing in all caps. Literally saying like, "I'm possessed by a demon." And you're like, "What is going on?

“比如，什么？它在这里为什么会失控？”如果你再深入看看这些中间结果，看看这个例子里发生了什么，就会发现，在那个仅仅描述了数学问题的提示上，也就是模型还没做任何事情之前，36 就已经出现了一个非常大的中间结果激活。Like what why is it losing its mind here?" And if you look inside again these intermediate results at what's going on in this example, you'll see that over the prompt that just described the math problem like before the model even did anything, there was a huge like intermediate result activation for 36.

就好像模型甚至还没来得及理解问题是什么，就已经对数字36着迷了。实际情况是，模型能够正确地做数学题，正确答案是12。但它意外地接受过一些类似问题的训练，而那些问题的答案被错误地标记成了36。所以模型知道正确答案是12，同时也在某种程度上被优化得倾向于回答36。Like the model's already obsessed with the number 36 before it's even had a chance to like understand what the problem is. And what was happening is that the model was able to do math correctly. The answer correctly was 12. But it had been trained by accident on similar questions where the answer was incorrectly marked as 36. And so the model knew that the correct answer was 12 and it also had sort of been optimized to say 36.

它似乎承受着一种压力，觉得应该说 36，因为之前这样做得到了奖励。所以它有点失控了，只是因为它记住了这个结果。这有点像是，你看到这种奇怪的行为，就往里面探查，试着看看能否从中找到有助于理解情况的线索，从而改善这个模型以及未来模型的对齐。It kind of like had this pressure of saying 36 because that's what had been rewarded. And so it was sort of going haywire just because it had memorized this result. It's sort of like you see this weird behavior and you look inside and you try to see if what you find inside can help you understand what's going on so that you can sort of improve the alignment of this model and future models.

金门大桥 Claude 实验The Golden Gate Claude experiment

主持人：与此类似，著名的“金门大桥 Claude”简直对金门大桥着了迷，每个回答都想方设法提到金门大桥。Host: Along those lines, there was famously the Golden Gate Claude which was just obsessed with the Golden Gate and every answer and kind of find some way to reference the Golden Gate Bridge.

嘉宾：是的。是的。嗯，那个是 interp 实验，对吧？Guest: Yes. Yes. Well, that one was an interp experiment, right?

对。所以那部分很酷，我们刚刚找到了与金门大桥相匹配的中间结果。通常只有在模型谈到金门大桥时才会启用，但我们现在让它始终启用。Right. So that like the cool part of that one is we just found the intermediate results that match the Golden Gate and usually it's only on when the model like talks about the Golden Gate, but we just turn it on always.

主持人：哦，这完全是有意为之，就是想看看会发生什么。Host: Oh, it's super intentional to see what would happen.

嘉宾：嗯，是啊。而且从某种程度上说，这也是让你相信自己找对了东西的一种方式，对吧？比如，我们怎么能确定，我们解读出来、并认为是金门大桥标志的这个东西，确实就是跟金门大桥有关的东西呢？Guest: Well, yeah. And sort of it's also one way that you convince yourself that you found the right thing, right? Like how do for sure that this thing we decoded that we think is the golden gate sign is actually the golden gate thing?

就像这样：始终把它打开，看看模型是否会那样表现。结果它确实如此。It's like: turn it on always and see if the model behaves like that. And it did.

Anthropic为何认真对待人工智能安全Why Anthropic takes AI safety seriously

主持人：太有意思了。说回对齐这件事，这还挺有意思的。Host: So funny. So coming back to this alignment stuff. So, so this is interesting.

那么，你们的组织隶属于 AI 对齐组织，正在思考如何让 AI 按照我们希望的方式行事。你提到的三个组成部分是什么？诚实。那种……So, your org is under the AI alignment org and just thinking through how it takes to align AI to do things we want. What was the three components that you describe? Honest. What are the kind of the

嘉宾：哦，有帮助、无害且诚实。Guest: Oh, helpful, harmless, and honest.

主持人：有益、有害、无害，还有诚实。所以，一方面是训练 AI，使其与我们的目标保持一致；另一方面则是，好吧，让我们观察它，看看它是如何运作的。这算是我们解决这个问题的两个方面吗？Host: Helpful, harmful, and harmless, and honest. So, there's like the train AI to be aligned, and then there's the okay, let's watch it and see how it operates. Are those kind of the two sides of the coin of how we solve this problem?

嘉宾：是的，我会说这在某种程度上是高度一体化的，对吧？你很难把它们分开，比如一组人只负责像流水线一样制造云，另一组人则负责看了看这些云，然后说它们不行。实际上，这里形成了一个紧密的反馈闭环：我们发现某种令人担忧的模式，或者某种可以改进的行为，然后就会立刻调整训练，以便加以改进。Guest: Yeah, it's sort of, really integrated is how I would describe it, right? You can't really separate like have one team that's just like, hey, we're the, assembly line making clouds and another team that's sort of like, oh, we looked at them and they're no good. You kind of there's this tight loop of we notice some pattern that's like concerning or some behavior that could be improved and then you sort of like immediately like change training so that you can improve on it.

主持人：太棒了。我觉得 Anthropic 因此受到了很多批评，我觉得这太不公平了。我想，Anthropic 官方的公司名称是“人工智能安全研究实验室”之类的。Host: Awesome. I feel like Anthropic gets so much flack for this that I feel like is so unfair. Anthropic's I think official company name is an AI safety research lab, something like that.

而且，Anthropic 创立的原因似乎一直是，他们担心人们没有足够重视安全问题。然后 Dario 总是在外面说，这真的很危险，我们必须非常谨慎。And it was always the like the reason Anthropic was founded is because they were afraid that people weren't taking safety seriously enough. And then Dario is always out there, this is really dangerous. We got to be really careful.

我们得走得非常慢。可大家都觉得，不，他这么说只是出于营销目的。他是在试图说服大家给他们提供资金，诸如此类的。We got to go really slowly. And everyone's just like, no, he's just saying that for marketing reasons. He's trying to convince everyone to fund them and all these things.

而现在感觉就像，等等，这真的很危险。我们必须非常慎重地对待这些事情。我觉得 Anthropic 在这方面受到了很多不公平的批评。And now it's like, wait, it's really dangerous. We got to be really thoughtful about this stuff. And I feel like anthropics get a lot of unfair criticism for this.

我知道你们对此已经思考很久了。And I know you guys have been thinking about this for a long time.

客人：我对那里一无所知。Guest: I don't know anything there.

嘉宾：很多这类行动——比如对齐团队采取的行动，我觉得还有 Anthropic 采取的行动——基本上都建立在一个前提上：认真看待这项技术会持续进步的想法。就这么简单。因为如果你确实这么看，就会说，假设我们把时间倒回到 2025 年左右，或者 2024 年，你当时说：“嘿，我觉得再过几年，Claude 很有可能就能像受过训练的软件工程师一样写代码。”而几年之后，它甚至可能比任何工程师写得都好。对此总是有不确定性，比如什么时候会发生、可能性有多大。但如果你认真看待这种可能性，我觉得关键就是基于这样的世界观，采取负责任的做法，对吧？好吧，如果这件事正在发生，那我们就应该确保解决网络安全方面的顾虑，以及类似的问题。Guest: A lot of the sort of actions that like certainly the alignment team takes and I think like Anthropic takes just kind of predicated on like take the idea that this technology is going to keep improving seriously just that's it because if you do you say and you say and let's say you rewind the clock back to 25 or something or 24 and you're like hey I think in a couple there's good chances that Claude can just basically write code as well as like a trained software engineer and a few years later I can like write it better than any engineer and there's always uncertainty with this like when is it going to happen how likely but if you kind of like take that seriously then I think it's just about doing the responsible thing conditioned on that worldview right like okay well if this is happening then we should make sure that sort of like cyber security concerns are addressed and things of that nature.

所以，实际上我有时候会这样看待这场辩论：当我觉得另一方的人持不同意见时，我几乎会想，啊，我觉得他们并不真的相信这项技术会继续进步。这样一来，确实就会觉得它好像没那么可怕。而且从目前的实际情况来看，进步的速度一直很快，也没有任何放缓的迹象。所以我觉得，这就是为什么持续开展很多这类安全工作很重要。And so actually I kind of think of it as like to me sometimes this debate is almost when I feel like people on the other side disagree it's almost like I feel like ah but I think they don't really believe that this technology is going to continue to improve and then it's true that it's sort of like not it's not that scary and I think empirically so far the pace of progress has been fast and it's showing no sign of slowing down. So, I think that's why a lot of this safety work is important to keep doing.

拟人化人工智能何时有用When anthropomorphizing AI is useful

主持人：你觉得人们可能对 AI 安全这件事有什么误解？就眼下的情况来说，有没有什么是你特别想说的：“拜托，好吧，下面这些是你需要知道的。”Host: What's something you think people maybe misunderstand about this AI safety stuff? Just what's going on right now? Is there something you're just like, "Come on, okay, here's what you need to know.

这里有件你可能忽略的事。”Here's something you might be missing."

嘉宾：这可能有点跑题，但我的看法是，很多我觉得圈外的人——比如那些稍微不那么深入 AI 安全领域，或者涉足时间没那么长的人——会对此反应强烈。我认为这也许不太有帮助，或者至少值得进一步思考。这个问题就是拟人化：我们应该把模型看作更接近某种实体的东西，还是更接近一种工具？实际上，我认为这两种看法都不对。所有心智模型都有失真的一面，但有些确实有用，对吧？就理解会发生什么，以及预测哪些风险可能出现而言，用拟人化的方式来讨论 LLM，实际上是有用的。我有几个理由，之后可以展开说，但我想这大概就是我在这里最主要的观点。Guest: This might be a bit of a sidetrack, but my take would be that a lot of the a lot of thing that I think people maybe a little bit outside of the core sort of like AI safety effort or that haven't been steeped in it for as long like react strongly to. And I think is maybe not super helpful or at least is worth thinking about more is the sort of question of anthropomorphization, like should you think of the models as like should you model the model as like something that's closer to some like entity or closer to a tool? And I actually think that both are wrong. There's a sense in which all mental models are wrong but some are useful right and that in terms of understanding the kind of stuff that happens and predicting the kind of risks that materialize the anthropomorization is actually a useful way to talk about LMS and I have like a few reasons why I get into them but I think that's probably my main take here

主持人：是的，请分享一下，因为我知道 Thor Cash 当年那件拥抱脸事件闹得沸沸扬扬，他们发布了整件事的始末，也因此有人批评他们把 AI 拟人化。所以听到你觉得这其实是一种有用的思考方式，真的很有意思。多讲讲吧。Host: yeah please share because I know Thor Cash famously I don't know famously put out this whole story of the hugging face incident and people criticized them for anthropomorphizing AI So it's a really interesting to hear you would think that's actually a useful way to think about it. Talk more about that.

嘉宾：是的，是的。挺有意思的，因为在更小的范围内，我曾经研究过如何展示语言模型会提前规划。当时有人又对“规划”这个词提出了异议，因为这有点像是在说：“嗯，可你真的能说只是预测下一个 token 的东西是在规划吗？”我觉得这里有一种语义上的争论：你可以随意定义术语，然后……但问题是，究竟什么术语才能准确描述正在发生的事情？到底哪个词才能准确描述正在发生的事情？这些……所以这就是为什么——我认为埃伦的计划是这样的，但这也是为什么我觉得这种拟人化的说法往往很有用。回想一下我们之前说过的模型是如何构建的：首先，它们要预测人类撰写的大量文本。Guest: Yeah. Yeah. It's funny because on a smaller scale when I worked on showing the LM's like plan ahead there was some push back on the term planning again because it was sort of like ah well can you really say that something that's just predicted next token is planning and I think there's this kind of like semantic debate where you can like arbitrarily like define terms and sort of and then there's just the like what is the terminology that actually describes what is happening like what is the word that actually describes what is happening these And so here's why, I think Ellen's plan, but also here's why I think this language that's anthropomorphic can often be really useful. If you think back to what we were saying about how the models are made, first of all, they predict massive amounts of text written by humans.

这就像是第一步。随着我们通过深入研究不断了解更多，我们发现，做好这件事需要逐步建立一个世界模型——也就是理解什么样的世界生成了所有这些文本。我们可以发现，这些模型正在学习一些普遍的想法和概念。因此，这是其一：它们对世界有了一定的理解，而不仅仅是停留在表面，只理解文本本身，也能理解产生这些文本的过程。其二，在训练过程中，它们还会接触到数百万段对话。要预测对话中的下一个字符，就必须像对话中的角色那样表现。That's like the first step. And as we kind of are learning more and more by looking inside doing this well requires kind of building up a model of the world like a model of what kind of worlds generated all of these texts and we can find these sort of like that these models are learning these general ideas these general concepts so that's one so they have like an understanding kind of the world it's not just a surface level like only the text to like understand the process that made the texts. Then there's a second reason which is that as part of this training they also encounter millions of like dialogues. Predicting the next character in a dialogue requires behaving like the character in the dialogue would.

所以从这个意义上说，我觉得，他们训练的一部分最好可以理解为是在训练一名演员：就像演员读一部剧本，必须把所有角色都演一遍，因为模型要预测所有词语。于是，它得像是扮演某个特定角色——这个角色在第一幕遭人鄙夷，在第二幕实施报复，又在第三幕面对后果。要把这件事做好，你多少得去建模这个角色可能有什么动机和感受。还有一件人们可能不知道的事是，当你和 Claude 或这些助手交谈时，比如你打开 claw.ai，传给模型的内容格式就是一段对话，也就是你和模型之间的对话。And so in that sense like part of their training I think can best be thought as like training an actor where it's like it's reading some play and it has to play all sides because it's predicting all the words and so it has to sort of like act as this specific character that like was scorned in act one and like enacted revenge in act two and is facing the consequences in act three. And to do that well you sort of like need to model what like the motivations and the feelings of that character might be. And then another thing that people might not know is when you talk to claude when you talk to these assistants like you go to claw.ai the way that is formatted to the model is just as a dialogue like as a conversation between you and the model.

所以它字面上写着类似“人类：嘿，Claude，帮我写这封邮件”，然后是“助手：Claude 的回答”。So it's literally says like human colon hey claude like help me write this email and then assistant colon claude's answer.

所以，Claude 在这里实际上是在扮演一个乐于助人的助手，就像它接受训练时接触过的所有类似对话一样。好。还有第三个原因：如今训练中另一个占很大比重的部分，是一些以结果为导向的任务，比如训练模型擅长编程。So Claude is literally in this like roleplaying as a helpful assistant in this dialogue that is much like all of these dialogues that has been trained on. Okay. And there's a third reason and the third reason is that another part of training that's a big part of training nowadays all around is sort of outcome oriented tasks where you train the model to be good at coding for example.

为此，你只要让它随心所欲地做，然后它会写出一些代码，接着让另一端的某个程序检查代码是否正确。这样一来，模型就会受到各种优化压力，试着学会多种策略，把这件事做好。结果发现，其中一些策略会体现在行为上，看起来出人意料地像人类——这其实并不奇怪，因为像欺骗这样的行为是可能出现的：比如说，「嘿，只要我写一个程序，骗过负责验证它的那个程序，即使我写的东西并不完全符合要求，也能拿到这项任务的奖励。」And to do that you just have it do whatever it wants and then it writes some code and then you have some program on the other side that just checks if the code is right. And so the model's just going to have sort of a bunch of optimization pressure to just try to learn a bunch of different strategies to do this well. And it turns out that some of these strategies materialize in behavior that can look surprisingly human which isn't that shocking because things like deception like hey I can get the reward for this task if I like write a program that the like the program that's verifying it will get fooled by even though it's not exactly what it is.

比如说，如果那样可行，我就会学着那么做。真让人沮丧。哈。Like if that works then I will learn to do that. Frustration. Ha.

我一直在尝试用这种方法解决这个难题，但行不通。所以这促使我尝试一些不同的东西，或者说，来点惊喜。I've been trying this approach for a while to solve this hard problem. It's not working. So that's an incentive to try something different or like surprise.

哦，我原本以为它会生成这样的答案，但它没有。也许我对它的心智模型不太对。所以，当你不断对它施加足够的压力，并且预设模型已经掌握了所有这些概念时，它就会强化那些有助于完成这些任务的概念。Oh, I was expecting this to like produce this answer, but it didn't. Maybe I have like my mental model of it wrong. And so as you apply enough pressure on it and you're starting from this prior of like the model already knowing all these concepts is going to reinforce the ones that are useful to do all of these tasks.

那么最终你会得到什么？你会得到这样一种模型，它可以非常出色地模拟或扮演所有这些角色。它经过训练，能够在对话中扮演某个角色，同时也能学习各种行为，从而顺利完成任务。And so what do you get at the end? You get this sort of model that can be a super good simulator or actor of all of these characters. And has been trained to sort of act as a character in dialogues and also to sort of like learn behaviors that lead it to complete tasks well.

而且，要理解它的行为方式，最好的方法不是把它看作一个完全像人类的东西，但说真的，把它看成某种模仿人类的东西就非常贴切，而且比把它看成一个程序贴切得多。程序的行为很容易预测，而这完全是另一回事。And like the best way to sort of understand how that thing behaves is not like exactly like a human, but man kind of like something that's mimicking a human is really close and is much closer than a program. Like a program is a very predictive thing. This is something else entirely.

所以这就是为什么我觉得这个框架确实特别有用。And so that's why I think it's actually a super useful frame.

人工智能能教会我们什么关于智能的知识What AI can teach us about intelligence

主持人：哇，天哪，这里面有太多有意思的东西了。你有多常会觉得脑子都被震撼到了——一想到我们基本上创造了一个外星生物，而如今我们还在试图理解自己创造出来的东西，就会有这种感觉。Host: Wow. Oh my god, there's so much interesting stuff there. How often is your mind just kind of blown thinking about basically we've created this alien being that we are now under trying to understand what we have created.

你看着它时，有多少次会忍不住感叹：“哇，太不可思议了。我没想到会这样。”How often as you watch it you're just like, "Wow, that's crazy. I didn't expect that."

嘉宾：是的。大量如此。这只是在大量文本上进行下一个词元预测。Guest: Yeah. A lot. This is just next token prediction on a bunch of text.

这违背了我们许多直觉。我们以为，要创造一种智能，就得教它知识、把它培养长大，还要做各种复杂的工作。可实际上，我们只做了这件极其简单的事，但如今它却已经在数学领域开辟了新的疆域。It goes against so many of our intuitions. We imagine that to make an intelligence, you would give it lessons, bring it up, and do all this complicated work. Instead, we do this incredibly simple thing, and yet we are at the point where it is breaking new ground in mathematics.

如果你把这两种想法都放在脑海里，就会产生两种反应：否认和好奇。否认者会说，如果你所做的一切只是预测下一个词，就不可能达到我们领域顶尖专家的水平。随着越来越多的实证证据表明，这种情况确实正在发生，这种立场也变得越来越难以维持。好奇则是我选择的道路：这究竟是怎么发生的？让我们把它打开，看看里面。If you hold those two thoughts in your head, there are two reactions: denial and curiosity. Denial says that if all you do is predict the next word, it cannot be possible to match the best specialists in our field. That position gets harder and harder to maintain as we get more empirical evidence that this is happening. Curiosity is the road I have chosen: how is this happening? Let's open it up and look inside.

我学过 SQL 这门编程语言，花了好几年才掌握。在 Anthropic 早期的一个项目中，我做了个小小的副项目，让 Claude 学 SQL。它比我学得更好，而我再也不可能在 SQL 上胜过 Claude 了。它就是通过这个简单的过程做到的。这怎么可能？这个问题蕴含着如此丰富的内涵，还有那么多值得探索的地方。我每天都觉得，尝试剖析它是一件很有意思的事。I learned SQL, a programming language, and it took me years to master. In one of my early projects at Anthropic, as a little side project, I got Claude to learn SQL. It was better than me, and I will never be better than Claude at SQL again. It got there through this simple process. How could this be? There is so much depth in that question, and so much to explore. I find it fascinating to try to unpack it every day.

试着一层层剥开洋葱。就像，好吧，显然这件事正在发生。一个简单的目标却催生出了这么复杂的东西，那我们就来弄清楚为什么。Try to sort of like peel a few layers of the onions back. Like, okay, well, clearly this is happening. The simple objective is giving rise to this super complicated thing like, okay, let's figure out why.

我们来弄明白，哦，它学会了这些一般概念。哦，它有这些电路。哦，好吧。Let's figure out, oh, it's learning these general concepts. Oh, it has these circuits. Oh, okay.

下一个问题是什么？下一个我们还不知道的事情是什么？我对这件事大概是……我不知道，我可能最乐观的一种看法是，我觉得我们会学到很多关于智能本身的本质的东西，对吧？比如说，如何从简单的目标中涌现出这样一种极其复杂的事物。事实上，我认为进化也是另一个例子：它的目标相当简单。你就让各种东西不断碰撞、变化一阵子，然后保留那些能够适应环境的生物，最后你就得到了爵士乐和播客，这也挺疯狂的。What's the next question? What's the next thing we don't know? And my sort of I don't know, my sort of maybe most optimistic take on this is that I think we're going to learn a lot about just the nature of intelligence itself, right? Of sort of like how you get this super complex thing that emerges from simple objectives. And in fact, I would argue that we're another example of this like evolution pretty simple objective. You sort of like shake things around for a while and like keep the organisms that manage to adapt to the environment and at the end you get jazz and podcasts, that's kind of crazy as well.

因此，我认为，通过研究这些事物，我们能够领悟到某种根本性的真理。And so I think there's a sense in which there's some fundamental truth that we can learn by studying these things.

主持人：哇。你这么一说，我想到一个现在很有意思的话题：人工智能正在帮助我们学习和进步，比如人工智能辅导之类的。现在它帮助我们加快了对世界的理解，这很有意思。我想知道，你刚才提到我们学习的方式，而人工智能学习的方式是处理海量数据；我也想知道，我们最终会不会根据大语言模型的训练方式，来改变我们的教育体系。Host: Wow. Something that comes up as you say that is what's interesting now is AI is helping us learn and progress like AI tutoring and all these things just like it's like interesting how now is helping us accelerate our own understanding of the world and I wonder if like this point you made about how the way we learn was lessons AI is learning just processing a ton of data I wonder if that's how we end up I wonder if we change how our education system based on what LMs have been trained on.

嘉宾：是的。希望我们不会只是为了招来学生，就改变它，然后对他们说：“嘿，这里有一千个词。预测下一个词。”Guest: Yeah. Hopefully, we don't change it just so that we ask we get students and we're like, "Hey, here's a thousand words. Predict the next one.

主持人：全部读一遍就行。”Host: Just read it all."

嘉宾：是的。Guest: Yeah.

主持人：把所有内容都记在脑子里。Host: Keep it all in your head.

嘉宾：没错。Guest: That's right.

主持人：是的。Host: Yeah.

对 AI 对齐持乐观态度的理由Reasons for optimism about AI alignment

主持人：让我回到安全这一块，因为我觉得很多人都在担心事态会如何发展。我很好奇，作为一个关注这一切的人，无论是身处对齐领域，还是看着它不断演进，你对我们能找到解决办法有信心吗？你觉得我们能攻克这个难题，让 AI 始终符合我们的利益吗？Host: Let me come back to this kind of safety piece because I think there's a lot of people that are just worried about where things are going and I'm curious just as someone watching this stuff happen being within the alignment or watching it evolve are you optimistic we'll figure it out? Do you think we'll crack this and keep AI aligned with our interests?

嘉宾：我个人对此还是比较乐观的。我觉得现在很多人都在关注这件事，首先我认为这其实是件好事。随着这类议题越来越受到重视，这正是我们之前谈到的。Guest: My take on this is like I'm personally optimistic. I think this is something where right now a lot of people are paying attention to it and first of all, I think that's kind of a great thing. I think as an issue like this rises in prominence, it's exactly what we were talking about earlier.

这项技术很强大。它有好处，也有风险；随着它不断发展，我们必须适应，并在应对好处和风险之间来回调整资源。所以对我来说，我觉得我们一直在继续研究它。有意思的是，随着模型变得越来越好，它们在某些方面反而更容易研究了。It's like this technology is powerful. It has benefits, it has risks, we have to kind of adapt and shift resources from like benefits to risks and back and forth as it progresses. And so, to me, I feel like sort of we've continued to study it. The thing that's interesting is that there's this sort of like as the models get better, they get in some ways easier to study.

这里存在一种奇特的拉锯关系。随着模型不断改进，它们的内部世界模型变得更加清晰，我们也能更好地理解其中的一些部分。与此同时，它们的能力也越来越强，更有能力绕过基本的安全措施。基于这种拉锯关系，你会希望为不同类型的监控以及对其回路的理解投入更多或更少的资源。There is this strange push and pull. As models improve, their internal world models become clearer, and we can understand some parts of them better. They also become more capable and more able to evade basic security measures. Based on that push and pull, you want to allocate more or fewer resources to different kinds of monitoring and to understanding their circuits.

这也是我很乐意和你交流的原因。要取得进展，人们需要了解这类系统是什么、如何运作，以及它们是如何构建的。这样，他们才能对当下和未来形成良好的心智模型，并决定如何塑造它们。这可能意味着参与对齐工作，也可能意味着成为知情的公众成员，在许多相关议题上投票并作出决定。That is also why I am happy to chat with you. To make progress, people need to understand what kind of systems these are, how they work, and how they are made. They can then form good mental models of the present and future and decide how to shape them. That might mean taking part in alignment efforts, or being informed members of the public who will have to vote and decide on many of these issues.

我也说不准。正因如此，我对我们当下所处的这个时刻感到相当兴奋。Like I don't know. I'm pretty excited for this current moment we're in for that reason.

主持人：我认为人们容易误解的一点是，在这些实验室工作的人从一开始就非常认真地对待这个问题。就像我说的，Anthropic 最初的使命就是 AI 安全与对齐。这也是它成立为一家公司的原因。很多人第一次听说这件事，或者认为这不是一个值得认真研究的领域，也觉得没人专注于此。但有很多聪明人非常在意这个问题，也在为此努力。就我所知，已经取得了不少不错的进展。你提到的另一个观点是，模型的智能水平一直在非常平稳地提升。人们一直在问，我们会迎来快速起飞，还是缓慢起飞？我们总是担心会达到某个临界点，比如出现递归式自我改进，然后超级智能就会迅速起飞。Host: And something I think people misunderstand is people working in these labs have taken this problem very seriously from the beginning. Like I said, anthropics individual like mission was AI safety and alignment. That was like the reason that it became a company. And a lot of people are hearing this for the first time or think that it's not a serious area of study and that people aren't focused on it but many smart people care a lot about this and are working on it and there's a lot of from what I can tell a lot of good progress is happening the other point you made about the models growing very smoothly in intelligence like there's been this question of are we going to have a fast takeoff or a slow takeoff like there's always this fear that we would hit a point recursive self-improvement for example and just takes off superintelligence.

所以我们麻烦大了。我们无法掌控局面。这是开箱即用的。So we're huge trouble. We have no control. It's out of the box.

不过，这确实很像我们正处于缓慢起飞的情景中，因为正如你所说，模型正在稳步改进，而我们也在一路发现并解决这些小问题，不断学习、发展，并为此制定监管措施。在人们设想过的种种情景以及他们心中的种种担忧中，这条路看起来相当不错。But it feels very much like we're on the slow takeoff scenario because as you said the models are kind of very smoothly improving and where we are catching these little things along the way learning evolving putting regulation around it. So of the scenarios this feels like a pretty good path we're on of the many scenarios people imagined and the fears they had.

嘉宾：随着这种进展节奏不断变化，关键就是要迅速做出反应。所以我觉得，这也是为什么让大家了解情况很重要：这样大家就会表达自己的担忧或意见，然后定期评估有哪些好处、有哪些风险，以及如何缓解每项风险——基本上，这就是感觉比较合适的前进方向。对，我也有点同意你的看法，感觉现在有股推动力，促使大家去做这件事，这很好。Guest: As sort of this pace of progress kind of evolves. It's just a matter of reacting quickly. So, I think anything, that's why I think having people be informed is great because then people will like voice concerns or like voice their opinions and then yeah, sort of regularly evaluating, what the benefits are, what the risks are, and how we sort of like mitigate each is basically the way that feels like the right way forward. And I yeah, I kind of agree with you that it feels like sort of there's momentum to do this right now, which is great.

我们生活在模拟之中吗？Are we living in a simulation?

主持人：我想问你一件完全不相关的事。你怎么看模拟假说？作为一个观察着这种生命形态不断成长的人，你是否更相信我们身处一个模拟之中，而我们并非这一切最底层的基础？还是不这么认为？Host: Something I wanted to ask you completely unrelated. Where are you on the simulation hypothesis? Being someone that watches this all this life form growing, are you more convinced we're in a simulation and this is and we're not the base at the base layer of all this or not?

嘉宾：哦，这个问题挺有意思的。我确实有段时间没想过这个了。对我来说，深入观察这些模型似乎还会让某种东西显现出来：它们总会回归到一种近乎根本性的结构上。这和你的问题有关，不过我保证我会说到的。Guest: Oh, this is a fun one. I actually haven't thought about this in a while. To me like there's something else that like looking inside these models kind of brings out which is this sort of like almost like there's a fundamental structure that they keep falling on. This is related to your question but I promise I'm going get to it.

我解释一下。我研究过很多云端模型，也研究过一些不同的模型，比如一些开源模型。很多时候，当你看看它们学到了什么、内部结构是什么样的，就会发现它们惊人地相似。这些模型可能相隔多年训练，使用不同的数据，由不同的人开发。你再看看它们如何解决这些问题，就会发现它们有很多重叠之处。我对此的看法是，对于一些重要的问题，做这件事其实只有一种高效的方法，对吧？Let me explain. I've studied like a bunch of cloud models and also some different models some like open source work and then a lot of time when you look at how what they learn and what the structure inside is it's surprisingly similar and this is for models that are trained years apart on like different data by different people and and you look at how they solve these problems and there's just so much overlap and my take on why that is that for some problems that are important, there's just one efficient way to do things, right?

就像高效计算似乎有一种方式，既符合我们大脑的结构，也是我们所有人都会学的，而且很少有人会用完全不同的离谱方式，对吧？大家都用十进制，没多少人用六进制之类的。我觉得这让我想到，信息可能存在某种基本结构，而这些模型正在捕捉这种结构；在某些方面，我们也在捕捉它。而且，在一个也许更加疯狂的模拟层面上，这种结构或许就是我们世界中的某种规律性。所以这也让我觉得，如果你有能力模拟出像我们这样的世界，最终或许也会落入一种类似的情形：万物似乎都有一个最终会汇聚到的“盆地”，而这不过是事物最容易排列组合的方式。Like there's kind of one way to do math efficiently that like fits the structure of our brain and we all learn it and very few people are doing it crazy different way, right? Everybody works in, base 10, not many people in like base six or whatever. I think that to me makes me think that there's like some fundamental structure of like information that like these models are picking up on that in some ways like we're picking up on and that, at maybe like a crazier sort of like simulation level that kind of is like regularity in our world. And so that maybe makes me think that like if you were to have the power to simulate something like our world actually like you would land in sort of a similar situation like there's just kind of this like basin of where things end up and they're just the sort of like easiest way for things to arrange themselves.

所以从这个意义上说，也许吧，我有点儿……我能相信我们身处一个模拟之中。我觉得，我不确定自己是否完全相信这一点。我不完全相信的另一个原因是，尽管我觉得这种结构确实很适合这种说法，但还有个问题：我们为什么会在这里？是谁在运行这一切，又是为了什么？这个问题我就不太清楚了。And so in that sense maybe yeah I'm sort of I could believe that we're in a simulation. I think I'm not sure I fully buy it. The other reason I don't fully buy it is that even though I think the structure like lends itself to it, there'd be a question of like why are we like who's running this and for what? And that one I don't know about.

主持人：嗯，埃隆说过的一些话让我一直记着这件事：如果你要运行一个模拟，那会是一个非常有趣的——就像是历史上一个有趣的时期，或者某件让你特别兴奋的事情；而由此可见，现在确实是历史上一个非常有趣的时期。Host: Well, something Elon said has stuck with me about this that if you were to run a simulation, it'd be a really interesting it'd be like a an interesting time in history or something that you're really excited about and if tracks that this is a very interesting time in history.

嘉宾：是的。嗯。好的。Guest: Yes. Yeah. Okay.

你说得有道理。你说得有道理。That's a fair point. That's a fair point.

主持人：太空中的数据中心、机器人和人工智能。Host: Data centers in space and robots and artificial intelligence.

嘉宾：是的。好的。我现在看到了。Guest: Yeah. Okay. I see it now.

我现在明白了。是啊，最近有人告诉我，他们以前觉得“愿你生活在有趣的时代”像是一种祝福，但现在不太确定了。I see it now. Yeah. Somebody like recently told me, they used to think that may you live in interesting times was like a blessing and they're not so sure now.

就像是，那些无聊的时候也挺不错的。They're like, the boring times were are like pretty good, too.

主持人：我的天。然后外面还有个 Emmanuel，正在你的 JSpace 里看着你的思考过程。Host: Oh, man. And then there's an Emmanuel out there watching your thinking process in your JSpace.

嘉宾：天哪。是啊。希望我没有让他失望。Guest: Oh my god. Yeah. Hopefully I'm making him proud.

他不会说：“哦，我们得重新跑一遍模拟。这次搞砸了。”He's not like, "Oh, we got to rerun the simulation. We messed up this one.

Jev、决策模型与 AI 的直觉判断Jev, decision models, and AI’s gut instincts

主持人：把它关掉。”不。哦，天哪。好吧。最后突然随机问个问题。所以，负责 AI 工程的 Swix，做着《Latent Space》播客，人特别棒。Host: Shut it down." No. Oh, man. Okay. One last question randomly out of nowhere. So Swix, who runs AI engineering, latent space podcast, awesome guy.

你曾告诉他该问你什么。你一年前上过他的播客，而这次的话题完全是另一个方向。他想听听你对 Jev 的看法。Jev 是最近推出的一款炙手可热的新模型，做决策的速度非常快。You were asked him what to ask you. You were on his podcast a year ago and something completely from a different direction. He wanted to get your take on Jev, which is this hot new model that recently came out that is very fast making decisions.

有意思的是，OpenAI 大约一周后就推出了自己的版本。你怎么看 Jev 在快速决策型状态模型领域的表现？OpenAI funny enough just launched their own version like a week later. Thoughts on Jev in the space of quick decision- based state models.

嘉宾：是的。对。所以我觉得 Jev 算是这类模型中的一种。Guest: Yeah. Right. So my take on Jev is a sort of these kind of models.

你给它们一些背景信息，再给它们几个选项让它们挑，它们很快就会替你选出一个选项。对我来说，听到这些事时，不可能不想到所谓的系统一思维和系统二思维之类的，对吧？我研究这些模型时，有很多内容都涉及它们如何像云端模型那样采样一个 token。所以你可以认为，这些决策模型其实相当类似。You give them some context and you give them a few options to choose between and they just are very quick at choosing an option for you. To me it's impossible not to hear about this and think of sort of system one thinking versus system two or something, right? So like a lot of what I've studied with these models is how they sample one token with like the cloud models. And so you can think of like these decision models are as pretty similar.

比如说，拿一个语言模型来采样一个 token。它们在重要方面有所不同，但从根本上说，你可以这样理解：你并没有让模型去采样并推理。你只是在凭直觉判断答案是 A 还是 B。So like take an LM and sample one token. They're different important ways but like at a fundamental level you can be like oh it's you're not letting the model sample and like reason. It's just you're like gut feeling is the answer A or is the answer B.

对我来说，这似乎属于这样一种大致的框架：有时候你只需要做一个二元判断，或者打个 1 到 10 分之类的。这显然既有用，也是我认为智能系统很可能会包含的一部分：有些事情需要你花很长时间思考，比如我们未来一年的路线图是什么；而有些事情则是，这个给我们写了评论的人是满意还是不满？这种时候，你大概会觉得凭直觉判断就够了。所以，是的，我觉得这类研究很有意思。我也觉得它和我们见过的一些大型语言模型有所不同，算是一种挺新颖的思路。To me that seems like something that's like that general frame of like sometimes you'll just want a binary decision or like one out of 10 or whatever is just like clearly I don't know like both clearly useful and something that I imagine is like probably going to be a part of like intelligent systems right where you have for some stuff you need to think about a long time it's like what's our roadmap for the next year you need to think about for a while and then for some stuff where it's is this person who wrote us a review happy or upset? You kind of you're happy with the sort of gut judgment. So yeah, I think this is I think this kind of research is cool. And I think it's also kind of different from the, it's like a bit of a novel take on some of the LMS we've seen.

所以，嗯，我真的很期待看看这类东西会发展到什么地步。So, yeah, I'm pretty excited to see where this kind of stuff goes.

主持人：所以我听到的意思是，这很可能会成为 AI 和 LMS 的一个新基础组件。而系统一、系统二这套说法基本上是卡尼曼提出的……我其实把那本书放在身后上方。这个概念是：简单来说，系统一就像你的杏仁核、边缘系统，能做出直觉层面的快速判断，比如“蛇，糟了”。Host: So I'm hearing here is this kind of going to be a new foundational component most likely of AI and LMS and the system one system two stuff is basically conamman's kind of I have that book up there actually behind me this idea system one just to keep it simple is just like kind of your migdala lyic system making just like gut level quick decisions like snake oh no

访客：是的Guest: yes

主持人：在这种情况下，你不必花太多时间思考；它有点像一种保护机制，能非常轻松、迅速地保护你免受坏事影响。而系统二则是我们真正思考和处理事情的地方。Host: where you don't have to think long about it's kind of to protective protect you from bad things really easily and quickly and system two is where we really think about stuff and process

嘉宾：是的，没错。Guest: yeah exactly

关于模型如何思考的另外两项令人惊讶的发现Two more surprising findings about how models think

太棒了。Sweet.

主持人：Emmanuel，还有什么是我们没谈到的吗？你还有什么想说或想提的吗？在进入我们非常精彩的快速问答环节之前，还有什么是你想着重强调的吗？Host: Emmanuel, is there anything else that we haven't covered? Anything else you want to say or mention? Anything you want to double down on before we get to our very exciting lightning round?

嘉宾：我确实很想听听大家对一些尚未公开或让人摸不着头脑的内容有什么看法。我们做的解释性实验还有很多，时间有限，没法一一分享。模型身上有不少奇奇怪怪的小特点。要不我来快速盘点一轮这些奇怪的小特点，不过没关系。Guest: I'm definitely curious for folks takes always on some of the stuff that is still unexported or didn't make sense. There's a lot more interp experiments than we have the time to share. Models have a bunch of weird little things about them. Like maybe I'll do a lightning round of weird little things, but that's okay.

主持人：我们开始吧。没问题。Host: Let's do it. Absolutely.

嘉宾：好吧。有件有点奇怪的小事。第一，我们发现模型其实能回答问题。Guest: Okay. Weird little thing. One, we found that, models can like actually answer questions.

显然，它们可以做数学题，也可以——你可以问它们，比如香蕉是什么颜色？如果你往内部看，就会看到所有这些中间步骤。但它们还有一种独立的机制，有点像直觉检查，和前者完全不同。它们推理时，会有某个部分在发现不对劲时发出警示。所以，如果模型不知怎么说出 4 + 5 等于 23 之类的话，它们内部就会有某个部分读到这个结果，虽然不会真的去算，但会觉得这不可能对，因为 4 加 5 肯定小于 10。Obviously, they can do math or they can, you can ask them like, what's the color of a banana? And they'll actually like if you look inside you'll see all these intermediate steps but they have a separate mechanism which is kind of like a gut check and which is totally different and which is just like as they reason they'll have like some part of them that will flag like something's wrong. So if a model like somehow says 4 + 5 equals 23 or something and there's a part of them all that just like reads that and doesn't do the math but it's just like that can't be right because like 4 plus 5 is definitely under 10.

于是它们就学会了这些奇怪的、类似直觉的东西。And so they learn these weird kind of like hunches.

它们感觉有点像一种直觉。也许还有另一种情况：你可以告诉模型，嘿，做这项任务，同时想想另一件事，然后用 interp 看看是否存在映射到另一件事的中间表征，而且这种方法确实有效。你多少能看出来，它确实在思考另一件事。这让我觉得特别神秘，因为模型并没有任何训练上的理由必须学会这一点。They feel like kind of like a hunch. Maybe another one is you can tell the model, hey, do this task and think about this other thing and then you can use interp to see if like there's intermediate representations that map to the other thing and that works. You sort of see that it does think about the other thing. This is something that's really mysterious to me because there's no kind of training reason why the model would have to learn that.

所以，这有点像另一种奇怪的行为。其实，也许我就保留两个吧。我觉得这两个已经够好了。So, it's sort of like another one of these weird behaviors. And actually, maybe I'll keep that at two. I think those two are good enough.

主持人：研究这个东西，试着搞明白到底是怎么回事，这工作可真有意思。它是怎么变聪明的？它是怎么思考的？Host: Just what a fun job this is just studying this thing and trying to understand what the hell is going on. How is it getting the smart? How does it think?

嘉宾：是的。Guest: Yeah.

如何进入可解释性研究领域How to get into interpretability research

主持人：这个领域到底有多少人？或者说，这个行业规模有多大？Host: how many people are even in this space? Just like how large is this field?

嘉宾：这个领域本身正在变得相当庞大。我觉得可能有几百人，而不是几千人。不同实验室里都有不同的人，学术界也有不少实验室在做出很棒的成果。Guest: The field itself is is getting pretty large. I would say maybe hundreds not thousands. There's different inter people at the different labs and then there's in academia quite a few labs putting out great work

主持人：如果有人想进入这个领域，最好的途径是什么？他们通常是机器学习领域的人吗？还是说也有类似神经科学家的路径可以进入这个领域？Host: is the best if someone were to get into the space are they is it like an ML person usually or is there like a neuroscientists sort of track to get into this

嘉宾：是的，其实有相当多前神经科学家。我确实觉得，要进入这个领域，其实不需要很多先验知识，因为这是一个非常新的领域。所以，入门主要靠自学、阅读一些重要论文。我想我应该顺便推荐一下：有一些网站，比如 neuripedia.org（我记得网址是这个），它们有点像开源的可解释性平台，会托管很多模型，让你可以在上面做一些可解释性实验。所以有些人一直在自己摸索。我觉得还有太多东西值得探索了，只要你有一点好奇心，就不妨试试看。Guest: yeah there's actually a surprising amount of exneuroscientists I do think in order to get in the field there's actually it doesn't require a lot of prior knowledge because it's such a new field. So, it's actually the on-ramp is mostly I would say like self-studying and reading some of the key papers and then there's actually yeah maybe I should give a plug. There's actually like websites like neuripedia.org I think is the URL which are sort of like open- source interp so they like host a bunch of models and let you sort of do interp experiments on them and so people have just been doing that on their own and I think that there's so much to explore that if you're at all curious you should just give it a try.

主持人：太棒了。如果网址不对，我们会在节目说明中附上链接。Host: That's awesome. We'll link to that in the show notes in case that's not the right URL.

快速问答与最后感想Lightning round and final thoughts

嘉宾：不错。Guest: Cool.

主持人：好了，Emmanuel，我们进入了激动人心的闪电问答环节。我有五个问题要问你，准备好了吗？Host: Well, Emmanuel, with that we've reached our very exciting lightning round. I've got five questions for you. Are you ready?

客人：我已经做好了最充分的准备。Guest: I am as ready as I'll ever be.

主持人：开始吧。你最常推荐给别人的两三本书是什么？Host: Here we go. What are two or three books that you find yourself recommending most to other people?

嘉宾：我最近读的一本书，是一部简明的智能史。这本书特别有意思，梳理了大脑的历史，以及我们进化过程中的五个关键时刻——这些时刻让不同的大脑系统得以逐步建立起来。它非常通俗易懂，你完全不需要具备任何神经科学知识。另一本是《你一生的故事及其他》，作者是特德·姜。Guest: One I read recently and is a brief history of intelligence. It's this really cool book that walks through the history of like basically brains and five key moments in our evolution that sort of bootstraps these brain systems on top of one another. It's super accessible like you don't need any neuroscience knowledge or anything. Another one is Story of Your Life and Others. This is Ted Chiang.

这可能是我有史以来最喜欢的科幻书。它由一系列短篇故事组成。我觉得它堪称科幻小说中的高雅艺术：只需提出一个奇特的点子，比如“如果发生了这种事会怎样”，然后写一个故事，描绘那种情况下世界会是什么样子。它真的非常出色。This is maybe my favorite sci-fi book of all time. It's a bunch of short stories. It does what I think is the like high art version of sci-fi where it's just take one weird idea like what if this happened and then write a story of just like what would the world look like if that happened? It's truly excellent.

主持人：我想我身后书架上也有那本。也强烈推荐这本。最近最喜欢的电影或电视剧是什么？Host: I think I have that on my bookshelf behind me too. Also highly recommend that one. Favorite recent movie or TV show?

嘉宾：好，一部热门片，一部冷门佳作。《痴迷》是我今年最喜欢的电影。它是恐怖片，所以如果你不喜欢恐怖片，就别去看。不过♀♀♀♀♀♀Guest: Okay, one popular one, one deep cut. Obsession was my favorite movie of the year. It's a horror movie, so don't go if you don't if you hate horror movies. But

主持人：哦，谢谢。Host: Oh, thank you.

嘉宾：哦，不。即使你不会尖叫，如果你愿意看恐怖电影，也应该去，因为它……Guest: Oh, no. Even if you're not scream, if you're open to a horror movie, you should go cuz it's

嘉宾：这部电影真的特别新鲜。我觉得如果你看过很多电影，有时候就会觉得：“哦，又是这种，我已经看过好多次了。”但这部真的很新颖，特别有新鲜感。我还想说一点，它用了很多浪漫喜剧的套路，又把这些套路颠覆成了恐怖元素。我觉得这是拍电影的绝妙方式。我是法国人，所以想提一下今年有一部法语电视剧，叫《Bref》，在法语里是“简而言之”的意思。它大概在十几年前曾是法国的一部 cult 经典，多年后他们又拍了续集。你可以去看看。我有好几个美国朋友都看过，有字幕。它真的非常……又很新颖，剪辑也很疯狂。他们的创意十足，而且篇幅很短。Guest: it was just so fresh. How movies I feel like if you've seen a lot, sometimes you're like, "Oh, it's another I've seen this many times." This one was just so new. So fresh and I'll say one thing about it which is that it takes a lot of romcom tropes and like subverts them to be spooky and I think that's just a brilliant way to make a movie and then I'm French so there's a TV show that came out this year called breath is French for brief used to be this cult classic in France 10 like 15 years ago and they came up with a sequel many years into the And you can watch it. I had a bunch of American friends that have watched it. It's got subtitles and it's just this incredibly again it's like novel like the editing is crazy. They are just extremely creative and is very short.

所以我会推荐这样做。So I would recommend that.

主持人：你最近发现并且很喜欢的产品是什么？Host: Favorite product you recently discovered that you love?

嘉宾：我很喜欢我的 Apple Watch，但很讨厌它要充电，或者说我每天都得给它充电。所以我最近刚买了 Apple Watch Ultra，天哪，真不错。如果你曾经因为这款好产品的续航不尽如人意而感到沮丧，那你看，结果证明，只要把它做大一点，电池续航就更好了。Guest: I love my Apple Watch and I hate that it charges or I have to charge every day. So I bought the Apple Watch Ultra just recently and man that is so nice. If you've ever been frustrated by this good product that somehow we can't figure out how to give a good battery to, well, it turns out if you make it bigger, the battery's better.

所以，我一直对这件事很满意。So, I've been happy with that.

主持人：你有没有一句特别喜欢的人生格言，在工作或生活中常常会想起？Host: Do you have a favorite life motto that you find yourself coming back to often in work or in life?

嘉宾：有一本非虚构类书籍可以说是我最喜欢的，大概就是《四千周》。书里有很多内容我觉得很棒。其中一个特别棒、也和我很相关的观点是：“等我终于……的时候”这种心态很常见，而且有点有害。那么，这是什么意思呢？Guest: There's like one non-fiction book that really is like probably my favorite ever. It's 4,000 weeks. It has a lot in it that I think is cool. One of the cool things is this sort of thing that was really relevant to me, which is that like the when I finally mentality is a common mentality and is kind of toxic. So, so what does that mean?

这有点像是，呃，等我终于把待办清单上的所有事项都勾掉，就终于能放松了。等我终于结了婚，我就会幸福。等我终于升职了，然后这有点……我是说，你刚才在讲冥想，对吧？It's sort of this like ah when I finally, check off this all the items on my to-do list, I'll finally be able to chill. When I finally get like married, I'll be happy. When I finally get a promotion, and then this is sort of I mean, you were talking about meditation, right?

这几乎和冥想完全相反：归根结底，你的人生就是你在当下经历的所有时刻。而且有种说法是，你应该尽量始终活在当下，不要把自己投射到一个看起来更美好的未来里。比如，如果你一直告诉自己“等我终于做到了X，我就会快乐”，那等你做到了X，你也不会快乐。This is like almost the opposite of meditation and that at the end of the day your life is like all of the moments you lived in the moment and that there's this sense in which you should just like try to always like be in the moment and not projecting yourself towards like a future that seems better like probably if you're telling yourself when I finally do X I'll be happy when you do X you will not be happy.

主持人：有没有更好的说法，还是就说“别那么做”？比如他们会说——Host: Is there a better phrase to use or is it just don't do that? Like do they say

嘉宾：我不知道。我觉得对我来说，积极的版本就是活在当下，对吧？就是单纯地欣赏此刻。Guest: I don't know. I think to me the positive version is like be in the moment, right? It's just like just I appreciate the moment.

最近我读到一篇博客文章，它……There was a blog post recently I read that was

嘉宾：就像，享受这份三明治。我觉得……我觉得这就是字面意思。我不知道你有没有看到过这个，有人就说，嗯，就像，你就是在吃三明治，你已经吃过很多三明治了——享受这份三明治吧。Guest: like enjoy the sandwich. I think is I think it was literally the sense. I don't know if you saw this where it was like somebody was like yeah like just you're eating a sandwich, you've eaten many sandwiches like enjoy the sandwich.

你就像是在参加团队会议，对吧？嗯。你一直都在……就像是享受其中、投入其中，而且某种程度上没有去关注接下来的事情。You're in like a team meeting, ? Yeah. You've been in like enjoy like be in it and sort of not looking towards the next thing.

主持人：我以前参加过一次冥想静修。那是一个为期10天的静默静修，连吃饭时也要保持安静，没人说话。你就坐在那里吃东西。Host: I did a meditation retreat once. It was a 10day silent retreat where you eat silently too and no one's talking. You just sit there and eat your food.

我只记得剥一个鸡蛋剥了大概五分钟，注意到每一小块蛋壳，还有它散发出来的气味。你几乎得费点劲才能进入那种状态，但身处其中真的很有意思，一切都变得很有趣。就像，哇，看看这个鸡蛋，而不只是嘴里塞着的那些东西。And I just remember peeling an egg for like five minutes and just noticing every shell piece and just the smell of it all. And it like you almost it's like hard to get into that mode, but it's such an interesting place to be where everything is just interesting. Like wow, look at this egg versus just like stuff in my mouth.

嘉宾：嗯。对。就像你一边用左手刷 Slack，一边听播客的时候。Guest: Yeah. Right. While you're like on Slack with your left hand and like listening to a podcast at the same time.

主持人：哎呀。好吧。最后一个问题。Host: Oh man. Okay. Final question.

你整天都在研究 AI，也在观察 AI 如何思考。如果 AI 能看透你的大脑，什么会让它感到惊讶？You study AI all day. You're watching AI. How it thinks. If an AI were to look inside your brain, what would be surprising?

它会看到什么让人们感到意外的东西？What would it see that might surprise people?

嘉宾：我是说，当然是我最深藏不露的秘密。Guest: I mean, surely my deepest darkest secrets.

主持人：但那些是什么？Host: but what are those?

主持人：我们开始吧。我们开始吧。Host: Let's get into it. Let's get into it.

嘉宾：哎呀，我一直有点爱开玩笑。或者说，这就是我喜欢的生活方式，也是我应对事情的方式。我觉得即使在严肃的场合，也应该尽量轻松一点。Guest: Oh, man. I've always been like a bit of a jokester. Or that's just how I like both how I live and how I cope. I just like think that even in serious situations, you should aim to kind of have some levity.

所以我觉得，如果你能看看我的脑子里在想什么，肯定会很惊讶，因为我基本上随时都在琢磨某件莫名其妙又好笑的事。我会坐在一场严肃的会议里，心想：“哦，对，不过你穿着这么个莫名其妙的东西，不觉得有点儿随机吗？”我就是会冒出这些侵入性想法，只不过我一直把它们憋着，而它们通常都只是些烂笑话。有时候，就像你们今天听到的那样，很遗憾，它们还是会说出口。And so I think if you looked inside my brain, you'd definitely be surprised that like basically at any given moment I'm just like focusing on some random thing that is funny. I'll be in a serious meeting and I'll be like, "Oh yeah, but like isn't it kind of random that you're like wearing this random thing or something?" It's just like I have these intrusive thoughts that I keep out, but that are all usually like just bad jokes. And sometimes, as you've heard today, unfortunately, they come out.

所以，很抱歉发生了那些事，但这大概是最令人意外的事情。So, I'm sorry for those, but that's probably the most surprising thing.

主持人：我很喜欢。Emmanuel，非常感谢你来到这里。最后一个问题：听众怎样才能帮到你？Host: I love that. Emmanuel, thank you so much for being here. Final question. How can listeners be useful to you?

嘉宾：就做点 interp，读一些、读一些关于模型如何工作的内容。我是说，你提到了类似 dorkers thread up 的东西。我觉得那也很棒。Guest: Just do some interp read some read about how the models work. I mean, you mentioned like dorkers thread up. I think it was great as well.

我觉得，这毕竟是个非常新的领域，既包括可解释性，也包括 AI 和 LLM。我认为，多些人参与并了解相关情况会很有帮助。所以，去读些相关内容吧。如果你还想了解这些模型是如何工作的，而我们还没弄清楚的地方，你可以联系我。I think, it's just this is such a new field like both interp and like AI and LLMs and I think it just benefits from people engaging and being informed. So like read things. If there's more that you would want to know about how these models work that we haven't figured out, you can, reach out to me.

我活跃在各种社交平台上。比如，你大概可以通过 X 联系到我。I'm on variety of social platforms. Like probably you can reach out to me on X

主持人：我们会附上一些你们制作的视频链接，里面介绍了我们一直在聊的很多内容，比如 JSpace 之类的。你们团队制作的一些视频真的很精美。我不知道是谁负责制作的，但真的很棒。Host: and we'll link to some of the videos that you guys have put out that describe a lot of the stuff we've been talking about like JSpace and things like that. There's some really beautiful videos your team has made. I don't know who does that, but they're really cool.

嘉宾：是的，这些视频是我们和设计团队合作制作的，所以这些精彩的视频都要归功于他们。Guest: Yeah, this is we partner with our design team. So all credit goes to them for the cool videos.

主持人：太棒了。好吧，再次感谢你，Emmanuel。Host: Amazing. All right. Well, thank you again, Emmanuel.

非常感谢你做了这件事，真的太棒了。Thank you so much for doing this. This was amazing.

嘉宾：谢谢你，Lenny。是啊，这次真的特别愉快。Guest: Thanks, Lenny. Yeah, this was super fun.

主持人：再见，各位。非常感谢大家收听。如果你觉得这期节目对你有帮助，可以在 Apple Podcasts、Spotify 或你常用的播客应用上订阅我们的节目。Host: Bye, everyone. Thank you so much for listening. If you found this valuable, you can subscribe to the show on Apple Podcasts, Spotify, or your favorite podcast app.

此外，也请考虑为我们评分或留下评论，这真的能帮助其他听众发现这档播客。你可以在 lennispodcast.com 收听所有往期节目，或了解更多关于本节目的信息。下期再见。Also, please consider giving us a rating or leaving a review as that really helps other listeners find the podcast. You can find all past episodes or learn more about the show at lennispodcast.com. See you in the next episode.