LangChain CEO 谈推理市场与模型路由：走向商品化，还是持续增值？

简要说明Brief Description

推理市场与模型路由的未来：会沦为大宗商品，还是日益增值？| LangChain 首席执行官The Future of Inference Market & Model Routing: Commodity or Increasing in Value? | LangChain CEO

目录Table of Contents

• 简介
  Intro
• 什么是 LangChain？为什么智能体仍然如此难以构建？
  What Is LangChain and Why Are Agents Still So Hard to Build?
• 为什么更强大的模型迫使 LangChain 删除产品中的部分功能
  Why Better Models Force LangChain to Delete Parts of Its Product
• 为什么中立的智能体测试框架仍然重要
  Why Neutral Agent Harnesses Still Matter
• 为什么 Harrison 极度看好垂直领域智能体
  Why Harrison Is Massively Bullish on Vertical Agents
• 企业真的在构建自己的 AI 模型吗？
  Are Companies Really Building Their Own AI Models?
• 专业模型真的胜过前沿模型吗？
  Do Specialized Models Actually Beat Frontier Models?
• 模型路由的真正价值究竟在哪里？
  Where Does the Value in Model Routing Really Sit?
• 你能相信 AI 评测吗？
  Can You Trust AI Evals?
• 智能体技术栈中的价值究竟沉淀在哪里？
  Where Does the Value Actually Accrue in the Agent Stack?
• 持续学习如何改变智能体
  How Continual Learning Changes Agents
• 为什么智能体记忆层仍然门户大开
  Why the Agent Memory Layer Is Still Wide Open
• 三年后，今天关于智能体的哪些事会变得显而易见？
  What About Agents Today Will Be Obvious in 3 Years?
• 每款消费级应用都会拥有自己的垂直领域智能体吗？
  Will Every Consumer App Have Its Own Vertical Agent?
• 当智能体生成的请求量达到人类的 1,000 倍时，会发生什么？
  What Happens When Agents Generate 1,000x More Requests Than Humans?
• 为什么智能体成本可能会成为一个大问题
  Why Agent Costs Could Become a Huge Problem
• LangChain 将工程团队薪资的约 30% 用于编程智能体
  LangChain Spends ~30% of Engineering Salary on Coding Agents
• AI 智能体会让工程团队规模大幅缩小吗？
  Will AI Agents Make Engineering Teams Dramatically Smaller?
• Anthropic 的估值取决于什么
  What Anthropic’s Valuation Depends On
• 企业是否害怕中国开源模型？
  Are Enterprises Afraid of Chinese Open-Source Models?
• 大多数人对开放模型的误解
  What Most People Get Wrong About Open Models
• 为什么 Harrison 认为推理市场规模巨大
  Why Harrison Thinks the Inference Market Is Massive
• 哈里森如今会把资金投向哪些人工智能领域？
  Where Would Harrison Invest in AI Today?
• 在人工智能领域，先行一步是否比以往任何时候都更加重要？
  Is Being First More Important Than Ever in AI?
• Jev估值达90亿美元的看多逻辑
  The Bull Case for Jev at a $9BN Valuation
• Benchmark 和红杉资本究竟如何帮助创始人
  How Benchmark and Sequoia Actually Help Founders
• 为了顶级风投而接受较低估值，值得吗？
  Is It Worth Taking a Valuation Discount for a Top-Tier VC?
• 人们对智能体最容易产生的误解
  The Most Important Thing People Get Wrong About Agents
• OpenAI估值1.5万亿美元，Anthropic估值2万亿美元
  OpenAI at $1.5TN vs Anthropic at $2TN

简介Intro

我认为我们把工程师薪资的30%花在了编程工具上。实际上，过去两个月这项支出下降了。与此同时，我们使用的 token 数量反而增加了。今天，我们欢迎 LangChain 创始人哈里森·蔡斯（Harrison Chase）做客。他创办的公司价值数十亿美元，在创建和监控智能体方面处于领先地位。今天，我们请他来到访谈席，讨论如今构建智能体为何如此困难；关于智能体的发展，有哪些大家都应该知道、却没人知道的事；最后还有我最喜欢的环节——一些相当古怪的预测，别人都不相信会发生，但他确信一定会发生。I think we spend 30% of our engineering salary on coding tools. It's actually gone down the last 2 months. While tokens, the number of tokens that we've used has actually gone up. Today we welcome Harrison Chase, the founder of Lang Chain, multi-billion dollar company. It is a leader in the ability to create and monitor your agents. And today we welcomed him to the hot seat to discuss why it is so hard to build agents today. What no one knows that everyone should know about what is coming with agents. And then my favorite, some really kooky predictions which no one else believes will happen, but he knows will definitely happen.

主持人：我真的低估了对推理的需求。一切都运行得很顺畅。这个市场太大了，对它的需求实在是太旺盛了。Host: I really underestimated the demand for inference. Everything just works. This market is so big. There's just so much demand for it.

嘉宾：准备好了。哈里森，老兄，我真的特别期待这次聊天。我期待有一阵子了，因为我记得迈尔斯在 Benchmark 的时候跟我提起过你，对你赞不绝口。所以我一直想促成这次聊天。谢谢你来参加。Guest: Ready to go. Harrison. Dude, I'm so looking forward to this. I've been looking forward to it for a while cuz I remember Miles telling me about you when he was at Benchmark and just singing your praises. So, wanted to make it happen for a while. So, thank you for joining me.

嘉宾：我很高兴来到这里。Guest: I'm excited to be here.

主持人：现在，对于那些还不知道的人，60Host: Now, for those that don't know, 60

什么是 LangChain？为什么构建智能体仍然如此困难？What Is LangChain and Why Are Agents Still So Hard to Build?

第二，什么是 LangChain？你认为它在技术栈中处于什么位置？seconds, what is Lang Chain and where do you like sit in the stack?

嘉宾：我们开发工具，让构建智能体变得更容易。因此，我们主要开发面向开发者的工具。我们有一个开源框架，基本上可以将模型连接到构建实用智能体所需的一切，比如工具、数据、计算能力等等。此外，我们还有 Langmith。Langmith 提供可观测性和评估功能。智能体经常会出错。你怎么知道它们是怎么出错的？又该如何衡量它们，才能随着时间推移提升其性能？这就是 Langmith 的作用。Guest: We build tools to make it easy to build agents. So, we build developer tools primarily. We have an open source framework which basically connects the model to everything else that you need to build a useful agent. So tools, data, computation, things like that. And then we also have Langmith. Langmith is observability and evals. Agents go wrong all the time. How do you know how they go wrong? And how do you measure them so that you can increase their performance over time? That's what Langmith does.

主持人：好吧。我靠问蠢问题谋生。嗯，这就是我做播客的原因。构建智能体为什么这么难？Host: Okay. I ask stupid questions for a living. Um, that's why I'm a podcaster. Why is it difficult to build agents?

嘉宾：构建智能体最难的一点是，它们的行为往往不如你所愿。模型就像一个个非确定性的黑箱，对吧？它们接收自然语言输入，然后输出自然语言。而且在这个黑箱内部，它们也是非确定性的。这意味着输入的维度几乎是无限的，什么都有可能。你在某人面前放一个聊天机器人，对方想输入什么都行，对吧？所以，你很难知道他们会输入什么。接着，这些内容进入模型——顺便说一句，智能体并不只是调用一次模型，而是一系列模型调用，还会与其他工具交互。于是，你就把这些小型黑箱放进一个大型系统中，最终出现各种层层累积的错误。Guest: The hard part about building agents is uh they don't behave how you want them to behave. So, so the models are these like non-deterministic black boxes, right? And they take natural language as input and then they output natural language. And inside the black box, they're also nondeterministic. And so what that means is that the input you get is like infinite in dimension. It can be anything. You put a chatbot in front of someone, they can type anything that they want, right? And so that makes it really hard to know what they're going to type. And then once it goes into this model, and by the way, agents are not just one model call. There's a they're a series of model calls and they're interacting with other tools. And so you have these small block black boxes that are then put inside this large system and you get basically these cascading types of errors.

所以你有一个非常非常大的、可以说是输入空间。你的输出空间也非常大，因为两个相似的输入进去之后，可能会得到完全不同的答案。而让这一切变得困难的，正是 LLM 本身。没错，它们非常非常聪明，但你无法确切知道它们会做什么，而且输入稍有变化，它们给出的答案就会改变。所以你构建了这个系统，但它实际上并不可靠。让这个系统能够可靠地运行，这才是最难的部分。比如说，和三四年前相比，如今的智能体已经进步了很多。主要发生了两件事：第一，模型变得更好了；第二，我们摸索出了如何构建系统，更好地利用这些模型。So you have this this this very big kind of like input space. You have this very big output space because two similar inputs can go in and they can arrive at completely different kind of like uh answers. And the thing that's uh making all this hard is the LLM itself. Yes, they're very very smart, but you don't know exactly what they're going to do. and they change answers based on small changes to the input. And so you build this system, but it's really non-re. And so getting this system to behave reliably, like that is the hardest part. Like when we talk about agents 3 years ago, four years ago to agents today, they've gotten a lot better. And the main thing that's happened is one, the models have gotten better, and two, we've figured out how to build systems that basically harness these these models better.

而我们关注的，是第二部分，也就是如何利用这些模型。And and that second part, the the harnessing of these models is what we focus on.

为什么更好的模型迫使 LangChain 砍掉部分产品功能Why Better Models Force LangChain to Delete Parts of Its Product

主持人：随着模型不断改进，你们的业务会发生哪些人们看不到的变化？Host: How does your business change in a way that people don't see as the models get better?

嘉宾：随着模型变得越来越好，我们的业务发生变化的主要方式，就是我们基本上得把产品的大块内容删掉。嗯，我的意思是，Cursor 也把这一点说得特别好。我记得这甚至是他们公司的一项实际运营原则，好像叫“抹掉产品”或“删除产品”。基本上，模型不断进步，六个月前你还必须做的事情，现在已经不必做了。这一点和我们的开源项目特别相关。开源说到底，就是提供一套工具链和框架，让这些模型能够完成各种任务。而使用这些模型、帮助它们完成任务的正确方式，每个月、三个月、六个月都会变——你可以选一个时间跨度，但变化确实非常快。Guest: The main way our business changes as the model gets better is we basically have to erase big parts of our product. Um so what I mean by that and cursor said this really well as well. I think this was one of their actual operating principles as a company is like erase the product or delete the product I think is what they called it where basically like these models get better and things that you had to do um 6 months ago you no longer have to do and so this is very relevant in our open source. So open source is all about basically the the harness and the framework for taking these models and helping them do things. And the right way to take these models and help them do things changes every month, 3 months, 6 months, choose your time period, but it changes pretty rapidly.

主持人：面对如此不确定的未来，你们该如何进行开发？是不是只能和那些实验室打成一片，才能了解他们的路线图，避免花六个月开发出某样东西，结果他们随后就把它淘汰了？Host: How do you build for such an uncertain future? Do you just have to get in bed with the labs to know the road map so you don't waste 6 months building something that then they erase?

嘉宾：一方面，这确实没有什么简单的答案。那你打算怎么办？什么都不做吗？我常对问我这个问题的创始人说的就是这个：我该怎么办？情况一直在变化，你知道，模型也在变化。如果我现在做点什么，可能六个月后就不再适用了。那替代方案是什么？难道你就什么都不做吗？所以我觉得，你必须做出一些有根据的判断，看看哪些部分未来还会存在，哪些不会。比如说，早期我们在 LangChain 里实现工具调用的方式，是让模型生成 JSON DSL，然后再解析它。我想我们当时都觉得，或者说都知道，这多少有点取巧，但我们不得不这么做，因为这是唯一能让它真正控制流程的办法——你得让它能够进行这种工具调用。Guest: There there's no easy answer on on one hand. What are you going to do? Not build. Like that's that's what I like to say to founders who ask this like what do I what do I do like stuff is changing um you know the models are changing uh uh if I build something now it might not be relevant in 6 months what's the alternative you just don't build anything and so I think you have to make kind of like educated guesses as to what parts are going to be around and what parts aren't going to be around in the future so for example early on the way that we would do tool calling in lang chain was by having the model generate JSON uh DSL and then parsing it and Uh that was something that I think we all thought or knew was like a little bit of a hack at at at the time, but we had to do it because that was the only way to get it to to actually steer things cuz you have to get it to do this type of tool calling.

然后又过了几个月，OpenAI 推出了工具调用。我想我们很早就意识到，这是一种更原生的工具调用方式，它直接集成在模型和 API 中。我想我们很早就意识到，这就是未来。因此我们投入了大量精力，彻底从 JSON 解析转向了这种工具调用。嗯，所以我觉得，你基本上必须分辨哪些东西只是短期权宜之计，哪些东西是长期存在、可以说会一直存在的。模型永远不会知道你的专有数据。它们永远无法真正获取 API 的输出。你必须拿到它们发出的 API 调用，把它传给 API，获取输出，然后继续这个循环。因此我觉得，围绕模型、将它连接到其他地方的系统会一直存在。And then and then a few months later, OpenAI came out with tool calling. And I think we recognized pretty early and this is like a more native version of tool calling that's directly in in in the model and in the APIs. And I think we recognized pretty early on that that was the future. And so we invested a lot of efforts and just completely switched from from JSON parsing to this tool calling. Um and and so I think you have to basically recognize what things are short-term hacks and what things are long-term kind of like will always be there. The models will never know your proprietary data. They will never be able to to kind of like actually get the output of of APIs. You have to take their like API call, pass it to the API, get the output and then continue in the loop. And so I think the system around the model connecting it to other places will always be there.

不过，我们围绕思维链、规划或工具调用所做的很多变通方法——那些其实可以更紧密地集成到模型或模型 API 中的东西——都会消失。我觉得你只要迅速意识到这一点，然后继续前进。But a lot of the hacks we do around uh chain of thought or planning or tool calling things that can just be much more closely integrated into the model or into the model API will go away. And I think you just have to be quick to recognize that and and go

为什么中立的智能体运行框架仍然重要Why Neutral Agent Harnesses Still Matter

开启。on.

主持人：为什么我们需要中立的运行框架？我们看到，你们显然能够从模型本身打造出非常出色的智能体。为什么这一层不会被它们直接吞掉？我认为，我们需要中立的运行框架，是因为在构建智能体之前，或在智能体运行过程中，会积累大量状态。这些状态可能包括记忆，比如短期记忆或长期记忆。记忆也有不同的形式，可能来自与智能体的交互，也可能来自智能体向环境学习。之所以会积累这些状态，是因为智能体会进行交互：它会走出去、学习、聊天，并从中学到东西，让自己变得更好。我认为目前最有意思的问题之一，就是如何让这些智能体随着时间推移表现得更好、不断学习。Host: Why do we need neutral harnesses? We see you know bluntly very brilliant agent builders from the models themselves. Why does your layer not just get eaten by them? I think the reason we need neutral harnesses is that there is a lot of state that you build up either before you build the agent or as part of the agent running. And the state can include things like memory uh whether that's short-term memory or long-term memory. Memory also comes in different form factors. It can be from interacting with the agent versus from the agent learning from the environment. And the reason that this state gets built up is the agent interacts. It goes out, it learns, it chats and and it learns things to make it better. I think I think one of the most interesting things right now is how to make these agents do better over time and learn.

其中一些学习成果，以及随着时间推移不断改进的部分，会体现在模型权重中，但很多都在模型之外，也有很多体现在这个——这个上下文里。现在，当你构建这个上下文时，真的不希望它绑定到某一家模型提供商。嗯，原因是这样一来，你就会被绑定在那家提供商那里，无法改用其他模型。能够更换模型，既有进攻性，也有防御性。进攻性在于，无论最好的模型来自哪家，只要它一推出，你就希望能够利用它。防御性在于，你不希望被锁定，以至于他们可以提高费率之类的。所以你希望能够切换模型。这个运行框架可以与模型紧密绑定，也可以保持中立。And some of that learning and some of the doing better over time will be in the model weights, but a lot of it is outside the model. And a lot of it is in this in this context. And right now when you build up this context, you really don't want it to be linked to a single kind of like model provider. Um, and the reason for that is that then it becomes linked there. You can't go to other models. Being able to change models is very offensive and defensive. So it's offensive in the fact that you want to be able to take advantage of the best model when it comes out no matter who it's coming from. It's defensive in the fact that you don't want to get locked in so they can't raise rates, things like that. So you want to be able to switch models. The harness is uh can be very tied to the model. It can also be neutral.

目前，上下文与运行框架紧密耦合。运行框架的职责基本上就是接入模型，并与其他所有东西交互。这些东西包括数据和 API，也包括记忆。因此，如果你构建的记忆与运行框架绑定，而运行框架又与模型绑定，那你就有点被困住了。你就有点……The context right now is really intricately linked to the harness. So the harness's job is to basically take the model and interact with all the other things. All the other things includes data and APIs, but also includes memory. And so if you build this memory in a way that is linked to the harness and the harness is linked to the model, then you're kind of stuck. You're kind of

嘉宾：抱歉，harness 结构确实如此，它允许用户实现数据可移植性。Guest: true of the harness structure, I'm sorry, is the portability of data that it allows the user.

主持人：是的，是的。你基本上不想被某些模型锁定。如果记忆、上下文开始以某种特定于某个模型处理记忆或上下文的方式不断积累，那就会严重影响可移植性，进而让你在未来发展业务时处于劣势。我们和很多人聊过如何掌握自己的智能，以及这意味着什么；对此有很多不同的理解。我们认为，掌握自己的智能包含几个不同的部分，其中一个关键部分是拥有这样一种开放且可控的智能体——你要能够调整所有不同的组成部分，也要能够调整模型。Host: Yeah. Yeah. You you want to you want to basically not be locked in to models. And if if if memory if context starts building up in a way that is specific to one how one model wants to interact with memory or wants to interact with context, that's very bad for portability. and then that puts you at a disadvantage building your your business going forward. We talk with a lot of people about owning their intelligence and what that means and there's a lot of different versions of that. The version that that that we think like owning your intelligence means we we think there's a few different components of that but one key component is having kind of like this this open and controllable agent like you want to be able to adjust all the different pieces. You want to be able to adjust the model.

你也希望能够调整运行框架，因为有时候这些模型实验室会把运行框架训练得非常擅长处理它们训练分布内的任务，而这些任务通常是编程。编程当然很棒，围绕编程也有大量用例，但还有很多用例并不是编程。也许它们会把编程作为其中一环，但实际上，你是在用它来做销售、做研究、处理法律事务，或是处理其他垂直领域的特定任务。因此，当你遇到模型实验室训练范围之外的任务时，你会希望能够控制运行框架的行为。你还会希望能够控制上下文。所以，模型、运行框架本身和上下文，共同构成了一种开放且可控的智能体架构。You want to be able to adjust the harness as well because sometimes the these these model labs are training harnesses to be really good at the things that are in distribution for them which oftent times is coding and coding is great and there's a ton of use cases around coding but there are a ton of use cases that are aren't coding and maybe they'll write code as part of it but they'll you they're using that as as a way to do sales as a way to do research as a way to do these the law as a way to do these other vertical specific things. And so when you get out of distribution of what the the model labs are training on you want to have control over like what the what the harness does. And then you also want control over the context as well. So model, the harness itself, and the context make up kind of like this open and controllable uh agent architecture.

我们认为，这就是拥有一个智能体意味着什么的重要组成部分。And we think that's a big part of what it means to own one intelligence.

为什么 Harrison 极度看好垂直领域智能体Why Harrison Is Massively Bullish on Vertical Agents

嘉宾：我想稍微回过头来聊聊我们所说的智能体价值。有些人认为，真正的价值在于验证——也就是判断它做得好不好，因为这能开启下一步：自动化。如果价值是通过验证体现出来的，那谁会胜出？智能体框架。Guest: I just want to back up a little bit when we think about like um agent value so to speak. Some people say that the value is actually in the verification like whether it did well or not because that unlocks the next step which is automation. If value is derived through verification, who wins? The harness.

主持人：如果价值是通过验证得来的，我认为最终获胜的是领域专家。也就是那些能够量化这种价值意味着什么的人，而这非常难。比如，构建评测就是一项非常困难的工作。所以我认为，很大一部分价值会归于那些能够编写验证器、捕捉领域知识的人。出于这个原因，我非常看好面向不同领域的垂直智能体，因为我认为，最适合编写这些验证器或构建我们所说的那些评测的人，正是领域专家。他们能够构建 harness 和模型，而且最终起作用的不会只是 harness，也不会只是模型，而是两者的某种组合。所以，给那些正在着手构建垂直智能体的人一个非常实际的建议：我强烈建议你们从 harness 层入手，做好上下文工程，调整 harness；等你们有了用户之后，再开始在后训练阶段调整模型。Host: If value is derived through verification, I think the people who win are the domain experts. The people who are able to kind of like quantify what that value means, which is very hard. Like building evals is a very hard job. And so I think the the the a lot of the value will acrue to whoever can kind of like uh write those verifiers, capture that domain knowledge. I'm I'm massively bullish on vertical agents for different domains for for this reason because I think the domain experts who who are best suited to to write those verifiers or build those eval as as we kind of talk about it um they will be able to build harnesses and models and it it won't be just the harness it won't be just the model it will be some combination of them. So, so very practically if you're getting started building a a vertical agent, I would highly suggest you start at the harness layer, you do context engineering, uh you you you kind of like tweak the harness and then when you've got users, then you start tweaking the model in post- training.

我认为，我认为，先从上下文层或 harness 层入手，ROI 会高得多。所以我觉得我们会有这些不同的垂直领域。他们能够以非常具体的领域化方式，量化对他们来说什么才算好，这样他们就能打造的不只是垂直领域智能体，还有垂直领域智能体体验。不同的智能体可能会有不同的体验。所以我觉得很多价值都会来自这里——我非常、非常看好垂直领域智能体，因此我们希望能让这类智能体大量涌现。I think I think it's it's much higher ROI to start at the context or harness layer first. And so I think we'll have all these different verticals. they'll be able to quantify what good looks to them um in in in a very domain specific way and that'll let them build uh not only like vertical agents but also vertical agent experiences. Uh different agents might have different experiences and so I think like a lot of the value will will I I massively bullish on vertical agents and so we want to enable a lot of that.

公司真的在构建自己的 AI 模型吗？Are Companies Really Building Their Own AI Models?

你刚才说，如果由你来构建那个东西，你会从 harness 层开始。我们看到很多公司——不管是 Harvey 还是我想 RAMP——都在构建自己的模型。它们实际上在多大程度上是在构建自己的模型？Mike Cannon-Brookes 在我们今天发布的节目里说，这种说法在很大程度上，恕我直言，是胡扯；它们其实是在微调中国的开源模型。这是真的吗？你怎么看？You said like if I was building that I would start at the harness layer. We're seeing a lot whether it's Harvey or I think it's RAMP um build their own models. To what extent are they actually building their own models? Mike cannon Brooks said the last on show we released today that it's largely kind of respectfully BS and they're fine-tuning open source Chinese models. Is that true? And and how do you think about that?

嘉宾：我认为其中很大一部分是，他们在对开放权重模型进行后训练。没错。嗯，我仍然认为这件事很有价值。我不觉得这有什么问题。我认为，掌控自己的智能有不同的层次。最便宜的方式就是只控制上下文，使用现成的 harness 和现成的模型。再往下一层，你可以开始调整 harness，这样你就比只控制上下文拥有更多控制权，但这样做会稍微贵一些。再下一层，就是对开放权重模型进行后训练。比如，你可以拿一个现有的开放权重中文模型，针对你的领域对它进行后训练。最底层则是从头开始训练一个模型。Guest: I think a lot of it is they are posttraining open openw weight models. Absolutely. Um, I I I still think that's a valuable thing to do. I don't think there's I don't think there's kind of like anything wrong with that. I think I think there's different like dimensions that you can go in terms of owning your intelligence. I think the cheapest way is you just control the context. You use an off-the-shelf harness, off-the-shelf model. The next level down is you start tweaking the harness and and that gives you more control than just the context, but it's a it's a little bit uh uh more expensive to do. The next level down is is you post-train an openw weight model. And so you might take an an existing uh openweight Chinese model and post- train it for your domain. And then and then the very kind of like bottom layer is they actually train a model from scratch.

我不认为，我不认为有很多人是在从头训练某种模型。我觉得大多数人做的都是后训练。我觉得如果你在从头训练模型，那你很可能是在 Neolab。我觉得他们是少数这么做的团队之一。嗯，不过是的，Ramp 和 Harvey 这样的公司人才济济，数据也非常丰富，还有大量使用量，并且正努力思考如何真正对其进行优化。所以他们已经充分挖掘了上下文的价值，也充分挖掘了 harness 的价值，现在他们基本上已经开始进行……的后训练。I don't think I don't think many folks are training kind of like a model from scratch. I think most are post- training. I think if you're training a model from scratch, you're probably at Neolab. I think those are one of the few that are doing that. Um but yeah, the ramps and the harveies of the world, they uh they have a ton of talent. They have a ton of data. They have uh a bunch of usage and are trying to think about really optimizing it. And so they've gotten all the juice they can out of the context, all the juice they can out of the harness, and now they've arrived at basically post- training of

专业模型真的胜过前沿模型吗？Do Specialized Models Actually Beat Frontier Models?

这些模型。these models.

主持人：我能问一下吗？关于这些专用模型在后训练领域的效果，目前还没有定论。就像我们刚才提到的，我看到一些报告称它们效率高得多、效果也好得多，但也有一些报告没那么乐观。你觉得最终会是怎样？我认为，我们目前看到的是，现阶段很多实际可行的后训练，大概都是针对一些比较简单的任务，而这些任务会构成一个更大的工作框架。我的意思是，当你搭建一个工作框架，或者说一个能执行有趣任务的智能体框架时，实际上很少会只是一个模型在循环运行。它可能是核心部分，但还会有大量其他模型调用，用来充当防护措施、负责路由，或作为子智能体、子工作流之类的组件。Host: Can I ask you, the jury's kind of out on the efficacy of these specialized models in the post-training realm, like we mentioned there, some reports I've seen say they're massively more efficient and better and some less so. How do you think this will play out? I think what we've seen in regards to post- training is that a lot of the post- training that practically makes sense right now is probably for like simpler tasks that make up a larger harness. So what I mean by that is like when you have a harness when you have an agent harness that's doing interesting work, it it it's actually very rarely just one model running the loop. That might be the core of it, but there's a bunch of other model calls that happen either as like guard rails or routing or sub aents or subworkflows or things like that.

因此，我们现在看到后训练效率较高的很多场景，都是一些比较简单的任务，比如分类、提取或总结。这些任务可能由智能体作为工具使用，也可能用作护栏之类的功能。我认为，现阶段这类后训练非常有意义，而且今后也会继续发挥作用。更没有定论的问题是，驱动这个循环的核心模型会怎样运作。因为智能体的主要工作方式，是让一个 LLM 在循环中运行并调用工具。那么，驱动这个循环的模型会是经过后训练的模型，还是前沿模型？我认为，这方面的情况更像是来回变化：每当有新的前沿模型推出时，它们运行这种核心循环的能力就会更强。我们确实看到 Harvey、Ramp 等公司在尝试对用于这个核心循环的模型进行后训练。And so a lot of the places where we see posttraining being uh efficient now are for simpler tasks like classification or extraction or summarization that might be a tool the agent uses or a guardrail or things like that. I I I think those types of post training right now make a ton of sense and will continue going forward. I think the question uh that is more unanswered is for this core model that's driving the loop because the main way an agent works is there's an LLM running in a loop calling tools and so the question is for for this model that's driving that loop will those be kind of like post-trained models or or will those be kind of like frontier models and I think this is where we see uh more of like a back and forth where like um every time a new frontier model comes out the the the better they are at just running this kind of like core loop we do see people like Harvey and Ramp and others kind of like experimenting with post-training models for this core loop.

所以我想，Harvey 最近大概在过去一个月左右宣布了一个完全符合这一思路的东西。嗯，我……我觉得这仍是个悬而未决的问题。我觉得，嗯……And so I think Harvey recently in the past month or so announced something exactly in that vein. Um I I I think that's an open question. I think um

主持人：如果你要打造自己的垂直领域应用，你会自己做后训练吗？Host: if you were building your own vertical application, would you post train your own or not?

嘉宾：一开始不会。绝对不会一开始就做，不过如果我们说的是五年后，假设我的使用量非常大，嗯，我觉得这确实有可能实现。其中一件事是……Guest: Not to start. Absolutely not to start, but like if we're talking like 5 years in the future, assuming that I'm getting a ton of usage, um I think it could be a realistic possibility. One of the things

模型路由的真正价值在哪里？Where Does the Value in Model Routing Really Sit?

我想，这些 harness 也需要做的一件事，就是根据请求进行模型路由。模型路由层有多大价值？模型路由真的很有意思。我觉得这是当下最热门、也可能最具争议的话题之一。嗯，有几点想法。首先，有个问题是，模型路由究竟应该放在哪里？应该放在 harness 层，还是网关层？这就稍微涉及一些技术层面了。that harnesses also need to do I imagine is model routing depending on request. How much value is in the model routing layer? Model routing is so interesting. I think it's one of the hottest and maybe most divisive topics right now. Um so so a few thoughts. First there there's there's a question of where the model routing should even be. Should it be at the harness level or should it be in the gateway level? And this is this is like getting a little bit into the technical

主持人：我们……这里有区别。抱歉。区别在于，harness 层面的路由属于它运行方式的代码，而在网关中，它只是一次 API 调用。把它放在网关层的好处是，因为它只是一次 API 调用，所以可以把它接入任何 harness。你可以把网关层正在进行的这种路由接入 Claude Code，或者任何 harness，而不必改动代码。缺点在于——这也是我认为我们更看好在 harness 中进行路由的原因——在 harness 里，你对正在发生的事情了解得更多，也更清楚当前状态。我认为 Factory 和 Cognition 也写过相关内容。使用 API 时，通常默认它类似于 OpenAI 的 Chat Completions API，因此它只能接收一定量的状态信息。Host: we there's a difference. I'm sorry. The difference is in the harness level it's in kind of like the code of of of how it runs and in the gateway it's just an API call. So the benefits of putting it in the gateway level because it's just an API call you can drop it into any harness. You could put this this this routing that's happening at the gateway level into clawed code into into any harness without having to change the code. The downside and why I think we're more bullish on routing happening in the harness as are I I think factory and cognition have also written things about this is that in the harness you just have more knowledge of what's going on. You have more awareness of the state. When you have an API it's generally assuming that it's kind of like an open AI uh chat completions API and so it only takes in a certain amount of state.

你可以在 harness 中掌握更多状态，从而知道提示缓存的情况。如果你想让提示缓存保持温热并可用，也可以更轻松地更改提示。因此，很多这类模型都有不同的提示策略。所以，如果你在 harness 层进行路由，就能更轻松地更改提示。我认为我们看到的很多讨论，首先就是围绕网关和 harness 的争论，以及路由应该放在哪里。我想我们大概认为它应该属于……You have more state in in the harness to know hey uh like what's going on with the prompt cache. If you want to try to keep the prompt cache uh warm and and and available, you can also change the prompt more easily. So, a lot of these models have uh different prompting strategies. And so, if you do the routing at the at the uh harness layer, you can more easily change the the prompt. And so, I think a lot of what we've seen is one this debate between kind of like gateway and harness and where does routing go. I think we probably believe it belongs in the um

嘉宾：为了让我们了解开放式路由器，以及其他还有好几家的服务提供商。Guest: and just so we understand the open routers and all the other providers which there are several

主持人：有，他们在网关里。Host: there are they are in the gateway

嘉宾：他们在网关领域做了很多，我显然……OpenRouter 是一家了不起的公司，使用量也惊人。Guest: they are in the gateway a lot of I obviously open router incredible company incredible usage

主持人：曾邀请他们上节目。爱你的，亚历克斯。Host: had them on the show love Alex

嘉宾：Alex 很棒。我觉得，呃，我觉得人们使用 OpenRouter、Vercel 和其他网关，很大程度上是为了访问模型。所以我觉得路由器和网关之间也有区别。我可能会说，OpenRouter 的大部分使用量——当然，Alex 显然比我更清楚具体数字——但如果让我猜，我会觉得其中很大一部分只是因为它是个网关，而不是路由器。人们会预先指定自己想用的模型，而 OpenRouter 只需提供一个 API 和一个 API 密钥，就能访问该模型。至于智能路由之类的功能，我觉得实际使用它的人远没有大家想象的那么多。我觉得，当有人用到它时，通常是像 Factory 这样的团队把它集成到了工具框架里。我觉得 Factory 在这方面做得非常出色。Guest: Alex is great I think uh I think a lot of why people use open router and versell and other gateways is largely for the model access so I think there's also a difference between a router and a gateway and So I would I would probably say most of open routers usage and Alex obviously knows the numbers a lot better than I do but if I had to guess I would guess a lot of it just comes from being a gateway not a router. So people specify the the model they want up front and open router just provides a single API with a single API key for accessing that the ter in terms of like smart routing I think that's actually being done like way less than people think. I think it and I think when it's done it's it's built into the harnesses by folks like factory. I think factory does a tremendous job about this.

他们写过不少技术博客，在路由方面做得非常出色。不过我认为现在，我相当坚定地认为，路由必须了解 harness。你得知道任务是什么，也得知道 harness 是什么样的。所以我不认为有一种路由方案能同时适用于法律、金融和编程，而且在这些领域都完全一样。它会非常依赖具体领域，也非常依赖具体 harness。不好意思。我听你说的时候，觉得考虑到 harness 中存储的上下文和记忆，路由应该放在 harness 层，这简直显而易见、毫无疑问。每当我有这种显而易见的反应时，我就想弄明白：是不是我很蠢？支持把它放在网关层的最有力理由是什么？They've written a number of technical blogs that are very well done in terms of how they do they do uh routing. But I think right now I I I pretty strongly believe that like routing you have to be harness aware. You have to know what the task is. You have to know what the harness looks like. So I don't think you'll have routing that works for law and works for finance and works for coding and have it all be the same. It's going to be very domain and very harness specific. I'm sorry. When I listen to you, it seems so unwaveringly obvious that routing should be at the harness layer given the context and memory that's is stored there. When I have such an obvious response, I like to understand why am I stupid? What is the bull case for it being at the gateway layer?

主持人：我觉得人们谈到网关层时，可能指的是两件事。第一，这会让迁移容易得多。你可以直接用 Claude Code。大家都喜欢 Claude Code，对吧？如果你告诉大家：“嘿，你不能用 Claude Code，得用另一个智能体框架。”嗯，市面上确实还有其他不错的框架，但我们姑且假设你很喜欢 Claude Code，也想让大家继续用它，同时又想做某种路由，比如路由到 DeepSeek 或某个开源模型之类的，那你唯一能做到这一点的地方就是网关层，因为你无法控制智能体框架。顺带一提，这也是支持开源智能体框架的另一个理由：你可以控制并修改它们。Host: I think when people talk about it at the gateway layer, I think there's maybe like two things that they're talking about. So, one, it makes it way easier to port. So, you don't you can you can take cloud code. People love cloud code, right? If you if you if you tell people, hey, you can't use cloud code, you have to use this other harness. Um, and there are other good harnesses out there, but let's pretend for a second that you love cloud code that and you want you want to let them keep on using that, but you want to do some semblance of of kind of like routing to maybe like a DeepSk or or an open source model or something like that, then the only way you can do that is at the gateway layer because you don't control the harness. Side note, this is another argument for open source harnesses that you can control and modify.

嗯，然后第二件事是，人们谈到在网关层进行路由时，我觉得他们往往会默认存在某个领域。大致来说，你可以认为有两个领域适合构建通用路由器。我其实觉得 OpenRouter 在某种程度上已经做到了这一点。你可以说是编程。编程是最热门的领域，所有使用都集中在这里。我们来构建一个特别擅长编程的路由器吧。也许它不会感知 harness，但会感知使用场景。这样我们就知道不同模型各自擅长编程中的不同任务，可以在网关层进行路由。另一种思路是：我们有一个通用聊天机器人，不打算在法律或金融领域做到顶尖，只想做一个面向整个公司的通用聊天机器人。我们可以足够准确地判断，如果你说“嗨，你好吗？”……Um, and then the second thing is when when people talk about kind of like routing at the gateway layer, I think they're oftentimes implicitly assuming that there's a domain. So there's kind of like two general domains that you could maybe argue that you can build general routers for. And I actually think open router has uh has done this to some extent. You could argue coding. Coding is like the the most popular domain. That's where all the uses going. Let's build a router that works really good for coding. Maybe we won't be like harness aware, but we'll be like use case aware. So we know different models are good for different things in coding and we can put it at the gateway. The other thing you could say is like hey we've got this like general chatbot like we're not trying to be best in the class at law or best in the class at finance but we're just going to be like a general chatbot across the company and like we can know well enough that if you say hi how are you doing?

为此你不需要用 Opus，可以用便宜的模型。所以我觉得，人们谈到在网关层进行路由时，其实往往默认你做的是通用聊天或编程，而不是某种特定领域的事情。一旦涉及特定领域，我真的认为路由就必须放在 harness 层。你不会用开放式的、现成的网关路由器来处理这种情况。那么，你觉得两到三年后，大多数路由都会在 harness 层吗？毕竟上下文带来的好处很明显；但对于非常通用的任务，路由还是会放在网关层。You don't need like opus for that. You can use a cheap model. And so I think when people talk about routing at the at the gateway level, they're implicitly kind of assuming that you're doing either like general purpose chat or coding um and and you're not doing like a a domain specific thing. As soon as you get into a domain specific thing, I really think it has to be at the harness layer. You're not going to use an open you're not going to use an off-the-shelf uh gateway router for that. So in 2 to 3 years time you think the majority is within the harness given the benefits that come from that context but for very horizontal task it will sit in gateway.

嘉宾：我认为完全正确。如果你在构建垂直领域的智能体，我觉得路由应该放在执行框架中。如果你考虑的是通用的横向任务，比如普通聊天或编程，那么路由或许可以放在网关中。Guest: I think that's exactly right. If you're building a vertical agent I think routing needs to be in the harness. If you're thinking about a general purpose horizontal task like just general chat or coding it it could maybe be in the gateway.

主持人：这是种商品吗？大家是不是都直接自己搭建？Host: Is it a commodity? Like does everyone just build it?

嘉宾：我不认为路由是一种同质化商品。我、我、我确实认为人们会去构建它，但我觉得即使在今天，构建路由也相当困难。而且我认为，部分原因在于模型变化得实在太快了。比如说，你可以认为某些模型在不同方面更出色，但随后就会推出一个新的 Opus 或新的 OpenAI 模型，而且它在所有方面都更好，于是路由的价值就会有所下降。Guest: I don't think routing is a commodity. I I I do think people will build it but I think building it is quite hard uh even today and and I think part of the reason for that is the models just change so rapidly. So like you can argue that certain models are better at different things but then a new opus or a new open AI model comes out and it's just better at everything and so then the value of routing kind of goes down.

主持人：你相信评测吗？Host: Do you trust evals?

你能相信 AI 评测吗？Can You Trust AI Evals?

嘉宾：我觉得评测很有用，但我不会完全信任它们。我的意思是，我觉得现在很多模型发布时使用的学术评测，已经没那么有用了。我觉得现在出来的模型基本都针对这些评测把基准测试刷到了极致，所以你学不到太多东西。我觉得你自己做的、针对特定领域的内部评测，绝对还是有用的。所以我想，这也是为什么构建评测是打造垂直领域智能体如此重要的一部分。我觉得你必须有办法衡量表现，而且我认为通常可以信任这些评测。不过，即便如此，最重要的事情之一还是看看数据。就是看看数据，了解它到底在做什么。Guest: I think evals are useful but I wouldn't trust them fully. And so what I mean by that is I I I think I think a lot of the academic evals that are out there that are used when a lot of the models come out are not really that useful anymore. I don't I think all the models that come out are kind of benchmark maxed against those evals and so you don't learn a lot. I think internal evals that you have and that you build for your domain absolutely can still be useful. And so I would I mean this is the main reason that building evals is such an important part of building a vertical specific agent. I think you have to have some way of of measuring things and and I think you can generally trust th those evals. Um but even then one of the most important things to do is just look at the data. Just like look at the data and see what it's doing.

不要过于依赖某种单一数字。我觉得这就是为什么你需要有追踪记录。你想看看实际发生了什么。也许你的评测并不完美。评测真的很难。你可以有个大致判断，但也许模型的表现确实更好了，只是方式不同，而你之前不知道该如何捕捉这种变化。所以我确实认为，嗯，是的，你可以相信垂直领域的专用基准，但你也必须不断改进它们。这正是我们看到的主要情况之一，因为这些模型只是在以不同的方式变得更好。我能问你吗？你……Don't rely too much on kind of like a single number. I think this is why you want to have traces. You want to look at what's going on. Maybe your evals aren't perfect. Evals are really hard. You can have some sense but maybe models are doing better but in just a different way that you didn't know how to capture before. And so I do think uh yes I think you can trust vertical specific benchmarks but you have to evolve those as well and that's one of the main things that we've seen because these models just get better in different ways. Can I ask you? You

智能体技术栈中的价值究竟在哪一层积累？Where Does the Value Actually Accrue in the Agent Stack?

那里提到了 traces。如果 traces 就像 agent 工作产生的废气，那么价值在哪里？是在应用层、可观测性层，还是模型层？究竟在哪里？mentioned traces there. If like traces are the like exhaust of agent work, where is the value then? Is it the app, the observability layer, the model layer? Where is it?

主持人：如果你在构建一个智能体，我认为它的价值在于，是什么让这个智能体擅长你所从事的领域或垂直行业。我的意思是，我觉得控制智能体行为有不同的方式：有上下文、执行框架，还有模型。在其中任何一方面拥有特定的专有内容，都可能成为优势。我认为，开始时最容易、也最显而易见的做法是控制上下文，然后再控制执行框架。这里包括数据、记忆、特定的 API，以及可以使用的特定工具。控制执行框架就稍微难一点。Host: If you're building an agent, I think the value is at what makes that agent good at the domain or vertical that you're you're working in. So what I what I mean by that is again I think there's different ways to um to to control what an agent does. There's the context, there's the harness, and there's the model. Having uh specific proprietary things for for any of those can be an advantage. I think controlling the the context is is the easiest and and the most uh obvious thing to do when starting then controlling the harness. And that would be data, that would be memory, um that would be specific APIs, um specific tools that you can use. Controlling the harness is like a little bit harder.

不过，你仍然可以把大量领域知识融入其中，比如哪些事情应该由路由来处理，以及其他相关内容。模型是最难控制的部分，但潜力上限也最高。让你能够在这些方面不断迭代的，是评估。因此，这些东西都很有价值，但前提是你能从中提炼出洞见，再把它们带回上下文、执行框架或模型中。具体流程可能各不相同：后训练非常适合把这些洞见从轨迹中带回模型。要把信息带回上下文或记忆中，你可能需要在这些轨迹上再运行一个智能体，让它输出一些可供记忆的经验。要把轨迹中的这些经验带回执行框架中，你可能需要在这些轨迹上运行一个智能体，并更新执行框架本身的代码。uh but still you can still put a lot of your domain knowledge around what should do both routing and things in there and then the model is is uh is the hardest control but potentially the highest ceiling the the the the thing that lets you iterate on each of those is eval so there is a ton of value in these but there's only value in that you like capture insights that you bring back to the context or bring back to the harness or bring back to the model and that exact process can look different so post- trainining great for bringing these insights from traces back to the model. For bringing things back to the context or memory, you're probably going to want to run like another agent over those traces and then just and then have that agent output kind of like learnings for memory. And then for bringing the these learnings in the traces back to the harness, you're probably going to want to run an agent over those traces and update the code of the harness itself.

而且我们确实有一个能帮人做这件事的工具，叫作 engine。它是 Langmith 里的一个智能体，会分析 traces 并尝试更新代码。然后，当你更新这些内容后，你需要知道它是不是真的变好了。这时候 eval 就派上用场了。And we actually have something that does this for people. It's called engine. It's an agent in Langmith that runs over traces and tries to update code. And then and then when you update these things, you're going to need to know that it's actually better. And this is where eval come in handy.

持续学习如何改变智能体How Continual Learning Changes Agents

主持人：我能问一下，持续学习会如何改变智能体及其工作方式吗？Host: Can I ask how does continual learning change agents and how they work?

嘉宾：持续学习，我不认为它会太大程度地改变智能体的工作方式，但我认为它会改变你希望围绕智能体运行的系统。我的意思是，我认为持续学习智能体的核心循环——或者说核心内部循环——大体上还是一样的。它会是一个 LLM 在循环中运行、调用工具，并与模型和环境交互。我认为持续学习增加的是一个外部循环。嗯，一个观察智能体运行产出、然后据此采取行动的外部循环。而且再说一次，它具体做什么可能会因你对持续学习的理解而有所不同。所以我认为持续学习也是那些没有特别精确定义的词之一，或者说，也许……Guest: Continual learning, I don't think so much changes how agents work, but I think it changes the systems that you want to have running around the agent. So what I mean by that is is is I think the core loop for a continual learning agent the core inner loop maybe to say will will still largely be the same. It will be an LLM running in a loop calling tools interacting with a model and in an environment. I I think what continual learning adds is an outer loop. Um an outer loop that looks at the exhaust of of agents and then and and then does things with it. And again, what what it does could be different depending on on what continual learning means for you. So, I think continual learning is also one of those words that doesn't have a super precise definition or maybe

主持人：我会把它定义为一种在运行过程中会随着时间推移而不断改进的系统。而且，它可以通过不同方式逐渐变得更好。它可以更新上下文，可以更新运行框架的代码，也可以更新模型的权重。这些方式的外层循环大致相同：获取运行轨迹，总结经验，修复某些问题，衡量效果以确认是否有所改善，然后重复。但具体到如何修改，比如更新模型权重，就和更新一个作为记忆文件的文本文件非常不同。Host: I I would define it as a system that gets better over time um as it runs. And again, there's different ways it could get better over time. It could update the context. It could update the code of the harness or it could update the weights of the model. Um and the outer loop will look roughly the same for all of those. It will look like take traces, get learnings, fix something, measure it to make sure it improved and repeat. But that exact kind of like changing again updating weights of model very different from updating a text file that is like a memory file.

为什么智能体记忆层仍然漏洞百出Why the Agent Memory Layer Is Still Wide Open

嘉宾：如果我们看看刚才提到的背景，比如数据是其中的一部分，我们会发现数据公司正以有史以来最快的速度将营收做到数亿美元。我的意思是，真的是有史以来最快，而且每个人都在卖数据。嗯。Guest: If we look at the the context like we mentioned data being a part of that, we're seeing data companies scale to hundreds of millions of revenue in the fastest time ever. I mean like ever ever and everyone is selling data. Mhm.

主持人：考虑到你们保持中立、不直接参与其中，但又与这个市场相邻，你们预计这个市场会如何发展？未来几年，这个市场会出现整合吗？你们怎么看它的发展？Host: How do you expect this market to play out given your neutrality on not being involved in it, but being adjacent to it? Will this be a market of consolidation in years to come? How do you see it play out?

嘉宾：我其实觉得，从整体来看，数据市场还处于相当早期的阶段，尤其是我们谈到为智能体提供上下文层的时候。我的意思是，有很多数据库，比如 ClickHouse，可能就是一个很好的例子，最近发展得非常好。我觉得它们主要还是专注于做数据库。它们就是传统意义上的数据库，擅长存储和查询数据，但方式仍然相当传统。我们谈到为智能体提供上下文层时，ClickHouse 在 AI 出现之前就已经存在了。再说一次，软件系统会产生大量数据，我觉得现有的数据库，比如 ClickHouse、[ __ ]、Elasticsearch 之类的，都很适合处理这些数据。但当我们谈到为智能体提供上下文层时，我认为其中一个关键部分是：那里确实有一个数据库，但我其实觉得数据库周围还需要一套系统。Guest: I actually think the the data market is still like pretty early in the grand scheme of things in so much as we're talking about kind of like a context layer for agents. So, what I mean by that is like there are uh there are a bunch of databases that are Click House is maybe like a really good example of one that's doing incredibly well recently. Um I I think they're still largely focused on kind of like they're they're they're a database. They're good old fashioned database. They're good at storing data and querying data, but it's still um in in pretty kind of traditional ways. Click house existed pre prei when when we talk about kind of like a context uh layer for agents. And again like there will be a bunch of data produced from like software systems and great like I think like existing databases click house [ __ ] elastic whatever are are are great for that when we talk about a context layer for agents I think a key part of what what exists there is yes there's a database but I actually think there's a system around the database.

我认为，智能体的上下文层会更像是一个围绕数据库构建的智能体，而不只是一个数据库。原因在于，要把所有这些经验都摄取到数据库里，需要某种智能能力。我认为，这种智能能力其实非常、非常难。记忆最难的部分其实根本不是数据库。我们和不少公司聊过，他们实现记忆的方式就是把内容存进一个单独的文件里，非常非常简单。难点在于，如何把所有发生的观察结果——无论是用户与软件交互（比如传统的 SaaS 产品），还是用户与智能体交互时产生的观察结果——提炼成足够有价值、能够放进这个上下文层的信息。I I think like a context layer for agents will look more similar to like an agent around a database than just a database. And the reason for this is that taking all of these learnings and ingesting them into a database, you need some sort of like intelligence to do that. And I think that intelligence is actually really really hard. The hardest part about memory is actually not the database at all. When we talked to a number of uh companies, the way that they do memory is they just store it in a single file. Very very kind of simple. the the hard part is taking all of the observations that happen whether it's from the user interacting with like the let's say it's a traditional SAS interacting with the the software or the user interacting with the agent and distilling that into information that's valuable enough to go into this context layer.

这就是目前记忆中最难的部分。That's the hard part in memory right now.

主持人：为什么这件事这么难？Host: Why is that so hard?

嘉宾：因为要让 LMS 按照你的要求行事并不容易。我觉得，这种元反思——比如反思提示词并改进提示词，反思智能体架构并改进智能体架构——也是 LM 仍然很难做到的事情。我们应该在内部为此构建一个基准测试。我们确实有一些针对这方面的基准测试，是关于 Engine 的，也就是那个在各种事物上运行的智能体。但我认为，对智能体来说，这种元反思仍然非常困难。我觉得这对人类来说通常也很难做到。比如说，我不知道我们是否真的理解——可能有些人确实理解，所以我不该这么说——但我不知道我们是否理解我们的大脑内部究竟发生了什么。Guest: Because it's tricky to get LMS to do what you want it to do. And this type of like this type of like meta reflection I think is something that LM actually still struggle at a lot like reflecting on um on on on kind of like prompts and improving prompts reflecting on agent architecture improving agent architecture. We should build a benchmark for this internally. We we do have some benchmarks for this in terms of engine which is our agent that's running on things. But I think this type of meta reflection is still really difficult for agents. I think it's also generally like a hard thing for humans to do as well. Like I don't know if we actually understand, some people probably do, so I shouldn't say this, but I don't know if we understand like like what like what what goes on inside our brain.

就像，我觉得我们的大脑仍然是某种，呃，呃，复杂的东西，而我们并没有完全理解它。所以，如果我们现在要在一个系统里复现这种学习过程，我觉得其中有很多不同的组成部分。所以我觉得你得……我……对，我觉得模型不太擅长反思。我也不认为我们知道这个系统应该是什么样子。输入应该是什么样的？输出应该是什么样的？这个智能体的运行框架应该是什么样的？不同的工作流程又应该是什么样的？Like I feel like our brains are still kind of like uh uh uh complex things uh that we don't fully understand. And so if we're trying to now replicate this kind of like learning process in uh in in in in a system, I think there's a bunch of different pieces. And so I think you have to I Yeah, I I I think models aren't great at kind of like reflection. And I don't think we know what this system should look like. What should the inputs look be? What should the outputs be? What should the harness for this this agent be? What should the different workflows be?

主持人：12 个月后，这些模型的反思能力会有显著提升吗？Host: Will the models be meaningfully better at reflection in 12 months?

嘉宾：可能吧。我的意思是，他们似乎会在各方面都变得更好。Guest: Probably. I mean, it seems like they'll get better at everything.

主持人：你认为模型改进的速度会继续像现在这样加速，还是说我们正接近某种渐近平台期？Host: Do you think the speed of model improvement will continue in the same manner of acceleration that it has done or do you think we are approaching some form of asmtotic plateauing?

嘉宾：我觉得，不在实验室里，恐怕很难说清模型进步的速度。我我我会说，我觉得有很多任务上，模型的改进可能会，呃，趋于平稳，意思是它已经足够好了。我觉得有一大堆任务，模型已经足够好。Guest: I think the speed of model improvement is something that's probably really tough to say without being inside of a lab. I I I would say I think there are a lot of tasks for which the model improvements will will will kind of like uh flatline in the sense that like it's already good enough like I think there are a bunch of tasks where the models are just good enough

主持人：这就像编程一样Host: which would be like coding

嘉宾：是啊，编程方面正在逐渐进步，嗯，比如摘要、任务路由。可能还有更难的任务，或者一些我们甚至还想不到该交给模型去做的任务。我想说，这类元学习就是模型目前还不擅长的任务之一。我猜这对实验室来说非常重要，关系到能否突破 RSI 之类的问题，或者其他类似的事情。嗯，所以我确实认为，随着时间推移，模型在这方面会越来越好，而且这种进步有可能看起来比以往快得多。嗯，我觉得如果没有在实验室里亲身参与，很难形成有根据的看法。有些任务的进展已经停滞了；但在其他任务上，我认为进步速度可能会比以前更快。Guest: yeah coding's getting there um uh like summarization uh routing tasks um there are probably more difficult tasks and or tasks that we don't even know to think about know to throw a model at I I would probably say like this type of like metalarning is one of those tasks that the models aren't good at now. I would imagine is very important to the labs in terms of kind of like cracking RSI or or or things like that. Um and so I do think the models will get better at that over time and it's possible that that progress actually looks uh much faster than that than than previous. Um I think it's really tough to have an educated opinion without being inside of a lab. There are some tasks where it's saturated. There are other tasks where I think it will probably just increase faster than before.

如今关于智能体的哪些看法，三年后会变得显而易见？What About Agents Today Will Be Obvious in 3 Years?

主持人：如今关于智能体的哪些说法听起来很疯狂，但你认为两到三年后会变得非常普遍？Host: What about agents today sounds crazy that you think will be incredibly common in 2 to 3 years?

嘉宾：我是说，我觉得代理的记忆层仍然是一个远未解决的问题。我认为，代理领域有一项目前还不常见、但未来几年会变得极其普遍的东西，那就是记忆层。我这么说，是因为你看看 Instinct 或 Muse，我觉得它们做得特别好的一点，就是个性化和学习，让人感觉它是专为你打造的——无论是因为它说话的语气，还是因为它会主动跟进。我觉得记忆这方面还非常非常早期。我们和业内人士聊记忆已经聊了大概两年，也一直在尝试做开源项目。我们推出过一个 alpha 版，后来又放弃了一个商业化的记忆解决方案。记忆这块确实还处于非常早期的阶段，而且非常依赖具体领域。Guest: I mean, I think this memory layer for agents is still massively unsolved. I think one thing about agents that is is rare today but will be incredibly common in the next few years is is this memory layer. So, what I mean by that is if you look at like instinct or muse, I think one of the things that they've nailed is this kind of like personalization and learning and seeming like it's it's something for you, whether it's because it's speaking in the tone or because it's just following up like that proactiveness. And and so I think I think memory is super super early. We've been talking with folks about memory for like 2 years and trying to do things in the open source. We we we launched an alpha and then and then killed a commercial solution for memory. It's just so early on in memory and it's so domain specific.

Instinct 或 Muse 的记忆机制，实际上和公司或运行框架的记忆机制很不一样，也和法律、金融之类领域的记忆机制很不一样。所以我认为，记忆确实是一个能带来显著差异的因素。你可以看到它带来的不同；我觉得 Instinct 和 Muse 在这方面做得很好，所以也能看到记忆给它们带来的不同。嗯，不过我认为现在还非常非常早期，我预计几年后所有产品都会包含某种形式的记忆。恕我直言，这难道不是相当显而易见吗？What memory looks like for Instinct or Muse is actually pretty different than what it probably looks like for a company or or harness and it's pretty different than what it looks like for legal or finance or things like that. So I think memory is is is one thing that really makes a difference. You see this make a different I think instinct and muse have done a great job at this and so you see how it makes a difference there. Um but I think it's still very very early on and I would expect uh all products in a few years to have some aspect of memory in it. With respect, is that not quite obvious?

主持人：我觉得，我觉得这件事……我认为记忆会产生影响这一点相当明显，这也是为什么过去几年里你会看到那么多人对它感兴趣。我仍然觉得答案并不显而易见。我仍然觉得答案极其不显而易见。我认为没人知道记忆问题的答案是什么。Host: I think I think it's I I think the fact that memory will have an impact is quite obvious and that's why you see so many people interested in it and have having it be interested in it for for the past few years. I still think it's nonobvious what the answer is. I still think it's incredibly non-obvious what the answer is. I don't think anyone knows what the answer for memory is.

嘉宾：既然它显而易见，而且几乎所有场景都明确需要它，你们为什么还要放弃这个商业化方案？我们没有……我们放弃它，是因为没有找到产品市场契合点。我想，嗯，我们为什么没找到产品市场契合点？我觉得其实是因为我们想做得太通用了。我觉得现在记忆（memory）还处于非常早期的阶段，所以我认为你想构建的很多解决方案，实际上都非常偏向特定领域或特定智能体。Guest: Why did you kill the commercial solution for it given the obviousness of it and the clear need for it in in almost everything? We didn't we killed it because we didn't find product market fit. I think um why didn't we find product market fit? I think we actually tried to be too general. I think memory is so early right now that I think a lot of the solutions that you want to build are actually very like domain or agent specific

主持人：那么你会构建垂直领域专用的内存解决方案。你会做出哪些不同的选择？Host: and so you would build memory spec like vertically specific memory solutions. What would you do differently?

嘉宾：回到记忆中困难的部分是什么这个问题。我觉得记忆中困难的部分，是从事实中学习该记住什么的过程。嗯，还有负责记忆的那个智能体。嗯，正如我说的，我非常看好垂直领域智能体。所以，呃，我认为让这些智能体善于学习特定领域的知识，是其中的关键部分。我确实觉得，这类学习型智能体至少在起步阶段可能必须是特定领域的。嗯，我也不知道有没有什么通用的模式：这些学习型智能体的运行框架会是什么样？它大概会和我们现在的很多运行框架非常相似。它大概会是……我们构建 Deep Agents。Deep Agents 是一个智能体运行框架。Guest: Going back to what the hard part of memory is. I think the hard part of memory is that process for learning from facts about about what to remember. Um, and that agent that that that does that remembering, um, as I said, I'm I'm massively bullish in kind of like vertical agents. And so, uh, and I think having those agents be good at learning things specific to their domain is is is a key part of of that. And so, I do think these kind of like learning agents, at least to start, probably have to be domain specific. Um, and and and I don't know if there's like a general like what what does the what does the harness for these learning agents look like? It probably looks pretty similar to a lot of the harnesses we have now. It probably looks pretty we build deep agents. Deep agents is an agent harness.

可能看起来差不多。一个运行循环的 OL，但它有特定的指令和工具，也许还有一些擅长浏览现有记忆结构，或者提炼事实的特定子代理。所以我认为会有专门负责记忆的代理。我只是觉得它们会是领域专用的。Probably looks similar to that. An OL running the loop, but it's got specific instructions and specific tools and maybe there's some specific sub aents that are really good at, you know, uh, navigating existing memory structures or or distilling kind of like facts. And so I think you'll have you'll have agents for remembering things. I just think they'll be domain specific.

每个消费类应用都会拥有自己的垂直领域智能体吗？Will Every Consumer App Have Its Own Vertical Agent?

主持人：你认为消费类应用会出现垂直领域智能体吗？我理解并认同你的看法：你提到了金融、会计、销售、营销和法律。你提到的 Muse 和 Instinct 会是通用型智能体。你认为我们会有垂直领域的消费类智能体吗？Host: Do you think you will have vertical agents for consumer applications? I understand and I agree with you when you look at finance, accounting, sales, marketing, legal. You mentioned muse and instinct. They would be horizontal agents. Do you think we'll have vertical consumer agents?

嘉宾：我认为，在消费领域，这两种智能体都会存在：横向智能体和垂直智能体。而横向智能体会与垂直智能体交互。什么情况下适合采用垂直智能体，什么情况下不适合？Guest: I think my take on the consumer space is that you'll have both. You'll have both horizontal and vertical agents. And the horizontal agents will interact with the vertical agents. Where does it make sense to have a vertical agent versus not?

主持人：我觉得，对大多数应用来说，开发一个垂直领域智能体可能是有道理的。我认为垂直领域智能体之所以有意义，可能有两个原因，也许不止两个，但我首先想到的是这两点。嗯，第一，很多这类应用都了解你，也能比通用型智能体更好地了解你如何使用这些应用。嗯，回到领域特异性这个问题，比如说，你使用 Spotify 听音乐的方式，和你使用 Strava 的方式非常不同，对吧？所以，这些应用其实可能最擅长了解你。而我认为，展现这些了解成果的最佳方式可能就是通过智能体。另一点是，很多这类应用都有大量 API。所以你可以说：“嘿，Spotify，别开发智能体了，直接开放一堆 API 就行。”Host: I think for most apps it probably makes sense to have a a vertical agent. And I think the reason that having vertical agents makes sense at all is is is probably twofold. Maybe more, but two that jumped to mind. Um, one, a lot of these apps have uh knowledge about you and can learn things about how you interact with that that app better than a general horizontal agent could. Um, so back to the point about domain specificity, how how you interact with music for say like Spotify um, is is is very different than how you might interact with Strava, right? And so those those apps are actually probably best at learning things about you. And I think the best way to expose those learnings is probably through an agent. The other thing is a lot of these apps have a ton of APIs. And so you could say, hey, you know, Spotify, don't build an agent, just expose a bunch of APIs.

我其实觉得，如果有一个智能体把这些 API 封装起来，并向横向智能体提供一个——或者也许多个，但大概是一个——自然语言接口，让它与垂直智能体交互，体验会更好。I actually think it's better for it's a better experience if there is an agent that kind of like wraps those APIs and exposes a single or maybe multiple but like probably a single like natural language interface for a horizontal agent to interact with with that with that vertical agent.

当智能体生成的请求量达到人类的 1,000 倍时，会发生什么？What Happens When Agents Generate 1,000x More Requests Than Humans?

嘉宾：当智能体成为与网站交互的基本界面时，基础设施会发生怎样的变化？我的意思是，嗯，我想买某场演唱会的门票，于是我说每隔 5 分钟查一次，只要有座位放出来就买。接下来的 3 天里，它们每隔 5 分钟就查一次。查询量是人工操作时的 1000 倍。在智能体交互或人类交互的世界里，所需的基础设施会发生怎样的变化？Guest: How does the infrastructure change when agents are the fundamental interface with sites? And what I mean by that is, you know, um I wanted tickets to a specific concert and I said check every 5 minutes and buy any seat that comes available and for 3 days they checked every single 5 minutes. The volume of queries was a thousand times what it would have been with humans. How does the infrastructure required change in a world of agent interaction or human interaction?

主持人：当智能体进行交互时，嗯，请求数量会大幅增加。我想你应该看到过一个故事：我记得是 Resi 封禁了某个用户的账户，因为他们用 Instinct 来拉取数据——我不知道具体频率，但应该挺频繁的。所以我觉得，基础设施面临的压力确实大得多。这种影响会怎样传导到更底层？我现在还没有明确看法。我认为你必须构建能够应对更大突发流量、扩展能力更强的系统。我没有更多具体想法了。你觉得服务提供商会允许智能体使用吗？Host: When agents are interacting uh it it it makes the number of requests go way up. I think I think you saw a story about how I think it was resi that blocked uh a user's uh account because they were using instinct to to pull it uh I don't know how frequently but pretty frequently and and and so I think yeah the the infra there's just way more strain on on the infrastructure. How does that percolate down to lower layers? I I don't know if I have an opinion on that yet. I think you have to build systems that can be much more bursty that can scale way broader. I don't I don't know if I have more specifics. Do you think providers will allow agent usage or not?

嘉宾：我认为，未来服务提供商会推出自己的智能体，人们要么直接与这些智能体互动，要么让自己的通用型智能体与它们互动。Guest: I think the future is providers exposing their own agent and having people either interact with that agent or having people's horizontal agents interact with that agent.

主持人：那么，我的水平代理——抱歉，我刚明白，它会与 res 的垂直代理交互。Host: So my horizontal agent, sorry, I just understand this would interact with res's vertical agent.

嘉宾：我认为这完全正确。Guest: I think that's exactly right.

主持人：这为什么会行得通？因为这仍然没有解决很多这类企业面临的两个核心广告问题。他们会失去广告模式，也会失去追加销售的机会——比如顺便推荐其他你可能喜欢的东西，结果就是购物篮金额减少。为什么智能体会行得通？我觉得可以把其中一些内容纳入智能体的回复。比如，你可以针对 Resi、Etsy 或其他平台上的特定商品投放广告，并把广告内容放进智能体的回复里，比如：“嘿，我找到了这三样东西，你可能还想考虑一下这件东西。”这样就成了一条自然语言回复，再传回主智能体那里。我觉得，关于系统负担这一点也是如此：如果你有一个垂直领域智能体，它与底层系统交互时，会比第三方智能体做得更好、更高效。Host: Why would that work? Cuz that still doesn't solve the two core problems of advertising for a lot of these businesses. they lose that ads model and then upsell they lose the like anything else you might like also on the side so you get reduced basket size why why would agent agent work I think you could include some of those things in the agent response um so you could you could do advertising for specific things on on resi or Etsy or whatever and include that in the agent's response like hey I found these three things you might also want to consider this other thing that's now a natural language response that goes back to kind of like the the main agent I think also to the point around the strain on the systems if if if you have a vertical agent, it's going to be better and more efficient at interacting with the underlying systems than a third party agent will.

所以，我认为这样会让他们做这件事的效率大大提高。So, I think it will make it a lot more efficient for for them to do that.

嘉宾：当我们查看查询数量时Guest: When we look at the amount of queries

为什么智能体成本可能会成为一个大问题Why Agent Costs Could Become a Huge Problem

而且，随着智能体数量变化，工作量也会变化。没错，完成所有这些工作需要消耗很多 token。随着时间推移，token 的成本以及这种智能的成本会如何变化？因为我们一直以为成本会越来越低，但实际上并没有便宜多少。我们该如何看待这个问题？and the volume that changes with agents, yeah, it takes a lot of tokens to do all of this work. How does the cost of tokens and cost of that intelligence change over time? Cuz we always thought it would get cheaper. Hasn't got much cheaper. How do we actually think about that?

主持人：成本是我们现在经常和大家讨论的一个重要问题。我觉得这里面有几个不同的方面。第一，我觉得前沿模型的价格总体上在逐渐降低。与一年前存在的许多产品相比，Astra 的性价比非常出色。第二，需求大幅增加了。当你开始讨论智能体与智能体之间的交互，或者多智能体系统时，消耗的 token 数量就会多得多，因此成本也在上升。嗯，我们主要在编程领域看到了这一点。编程成本飙升了，我觉得包括我们在内的许多组织现在都在考虑如何削减这部分成本。嗯，这最终会如何解决？呃，我的意思是，如果你看看我们如何应对编程智能体，我觉得其中已经能看出一些端倪。Host: Cost is one of the big things that we talk with people about right now. I think I think there's a few different dimensions to this. One, I think like the the Frontier models are relatively getting cheaper over time. Astra is like price performance fantastic compared to a lot of things that existed a year ago. Two, there's just way more demand. When you start talking about agents talking to agents or multi- aent systems, there are just so many more tokens that are getting burned and so the costs are going up. Um we see this in coding primarily. coding costs have skyrocketed and I think a lot of organizations including ours are now thinking about ways to kind of like pair pair those costs back. Um h how does this end up getting resolved? Uh I mean if you look at how we're tackling coding agents I think there's a few hints there.

所以，嗯，在处理编码智能体时，我们会把它分成三个步骤。第一步，基本上就是提供可见性，让管理员知道各项内容会花多少钱、已经花了多少钱。第二步是成本控制。比如你可以说：“嘿，帮我订个预订，但别花超过 5 美元。”嗯，然后第三步实际上是影响 harness 本身。所以我认为，迄今为止，大部分 harness 工程主要都是为了提高智能体的效率，因为这是——哦不，抱歉，大部分 harness 工程都是为了让智能体变得更好、更准确，因为这是最大的瓶颈。嗯，我觉得随着人们越来越关注成本，我们看到大家也开始更重视提示缓存之类的技术，它们确实很有助于降低成本。So so um when we're tackling coding agents we think of it in three steps. One we think of basically just like visibility showing showing kind of like how much things uh will cost how much things have cost to admins. Two is cost control. So you could say something like, "Hey, you know, go get me a reservation, but don't spend more than $5 like doing it." Um, and then uh and then third is actually impacting the harness itself. So I I I think primarily a lot of harness engineering has been done around making the agent more efficient because that's the or sorry a lot of the harness engineering has been done to make the agent just like better, more accurate because that's the biggest blocker. Um I think as cost becomes more concerned, we're seeing people care more about things like prompt caching which are really good for kind of like reducing costs.

我们看到模型提供商实际上让这件事变得更容易了。这有点深入细节了，不过他们引入了一些功能，比如可以在代理运行过程中更改其可用工具。以前，这会破坏缓存并导致成本大幅增加。现在，你可以动态添加工具，同时保持成本不变。所以我认为，我们看到人们越来越关注模型 API 和代理框架，以控制成本。We're seeing model providers actually make this easier by and this is getting a little bit in the weeds, but they introduced things like where where you can change the tools that are available to an agent mid-flight. Previously, this would have busted the cache and cause like a massive increase in in cost. Now, you can just add those dynamically um and and keep the cost the same. So, I think we're seeing more attention to both the model APIs and and harnesses in terms of like controlling costs.

主持人：如今与大型企业交流时，他们最看重什么？对他们来说最重要的是什么？是准确性、成本，还是安全性？Host: When you talk to large enterprises today, what do they matter most? So what matters most to them? Is it accuracy? Is it cost or is it security?

嘉宾：嗯，还有一点是延迟。顺便说一下，我们每年都会做开发者调查，询问开发者关心什么。过去两年里，排名第一的回答基本上是后面两项加起来的两倍，那就是准确性。所以这仍然是首要因素。接下来两项是延迟和成本，安全性则稍微靠后一些，但差距不大。嗯，最近我们和大型企业交流时，我觉得成本受到的关注有所上升。我仍然会说它低于准确性，但可能高于延迟。而对于大型企业来说，安全性确实更受重视，但可能仍低于准确性和成本。Guest: Uh another thing would be latency by the way. So we run developer surveys every year where where we ask developers what they care about. For the last two years the number one response uh basically double the next two has been accuracy. So that's still the the primary thing. The next two are latency and cost and security is like a a little bit after that pretty close. Um when when we talk to large enterprises recently, I'd say like cost has has risen in terms of concern. I'd still probably say it's below uh below accuracy, but it's probably higher than than latency. And and then security is for large enterprises, security is is is is higher, but still probably below accuracy and cost.

LangChain 将约 30% 的工程师薪资用于编程智能体LangChain Spends ~30% of Engineering Salary on Coding Agents

主持人：我能问一下，你大概会把工程师薪水的百分之多少花在编程工具上？我不是特指……不过你会花钱买编程工具吗？Host: Can I ask you what percent of your engineering salary give or take? I'm not specifically, but do you spend on on coding tools?

嘉宾：我觉得我们的工程预算大概有 30% 花在了编程工具上。其实过去两个月左右，这个比例还下降了。嗯，虽然我们使用的 token 数量实际上增加了。我们是通过三个步骤做到这一点的。第一步基本上就是提高可见性。我们把所有编程智能体的使用情况都追踪到 Langmith，这样就能知道谁花了多少钱、钱花在了什么地方，也能据此做分析。第二步就是基本的成本控制。我们设定了一个预算，如果你用到了预算上限，可以找我们的工程副总裁聊聊，他几乎总会给你提高额度，但这只是为了防止智能体被不当使用，或者陷入糟糕的循环。第三步则是控制运行框架。Guest: I think we spend probably roughly 30% of our engineering salad on coding tools. It's actually gone down the last uh 2 months. Um while tokens, the number of tokens that we've used has actually gone up. And so this has we've done this by doing uh three steps. First is basically just visibility. We tracked all of all of our coding agent usage to Langmith. So we could tell who was spending how much, what they were spending it on. We could do analytics over that. Uh second is just basic cost controls. We we set kind of like a budget and and if you if you hit the budget, you can talk to our VP of engineering and and he'll he'll almost always like increase it, but it's just a safeguard against kind of like agents, you know, being used uh incorrectly or going in a in in a bad loop. Um and then third would be we control the harness.

我们做过的一件事，就是自己搭建了一个内部云端编程代理。它不是 IDE，也不是基于终端的代理；这类工具我们让大家随意使用，但我们自己搭建了一个云端代理，叫 Open Suite。实际上，这是我现在唯一使用的编程方式，因为我一直都在 Slack 上，所以只要在 Slack 里给它发消息，它就会开始运行。 在这个代理框架里，我们做了不少工作来控制成本，比如提示缓存，以及集成到框架中的路由功能。我觉得，要控制编程代理的开支，这是一套相当容易复用的三步流程：首先提高支出的可见性，其次实施一些基本控制，最后尝试改进代理框架本身。Um so one of the things we've done is we actually uh built our own uh in-house cloud coding agent. So not not a not a not a IDE, not a not a terminal based agent. We we let people use whatever they want for that, but we built our own cloud-based agent, open suite is what we call it. Um it's actually the only way I code anymore because I'm in Slack all the time and so I just message it from Slack and it kicks off. But in that harness, we've done a bunch of work to control costs. Um so things like prompt caching, but also things like routing we've built into this harness and I actually think that's like a pretty repeatable three-step process for kind of like getting coding agent spend under control. Just first visibility, second like basic controls, third like try to improve the harness itself.

主持人：完全理解。我确实想进一步探讨一下。你提到了可见性，也就是看看谁花了多少钱——花钱最多的人是不是最有效？Host: Totally get that. I do just want to unpack that. You mentioned that visibility, see who's spending what were the people spending the most the most effective.

嘉宾：有时候，花费最多的人效率最高；有时候却不是。我的意思是，我们有几位资深工程师花费很高，而我百分之百信任他们，相信他们做的事情完全值得。对此我毫不怀疑。我们也遇到过一些情况，使用量突然大幅飙升，原因与其说是别的，不如说是一些愚蠢的失误或不恰当的配置。举个例子，我们有一两个任务用了某个插件，它会在智能体每一轮执行时截取网页 DOM 的屏幕截图或快照并发回去，因为他们在做一些前端工程，想把页面的样子捕捉下来再发回来。结果它就这样不断地运行，形成了一个巨大的循环。Guest: Sometimes the people spending most are the most effective and other times they're not. So what I mean by that is we have a few senior engineers who spend a lot and I I 100% trust that what they're doing is well worth it. No questions there. We've also just had some cases where um due more to like kind of like uh silly mistakes or kind of like improper setup than anything, we've had like massive spikes in usage. So, so an example of this um we we had uh one or two tasks where the some plugin was being used that basically like would would screenshot or or or take a snapshot of of the DOM of a web page and send that back every turn of the agent because they were doing some front end engineering and they wanted to take what it looked like and just send it back. It would just run this massive loop.

所以，我想我们一周在那一件事上花了大概四万美元。你知道，到了这种程度，就像我刚才说的，我觉得我们公司整体上还是挺宽松的，也愿意信任员工，但我也知道，这样做可能不值得。所以，这只是一个例子。再说一次，这并不是恶意的，也不是有意为之，甚至算不上是有意的使用方式。基本上就是弄错了。所以我觉得，能发现这种情况还是很值得的。具体要看你处在这个光谱的哪一端。So, I think we spent um 40k in a week on on that one thing. And so, you know, when it gets to that, like again, like, you know, I I think we're pretty kind of like uh liberal as a company and trust people, but I I know that's probably not worth it. And so, that's just like an example of like, you know, again, it wasn't malicious. It wasn't uh it wasn't um it wasn't even intended usage really. It was kind of just a mistaken. So, I just think just like catching that is is is well worth it. So, it depends a bit on on side of the spectrum.

AI 智能体会让工程团队规模大幅缩小吗？Will AI Agents Make Engineering Teams Dramatically Smaller?

主持人：正如你提到的，工资的 30%。Host: 30% of salary as you mentioned.

嘉宾：这只是个粗略估计。对，Guest: That's a guesstimate. Yeah,

主持人：估算一下就行。完全没问题。两年后会是多少？Host: guesstimate. Totally fine. What is that in two years?

嘉宾：我认为，估算编程智能体的开销，难点在于各个组织都在飞速变化。我想表达的是，我觉得未来的组织规模会小得多，编程智能体的开销可能会和人工薪资一样高，甚至更高，或者高出一个数量级；而少数几个人就能以极快的速度、极高的效率来推动并执行工作。我觉得这是一个很诱人的愿景。不过，我认为，包括我们在内的很多公司，可能还没有发展到那一步。Guest: I think the hard part about estimating coding agent spend is that orgs are just changing so rapidly. So what I mean by that is I I I think there's a future where orgs are much smaller and coding agent spend is uh you know the same as as human salary or or or more or an order of magnitude more and you have a small number of of people that are uh driving and executing uh incredibly rapidly and incredibly performantly and I and I think that's uh you know an enticing vision in terms terms of of of what you think about um I think I I don't think many companies including us are are probably there.

我觉得我们仍然发现，我们需要很多来自不同背景、拥有不同专长的人一起协作，而且会有大量沟通、协调开销和审查。没错，理想情况下，一个人能驾驭 100 个智能体，推动构成产品很大一部分的所有工作。但我觉得我们仍然看到，这在很大程度上还是一件非常、非常依赖人的事情，其中仍有一个非常人性化的组成部分。所以我花了很多时间思考组织设计之类的问题，比如能不能组建小团队，甚至是一个人的团队、两个人的团队，由他们来驾驭智能体。我觉得这种做法很诱人。在这种情况下，我会说，智能体的开销可能是原来的三倍，但与此同时，能做到这一点的工程师也极其宝贵。所以也许实际开销最终还是会保持不变，因为那些真正能做到这一点、能驾驭所有这些智能体的人——人数会更少，但你会给他们付更多的钱，因此实际开销最终还是一样。I think we still see that uh we need a bunch of different people with a bunch of different backgrounds and a bunch of different expertisees collaborating on things and there's a bunch of communication and overhead and review and like yes it would be ideal to have one person driving a 100 agents and just like driving everything that like makes up you know a large part of the product but I think we just see that it's still a very kind of like human human there's still a very human component to it And so I spend a lot of time thinking about kind of like org design and like is it possible to have like small teams, one person teams, two person teams that are kind of like driving agents. Um I think it's very tempting and in that situation I would say like you know agent spend maybe 3x the cost of of but then also you know the engineers who can do that are incredibly valuable and so maybe it actually will just stay the same because the people who can actually do that and who can drive all those agents you're you're you're there's fewer people but you're paying them a lot more and so the spend actually stays the same.

所以我觉得，组织设计现在非常非常有意思，因此我认为这个问题其实很难回答。So I think org design is very very interesting right now and so because of that I think it's actually really hard to answer that question.

主持人：对我来说，这个数字如此重要，因为Host: To me that number is so central because

Anthropic 的估值取决于什么What Anthropic’s Valuation Depends On

这是决定 Anthropic 企业价值的根本因素。如果这个数字降到 5%，那么 2 万亿美元的 IPO 估值可能就有点过高了。如果这个数字实际上达到 300%，就像 Mccor 的情况一样，那估值就明显偏低了。这就是我如此关注它的原因。你同意这个数字是 Anthropic 价值的基础吗？it is the fundamental determinant of Anthropics enterprise value. If that number goes to 5% then the $2 trillion IPO might be a little bit overpriced. If that number actually goes to 300% like it is in Mccor's case then it's dramatically underpriced. And so that's why I focus so much on it. Do you agree that number underpins anthropics value?

嘉宾：我认为，这个数字支撑着 Anthropic 的估值，可能是合理的。不过棘手之处在于，到目前为止，我们唯一看到智能体真正发挥作用的领域是软件工程；但在企业内部，还有很多其他领域，我认为智能体尚未对它们产生影响。所以我认为，这仍然是有可能的。软件工程方面，我觉得很难预测，但我认为基本上有两个关键因素：一是软件工程领域会发生什么——会和今天一样多、更多，还是更少？但我认为，毫无疑问，在其他领域，智能体带来的影响会远超今天。从这个角度来看，我认为这对 Anthropic、OpenAI 和其他模型提供商来说是非常利好的。我认为……Guest: I think it's probably right that that number underpins anthropics value. I think the tricky part is that the only place we've seen agents really have effect is in software engineering so far though and there are a ton of other places within an enterprise that I I would argue have yet to kind of be affected by by agents. And so I think it's possible again software engineering I think very difficult to to to predict but but I think there's basically two levers here. One is like what happens in software engineering is it the same as today? Is it more than is it less than but I think I think undoubtedly for the other domains it will be a lot more than it is today. And so I think from that perspective I think that's very bullish on anthropic and open AI and other model providers. I think

企业是否害怕中国的开源模型？Are Enterprises Afraid of Chinese Open-Source Models?

另一个很多人都在问的问题是：开放模型的激增会如何改变格局。嗯，你与一些大型企业有合作。我有一位很要好的朋友 Jason Lmin，来自 Saster。他说……我就不模仿他的口音了，不然肯定会弄得一团糟。我以前模仿过，他觉得不太像。嗯，你发现大型企业会担心使用中国开源模型吗？还是说他们其实并不介意？我们看到有些企业确实担心使用中国开源模型；也看到另一些企业不太在意。你……another question that a lot of people are asking is about kind of the proliferation of open models and how that changes the landscape. Um you work with some very large enterprises. One of my dear friends Jason Lmin from Saster says that I'm not going to do his accent because I'll butcher it and I've done it before and he got rather he said it wasn't very good. Um uh do you find that large enterprises are fearful of working with open source Chinese models or actually they don't mind? We see that some enterprises are fearful of working with open source Chinese models. We we we see others that don't don't kind of like care. Do

主持人：你认为这种恐惧是有道理的吗？Host: you think that fear is justified?

嘉宾：我不太觉得这种担忧有道理。我觉得，嗯，确实还有别的顾虑。我想其中一个顾虑是，你知道，它运行所依赖的数据会 somehow 回流到中国的实验室。再说，如果你用 Fireworks，或者自己托管，我觉得这种担忧简直荒唐。我觉得还有其他担忧，比如说，他们训练模型所用的内容会让它以某种方式作出回应，嗯，你知道，不符合企业的价值观之类的。我觉得这就是 eval 公司所关注的，而我认为这种担忧大体上没有根据。不过，当然，在某些领域，可能确实有那么一点道理。我觉得大概是这样：这种担忧大体上没有根据，但在某些领域，考虑到那种回应方式，我能理解其中可能有一些微妙之处。Guest: I don't really think that fear is justified. I I think there's um I think there's different concerns. I think one concern, you know, is that uh the the the data that it runs on will will somehow go back to the Chinese labs. And again, if you're if you're using fireworks or hosting it yourself, I think that's just ridiculous. I I I think there's other fears that, you know, the stuff that they've been trained on will cause it to respond in a way that's uh you know, not um consistent with the the values of of the enterprise or something like that. I think that's where eval comey and I think that's largely unfounded, but like sure maybe for some domains there's there's some realness there. I I think it's probably so I think it's probably largely unfounded but for some domains because of that kind of like response I could see there being some some nuance there.

主持人：你认为在接下来的 12 个月里，我们会有实力相当的美国开源模型吗？Host: Do you think in the next 12 months we'll have comparable US open source models?

嘉宾：我认为在接下来的12个月里，我们肯定会拥有可与之媲美的美国开源模型。看看英伟达正在做的一切，看看他们如何与生态系统合作。我认为他们非常重视这件事。再看看 Reflection 最近发布的模型。我觉得它在一些基准测试上的表现仍然略逊于某些中国模型，但我认为这个领域受到了极大的关注，所以我相信这些模型肯定会迎头赶上。Guest: I think in the next 12 months we will absolutely have comparable open source uh US open source models. I think you look at uh everything that Nvidia is doing to kind of like uh work with the ecosystem. I think they they care about it tremendously. I think uh you look at uh reflection had a model launch recently. Um I think it was still a little bit behind some of the Chinese models on some benchmarks but I think there is a tremendous amount of attention in in that domain and so I think absolutely those will catch up.

大多数人对开放模型的误解What Most People Get Wrong About Open Models

主持人：鉴于你的立场，当你审视当今开放模型领域的现状时，有哪些事情是世人看不到、而你却看得很清楚的？Host: Can I ask you when you look at the state of landscape today in terms of open models what does the world not see that you see very clearly given your position?

嘉宾：我觉得说到开放模型，我认为它们的使用确实很广泛，但我也认为它们承担的任务比人们想象的要狭窄得多。所以，所以我觉得，现在有很多人在使用更多开源模型和更多经过后训练的模型，这比我一年前或一年半前预想的要多。不过我仍然认为，它们主要用于一些更简单的任务，比如路由、护栏、信息提取或摘要。我不认为它们真的在驱动许多……呃……这些主要智能体的核心智能体循环。Guest: I think when it comes to open models I think there is a lot of usage of them but I do think they're for more narrow tasks than people think. So, so I think we see a lot of people using more open source models and more post-trained models than than I would have maybe expected a year or a year and a half ago, but I still think they're primarily for these simpler tasks like routing or guard rails or extraction or summarization. I don't think they're really driving the the core agentic loops for a lot of uh the these these these main um these these main agents.

所以我觉得开源模型有点容易引发分歧，因为有些人站在一端，觉得“太好了，它们已经够用了，什么都可以用它们”；而我觉得它们还没到那个程度。还有些人则觉得，前沿模型只会越来越好、越来越便宜。没错，这确实如此。但如果你非常清楚自己要做什么，也就是说，你有一个高使用量的应用场景，而且其中有几个环节只是足够简单的任务，我觉得针对这些应用场景对模型进行一些后训练，投资回报率会非常高。所以我想说，遗憾的是，我的立场可能比较微妙，大概处于我在 Twitter 上看到的两种极端观点之间。And and so I think open source models are are kind of divisive because you have some people on on one end who are like great they're they're good enough let's use them for everything and and I don't think they're quite there and then I think you have other people who are like you know the frontier models are just going to keep on getting better and better and cheaper and cheaper and like yes that's true but if you really know what you're going to do so you have to have a high volume use case and if there are a few components of that that are simple enough tasks I think it's I think it's I think it's very good ROI to start kind of like post- training or or yeah post- training models for for those use cases. So, I I'd say, you know, unfortunately, it's probably like a nuanced uh position that's somewhere in the middle of of what I feel like the extremes on Twitter are.

哈里森为何认为推理市场规模巨大Why Harrison Thinks the Inference Market Is Massive

主持人：你怎么看推理市场接下来的发展？你刚才很坦率地提到了 Fireworks。我们是 Fireworks 的重要投资方。我很喜欢 Lynn 和她的团队，但每个人都在尝试做这件事。你……我们请来了 Crusoe 的 Chase，他所在的公司也把这作为业务的一部分。你知道，我和 Nebius 的人也很熟。他们也把这作为 Rebbit 业务的一部分。这门生意会如何发展？你怎么看有必要做完整个技术栈，还是只做 Fireworks 或 Base 10 这样的推理服务商？Host: How do you see the inference market playing out? You mentioned that Fireworks very openly. We're a big investor in Fireworks. I love Lynn and the team, but everyone's trying to do it. You you have you we had Chase on the show from Crusoe who also have that as part of their business. You know, I'm friends with the Nebius guys. they also do it as part of Rebbit. How does this business play out and how do you think about the need to do all of the stack versus inferenceon providers like a fireworks or a base 10?

嘉宾：我觉得说到推理市场接下来会如何发展，有两件事是我从 LangChain 起步到现在一直都严重低估了的。第一是对推理的需求，第二是对编程智能体的需求。我觉得这两个市场都比我两三四年前想象的要大得多。所以我觉得，最近 Benchmark 的 Eric Visher 在推文还是播客里说过类似的话：这个市场大到什么都能行得通。我觉得 Thomas Tungst 也提到过，推理市场实在太大了。你知道，我觉得 Eric 说的范围比推理更广，但如果把这话套用到推理上，我觉得任何推理服务都会行得通。Guest: I think when it comes to how things will play out with respect to inference, one of the things that I there's two things that I I think I really underestimated uh over the from the start of lang chain to now. One is the demand for inference and two is the demand for like coding agents. I think both of those markets are just tremendously bigger than than I thought two 3 four years ago. And so I think um I think actually it's it's Eric Visher at Benchmark who who said this on uh either a tweet or podcast recently where I think he basically said like everything just works like this this market is so big. I I think uh Thomas Tungst had something around just like the market for inference is just so big that you know I think Eric said it about more broad stuff than just inference but I think applying it to inference I think any inference will work.

这就像，你知道，对它的需求实在太大了，而且我不知道短期内需求会不会减少。所以，没错，Fireworks 能行，AWS 能行，Crusoe 也能行。我只是觉得，对它的需求实在太大了。That's like, you know, there's just so much demand for it and and and I I don't know if it's going to get less anytime soon. And so, yeah, sure, Fireworks will work, AWS will work, Crusoe will work. I just think there's so much demand for it.

哈里森今天会把资金投向哪些人工智能领域？Where Would Harrison Invest in AI Today?

比如说，我可以给你举几个例子：你觉得现在有哪些领域你不会投资？我觉得客户支持领域实在乏味透顶。过去18个月里，有20家供应商获得了超过1亿美元的融资。那儿还有 Salesforce、Atlassian 这样的巨头，以及一些老牌公司，比如 Intercom 和 Talkdesk，也在跟你抢生意。好吧。再比如机器人领域——这可能是个大胆的观点，但现在有海量资金涌入机器人领域，而我们距离机器人领域的 ChatGPT 时刻还很远。尤其是人形机器人，在形态设计上确实面临很大困难。我的回答是，我觉得目前的融资规模和市场热度，远远超过了实际能力。我愿意把自己看作一个相当乐观的人，所以我认为这项技术确实会带来变革，也会创造大量机会。Where would you not be investing today like I so I can give you some examples. I think the the customer support space is like wildly uninteresting. 20 providers all funded with over 100 million in the last 18 months. You got Salesforce and Atlassian and big incumbents there too. You got kind of mildly old ones like intercom and talk desk doing it to you. Okay, cool. Or like robotics, like bold take, but there's so much money going into robotics. We're massively far away from having a chat GPT moment. Humanoids in particular really struggle in form factor. I just think the funding and the excitement exceeds the capabilities significantly would be my answer. I like to think I'm somewhat of an optimist and so I I I I think there I think I do think this technology is transformative and so I think there will be ton of opportunity.

嗯，我想我可以回答这个问题，然后我们可以据此反推出我不会投资什么。我喜欢……我我我觉得，呃，推理方面我们已经谈过了。我觉得有一个巨大的……呃，有巨大的……Um I think and so I might answer that and then through that we can back out what I wouldn't invest in. I like I I I think uh inference we already talked about. I think there's a tremendous uh there's tremendous

主持人：开始吧。你、你、你是我们基金里最新加入的合伙人。欢迎你。很高兴你第一天就来加入我们。首先，我们应该进行推理。Host: Let's do this. You're you're you're my newest partner in the fund. Welcome. It's so good to have you on your first day. Number one, we should have inference.

嘉宾：我们应该具备推理能力。嗯，未来会有更多的软件被开发出来，所以我们也应该投资一些更底层的基础设施原语之类的东西。Modal 是一家非常出色的公司。呃，Temporal 也是一家非常出色的公司。我觉得，呃，Guest: We should have inference. Um, there's going to be way more software written, so we should also invest in like lower level kind of like infrastructure primitives. Modal, incredible company. Uh, Temporal, incredible company. I think uh,

主持人：它们与烟花争相媲美。Host: they compete with fireworks.

嘉宾：嗯，模态在某些方面确实如此。是的。Guest: Um, modal does on some things. Yeah.

主持人：好的。Host: Okay.

嘉宾：但他们还做更多的事情。他们还提供沙箱。所以我们有推理层、第一层，也就是编码层。Guest: But they also do more. They do sandboxes. So we've got inference layer one the coding layer

主持人：协作编程，呃，或者我会说推理，然后像基础设施，比如低层基础设施、Temporal、Modal，呃，Supabase、Convex，嗯，我觉得垂直领域智能体会是一个不错的层级，我觉得，嗯……Host: co coding uh or I would say inference and then like infrastructure so like low-level infrastructure temporal modal um uh superbase convex um vertical agents I think vertical agents are are are going to be a good layer I I think um

嘉宾：有什么特定的垂直行业吗？Guest: any specific verticals

主持人：我不确定有没有哪类垂直领域值得我特别看重。我觉得重要的是，要有一种针对特定领域的观点，并且真正深入钻研。我记得 Patrick Collison 最近在一档播客里说过这件事。我想他当时是在和 Omrad（Replet）聊天，谈到接下来会创办什么样的初创公司时，他说大概不会是那种很炫的领域，而是某个技术渗透率可能较低、至少 AI 渗透率较低，但已经积累了大量领域专业知识的行业，然后进入这个行业，真正对它进行改造。我觉得关键在于拥有某种领域专业知识，它可以被沉淀为数据或逻辑，放进框架里，最终或许也能体现在模型权重中。不过我不会从模型权重入手，而是先积累一些可以沉淀在其中的领域专业知识，然后把它用起来。Host: I don't know if there's any kind of like verticals that I would overweight so much. I think the important parts are having like a domain specific kind of like point of view and and really diving deep there. I I think Patrick Collison said this on a recent podcast. I think with Omrad from Replet when he was talking about like what the next uh what startup he would build it's it probably wouldn't be like a sexy area. it would be some area where like technology maybe or at least AI penetration is low but there's a lot of domain expertise built up there and kind of going in there and and really transforming that. I I I think the key the key part is basically just having some domain expertise that can either be captured in data or logic that goes in the harness um and eventually maybe the model weights but but I wouldn't start there having having some domain expertise that can be captured there and and then putting it to work.

嗯，我想顺便说一下，在这些事情里，我可能会关注三点。我觉得我们的 prerex，如果你看看所有这些成功的团队，我认为第一点是，他们的人才实力非常强。比如 Modal，它的人才团队实力强得惊人。Lynn 和 Fireworks，对吧？他们写了 PyTorch，这多酷啊，对吧？嗯，第二点……Um, I think by the way uh there there there are maybe like three things that I would uh look for in all of these and I think our prerex if if you look at all of the uh if you look at all of these teams that that are succeeding one I think like the talent that they have is tremendously strong like like modal has an incredibly strong kind of like talent team. Lynn and fireworks right they wrote pietorch like how cool is that right? Um two

嘉宾：我们见到他们的时候，你知道，就是你跟创始人见面时，上次做简短介绍就像是在说：“我开发了 PyTorch。”Guest: when we met them you know when you're kind of meeting founders the last time you do the brief intro it's like I built PyTorch

主持人：那个，那个行。对，对。这一项可以勾上了。Host: that that works. Yeah. Yeah. Good tick in that box.

嘉宾：所以，我觉得要打造这种特别底层、深入的基础设施，你必须有非常强的技术实力，而能做到这一点的人并不多。第二，你必须行动得非常非常快，能够快速迭代，我认为这一点极其重要。嗯，市场变化实在太快了。我觉得速度和执行速度——说到 LangChain 的优势，我认为我们整合得非常快，这也是我们相较于其他公司的优势之一。最后一点是品牌。我觉得，很多在 AI 领域、AI 时代取得成功的公司，基本上都是偶然抓住了机会，然后乘势而上，把握住了这波浪潮。比如早期的 LangChain，我们迅速走红，并且把这种势头转化成了实际成果。Guest: So so I think to build like especially this really deep infrastructure you have to be like super technically strong and there's just not that many people that that can do that. Two you have to go like really really fast like this this like being able to iterate quickly I think is incredibly important. Um uh the market just changes so rapidly. I think I think speed and execution speed when when I think about kind of like our advantages at lingch chain I I think we integrate really really quickly and I think that's one of the uh advantages we we have over other companies and then the last thing would be brand um I think like a lot of companies that uh uh you know made it in this AI area AI era basically caught lightning in a bottle and then like rode that wave and captured that. You look at like lane chain early on like we went very viral and have executed against that.

Cursor 就是个很好的例子，他们火得一塌糊涂，能够与那个 Jev——呃，甚至是 Claude Code、GPT——抗衡。我觉得这个领域发展得实在太快了，而且竞争者又这么多；也许这恰恰是因为竞争者太多。你知道，LangChain 出来后的两个月里，就有大概一百种不同的框架问世。但我认为，我们积累起了品牌优势，而且我觉得那是一种持久的优势，我们也算是凭借它一路走了过来。所以 Instinct 可能也是另一个例子，此外大概还有一堆个人助理。不过我觉得，如果说到估值，它们可能会高于理性水平；但对于一些正在进行的融资轮次，如果有一家脱颖而出的公司，技术团队非常强，而且已经证明他们能够快速执行，我觉得你就别盯着价格看，直接投资就行。cursor a great example they went very viral execute against that Jev uh even claude code GPT like it seems like there's just because I it seems like the space is moving so fast and because there are so many competitors and maybe this is even because of because there are so many competitors you know there were like a hundred different frameworks that came out in the two months after Lingchain came out but I think we had a brand advantage that was built up and I think that was a durable advantage that we kind of like rode and so instinct another example of is probably a bunch of personal assistance, but like I think if there is a uh like in terms of like it would probably be at like maybe higher valuations than is rational, but in terms of some of these funding rounds that are getting done where there's a a breakout company with like a really strong technical roster that has proven that they can execute quickly, I think you just don't look at the price and you just invest.

在人工智能领域，抢占先机是否比以往任何时候都更加重要？Is Being First More Important Than Ever in AI?

主持人：成为第一名是否比以往任何时候都更加重要？Host: Is being first more important than ever before?

嘉宾：我觉得是，绝对是。是啊，你看，我的意思是，追上来是有可能的。我觉得 OpenAI 在追赶编程领域方面做得非常出色，但 Anthropic 和 Claude Code 抢先一步，呃，哇，那真是巨大的优势。Guest: I think so. Absolutely. Yeah, you look at I mean it's possible to catch up. I think OpenAI does an incredible job of catching up with with codecs but being first uh anthropic and cloud code that wow what what an advantage that was.

主持人：不过，正如你所说，LangChain出现了，几个月后又有越来越多的产品出现，但直觉上都差不多。不过，率先拥有品牌光环所带来的持久品牌垄断，确实非常真实。Host: But to your point you lang chain comes out and then months after more and more come out but instinct same same but actually the durable um brand monopoly that comes from being the halo first is very real.

嘉宾：我觉得这确实非常真实。嗯，我也觉得之后你必须把执行做得特别出色。呃，举个反例来说明我刚才说的话：你只是先行一步，并不意味着就能确保成功。AutoGPT，我不知道你还记不记得 AutoGPT 项目，它是在 LangChain 出现几个月后推出的，当时是有史以来增长最快的 GitHub 项目。嗯，你知道，我……我……呃，我之后就没怎么听到他们的消息了。这不是在贬低他们，想要快速执行确实很难。但我觉得，仅仅因为你在某个领域先行一步，并不意味着你就能……就能活下来。我从来不喜欢……Guest: I think that is is super real. Um I also think that you have to execute incredibly well after that. uh counter example to what I just said like just because you're first doesn't mean that you're guaranteed success. AutoGPT I don't know if you remember the AutoGPT project but it came out uh a few months after Langchain was the fastest growing GitHub project of all time. Um and you know I I I uh I I haven't heard much uh from from them since. And that's not a knock is very hard to kind of like execute with speed. But I think like just because you're you're first in an area doesn't mean that you'll you'll kind of like survive. I never like to

Jev估值90亿美元的看多逻辑The Bull Case for Jev at a $9BN Valuation

批评别人，所以我觉得在某些方面，换成赞美他们会更有帮助。如果你要写一份备忘录，说明我们为什么应该投资 Jev 的 90 亿美元融资轮，你看好 Jev 的理由会是什么？knock people so I find it like helpful to like praise them instead in some respects. If you were to write a memo joining the firm for why we should invest in $9 billion Jev round, what would your bull case be for Jev?

主持人：看好 Jev 的理由是，出现了一种新型推理和一种新型模型，而且这个模型类别的名字就直接以他们命名。所以他们会乘上这股浪潮。嗯，他们是一支非常有才华、能够把这件事做成的团队。嗯，我不太了解这支团队。我觉得，根据我所看到的一切，要以那个估值进行一轮融资，前提大概就是这个。我的看法似乎也是如此，但这就是看好他们的理由：他们基本上开创了一个新的模型类别，与之紧密相关，而且拥有一支有才华、能够把事情做成的团队。Host: The bullcase for Jev is that there's a new type of inference and new type of model and they're like the the model category is literally named after them. And so there's they're going to ride that wave. um and they're uh uh uh they're a very talented team that can execute on that. Um I don't know the team super well. I think that would have to be the assumption for for a round at at that value from everything that I've seen. I I that seems to be the case, but but that would be the bull case. They basically invented a new model category are associated with it and have a talented team that can execute on that.

Benchmark 和红杉究竟如何帮助创始人How Benchmark and Sequoia Actually Help Founders

主持人：从基准资本（Benchmark）拿钱会改变一家公司的发展轨迹吗？Host: Does taking money from benchmark change the trajectory of a company?

嘉宾：百分之百。我觉得，当 Benchmark——Miles，以及整个 Benchmark 合伙团队——投资我们时，团队里只有我和 Ankush。呃，我们当时就两个人。嗯，我们真的把他们当成第三位联合创始人。我觉得，呃，你知道，当我回想公司如何从两个人发展到如今的 400 人时，他们可以说是这一路上最始终如一的合作伙伴。你知道，我不是风投人士，也从来没做过风投。市场上可能还有其他一些因素，比如优秀的 Benchmark 参与了那个 signals 项目，这本身就是一种信号。Benchmark 这一轮融资后大概两三个月，红杉就加入了我们，我们非常幸运。他们也一直是非常出色的合作伙伴。Guest: 100%. And I I I think Benchmark when when Miles and really the whole Benchmark partnership invested in us, it was me and Ankush. Uh we were two people. Um we we really thought of them as kind of like a a third co-founder. I think there are uh you know when when I think about company building from from two people um to where we are now today, which is like 400 people. They've been kind of like uh our our most consistent kind of like partner in that in that journey for sure. You know, I am I am not a VC. I've never done VC. There's probably also other market dynamics where great benchmarks involved in that signals thing. And we're very lucky to have Sequoia join us after like 2 months, 3 months after the benchmark round. And they've been a tremendous partner as well.

主持人：那是 Sequoa 的谁？Host: Who was it from Sequoa?

嘉宾：来自红杉资本的索尼娅，我觉得她是这个领域里消息最灵通的人之一。Guest: Sonia from Sequoia who is one of the most plugged in people in the spaces I think.

主持人：他们的风格有什么不同？Host: How do their styles differ?

嘉宾：所以他们两家都非常、非常聪明敏锐，也有一定的经验，是真正的实战经验。我觉得 Benchmark 可能更像是关注公司的核心，也就是公司的核心本质；他们会围绕这一点为我们出谋划策，也会在这方面提供帮助。我的意思是，我觉得公司建设、定位之类的很多事情，我都会去找他们，他们也会花时间帮忙。Sequoia 的风格，我觉得是方方面面都会帮，他们简直就是一台机器。他们会帮忙招聘个人贡献者和高管，也会帮忙做营销、发布产品，还会真正把各项事务运作起来。我觉得是这样。对。Guest: So they're both very very kind of like intelligent and sharp and and h and have kind of like experience, real experience that they bring. I think benchmark is maybe a little bit more like um what's the uh like what's the core nucleus of of the of the company and like that's like we're we're going to be a sounding board for that and we're we're going to kind of like help with that. So, so what I mean by that is I feel like uh yeah, a lot of the company building things, positioning things I would go uh I would go to them for and they would spend time there. Sequoia's style I think is they help with like everything like they are just a machine. So they help with recruiting of IC's of executives, they help with marketing and launches and they'll really operationalize. I uh I think so. Yeah.

我的意思是，我们有很多候选人都是通过红杉资本的推荐找到的。I mean we've gotten a lot of our candidates from from Sequoia references.

主持人：是的。他们有一个人才网络，我们从那里找到了不少非常非常优秀的候选人。Host: Yeah. They they have a talent network and we've gotten a bunch of really really strong candidates from there.

嘉宾：你会对创始人说，这值得……吗？Guest: Would you say to founders it's worth

为了顶级风险投资机构接受较低估值，值得吗？Is It Worth Taking a Valuation Discount for a Top-Tier VC?

给 Sequoia 打折？taking a discount for Sequoia?

主持人：我觉得，绝对值得为了红杉资本、Benchmark，以及可能还有其他这些顶尖——呃，一级风投公司而接受折价，不过我们合作的就是这些公司。Host: I think absolutely it's worth taking a discard for Sequoia and Benchmark and and probably other of these top one uh uh or tier one VC firms, but those are the ones that we work with.

主持人：关于筹款，你们现在有哪些认识，希望当初只有你们两个人时就能告诉自己？Host: What do you know now about fundraising that you wish you could tell yourself when it was just the two of you?

嘉宾：我觉得融资几乎就像一场游戏。我一开始确实完全不知道这一点。我是第一次创业。我也觉得自己把很多精力都放在产品上，基本上我所有时间都花在产品上，所以我非常专注于这一点。嗯，我确实觉得融资有这么一套玩法：你知道，你想要在不融资的情况下去融资。你想要，呃，一旦拿到一份条款清单，就再拿到一份，然后你知道，告诉其他人你已经拿到一份条款清单了，然后……Guest: I think there is like this game to fundraising almost. And I I I definitely didn't know that going in. I'm a first-time founder. I also think I I I focus a lot like where I spend all my time is on product basically and so I'm very focused on that. Um I do think there is this game to fundraising where you know you you um you want to fundra without fundraising. You want to like uh once you get one term you want to get one term sheet and then you know tell others you have a term sheet and

主持人：在 Twitter 上发布一张营收图表，然后这就带来了所有这些入站线索以及Host: post a revenue graph on Twitter and then it leads to all these inbound and

嘉宾：嗯。Guest: yeah.

主持人：你[ __ ]喜欢这样吗？我[ __ ]讨厌这样。也许我现在年纪太大，不适合玩这种游戏了，但这根本就是在摆架子。就像，哈里森，你是真的喜欢我，想和我一起做生意吗？不管怎样，我都会给你同样的价格和同样的钱。来吧，就这么做。别再[ __ ]摆架子了。Host: Do you [ __ ] like this? I [ __ ] hate it. Maybe I'm too old for this game now, but like it's just posturing. It's like Harrison, do you really like me and want to build a business with me? I'll give you the same price and the same money than whatever. Like let let's just do this. Stop the [ __ ] posturing.

嘉宾：我觉得，嗯，我一点儿也不喜欢融资。我觉得，我的意思是，有点像你刚才说的，我想和我喜欢共事的人一起做出很棒的东西。我觉得，嗯，是啊，考虑到我们的雄心有多大，我们尤其需要融资才能实现目标。嗯，我觉得，你知道，当初我们向 Benchmark 融资时，他们采取的方式大概也是这样：嘿，咱们先看看彼此是否合得来、是否愿意一起共事。顺便说一句，他们的谈判策略非常高明。所以我觉得，他们确实也很擅长谈判。嗯，不过回头看我们最近几轮融资，比如我们的 B 轮，我们是从 IVP 的 Tom 那里融的，Guest: I think um I I don't like fundraising at all. I I I think I mean kind of what you were saying like I I want to build something great with people I enjoy working with. I think like uh yeah, like we we especially for the scale of ambitions that we have, we need to fund raise to to get that done. Um I think uh you know I think that was uh I think that the that was the approach that that Benchmark kind of took when when we were fundraising from them as well like hey let's figure out like you know if if we like each other and want to work together by the way that was very good negotiating by them and so I think they you know uh they're very good negotiators as well. Um, but when we when I look at like our most recent rounds uh that we've done, like we raised from Tom at at IVP, our series B,

主持人：我和 Tom 聊天已经有一段时间了，我想大概是从 Sequoia 那轮融资前后开始的，聊了两年半多。所以我们就是这样慢慢建立起了关系，彼此逐渐了解。当时到了融资的时候，我觉得其中少了些套路，这一点我真的很感激。Host: I I'd been chatting with Tom from like I think right around the Sequoia round for like over like 2 and a half years. And so that was like a relationship that we just built up over over time and I got to know him and he got to know me. And so when it when it came time to to fundraising, I feel like there was like less games in involved there and I really appreciated that.

主持人：我想来一轮快速问答。我说一句简短的话，你立刻说出你的想法。这样可以吗？Host: I would love to do a quick fire around. So, I say a short statement and you give me your immediate thoughts. Does that sound okay?

嘉宾：我们开始吧。Guest: Let's do it.

主持人：好的。你最喜欢的一次风投会面是什么？Host: Okay. What was your single favorite VC meeting?

嘉宾：第一次向 Benchmark 路演。Guest: First pitch with Benchmark.

主持人：为什么？Host: Why?

嘉宾：房间里真是群英荟萃。Guest: So many legends in the room.

主持人：和谁一起玩最酷？Host: Who is the coolest to hang with?

嘉宾：呃，比尔·格利。Guest: Uh, Bill Gurley.

主持人：你当时感到害怕吗？Host: Were you intimidated?

嘉宾：我想我当时太天真了，才会被吓住。Guest: I think I was too naive to be intimidated.

主持人：他也就是个大块头。就像是……Host: He's also just like a big man. like it's

人们对智能体最常见的误解，也是最重要的一点The Most Important Thing People Get Wrong About Agents

这是一个令人望而生畏的身影，笼罩在你头顶。嗯，呃，你对智能体最重要的看法是什么，而整个生态系统中的大多数人都不会认同？it's an intimidating figure looming over you. Um uh what is the single most important thing you believe about agents that most of the ecosystem would disagree with?

嘉宾：构建它们仍然非常难，真的非常非常难。也许大家都会认同这一点，但我觉得人们还是低估了这件事有多难。Guest: It's still really hard to build them like really really hard. Maybe people would agree with that but I think it's just underrated how hard it is.

主持人：过去12个月里，你在哪件事上改变了最大的想法？Host: What have you changed your mind on most in the last 12 months?

嘉宾：推理服务市场规模巨大。我记得大约一年半前，我们和董事会讨论过是否应该做一个网关。当时我们没看出相比 OpenRouter 这样的产品，我们能增加什么价值。我认为那绝对是个错误的决定。Guest: Market for inference is massive. I remember talking with the board about whether we should do a gateway a year and a half ago and we didn't see what value we could add over something like open router and I think that was 100% the wrong decision.

主持人：对你和团队来说，构建过程中最棘手的技术难题是什么？Host: What's been the hardest technical thing for you and the team to build?

嘉宾：我们为存储智能体轨迹构建了自己的数据库，名为 Smith DB。Guest: We built our own database for storing agent traces Smith DB.

主持人：你怎么看待既与 ClickHouse 竞争、又是它的客户？这感觉有点奇怪，还是很正常？我觉得我们和五百多家公司都是既竞争又合作的关系，所以我个人已经很习惯了。Host: How do you think about competing with ClickHouse and being a customer? Kind of weird or normal? I think there are 500 different companies that we are fremies with and so I think I've personally gotten very used to it.

OpenAI估值1.5万亿美元，Anthropic估值2万亿美元，谁更高？OpenAI at $1.5TN vs Anthropic at $2TN

嘉宾：你可以投资一家即将上市的公司：估值1.5万亿美元的 OpenAI，或者估值2万亿美元的 Anthropic。Guest: You can invest in one IPO open AI at 1.5 trillion or anthropic at 2 trillion.

主持人：Open AI。Host: Open AI.

主持人：为什么？Host: Why?

嘉宾：我认为，他们有更多元、各不相同的路径，能够实现非常高的退出估值。Guest: I think they have more diverse different paths to really tremendous exits valuations.

主持人：完全理解。你现在知道了哪些事情，是希望自己在 2022 年刚开始时就知道的？Host: Totally get that. What do you know now that you wish you'd known when you started in 2022?

嘉宾：真希望我能知道如今的安全带是什么样的，这样我们就能早点把它们造出来。Guest: I wish I knew what harnesses today looked like so we could have built them earlier.

主持人：最后一个问题。展望未来5年，你最期待什么？Host: Final one for you. What are you most excited for when you look forward 5 years?

嘉宾：我真的很期待，看看我们能用 AI 做出哪些有创意的事情。我觉得，AI 在很多方面——你知道，在我们做的很多工作中——都是在帮助企业提高效率、交付成果。我觉得像 Sunno 这样的创意应用非常有意思，我希望能看到更多这类东西。Guest: I'm really excited for what we what creative things we can do with AI. I think a lot of AI, you know, in a lot of what we work on is, you know, helping enterprises be more efficient and and delivering outcomes. I think creative things like Sunno are are tremendously interesting and and I wish we'd see more of those.

主持人：完全同意。我记得迈奇上过我们的节目。那期其实特别有意思。嗯，做这期节目也特别开心。哈里森，非常感谢你包容我总是跑偏、时间安排也不太一致，不过你真的是一位非常棒的嘉宾。Host: Totally agree. I remember having Mikey on the show. That was a fun one actually. Um and this has been so much fun to do. Harrison, thank you so much for putting up with my waywardness and divergent schedules, but you've been an amazing guest.

嘉宾：不，还是非常感谢你们邀请我来。Guest: No, thank you so much for having me.