别让模型编造数字｜Kepler 的 Vinoo Ganesh 谈金融 AI 架构

简要描述Brief Description

Vinoo Ganesh 解释了 Kepler 处理财务问题的方法：检索可追溯的来源事实，让确定性代码计算数值，并剔除任何无法验证的生成数字。Vinoo Ganesh explains Kepler’s approach to financial answers: retrieve traceable source facts, let deterministic code compute values, and remove any generated number that cannot be verified.

目录Table of Contents

• 简介
  Intro
• 为什么 LLM 与财务数字格格不入
  Why LLMs and financial numbers don't mix
• 模型引用数字，代码负责计算
  Models cite numbers, code does the math
• 从 SEC 文件中构建引文注册表
  Building a citation registry from SEC filings
• Kepler 如何端到端地回答问题
  How Kepler answers a question end to end
• 组织事实：数值、单位、尺度、期间
  Structuring facts: value, unit, scale, period
• turbopuffer 上的混合搜索与重排序
  Hybrid search and re-ranking on turbopuffer
• 租户隔离与安全性
  Tenant isolation and security
• 让每个数字都可追溯
  Making every number traceable
• 剔除无法验证的内容
  Stripping what can't be verified
• 普通 RAG 的不足之处
  What plain RAG gets wrong
• Kepler 实战展示
  Kepler in action
• 保障的边界
  Where the guarantees end

简介Intro

嗨，感谢你们的到来。我叫 Vinoo Ganesh，是 Kepler 的首席执行官。我的几位队友也在这里。Hey, thanks for coming. My name is Vinoo Ganesh. I'm the CEO of Kepler. We have a few of my teammates here as well.

很高兴能来到这里和大家交流。这次分享最初源于 AIE 世界博览会。当时我们从宏观层面探讨了 Kepler 的整体情况，以及我们是如何开展工作的。嗯，我们目前最引以为傲的成果，还有一大堆新进展，就是 Anthropic 对我们的架构和技术进行了全面深入的研究。世界博览会那场演讲之后，我收到的最大问题就是：那么，你们到底是怎么做到的？I'm excited to be here to chat with you. So, the initial inception of this session comes from the World's Fair for AIE. Where we discussed at a high level Kepler as a whole and how we do what we do. Um and our big claim to fame right now and there's a new bunch of other stuff is Anthropic did a full deep dive into our architecture, our technology, and the biggest question that I got after the World's Fair presentation was well, how do you actually do this?

嗯，我们的合作伙伴和朋友 turbopuffer 把这次分享的机会交给了我，让我来谈谈我们如何将 turbopuffer 用作整个生态系统中的一个组成部分。嗯，顺便打个广告：大约 25 分钟后，Jack 和我会在主舞台上做一场分享。所以我们会尽量快速讲完。如果你想了解 AI 时代的基本面投资，欢迎 25 分钟后过来听。嗯，那么，先简单介绍一下我自己。Um so, our partners and friends at turbopuffer have given me this session to talk about how we use turbopuffer as a component of this overarching ecosystem. Um also shameless plug, Jack and I are doing a talk on the main stage in about 25 minutes. So, we're going to power through this and if you want to hear about fundamental investing in the age of AI, uh feel free to come to that 25 minutes from now. Um so, to talk a little bit about uh so, quick background of myself.

嗯，再介绍一下，我是 Vinoo，Kepler 的 CEO。我职业生涯始于 Palantir，并在那里负责计算层。Um again, I'm Vinoo. I'm the CEO of Kepler. Uh began my career at Palantir. I led the compute layer there.

嗯，后来我创办了自己的数据即服务公司，之后在 Citadel 担任业务工程负责人。嗯，我们在打造 Kepler 时学到的很多东西，都直接来自于我们在相当严苛的环境中积累的经验：在这些环境里，准确性和保真度都至关重要，也因此我们明白了如何构建并利用某种底层平台——或者说，管它是什么时髦的新词。接下来，我们来聊聊 Kepler 的作用。Um then built my own data as a service company and then was head of business engineering at Citadel. Um so, a lot of what we learned here in building uh Kepler comes directly from the experiences of understanding how to build and harness or a substrate or whatever the cool new word is in pretty aggressive environments where accuracy and fidelity matter pretty aggressively. So, let's talk a little bit about what Kepler does.

为什么 LLM 与财务数字格格不入Why LLMs and financial numbers don't mix

嗯，我知道我们嗯已经去过很多地方了。我们也很高兴今天能参与 Gemini Force 的博客文章。呃，他们推出了一个新模型。嗯，核心原则是，作为一名分析师，如果你确实在金融领域工作，就绝不会向客户或你的 PM 展示一个他们无法明确验证或追溯的数字。Um I know we're um we've been in a lot of places. We're excited to be part of Gemini Force um blog post today as well. Uh they've a new model out. Um so, the core tenant is um, an analyst, if you're actually working in finance, you never show a client or your PM a number that they can't explicitly validate or trace.

所以，当你看到“营收增长了 12.4%，达到 4182 亿”这样的内容时，这些都是真实数字，必须准确无误。难点在于，我们所依赖的都是概率机器。大语言模型从根本上说就是概率机器。让它们如此富有创造力、如此出色的特性，也正是让它们难以应用于任务关键型场景的原因。So, when you see something like revenue grew 12.4% to 418.2 billion, these are actual numbers that need to be correct. The challenge is we're all operating off of probability machines. LLMs are at their core probability machines. The same thing that makes them so creative and cool is the same thing that makes them very difficult to use in mission-critical situations.

因此，一个很常见的做法是事后附上来源。这听起来不错，但事后才附上来源会带来一个问题。首先，尽管很多人这么认为，大语言模型实际上并不会阅读你的文档。它们会把许多内容切分成块，进行语义搜索，然后检索出一些可能有一定相关性的信息——运气好的话是这样。So, a natural thing you see is people that attach sources after the fact. Now, this is cool, but here's the problem that happens when you just attach a source after the fact. First, LLMs don't actually read your documents, despite what many people think. What they do is they chunk a bunch of things together, they do a semantic search, and they retrieve information that may be semi-relevant, if you're lucky.

所以，很多人面临的第一个大挑战是，我们找错了信息来源。比如，我们从 Seeking Alpha 或 Venu 随手写的 YOLO 博客上获取信息，那里说 Palantir 每股会值 10 万美元。又或者，我们找对了信息来源，却看错了数字。比如，读 10K 或 10Q 时，你可以想象这样一种情况：我看到数字 17。So, the first big challenge a lot of people have is we have the wrong source. Like, we've pulled information from Seeking Alpha or Venu's random YOLO blog that says Palantir will be worth $100,000 a share. Or, we have the right source and the wrong number. For example, when you read a 10K or a 10Q, you can imagine a world where I see the number 17.

那么，这对应的是什么？是1700万、170亿，是1700万日元，还是1700万美元？这是这家公司2023年的收入，在2025年的10-K文件中记为1700万吗？还是说17是页码？Well, what does that correspond to? Is it 17 million, 17 billion, is it 17 million yen, 17 million dollars? Is it 17 million was the revenue of this company in the year 2023 in the 10K that's in 2025? Or, is 17 the page number?

比如，所有这些都是我们在处理任何需要可验证性或数值准确性的环境时会考虑的因素。最后，还有心算这一点。增长率、利润率、倍数这些计算，模型实际上并没有真正进行。现在，在某些情况下，它们会编写代码来进行数学运算，但模型本身并没有以能确保忠实度和准确性的方式进行数学运算。Like, all of these things are considerations that occur when we're actually working on any kind of environment that requires verifiability or numerical accuracy. Finally, there's a piece of mental math. Growth rates, margins, multiples, models are not actually doing Now, in some cases, they write code that does do math, but models themselves are not actually doing math in a way that ensures fidelity and accuracy.

模型引用数字，代码负责计算。Models cite numbers, code does the math

因此，我们最终需要开发出一种架构：模型引用数字，但绝不编写数字。Kepler 的核心及其背后的魔力在于检索——找到正确的来源，感谢 turbopuffer，稍后我们会谈到它。数学运算由代码完成。这里说的代码，意思是模型不编写数学运算，而是使用我们人类预先编写的确定性代码。我们还有一个后处理过滤器。So, what we end up having to develop is an architecture where the model cites the numbers, but never writes them. At its core and the magic that powers Kepler is retrieval finds the right source, thank you turbopuffer, which we'll talk about. Code does the math, and code meaning the model is not writing the math, but predetermined a deterministic code that we humans have written. And we have a post-processing filter.

那么，我们来逐步分析一下。So, let's break this down.

从 SEC 文件中构建引文注册表Building a citation registry from SEC filings

所以，我们处理了整个 SEC 数据语料库，总计超过三千万份文档。嗯，我们的朋友 turbopuffer 希望很快就能和我们一起，研究如何把这些内容呈现给大家。嗯，不过我们先做了一步预处理，并登记了我们称为“引用”的内容。So, what we did is we processed the entire corpus of SEC data. Now, that's 30 plus million documents. Um our friends at turbopuffer will be hopefully working with us fairly soon on how we actually surface this to all of you. Um but we ran a pre-processing step, and we registered what we're calling citations.

因此，引用实际上就是我们从申报文件中提取出的确定性数值，例如营收数据、利润表、资产负债表，以及管理层讨论与分析（MD&A）中用人工撰写的文字所表述的内容，比如：“嘿，我的营收增长了 37%。”这类内容大约有 20,000 条。我们已将三项单独的申报事实公开为可引用的锚点，这实际上意味着，我们能否告诉你某个数字的出处，取决于能否点击该数字，并追溯到它所出自的核心文件。在生产环境中，对于无法核实的引用，我们会直接拒绝。And so, citations are effectively the deterministic numerical numbers that we have pulled out of filings. Things like revenue numbers, things like an income statement, a balance sheet, things like MD&A that actually says, "Hey, my revenue grew 37% in human-written text." There's about 20,000 of those. There's three individual filing facts that we've revealed as citable anchors, which effectively means our ability to tell you where a number came from comes down to the ability to be able to click on a number and actually reference back a core document where that came from. When we're running in production, we actually refuse citations that we can't verify.

也就是说，如果我有一个没有被引用的数字，那么模型在结构上就无法生成这种输出。最后，过滤器正确提取出的引用数量大约是 21,000 条。我稍后会更详细地讲讲这个。关于 Kepler 的工作原理以及 Anthropic 的相关内容，如果你想了解更多，可以搜索“Kepler Anthropic”；你可以读到语言模型本身是如何进行规划的。Meaning, if I have a number that's not cited, the model is structurally incapable of producing this output. Finally, uh the number of citations that the filter has correctly extracted, we got about 21,000. I'll I'll through this in a little bit more detail. How Kepler works and the piece and anthropic, if you want more information Google Kepler anthropic, you can read about is the language models themselves are planning.

确定性软件处理每一个数字，甚至更广泛地说，处理每一条事实。Deterministic software handles every number or even more generally every single fact.

Kepler 如何端到端地回答一个问题How Kepler answers a question end to end

那么，我们来逐步分析。用户提出问题后，LLM 会根据对工具的全面了解，制定一套计划，其中包括确定性的检索步骤、呃，引文筛选器、边界框，以及这一整套流程。现在，我们来具体说明。So, let's break this down. A user asks the question. The LLM then creates a plan with full knowledge of tooling, deterministic retrieval steps, uh citation filter, bounding boxes, and this whole thing. Now, let's actually make this specific.

假设我在问：“嗯，Google 的调整后 EBITDA 是多少？”想想要回答这个问题需要做些什么。我得先从多份 10-K 文件中提取一堆事实，而且这些文件可能涉及不同年份。还得弄清楚，嗯，你知道，在投资者演示材料和财报电话会议中披露了哪些新信息，或者是否发布了重述，以至于我现在必须实际更新一个数字。模型会推演的就是这些。Let's say I'm saying, "What is the adjusted EBITDA of, you know, Google?" Think about what's involved in getting answering that question. I have to first retrieve a bunch of facts across 10-Ks, arguably over potentially different years. I have to figure out, uh you know, in an investor presentation and an earnings call, what new information was disclosed or if a restatement came out and now I have to actually update a number. All of that is what the model thinks through.

从这里开始，我们有一些工具可以实际检索、计算、构建表格，并将整个基础设施注册到引文注册表中。这个引文注册表不仅能让我们为数字或事实提供数值大小或相关元数据，还能为这些事实标注类型。举个简单的例子，假设我查看 Palantir 的 2025 年 10-K 年报。From there, we have tools that actually retrieve, compute, build tables, and register this entire infrastructure into a citation registry. That citation registry is what allows us to not only give magnitude or metadata associated with numbers or facts, but type those facts. So, a simple example here. Let's say I do look at Palantir's 2025 10-K.

里面写着 Palantir 2023 年的收入是，你知道，就说是十亿美元吧。我刚才提到的所有数字都需要结合背景来理解。虽然这十亿美元出现在 2025 年的 10-K 文件中，但仍必须标注并作为 2023 年的数据点处理。我说的是十亿美元。In there, it says Palantir's 2023 revenue was, you know, let's say a billion dollars. All of those numbers that I just said need to be contextualized. That billion dollars, even though it appears in the 10-K from 2025, has to be annotated and treated as 2023's data point. I said billion.

嗯，如果你看过 10-K 或 10-Q，里面通常会写成 1。数十亿这个数字则在某个表格里，远远地飘在某处。所以，我们必须设法处理这些复杂对象，并为它们添加注释，才能获得有意义、可追溯且确实正确的输出。现在，事后看来，模型就喜欢说些这种话。Well, if you looked at a 10-K or a 10-Q, it usually says one. The billions is up in the table somewhere far away floating around. So, we have to somehow take these complex objects and actually give them the annotations to be able to pursue meaningful and traceable output that's actually correct. Now, after the fact, models like saying stuff.

这就是他们的做法。那么，当模型实际编造了一个数字，而我们无法确定性地证明它来自可信来源时，会发生什么？我们必须设置一个后处理流程——一个后处理步骤——来真正剔除这条信息。我们就是这么做的，并以我敢说（而且我认为确实如此）市场上任何工具中最细粒度的形式，为你提供完整的可追溯性。That is what they do. So, what happens when a model actually invents a number that we cannot deterministically prove came from a trusted source? We have to have a post process a post processing step to actually strip this information out. And so, we do this and we give you the full traceability in the most I'm going to claim this and I think it's true in the most granular form of any tool on the market.

这意味着你可以点击 Kepler 平台上的任何内容，我会向你展示截图，证明这一架构在全球范围内确实可行。Which means you can click on everything in the Kepler platform and I'll show you screenshots that actually allow us to prove this architecture works globally.

组织事实：数值、单位、尺度、期间Structuring facts: value, unit, scale, period

那么，呃，我们来聊聊这在 turbopuffer 领域里是如何运作的。turbopuffer 很友好地为我们提供了一个舞台，让我们进行这场对话。首先，我们来看一份 10-K。假设我们看到总净销售额。416,000 美元这个事实，是个不错的点，对吧？So, um let's talk a little bit about how this works in the realm of turbopuffer who's uh been nice enough to give us a stage to have this conversation. First and foremost, let's look at a 10K. Let's say we see total net sales. That fact that $416,000 It's a good point, right?

我们不知道这意味着什么。这个 416,000 美元的数字，嗯，需要有相关事实信息。比如数值、单位、尺度、时间段。现在，问题来了。We don't know what that means. That $416,000 number um needs to have facts associated with it. Things like a value, a unit, a scale, a period. Now, here's the problem.

对于在这个领域工作的人来说，大家都可以说 XBRL 能解决所有这些问题，我们不必担心从文档中提取信息。如果 XBRL 不是大约有 10% 的时候会出错，这话倒也没错。但更重要的是，做投资决策时，有多少人只看公开信息？For those of you who work in this space, everyone can say XBRL solves all this issue. We don't need to worry about pulling information out of documents. That's true if XBRL weren't incorrect like some 10% of the time. But even more so, how many people are just looking at public information when they're making investment decisions?

数量相当少。他们有自己的 One Drive、自己的 BIP 同步，以及自己整理的各种投资备忘录，还有大量内部信息。因此，要让我们的架构发挥作用，就不能只处理 10-K 和 10-Q 文件，还必须基于你们的内部数据，包括 SharePoint、Box、Ignite、One Drive 和 Outlook。因此，我们会在数据已有结构的地方对其进行整合，并在数据没有结构的地方实现搜索。Fairly few. They have their own One Drive, their own BIP sync, their own various investment memos that they've put together and a slew of internal information. So, for our architecture to work, we have to work not just across 10Ks and 10Qs, but also on top of your internal data, your SharePoint, your Box, your Ignite, your One Drive, your Outlook. So, we structure the data where the data structured and then we implement search where it isn't.

这使我们能够在 turbopuffer 中创建一个命名空间，将 BM25 和混合搜索与 Kepler 平台的完整保障真正结合起来。And what that allows us to do is create a namespace in turbopuffer that actually combines both the BM25 and hybrid search with the full guarantees of the Kepler platform.

在 turbopuffer 上进行混合搜索和重排序Hybrid search and re-ranking on turbopuffer

我们就是这样做的。这大概就是我知道大家会拍照的那张幻灯片。这是神奇的一页。我不担心透露这些，因为这是个非常难的问题，而且变化很快。Here's how we do it. And this is probably the slide I know people are taking photos. This is the magical slide. I'm not worried about revealing this because this is a very hard problem and it moves quickly.

所以我们使用零熵，因此使用零嵌入。这里我们使用 1280 维。ANN（近似最近邻）加上 BM25，就是我们实际跨越整个命名空间检索信息的方式。看着大家都在拍照，真是太棒了。这就是我们实际跨越整个命名空间检索这些信息的方式。So we use zero entropy so zero embeddings we use 1280 dimensions here. ANN is approximate nearest neighbors plus BM25 is how we actually grab information across I look at everyone's taking photos. This is amazing. This is how we actually grab this information across our namespace.

我们确实会进行互惠重排序，以确保实际检索回正确的、通常是 60K 或 60 份文档，然后再进行另一种形式的重排序。这是一种可信的重排序方式，我们会检索回比其他来源更值得信任的来源。比如，不同来源可能会提供不同的信息片段，却都声称说的是同一件事。财报电话会议的文字记录中，演讲者说“我的营收是这个数”，但这与 10-K 文件中的内容不一致——这其实不是漏洞，而是特性。We do do reciprocal re-rankings to actually make sure that we pull the right usually 60K or 60 documents back and then we do another form of re-ranking. And this is a trusted form of re-ranking where we pull back the sources that we actually trust more than others. For example, different sources will have different pieces of information that claim to be the same thing. An earnings transcript an earnings call transcript where a presenter says my revenue was this that disagrees with exists in the 10K is actually a it's not a bug it's a feature.

我必须在这些内容之间实际协调两条不同的信息，所以你也需要注意这一点。我们使用交叉编码器进行重排序，然后将结果返回给模型。这个流程中的这一步基于两件事。There's two different pieces of information I have to actually deconflict across these. So you need to be aware of that as well. We use a cross encoder to actually do the re-ranking and then we return that result to the model. This step in this process is predicated on two things.

这依赖于我们能够在有新信息输入时，以极低的成本重新生成嵌入。生成嵌入并不是一件便宜的事。如果你能做到，而且效率很高，那很好；但如果你要托管自己的嵌入模型，这就会变得——除非你托管的是自己的嵌入模型——非常昂贵，也非常困难。所以这里的策略是先进行广泛搜索，然后重新排序。It's predicated on one our ability to rerun embeddings as new information comes in incredibly cheaply. So generating embeddings is not a cheap process. If you can do it and you can do it efficiently great but if you're hosting your own embeddings model it becomes unless you're hosting your own embeddings model it becomes really expensive and very difficult. So strategy here is we search wide then we re-rank.

租户隔离与安全性Tenant isolation and security

现在，租户隔离也变得非常重要。当我们实际访问任何类型的文档时，有很多重要因素需要考虑。例如，如果 Citadel Desk 1 看到了 Citadel Desk 2 的研究成果，那就很糟糕了，对吧？因此，我们也必须在文档级别内置安全机制。Now there's tenant isolation that also becomes really important. Now, when we actually hit any kind of a document, we have a bunch of different pieces that are important. For example, it would be very unfortunate if Citadel Desk 1 saw Citadel Desk 2's research, right? So, we have to have security built into the document level as well.

所以，重——我们做的第一件事是，如果从那个指向文档的中间查询开始，那么每条命中记录都会在 Postgres 中重新授权，也就是说，在每一次计算中，我们都会检查引用、值和文档。每个构建器都会添加一个新的筛选条件。随着时间推移，我们会逐步构建起这些复杂的筛选条件，从而确保在查询信息和数据时，不会无意间在众多不同来源之间泄露数据。只有 Auth 本身能够创建受限范围的令牌，来实际创建这些信息。So, re- The first thing we do is if you start at that middle query that goes to the document, um every hit is reauthorized in Postgres, meaning we check the citation, the value, the document across every one of these computations. Every builder adds a new filter. So, we're building up these complex filters over time that actually allow us to make sure that as we're querying information and querying data, we're not leaking data inadvertently across a bunch of different sources. Only Auth itself can create a scoped token to actually create this information.

当然，用户和 Aura 都来自经过签名的设备。因此，我们正是通过这种方式确保安全，并确保你的 Alpha 留在贵公司的生态系统内。在如今人人都能使用 LLM 的世界里，Alpha 是个有趣的话题，我们也可以聊聊。不错。And of course, the user and aura come from a signed device. So, this is the way that we ensure security, and this is the way that we ensure your alpha stays within your firm's ecosystem. Alpha's an interesting question which we can talk about in a world where everyone has access to LLMs as well. Cool.

让每个数字都可追溯Making every number traceable

那么，我们如何确保一个数字能够出现在 Kepler 平台上？首先，我们会为平台中的每条事实分配唯一标识符。也就是说，我们需要能够进行实体识别和实体消歧的标识符。每个源 ID 都对应着存储在确定性引擎中的某条事实的一组元数据。Now, how do we make sure a number can appear in the Kepler platform? First and foremost, we assign unique identifiers to each fact in our platform. Meaning, we need to have identifiers that both can do entity recognition and entity resolution. Each one of these source IDs corresponds to a bunch of metadata in a fact stored in a in a deterministic engine.

为了以可定制的方式向您展示输出结果，我们需要一个显示值。因此，如果您希望整个 Kepler 平台都以千美元、日元或其他单位显示，我们就需要能够对结果进行后处理，并向您展示实际输出。我们还必须向您说明结果是如何得出的，以及数据来源于何处。因此，在 Kepler 中有一项不变原则：您可以点击平台上的每个数字、每项事实，查看其实际来源。In order for us to show you the output in a way that's customizable, we need to have a display value. So, if you want your whole Kepler platform in thousands of dollars or in yen or in any of these, we need to be able to post process and show you the actual output. And we have to show you how we got it and where we got it from. So, the invariant in Kepler, you can click on every number, every fact in the platform and see the actual source that we got it from.

这就是我们能够为你提供真正可引用、具有确定性且可复现的数据的方式。这也是我们能够向用户提供的幂等性保证。有了这项保证，我们就能在任何地方反复运行同一个查询，并得到相同的结果。那么，接下来就是我们呃必须处理的问题。And that's how we actually give you numbers that are truly citable, deterministic, and reproducible. This is also the idempotent guarantee that we're able to give to people. By having this guarantee, we can run the same query over and over again wherever and get the same results. So, here is what we uh have to deal with.

当我们跑完整个数据语料时，嗯，我们所有引用中有三分之一涉及模型没有做过的计算。这意味着要把这些引用逐一提取出来，无论它们来自网页搜索结果、电子表格，还是申报文件的附件。模型本身也可能从自己的语料库或训练集中编造出许多内容，这些内容真的很难处理，尤其是在涉及历史数据时。比如，查找 2005 年的某个营收数字，这个数字很可能就在模型的训练语料中。但如果我们从那个语料库中检索它，就无法保证它确实是从正确的年份和口径中提取出来的，也无法确保我们能追溯到它的来源。When we ran this whole corpus of data, um a third of all of our citations are the math that the model didn't do. What that means is pulling every one of these citations out, whether it's web results, spreadsheets, whether it's coming from a filing exhibit, the model itself can make up a bunch of things from its own corpus or its own um training set that are really, really difficult to work around, especially when you go to historical data. For example, looking up a revenue number from 2005 likely lives within the model's training corpus. But it if we retrieve it from that corpus, we can't guarantee that it was pulled correctly from the right year and right scope, and that we can actually trace it back.

所以，粗略来看，其中三分之一是数学内容；也就是说，所有引用中有三分之一涉及模型没有做过的数学计算。那么，我们实际上该如何实时完成这件事呢？So, just off of a whole, a third of this are math that the model that a third of all citations are the math the models did not do. Now, how do we actually do this live?

去掉无法验证的内容Stripping what can't be verified

随着 token 向前推进，就像我们在进行推理一样，我们会把这些筛选条件和引用交给 LLM，让它实际解析它们。可以把这理解为 LLM 对指针进行解引用。接着，我们有一个注册表，可以在其中查找源本身，并从注册表中提取信息，为你提供可追溯性。这里真正强大的一点是，当找不到匹配项时，我们会去掉那个数字。As tokens move forward as we're flowing kind of like we're doing inference, um we give these filters and these references to an LLM that has to actually resolve these. Think of this as the LLM dereferences the pointer. We then have a registry where the sources themselves are looked up, and we pull information out of that registry to give you the traceability. Now, the really powerful thing here is when we don't have a match, we strip that number.

这么做实在太激进了。这意味着，你原本以为 LLM 会是一个很棒的思考伙伴，能发挥想象、进行推测和展望，但它不会成为那样的伙伴。也就是说，就目前而言，我可以这么说：如果你依赖模型幻觉，Kepler 就不是特别有用的工具。换句话说，如果你想发挥创意，或者想写营销内容之类的东西，Kepler 就不是合适的产品。That's a very aggressive thing to do. It means your LLM will not be that wonderful thought partner that you thought it would be, where it would make up, speculate, and project. This means at this moment, and I'll say this, Kepler's not a super useful tool if you rely on model hallucinations. Meaning if you're trying to be creative or you're trying to I don't know, write marketing content, Kepler's not the right product to use.

这意味着，在金融领域，某些行业（比如银行业）靠抓住机会推销产品，可能并不依赖数字的准确性，因为你的工作是销售产品，而不是根据准确、可追溯的信息构建 DCF。因此，通过对照我们的信息来源，我们实际上为任何数据语料库都构建了一个真实依据检索引擎，既涵盖公开股票数据，也涵盖你们自己的内部数据。不过，这挺烦人的。That means there are certain industries in finance where opportunistically selling, like banking, probably don't rely on numerical accuracy because your job is to sell an offering. It's not to create a DCF that's backed by accurate traceable information. So, by checking against our sources, we've effectively created a ground truth retrieval engine across any corpus of data, including public equities as well as your own internal data. Now, this is annoying.

这意味着，这个平台不可能像许多人期望的那样神奇强大。那么，我们究竟该如何判断是应该剥除引用，还是尝试修复引用？这里的核心要点是，我们有一个后处理步骤，专门尝试修复引用。这实际上意味着，如果我们发现某些信息无法可靠地追溯来源，是否应该尝试修复它，并即时通过智能体推断来判定这些信息来自哪个来源？This means the platform cannot be as magically powerful as many people want it to be. So, how do we actually ensure whether we should strip a citation or whether we should try to repair it? And so, the core insight here is we have a post-processing step that actually tries to do a citation repair. Which effectively means if we find information that we can't reliably trace, should we try to repair that and do an on-the-fly agentic determination that says this information came from this source?

答案是，在某些领域我们确实会这么做。这可能是这场演示中第二个我们可以稍微说得含糊一点的部分，也就是 Kepler 如何维护自己的知识产权。很好。The answer is in certain spaces we actually do do that. And that's probably the second piece of this presentation that we can keep a little bit, um, kind of like wishy-washy, which is how Kepler maintains its IP. Cool.

普通 RAG 的错误之处What plain RAG gets wrong

嗯，现在，当我们实际运行这些查询时，有一些数字我觉得值得分享。因为如今这个行业里的大多数人，都会用 LLM 来处理大量金融研究，然后说：“太好了，只要资料都经过恰当的 RAG 处理，一切就没问题，我就可以放心用了。”所以，我们的筛选器在原生环境下——就像在 Claude 或 ChatGPT 上运行一样——从评估中拒绝了 10,981 条引用。也就是说，大约有 10,000 条；我想，其中有 3,810 条直接就是错误的数字。如果你从事金融分析，这绝对是最可怕的事情。Um, now, when we actually run these queries, there are some numbers that I think are worthwhile to share. Because most people in this industry right now are wrapping a lot of financial research with an LLM and they're saying, "Great, everything is right as long as things are, you know, ragged properly, I should be good to go." So, our filter across native, just like running it on Claude or ChatGPT, um, refused 10,981 citations from our evals. Meaning 10,000 about, I guess, 3,810 of these were just straight-up wrong numbers. This is the scariest thing ever if you live in financial analytics.

这意味着，仅仅把 Claude，呃，呃，或者 OpenAI，或者这些模型接入一个原生 RAG 工具或向量数据库，实际上是不够的。如果你只关注语义搜索，那确实有很多很酷的事情可以做。但如果你要处理任何要求数值准确性和可审计性的场景，就不能信任常规的 RAG 平台。这对很多实际上只是 ChatGPT 套壳产品的初创公司来说，是件挺可怕的事。This means hooking up Claude to uh, uh or open AI or any of these models to just a native rag tool or a vector database actually isn't enough. Now, there's really cool things that you can do if you're looking at just semantic search and that's it. But if you're looking at anything that requires numerical accuracy and auditability, you actually can't trust regular rag platforms. This is a scary thing for a lot of startups that are effectively chat GPT wrappers.

我这么说并不是在表达负面意思。这就是我们目前所处的现实。因此，必须出现一种新型架构，将可审计性和可追溯性与确定性要素结合起来，当然也包括数值准确性。嗯，模型不会做数学。I'm not saying that in a negative way. It's the reality of where we are right now. So, a new kind of architecture has to emerge and that architecture has to combine both the auditability and traceability with the deterministic elements of course like numerical accuracy. Um models don't do math.

这其实很简单，我认为这里的见解相当关键。呃，任何时候，只要我们遇到某种情况，模型被迫做数学计算，或者无法推理某件事，我们实际上就让那件事变得不可能。通过增加这个后处理步骤，模型实际上无法写出自己无法引用的内容。也就是说，模型在结构上就不具备这种能力——我会谨慎地这么说。This is pretty simple and I think it's actually pretty critical um insight here. Uh anytime that we see some kind of situation where the model is uh forced to do math or if it can't reason through something, we effectively make that impossible. By having that post post processing step, the model can't actually write something it can't cite. Meaning the model is structurally incapable and I'll be careful.

该平台在结构上不可能生成虚构数字。模型仍然可能产生幻觉，而且它确实会这样做。The platform is structurally incapable of producing hallucinated numbers. The model can still hallucinate and it definitively does.

Kepler 实战展示Kepler in action

好。我们来简单聊聊 Kepler 是什么样的。这只是我们的平台所能做到的一小部分：展示确定性的可追溯性，以及诸如此类的功能。嗯，如果有用的话，我很乐意进一步聊聊。比如，像“苹果 2025 年的营收是多少，增长有多快”这样一个简单的问题，所有数字都可以点击，这意味着点击这些数字后，你不仅能看到它们来自哪份 10-K、10-Q、8-K 或投资者演示文稿，还能看到背后实际进行的计算过程。Cool. Let's talk a little bit about what Kepler looks like and this is a tiny sliver of what our platform does when we show the deterministic traceability and all that. Um happy to chat through uh this as if it's if it's useful. But a simple question like Apple's 2025 revenue and how fast it grew, every number being clickable means by clicking on these, you see not only the 10-K or the 10-Q or the 8-K or the investor presentation that it came from, but you see the actual calculations that are done behind the scenes.

像是我算出了这两个数字，得到了6.4%的增长，这些都会在平台上呈现出来，并且可以追溯。因此，我认为这种架构会成为标准。对于开发金融研究工具或在 AI 领域开展工作的人来说，眼下的机会受到限制，因为目前缺乏真正支持你想要执行的工作流程的架构。现在你可以搭建全托管智能体，让它们根据生成的信息提出一些想法，但你仍然无法生成 DCF（贴现现金流）模型。这就是为什么我们附近仍有分析师，或者现在仍有分析师在忙着从 S-1 文件中提取信息，再把 PDF 里的信息整理到 Excel 中。Things like I did these two numbers and I have a 6.4% um increase all becomes present and traceable on the platform. So, this architecture I think will become the standard. Um, the opportunity in front of people that are building financial research tools or operating in the realm of AI is right now limited by the lack of architecture that actually supports the workflows you want to do. You can set up fully managed agents right now that try to ask for ideas based on, you know, generated information, but you still can't produce a DCF, which is why we still have analysts down the street from here or working right now on stripping information out of S-1s and taking information from a PDF and putting it into Excel.

我们的目标是结束这种状况。Our goal is to end that.

保障的边界Where the guarantees end

所以，嗯，我确实想以一件非常重要的事情作为结尾：我们的保证也是有边界的。我们让模型引用数字，而不是编造数字。我们会核实每一条引用。嗯，我们把算术运算交给计算层，而不是模型层。And so, um, I do want to end on a kind of one very important thing, which is our guarantees do end somewhere. We make models cite numbers, not write them. We verify every citation. Uh, we push arithmetic to the compute layer, not to the model layer.

我们会隔离各个租户，并衡量召回率。这意味着，我们实际上可以保证，你在平台上开展的真实研究独特而有力。这并不意味着我们现在就能打造一家全面取代所有人的对冲基金。接下来大概30秒后我们会聊到这个话题，然后再谈下一个地方。不过，我看到的趋势是，大家会更加注重量化基本面分析。We isolate tenants and we measure recall. This means we can actually guarantee that the ground truth research that you do in your platform is unique and powerful. It does not mean that we can create a full-scale hedge fund that's replacing everyone, at least right now. And so, I see the move, and we'll talk about this in, I guess, 30 seconds as we go, uh, to the next place, but, um, the move that I'm seeing is folks will be more quantamental.

这就是未来。这就是机遇。我们越能将独特的检索机制与确定性检索结合起来，嗯，最终就越能开创一个非常强大的局面，让 AI 终于能替我们做那些我们不想做的事，而不是坐在那里做我们真正想做的创意构思。接下来还有一个附录，嗯，不过非常感谢大家。This is the future. This is the opportunity. The more that we can use unique retrieval mechanisms in conjunction with, of course, deterministic retrieval, um, we'll end up in a really powerful place where AI can finally do the stuff that we don't want to do instead of sitting there and doing the ideating that we do want to do. With that, um, there's an appendix here, but thank you so much.

呃，谢谢你抽出时间。Uh, I appreciate the time.