Al Chen：用可验证的工程证据缩短大型银行 AI 部署审查

简要说明Brief Description

无需等待 12 个月的合规审批，即将 AI 部署到 5 家大型银行——思科的 Al ChenShipping AI Into 5 Major Banks Without a 12-Month Compliance Wait — Al Chen， Cisco

目录Table of Contents

• 简介
  Intro
• 银行会核查什么
  What banks check
• 使用代理修复 CVE 漏洞
  Fixing CVEs with agents
• 验证备份与恢复
  Proving backup and restore
• 通过 LLM 评审器泄露的个人身份信息 (PII)
  PII leaking through LLM judges
• 将客户问题转化为工程问题
  Making customer problems engineering problems
• 要点
  Takeaways

简介Intro

呃，我上十年级时，有一位数学老师，赫夫先生，他出的数学考试出了名地难。他会让我们把答案写到小数点后第四位，以防你碰巧靠运气猜对答案。实际上，这道题的大部分分数都来自你写下的证明，用来说明你是如何得出答案的。Uh when I was in the 10th grade, I had a math teacher, Mr. Huff, who was uh notoriously brutal on our math tests. He would uh make us write our answers down to the uh fourth decimal place in case you happen to get to the answer by luck or by chance. And most of the credit for the question was actually from the proof that you wrote to show how you got to the answer.

当我刚开始使用 AI 可观测性软件时，呃，我意识到，我每天合作的这些合规和风险团队大多有点像哈夫先生，只不过他们拥有数百万美元的合规预算。我还想告诉你，在银行内部部署 AI 智能体需要很长很长的时间。And when I first started playing AI observability software, uh I realized that most of these compliance and risk teams that I work with every day, they're kind of like Mr. Huff, but with a multi-million dollar compliance budget. And I would let you know that playing agents AI within banks takes a long long time.

原因在于“生产”这个关键词。我说的不是开发或预发布环境中的原型，也不是那种用于提升内部员工生产力的内部 RAG 智能体。And the reason is because that keyword production. I'm not talking about prototypes in dev or staging or even like internal rag agents to increase, you know, internal employee productivity.

我说的是数百万银行客户每天都会使用的应用程序。也就是你登录银行网站或移动应用时使用的那些应用。而这些才是真正重要的使用场景。在我的团队，我们身处一线，负责将我们的软件以及工程流程部署到全球最大的银行。I'm talking about the applications that these banks millions of customers use every single day. The same ones you use when you log into your bank's website or mobile app. And those are the use cases that matter. In my team, we're on the front lines of deploying not only our software, but also engineering processes into the largest banks in the world.

我想分享一些客户向我们提出的问题，以及我们采用的策略，帮助他们将投入生产所需的时间从一年缩短到几周。那么，我们先来看看标准 SaaS AI 在银行是如何部署的。先剧透一下：行不通。And I want to share some of the issues that our customers bring to us and the strategies we use to help reduce the time to production from a year down to a couple weeks. So, let's take first take a look to see how standard SaaS AI gets deployed at a bank. Uh spoiler alert, it doesn't work.

存在向互联网的出站流量，敏感信息可能会因此泄露。在隔离网络的银行内部部署本地 AI 时，流程会稍微复杂一些。There's outbound egress to the internet and sensitive information can potentially be leaked out. Within when you deploy on prem AI at an air-gapped bank, the flow is a little more complicated.

银行会核查什么What banks check

今天我将介绍三个支柱。第一是实际的构建流水线：银行如何将我们软件中的镜像拉取到自己的集群中。如果某个区域在请求处理中途发生故障，数据弹性方案是什么？最后，也是最重要的部分之一：如何确保敏感信息不会泄露给银行内部员工，也不会流出银行的 VPC。I'm going to walk through three pillars today. Number one is the actual build pipeline. How our banks pull images from our software into their cluster. If a region fails mid-request, what is the data resiliency plan? And then finally, one of the most important parts is how do you ensure no sensitive information leaks out to internal bank employees and outside of the bank's VPC.

合规速度与 CVE 密不可分。任何称得上合格的平台，都会制定计划来应对这第一道合规门槛，也就是常见漏洞与暴露（CVE）。接下来，我会带你了解这个流程为何如此复杂。首先，你可以针对某个 CVE，查阅 NVD、CISA 或 MITRE 网站上的资料。Compliance velocity uh and CVEs go hand in hand. And no platform worth their salt does not have a plan for how they address this first compliance hurdle, which is common vulnerabilities and exposures. I'm going to walk you through why this process is complicated. So, number one, you can take one CVE and you can research on the NVD, CISA, or MITRE websites.

但实际上，我们的平台——大多数平台都是如此——有多张镜像，而每张镜像都存在多个 CVE。But, in reality, our platform, most platforms, you have multiple images and each image has multiple CVEs.

我不是要为一家银行修复 CVE，而是要为多家银行修复。因此，我们必须围绕这项工作建立一套系统和流程。复杂之处在于，各家银行使用的镜像版本不同，各自面临的 CVE 也各不相同。此外，各家银行使用的扫描工具也不同，这些工具会生成报告，列出存在的 CVE。And I'm not trying to remediate CVEs for one bank, but rather for multiple banks. So, we have to create a system and process around this. And the complication there is that our banks are on different versions of our images, each with their own unique CVEs. On top of that, banks utilize different scanning tools, which output a report on what CVEs are present.

此外，他们还制定了具体政策，规定镜像中实际存在哪些 CVE、严重级别如何，以及修复时限的 SLA 是什么。我们的镜像依赖许多开源项目，而这些项目包含许多带有 CVE 的传递依赖，我们的工程团队无法修复这些问题。因此，我们的团队一直在查看许多这类开源项目的发行说明，因为我们必须向这些银行的安全团队详细解释为什么无法修复这些问题，以及为什么我们依赖这些开源项目的维护者。Furthermore, they have specific policies around what are the actual CVEs in the images, what the severity level is, and then what the SLA is for time to remediation. Our images rely on many open-source projects and those projects contain many transient dependencies which have CVEs that our engineering team cannot remediate. So, our team is we're constantly looking into the release notes for a lot of these open-source projects cuz we have to provide a detailed explanation back to the security teams at these banks on why we cannot remediate and why we're dependent on these open-source project maintainers.

使用智能体修复 CVE 漏洞Fixing CVEs with agents

所以，一旦你确定了团队——也就是工程团队——实际能够修复的 CVE，就有多种修复方式。你可以将软件包升级到更高版本，或者基于基础 Docker 镜像重新构建镜像。不过，我们还会遇到一个复杂情况：有些银行会说，他们希望使用该软件包的 3.8 版本。So, once you've identified the CVEs that your team your engineering team can actually remediate there's multiple ways of remediation. You can bump the package version up to a higher version or rebuild your image against the base Docker image. But, then we also have this complication of certain banks saying we want to be on version 3.8 of the package.

不过，另一家银行可能会说：“等等，先等等。那个版本太高了。CVE 太多了。我想用那个软件包的旧版本。”因此，考虑到所有这些情况，你的软件仍然必须能够正常运行。一个可行的解决方案是建立一套评估标准。我们有这样一套标准，可以承担大量繁重的工作，包括报告和内部研究。But, another bank might say hold wait, hold on. That version is too too high. There's too many CVEs. I want to be on an older version of that package. So, given all of this your software still has to work and function. And so, one solution that you can use is creating a scale. So, we have a scale that does a lot of the heavy heavy lifting and reporting and the internal research as well.

输出采用清晰的表格格式，告知工程团队需要修复哪些 CVE，并向银行的安全团队说明，受这些传递依赖项的影响，哪些问题可以修复、哪些无法修复。And the output is a nice kind of table format that has tells our engineering team what CVEs we have to fix and also provides explanations back to our security teams from the bank on what can we cannot fix due to these transient dependencies.

这些扫描工具中的大多数，比如 Sysdig、Wiz、Prisma，只会检查软件包是否已升级到特定版本。但这完全无法说明那个漏洞利用是否仍然存在于你的应用程序代码中。那么，究竟该如何测试这一点呢？一种解决方案是创建一个 Harbor 任务，这有点像是为你的智能体搭建一个测试框架或环境，让它尝试各种操作。Most of these scanning tools Sysdig, Wiz, Prisma they just check to see if the package has been bumped to a specific version. But, it says absolutely nothing about whether or not that exploit is actually still in your application code. And so, how do you actually test for that? So, one solution is creating a harbor task and this is kind of like creating a harness or a world for your for an agent to try things out.

因此，在这个 Harbor 任务中，你有给智能体的指令、包含应用实际 Docker 镜像的环境，以及可针对智能体运行以检查其是否通过的测试。这里的指令是关于这个服务、这个软件包的一个 CVE。我告诉智能体的指令是：据称将这个软件包升级到 2.6.2 版可以修复该 CVE。And so within this harbor task you have instructions for the agent, you have an environment which contains the actual docker images for your application, and then tests that you can run against the agent to see if they passed. The instruction here is a CVE for uh this service, this package. And the instruction I tell the agent is 2.6.2 of this package supposedly remediates the CVE.

我希望你基于旧版本编写一个漏洞利用程序，看看这个新版本是否仍然存在这个漏洞。然后编写一个测试，检查这个漏洞利用是否确实通过了。接着，你可以用 Harbor 运行一批 Docker 容器，或者说，用 Harbor、Modal 或 Daytona 沙箱并行运行这些任务，这样就能进行多轮试验，看看智能体在不同试验中的表现。I want you to go and write an exploit based on the older version to see if this new version still has that exploit. Then write a test to see if that exploit actually passed or not. And then with harbor you can run a bunch of docker containers or um harbor uh modal or Daytona sandboxes, run these all in parallel, and you have a bunch of trials to see how the agent performed over various trials.

我想说的是，光说我们把软件包版本升级到了某个特定版本还不够。我们必须提供证据，证明这个漏洞利用代码确实已经不在应用程序代码中了。呃，第二个支柱——灾难恢复——对银行来说并非可有可无的基础设施维护事项。And the What I'm trying to say here is that it's not enough just to say we bumped the package version up to a specific number. We have to provide the evidence to show that the exploit is no longer actually in the application code. Uh the second pillar, disaster recovery, is not like optional infra hygiene for a bank.

这里的复杂之处在于，大多数 AI 可观测性工具所使用的主要数据库平台（包括一些赞助本次会议的公司所使用的平台），并没有提供跨区域故障转移和数据复制的托管服务。The complication here is that the main database platform most AI observability tools use, including some of the companies that are sponsoring this conference, there's no managed service for failover across regions and data replication.

有人知道我说的是哪个数据库平台吗？它不是这个……所以，Redis、Postgres 都支持跨区域托管故障转移，但这个数据库不支持。ClickHouse 是一个很棒的数据库平台，但即使在他们的数据弹性文档中，他们也明确指出，没有跨区域托管故障转移服务。Anyone happen to know what database platform I'm talking about? It isn't this So, with Redis, Postgres, there's managed failover across regions, but this one database does not have that story. ClickHouse is an amazing database platform, but even in their data resiliency docs they call out that there's no managed service managed failover across regions.

所以，这意味着银行需要自行想办法解决灾难恢复的问题。一种简单可行的模式是：在主区域部署 ClickHouse，运行一个将数据备份到存储桶的作业，再运行另一个作业，将存储桶中的数据恢复到灾难恢复区域。在该区域内，你可以部署多个 Pod，保存这些数据的副本。So this means the banks are kind of up to themselves to figure out that disaster recovery situation. So a simple pattern that you can employ is you have ClickHouse in a primary region, a job that backs up to a bucket, and then another job that restores that bucket to a DR region. Within that region, you might have multiple pods that have replicas of that data.

整个流程都依赖于 ClickHouse backup，这是一个由 Altinity 维护的开源项目，而 Altinity 本身也是 ClickHouse 在跨区域管理数据故障切换方面的竞争对手。This whole procedure relies on ClickHouse backup, which is an open-source project maintained by Altinity, who they themselves are a competitor to ClickHouse in terms of managing this data failover across regions.

验证备份与恢复Proving backup and restore

嗯，所以通过这个流程，ClickHouse 备份会共享日志，说明备份和恢复是否成功，因此你可以得到这样一张不错的表。不过，此时这个状态并不能说明你的数据情况。磁盘故障、节点故障、迁移出错、复制延迟，这些问题都可能导致你的数据过时，也就是说，数据状态可能与这项任务实际运行时不同。It's uh so with this procedure, they ClickHouse backup shares logs on whether or not the backup and restore happens, so you can get this nice table. But this status says nothing about your data at this point in time. A bad disk, a bad node, um bad migration, replication lag, these are all issues that can cause your data to become stale, that is different from when this job actually ran.

因此，你还可以多做一步，检查表之间的行数，并运行检查，确认同一区域内的副本是否已经彼此收敛。这里的要点是数据正确性，而不只是勾选一项，确认备份和恢复作业运行过。和 CVs 类似，你不能只是升级版本，就以为漏洞已经消失了。So you can go the extra step in checking the row counts between tables, uh running a check to see if the replicas within a region have converged to each other. And the takeaway here is that it's about data correctness, and not necessarily just checking a box and seeing if the backup and restore job ran. Similar to the CVs, you can't just bump the version and assume the exploit's gone.

你们必须进行这些额外检查。还有一点要提醒所有在集群内构建这类备份和恢复机制的架构师：无论你们愿不愿意，你们也在对数据驻留作出决定。You have to do these extra checks. And uh one call out for any of the uh architects who are creating this kind of backup and restore within your cluster is that you're also making a decision around data residency, whether you like it or not.

当你将数据复制到存储桶时，存储桶并不在意它位于俄勒冈州还是弗吉尼亚州。但数据驻留会关注这些单条记录存储在哪里。因此，一种解决方案是只将 ID 以及与你的追踪数据和数据相关的元数据复制到不同区域。When you're replicating to a bucket, the bucket doesn't care if it's in Oregon or Virginia. Data residency, however, cares about where those individual records are stored. So, one solution is you can replicate just the IDs and the metadata around your traces and your data into different regions.

但是，实际的核心追踪数据存储在一个区域内，由于某些司法管辖区的法律规定，不会离开该区域。呃，最后一个支柱是，大多数银行认为自己没有向内部员工或 VPC 之外的用户泄露任何敏感信息，但可能存在敞开的侧门。这种情况表现为 PII 和 MNPI 被遮盖。But, then the actual core trace data, that's stored within one region and doesn't leave that region because of certain jurisdiction laws. Uh the final pillar is most banks think that they are not leaking any sensitive information to their internal employees, their users outside of their VPC, but there may be side doors that are open. And this comes in the form of PII MNPI being redacted.

对此有许多不同的策略。你可以在 SDK 层、网关层或通过各种中间件来对信息进行脱敏。And there's many different strategies for this. You can redact information at the SDK level, at the gateway level, using various middleware.

通过 LLM 评审器泄露 PIIPII leaking through LLM judges

而在实践中，这可能是这样的：在 UI 上，你可能会看到一些客户信息，其中的敏感信息已被隐去。但是，你们使用的评审模型呢？我希望你们所有人都去查看 LLM 评审模型的系统提示，并检查追踪记录，看看这些包含评审模型推理过程的记录中是否含有任何个人身份信息（PII）或重大非公开信息（MNPI）。And what this looks like in practice is on the UI, you might have some customer information, sensitive information is redacted. But, what about the judges that you're using? I challenge all of you to look at the system prompts for your LLM's judges and check the traces to see if those traces with the reasoning from those judges contain any PII or MNPI.

这是一个简单评审器的系统提示词。它会针对给定任务，为智能体的表现打分并说明理由。不过，提示词中有一行乍看之下似乎无伤大雅，因为你希望人工审核者能够核实智能体是否表现出色、是否准确。Here's a system prompt for a simple judge. It outputs a score and explanation on how well the agent performed for a given task. But, there's one line here in the prompt, which is seemingly innocuous because you want the human reviewer to be able to verify if the agent did a good job and was accurate.

这可能会导致裁判泄露并取消对 PII 或 MNPI 的脱敏。因此，你的 UI 看起来依然不错，但在后端，作为裁判的 LLM 在推理过程中可能会对客户信息和敏感信息取消脱敏。This just potentially may have allowed the judge to leak and unredact PII or MNPI. So, your UI still looks good, but then on the back-end your LLM as judges reasoning may un-redact customer information and sensitive information.

因此，一种策略是仍然让你的评审器读取原始追踪数据，但不保留评审器的推理过程。不过，评分是有用的，对评估也很有帮助，因此你希望评分始终通过 UI 持久化。但在另一条分支上，正如我们所见，推理过程仍可能泄露 PII。不过，你可以在应用程序代码中添加检查，告知代理这条记录不包含敏感信息，因此可以持久化到 UI。So, one strategy for this is you can still have your judge read the raw trace data, but the reasoning for that judge is not persisted. The score, however, you want that score it's for good good for evals, that always persists through the UI, but then on this other branch the reasoning, as we saw, can still leak PII. But in your application code, you can have a check that tells the agent this record does not contain sensitive information and therefore can persist to the UI.

这样，你现在就有了该记录的评分和推理过程。如果检查结果为阴性，也就是说其中确实包含敏感信息，那么你只需在 UI 中隐藏这段推理过程。我们合作的大多数银行都希望看到 LLM 作为评审者针对这些敏感信息给出的推理过程。So, now you have a score plus the reasoning for the for this record. And then if that check comes back negative, it does have sensitive information, then you just suppress that reasoning from the UI. Most of the banks we work with, they want to see the LLM as judges reasoning on this sensitive information.

你还可以为这个流程增加一项改进：使用第二个 LLM 作为评判者，并对输入中的敏感信息进行脱敏处理。这样，你就可以确信评判者的输出是干净的，然后将其显示在 UI 中。And the extra upgrade you can add to this flow is you have a second LLM as a judge where the input has is redacted of sensitive information and therefore you can have confidence that the output from the judge is clean and then you can have that show up in the UI.

这里的要点是，实现零泄漏和数据主权并非没有代价。在这个案例中，代价就是针对同一个请求再调用一次 LLM。所以，我们与这些安全和合规团队建立信任的一种方式，是不只是说“相信我们，这是答案”。The uh takeaway here is that having zero leakage and data sovereignty is not free. In this case, it's the cost of running another LLM call on the same request. So, one way we're able to build trust with these security and compliance teams is that we're not just saying trust us, here's the answer.

我们必须展示证据，并说明我们是如何得出答案的。许多供应商会告诉你：“我们的部署上线只需要几周时间。”但他们没有说的是，他们并未根据你的基础设施、安全状况以及特定集群的细节来核实这个时间。因此，我们会关注一些能够缩短上线时间的信号；归根结底，就是看是否已经建立了许多流程，例如如何从镜像仓库拉取镜像，以及是否已准备好用于推理的 GPU。We have to show the evidence and the work on how we arrived at the answer. And many vendors will tell you, you know, we're our deployment time to production takes a few weeks, but what they're not saying is they didn't check that time against your infrastructure, your security posture, and the nuances about your specific cluster. And so certain signals that we look for in terms of shortening that time to production is really just having a lot of processes set up in terms of, you know, how you pull images down from an from a registry, having GPUs sourced and ready to go for running inference.

如果要进行正式的安全审查，或者要构建定制的 LM 网关或 RBA 配置，时间线就会延后几个月。如果连将镜像拉取到自己的镜像仓库都没有审批流程，也没有关于 GPU 来源和采购的方案，时间线还会进一步延后。我发现，导致时间线延后的大多数因素，技术层面的原因较少，更多是银行内部在人际、组织和协同方面面临的挑战。That timeline gets pushed out a couple months when you have formal security reviews, or you're building a custom LM gateway or RBA configuration. And then that timeline gets pushed even further out when you don't even have an approval process for pulling images down to your registry, and not having a story around sourcing and procuring GPUs. Now, I found that most of these signals that push the timeline out are less technical and more about internal human and organizational challenges and alignments at the banks.

不过，仍有一些重要因素需要弄清楚。今天的主题是展示你的工作成果。这才是你与银行合作时，真正能够撬动合规进度的杠杆。Uh but still important factors to figure out. So, the theme for today is showing your work. That is the real lever that you have on compliance velocity when working with a bank.

将客户问题转化为工程问题Making customer problems engineering problems

当我们的银行客户——当其中一位客户来找我们，说：“嘿，我们的客户遇到了问题。”这些问题就成了我团队的问题。我们会把这些问题拆分为工程、工作流程，以及人员和组织工作流。我们的目标是尽可能实现自动化，但这些问题总让我夜不能寐。And when our banks, when one of our customers come to us and tell us, "Hey, our customers are having problems." Those problems now become my team's problems. And we break those problems down into uh engineering, workflow, and human and organizational work streams. The goal is to automate as much as possible, but those problems are the ones that keep me up at night.

我希望确保竞争对手在这些方面都不会比我们更努力，也不会比我们更聪明。我还想说，你不应该只满足于成为又一个供应商，而应该真正着手解决客户的问题。我从与这些银行和安全团队合作中学到的最大一课是，事关重大。And I want to ensure that our competitors don't outwork us and don't outsmart us on any of those vectors. And I would say that you don't want to just be another vendor, you want to actually be solving your customers' problems. And I would say the biggest lesson I've learned from working with these banks and security teams is that there's so much at stake.

支票账户、抵押贷款、信用卡。在这些情况下，精确到小数点后四位很重要。Checking accounts, mortgages, credit cards. Getting your answers down to the fourth decimal place matters in these situations.

这些银行里的许多软件之所以陈旧过时，是有原因的。但传统软件、老旧银行流程与创新之间的界线，每天都在发生一点变化，而这正是因为你们每一个人。所以我想，你们都在探索不同的智能体框架，用新的工具链进行构建，还在尝试 Jev。And there's a reason why a lot of the software within these banks is old and outdated. But the line between legacy software and old banking processes and innovation is moving a little bit every day and it's because of all of you. So I think all of you're, you know, exploring different agent frameworks and building with, you know, new harnesses, playing with Jev.

对于你们当中少数那些关注银行内部缓慢、乏味、官僚、讲究章法，以及混乱的人际和组织挑战的人来说，在金融服务领域部署 AI 再合适不过了，因为其中利害攸关。而且，如果你们想根据这些支柱来评估自己的部署情况，尽管联系我。我今天只讨论了三大支柱，不过我们还有一整套生产环境检查清单，用来支撑这些支柱。And I would say for the few of you who care about the slow, the boring, the bureaucratic, the methodical, the messy human and organizational challenges within a bank, there's nothing like deploying AI within the financial services sector because there's so much at stake. And yeah, if you all want to score your own deployments against um these pillars, just just reach out. I only discussed three pillars today, but um you know, there's a whole production checklist that we use to uh that underpins these pillars.

要点Takeaways

回想我十年级的数学考试，那些考试最棒的部分之一，就是考完之后，我和朋友们会去餐桌旁，尽量把还记得的所有题目都匆匆写下来。然后我们就利用那段时间讨论：“嘿，我们是怎么着手解这道题和那道题的。”这让我能够思考解决这些题目的不同思路和不同方法。像这样的会议，就有点像我十年级时的那张午餐桌，因为我想向大家学习，看看你们是如何为客户解决问题的。Going back to my 10th grade math test, one of the best parts about those tests were afterwards we me and my friends we, you know, go to the lunch table and we'd try to scribble down all the problems we remembered from the test. And then we use that time to talk about, "Hey, here's how we approached solving this problem and that problem." And it allowed me to kind of think about different approaches and different ways to solving these problems. And conferences like this are kind of like that uh lunch table back from when I was in 10th grade because I want to learn from all of you on how you're solving your problems for your customers.

非常感谢。Thank you very much.