当融资材料写着「70% 自动化」，技术尽调该查什么？

简要描述Brief Description

当演示文稿宣称“70% 已实现自动化”时，技术尽职调查会发现什么——Soukaïna AbelhadWhat Technical Due Diligence Finds When the Deck Says “70% Automated” — Soukaïna Abelhad

目录Table of Contents

• 自动化比例未能体现的内容
  What an automation percentage leaves out
• 检查一：工作量与人力劳动的对比
  Check one: workload against human labor
• 第二项检查：定义自动化边界
  Check two: define the automation boundary
• 第三项检查：评估与专家审查
  Check three: evaluation and expert review

自动化率未能体现的内容What an automation percentage leaves out

我叫 Sukaina，是一名 AI 技术负责人。今天我想谈谈，当演示文稿上写着“70% 自动化”时，技术尽职调查会发现什么。可能有人给你看过这样的数字：快了 55.8%。你一直在琢磨，这个数字到底能不能信。这个数字来自一项受控实验，实验对象是 95 名专业开发者。但实验任务仅限于：基于模板构建一个 JavaScript HTTP 服务器，并通过 12 项公开测试。今天，我会介绍 AI 时代技术尽职调查的三项检查要点。实际上，我在这个问题的两边都积累了超过 80 年的经验。在买方一侧，我构建过投资专业人士使用的 AI 系统；在构建一侧，我现在负责生产环境中的 AI 评估，并协助生成提交给高管的各项数据。危险的数字并不总是捏造出来的。My name is Sukaina, and I'm an AI tech lead, and today I'm here to talk about what a technical due diligence finds when the deck says 70% automated. Someone has probably shown you a number like this. 55.8% faster. And you've been wondering whether to believe it or not. Well, this one comes from a controlled experiment with 95 professional developers. But the task was one JavaScript HTTP server built from a template against 12 visible tests. Today, I'll give you three checks for what technical due diligence finds in the AI era. I've actually spent more than 80 years on both sides of the question. On the front side, I built AI system that investment professional use. On a build side, I now lead AI evaluation in production and help create the numbers that reach executives. The dangerous number is not always fabricated.

它往往是一个真实的测量结果，被带入了一个它……的结论中It's often a genuine measurement carried into a conclusion it

检查一：工作量与人力劳动对比Check one: workload against human labor

无法支持。Chun Chan 很好地描述了买方视角：为备忘录而建，而非为演示而建。我想把这一标准延伸到运营指标上。第一项检验不需要了解模型，只需做纯粹的算术。在 Faros 的观察性数据中，AI 采用率高的团队合并的拉取请求多了 98%。拉取请求的审查时间增加了 91%，而平均每个拉取请求的规模扩大了 154%。样本涵盖了 1,255 个团队中的 10,000 多名开发者。这并不能证明 AI 增加了员工人数。这是观察性遥测数据，而不是受控的劳动力研究。但它仍然给我们敲响了警钟：进入系统的工作量增加了，下游审查负担也随之加重。因此，在尽职调查中，要询问公司对未来 18 个月的判断，并把工作量与人工的全成本放在一起比较。cannot support. Chun Chan described the buyer side in a nice way. Built for the memo, not for the demo. I want to extend that standard to operational metrics. And the first check requires no knowledge of models. It's pure arithmetic. In Faros observational data, high AI adoption teams merged 98% more pull pull requests. Pull request review time increased by 91% and the average pull request became 154% larger. And the sample covered more than 10,000 developers across 1,255 teams. This does not prove that AI increased head count. It's an observational telemetry, not a controlled labor study. But it still give us a warning. More work entered the system and the downstream review burden grew with it as well. So in diligence, ask for the company's own 18-month theory. Put workload volume next to the fully loaded human labor.

统计所有因 AI 而产生工作的人员，例如操作人员、审核员、复核员、培训人员、升级处理人员和返工人员。如果人工劳动量仍随工作量变化，那么这份演示文稿描述的是辅助，而非减少人工劳动。但当这笔账算得过来时，Count everyone whose work exists because of AI, for example, operators, reviewer reviewers, trainers, escalation and rework. If human labor tracks workload volume, the deck has described assistance, not labor removal. But when the arithmetic survives, the

检查二：定义自动化边界Check two: define the automation boundary

定义调用仍然可能失败。比如，Uber 表示，其 70% 的拉取请求都归因于 AI。这很了不起，但仍无法告诉我们完整的自主完成率。披露的信息也没有说明统计周期和总数。因此，每一项自动化声明都处于这条阶梯上的某个位置：要么由 AI 起草，要么经人工编辑，要么经人工批准，要么完全没有人工干预。金融领域已经有一个实用的术语来描述最后一种情况，叫作直通式处理。可以把它定义为：在你指定的边界范围内，端到端地完成整个流程，且无需人工干预。那么，这里的边界是止于代码起草完成，还是止于拉取请求获批，抑或要等到变更部署到生产环境？所以请记住，“归因于 AI”和“自主完成”是两种不同的说法。definition call still can fail. Uber, for example, says that 70% of its pull requests are AI attributed. This is impressive. It still does not tell us the full autonomous rate. The period and total count were not disclosed, either. So, every automation claim sits somewhere in this ladder. It's either AI drafted it, or edited by humans, or approved by humans, or no human intervention. Finance already has a useful term for the last one. It's called straight-through processing. Define it as a complete end-to-end without human intervention inside a boundary you name. So, does the boundary here end when the code was drafted, or when the pull request is approved, or when the change reaches production? So, remember, AI attributed and autonomous are two different claims.

检查三：评估与专家评审Check three: evaluation and expert review

好，现在假设定义足够精确。我们能相信这项测量吗？Meta 发现，在以人工基准进行调整后，自动评分器与合并决策之间仍有 24 个百分点的差距。来自三个代码仓库的四位维护者审查了 296 个针对不同问题、由 AI 生成的拉取请求。他们不知道补丁来自 AI 还是人类。自动评分器的评估明显比那些负责决定是否接受这些成果的人乐观。这并不意外。我也从另一个角度见过同样的情况。我参与过的一个项目中，10 个输出里有 10 个都能通过自动验证。但一旦让专家参与进来，能过关的就寥寥无几。这就是为什么我会把评估成熟度视为判断可信度的首要信号。所以，一定要问清楚：谁标注了真实标签？专家是否认同？有没有人衡量过这种一致性？Okay, now suppose the definition is precise. Can we trust the measurement? Meta found 24% point gap between automated grader and merge decision after adjustment to the human baseline. Four maintainers from three repositories reviewed 296 AI generated pull requests across different unique issues. They did not know whether the patch came from AI or human. The automated grader was substantially more optimistic than the people responsible for accepting the work. Not a surprise. I have seen the same thing from the other side. One of the projects I worked on, 10 out of 10 output would pass the automated validation. Once you bring the the experts on the table, barely few survive. This is why evaluation maturity is first credibility signal I would look for. So, always ask who labeled ground truth, and whether the expert agrees, and whether anyone measures that agreement.

询问自动化评审器是依据什么进行验证的，并询问有哪些糟糕的输出漏了过去。那么，当算术、定义和评估都出了问题时，一个词仍然必须转化为决策，对吧？这并不是说，薄弱的 AI 定位导致了 Builder.ai 的崩溃。它资不抵债还涉及 AI 相关声明以外的营收融资问题。尽职调查的教训早在 2019 年就已显现，当时《华尔街日报》公开质疑其 AI 能力声明。2025 年 5 月，《金融时报》也报道称，该公司对 2024 年营收的估算已从 2.2 亿美元下修至约 5,500 万美元。6 月 2 日，该公司申请了《破产法》第七章破产程序。因此，警讯早已存在，但流程并未将其转化为一道关卡。所以，即便一项声明通过了这些检查，它仍然必须经受住经济层面的考验。那就从符合条件的工作流价值开始。Ask what the automated judge was validated against, and ask what bad output escaped. So, when arithmetic's definition and evaluation all fell, a word it still has to become a decision, right? This is not a claim that weak AI positioning caused builder.ai collapse. Its insolvency involved revenue financing issues beyond AI claims. The diligent lesson started way back in 2019, when the Wall Street Journal publicly questioned the claim of the AI capability. In May 2025, the The Financial Times also reported that its 2024 revenue estimated had been revised from $220 million to roughly $55 million. On June 2nd, the company filed Chapter 7. So, the warning existed, but the process did not turn it into a gate. So, even when a claim survives those check, it still has to survive the economics. So, start with eligible workflow value.

应用实际采用率、端到端流程成功率和质量验收率。然后，扣除 AI、人工审核、升级处理和返工的成本。我想要的指标是每项已验收自主任务的成本。如果任务失败、需要升级处理或返工，这些成本都应计入。因此，务必把这些检查放在一起；这张卡片大概就是我会带去下次会议的内容。 总结一下，主要有三项检查。第一，算术：将工作量与包含全部成本的人工劳动进行比较。第二，定义：找出具体工作流中的错误项，并确定其端到端处理率。第三，判断标准：询问由谁定义质量，以及专家是否认同该输出能够取代什么。然后，只把经得起检验的说法转化为单位经济效益。当然，要么相信它已经过测试，要么就离开。 所有来源资料中的可打印卡片都在网站上，扫描这个二维码即可查看。Apply actual adoption, straight-through process success, and accepted quality. Then, subtract the cost of AI, human review, escalation, and rework. The metric I would want is cost for accepted autonomous task. If the task fails, escalates, or need rework, this belongs to the cost. So, always put those checks together, and this is the card I would probably take with me to the next meeting. To sum up, three main checks. The arithmetic's, compare workload volume with fully loaded human labor. Two, definition, identify the wrong and and the straight-through rate inside a name workflow. Three, the tell. Ask who defines quality. Whether expert agrees in what that output escapes. Then translate only the surviving claims into a unit economics. And of course, believe it tested or walk away. The printable card in all the source notes are in the website and this QR code will lead you to it.

那么，拿一张卡片，扫描它，再打印出来，带去你下一次供应商会议或投资讨论。请始终记住，尽职调查的目的不是问这个数字是否令人印象深刻，而是要问：这个数字要有意义，必须满足什么条件。谢谢。So, take a card, scan it, print it, and take it to your next vendor meeting or investment discussion. And always remember, the job of diligence is not to ask whether the number is impressive. It's to ask what must be true for the number to matter. Thank you.