Scion:Anthropic 为智能体编程重建版本管理
原题Redesigning source control at Anthropic
推荐理由Anthropic 的日均合并量从 5 月约 2500 次增长到 9 月超过 3 万次,逼着团队重新审视版本管理的默认假设。Scion 将更多操作放到服务端,以变更日志和按需获取文件处理大量持续在线的客户端。真正值得借鉴的是如何从工作负载反推架构,而不是得出「Git 已经过时」的结论。团队明确说明,这套系统还依赖内部的审查、搜索和持续集成工具,外部组织更适合借鉴其中的设计,而非直接复制整套方案。
读前速览
Anthropic 用 Scion 说明,智能体带来的提交增长会改变版本管理的架构假设。
提交增长改变基础设施的约束。
正文报告日均合并从 2026 年 5 月约 2500 次增至 9 月超过 3 万次,增长口径与时间段应一起保留。
常在线客户端允许不同的服务端设计。
Scion 将更多操作交由服务端,以变更日志和按需获取文件承接高吞吐任务。
内部方案依赖配套工具。
团队明确说明审查、搜索和 CI 高度定制化;外部组织更适合借鉴局部设计,而非直接照搬整套系统。
完整摘要
Anthropic 报告,日均代码合并量从 2026 年 5 月约 2500 次增至 9 月超过 3 万次,传统 Git 部署逐渐成为瓶颈。团队构建 Scion,将常在线客户端、服务端操作、变更日志和按需获取文件结合起来。它适用于团队自身的高吞吐需求,依赖配套审查、搜索和持续集成系统,并非可以直接替换所有组织 Git 的通用产品。
目录 · 8 节
在 Anthropic,编码已不再是构建系统或开展研究的瓶颈。Claude 能够自主迭代研究循环、对复杂程序进行微优化,并从头构建新服务,其速度和规模前所未见。如今的瓶颈在于那些为人类规模而构建的开发者基础设施:源代码管理、构建系统和 CI。At Anthropic, coding is no longer the bottleneck to building systems or doing research. Claude is able to autonomously iterate on research loops, micro-optimize complex programs, and build new services from the ground up at a speed and scale we haven’t seen before. The bottleneck, now, is all of the developer infrastructure built for human scale: source control, builds, and CI.
2026年5月,Anthropic平均每天合并约2,500项代码变更;截至9月,我们每天合并的代码变更已超过30,000项。我们预计这种增长速度短期内不会放缓;合并代码的潜在需求几乎是无限的,而我们每消除一个瓶颈,产出的代码量就会立即进一步增加。In May 2026, Anthropic averaged around 2,500 code changes merged per day; as of September, we merged over 30,000 per day. We don’t expect this rate of growth to slow down anytime soon; the latent demand for merging code is practically infinite, and every bottleneck we remove immediately leads to a further increase in the amount of code we produce.
为了支持这一增长,我们构建了内部版本控制系统 Scion,并在五个月内将我们的单体仓库从 git 完全迁移了过来。在同一时期,我们还构建并推出了代码审查界面(名为 Antique)、代码搜索、功能丰富的源代码控制 IAM 系统,以及管理和合入大量已审查代码的新方法,等等。这篇文章将重点介绍我们具体如何构建 Scion。这是一个有趣的案例:Claude 编写代码的能力,既是我们最初陷入这一境况的原因,也是我们得以迅速构建出高质量、可扩展系统来应对这一挑战的关键。In order to support this growth, we built Scion, an in-house version control system, and fully migrated our monorepo from git in five months. We also built and shipped a code review surface (called Antique), code search, a rich source control IAM system, novel ways of managing and landing large amounts of reviewed code, and more, all in that same time period. In this post, we’ll focus on how we built Scion specifically. It makes for an interesting case study: Claude’s ability to write code was both what put us in this situation in the first place and how we were able to quickly build a high-quality, scalable system to deal with it.
扩展我们的单体仓库Growing our monorepo
在探讨我们构建的系统之前,我们应该先承认,我们的工作负载相当特殊。Anthropic 使用 Claude 编写软件的程度远超几乎所有其他公司,因此,我们往往最先发现那些为人类规模设计的工具开始吃力的地方。源代码管理就是其中一例。虽然我们预计在近期内不会有很多组织遇到这些问题,但我们怀疑,用 AI 编码工具的其他人用不了多久也会面临同样的瓶颈。这正是我们想分享自己构建的系统以及从中学到的经验的原因。Before we explore what we built, we should acknowledge how unusual our workload is. Unsurprisingly, Anthropic uses Claude to write software far more than almost any other company, so we’re often the first to see where tools built for human scale start to strain. Source control is just one example of this. While we don’t expect many organizations to have these problems in the very near term, we suspect it won’t be too long before others who use AI coding tools experience these same pressure points, which is why we wanted to share what we built and what we learned.
Git 是一种去中心化系统:每个克隆版本都是一个完整、独立的代码仓库,即使完全离线,也能计算差异、查看历史记录和代码归属,并检出任意版本。对于绝大多数代码仓库来说,这是恰当的设计,也是 Git 成为标准的重要原因之一。Git is a decentralized system: every clone is a complete, self-contained repository that can compute diffs, read history and blame, and check out any revision while fully offline. For the vast majority of repositories, this is the right design, and it’s a large part of why git became the standard.
随着我们的单体仓库不断增大,它越来越不适合这种设计。仅主分支的历史记录就接近 100 GB,压缩后的工作副本也超过 25 GB。我们每天为 CI 执行约 2500 万次代码检出,还未计入开发者环境、智能体沙箱和研究任务。如果不加优化地获取代码,每天产生的 Git 流量将超过 600 PB。As our monorepo grows, it increasingly doesn’t fit that design space. The history of our main branch alone is almost 100 GB, and a compressed working copy is over 25 GB. We perform on the order of 25 million code checkouts per day for CI, before counting developer environments, agent sandboxes, and research jobs. Fetched naively, that would be over 600 PB of git traffic per day.
Git 提供了多种工具来处理仓库的一个切片,例如浅克隆、稀疏检出和 promisor 远程仓库。除此之外,大型 monorepo 还有一些常见的变通方案,比如内部 Git 镜像,以及存放在 blob 存储中的仓库快照(种子)。我们全都用过——在业务高峰期,为了应对负载,我们运营着数百个内部 Git 镜像。这为我们争取了时间,但随着合并速度加快,维护它们变得越来越困难。两次刷新之间,种子与 main 分支最新提交的距离越来越远;而要确保依赖历史记录的操作(例如 diff 和 blame)在部分克隆上正常运行,也需要投入越来越多的精力。Git offers tools for working with a slice of a repository, such as shallow clones, sparse checkouts, and promisor remotes. Beyond these, there are standard workarounds for large monorepos, like internal git mirrors and repository snapshots (seeds) kept in blob storage. We’ve used all of them—at our peak, we operated several hundred internal git mirrors to deal with the load. This bought us time, but they became harder to operate as our merge rate increased. Seeds fell further from the tip of main between refreshes, and it took more and more care to keep operations that depend on history, like diff and blame, working on partial clones.
退一步来看,我们注意到,和大多数公司一样,我们把 git 当作集中式系统使用,只有一个权威副本;然而,我们的客户端几乎全都是代理和任务,顾名思义,它们始终在线。随着仓库不断增长,git 在分布式离线工作中表现出色的特性,反而在主动削弱我们扩展规模的能力。于是我们开始思考:如果从一开始就考虑到代理始终在线这一点来设计源代码管理系统,它会是什么样子?Taking a step back, we noticed that, like most companies, we were using git as a centralized system with one authoritative copy, but nearly all of our clients were agents and jobs that are online by definition. The properties that make git excellent for distributed, offline work were actively undermining our ability to scale as the repo grew. So we asked ourselves: if we designed source control with the fact that agents are always online in mind from the start, what would it look like?
面向智能体规模的源代码管理原则Principles for agent-scale source control
随着智能体式编程的发展,许多关于软件开发方式的常见假设正在发生变化。我们在构建 Scion 时拥抱了这些变化,因此得以在若干核心原则上采用与 git 截然不同的做法。针对超大型单体仓库的源代码管理已有先例,尤其是 Google 的 Piper 和 Meta 的 Sapling,因此我们得以汲取它们来之不易的经验教训来构建 Scion。我们的原则是:With agentic coding, many common assumptions about how software is developed are evolving. We embraced these changes when building Scion, which allowed us to take a fundamentally different approach than git on several core principles. Source control for huge monorepos has prior art, notably Google’s Piper and Meta’s Sapling, so we were able to build Scion with their hard-won lessons in mind. Our principles are:
客户端始终在线。Git 假设客户端会离线工作,因此需要拥有仓库的完整副本。由于我们的客户端几乎都是代理和作业,我们围绕始终可访问的服务器设计了 Scion 的客户端和协议,让客户端在需要时获取数据。Clients are always online. Git assumes that clients work offline and need a complete copy of the repository. Since nearly all of our clients are agents and jobs, we designed Scion’s client and protocols around a server that is always reachable, and have clients fetch data when they need it.
客户端-服务器不对称性。在 git 中,每个客户端都能充当功能完备的服务器。在 Scion 中,服务器负责回答有关文件历史和 blame 的问题、创建提交、执行合并和变基等操作。客户端维护一个容量有限的本地对象缓存,将工作副本文件写入磁盘,监控文件系统的变化,并为用户提供一种便捷的方式来使用服务器。客户端自身能执行的计算很少,但作为交换,它在本地所需的数据也少得多。Client-server asymmetry. In git, every client is capable of being a full-fledged server. In Scion, the server answers questions about file history and blame, creates commits, performs merges and rebases, and so on. The client maintains a bounded local object cache, lays out the working copy files on disk, monitors for filesystem changes, and gives users an ergonomic way to work with the server. It can compute very little on its own but, in exchange, it requires much less data locally.
全面的源代码管理服务可以为许多新颖的应用提供支持。使用 git 时,CI 任务或工具如果需要查询仓库中的信息,例如某一行最近由谁修改,或两个提交之间有何关联,就必须维护一份本地仓库副本,或者寄希望于供应商特有的 API 能提供答案。借助 Scion,客户端使用的所有源代码管理查询也都通过 API 对外开放,因此在许多情况下,任务完全不需要本地仓库。许多内部工具(例如我们的自动化二分查找和回滚流水线)如果没有这种成本低廉、可通过程序访问的源代码管理能力,就无法大规模投入实际使用。A comprehensive source control service can power many novel applications. With git, a CI job or tool that needs to ask a question about the repository, such as who last changed a line or how two commits are related, has to maintain a local copy of the repo or hope that vendor-specific APIs provide the answer. With Scion, all of the source control queries the client uses are also exposed via API, so in many cases jobs don’t need a local repository at all. Many internal tools, such as our automated bisect and revert pipelines, wouldn’t be practical at scale without this cheap, programmatic access to source control.
一切都必须能够横向扩展。在业务高峰期,我们运行着数百个 Git 镜像,这极大地加重了容量压力;一旦仓库超出镜像的磁盘或内存限制,扩展就不再可行。在 Scion 中,仓库数据是分布式存储的,每个组件都可以独立扩展;任何地方都不存在仓库的完整物化副本,即使在服务器上也没有。Scion 中没有与 packfile(Git 的压缩对象包)等效的机制,因此无需维护仓库。Everything must be horizontally scalable. At our peak, we ran several hundred git mirrors, which significantly strained capacity and no longer scaled once the repo outgrew the mirrors’ disks or memory limits. In Scion, the repository’s data is distributed, and every component is independently scalable; there is no fully materialized copy of the repo anywhere, even on the server. There is no equivalent of packfiles (git’s compressed bundles of objects), so no repository maintenance is needed.
提交必须轻量、增量式且无锁。在 Git 中,你先在本地提交,然后推送以分享你的工作。而在 Scion 中,每次提交、变基或合并都在服务器上进行,因此无需推送,但这些操作都必须足够快。创建提交并更新其下游数据结构的成本必须与改动的大小成正比,而不能取决于整个仓库的大小,否则服务器就会不堪重负。Scion 的开发生命周期不鼓励使用引用(指向提交的指针,例如分支和标签),主引用或发布分支除外,因为所有服务器都必须达成一致的全局状态,是一个迫在眉睫的扩展性问题。Commits must be cheap, incremental, and lock-free. In git, you commit locally and then push to share your work. In Scion, every commit, rebase, or merge happens on the server, so there’s nothing to push, but each of those operations needs to be fast. The cost of creating a commit and updating its downstream data structures has to scale with the size of the change, not the overall size of the repo, or the server can’t keep up. The development lifecycle in Scion discourages refs (pointers to commits, such as branches and tags), except for the main ref or release branches, because global state that every server has to agree on is a looming scaling problem.
工作副本是可选的,而且从设计上就是稀疏的。我们的单体仓库规模非常庞大,并且每天为了 CI 要检出代码数千万次,因此我们希望尽可能避免检出。以 Scion 为例,CI 作业如果需要某个提交与其合并基点之间的差异,就可以向服务器请求。同样的原则也适用于写入操作:许多写入型工作负载也可以跳过检出,完全通过 API 执行。Scion 的协议以稀疏检出为核心——只包含客户端请求的目录;同时也支持虚拟化检出,按需物化文件。因此,当作业确实需要在磁盘上使用文件时,客户端只需为实际读取的内容付出代价。The working copy is optional, and sparse by design. Because our monorepo is so large, and because we check out code tens of millions of times a day for CI, we wanted to avoid checkouts whenever we could. With Scion, a CI job that needs the diff between a commit and its merge base, for example, can ask the server for it. The same goes for writes: many write workloads can also skip the checkout and function entirely via API. Scion’s protocol is centered around sparse checkouts, which only contain the directories the client asks for, and virtualized checkouts, which materialize files on demand, so when a job does need files on disk, the client only pays for what it actually reads.
Scion 的工作原理How Scion works
了解这些原则在构建 Scion 的过程中如何付诸实践,会很有帮助。这些经验也适用于其他源代码控制系统,或者更广泛地说,适用于软件开发生命周期的其他阶段。It’s helpful to understand how these principles came together in practice when building Scion. These insights can apply to other source control systems or, more broadly, to other stages of the software development lifecycle.
数据结构Data structures
在 Scion 中,工作单元是“变更集”(changeset),即写前日志中的一条记录。变更集保存了你通常会认为 git 提交所包含的大部分内容。不过,git 提交保存的是指向树的指针,而变更集保存的是相对于其第一个父项发生变化的文件列表¹,以及指向 blob 存储中压缩文件内容的内容寻址指针。变更集在服务器端创建,并可立即在任何地方使用。In Scion, the unit of work is a “changeset,” an entry in a write-ahead log. A changeset stores most of what you’d expect from a commit in git. But where a git commit stores a pointer to the tree, a changeset stores a list of the files that changed relative to its first parent,1 along with content-addressed pointers to the compressed file contents in blob storage. Changesets are created server-side and are available everywhere, instantly.
这种结构使我们能够以极高的吞吐量接收变更集,因为无需接收包文件,也无需锁定或跨服务器协调,更无需移动引用。Scion 确实支持引用,但由于创建和获取提交都不涉及引用,移动引用只需对 Spanner 中的一行执行比较并交换操作,而且这与创建变更集是彼此独立的。This structure lets us accept changesets with very high throughput because there are no packfiles to accept, no locking or cross-server coordination, and no refs to move around. Scion does technically support refs, but since neither creating nor fetching commits involves them, moving one is a compare-and-swap on a single Spanner row that happens separately from creating changesets.
变更集只记录发生了哪些变化,因此仅凭它本身,无法快速回答诸如某一行最后由谁编辑,或哪些提交影响了某个文件等问题。为回答这些问题,Scion 会根据变更集构建额外的数据结构,包括专用于文件 blame 和路径历史的数据结构,以及用于提交图的跳表。我们把所有不直接属于变更集的内容都称为“派生数据”,并通过一种通用机制来计算这些数据。每种派生数据都以递归方式定义,因此,某个变更集对应的值是根据其父变更集对应的值计算得出的。A changeset records only what’s changed, so on its own, it can’t quickly answer questions like who last edited a line or which commits affected a file. To answer those, Scion builds additional data structures from the changesets, including dedicated structures for file blame and path history and skip lists for the commit graph. We call everything that isn’t directly part of the changeset “derived data,” and we compute it all through one generic mechanism. Each type of derived data is defined recursively, so its value for a changeset is computed from its values for that changeset’s parents.
这种机制有三项好处。第一,当新的数据结构能够加快某项操作时,我们可以轻松添加一种新的派生数据。第二,对于每种类型,我们都可以选择立即计算,或将工作推迟,从而避免物化不需要的数据。第三,由于客户端永远看不到这些数据结构的内部细节,我们可以对其进行优化,而无需修改客户端。随着系统不断发展,我们多次添加了新的派生数据类型,例如用于执行副本检测或基于路径的文件搜索的新索引。This mechanism has three benefits. First, we can easily add a new type of derived data when a new data structure would make some operation faster. Second, for each type, we can choose whether to compute it eagerly or defer the work, so we avoid materializing data we don’t need. And third, because the client never sees the internals of these data structures, we can optimize them without changing the client. We’ve added new types of derived data many times as the system has grown, such as new indices for performing copy detection or path-based file search.
扩展 ScionScaling Scion
少量 Scion 服务器即可为客户端所需的任何操作提供源代码管理 API,例如获取变更集、计算差异、查找合并基点以及获取数据块。所有源代码管理数据都采用内容寻址,因此数据一经存储便不会改变,任何服务器都可以缓存这些数据,而无需与其他服务器确认。因此,这些服务器可以横向扩展,且只需很少的协调。它们本质上是大型缓存,还能够即时计算派生数据,并将结果写回 Spanner。A small fleet of Scion servers provides source control APIs for any operation a client needs, such as getting changesets, computing diffs, finding merge bases, and fetching blobs. All source control data is content-addressed, so a piece of data never changes once it’s stored, and any server can cache it without checking with the others. As a result, the servers are horizontally scalable and require little coordination. They are essentially large caches that can also compute derived data on the fly and write the results back to Spanner.
Scion 的许多使用者如今不再需要维护自己的本地仓库。过去,对于需要查询源代码管理信息(尤其是提交祖先关系和 blame 信息)的 CI 作业或工具来说,获取并维护仓库是一项相当繁重的维护工作。在迁移初期,也就是我们还没有自己的客户端、仓库格式或协议时,Scion 只处理这些“元数据”查询。获取文件内容和生成工作副本所需的数据量大得多,因此需要专门设计方案。Scion’s many consumers no longer need to maintain local repositories of their own. Fetching and maintaining the repository used to be a significant maintenance burden for CI jobs or tools that needed to ask source control questions, particularly about commit ancestry and blame. Early in the migration, before we had our own client, repo format, or protocols, Scion only answered these “metadata” queries. Fetching file contents and materializing working copies are far more data-intensive operations, and they needed a design of their own.
Scion 的 scn 客户端在几个重要方面不同于 git 的 fetch 协议,因此能够采用高度可扩展的架构,所需 CPU 资源也显著少于 git 服务器。Scion 的协商过程不会告诉服务器客户端已经拥有哪些内容;相反,它只针对客户端需要的树和 blob 进行操作。客户端从根树开始,在本地存储中检查是否缺少树对象,并递归请求缺失的对象,然后对 blob 重复这一过程。每个请求都针对一个特定且永不改变的对象,因此任何缓存都能响应请求,整个架构也与传统的分层缓存非常相似。这个过程效率高且可断点续传,因为它不需要一次性接收一个数 GB 的 packfile 响应。稀疏检出甚至虚拟化检出都不需要特殊支持,因为客户端只需不请求自己不需要的树对象。Scion’s scn client deviates from git’s fetch protocols in several important ways and, as a result, lends itself to a highly scalable architecture that requires significantly less CPU than git servers. Scion’s negotiation doesn’t tell the server what the client already has; instead, it operates exclusively on trees and blobs the client wants. Starting from a root tree, the client checks its local store for missing trees and requests them recursively, then repeats the process for blobs. Every request is for a specific object that never changes, so any cache can answer it, and the architecture closely resembles traditional tiered caches. The process is also efficient and resumable, because there is no single multi-gigabyte packfile response. Sparse and even virtualized checkouts don’t need special support, because the client simply doesn’t ask for trees it doesn’t want.
在集群中,客户端通过套接字与守护进程 scion-daemon 通信,该进程为整个节点存储对象。我们的大多数节点都需要访问单体仓库,而且同一节点上的作业会重复使用大部分相同的对象。每个守护进程都从定期刷新的快照启动,之后只与其所在区域的树缓存服务器和 Blob 缓存服务器通信,这些服务器称为 scion-cache。这些缓存服务器会直接从内存中以守护进程可使用的磁盘格式流式传输对象,因此无需花时间计算客户端缺少哪些对象,也无需执行差分压缩。如果缓存中没有某个对象,它就会向权威的 scion-servers 发出请求;后者会根据需要从 Spanner 或云存储读取数据。In the fleet, the client talks over a socket to a daemon, scion-daemon, that stores objects for the whole node. Most of our nodes need access to the monorepo, and jobs on the same node reuse most of the same objects. Each daemon starts from a periodically refreshed snapshot, then talks only to the tree and blob cache servers in its region, called scion-cache. These cache servers stream objects directly from memory, in the disk format that the daemon will consume, so they don’t need to spend time computing the client’s missing objects or performing delta compression. When a cache is missing an object, it asks the authoritative scion-servers, which read from Spanner or cloud storage as needed.
scion-daemon、scion-cache 和 scion-server 对此协议中的相同 RPC 都会作出响应,因此客户端无需了解缓存拓扑结构。理论上,任何层级都可以与其他任意层级通信,但我们会严格强制分层,以隔离故障域。我们要避免区域级 scion-cache 出现故障后压垮上游 scion-server,进而导致整个系统瘫痪。The scion-daemon, scion-cache, and scion-server all respond to identical RPCs for this protocol, so the client doesn’t need to know the cache topology. Any tier could theoretically talk to any other, but we strictly enforce the tiering to isolate failure domains. We want to avoid a situation where a failure in a regional scion-cache overwhelms the upstream scion-servers and brings down the entire system.
该系统的每一层缓存命中率都超过 99%,且无需维护长期状态,因此可按需轻松扩容。如今,与我们之前的 Git 镜像基础设施相比,Scion 基础设施所需的 CPU 和内存资源减少了 200 倍。CI 和研究任务检出代码的速度快了两个数量级(从几分钟缩短到几秒),为整个集群释放了大量容量;仅 CI 就节省了 25% 的 CPU 容量。客户端和缓存都无需执行仓库维护工作,从而简化了任务的生命周期管理。Each tier of this system maintains a cache hit rate of over 99% and does not require long-lived state, so it’s easy to scale up if needed. Our Scion infrastructure footprint is now 200x smaller than our previous git mirroring infrastructure in terms of CPU and memory requirements. CI and research jobs check out code two orders of magnitude faster (in seconds rather than minutes), which frees up a sizable amount of capacity across the fleet; CI alone realized a 25% CPU capacity win. Neither clients nor caches need to perform repository maintenance, which simplifies the lifecycle management of our jobs.
构建 Scion 的经验教训What we learned building Scion
我们开始构建 Scion 的时间,恰好与 Claude Mythos Preview 和 Claude Tag 开始在内部开放使用的时间相近。这两项发布一方面进一步凸显了 Scion 的必要性,另一方面也显著加快了我们的构建进度。We started building Scion around the same time as Claude Mythos Preview and Claude Tag became available internally. Both of these releases simultaneously amplified the need for Scion and meaningfully accelerated how quickly we could build it.
这里有一个特别引人注目的例子。今年早些时候,我们基本上是手工编写了 git-to-Scion 摄取流水线的第一个原型版本。我们原以为它已经不错了——直到我们拿到 Claude Mythos Preview 的早期内部快照。我们让它重写整个项目,它在没有任何进一步指示的情况下,生成了一个内存消耗减少 20 倍、速度提升 4 倍的版本,同时保留了有关摄取顺序和数据结构的所有隐含不变量。Here’s a particularly striking example. Earlier in the year, we had largely handwritten the first version of the git-to-Scion ingestion pipeline as a prototype. We thought it was good—until we got our hands on an early internal snapshot of Claude Mythos Preview. We asked it to rewrite the whole thing, and with no further instructions, it produced a version that consumed 20x less memory and was 4x faster, while preserving all of the implicit invariants around ingestion ordering and data structures.
构建和运营 Scion 的经验,为构建代理规模的软件可能是什么样子提供了一些有趣的启示。What we learned building and operating Scion offers some interesting lessons for what building agent-scale software could look like.
小团队快速开发将会变得更加普遍。一个由七人组成的团队开发了 Scion 和 Antique(我们的代码审查工具,光是它就足以另写一篇完整的博客文章),以及另外六个相关系统,并在五个月内完成了全面迁移。与此同时,我们每天合并的代码量还增长了十倍。过去,公司要开发并部署这样一套源代码管理系统,需要投入更多工程师,经历数年循序渐进的迁移。我们认为,未来这种快速、灵活的迁移方式将会越来越普遍。Rapid development with a small team will become more common. A team of seven people built both Scion and Antique (our code review tool, which we could easily write a whole other blog post about), not to mention half a dozen related systems, and fully migrated to them in five months. We did all this while the volume of code merged per day increased by 10x. In the past, developing and deploying a source control system like this would have taken a company years of iterative migration, with many more engineers. We think this sort of fast, nimble migration will become more common in the future.
工程师的价值并未降低,只是他们的工作正在发生变化。所有参与构建这个系统的工程师现在都会认为,Claude 编写代码的能力比他们更强。但我们的工程师仍然提供了不可或缺的洞察:他们理解系统间组件的整体架构以及设计层面的权衡,并能围绕我们想要解决的问题和已知需要达到的规模,做出明智的选择。我们经常发现,工程师的职责就是对 Claude 说“不”。智能体有时会违反核心系统约束,以满足一种本不该被允许的模式。这在我们的迁移过程中尤其常见。例如,有多个系统没有询问我们的代码审查工具,而是通过线性读取提交日志来暴力查找某项更改何时合并到 main。Anthropic 也广泛采用对抗性审查。尽管这种做法在很多方面都极其有效,但也可能会目光短浅,因此需要经验丰富的工程师引导 Claude,避免它为了应对假设性的担忧而对系统的某些部分过度设计。例如,有一次审查对信任另一项内部服务过于防备,导致 Claude 添加了过多的验证逻辑和复杂性。团队成员仍会审查系统的每一项变更请求,但如今他们关注的是变更的意图,而不是逐行阅读代码。Engineers are no less valuable, but their work is changing. All of the engineers who worked on this system would now rate Claude as better at writing code than them. But our engineers still supplied the invaluable understanding of the high-level inter-system components and the design-level tradeoffs, making informed choices around the problems we wanted to solve and the scale we knew we needed to achieve. Very often, we found that the role of the engineers was to tell Claude “no.” Agents will sometimes violate core system constraints to satisfy a pattern that should never have been allowed in the first place. This was particularly common in our migration. For example, there were multiple systems that tried to brute force looking up when a change merged to main by linearly reading the commit log instead of asking our code review tool. Anthropic also makes extensive use of adversarial review, and, while this is incredibly effective in many ways, it can also be myopic, so skilled engineers needed to steer Claude away from over-engineering parts of the system in response to hypothetical concerns. For example, one such review was overly defensive about trusting another internal service, which resulted in Claude adding excessive validation and complexity. Our team members still review every change request for the system, but they now examine the intent of the change as opposed to reading every line of code.
我们大幅提升了优化水平和质量。得益于 Claude 编写代码的速度远超我们自身,且能处理海量代码,Scion 成了我们参与构建过的经过最充分测试、防护最完善的代码库之一。即使时间紧迫,我们也构建了隔离式模拟环境,涵盖我们遇到过的所有复杂基础设施交互和源代码管理交互,还加入了模糊测试和精密的速率限制。Claude 还轻而易举地找到了显著的优化方案——例如,针对祖先查询的一项算法优化,弥补了我们一个索引未覆盖的情况;专家可能需要数天甚至数周分析和调优才能做到这一点……而第二天,Claude 又把这一切重做了一遍。We were able to substantially improve optimization and quality. By virtue of Claude’s ability to write immense amounts of code much faster than we could ourselves, Scion is among the most well-tested and well-defended codebases we have ever worked on. Even on our tight timeline, we were able to build hermetic simulations of every non-trivial infrastructure and source control interaction we’ve ever encountered, as well as fuzzing and sophisticated rate limiting. Claude also casually found significant optimizations—for example, an algorithmic optimization to ancestry queries that missed one of our indexes—that would have taken an expert days or weeks of analysis and tuning… and then did it all again the next day.
我们为单体仓库安全的新形式奠定了基础。在庞大的单体仓库中管理安全性、信息安全和合规,与管理其规模和扩展性同样重要。从一开始,我们在构建 Scion 时就考虑到了这些需求。它支持对代码仓库进行细粒度的访问控制,还能为单个目录设置权限,从而在不失去单体仓库优势的同时,保护最敏感的代码。此外,我们构建的代码审查工具可以针对审批设置复杂的语义约束,使我们能够区分仅涉及注释的简单变更与敏感的逻辑变更,让人工审查者能够专注于影响最重大的变更。We laid the groundwork for new forms of monorepo security. Managing safety, security, and compliance in a massive monorepo is as important as managing the size and scale. We built Scion with these in mind from the start. It supports rich access control on repositories, as well as the ability to apply permissions to individual directories in order to protect the most sensitive code without losing the benefits of a monorepo. Furthermore, the code review tooling we built can express complex semantic constraints on approvals, which lets us differentiate simple things like comment-only changes from sensitive logical changes, so that human reviewers can focus on the most consequential changes.
运营卓越有了新的可能。我们的团队大量使用 Claude Tag,这帮助我们监控系统的每个组件。例如,我们创建了 Slack 频道,让 Claude 专注到近乎执着的程度,持续关注单个组件的运行状况。我们把这些 Claude 称为“超级粉丝”。我们有一个 scion-daemon 超级粉丝、一个 scion-cache 超级粉丝、一个 Antique 合并队列超级粉丝,等等。它们可以访问我们的遥测数据和日志,就像一位经验丰富的工程师,除了刷新图表、寻找任何异常之外别无他事。我们也为所有系统设置了传统告警,但与之相比,我们的超级粉丝能发现细微而新颖的问题,让这些告警显得单一片面。A new level of operational excellence is possible. Our team uses Claude Tag heavily, which has helped us monitor each component of the system. For example, we created Slack channels where we tasked Claude with being obsessively focused on the health of a single component. We call these Claudes “superfans.” We have a scion-daemon superfan, a scion-cache superfan, an Antique merge queue superfan, and so on. They have access to our telemetry and logs, and act like an experienced engineer who has nothing better to do than to refresh charts and look for anything strange. We also have traditional alerts for all of our systems, but our superfans can detect subtle and novel issues that make these alerts seem one-dimensional by comparison.
我们正在彻底重新审视自己的先验认知。这个项目促使我们认真审视其他为人类工作规模设计的工作流。例如,正规的合并队列在智能体规模下就不太说得通了。当测试仍在运行时,数百甚至数千项更改都可能被合并,这时 CI 到底意味着什么?我们一直没能有效复现业界标准的做法——在合并队列中对整个代码仓库加锁。因此,我们转而投入资源进行合并后和部署前的自动化测试。这确实意味着并发更改可能会产生冲突,但 Claude 通常能比人类发现这些冲突还快地将其修复。We’re completely rethinking our priors. This project prompted us to take a very hard look at other workflows designed for human scale. A proper merge queue, for example, doesn’t make much sense at agent scale. What does CI even mean when hundreds or even thousands of changes can merge while tests are running? We haven’t been able to effectively replicate the industry standard of a global repository lock on the merge queue, so instead, we’ve invested in post-merge and pre-deploy automated testing. This does mean that concurrent changes can conflict, but Claude can typically fix these conflicts faster than humans can even identify them.
为更广泛的生态系统做贡献Contributing to the wider ecosystem
我们认为,Scion 只是应对日益强大的 AI 智能体所引发的扩展性问题的一种方案。我们分享这些细节,是想展示团队如何应对类似挑战;但我们并不认为,我们的解决方案或考量适用于所有情况。We see Scion as just one part of the answer to the scaling concerns that increasingly capable AI agents have raised. We’re sharing these details as an example of how teams can tackle similar challenges, though we don’t expect our solution, or our considerations, to suit every case.
Scion 绝不是适合一般用途的 git 替代品。我们在这里概述的这些概念,只有在真正庞大的单体代码库中才有意义:这类代码库要么有数万名工程师,要么有成群的智能体大量编写代码。此外,Anthropic 的独特之处在于,我们在内部使用时能够消耗的 token,远远超过其他任何企业实际愿意投入的量。我们这样做,既是为了满足自身的软件开发需求,也是为了测试 Claude 的能力。Scion is by no means a suitable replacement for git for general use. The concepts we’ve outlined here really only make sense for truly enormous monorepos, with either tens of thousands of engineers or swarms of agents prolifically writing code. Plus, Anthropic is unique in that our ability to spend tokens for internal use far exceeds what any other business would realistically want to spend. We do this both to address our own software development needs, and to test Claude’s capabilities.
从实际角度来说,很难以一种能广泛适用于其他组织的形式提供 Scion。它与 git 有很大差异,这意味着企业需要迁移开发生命周期中的所有环节,而且还需要配套的代码评审、代码搜索和 CI。我们内部使用的这些组件高度定制,其他组织很难采用;我们也担心,试图打造一个面面俱到的公开版本,最终可能并没有多大用处。Practically speaking, it’s difficult to offer Scion in a form that would be broadly useful to other organizations. It diverges significantly from git, meaning companies would need to migrate every piece of the development lifecycle, and it requires its own code review, code search, and CI. Our internal versions of these components are highly bespoke, so they would be difficult for other organizations to adopt, and we worry that attempting to build a public version that tries to please everyone wouldn’t actually be very useful.
不过,我们预计,在未来 6 到 12 个月内,这类问题会越来越常见。对于每一家使用 git 的公司来说,都独自面对类似挑战,将是对工程资源的低效利用。因此,我们认为最有意义的做法是分享我们的思路,让其他开发者能够在不偏离整体模型的前提下,将 Scion 中一些最成功的做法应用到 git 上。git 服务器不必像 git 客户端那样运作;新颖的服务器架构能够显著提升扩展能力。git 平台可以像 Scion 一样提供全面的服务器端 RPC,甚至可以在各个 forge 之间采用标准化方式,让 git 客户端能够与之通信。协商协议也不必与如今完全相同,而且 git 中各种稀疏和浅克隆功能也可以很好地组合使用。However, we anticipate that this class of problems will become more commonplace over the next six to 12 months, and it would be an inefficient use of engineering resources for every company using git to face similar challenges alone. Given that, we think the most impactful thing we can do is to share our thinking, so that other developers can apply some of the most successful parts of Scion to git without departing from the overall model. A git server doesn’t have to work the same way as a git client does; novel server architectures exist that provide significant scaling improvements. Git platforms can provide comprehensive server-side RPCs like Scion, perhaps even in a standardized way across forges that the git client can learn to talk to. The negotiation protocol doesn’t have to look exactly like it does today, either, and the various sparse and shallow features in git can be made to compose well.
最后,我们想感谢 Git 社区:近二十年来,他们一直在扩展这一平台,以服务数百万开发者。我们的贡献要小得多:分享我们对智能体式软件开发如何与这些系统交互的早期观察,以便与 GitHub、GitLab 和开源软件社区携手,满足开发者不断变化的需求。随着软件开发生命周期与人工智能的指数级发展交汇,我们都在共同面对这一切。Lastly, we want to acknowledge that the git community has spent nearly two decades scaling the platform for millions of developers. Our contribution is far narrower: sharing our early view into how agentic software development interacts with these systems, so that we can partner with GitHub, GitLab, and the OSS community to support the changing needs of developers. We’re all in it together as the software development lifecycle intersects with the AI exponential.
致谢Acknowledgements
作者:Nick G. 和源代码管理团队:S.G.、R.K.、Dan C.、Garrett H.、Ben L.、Colin M.、Joe P.。我们感谢 Vilas J. 和 Sergiu D. 在这段雄心勃勃的旅程中支持源代码管理团队,也感谢 Amir O. 早期为 Antique 做出的贡献。Written by Nick G. and the source control team: S.G., R.K., Dan C., Garrett H., Ben L., Colin M., Joe P. We’re grateful to Vilas J. and Sergiu D. for supporting the source control team on this ambitious journey and Amir O. for early contributions to Antique.