AI 编码时代的基础设施：Blacksmith 如何加速 CI

简要描述Brief Description

面向 AI 编码时代的基础设施Infrastructure for the AI Coding Era

目录Table of Contents

• 利用游戏 CPU 加快 CI
  Making CI Faster With Gaming CPUs
• 赢得投资者和开发者的青睐
  Winning Over Investors and Developers
• AI 编码时代的 CI 规模化
  Scaling CI for the AI Coding Era

利用游戏 CPU 加快 CIMaking CI Faster With Gaming CPUs

每当开发者编写代码时，都必须运行一系列检查，确保没有任何东西出问题。这个过程叫作 CI，而 Blacksmith 编写基础设施，让这些检查运行得非常非常快。我们在过去一年增长了 20 倍。Every time a developer writes code, they have to run a bunch of checks to make sure that nothing is broken. That process is called CI and Blacksmith writes infrastructure to run those really, really fast. We've grown 20x in in the last year.

我们最近完成了C轮融资。本轮融资由Seoia领投，公司估值达10亿美元，融资金额约为6600万美元。我们当时是局外人。We recently raised our series C. This was done at a billion dollar valuation led by Seoia and we raised about 66 million. We were outsiders.

我们不是平台工程师，也不从事 DevOps。事实上，我们之前都没有编写过 CI 流水线。We were not platform engineers. We didn't work in DevOps. In fact, none of us had written a CI pipeline before.

当我思考为什么我们是少数几个以这种方式着手解决问题、并这样思考问题的人之一时，我觉得答案的一部分在于，我们恰到好处地保持了一些天真，没有在一个好点子进入市场之前就把它扼杀掉。所以如今，借助智能体式编码工具，开发者写的代码比以往任何时候都多。但如果你无法以同样的速度交付，那么写出多达 100 倍的代码也没什么用。When I think about why we were one of the only ones to to approach the problem this way and to think of it this way, I think the part of the answer here is just being naive enough in the right way to not kill a good idea before it meets the market. So today with um with Agentic coding tools, developers are writing more and more code than they ever were. But it's not very useful to write 100 times the code if you can't ship it at that speed.

所以现在，瓶颈不再是编写软件，而是验证软件。这通常意味着要等待 CI。也就是说，你得确保测试通过，并且确信自己交付的东西确实能正常运行。So now the bottleneck has become not writing the software but validating it. And that usually means waiting on your CI. So that means making sure that the tests are passing and that you have confidence that what you're shipping actually works.

我一直想来硅谷创业。上高中时，我最喜欢做的事就是在维基百科上到处浏览，阅读科技公司的相关信息，了解它们是如何创立的、做些什么。我也想做点属于自己的事。其实我去了滑铁卢大学，因为那里的创业项目相当不错，而且很多创始人都毕业于滑铁卢大学；这是一所非常偏重技术的学校。I always wanted to come to Silicon Valley and start a company. When I was in high school, my favorite thing to do was surf Wikipedia pages and I would read about technology companies and how they were founded and what they did and I wanted to do something of my own. I actually went to Waterloo cuz they had a pretty great startup program and a lot of founders came from Waterloo and they were very technical school.

在我开始在 Fair 工作后，我就开始大量思考各种想法，还有、还有、还有我该如何行动。实际上，Blacksmith 是我第二次申请 YC。我们三个人是本科时的朋友。当时我们都在多伦多，也都想创办一家公司，于是围绕着我们认为应该做什么展开头脑风暴，而我们全都是工程师。After I started working at fair, I would start thinking a lot about ideas and and and how I can move and actually blacksmith is my second application to YC. The three of us were friends from undergrad. We were all in Toronto at the time and we all wanted to start a company and we were riffing ideas around what we thought we should work on and all of us were engineers.

我们几乎在每家公司工作时都觉得 CI 糟透了，主要是因为它特别慢。但管理这部分基础设施到处都是件麻烦事，而我们觉得自己有更好的办法。最初讨论后来发展成 Blacksmith 的那个想法时，我们其实有点怀疑它是否行得通。但随着时间推移，做了更多研究之后，我们的信心也越来越强。Pretty much at every company we worked at CI universally sucked and it sucked because it was super slow but managing this piece of infrastructure was a pain across the board and we felt we had a better way. When we initially talked about the idea for what became blacksmith, we were initially kind of skeptical whether this could work. But over time, as we did more research, our conviction continued to grow.

我的联合创始人都是数据库工程师。我当时在做搜索基础设施。但从外部来看，我们多少看出，管理 CI 信息之所以这么难，是因为这种工作负载的形态与 AWS、GCP 和 Azure 等超大规模云服务商真正擅长处理的工作负载大不相同。My co-founders were database engineers. I was working on search infrastructure. But from the outside, we sort of saw that the reason managing CI info was so hard, the shape of the workload is very different from what hyperscalers like AWS, GCP, and Azure are actually really good at.

这几乎让人觉得，造成痛苦的是一种不匹配。我们觉得，真正的出路在于构建一个非常擅长运行 CI 工作负载的云平台。归根结底，就是为具体任务选用最合适的工具。And it almost felt like a mismatch that caused the pain. And we felt like the real way out is building a cloud that is extremely good at running CI workloads. It comes down to using the best tool for the job.

我们逐渐意识到，CI 工作负载的特性和桌面应用程序，尤其是你在普通游戏电脑上运行的电子游戏，非常相似：它们不太在意有多少个执行线程，更在意单个执行线程能不能跑得非常非常快。起步时，我们会从一家德国数据中心服务公司租用服务器，这家公司以托管《Minecraft》和《使命召唤》游戏服务器而闻名。随着规模不断扩大，我们开始逐步掌控技术栈中越来越多的部分。What we started to realize was the characteristics of CI workloads are a lot similar to you know desktop applications in particular video games that you would run on your typical gaming PC in that they don't care as much about multiple kind of threads of execution but they care about that one single thread of execution being really really fast. When we started off, we would rent these servers from this German data center services company that was known for hosting Minecraft servers and Call of Duty game servers. Over time, as our scale has grown, we've started to own more and more of the stack.

所以现在，我们某种程度上是在更大规模上与数据中心服务商签订协议，实际上购买的是兆瓦级的电力，而不是单台机器。So now we sort of operate at at, you know, much larger uh levels of agreements with data center providers where we're effectively buying megawatts of power rather than individual machines.

赢得投资者和开发者的青睐Winning Over Investors and Developers

所以我实际上投资了 Blacksmith 两次。我领投了他们从 Y Combinator 毕业后的首轮融资，当时用的是 SAFE；几个月后，我又领投了他们的 A 轮融资，并加入了董事会。我觉得他们用游戏 CPU 这件事特别有意思，因为我以前从没见过有人尝试用这种方案来做 CI。So I actually invested in Blacksmith twice. I led their initial round out of Y Combinator which was on a safe and then a few months later I actually led their series A and joined the board. I think it was super interesting that they were using gaming CPUs because I'd never seen anyone else try that solution for CI.

甚至让我有点震惊的是，居然有人会把这么多这样的东西放在同一个地方。这才是有点奇怪的地方。通常你会在家里有一台电脑，就是那种单机，你知道，桌子底下那台大主机，然后你就玩个什么游戏之类的。It was even kind of a shock to me that somebody would even put a bunch of these things in one place. That was sort of the weird thing. You usually would have it at home for your like single, you know, your big box you have under your desk and you're like, you know, you're playing whatever game.

但没错，就像是：“不，我们得买个一百台这样的设备，用它们来编译。”我当时就觉得：“哦，这确实有点不一样。”让 CI 变快，很大程度上就是跳过之前已经完成的冗余工作。从宏观上说，开发者把这称为缓存。Blacksmith 的一大独门秘诀，就是把缓存产物移到靠近计算节点的地方。But yeah, being like, "No, we need to have buy like a hundred of these things and use them to compile." Like that that's where I was like, "Oh, that's that's kind of different." A large part of making CI fast is skipping redundant work that has already been done before. At a high level, this is what developers refer to as caching. A big part of Blacksmith's special sauce is moving cache artifacts close to the compute nodes.

最初的 200 位客户特别难争取。一开始，我们其实是请在各家公司工作的朋友给我们一个机会。我们向他们征求展示公司标志的许可，之后就能把这些标志放到网站上，积累一些可信度。借助这一点，我会亲自给 CTO、工程副总裁等目标人群，以及平台工程师打外呼电话，努力让他们使用我们的产品。The first 200 customers were extremely hard. Initially, we actually asked our friends who were working at companies to give us a shot. We asked them for logo rights and we were able to put these logos on our site and build a bit of credibility and using that I would personally called outbound CTO's and VP of engineering personas and platform engineers and try to get them to use the product.

我们最早的客户和我们一样，都是小公司。他们很看重快速推进，因此希望 CI 也能跑得非常快，而且流程上的繁文缛节也很少。真正赢得工程师认可的唯一办法，就是拿出出色的表现，再次强调，就是以一种他们自己可能做不到的方式解决问题。The earliest customers we had were small companies like ourselves. They cared about moving really fast and so they wanted really fast CI and there was also very little red tape. The only real way to win over engineers is to really perform and again to solve a problem in a way that they probably wouldn't be able to do themselves.

你知道，他们眼光很挑，也喜欢尝试各种新工具。有时候，他们并不是最容易留住的客户。You know, they're very discerning. They like to try a bunch of new tools. Sometimes they aren't the stickiest customers.

在没有任何销售团队的情况下，他们就拿下了金额相当可观的五位数、六位数合同，这全是因为 Blacksmith 让 CI 优化和高速运行变得如此简单。我们让这一切变得非常简单。只需点击一下，你就可以把所有 CI 都切换到我们这里。Without any sales team, they were able to get kind of large five six figure contracts because of just how easy Blacksmith made it to optimize and run your CI as fast as possible. We make it really easy. In one click, you can switch over all of your CI to us.

最近接入 Blacksmith 的客户之一是 Langchain。此前，Langchain 在 GCP 上运行 CI。在 GCP 上，他们的开发人员通常要等 15 到 20 分钟，才能获得运行 CI 作业所需的计算资源。One of the customers that onboarded onto Blacksmith recently was Langchain. Langchain was previously running their CI on GCP. On GCP, their developers would routinely wait 15 to 20 minutes to even acquire compute to run the CI job.

在 Blacksmith 上，由于我们实际上提供了无限的计算资源池，等待时间从 15 到 20 分钟缩短到 10 到 15 秒，大幅提升了开发者提交代码的效率。Blacksmith 目前的一项重要工作，是让更多人知道：只要任何工程团队考虑 CI 效率，或是如何信任他们生成的代码，他们就会立刻想到 Blacksmith。为此，我们正在旧金山和纽约两地推出大规模营销活动。On Blacksmith, because we effectively expose an infinite pool of compute, this wait time went down from 15 to 20 minutes to anywhere from 10 to 15 seconds, significantly improving the velocity at which their developers could push code. A big push for Blacksmith now is to get the word out that when any engineering team thinks about CI velocity or gaining trust in the code that they're generating, they immediately think of Blacksmith. To this effect, we are rolling out a massive marketing campaign across both San Francisco and New York.

我们将在纽约包下15个地铁站，还会把一列地铁列车整体装饰成我们鲜亮的黄色。我加入时，团队大约只有五个人。We're taking over 15 subway stations in New York. We're wrapping a subway train in our bright yellow color. When I joined, the team was about five people.

我们当时运行着一个小型集群，大约有几百台服务器；而今天，我们看到的规模已经大了好几个数量级。现在，我们团队有30人，其中12名工程师。We were running a small cluster of a couple hundred servers and today we've seen kind of many orders of magnitude more scale. Now we're a team of 30 people, 12 engineers.

为 AI 编码时代扩展 CI 能力Scaling CI for the AI Coding Era

归根结底，这取决于我们已经做出、并且会随着进展继续做出的一些关键决策。我们有很多定制软件，其中一项就是管理车队规模的增长。It boils down to a few key decisions we've we've made and kind of continue to make as we as we go along. We have a lot of bespoke software. One to just manage the growth of our fleet.

我们在技术栈的每一层都构建了大量定制化的可观测性能力。其次，我们严格控制产品的功能范围，确保它精简、易于管理，并且只随着团队规模扩大而扩展，而不会因为某个客户的定制需求就不断增加功能。专注于此带来的巨大回报是，我们只靠一支很小的团队，就能扩展到管理数量极其庞大的机器和客户。We have a lot of kind of bespoke observability at every layer of the stack. And secondly, we are ruthless about keeping the surface area of the product small and manageable and only scaling it with the size of the team rather than giving into one of bespoke customer requirements. And a big payoff of that kind of focus is that we're able to scale uh to a very large, you know, number of of machines and a very large number of customers with just a very small team.

因此，最大的挑战之一就是跟上需求。我们发现，每次有新的 LLM 发布，CI 使用量都会大幅飙升，因为大家都争先恐后地试用它，并交付更多代码。由于订购和安装机器需要时间，我们必须在技术层面想办法应对这波负载。So, one of the largest challenges is keeping up with the demand. One thing that we're seeing is every time uh new LLM drops, um we see a massive spike in CI usage as people rush to test it out and ship more code. So because it takes time for to order and install machines, we have to get creative technically to support that load.

这意味着我们需要更好的算法来安排作业，更巧妙地提高作业打包效率，并采用更高效的网络拓扑。因此，在夏初我们宣布完成 B 轮融资后不久，凤凰城发生了一次电涌，导致我们在当地运营的一座大型数据中心的主冷却系统和备用冷却系统都失效了。在停电期间，我们的客户受到影响，作业启动时间变长了。So that means better algorithms for placing our jobs, getting uh more clever around how we can pack jobs more efficiently, more efficient network topologies. So, right after our series B announcement early on in the summer, a power surge in Phoenix took out both our primary and backup cooling systems in one of the biggest data centers that we operate in Phoenix. Our customers were affected with, you know, longer startup times for their jobs during that window of outage.

这凸显了 Blacksmith 采取地理分布式布局的重要性。我们希望帮助开发者对自己的代码变更充满信心。为此，我们正在开发的一款产品是自主 QA：收集截图或视频片段等产物，以增强大家对变更的信心。That underscored the importance of being geographically distributed for Blacksmith. We want to help give developers confidence in their code changes. And as part of that, one of the products that we're working on is to do autonomous QA where we gather artifacts like screenshots or video snippets that induce confidence in a change.

随着代码产出量不断增加，开发者将没有时间逐一确认代码是否正常运行。他们会转而寻找证据，而 Blacksmith 能让他们更有信心、更快地合并代码。我们希望消除这一瓶颈，让开发者交付软件的速度尽可能接近他们的思考速度。With the amount of code that's being produced, developers won't have time to go and try to understand if things are working. They're going to look for evidence and Blacksmith is going to give them confidence to merge their code faster. We want to remove that bottleneck and allow developers to basically bring shipping software as close as they can to their speed of thought.