Factory 如何用 Claude 构建自主软件工程：从 Droids 到无人值守工厂

Brief Description

How Factory is building autonomous software with Claude

Table of Contents

• Intro
• From string theory to founding Factory
• The mission: autonomy for software engineering
• Lessons from being too early
• Droids and the "dark factory"
• A COBOL modernization with 90% less human effort
• When Droids need frontier models
• What Opus unlocks: less human oversight
• Shipping 300 things and rethinking what matters
• The Anthropic partnership
• What's blocking the dark factory
• Advice for would-be founders

Intro

I think that's some of the most human pieces of art or pieces of technology, rethink what we've been doing before and figure out this entirely new angle. And we have time to do that by using models like Opus and Fable, where they take on a lot of the work that used to occupy so much of our time.

From string theory to founding Factory

Host: Matan, super great to be here with you today. We were talking about this a little bit before we sat down, but you have such an unusual background. In 2023, there's this pivotal moment where on one side of it, you were a PhD student studying the AdS/CFT correspondence in 10-dimensional black holes and string theory. On the other side of it, you become a founder, of Factory. What happened?

Guest: I guess growing up in the Bay Area, the culture is startups. And I was a rebellious child. I was like, I want nothing to do with startups. I'm going to do the most opposite thing. I got into theoretical physics and did that for about 12 years. As one does. Yeah, right. That dividing line between March and April 2023, I think that was the culmination of realizing that while theoretical physics and string theory is extremely beautiful, as you've noticed, I was working on 10-dimensional black holes, and we don't live in 10 dimensions, nor do we live near a black hole.

But then I took a seminar in program synthesis, which we now call code generation, and that just completely nerd-sniped me. Head over heels, was obsessed with that. Then it just became very clear in 2023 that to solve this problem, academia was not the place. You had to go into the industry. You had to work with some of the world's largest enterprises, some of the messiest codebases where some of the code is 30 years old, some of it is brand new, some of it is in COBOL, half of it is in Python. Just this very weird dynamic situation is where the code generation problem becomes the most interesting.

The mission: autonomy for software engineering

The mission that we have in the very first deck that we ever made is the same as our mission today, which is to bring autonomy to software engineering. Software engineers are some of the best problem solvers in the world, but so much of their life, especially in a large enterprise, is not as high leverage as it could be.

Host: You had this conviction from day one that this is going to be an autonomous software factory, even though at the time, the models weren't really there yet.

Guest: Models are going to get better. They're going to be able to do more and more of these tasks. Before, we were selling the vision of agents, now it's like everyone knows.

Lessons from being too early

That 2-year period between 2023-2025 was your journey through the desert. I think you said elsewhere that sometimes being too early is the same as being wrong. What did that desert teach you?

Guest: I was obsessed with problems and solving problems. That was all I cared about. This is really the first job I've ever had. I knew nothing about anything relating to building a company. Whether it's hiring, engineering, building product, sales, marketing, none of it. If the timing was right and the world was ready, I was the wrong founder in my mind because I didn't know anything. We would have missed the opportunity. It's kind of a blessing in disguise because I think it allowed me the time to grow and learn the things that really matter to now scale.

Droids and the "dark factory"

You've mentioned this concept of these Droids working in a dark factory, lights off 24 hours a day, 7 days a week. How are you building the product now for that future?

Guest: As you get used to delegating tasks to agents that can go for days or weeks on end, you start getting used to the idea that there's something that you're not going to go and touch for a couple of days at a time. It slowly starts building that behavior change. Also, when they see that the output is actually meaningful, they start to build some of that trust. Trust is going to become all the more important. You're now having systems that are doing things without you looking, without you necessarily explicitly approving every time. Having a software factory that is always monitoring, seeing, oh, hey, we just deployed some change, and now the completion rate through this step has gone down.

Let's go and spin up a sandbox where we go and test and see why did that happen? We found a candidate solution. Let me go ping you and show you what might fix it, and then deploy it and monitor again. I think it's so important to meet our users where they are and respect their intelligence as developers. They know their business more than we do. And the best we can do is allow them to configure it and morph it to the silhouette of their organization.

A COBOL modernization with 90% less human effort

Do you have any hero examples of a Droid doing something really impressive or crazy end-to-end for a customer?

Guest: Every single bank has a backlog list of these crazy COBOL modernizations that they need to do. It's all like, no engineer wants to be assigned to this because that is like, "Oh, God, for the next year, my life is going to suck dealing with this." With one such bank, we did one of these modernizations. As it was working, it was extracting lessons. This is the way this bank does things. This is how we can learn from it in the future. That was one of our engineers sitting next to some of their engineers to go and do that first pass. Then after that, the next one, the amount of human involvement was 90% less. That to them, was a magic moment. There are tears of joy in these engineers' eyes because working on those migrations is the worst.

When Droids need frontier models

Where do you feel that the Droids need to be very, very smart, and they need the frontier?

Guest: If you have the CTO of a company implement every line of code, that is not an efficient use of their time. It turns out that typically we will have structure where there is someone who has maybe more context about a problem or more history of problem-solving, where they're going to come up with a general systems' solution. It's analogous here where, I myself, if I'm doing planning or something pretty complicated where I need to orchestrate a lot of moving parts, I'll use Opus.

What Opus unlocks: less human oversight

What has Opus enabled you to build with the Droids that you weren't able to before?

Guest: I think the biggest unlock, and this is the most important part for us, is reducing human oversight. That planning is a lot more difficult, or you'll only find out if you went awry, billions of tokens down the line. I think there's the gut sense of you can trust the plans a little bit more to the point where you literally might not even look at them, and you're like, yeah, sounds great. Let's go and do it. That's been pretty cool to see. Before, it was like, you see it, but you're really closely monitoring to now it's like… It's only a very crazy corner case where you'll go back and be like, wait, what did we do in the plan? It rarely, rarely happens. There are certain things that are the brutal reality of working at a company where there are just things that you have to do that need to meet a certain reliability threshold.

And what's cool is Opus and Fable, it takes a lot of those things that previously you had to spend a ridiculous amount of time doing, and it's like, "I got you covered there." Every company wants to ship 30 things, but they can only ship three and a quarter because of the resourcing that they have.

Shipping 300 things and rethinking what matters

Well, now you can ship 300 things. But will those 300 things actually make a difference? Are they going to actually make happier customers or better relationships with your customers or your end users? I think it shifts the question for the humans to now be like, what really is our core competency? How can we deliver on that better? That is the thing that's really hard because sometimes we put ourselves in such a box of like, this is all that we can do. To me, it's very exciting because I think that's some of the most human pieces of art or pieces of technology, rethink what we've been doing before, and figure out this entirely new angle. We have time to do that by using models like Opus and Fable, where they take on a lot of the work that used to occupy so much of our time.

The Anthropic partnership

When you think of Anthropic and you think of that partnership, what's top of mind for you?

Guest: I think the biggest thing is reliability in terms of how we work. Understanding that there are a lot of things that we have in common in terms of what we're excited about, and making sure that it's set up such that like, hey, here are the important things we should spend our time on. Here are the things that you guys should spend your time on. We very clearly want to see a world where human potential is unlocked, and this ambition is raised, and there are certain constraints that are required. There're some safety constraints that are required. There's some messaging that really matters. Sometimes these tools can be scary. They're very powerful, and that is inherently scary. But it's also a beautiful thing for the world that we should highlight the benefits that are coming from it and how Droid can be better with Claude and how Claude can be better with Droid.

It's really important that the people that we partner with take it as seriously as we do, as a today problem. Some people are like, "Oh, eventually it might be crazy." It's like, no, today this is extremely important across every piece of this AI engineering puzzle.

What's blocking the dark factory

Looking ahead a year in advance when you have these dark factories. We're not fully there yet. What are the blockers? What are the bottlenecks right now?

Guest: The way I think about it, there are three concentric circles. At the core, there's the model, then there's the harness, and then there's the company and the workflows of the company as a whole. These are all growing very quickly. The model capabilities now versus 2 years ago, it's insane. Completely, completely different. A model today could do what a model plus a harness could do 2 years ago. It's a cool call to arms for everyone to raise the scale of their ambition. Because as models get better, if you are lazy, and you don't change your harness, it's going to be superseded. As these models get smarter and smarter, they start to absorb more of the harness.

How do you build for that? Then also, how do you think about the leverage that type of really intelligent model can give you? We cannot settle. Things are moving way too quickly. We need to always have in one hand a sledgehammer to whatever we thought was the best solution. Every month, we need to rethink. Is this still optimal? As these models are getting better, and also having a deep partnership, understanding and having this relationship where we can see, hey, these are the types of problems they're thinking about. We can trust, okay, this is going to be going up there. Let's make sure we're filling this hole in the meantime, so that we can ride that wave together.

Advice for would-be founders

I want to go back to the very beginning. We were talking about your pivotal moment that led you onto this path. There are a number of would-be founders watching this, and their pivotal moment is right ahead of them. What advice would you have to those founders?

Guest: This is a bit funny, but anytime a person asks me if they should start a company, I always say no. Because if they need my affirmation, you know. I just like really getting obsessed with problems. Then everything is just so much fun. Find the thing that you literally cannot stop thinking about and just keeps you up at night. The beautiful thing about the era in which we're in now, normally you could say, oh, the thing that keeps me up at night is I want to be an astronaut, or whatever. Fifty years ago, it's pretty unrealistic. It is extremely realistic today.

Even if you have the most out there crazy ideas, and that's the only thing that keeps you up at night, it's actually way more achievable than it's ever been. Find the thing that you get obsessed about, the intellectual curiosity just pours out of you, and I think the rest will come from that. Thank you so much. This has been an amazing conversation. I really appreciate the time. Thank you for having me. This was fun.