斯坦福 CS146S：常驻智能体如何持续工作

Content

So, as mentioned, I'm Lee. We're going to talk a little bit today about how this new category of always on proactive persistent agents work and kind of give you some of the understandings of the building blocks for how these are made. Um, just as a, you know, fun statement up here, some of the things we'll talk about are in progress. We're working on at SpaceX. Um, and in week one, you all built this very kind of simple harness that helps you understand the core idea, which is that you have a model. It can call tools in a loop. And that's actually very powerful. It unlocks a lot of new capabilities for the models. Um, but it does come with some drawbacks. Notably, you close your laptop and the process stops. It's no longer running. Uh, and I want to talk about how you can take this simple idea, it's very powerful for how to get more out of models and extend it to this new type of agent which is an always on agent.

So rather than just running in your terminal, this can run on a server, it can run in the cloud, you can close your computer, uh, it can restart, it can survive crashes, for example. This agent has its own computer. So not just your laptop, but a Linux machine in the cloud where it has its own desktop files, browser, and more. maybe some 3D apps. Um, it can save files on the machine. It can understand more about your progress over time and build up a memory of how you work with it. Um, unlike a harness in the terminal that is responding to every message that you send, this type of always on agent is more like a colleague where it doesn't always immediately send you a message. It knows when to be quiet. Sometimes it might just want to work in the background a little bit and only come back to you when there's an important update.

Uh, and it can be triggered more than just you typing. So, it's not just you prompting it in the terminal. Maybe there was a Slack message or an email or some other type of event that wakes up this agent to go do work. Um, and ideally the hope with these type of always on agents is that they can do work over very long periods of time. Not just hours or days, but maybe even weeks or months. And that just requires a little bit of a different architecture, which we'll get into. So, we're going to go uh decently in depth on how we built this architecture, how we built the harness, um kind of what's got us here today and where we're going in the future. And we'll get into some of the specifics of how we've engineered this system to be proactive, to be persistent, and to follow instructions very well.

And of course, time at the end for questions if you all want to ask anything about this or uh anything outside of uh what we've covered today. So, first, how we got here. There's kind of been four eras of developers working with AI so far. I would say the first one was uh copy pasting code out of chat GBT and putting it into your editor of choice which feels like you know decades ago at this point but it was really only a few years ago. Uh and then once the model started to get better at being able to have kind of that simplified harness we talked about uh in you know in the last uh lecture uh they were able to edit files, run shell commands, kind of have this very simple interface for how you work with them in the terminal. And this really took off because all of a sudden the model had this new capability where it could go do more proactive work for you and be more you know productive.

Then this third era was taking the coding agents and where they were kind of simple to start and giving them these dedicated apps. And at this point, people were starting to run many agents in parallel. Maybe they're running them in the cloud, not just on their machine. Uh but still at this point, you were kind of the bottleneck in that you were reviewing every command that they were running. You wanted to make sure every shell command was just right. You're kind of really watching them and being in the loop on everything. And now with improvements in models, in harnesses, in products, we're now in this new era where you're starting to have these always on agents and they can kind of run ambiently in the background where they have their own computer. They have their own memory. They learn new skills on the job and they keep working after you go, you know, go to bed or go do other work.

A good example of this is we've kind of moved away from this world of prompting an agent every single day. Although we still do that sometimes, but you can set up agents where it will kind of just run in the background. So maybe in the the before times it was you would prompt an agent, you would wait. You prompt an agent, you would wait, and if it asked you to run a shell command, you'd have to approve or, you know, disallow that command. Uh and now instead of kind of queuing commands, you can steer the agent and you might say, "Hey, go check this thread." and then immediately after, oh, by the way, you know, actually go do that on Thursday. And prior generations of models would get kind of confused when you would immediately interrupt it and give it some other command. But now they kind of understand just like how you might ramble on talking to a friend and they can kind of parse that these two things are related.

The models and the harnesses can now handle that. And if you say, "Hey, go send this email, go send this Slack message." Um there are additional safety checks in place which a lot of companies are calling autoreview which is essentially a separate AI model that's checking all of the commands that get run. So you know kind of funnily enough that's actually more secure than the fatigue of having to watch hundreds of shell commands and approve each one. We've kind of reached the point where if you did a blind test with what the AI model's checking on the commands and what a human's checking on the commands. Most of the time the the model's actually doing a better job. um kind of paradoxically, but as the models have gotten better, the interface has gotten quite a bit more simple actually. And this type of always on agent we're calling a bot.

But as you'll notice, there's actually quite a few of these new types of products popping up right now and gaining a lot of traction. And I think the main reason is because we all know how to text. You know, everyone knows how to text. Everyone is pretty proficient with using their phones. And we've kind of distilled down this way of working with AI that feels like you're just texting a friend, but it can actually go and do real work for you. It's not just answering questions in a Q&A style setting anymore. It's actually going and doing longer tasks like you would hand off to a co-orker, for example. Um, so there's been a few changes specifically in the models that have enabled this to happen. Um, I'll go over six, but there's, you know, quite a bit more. The first one is that the models just over the past year or two years have really gotten better at being able to follow instructions over a longer period of time where they don't get confused or make mistakes and now models can pretty confidently run for hours or days with the right uh compression and compaction which we'll talk about in a little bit.

Two, models now can essentially hand off to helpers and those helpers are called sub aents. And the reason why a sub aent is helpful is the the model has kind of its working memory of the process that it's on and that's going to fill up eventually. Uh so it's nice if it can hand off to these helpers that have a fresh set of working memory and that effectively can give it unlimited context basically where it can go and do work that might be expensive in terms of the tokens it's using or the tools that it's calling and then come back to the main agent and report its updates. And I'll have some diagrams of that later. Um, models now not only are just better at calling tools. I mean, just a few years ago, they basically couldn't call tools very well and they would hallucinate all the time. It's funny, you don't really hear people talk about hallucinations anymore because it's kind of a solved problem mostly.

Um, but not only can they call tools, but you can give them very powerful tools the same way as if you onboarded somebody onto your team. So, obviously running shell commands and files, but also you give them a full computer, a full Linux uh computer. uh you allow them to make demo videos and such. Um they can click around on a screen just like a human would and they're getting increasingly better at doing that, otherwise known as computer use, which basically means that any SAS product or any website that didn't expose an API or an MCP server can be used headlessly basically so autonomously through uh computer use by these models and these bots. um you can store all of the information in files and the reason you can do that is because it's actually very easy for the models then to recall and search this information which we'll talk about and then finally you can just kind of show the bot how to do some work and it will kind of turn that into skills or memories or routines or things that it can reuse later.

So on the file thing for example, this is kind of a simplified example, but you might remember or you might uh imagine that when you're working with a bot under the hood, it might look something like this. You build up this profile or this understanding of the user and that's going to be included in every single prompt. Maybe you like, you know, lowercase text like messaging and every single message should have that. Also, as you're working with these bots over time, you're building up a kind of like a working log, just like if you were working with a colleague and they were writing down notes every day of the work that they've done. That's all being put into files that the model can then decide to search or use later. If you say, "Hey, what did I do last week?" It just can can go and look that up.

And the models are very good at using Linux commands, running shell commands to go grap and search and look through these files. Um, similarly, um, with skills, you are essentially taking your knowledge of how to do a specific task and encoding it into a markdown file. The very specific process you need to do the things that you believe make that uh you know successful or unsuccessful. And that's not just your skills. There's a whole ecosystem now of companies and developers um writing these skills for how to send a great email or you know how to build a great API or many different things. Uh and and for example in Grockbot there is a whole system of plugins. So basically any tool that you want to use there's a plugin you can install which uses skills or servers and then also routines which if you're familiar with cron jobs you know you're running some prompt basically on some kind of schedule which is pretty useful as well.

So basically everything is computer um part three. So let's go a little bit further into the kind of the mind of this always on agent or going behind the scenes. Um what does it mean for the agent to be always on? You can kind of read this chart from left to right, but it starts out with the server being asleep and also this computer that the agent can use being asleep. And so whether you send in a message or some kind of trigger uh decides to make the agent start up, which we'll talk about in a second, that's basically loading up and booting up all of the state for this agent. And it's running on a server. if it decides that it needs to use a computer, maybe it wants to go use the computer's browser or, you know, go click around in the computer, it can start up the computer as well. And then once you're done using it, it also has the ability then to basically put those to sleep.

Why is this important? It would be, you know, expensive and also uh there's not enough computers in the world to just have those computers running all of the time uh for everyone as well as that's a lot of server usage too. And so this state can then be persisted to you know a database the the conversation logs the memories the routines all of that. Um how does the agent get woken up then? Well obviously you can prompt it directly but I think what's increasingly interesting is having other things running in the background that can then decide when you want to go reach out to your bots. Uh maybe that is somebody pinging you on Slack and your bots can live in Slack. Maybe that's a phone call or a meeting that you put your bot in. It's increasingly common now at SpaceX where we see bots joining meetings for employees where they couldn't make it to the meeting, but they want their bot to go and just take notes and come back to them, which is kind of funny.

Um, maybe you had some routine or some background work that was taking, you know, hours or days to do and then you want to come back and give you a summary. That all happens. Then we have basically a router that ddupes that and figures out how to put it into the conversation. And then it's has this heristic of do I need to bug the user about this or not? Because not everything is actually important. Just like if you have a co a co-orker working on something, it would be pretty annoying if every single time they made progress on something they came and bugged me like, "Hey, check it out. Like I made progress on this thing." You really only need a status update maybe, I don't know, every day depending on the task. Um, so I talked about how the agent has this computer, this Linux box. Um, this is running a firecracker virtual machine.

And the reason this is really helpful is because it's it's very easy then to essentially snapshot the state of the machine and shut it down and start it back up. And on this machine you of course have files, but you can kind of connect into the machine and have a live view of the machine and actually control it yourself if you want. Um the machine can of course run commands which is great if you're going to run some shell commands. Maybe you don't want it on your personal laptop. You want it in this other environment that is you know secured and locked down. And then another interesting thing is of course it has Chrome and it can use a browser but you can also make it essentially have like virtual desktops. So if you're running five bots, 10 bots and each one of them needs their own browser you can essentially let them have different screens.

And all of this again is kind of shared infrastructure between these [clears throat] bots where they can go call on these computers. Um so we talked about kind of the the triggers that will wake up the agent that's kind of on the left. So different apps or Slack messages or emails. Um then you have this middle piece which is the server and then on the right is the computer that we just talked about where you can run files and commands and other things. So let's talk a little bit more about the server. I already mentioned you know commands will come in they get put into this queue and then it calls the purple box which is the agent loop and this is what we covered last week like this is a model calling tools in a loop and then when you shut it down you can kind of save the state to the database or storage.

Um, so it's like it's it's interesting, I think, to look at this and realize there's this one kind of small piece that can then have all of these abstractions built on top to create uh a very powerful product uh when you give it all of these helpful tools and you connect it to all of these different services. The reason agent runs on the server is uh a pretty common practice in computing where you want to have kind of a thin client and a thick server. And computing has kind of oscillated between these things over the years. But for example, a few good reasons for this is let's say you need to push an update to the, you know, the agent code. It would, you know, kind of suck if you had to push an app store update just to get that code out there versus just pushing a bunch of server updates very quickly.

Um, also sometimes you need to do heavy computation work, you know, like transcription or image generation or something. And that's probably going to be faster on a server than on device depending on uh what the devices are. Um, and you know, it's there's very established ways that we know how to scale servers, especially with variable traffic and variable growth. Um, but the most critical thing here is that if you put the agent loop on the virtual machine, then you can't shut down the virtual machine independently from the uh from the agent server. So yeah, you want to save everything on the server. Um, so you can always kind of rebuild that virtual machine. For example, let's say you need to do some critical zero day Linux uh update on the on the VM. Well, it would be great to be able to rebuild that and not, you know, blow away the entire agent loop and have downtime in the application.

Um um when you hit send, this is maybe a little in the weeds, so I'll just kind of gloss through it, but basically um if you want to dig more into learning more about distributed systems and thinking about um you know, strong versus eventual consistency, which are just fun topics to get into, you know, you send a message and you kind of want the client to immediately identify that and acknowledge it. Then when you go into the turn, you're obviously putting these um tools in a loop with the model, saving that to the database, and then the clients are essentially putting subscriptions with all of the servers. So they know if something changed and they're listening for these signals, and if they get a signal, then they can go and essentially fetch the latest information from the server. And there's some fancy dduping you can do to make sure if you know if there were some kind of retries um that you would not show, you know, multiple messages in the the list.

And so, uh, the like underlying infrastructure primitives to think about here, um, is a workflow versus a job queue. And the main difference here is with a job queue, let's say you're kind of going along on steps, you ask your agent to go and do some kind of complicated task, uh, and it's making its way from step one to step two to step three, but then for some reason there was an infrastructure issue. Um, there was some problem with a normal queue. If that crashes and you go to restart it, well, now you have to replay steps one, two, and three, which is obviously that that could be pretty problematic. So, what you really want here is you want a durable workflow, which means that it's safe for you to actually retry that. And when you retry after a failure, it picks right up on step three. And this is like kind of a solved problem in some ways in distributed systems and in infrastructure.

So, we use an open source uh tool called Temporal, which is interesting to look into if you're if you're curious, so that we don't have to rebuild a lot of this very complicated stuff from scratch. Of course, you still have to run the infrastructure for it, but some of the pieces to put together there. Um, so let's say a new message comes in. There's different priorities based on whether I'm messaging my agents or messaging my bots versus if it's in a group chat or a Slack thread or some background routine for example. So if I say oh actually you know change the date for this event, we actually want that agent loop to stop and my message is going to have priority over everything else that's happening which kind of makes sense. Um, for everybody else, you're kind of putting things into the queue in the back of the line.

And if you're if the the agent gets, you know, three or four messages very quickly, it's able to kind of batch those together. So, it's just one turn. You can kind of figure out whatever heristic you want there. Um, and then it kind of goes through and does each turn at a time. Okay. So, let's talk about the the harness itself. And this is going to kind of take the the basic harness that we talked about last week and add some some new tools, some new concepts to it. The biggest difference with the Grockbot harness um and probably some of the other tools like this uh on the market is the way we've done the thin client fix server architecture is there's only one tool that communicates between the client and the server and it's just send user. And the great thing about this is that it's actually very architecturally simple where the client is just texting the server in some ways and you're putting a lot of these heavier tools on the server.

And there's many of those tools. There's running shell commands, reading files, but also calling on a cloud agent or recording your screen or doing demos for example. Um, and this I I think has worked pretty well because now regardless of where you spin up a new client, whether it's a mobile app, a server, um, a desktop app, email, Slack, it all just has this one single tool that communicates back and forth with the server. On the client though, even though it just has this one tool, when it receives the tool, it can decide how it wants to display that in a rich and interactive way that makes the most sense for the user. So if it gets uh information back from the server that the user is trying to log into something, it can kind of show these unique sign-in forms and connect to tools like one password.

It can go and call on services like uh stripe link or use X money or uh other services to go and allow people to pay for their services. You can if you're trying to do something that is non you can't reverse it sending an email or a Slack message, it might come back and ask you to you know edit and approve a draft before you send it. And then sometimes it just you know you can just leave an emoji reaction if you don't actually need to send a message. Um so going back to what I was saying with dynamic tool discovery. So for uh for any model there's a context window which is kind of its working memory and that's some limit in tokens. And if you just jam every single tool possible into that uh context window, you're not going to have a lot of space. And the way the tools work is you have kind of the the name of the tool, you have the schema, you have all these different details.

And if you start installing your Google Drive connector, your Gmail connector, your YouTube connector, like all these different services that use MCP servers, which all have tools, you're going to start looking at something like the top where a lot of the blocks are already filled. Uh, and the problem with that is it doesn't give the model as much space to actually do the work of the conversation. Um, so one kind of context engineering or harness trick um that a lot of the the popular harnesses now use is uh it's called dynamic context or I think uh some other folks call it the tool search tool which is a fun name but basically you're only putting the names of the tools into the every message that gets sent to the model and then the model can decide okay I actually want to go use this tool.

I can go read the information from a file. It's kind of files and a file system all the way down for all of these things. Uh, and that just helps you save on your context usage, which then also helps save on cost, save on usage, just better for everybody. In general, the principle here really is to try the cheapest option or the most reliable option first and kind of work your way up the ladder here. So, the easiest one is just what you know what the agent or what the bot already knows based on the conversation history. uh if it doesn't if it's not in the immediate context, maybe there's a plugin, an MCP server, some kind of API that it can go and pull, you know, your uh your banking information from Plaid, for example. Uh maybe that doesn't work and it needs to go do some kind of web search to see, you know, what was the score of the Cubs game last night.

Um maybe if it doesn't have that, it actually needs to go to the computer and open up the browser and do some searches on there. And finally, like maybe it needs to go and actually use the entire desktop. So run scripts or run commands or build things on that Linux computer. And only if all of those fail, then go bug the user and ask them to go give input or approve something. And this really helps it feel like what it would be like to work with a colleague is like they're not going to bug you for all these different things. Ideally, they're going to try a lot of these options first. um on that computer. So the the Linux machine that the agent is able to use, there's kind of three different ways that it can retrieve information from it. Um of course it can just take a screenshot of the screen, but there's more efficient ways of doing this.

Um it could also look at the HTML of the page, but you can also look at the accessibility tree, which is a nice way of like cutting down a lot of the uh token usage and basically to help you find the buttons or links or other elements that you want. uh the browser to click on. Uh, of course, if none of that works, you can just click on pixels. So, some websites, for example, have modals or dialogues that pop up that you need to click. Uh, and so your model, your agent needs to have that functionality as well. Or again, kind of worst case on that ladder is you go back to the user and like, oh, there's like a two-factor off code or there's some kind of capture or or something like that. Um, so the idea is that kind of putting the pieces together, you have this main agent and it's kind of like your orchestrator and the main agent you want to keep the conversation as sparse as possible because it now has the ability to go call on all of these different helpers.

And the helpers are just sub agents. And so this is helpful because if you hand it off to a helper to go run a bunch of shell commands or do a lot of expensive tool calls, all of that work that ultimately is ending up in some answer to a question or a task. None of that is bloating the conversation in the context of the main agent. It's only coming back with the result. And this is how you get to a product that feels like it has infinite context. Obviously there infinite context is not a real thing. Every model has a context limit. But there are these tricks and workarounds such that especially when you train models to get better at this that it doesn't feel like you're running up against the limits of the context window and you don't feel a degradation of the model's quality as you get further along that as well.

So we have these different helpers each with different skills. Each one of these is its own durable workflow. So, going back to the cues versus workflows thing, that means if it fails, the video helper crashes uh something, you know, uses up too much memory or something that can restart and keep going without losing progress along the way. And then the main agent, it is kind of like a co-orker where it's managing a to-do list. You have a few things you can do. You can go check on it. You can send it a message, you know, you can stop the helpers, for example. Um, and so a lot of this is modeled after the ideal way you would scale, you know, a team of engineers working on a very complicated problem. So, uh, like just a few little interesting tidbits from building this out on the team. Um, I think we'll talk a little bit more about prompt caching here, but um, ideally you don't add or remove new tools during turns.

Um, and just like I mentioned with the slide with all of the uh boxes, if you have extremely long tool descriptions or schemas, that's going to take up a lot of space in the context window, which is bad for pricing. It's bad for usage. Ideally, you want to minimize that. So, there's plenty of optimizations you can make here on the engineering side to just trim that down. Um, another interesting one is like you might hear people talk about how, oh, like we can have a harness, it just has one tool and that one tool is like running shell commands. And this is true and you can model a lot of things this way, but if you see the model doing, you know, 90% of its time doing shell commands for this one thing, it actually can be more efficient to make a dedicated tool for some of that stuff. Um, and it's it's a bit more legible for people actually reviewing [clears throat] how the product is working.

Uh, and the last one's kind of funny, too. Uh obviously if you're writing errors for humans it's helpful to make it very descriptive like okay this thing failed like click on this link or here's what you need to do next. Turns out that's also very helpful for agents. So there there is some DX in designing um good error messages for agents too. And when you're trusting your bots to go and do work over a long period of time it is very important that you trust them to be secure with your information to be secure with how it operates. So even working on its own separate machine not polluting anything on your machine still you would like it if you know a model reviews every shell command before it goes. So auto review it would be great if you know when these untrusted inputs like a email in your inbox could be a fishing email or some web hook.

You don't want to send that to the model and act like it was a user instruction. And there's plenty of work to do here uh in the industry on just making sure that that is uh properly protected from um prompt injections. um for these kind of uh non-reversible things like sending emails, right? You want to really ask the user first and then also you can build in some workflows into the product to keep passwords and other things outside of the model conversations. Okay, context engineering is also known as harness engineering. Like I feel like it's kind of the same thing. they've evolved over time, but basically what are some tips and tricks we can do to minimize the amount of context that's being used in these harnesses. Um, which is good for efficiency, for reducing costs, um, and other reasons.

I talked about prompt caching. Um, this is a good example of why it's really important. So if you think about when you're working with a coding agent or any agent, most of the tokens used are actually ideally cached tokens because you're sending it a conversation and then you're doing a change. You're you're adding one more message and then you're basically resending the entire conversation over again. So if you look at the pricing for models, there's huge discounts of course on if a token is cached and you want to keep them in the cache window as long as possible. And so a way to do this is you try to keep the tool definitions, the system prompts, all of this information as static as possible between different calls so that you don't break the cache and then have to pay for the uncashed tokens every single time.

So if you have that fixed part at the top, then you know when you get down to the bottom and you only have the newest message that comes in, that's the only delta that you're paying the uncashed tokens for. And there's a a whole art and science uh to this as well. basically because that main agent in the most ideal sense, this is the one you're having this very very very long conversation with that's compacting or compressing so summarizing the context many many times over uh without you even noticing sometimes which we'll show an example of. So you really want to keep that context small. Um some ways of doing that is anytime there's a very long output rather than stuffing that in the context you can just put it in a file and the agents are very good at reading files. Uh so this actually works pretty well which is just like reading a skill or reading a memory for example.

If you give the model a screenshot um or some other very expensive work you just delegate that out to a helper again so it goes to the sub agent and it doesn't affect the main agents context. Um, and then also in an ideal world, you want to kind of be always gardening and trimming and removing unnecessary stuff from this conversation, from this context window as you're going, both for cost, but for also recall and just making sure the model is performing well. So, if there's a cash cache miss, it's kind of like a bug in the system really. Um, and if if you go all the way back to uh all the way back to January of this year, uh when OpenClaw was really getting popular and people were using it uh inside of subscription plans for uh model providers, this was a very new usage pattern that model providers hadn't really seen and the the harness hadn't really yet been optimized for this type of workflow.

So, there was a lot of cache misses. And when you're doing something like this where agents are running all the time, that's obviously problematic. So, lots of fixes uh had to go in to make that robust to keeping the cache uh uh keeping the prompt cached as much as possible. Um, so like a fun uh a fun trick here is just trying to keep everything in the prompt in order such that even if you deploy a new change where you want to test out some new change to the system prompt or add a new tool, you can you can almost hash each section and always keep a fixed order. Like let's say you're doing a AB test and you want to put a new piece of the prompt in there. You really don't want that to screw up the prompt order for everybody else. Otherwise, you just busted the cache. Uh, and then that's that's problematic.

Um, another fun one is when you open the chat and you start typing, um, the server can actually go and prepare the prompt early, uh, to basically warm the cache. Almost like if you're on a website and you go to hover on a link, it can go and prefetch the next page. Um, so there's like tons and tons of little engineering optimizations that compound how far these models can go in terms of token usage, uh, and how smart they can be when you use them for very long periods of time. And the key thing is that when you're working over a very long period of time, you essentially have to get very good at compaction or summarization because let's say the model has a 200,000 token context window. You're going and going and going. Eventually, it's going to hit a point where it needs to summarize and all compression is lossy.

So, you have to figure out the best algorithm to do the compression and not lose important details. And there's, I'm sure, entire uh PhDs just for this problem. like it's it's very very tough to do, right? But just to show an example, like let's say you're going back and forth with your with your bot. Uh you have over 100,000 tokens uh and then you go and do something else. Well, there is this time period where the model providers have uh like a cache window when the cache is still warm. Maybe it's 10 minutes, maybe it's 60 minutes. Uh one trick that we do which I think is really helpful is while you're still within the warm cache period, you should probably do the the compaction right now because it's you have this long conversation. it's going to be much cheaper. So just go ahead do the compaction and now when the user comes back in 30 minutes or whatever they're starting from this summary and the summary now is going to be much cheaper for them to continue adding messages on.

So lots of work to do there to make it u to make the algorithm very high quality for how you do the summarization. When you talk about compaction and what information you choose to include in the agent's memory, I kind of think about it in a few different layers. The first one is like facts about the user that are very important to be included every single time. Going back to earlier when I said, "Oh, like the user always wants to have lowercase text." That probably needs to be in every single message. Otherwise, it would be really weird if one message now has uppercase letters. Uh if you're going like full Sam Alman style, um then uh you probably also have some prompts that you want or some facts that you want to include in there. And there's many different ways you can make this algorithm to determine what are the most important facts to keep.

So ideally there's this kind of garbage collection of this fact. And like if you think about it, this is kind of how our brains work. It's like this fact right now is important to keep. It's relevant for this conversation today. You're probably not thinking about, you know, a soccer match right now. Uh and your brain is very good at keeping those ones in context right now and kind of remembering them for later. We try to model some of that here where the the the harness has of course these core facts. It also has these dated logs of kind of your past conversation history essentially and then also these kind of scratchpad notes. So just like little things that you've been working on right now that are designed to be ephemeral and they will go away but they might be relevant just for the next 30 minutes or an hour.

And everything else can be found through files. This is the magic of files of the models being good at running commands is like they can always just go look stuff up and they're actually very good at doing that. So where where is this going? There is this flywheel of training the model to get better at the product and just as a highle refresher on kind of the stages of training a large language model. Uh first you have pre-training where the model learns uh a general understanding of the world from large amounts of internet text or other private data. Then you have supervised fine-tuning where you're teaching the model how to behave uh in a certain way maybe as a chat assistant or as an agent. And then finally you have reinforcement learning where it's kind of like teaching the model how to play games and get better at those games.

Um, and in the context of these always on agents, it's very helpful, for example, to train the model to see examples of the harness and to know how to work inside of the harness in SFT. Um, so it can understand the tools that you have available. It's helpful to train the model to be better at instruction following during reinforcement learning. How to understand the right skill to call when there's hundreds of skills, for example. how to click on the right spots when you're given in a browser and it needs to kind of click around. Um, and not only the training itself, but also how you serve the model, the inference that you do. The main agent you're talking with, you know, ideally you want it to have very low latency. You want it to come back to you right away, give you really fast replies, and you can tune the harness and the inference to do that.

But then when you go hand it off to a helper and you're asking it to go do some work for a very very long period of time, the difference between 10 minutes and 12 minutes is kind of negligible at that point. So that allows you to kind of control uh how you optimize things. Um the way that this kind of flywheel works is that let's say your agent gets something wrong. Um ideally then that gets turned into some kind of evaluation. So, uh, the model, you know, accidentally, uh, called this tool and it wasn't supposed to. Um, then on the back end, you can go and fix the harness. You can fix the bug, you can fix the prompt, maybe it's an inference issue. Um, you can check and make sure it's actually fixed in the eval, and then ship the change. Uh, and you know, the teams uh, and the products uh, the teams working on these products, they're kind of doing this every single day.

They're finding bugs, they're fixing it, they're measuring it with eval. That's kind of this inner loop that you're always getting better at. But then there's this outer loop which is like every few months or every month or however long um you're training new models and ideally the new models are learning from all of these bugs and these failures and they're kind of climbing on the evals to get better at the things that you care about and you can drop the new versions into the product and make sure that they're improving at all these different things. And the fun thing about this is then when the new models come out they have new capability jumps. Um it allows you to use the product in new ways. So there hasn't really been a lot of training data for example on using bots in a group chat. This is kind of an emergent thing that's being figured out.

Uh and it's important to get it right. Obviously if you've seen things like multbook or the hugging face incident like there are ways that that goes poorly. So it's important to be able to train the models how to behave well in a multi- aent system where uh different bots are talking to each other for example. Um, and so ideally you're always kind of running these inner loops and outer loops as a product team building these type of harnesses at a company uh working on on AI together. Um, one of the principles that we had at Cursor uh and now at SpaceX is to delete the product. And what we mean by this is you kind of have to internalize and assume that the models today are the worst they'll ever be. which means that in six months uh you might need to completely rebuild the UI. Like most of the stuff that I've talked about today, uh it's a pretty big update from if I would have given this talk six months ago or definitely a year ago because we've learned a lot of uh new practical ways of building harnesses and doing context engineering.

So a good strategy here is you really want to try to build for the next generation of the model. like how do you build as little as possible and get by uh while making just a really really simple interface to work with these models and allow them to uh to to think and act for you. So um some things that I think will happen in the next 6 to 12 months. We'll see. I think these are decently safe bets. But uh you know right now let's say you have your agents or your bots working for maybe a few hours, maybe a day depending on how tough it is. I think it's very possible that pretty soon um your agents will be working for weeks or months. Uh and it does bring up some kind of interesting questions in terms of how we train and evaluate those models. I think the models are going to get better and better about how they recall on past information in in conversations.

Plenty of different approaches in research here for the type of algorithms used or the representation of the data under the hood. Maybe it's a graph, maybe it's files, maybe something else. Um, I think ideally if you have this digital colleague, you give it literally all the tools that you have if you were onboarding a real human. And so I think organizations are still warming up to that idea, of course. So there's a more to do there. I think the interfaces uh will maybe get even easier. I think some people will prefer to just talk to their kind of bots or agents uh as a in a speech-toech way. Maybe they'll have their like uh Johnny IV device that is hopefully really cool and they can like talk to that uh all day. Um I think the models and the products will get really good at um learning skills on the job and then encoding that into files in instructions skills uh and taking that raw intelligence and actually making it useful for whatever uh whatever task you're asking it to do.

and people and companies um are going to get much better I think about uh writing down how all of these kind of bespoke processes work so that they're legible for agents to use. So some interesting problems I think that like could just spark ideas for you all to to research or to look into. The first one is like going back to this idea of agents that run for a month. What do you do when you need to evaluate an agent that runs for a month but the models ship every month? you know, it's like you can't fully evaluate that the model is uh sufficiently uh behaving as expected. So, you either have to figure out how to do a better eval or you know not ship models as fast and I think that's going to be a problem that the industry has to figure out in the next six months. Um memory obviously we talked about the different algorithms.

I think there's so much to do on the security side for how to make a secure harness, how to make the infrastructure [clears throat] around these agents um very secure uh and so they can't be taken over by bad actors. I think we didn't really talk about it too much, but the heristics around when the agent is silent or when it kind of bugs you, uh I don't think that we have it perfect yet. And I think trying to thread the needle between being proactive and suggesting like, "Oh, hey, like I saw that you missed class yesterday. Like I I had a recording there and I took the notes and like here's the thing." Like maybe that's helpful or maybe that's kind of annoying, you know? And like trying to figure out the right the right balance there. Um, I also think that it's a pretty safe bet to assume that at this time next year there will be significantly significantly more tokens flowing in the world which means that optimizing the inference optimizing the models optimizing the harnesses trying to drive down the tokens to uh be more efficient will also the token usage to be more efficient I think will also be very important.

So there's kind of these new building blocks to think about. I believe it was Alex from Open Router that I saw on on X. He had a really good take on this. There was people saying like, "Oh, every app today it looks the same. You've got the sidebar, you know, you've got the agent when you got the chat box. Like everybody's building the same thing." And he was like, "Yeah, that's like in 2005 you said everybody's building the same thing. They've got a server, you got a database, they've got some like CRUD interface to like update and delete things." And the reality is this is just kind of the new normal now where every new product is going to have to have some kind of uh agentic side to it either integrating or it will become a uh a you know a headless piece of data for another agent to use.

Um, and so we've talked about a lot of these products today. Durable workflows, um, sandboxes, computers in the cloud, memory, skills, uh, connecting [clears throat] to other services, doing voice, payments, identity, evals, observability. Each one of those boxes is like multiple billions of dollars of VC capital, you probably trillions. There's 20 startups for each one of those. And there's so many different things to explore and all of those because they kind of are the new primitives for the next generation of companies, the next generation of products, and they all have to integrate and be part of harnesses. So there's there's definitely a lot to do there. A few things uh to take away and then we'll have time for questions. Obviously, you all are engineers. You're thinking about how to write software.

It's more important than ever to think about the broader system design and the architecture of the code that you're working on because that the decisions you make there will really compound. Um, so I'm still I'm still pro looking at code. You know, I still look at the code that I'm writing. Some people not looking to code. I still think it's helpful, especially to understand the architecture that you're shipping. Uh, it's crazy how much that's changed in just a couple years. Um, secondly, you know, it sounds like y'all were kind of messing around with building own versions or understanding of how the harness worked in uh, last week. Uh, I encourage you to think about some of the things that I did today and try to apply that and think about how you might build a version uh, for yourself. I'm pretty sure there's like open grobbot on GitHub somewhere where somebody's like, you know, slot forked our code and made and made a version.

Uh, and you could probably set up like the the guey for that with, you know, a sandbox somewhere and, uh, you know, a a VM and like all sorts of different things. Um, so that could be interesting if you want to learn a little bit more about how these pieces work. And then finally, um, I would not shy away from just learning a little bit more about how LLMs work. You know, we gave a little bit of the TLDDR here of how they're trained, but there's so much more you can go into here without necessarily going deep in ML or deep into the science part of it. But I think it's helpful to understand just like when you're building a web app, it's helpful to know how databases work because increasingly the there are so many decisions you make at the product in the harness uh level that are influenced by the models themselves.

And having an intuition of how that works, I think will make you very well positioned for uh whatever role you're operating in. And uh that's all I have today. Thank you.