智能体工程师必须回答的三个问题｜LangChain

Brief Description

Three Questions Every Agent Engineer Should Be Able To Answer

Table of Contents

• What it means to own your own intelligence
• The three questions every agent engineer should answer
• Question 1: mining production traces to find failures
• Fine-tuned judges for monitoring agents at scale
• Question 2: reproducing failures beyond input and output evals
• World specs and tasks that mimic production
• Example: the world and tasks for LangChain's GTM agent
• Keeping tasks hard and hard to reward hack
• Turning production data into tasks and worlds
• What app teams see that frontier labs can't
• Question 3: fixing failures at the harness and model layers
• Staying ahead as models and users change

What it means to own your own intelligence

Welcome, everybody. Really looking forward to covering how and why and the methodology around what we know at LangChain to be about owning your own intelligence. It's our new office. Really stoked to have everyone here. We've got an awesome lineup. I'm going to approach what it means to own your own intelligence from a practitioner's angle. For someone who's building with agents and for someone who is probably going through a lot of the processes that it takes that I'm about to cover when it comes to building and improving your agent.

The three questions every agent engineer should answer

I think that there are three main questions that any person who's building an agent needs to have a very solid grasp of. And if they can answer them well, then they can definitively say that they do own their own intelligence. So, that starts with when you're building agents, understanding what goes wrong and understanding where your agent fails. This is obviously a very, you know, this is a simple question, but actually there's a lot of engineering work that goes into understanding if an agent failed. There's also a lot of engineering understanding that goes into understanding how we can make it fail again. This is obviously key on the process to being able to build agents that improve and to be able to own all parts of your model, and maybe the weights, and the all your harness and the weights, and then the weights as well.

And then finally as well, being able to answer what needs to happen to make it stop failing in the future.

Question 1: mining production traces to find failures

I think to answer the first question and to understand really how and why your agent has failed requires a very strong grasp of tracing and understanding your agent's behavior. And this often comes from mining your agent's production traces. Building an agent, hopefully you're tracing it, you're going to have a lot of trajectories and understandings of all of the different kinds of behavior that your agent is taking. And amongst all of this, there's going to be a lot of valuable signal that you can use to move on the path to improving your agent. That might be, for example, times where the agent either successfully called a tool or it incorrectly called a tool, it led to frustrating the user because it didn't have the right context. There's a variety of different signal within your agent traces, particularly your production traces, that is a very important element to capture when it comes to owning your own intelligence and understanding whether or not your agent failed.

Fine-tuned judges for monitoring agents at scale

The issue with doing that at scale, obviously, is that if you have an agent that has thousands of runs a day, millions of runs a day, it's going to be very hard to understand at scale whether or not your agent is performing as you expect it to. Using frontier judges, LLM-as-judges, online evaluators, obviously is very effective, but also very expensive at scale. And so, as an example, a way that we approach this, and just one of the ways that you can look towards understanding and extracting signal from your agent traces, is by making the evaluation-specific judges to go and extract and actually mine your agent for production signal. So, we did this at LangChain recently. We worked with fine-tuning, labeling, and producing a judge to go and actually classify whether or not an agent was giving users a delightful experience, whether or not there was any perceived error.

So, that might be, for example, an agent frustrating user, or it might be a, you know, a string of tool calls that an agent makes that actually leads it to performing the wrong action when a user was looking for it to do something else. And so, being able to actually have some form of monitoring and understanding what your agent's doing at scale is a very important step towards understanding whether or not it failed. And also a very important way to be able to cluster the behaviors and the failure mechanisms that begin to emerge as you build and as you start to improve your agent. This is only the first step in the journey, though, because once we know and once we have classified agent behaviors, we then need to go and take a variety of next steps to go and be able to reproduce and then actually remediate these issues as they continue to develop.

Question 2: reproducing failures beyond input and output evals

So, that brings me to my next point about what it really means to be able to own your own intelligence. A very important part of that is being able to reliably reproduce where your agent is failing. And I've worked with a variety of different application teams, a variety of different teams building their agents, and really over the past six or seven months, this has begun to take a pretty drastic shift. The typical way of doing this, and actually, rather, the shallow way of looking at this, is just evaluating whether or not the inputs, or evaluating whether or not the outputs to your agent given some simple input produces the behavior that you'll expect. This is very powerful for capturing a variety of different agent failure mechanisms, but often doesn't encapsulate the full picture. The reason why is an agent's behavior might be influenced by external factors of the environment that it's in.

And so, that might be, for example, the state of the database that it's interacting with, the permissions that it has access to, the particular schemas that it's working with, or even the past actions that it's taken that has led to it being in the environment in the state that it has. And so, what we really need then is a way for us to be able to mock out the broader application machinery of your agent as it stands in production.

World specs and tasks that mimic production

And that brings me to building more robust evaluations by creating a way where we can mimic the world or the environment that your agent exists in. This is an iterative process, but really what we... a way that we found to do this quite successfully at LangChain, and again, this is something that's constantly evaluating, is through the idea of working with specifications to define the agent's world as well as the tasks within it. So, what I mean by that is having some kind of generic world spec which defines the things that don't mutate across that agent's world. And so, that might be things like the permissions, or that might be like the domain-specific rules that an agent has access to, or that might be things like the state of the file system and the dependencies that it interacts with.

This is things that don't change across the agent's actions that it takes in this action space. So, an example of this is we have a very commonly used internal agent

Example: the world and tasks for LangChain's GTM agent

at LangChain. It's called the GTM agent. And so, our go-to-market organization will use it, for example, to do things like do competitive research on competitors in deals, or go and understand and spec out the pipeline of a particular sort of opportunity. The world in that case would be all of the systems that the agent needs to interact with: Gong, Salesforce, Calendly, you know, Google Calendar. And effectively the systems that it touches and interacts with as this agent interacts and is engaged with by all of our sellers. Tasks within that world represent specific actions or tasks that the agent needs to take, which will go and actually test and hopefully surface different emergent behaviors and things you're trying to test for. And so, a task within that go-to-market agent world, for example, would be go and research all of the Gong transcripts for a particular customer that we're looking to investigate or are in the process of working with, and help me draft a next best reach-out.

And so, if I'm somebody who's building a... or if I'm somebody who's building an environment or a place where I can reliably test my go-to-market agent in production, I'm probably going to want to have some concept of a robust world where it mimics the environment that that agent is operating within production. And that extends far beyond just a simple input and output based

Keeping tasks hard and hard to reward hack

evaluation. One thing that we've noticed, though, when kind of going and creating these worlds and developing these specs and working with an agent, coding agent, side by side to actually go and develop these, is that agents themselves aren't very good, or rather, need actual constant supervision and guidance when building these out. And so something that's obviously very important in all of this is to make sure that the tasks that your agent is defining are both reliably hard and actually present a challenge for you to be able to continuously improve your agent with. And also there aren't methods where the agent has designed tasks that are easy to reward hack. You might have heard of the term reward hacking or, you know, adjacent terms. This is obviously really important as well because it makes sure that the world that we're developing and the tasks that we're giving our agent are sufficiently hard so that when we make improvements they're quantifiable and actually mimic really what the agent would be doing in production.

Turning production data into tasks and worlds

Something that I think is kind of really interesting and core to this, and once we have this kind of like mechanism now of knowing that the way for us to really kind of get a handle on our own intelligence and be able to create and develop an agent that is solving problems in a world that actually mimics the world that it's, that the kind of environment that it's operating in, is that we have the ability to go and do this through turning that data that we have from running our agent in production into this kind of, these kinds of tasks, into these kinds of worlds.

What app teams see that frontier labs can't

So it's no surprise that some of the best models that we use today, produced by frontier labs, are... it's no surprise that frontier labs are producing some of these models. And the reason why, obviously there are many reasons, but the capital that they can allocate towards acquiring data to build these worlds and to build these tasks far supersedes that of the average app-layer company. A classic example of this, by the way, is, you know, Google making a $10 million tender offer to Spirit Airlines not to buy the company, but to buy the internal Slack memos, the emails, the documentation, because obviously all of that's going to be used to develop tasks and develop you know, training datasets to be able to actually go in and produce and make better models. But this is a bit of a David and Goliath situation because, unlike frontier labs, application developers and application owners see the spiky edges around the ways that people are using their agents.

And for that reason, they're able to produce data from that telemetry, or they're able to understand how real people are actually interacting with and using their agent, and therefore develop tasks and develop environments that a frontier lab never could. And so, once we know that from the traces of our agent, from all of this exhaust, from all of this substrate, we can really help design powerful tasks and worlds to be able to go in and actually help improve our agent. Then obviously we know that it's a data-wrangling problem, because there's going to need to be deduplication, there's going to need to be all sorts of different ways that we turn these traces into reliable tasks. But once we do, we have the substrate and we have the foundation for a lot of further agent improvement and development, and really then we have a strong benchmark and starting point for us to be able to go and actually make quantitative agent improvement.

Question 3: fixing failures at the harness and model layers

And so, once we have a world, once we have a set of tasks within that world, ideally a task targets some kind of failure mode that you've seen in your agent, right? You would design tasks around where you've observed your production traffic, or you've observed your production agent to be failing. Really then changes come at two layers. And this is really where owning your own intelligence and owning the mechanisms around both the harness and the models that are powering your agents really begins to compound, because now you can have full flexibility to make changes at all levels here. Changes at the harness level might involve simple changes like prompts or tool schema changes, or changing the way that context is injected so that an agent given a certain task can answer in a certain way.

And then making changes at the model layer, I'll kind of, I won't dive into it too much because I know that Charlie here is going to cover some of the different post-training techniques that you can do at this level, but again, now you have the flexibility to be able to go in and choose your post-training algorithm of choice and make changes to help go and remediate whatever it is that you're trying to test for and ideally mitigate. Once you've got that, then we can rerun and then we can establish that the environment and the agent running in that environment can pass those particular tasks and begin to behave as we expect it to.

Staying ahead as models and users change

Finally, once we've established a robust environment or world and set of tasks, it's inevitable that as models develop, parts of what we test for will change and, you know, regressions will be introduced, but also parts of what we designed for to solve for particular deficiencies in our agent will be absorbed as part of the model. And I think that this is kind of no surprise to anyone, but we've constantly seen parts of the harness, whether it's, you know, long task decomposition or whether it's dynamic tool routing, be absorbed into the model layer. And so, we'll see that change. But we'll be able to test for it because we now have a robust way to actually simulate how our agent is behaving in production. Also, what will be... also, as well, the interaction patterns of people using your agent will obviously change, and having a strong way to be able to go and test and actually roll out and determine whether or not your agent holds up under those particular interaction patterns will still be available given that we have this really robust and powerful way to now test it.

And so really being able to own your own intelligence comes down to being able to adapt to changes as models improve, as user interaction patterns change. And as builders, once we are aware of, and when once we have put the mechanisms in place to be able to go and test different behaviors through robust environments and through different tasks, we really then can stay on top of this fast world that we're moving in. And so, thank you for listening.