我们如何打造自己的设计智能体
原题How We Built Our Design Agent
推荐理由LangChain 将设计原则、反馈理由与持久沙箱结合,让内部设计智能体持续交付品牌素材,并开源完整实现。文章具体说明为什么要删掉过时的硬编码规则、如何保留人的审美判断,为建设团队内容工具提供一条从原型走向协作生产的实践路径。
读前速览
LangChain 详细介绍了其 GTM 工程团队如何在 Slack 中构建 Content Studio——一个生产级设计智能体,通过设计原则教会模型设计品味,借助双渲染引擎的灵活媒体框架,以及技能-路由-门禁-维基-记忆架构,生成符合品牌调性的视频、图形和演示文稿。
设计品味来自上下文和原则,而不仅仅是品牌令牌。
模型能写出好代码,但默认倾向于分布收敛(熟悉、通用的模式)。LangChain 通过将设计原则(每帧一个焦点)、防范常见 AI 模式的护栏,以及每条设计师反馈背后的推理逻辑编码进技能中,教会 Content Studio 品味,使智能体能够用有意图的选择而非统计平均值来填补未指定的决策。
随着模型进步,灵活的框架架构优于预构建的组件库。
最初的 harness_v1 包含 115 个硬编码 UI 组件和运动规则,以弥补模型局限。当 Opus 5.5 到来时,移除大部分规则、仅保留高层原则反而带来了更好的多样性和创造力。这印证了「苦涩的教训」:工程投入应聚焦于环境(工具、上下文、人类指导),而非预先固化那些模型很快就能原生处理的创意决策。
双引擎媒体框架实现专业级视频和动画制作。
Content Studio 使用 HyperFrames(HeyGen 的开源框架)进行基于时间线的视频编辑——将片段、字幕和屏幕录制视为网页元素——以及自定义的 motion-studio 画布引擎,为原创动画逐像素绘制并添加运动模糊。两个引擎都按时间戳确定性渲染,支持帧级检查和并行渲染,同时保持音频同步。
结构化知识系统(技能、门禁、维基、记忆)将设计判断编码为可复用、可审计的逻辑。
路由技能将请求类型映射到专业技能,并决定是否需要画布。门禁是对可量化规则的确定性代码检查(如大小写、标志锁定)。维基存储公司上下文和优秀作品示例。记忆层随时间捕获人类教导。这将主观原则与客观检查分离,并使智能体的推理过程可审查。
生产指标验证了该方法:广泛采用、成本大幅降低、可量化的互动提升。
Content Studio 交付了 3,000+ 个素材,包括 80+ 份演示文稿;其中一份出现在观看量最高的自然 YouTube 视频中。使用其视觉内容的帖子获得了 2 倍点赞和 3.7 倍收藏。75% 的请求来自市场部门以外,75% 为每周回访用户。每次请求的模型支出中位数从 8 美元降至 0.50 美元,迁移到 Managed Deep Agents 后内部代码缩减了 80%。
完整摘要
LangChain 的 Content Studio 从一个低质量原型演变为广泛使用的内部设计智能体,交付了 3,000+ 个素材,成本降低 16 倍。团队发现模型已经能很好地执行代码,但缺乏设计品味——即区分「正确」与「出色」工作的判断力。他们通过将设计原则、防范「AI 垃圾」(如僵尸 UI)的护栏,以及设计师反馈背后的推理逻辑编码进技能中,来教会模型品味。突破来自拥抱「苦涩的教训」:与其预构建 115 个刚性组件来弥补模型弱点,不如构建一个灵活的框架——每个线程独立的 Linux 沙箱,配合 HyperFrames 进行视频编辑,以及自定义的 motion-studio 画布实现像素级动画。路由技能将请求导向专业技能;确定性门禁执行可量化规则;维基提供公司上下文;记忆层捕获持续的人类指导。多人协作的 Slack 用户体验展示计划、实时进度和审批,支持长达一小时的异步子智能体运行。成果:X 上获得 450 万次浏览,点赞数 2 倍、收藏数 3.7 倍,75% 的请求来自非市场部门,中位成本 0.50 美元。完整框架已开源,可通过 Managed Deep Agents 部署。
目录 · 13 节
核心要点Key Takeaways
- 品味源于背景信息。模型已经能写出不错的代码,但要让它们做出符合你们风格的成果,还需要设计师的原则、避免 AI 粗制滥造的准则,以及反馈背后的思考。Taste comes from context. Models already write good code, but they need your designers’ principles, guardrails against AI slop, and the reasoning behind feedback to make work that looks like yours.
- 给智能体配一台设计师的电脑。每个 Slack 线程都有自己的沙盒,智能体可以在不同轮次之间保留项目,在视频引擎、图像工具和共享画布之间协同工作,并检查自己生成的渲染结果。Give the agent a designer’s computer. Every Slack thread gets its own sandbox where the agent keeps the project between turns, works across a video engine, image tools, and a shared canvas, and inspects its own renders.
- 随着模型的改进,删除规则。用于弥补模型弱点的规则会逐渐过时,因此应保留设计原则,并随着模型变得更好而移除权宜之计。Delete rules when the model improves. Rules that make up for a model’s weaknesses expire, so keep the design principles and remove the workarounds as models get better.
- 长时间运行的智能体需要支持多人协作的用户体验。大型任务会交由异步子智能体执行数小时,因此 Slack 线程会展示计划和实时进展,并在任何内容发布前请求审批;团队中的任何人都可以在执行过程中引导智能体。Long-running agents need multiplayer UX. Big jobs run for hours on async subagents, so the Slack thread shows the plan, live progress, and an approval before anything ships, and anyone on the team can steer the agent mid-run.
- 你可以对它进行分叉。开源代码库包含完整的运行框架,而 Managed Deep Agents 会将其部署到你团队的 Slack 中。You can fork it. The open-source repo has the full harness, and Managed Deep Agents deploys it to your team’s Slack.
上面的宣传内容由 Content Studio 制作。它是我们的设计智能体,只需在 Slack 中发送一条提示,就能创建视频、社交媒体图片、重制的 UI 图像、幻灯片等内容。该智能体生成的每项素材都是用代码编写的。The promo above was made by Content Studio, our design agent that can create videos, social graphics, recreated UI images, slide decks, and more from just a single prompt in Slack. Every asset the agent produces is written in code.
Content Studio 完全基于 Managed Deep Agents 内部构建。如今,它已成为公司内部使用最广泛的 GTM 智能体之一,但最初并非如此。早期反馈……不太理想。Content Studio was built entirely in-house on Managed Deep Agents. It’s now one of the most widely used GTM agents within our company, but it didn’t start out that way. Early feedback was… not great.
那么,我们是如何从如此低的质量做到现在这样的?So how did we go from such low quality to this?
我们花了四个月教 Content Studio 理解我们的审美和设计原则,主要是靠看着它在全公司面前频频出错。We've spent four months teaching Content Studio our taste and design principles, mostly by watching it get things wrong in front of the whole company.
这篇文章介绍了我们的 GTM Engineering 团队如何与设计团队合作,从零打造 Content Studio:对我们而言,好的设计意味着什么,我们如何让智能体理解我们的审美,以及让它能够制作从静态图片到发布影片等各种内容的工具框架。This post walks through how our GTM Engineering team worked with our design team to build Content Studio from scratch: what good design means to us, how we taught the agent our taste, and the harness that lets it make anything from a still to a launch film.
我们已将设计智能体开源。Fork 这个开源仓库,添加你们的品牌设计令牌、字体和技能,再通过 Managed Deep Agents 将它部署到团队的 Slack。10 月 28 日,欢迎参加 GTM Engineering Live。我们将深入剖析执行框架和用户体验,并展示这篇文章中没能写下的内容。We've open-sourced our design agent. Fork the open-source repo, add your brand tokens, fonts and skills, and deploy it to your team's Slack with Managed Deep Agents.
Then join us at GTM Engineering Live on October 28. We'll go under the hood of the harness and the UX, and show the work that didn't fit in this post.
构建可规模化的设计智能体Building a design agent for scale
LangChain 发展迅速。我们不断推出产品、开展营销活动、举办网络研讨会、发布博客并参加各类活动。每项工作都需要相应的素材,从博客头图和图表,到演示文稿、视频和广告创意,应有尽有。我们的设计团队不堪重负,重复性的需求挤占了更具战略性和创新性的工作时间。作为一家精简高效的初创公司,我们需要一种便捷的方式,让公司里的每个人都能创作出高质量、符合品牌调性的作品。LangChain moves fast. We have constant product launch campaigns, webinars, blogs and events. Each initiative needs assets, from blog headers and diagrams to decks, videos and ad creatives. Our design team was stretched thin, with repetitive requests crowding out more strategic and innovative work. As a lean startup, we needed a low-friction way for anyone at the company to create high-quality, on-brand work.
为满足这些需求,我们打造了 Content Studio——一个驻留在 Slack 中的设计代理。团队成员只需描述自己的需求,并附上相关内容或简报,代理就会返回完成度高且符合品牌风格的文件。要打造 Content Studio,我们需要三样东西:To address these needs, we built Content Studio, a design agent that lives in Slack. A team member describes what they want and attaches accompanying content or briefs, and the agent returns finished, on-brand files. To build Content Studio, we needed three things:
- 我们设计师的专业判断被写入代理的技能和检查项中,让它知道什么样的结果才算好。Our designers' expert judgment, written into the agent's skills and checks so it knows what good looks like.
- 围绕该模型构建的定制环境,包括一台视频编辑电脑,以及用于设计工作各个环节的工具。A custom environment around the model, including a video editing computer with tools for every layer of the design work.
- 合适的用户体验,让整个团队都能通过一个 Slack 线程协作处理持续数小时的项目。The right UX, so a whole team can collaborate on a project that runs for hours from one Slack thread.
Managed Deep Agents 为我们提供了在公司内部将 Content Studio 投入生产所需的一切:托管式 LangSmith Sandbox、Slack 集成、托管工具、范围受限的智能体记忆,以及智能体运行调度功能。这样一来,小团队就能把时间用在设计工作、技能、工具和 Slack 体验上,而不是处理底层基础设施。Managed Deep Agents gave us everything we needed to productionize Content Studio inside our company: a managed LangSmith Sandbox, integration with Slack, managed tools, scoped agent memory and scheduling agent runs. This allows a small team to spend its time on the design work, the skills, tools and Slack experience, instead of the plumbing.
关键成果Key results
- Content Studio 已为公司各部门交付了 3,000 多项素材,其中包括 80 多份演示文稿。其中一份演示文稿甚至被用于我们观看次数最高的自然流量 YouTube 视频。Content Studio has delivered over 3,000 assets across the company, including more than 80 slide decks. One of its decks was even used in our most-watched organic YouTube video.
- 展示其作品的帖子在 X 上获得了约 450 万次浏览。Posts with its work drew about 4.5M views on X.
- 在 LangChain 账号上,带有其视觉素材的帖子获得的点赞数约为其他帖子的 2 倍,收藏数则达到 3.7 倍。On the LangChain account, posts with its visuals have earned about 2x the likes and 3.7x the bookmarks of other posts.
- 如今,四分之三的请求来自营销部门以外,75%的用户是每周都会回访的老用户。Three in four requests now come from outside marketing, and 75% of users are repeat visitors who come back every week.
- 现在,中位数请求的模型成本约为 50 美分,低于 9 月中旬的约 8 美元。迁移到 Managed Deep Agents 后,我们自己的代码量减少了约 80%。The median request now costs about 50 cents in model spend, down from about $8 in mid-September. Our own code shrank by about 80% after the migration to Managed Deep Agents.
AI 真的擅长设计吗?Is AI actually good at design?
在开始构建之前,我们先来讨论本世纪的争论:AI 究竟能不能做出优秀的设计?Before jumping into the build, let’s first address the debate of the century: can AI actually produce good design?
该模型在执行方面已经表现出色,能够编写 HTML 和 CSS、图表规范以及动画时间线。不过,它并不了解我们的背景,因此会倾向于采用熟悉的模式,也就是 Anthropic 所说的“分布收敛”。结果可能看起来很精致,却和其他上百个作品如出一辙;也可能动画做得很漂亮,却始终没能传达重点。熟悉的模式不一定不好,但我们希望智能体能做出符合自身目的的选择。The model was already good at execution and could write HTML and CSS, chart specs and animation timelines. However, it didn't have our context, so it reached for familiar patterns, what Anthropic calls “distributional convergence”. The result could look polished and still look like a hundred others, or be beautifully animated and never get its point across. Familiar isn’t always bad, but we wanted our agent to make choices that followed its purpose.
Stripe 设计主管 Katie Dill 做了一个颇有启发的对比。二战后,在快速建设的压力下,建筑师们一遍又一遍地照搬现代主义简洁的造型和朴素的表面。这些复制品保留了外观,却失去了背后的思考和个性。她警告说,AI 也可能对软件做同样的事。她称之为“僵尸 UI”,并表示我们需要有自己的立场,因为“如果你没有,AI 就会替你做主。”Katie Dill, Head of Design at Stripe, makes a helpful comparison. Under pressure to build quickly after World War II, builders copied the clean shapes and bare surfaces of modernism over and over. The copies kept the look while losing the thought and character behind it. Her warning is that AI can do the same to software. She calls it “zombie UI” and says we need a point of view, because “if you don’t, AI will give it for you.”
我们不认为 AI 不擅长设计。只是除非你赋予它鲜明的观点,否则它就没有自己的立场;而要让它拥有我们的立场,就得教会它我们的审美。We don't think AI is bad at design. It just doesn't have a point of view unless you provide it, and giving it ours meant teaching it our taste.
培养设计智能体的审美Teaching a design agent taste
品味是一种判断力,能区分作品只是正确,还是称得上优秀。在设计中,两种布局可能都用了恰当的字体、颜色和间距,但只有一种真正表达出了自己的意图。品味源自迪尔所说的“练就敏锐的观察力”。Taste is the judgment that separates work that's correct from work that's good. In design, two layouts can use the right fonts, colors and spacing, and only one of them makes its point. Taste comes from what Dill calls "getting really good at noticing."
正是通过这种观察,我们学会了识别 AI 劣质内容;这类内容最常出现在任务说明把某项决定留给智能体时。That noticing is how we learned to spot AI slop, which most often shows up when a brief leaves a decision to the agent.
例如,下方左侧的图片是由本地编程智能体使用我们的品牌技能创建的(其中包含颜色、字体、徽标等)。但最终结果遗漏了一些关键要素,例如正确的徽标组合形式、文本对齐方式,以及哪些颜色属于我们的品牌主色、哪些属于辅助色。For example, the image on the left below was created by a local coding agent with our brand skill (which includes colors, fonts, logos, etc.). But the final result missed critical components, such as the correct logo lockup, text alignment, and which colors were our primary vs. secondary brand colors.
同一条提示词在 Content Studio 中生成了一张完全符合品牌调性的图片。这是因为本地代理用自己见得最多的内容补全了简报中未明确的每项决策。而采用这种方法后,Content Studio 用我们的选择填补了同样的空白。The same prompt produced a perfectly on-brand image in Content Studio. This happened because the local agent filled in each decision that the brief didn't cover with what it had seen most. With this methodology, Content Studio filled the same gaps with our choices.
我们提供给 Content Studio 的指南从基础内容开始,包括颜色令牌、字体粗细、间距和对齐方式。但大部分改进来自我们在此基础上补充的内容。我们引入了设计原则,例如每个画面都应有一个视觉焦点;针对常见 AI 设计模式设置了防护规则;还说明了每条反馈背后的理由。结合更强大的底层模型,这些内容帮助 Content Studio 学会以更像设计师的方式思考和沟通。The guidelines we gave Content Studio started with the basics, including color tokens, font weights, spacing, and alignment. But much of the improvement came from what we added on top. We introduced design principles, like having one focal point per frame, guardrails against common AI design patterns, and the reasoning behind each piece of feedback. Combined with better underlying models, these helped Content Studio learn to think and communicate more like a designer.
起初并非如此。在我们早期制作的一段动态图形中,一只鸟飞出画面,身后拖着一道光带,持续不到一秒。我们要求不要出现锐利的边缘,但初稿显得笨拙,边缘生硬而圆钝。设计师本该明白我们的意思:边缘应该柔和下来,而尾迹则应朝着与鸟飞行方向相反的一端逐渐变细、淡出。It didn't start that way. In one of our early motion graphics, a bird flies off screen with a ribbon of light trailing behind it for less than a second. We asked for no sharp edges, but the first pass came back clunky, with harsh, rounded edges. A designer would have understood what we meant. The edges should soften, while the tail tapers and fades in the direction opposite the bird's movement.
到后来的迭代时,智能体已经开始以设计师的眼光审视这个细节。它先快速画了张草图,确定鸟尾巴应该在什么位置,然后自行建立了一套逐帧检查机制,确保丝带始终连接在尾巴上。大多数观众可能不会有意识地注意到这个细节,但一旦哪里不对,他们就会有所察觉。By a later iteration, the agent was evaluating that detail with a designer's eye. Using a quick sketch to identify where the bird's tail should be, it built its own frame-by-frame check to ensure the ribbon stayed attached. Most viewers would never notice the detail consciously, but they'd feel it if something was off.
可衡量的问题最容易解决。默认大小写规则过去经常在评审中被提及,但现在自动化检查会捕捉到每一处。更难的判断则更主观,例如一个营销活动创意是否足够出色、传达的信息是否清晰,或者执行效果是否能引起受众共鸣。当设计师做出这类判断时,我们会把其中的考量记录到一项技能中,让智能体能将其应用于今后的草稿。The measurable problems were the easiest to solve. Default capitalization used to come up frequently in reviews, but an automated check now catches every instance. The harder decisions are more subjective, like whether a campaign idea is strong enough, the message comes through clearly, or the execution resonates with the audience. When a designer makes one of those calls, we capture the reasoning in a skill so the agent can apply it to future drafts.
Content Studio 让更多人能够承担设计工作。产品经理、工程师或创始人都可以在 Slack 上向智能体说明需求,然后就能拿到符合品牌调性、可直接发布的成品。如今,四分之三的需求来自市场营销团队以外。对设计师而言,角色已转变得更接近创意总监:负责把握方向、审核作品,而不是亲自制作每一项素材。Content Studio changed who can do design work. A PM, engineer, or founder can brief the agent in Slack and walk away with something on brand and ready to ship. Three in four requests now come from outside marketing. For designers, the role has shifted closer to that of a creative director, setting the direction and reviewing the work rather than producing every asset themselves.
做出正确的设计判断只是流程的一部分。我们还需要为智能体提供工具和环境,让它能够落实这些判断。Getting the design judgment right was one part of the process. We also needed to give the agent the tools and environment to execute on it.
构建媒体测试框架Building a media harness
设计师会使用多种工具:用 Photoshop 处理静态图像,用视频引擎制作动态效果,还会用共享画布进行原型设计与协作。他们也会参考品牌指南、设计素材和公司文档。A designer works across several tools, from Photoshop for stills and a video engine for motion to a shared canvas for prototyping and collaboration. They also rely on brand guidelines, design assets, and company documentation.
智能体也需要类似的配置:一台可供工作的计算机、用于设计流程不同环节的工具,以及设计师开展工作时所具备的背景信息。运行框架将这些要素整合在一起。Deep Agents 提供底层运行框架,包括智能体循环、规划、文件、子智能体和上下文管理。Managed Deep Agents 将其打包并部署到生产环境中。我们围绕它构建了整个环境,为智能体提供一台计算机,使其能够维护项目、跨工具协作并检查结果。An agent needs a similar setup, with a computer to work on, tools for different parts of the design process, and the context a designer brings to their work. The harness brings these pieces together. Deep Agents provides the underlying harness, including the agent loop, planning, files, subagents, and context management. Managed Deep Agents packages and runs it in production. We built the environment around it, giving the agent a computer where it can maintain a project, work across tools, and inspect the results.
测试框架_v1harness_v1
我们最初版本的这个工具框架试图替模型完成设计工作。早期模型需要大量帮助,才能制作出我们愿意正式发布的动态图形或产品发布影片。为了弥补它们当时还无法自行解决的问题,我们本能地认为,应该把更多自己的设计判断融入指令和代码中。Our first version of that harness tried to do the designing for the model. The early models needed a lot of help to make a motion graphic or a launch film we'd want to ship. To compensate for what they couldn't yet figure out themselves, our intuition was to put more of our own design judgment into instructions and code.
因此,我们构建了一套设计系统,其中包含 115 个可复用的 UI 组件,涵盖标题、布局和动画,并提供了何时以及如何使用这些组件的指南。这为模型提供了一系列现成的设计作为参考,减少了它从零开始做出的创意决策数量。So, we built a design system with 115 reusable UI components for titles, layouts, and animations, along with guidance on when and how to use them. This gave the model a set of existing designs to work from, reducing the number of creative decisions it had to make from scratch.
这有助于保持品牌风格一致,但代理会在每一帧中重复使用相同的布局和动画,缺乏变化:That helped it stay on brand, but the agent would just repeat the same layouts and animations for each frame without much variety:
我们知道模型最终会非常擅长这项任务,因此我们一直在为那一刻打造环境,并在此过程中不断尝试各种技能和工具。有时,我们会增加更多参考资料或更详细的说明来帮助模型;有时则会把它们全部移除,看看模型仅凭自身能做到什么。We knew the models would eventually be great at this task, so we kept building the environment for that moment, experimenting with the skills and tooling along the way. Sometimes we'd add more references or more detailed instructions to help the model. Other times we'd remove them entirely and see what it could do on its own.
但结果仍未达到我们的预期,而要缩小差距,就需要投入越来越多的工程工作。我们不断为技能添加规则、编写更多动作代码,并构建更多组件,以弥补模型做不到的部分。But the results still weren't where we wanted them, and closing the gap required increasingly more engineering work. We kept adding rules to the skills, writing more motion code, and building more components to compensate for what the model couldn't do.
随后,Opus 5.5 问世,我们立刻看到了质量的提升。我们重新审视了这些技能,删除了大部分硬编码规则,只保留设计原则和高层指导。结果进一步改善,输出更加多样,也更少重复。模型终于开始自主做出创意决策,而这些决策此前我们一直试图提前为它设定好。Then Opus 5.5 arrived, and we saw an immediate jump in quality. We revisited the skills and removed most of the hard-coded rules, keeping the design principles and high-level guidance. The results improved further, with more variety and less repetition. The model was finally making creative decisions we'd previously tried to prebuild for it.
这就是我们的“苦涩教训”:旨在弥补模型弱点的规则,会随着模型能力提升而变得越来越无用。真正持久的是模型所处的环境,包括它使用的工具、获取信息的上下文,以及人们引导它工作的方式。我们没有预先设定创作决策,而是专注于让工具框架足够灵活,以便充分发挥每一代新模型的优势。This was our Bitter Lesson. Rules designed to compensate for a model's weaknesses become less useful as models improve. What lasts is the environment around the model, including the tools it uses, the context it draws from, and how people guide its work. Instead of prebuilding creative decisions, we focused on making the harness flexible enough to take advantage of each new generation of models.
设计师的电脑Designer’s computer
为了展示这个 harness 如何运作,我们让 Content Studio 完整制作了一个视频项目,讲述它的工作方式。成片的主角是一只鹦鹉,置身于像素风丛林中,旁白采用 David Attenborough 的风格(由 ElevenLabs 配音)。我们就用这个项目来逐层介绍 harness 的构成:计算机、工具,以及支撑其工作的上下文。To show the harness in action, we asked Content Studio to make a film about how it works with a video project end to end. The result stars a parrot in a pixel jungle narrated in the style of David Attenborough (voiced with ElevenLabs). Let's use it to walk through the layers of the harness: computer, tools and context behind its work.
设计是一个迭代过程。你先做出某样东西,看看效果再修改,往往要反复很多次。因此,智能体需要一台能在整个对话期间持续工作的计算机。每个 Slack 线程都有自己的 LangSmith Sandbox,这是一个配备文件系统和 shell 的隔离 Linux 机器;项目会在线程存续期间一直保存在那里。Design is iterative. You make something, look at it and change it, often many times. So the agent needs a computer it can keep working on across a whole conversation. Each Slack thread gets its own LangSmith Sandbox, an isolated Linux machine with a filesystem and a shell, and the project lives there for the life of the thread.
智能体创建的一切内容都是通过代码构建的,无论是幻灯片的 HTML 页面、图表的数据规范,还是视频的动画时间轴。这样一来,无需从头开始就能轻松迭代。如果有人要求让鹦鹉提前出场,智能体只需更改一个时间值并重新渲染,不会影响美术设计或文案。对于扁平图像或生成的视频片段,即使是细微的修改,也可能意味着要重新制作整个素材。Everything the agent creates is built with code, whether it's an HTML page for a slide, a data spec for a chart, or an animation timeline for a video. This makes it easy to iterate without starting over. If someone asks to bring the parrot in earlier, the agent can change a single timing value and render again without affecting the artwork or copy. With a flat image or generated video clip, even a small edit could mean recreating the entire asset.
项目简介、计划、场景代码、音频和渲染文件都存放在同一个项目文件夹中,参与该任务的每个子代理都基于这个文件夹开展工作。The brief, plan, scene code, audio and renders all live in one project folder, which every subagent on the job works from.
除此之外,我们还安装了设计师在工作各个层面都会用到的工具:On top of that, we installed the tools a designer would reach for at each layer of the work:
动效是最难的一层。它由两个引擎驱动:Motion is the most difficult layer. It runs on two engines:
- 用于编辑的 HyperFrames:HyperFrames 是 HeyGen 的开源视频框架,可将视频转化为网页。片段、屏幕录制、字幕和文本都只是页面上的元素,由 GSAP 时间轴控制移动。因此,当有人要求为网络研讨会添加字幕或剪辑产品演示视频时,智能体就像编辑网页一样进行剪辑。它的 CLI 基本上就是智能体的剪辑工具:检查合成内容、提取一帧进行核对、渲染剪辑成片。我们还会安装 HeyGen 官方技能,让智能体向打造这一格式的团队学习。HyperFrames for editing: HyperFrames, HeyGen's open-source video framework, turns a video into a web page. Clips, screen recordings, captions and text are just elements on that page, and a GSAP timeline moves them around. So when someone asks to caption a webinar or cut a product demo, the agent edits it like a web page. Its CLI is basically the agent's editing suite: lint the composition, pull a frame to check, render the cut. We also install HeyGen's official skills, so the agent learns the format from the people who built it.
- 用于动画的 motion-studio:网页无法模糊快速运动、让物体发光或将徽标旋转成 3D 效果,而这正是我们想要的原创动画质感。因此,我们打造了自己的 canvas 引擎。智能体编写代码,绘制每一帧中的每一个像素;引擎则将最多 16 个瞬间融合到每一帧中,就像相机快门模糊运动一样。鹦鹉短片就是用这种方式制作的。
motion-studiofor animation: A web page can't blur fast motion, make things glow or turn a logo in 3D, and that's the finish we wanted for original animation. So we built our own canvas engine. The agent writes code that draws every pixel of every frame, and the engine blends up to 16 moments into each one, the same way a camera shutter blurs movement. The parrot film was made this way.
两个引擎都根据时间戳渲染每一帧,因此,无论何时渲染,时间点为十秒的帧看起来都一样。这样,代理就可以检查单个时刻,而无需渲染整部影片;同时,我们也能并行渲染多帧。音频与画面共用同一时间轴,从而确保旁白、音乐和音效与视觉内容保持同步。Both engines render each frame based on its timestamp, so a frame at ten seconds looks the same regardless of when it's rendered. This lets the agent inspect individual moments without rendering the entire film, while also allowing us to render multiple frames in parallel. Audio runs on the same timeline, keeping narration, music, and sound effects synchronized with the visuals.
通过这样的设置,智能体几乎可以创建任何东西。但如果没有进一步指导,它就会默认采用模型自己的设计方案,而不是我们的方案。接下来的挑战是教会它遵循我们的品牌和设计规范。With this setup, the agent can create almost anything. But without further guidance, it defaults to the model's own design choices rather than ours. The next challenge was teaching it to work within our brand and design standards.
设计师的大脑Designer's brain
我们围绕四项不同职能组织了智能体的知识,每项都有明确用途。技能指导如何执行特定任务,而 wiki 则提供有关我们公司的背景信息,以及优秀工作的标准。凡是可量化的内容,都由 gate 负责处理——它是用代码编写的确定性检查。另有一个独立的记忆层,用于记录人们随着时间推移教给智能体的内容。We organized the agent's knowledge around four distinct functions, each with a clear purpose. Skills guide how to perform specific tasks, while the wiki provides context about our company and what good work looks like. Anything measurable is handled by a gate, a deterministic check written in code. A separate memory layer captures what people teach the agent over time.
技能Skills
代理接到请求后,首先要做的是判断这属于哪类工作。这由我们的媒体制作技能负责。我们称之为路由技能,因为它的大部分内容就是一张表。表格的每一行都将一种需求(营销活动、改版、缩略图、帖子、YouTube 套件、发布影片、短动画)映射到相应的处理路径和专门技能。路由器还会判断工作是否需要共享的视觉画布,并避免让小幅修改走完整流程。The first thing the agent does with a request is work out what kind of job it is. That's handled by our media-production skill. We call it a router skill because most of it is one table. Each row maps a kind of ask (a campaign, a restyle, a thumbnail, a post, a YouTube package, a launch film, a short animation) to the path and the specialist skill that handle it. The router also decides whether the work needs a shared visual canvas, and it keeps small edits out of the full process.
对于上面那样的电影,路由器会选择为指导大型动态影像制作而设计的 launch-film 技能。我们的 48 项技能总计近 1,000 个文件,每项都针对一项工作而非某个主题,因此其描述读起来像一条路由规则,其中还包括何时不应使用它。For a film like the one above, the router picks the launch-film skill designed for directing a large motion production. Each of our 48 skills, close to 1,000 files in all, is scoped to a job rather than a topic, so its description reads like a routing rule, including when not to use it.
各项技能相互组合,而不是彼此重复。launch-film 负责指导,motion-studio 负责制作画面和声音,因此动画技艺只需编写一次。launch-film 还包含它将启动的子代理的角色简介,以及一个真实案例研究:我们曾用代码逐镜头重建一部 88 秒的宣传片。代理正是通过这个案例研究学会如何将一部影片拆分给多个子代理。它会在明确的场景切换处剪开,让同一个子代理负责连续的镜头移动,并在并行作业开始前冻结共享代码。Skills compose instead of repeating each other. launch-film directs and motion-studio makes the picture and sound, so the craft of animation is written once. launch-film also carries role briefs for the subagents it will start, and a case study of a real 88-second launch film we rebuilt shot for shot in code. That case study is where the agent learned how to split a film between subagents. It cuts at hard scene changes, keeps a continuous camera move with one subagent, and freezes shared code before anyone works in parallel.
描述对于工具和技能同样重要。为了减少上下文占用,我们将 MCP 工具接入内置搜索。代理启动时只能使用我们自己的工具,并会根据需要通过搜索工具名称和描述中的关键词来加载其他工具的 schema。Descriptions matter for tools just as much as for skills. To keep context usage down, we put MCP tools behind a built-in search. The agent starts with only our own tools and loads additional tool schemas as needed, using keyword searches across tool names and descriptions.
维基百科Wiki
Skills 说明如何完成一项工作,而 wiki 则提供出色完成工作所需的背景信息和参考资料。内容涵盖品牌设计变量、视觉语言、品牌语调,以及平台、营销活动和产品信息等方方面面。另有专门的动态设计 wiki,涵盖导演、制作技艺、声音和评审,并收录了对 70 部参考影片的解析。Skills explain how to do a job, while the wiki provides the context and reference material needed to do it well. It covers everything from brand tokens, graphic language, and voice to platforms, campaigns, and product information. A dedicated motion wiki covers directing, craft, sound, and review, alongside breakdowns of 70 reference films.
我们没有在各项技能中重复这些信息,而是直接链接到 wiki。这样可以将共享知识集中在一处,也让更新更加轻松。比如字体规则发生变化时,我们只需更新一次,所有技能都能引用最新版本。Rather than duplicating this information across skills, we link directly to the wiki. This keeps shared knowledge in one place and makes updates easier. When something like a font rule changes, we only need to update it once, and every skill can reference the latest version.
了解我们的工作方式,并不一定能降低动态设计的成本。错误的创意决策仍可能耗费数小时的渲染时间。Knowing how we work doesn't necessarily make motion design cheaper. A wrong creative decision can still cost hours of rendering.
规范驱动设计Spec-driven design
动态效果一旦做错,代价尤其高,既耗费渲染时间,也消耗 token。渲染一帧分镜大约需要 50 毫秒,而完成影片中的一秒画面则需要 8 到 40 秒。再加上子代理要为每一幕编写、渲染和审核,成本很快就会累积起来。因此,代理会先在分镜中构思创意,以较低成本探索不同方向,等某个方向获批后,再进入完整制作阶段。Motion is especially expensive to get wrong, both in render time and tokens. A storyboard frame takes about 50 milliseconds to render, compared to 8 to 40 seconds for a single second of finished film. With subagents writing, rendering, and reviewing every act, those costs add up quickly. So the agent develops ideas in the storyboard first, where it can explore different directions cheaply, before moving into full production once a direction is approved.
为了支持这种探索,我们为智能体提供了一套参考影片库,供它汲取灵感,就像设计师根据灵感板开展工作一样。当它找不到合适的视觉风格时,就会使用图像模型生成新的参考图像:先快速出稿,再对最有希望的方案进行打磨。这些图像有助于确立视觉方向,而分镜和最终影片仍然是通过代码,使用真实的字体和品牌素材制作的。To support that exploration, we gave the agent a library of reference films to draw inspiration from, much like a designer working from a mood board. When it can't find the right look, it uses an image model to generate new references, starting with quick drafts before refining the most promising options. These images help establish the visual direction, while the storyboard and final film are still built in code using real typography and brand assets.
一旦我们确定方向,获批的分镜脚本就会成为后续影片制作的依据。接下来,项目将分为三个阶段:Once we pick a direction, the approved storyboard becomes the spec the rest of the film is built against. Then the project moves through three stages:
- 分镜脚本,用于就视觉构思、文案和镜头达成一致。A storyboard, to agree on the visual idea, the copy and the shots.
- 动态分镜是一种与音频同步的粗剪,用来检查静帧无法呈现的节奏。An animatic, a rough cut timed to the audio, to check the pacing that stills can't show.
- 制作阶段,子代理在此构建、检查并渲染已获批准的镜头。Production, where subagents build, check and render the approved shots.
我们最初在该技能的说明中描述了这一顺序,而智能体有时会为了节省时间而跳过审核。现在,流水线通过代码强制执行这一顺序,因此只有所有幕都通过审核后才会进行组装,而说明仍会指导创作决策。必须始终遵守的顺序由代码保证,判断则留在提示词中。We first described this order in the skill's instructions, and the agent sometimes skipped a review to save time. Now the pipeline enforces it in code, so assembly waits until every act passes review, while the instructions still guide the creative calls. The order that must always hold lives in code, and the judgment stays in the prompt.
指示,而非提示Point, don’t prompt
在这些方向之间做选择是团队的决定。Slack 很适合用来提出工作请求和接收工作,但线程是线性文本,而故事板是空间布局,因此反馈最终只能描述某个内容在哪里,而不能直接指向它;而且每次修改,哪怕只是改一个词,都必须重新通过代理提交为新的提示。Picking between those directions is a team decision. Slack works well for asking for work and receiving it, but a thread is linear text and a storyboard is spatial, so feedback ends up describing where something is instead of pointing at it, and every change, even a one-word fix, has to go back through the agent as a new prompt.
Paper 为我们解决了这个问题。它是一个专为智能体打造的设计画布,配有一个 MCP 服务器,让智能体可以直接控制文件。人们可以在同一个文件中协作,并排比较不同方向、针对特定画框发表评论,或亲自进行编辑。由于画布是共享的,智能体可以回到文件中,根据这些评论和编辑来指导下一轮迭代。Paper solved this for us. It's a design canvas built with agents in mind, with an MCP server that gives the agent direct control over the file. People can work in that same file, comparing directions side by side, leaving comments on specific frames, or making edits themselves. Because the canvas is shared, the agent can return to the file and use those comments and edits to guide its next iteration.
难点在于,Paper 的代理工具只能在其桌面应用中运行,而我们的代理则运行在云端沙盒中。为弥合这一差距,我们在一台始终在线的 Linux 工作节点上以无头模式运行 Paper Desktop,并通过虚拟显示器和经过身份验证的桥接来连接两者。一套配置即可支持整个团队。The challenge was that Paper's agent tools only run inside its desktop app, while our agent operates in a cloud sandbox. To bridge that gap, we run Paper Desktop headlessly on an always-on Linux worker, using a virtual display and an authenticated bridge to connect the two. A single setup supports the entire team.
即使获得批准,将分镜脚本制作成成片也可能需要数小时。到那时,这项工作更像一个小型项目:多个部分需要并行制作,最后再整合到一起。协调这一切,已超出单个智能体轮次所能处理的范围。Even after approval, turning a storyboard into a finished film can take hours. By that point, the work resembles a small project, with multiple pieces being built in parallel that need to come together. Coordinating all of this is more than a single agent turn can handle.
担任幕僚长的智能体Agent as a chief of staff
想想之前那个鹦鹉短片:制作进行到一半时,大家在 Slack 上协作。一个人分享参考素材,另一个人建议换个开场,还有第三个人对第四帧留下反馈,而代理还在制作中。这就像一场头脑风暴,只不过负责记录的人也要负责拍片。同步子代理在这种情况下效果不佳,因为主代理必须等它们完成,期间人们就无法与它互动。Consider the parrot film from earlier, midway through production in Slack. One person shares a reference, another suggests a different opening, and a third leaves feedback on frame four, all while the agent is still building. It's like a brainstorm where the person taking notes is also responsible for making the film. Synchronous subagents don't work well here because the main agent has to wait for them to finish, leaving no way for people to interact with it in the meantime.
为了解决这个问题,我们让子代理以异步方式运行,并将主代理的角色从实施者转变为幕僚长。它会把复杂项目拆分成更小、彼此隔离的任务,分配给子代理并行执行。与此同时,它会随着情况变化及时向团队同步进展,让大家能够继续提供反馈并把控项目方向。To solve this, we made subagents asynchronous and shifted the main agent's role from implementer to chief of staff. It breaks complex projects into smaller, isolated tasks, assigns them to subagents, and runs the work in parallel. Meanwhile, it keeps the team informed as things change, so people can continue giving feedback and steering the project.
对于每项任务,主代理都会创建一份子代理配置,明确说明其职责、相关背景、文件所有权以及如何验证工作成果。它还会根据任务的复杂程度选择模型和投入程度。For each task, the main agent creates a subagent profile with a clear brief covering its role, relevant context, file ownership, and how to validate the work. It also selects the model and effort level based on the complexity of the task.
我们还为主代理添加了一小组用于协调工作的工具:We also added a small set of tools for the main agent to coordinate work:
- start_task 会在后台将一部分工作交给子代理处理。
start_taskhands a part to a subagent in the background - check_tasks 显示正在运行的任务,并读取已完成的报告。
check_tasksshows what's running and reads finished reports - cancel_task 和 resume_task 可以停止不再符合需求的工作,或从中断处继续。
cancel_taskandresume_taskstop work that no longer fits the brief, or pick it up where it left off - report_progress 允许子代理在任务进行中发送更新
report_progresslets a subagent send updates mid-job - schedule_wakeup 为主代理设置一个计时器,以触发下一次运行并继续执行任务。
schedule_wakeupgives the main agent a timer to trigger another run and continue the job
在底层,每个子代理都是作为后台任务运行的普通 Deep Agents 任务工具。任务完成后,我们的代码会将结果放入队列,并据此启动主代理的新一轮运行。主代理运行期间,人们也可以随时引导它,因为工作过程中收到的消息会在它下一次调用模型前插入。子代理通过项目文件夹进行协作,各自只写入自己负责的文件,并将进度保存在磁盘上。Under the hood, each subagent is the ordinary Deep Agents task tool running as a background task. When it finishes, our code queues a new run of the main agent carrying the result. People can also steer the main agent mid-run, since a message that arrives while it's working is inserted before its next model call. The subagents coordinate through the project folder, each writing only the files it owns and keeping its progress on disk.
工作在后台运行时,线程必须展示当前进展。主代理会持续更新计划和预计完成时间、包含正在运行的子代理的实时状态行、渲染进度和开支;每次编辑都会收到确认,并且任何内容公开前都会显示一张审批卡片。With the work running in the background, the thread has to show what's happening. The main agent keeps it up to date with a plan and ETA, a live status line with running subagents, render progress and spend, an acknowledgment for each edit, and an approval card before anything goes public.
对这种方法最清晰的检验之一,是一项需要连夜为一份 52 页演示文稿更换视觉设计的任务。智能体先制作了三张样例幻灯片,并在继续处理演示文稿其余部分之前检查了自己的成果,之后按编号分批处理,以便逐批审批。过程中它还新增了两张幻灯片。整晚下来,主智能体运行了约一小时,而它的子智能体累计工作了六个多小时,整个过程都没有阻塞对话。One of the clearest tests of this approach came from an overnight job to reskin a 52-slide presentation. The agent started by creating three sample slides and reviewing its own work before moving through the rest of the deck in numbered batches for approval. It also added two new slides along the way. Overnight, the main agent ran for about an hour while its subagents collectively worked for more than six hours, all without blocking the conversation.
自我改进型评测框架The self-improving harness
与团队合作数月的设计师会逐渐了解团队的偏好,不必反复听取相同的反馈。没有记忆功能,智能体每次开启新对话都得从头开始。为避免这种情况,我们构建了两个学习循环:快速循环在大家给出反馈时及时记录,慢速循环则在后台从已完成的项目中学习。A designer who's worked with a team for months learns its preferences and doesn't need to hear the same feedback twice. Without memory, the agent would start from scratch with every new thread. To avoid this, we built two learning loops. The fast loop captures feedback as people give it, while the slow loop learns from completed projects in the background.
快速闭环:当有人分享一条规则或偏好时,代理会在同一轮对话中将其保存下来。Managed Deep Agents 将记忆以纯文本文件的形式分为两层存储:一层由整个团队共享,另一层则专属于每个人。The fast loop: When someone shares a rule or preference, the agent saves it in the same turn. Managed Deep Agents stores memory as plain files in two layers, one shared across the team and another specific to each person.
个人偏好会延续到不同对话线程中,但共享规范优先;为公司频道创作的内容则遵循公司的语气风格。每一层都有一个 AGENTS.md 文件,会在每次调用模型时加载。早期版本膨胀到 50 万个 token 后,我们现在将这些文件限制在 200 行以内。Personal preferences carry across threads, but shared standards take precedence, and anything created for a company channel follows the company voice. Each layer has an AGENTS.md file that loads with every model call. We now limit these files to a 200-line index after an early version grew to 500K tokens.
缓慢循环:每项任务完成后,我们的 learn_from_job 工具都会启动一个轻量级子代理,提炼最多五条有证据支持的经验教训,并将其与所用技能关联起来。之后的任务会读取并评估这些经验教训;每周一次,后台代理会将经证实有用的经验教训纳入技能和 wiki。The slow loop: After each job, our learn_from_job tool launches a lightweight subagent that captures up to five evidence-backed lessons and associates them with the skill used. Future jobs read and rate those lessons, and once a week, a background agent incorporates the ones that prove useful into the skills and wiki.
接下来是什么?What’s next
还记得早期的那些评审吗?最大的改进在于,我们把设计师的判断沉淀到一个智能体可以从中学习的共享环境里,而不是任由这些判断散落在各个评审讨论串中。在这个过程中,我们也让更多人能够参与设计工作。Remember those early reviews? The biggest improvement came from capturing our designers’ judgment in a shared environment the agent could learn from, rather than leaving it scattered across review threads. In the process, we made design work accessible to more people.
接下来,我们希望代理能够管理整个营销活动生命周期,其中部分功能已经实现。获得授权的审批人批准后,它会通过 Buffer 发布公司的社交媒体帖子,为员工起草可供他们从个人账号分享的帖子,并运行定时例行任务,包括每天审查并推荐值得引用的帖子。Next, we want the agent to manage the entire campaign lifecycle, and some of that is already working. It publishes company social posts through Buffer once an authorized approver signs off, drafts posts for employees to share from their own accounts, and runs scheduled routines, including a daily review that suggests posts worth quoting.
我们的目标是更进一步,让一个获批的创意方向就能在同一个对话线程中贯穿整个营销活动。智能体将撰写简报、为各个渠道制作素材、安排帖子发布、协助团队扩大传播范围,并利用效果数据为下一次营销活动提供参考。在所有内容公开发布之前,仍会由人工审核并批准。Our goal is to take this further, so a single approved creative direction can carry an entire campaign from one thread. The agent would write the brief, create assets for each channel, schedule posts, help the team amplify them, and use performance data to inform the next campaign. People would still review and approve everything before it goes public.
构建你自己的设计代理:这个开源仓库包含了本文介绍的运行框架:沙盒计算机、技能、维基和记忆、异步子代理,以及 Slack 用户体验。Fork 这个仓库,添加你们的品牌令牌、字体和技能,然后通过 Managed Deep Agents 将它部署到团队的 Slack 中。10 月 28 日,欢迎参加 GTM Engineering Live。我们将深入介绍运行框架和用户体验,并展示本文未能涵盖的内容。Build your own design agent: The open-source repo has the harness from this post: the sandbox computer, skills, wiki and memory, async subagents and the Slack UX. Fork it, add your brand tokens, fonts and skills, and deploy it to your team's Slack with Managed Deep Agents.
Then join us at GTM Engineering Live on October 28. We'll go under the hood of the harness and UX, and show the work that didn't fit in this post.