OpenAI 为何竞速打造超越人类的电脑操作能力

简要描述Brief Description

OpenAI 的 Ari Weinstein 和 James Sun 与 a16z 的 Seema Amble 和 Fabrizio Serafini 探讨了使用计算机的智能体。他们讨论了速度与可靠性、企业管控、日常自动化、遗留软件、产品设计，以及人与智能体之间更主动的协作等话题。OpenAI’s Ari Weinstein and James Sun discuss computer-using agents with a16z’s Seema Amble and Fabrizio Serafini. They explore speed and reliability, enterprise controls, everyday automation, legacy software, product design and more proactive collaboration between people and agents.

目录Table of Contents

• 简介
  Intro
• GPT-6 Astra 为消费者和企业带来的新可能性
  What GPT-6 Astra unlocks for consumers and enterprises
• OpenAI 团队如何使用点号
  How the OpenAI team uses dots
• 迈向超人类水平：速度、成本与控制
  Approaching superhuman: speed, cost and control
• 开放网络上的智能体与广告问题
  Agents on the open web and the ad problem
• 将哪些任务交给代理处理
  Which tasks to hand over to an agent
• 让企业信任计算机使用功能
  Getting enterprises to trust computer use
• 最佳企业级应用场景：平凡之处蕴藏机遇
  The best enterprise use cases: boring is opportunity
• 智能体会运行遗留软件，还是取而代之？
  Will agents run legacy software or replace it?
• 给基于计算机使用功能构建产品的创始人的建议
  Advice for founders building on computer use
• 计算机使用的下一步是什么
  What’s next for computer use

简介Intro

我们现在已经让计算机操作达到了比普通人更快的水平。我认为我们离超越人类的水平已经非常近了。We've gotten computer use now to a place where it's faster than the average human. And I think we're getting really close where it is superhuman.

参与者：你认为哪些类型的任务已经可以交给代理处理，哪些任务还需要检查和修正？Participant: What sorts of tasks do you think are ready to hand over to the agent versus the ones like okay it still requires some checking and correcting

参与者：现在是使用计算机的智能体，因为 Astra 现在已经能够做任何人能做的事。我们一直在思考，借助更快、更好的模型，我们能带来哪些新的消费者体验。至于企业端，安全问题自然就变得更值得关注。我们如何让人们能够控制其中人工参与的程度？又该如何确保可审计性？Participant: computer using agents now since Astra is now capable of doing anything that a person would do. We think a lot about what new consumer experiences can we enable with a faster better model. And then on the enterprise side that's kind of naturally why the safety becomes more interesting. How do we give the ability to control the amount of human the loop there is and then the auditability?

这和我们构建过的其他任何东西都不一样，因为开发者得做一些工作才能采用它。得有人构建一个连接器 MCP 服务器。你知道，计算机使用功能适用于所有东西。现在我们有了能力很强的模型。It's unlike any other thing that we've built where the developer has to do some work to adopt it. Someone has to build a connector MCP server. You know, computer use works with everything. Now we have like capable models.

它们的成本正变得足够低，可以大规模运行。公司应该考虑自动化哪些工作？They're getting cheap enough to run at scale. What should companies think of automating?

参与者：我认为它可以分为两大类。Participant: I think of it as in two big categories.

GPT-6 Astra 为消费者和企业带来的新能力What GPT-6 Astra unlocks for consumers and enterprises

第一类就像这样：参与者：欢迎收听 A16Z 播客。我是 Sema Amble，今天和我的同事 Fbritzio Saraphiniti 一起主持。八月，我们写了一篇关于使用计算机的智能体的文章，介绍它们如何开始在公司内部承担真正的工作。今天，我邀请到了 OpenAI 的 Ari Weinstein 和 James Sun，他们将从自己的视角帮助我们梳理使用计算机的智能体相关动态，并分享一些对未来的看法。Ari、James，感谢你们来参加节目。关于……The first category is like Participant: Welcome to the A16Z podcast. I'm Sema Amble. I'm joined by my colleague Fbritzio Saraphiniti. In August, we wrote about computer using agents and how they're starting to do the real work inside companies. today I'm joined by Ari Weinstein and James Sun from OpenAI who are going to help unpack what is going on with computer using agents from their perspective and share some thoughts on the future. Ari James, thanks for joining us. Of

参与者：当然，很高兴来到这里。是的，谢谢邀请我们。Ari，开始吧，跟我们聊聊 Astra 中的哪些进展推动了计算机操作智能体的发展，又有哪些真正实现了突破。Participant: course, happy to be here. Yeah, thanks for having us. Ari, kick it off and tell us a little bit about what in Astra has made computer using agents move forward and what's really unlocked.

参与者：是的。所以 Astra 是一个很棒的模型，很多方面都很出色。在 Astra 出现之前，我们的计算机使用模型就已经很厉害了，你知道，像 Soul 本身在使用计算机方面就非常出色。所以根据我的一些个人体验，Astra 确实在一些细节上提供了帮助。Participant: Yeah. so Astra's an amazing model. It's good at so many things. Our computer use models were already so good before Astra, you know, like Soul itself is amazing at computer use. So Astra really helps with a few things around the edges in my sort of anecdotal experience.

一是平均来说它快一点，因为完成同样的任务时用的思考 token 更少。它在某些类型的创意工作上也更出色。比如，我们的发布视频里有个演示：你跟它对话，它就在 Microsoft Paint 里画东西。它在这类事情上要好得多，我觉得这种能力可能也能迁移到其他类型的创意任务上。最后一点，我发现它在一些边缘情况下更可靠。就是，你知道，我觉得我更信任它一点。One is it is a little bit faster on average because it uses fewer thinking tokens to do the same task. It's better at certain kinds of creative work like for example we had this demo where in our launch video where you were talking to it and it's drawing things on Microsoft Paint. it's way better at that sort of thing which I think probably transfers to other kinds of creative tasks also. And then the last thing I find is it's just more reliable around the edges. Like you know it's kind of I feel like I trust it a little more.

就像是这样一类事情：99% 可靠和 97% 可靠之间有一点差别，不太容易察觉，但却非常重要。我觉得它在某种意义上就是高了一点，而且这种感觉相当具体。like it's sort of the kind of thing where there's a little difference that is hard to perceive but really important between something being 99% reliable and 97% reliable. And I feel like it's sort of just a little higher in a way that feels pretty concrete.

参与者：是的。我想补充一点，Astra 也是我们对齐程度最高的模型。是的，没错。所以，你知道，我们稍微聊聊信任这个话题。是的。Participant: Yeah. And I think adding on to some of those things, Astra is also our most aligned model. Yeah. Right. And so, you know, talking about trust a little bit. Yeah.

参与者：你知道，如果你看看它的系统卡，就会发现它在抵御提示注入方面强得多，也更不容易采取任何非预期的行动。所以你想想，企业用它来处理工作，尤其是公司用它做敏感事务，比如采购、和供应商沟通，或者和客户交流，就会觉得使用起来安全得多，这真的很令人兴奋。创意方面也有些有趣的事。我想我们内部让 Astra 和 5.6 玩电子游戏，它每次都赢。Participant: you know, if you look at the system card for it is much better at being resistant to prompt injection. it's much better at not taking any kind of unintended actions. And so when you think about it for like enterprise work, especially for companies using it for sensitive things, whether it's like procurement or you know talking to vendors or talking to customers, it's going to be just a you're going to feel a lot safer kind of using it and that's really exciting. And then so some fun things on the creative side. I think internally we had you know Astra play video games against you know 5.6 and it wins every time.

所以有人发了一些很酷的推文，比如说 Astro 能在大约 30 小时内通关《辐射》之类的。它玩电子游戏也确实非常厉害。And so there are some cool tweets that people have made where, you know, Astro is able to beat Fallout in like 30 hours or something like that. And so it's also just really amazing at playing video games.

所以我也不知道这大概意味着什么，不过，确实非常令人兴奋。So I don't know that probably translates to something, but yeah, it's very exciting.

发言者：也许可以再深入聊一下这一点：计量代理在消费者和企业用例中的应用，会对我所说的计算机使用型代理周边的各种支撑体系产生截然不同的影响。你们怎么看？如果要选的话，消费者和企业这两个方向，哪边更令人兴奋？Participant: Maybe just to double click on that point, consumer versus enterprise use cases of metering agents, very different implications around kind of like all the scaffolding around I would say the computer using agents. How do you guys think about where things are more exciting between consumer and enterprise if you were to pick?

参与者：我想你知道，我们在 OpenAI 的目标其实就是打造对所有人都有用的产品。所以你知道，这件事很有意思，因为 Ari 显然更偏向消费级产品，而我花在企业领域的时间多一些。我认为，消费级领域始终是我们尝试推动可能性边界的地方。你知道，正如 Ari 提到的，Astra 比之前的模型快得多，所以我们会认真思考：更快、更好的模型能让我们实现哪些新的消费级体验？而在企业领域，安全问题自然就变得更值得关注了，因为我认为，每当我们和大型公司交流时，企业计算的应用场景其实相当明显，对吧？企业内部有很多没有任何 MCP 工具的老旧软件，它们已经被使用了 15 年、20 年，而且很多时候，普通人其实很难使用它们。Participant: I think you know our goal at OpenAI is really about building products that's useful for everyone. And so you know actually it's interesting because Ari is definitely more of a consumer guy here and I spend a little bit more time on the enterprise side. I think the consumer is always the place where we try to push the frontier on what's possible. you know with Astra like Ari mentioned there is it's a much faster compared to the previous models and so we think a lot about like what new consumer experiences can we enable with a faster better model and then on the enterprise side that's kind of naturally why the safety becomes more interesting because I think the enterprise whenever we talk to large companies I think the use cases for comput is actually fairly obvious right inside enterprises there's lots of legacy software that doesn't have any MCP tools they are software that's been used for like 15 20 years and so there's and a lot of times it's actually kind of hard to use for a normal person.

对吧？很多界面都还是差不多十年前的样子。所以你甚至不知道该怎么操作一些关键功能。我们看到有些公司开发了一些技能，教模型更好地使用这些功能。我觉得或许可以这样理解：在企业领域，这些能力可能早就存在了一段时间，公司现在才开始跟上，试着弄明白该如何应对；而在消费领域，我们则是在探索下一类用例——这些用例或许是当下的软件还无法实现的。你也可以再多分享一些。right? Lots of like UI that is like from the last, you know, decade. And so you don't even know how to operate some of the key things. And so we've seen companies build skills that kind of teach the model to be able to use them better. And so I think maybe the way to think about it is enterprise is where, you know, the capabilities have been there maybe for a little bit longer and the companies are catching up trying to understand how to deal with it and consumers where we're trying to figure out the next types of use cases that's not maybe possible today with software. Maybe you can share more.

参与者：是啊，我的意思是，面向消费者的应用场景对我来说特别有意思，因为这有点像是在想：怎样才能让你的智能体帮你处理日常生活中那些特别烦人的事情，帮你节省大量时间，让你有更多时间专注于那些……Participant: Yeah, I mean the consumer use cases are so fun for me because it's kind of like how can you put your agent to work doing the things in your daily life that are so annoying and save you tons of time and just have more time to focus on the things that

参与者：你对此感到兴奋。我觉得看到 Dot 真的很有意思，你知道，每个 Dot 都有自己的电脑，你可以让它在云端帮你做事。它有一个完整的网页浏览器，可以帮你买东西，也可以帮你预约理发，或者做任何你想做的其他事情。所以这真的很有意思，我觉得未来还有很多值得期待的东西，会非常有趣。Participant: that you're excited about. I think it's been really fun to see dot where you know each dot gets its own computer and you can ask it to do things for you in the cloud. it gets a full web browser. It can buy things for you. can you know book haircut appointments and you know anything else you wanted to do. So that's been really fun and I think there's a lot more on the horizon. That's going to be really fun.

OpenAI 团队如何使用 DotsHow the OpenAI team uses dots

参与者：是啊。我知道这本来就是我想问的关于 Dots 的问题。我觉得 James，你可能会用 Dots 来梳理产品反馈。如果你能多分享一些这方面的内容就太好了；我也特别好奇，你们是怎么使用 Dots 的，以及电脑操作在你们能用它们做的事情中扮演什么角色。Participant: Yeah. Know that was going to be my question on dots. I think James you probably use a dot to walk through product feedback. It would be cool if you could share more about it or I'm just so curious to know on your end guys like how are you using dots and like how does computer use play into kind of what you can do with them.

参与者：是的，我觉得 Dots 确实有几个地方特别棒，对吧？其中一点是它非常主动，所以很多时候，无论是在我的个人生活还是工作中，Dots 会直接提醒我一些我完全没注意到、就这么错过的事情，这让我很惊讶。就前几天吧。我们内部的邮件系统有个问题：收到外部邮件中的会议邀请时，邀请不会显示在我们的日历上，所以我经常会错过一些外部会议。现在 Dots 会帮我把这些情况都揪出来。Participant: Yeah, I mean I think dots is really cool for a few reasons, right? One is like really proactive and so I've been very surprised a lot of times when in both my personal life and work when dots will just flag something that I just completely slip through the cracks, right? Just the other day. So internally like on our email system whenever we get an external email for an invite it doesn't actually show up on our calendars and so I end up missing a bunch of like external meetings and so dots now is like catching these things for me.

就像，哦，你收件箱里有个邀请，但日历上没有。要不要我直接在 Gmail 里帮你接受，这样它就会显示出来？这可真是帮了大忙。It's like oh there's this invite that's in your inbox. It's not on your calendar. Do you want me to like accept it in your Gmail so it shows up? So that's been a lifesaver.

这只是件小事，但确实很重要。不过我也让它主动去看看。我觉得它很擅长主动寻找那些任务形态并非每次都那么——怎么说呢——具体明确的任务。It's a small thing but it matters a lot. but I also have it proactively kind of looking at I think it's really good at proactively looking for tasks where it's the shape of it is not super let's say concrete every single time.

举个我还没设置好的例子：我们内部用 Slack，消息一直很多。我以前用自动化和编解码器搭了个 Slack 任务，每小时或每 30 分钟去拉取一次消息，但最后我把很多通知都屏蔽了，因为有些时候有用，有些时候没用。用 DOTS，我可以设定一个比较高层次的目标。比如说：“嘿，只要我的汇报链条里有人说了什么，就标记出来告诉我，对吧？如果有人给出了重要反馈，就根据你对某些产品的判断来处理。”而且因为它有一台云端计算机，我还搭建了一个相当酷的工作流：有人报告问题时，他们通常会附上 Sentry 日志。So, an example that I also haven't set up to do is we use Slack internally and there's a ton of messages that's going on at all times and I used to have this Slack job using automations and codecs that's like every hour or every 30 minutes go and try to pull things and I've just ended up tuning out a lot of the notifications because it's just like sometimes useful sometimes not. With DOTS, I can give kind of a high level goal. It's like hey if anybody in my reporting chain says anything flag it to me, right? if anybody who's giving some critical feedback, use your judgment about certain products, go for it. And then because it has a cloud computer, you know, I've had this pretty cool workflow where when someone reports, they usually have like a sentry log.

我可以让它使用云端电脑，你知道，实际操控我的本地浏览器，而 DOS 会同时控制你的本地电脑和云端电脑。I could have it go use the cloud computer, you know, actually drive my local browser while the DOS's going to control both your local computer as well as the cloud computer.

登录 Sentry，拉取日志，进行分析，然后给我反馈。这样一来，判断哪些反馈重要、查看反馈、再去 Sentry 下载日志、将日志输入编解码器并征求意见等整个流程，现在基本上都可以自动化完成。Log into Sentry, pull the logs, analyze it, give me feedback. And so it's saving that entire loop of figuring out which feedback is important, looking at it, then going to Sentry, downloading the logs, feeding into codecs, asking for thoughts, and all that can now just kind of be automated.

逼近超人水平：速度、成本与控制Approaching superhuman: speed, cost and control

所以，是的，DOS 至少很适合处理这些主动型、某种程度上长期运行的任务，因为它们每次的形式都不完全一样。And so, yeah, DOS is great for at least these proactive and kind of longunning tasks where they're not the same shape every single time.

参与者：我们稍微谈了谈可靠性、速度、token 使用量和效率。随着你们不断改进并推出新产品，你们是如何权衡这些因素的？我觉得 OpenAI 对 computer use 非常兴奋，也在这方面投入了很多；现在我认为，我们已经打造出了速度最快、成本最低、最可靠的 computer use 产品。而且我觉得，这项技术还有很大的发展空间。我们花了很多时间研究速度。你知道，速度这个问题很有意思，因为一方面，computer use 的优势在于它可以在后台执行任务。Participant: We talked a little bit about reliability, speed, token usage, and efficiency. How do you guys think about balancing all these things as you keep making improvements on what you're releasing? I think we're really excited about computer use at OpenAI and we've invested in it a lot and we've gotten to a place where I think we have the fastest, cheapest and most reliable computer use product. and I think there's actually so much further we can take that. we spent a lot of time looking at speed. You know, speed is really interesting because on one hand the power of computer use is it can do tasks in the background.

因此，在某些情况下，所需时间长短并不重要，因为它可以替你完成任务，而你可以同时去做其他事情。但有时你也会想要盯着它，确保它做得正确。So in some cases it doesn't matter how long it takes because you can be going and doing other things while it's accomplishing the task for you. But also sometimes you want to watch it to make sure it does it correctly.

你得等它完成，因为在它完成之前，其他事情都无法进行。所以速度非常重要，因为你肯定不希望产生这种感觉：哦，我自己做就行了，那样还轻松得多。现在，我们已经让计算机操作的速度超过了普通人使用计算机的速度。我觉得我们正越来越接近一个超越人类的水平，也就是说，它比像我们在座某个人这样的计算机使用专家还快。我认为，一旦达到这个水平，局面就会发生根本性的变化。You need to wait for it because it needs to complete this before something else can happen. And so the speed is so important because you know you don't want to get this feeling like oh I could have just done that myself and it would have been so much easier. So you know we've gotten computer use now to a place where it's faster than the average human at using a computer. And I think we're getting really close to a place where it is superhuman, meaning it is faster than an expert computer user like probably one of us at this table. and I think that's going to actually be a sea change when that happens.

我认为，这会改变我们所有人与计算机交互的方式，因为突然之间，凡是你知道可以手动完成的事情，它都能更快地完成。所以，除非你出于某种原因真的想亲自去做——我觉得其实人们可能有很多理由想这么做——否则通常情况下，你大概只会想让智能体替你完成。I think it's going to change the way we all interact with computers because all of a sudden anything that you know that you could do manually, it could do faster. So unless you really want to do it yourself for some reason, which I think actually there may be many reasons people would want to do that, but in a default case, you probably just want the agent to do it for you.

我认为，这将让我们能够打造各种很棒的用户体验。因此，速度是我们非常重视的一点。显然，我们的策略是提供多种不同的模型，涵盖不同的能力水平和价格区间。针对不同的使用场景，你可能会想使用不同的模型。与此同时，随着时间推移，我们价格较低的模型在计算机操作方面也越来越强。我认为，未来人们会根据不同任务选择不同模型，这种选择会变得更加多样。And I think that's going to let us build all kinds of cool user experiences. So speed is something we think about a lot. And then obviously we have a strategy of offering multiple different models at multiple different you know qualities and price points. And so for different use cases you may want to use a different model. at the same time our less expensive models are getting much better at computer use over time. And so I think there will actually be more variation over time in which models people want to use for which tasks.

补充一点，我觉得另一个值得思考的问题是：我们如何让客户能够掌控安全性、可靠性、自主性和速度之间的平衡？比如说，如果模型速度比较慢，你可以看着它做事，但这样你能做的任务就受限了，因为你不可能一直坐在这里盯着它做事。这其实效率更低，对吧？因为它比你慢。但一旦发展到你无法再观察和审核它所做的一切，我们该如何让企业和商家尤其能够控制人工参与的程度，以及可审计性呢？ 我记得几个月前，作为 Access 的一部分——几个月前，作为 Astra 的一部分——我们推出了一套企业控制功能，让企业可以选择人工参与的程度。比如，他们是否希望代理自动批准所有操作。我们有一个非常棒的系统，叫 Guardian；还有应用里的自动批准功能。我觉得它是一个非常智能的模型，会审查计算机使用代理采取的每一个操作，判断是否安全、是否符合预期。因此，消费者可能会想：“我就想让它把所有事情都做了，不想让它征求我的批准。”而企业则可以决定哪些类型的操作可以直接执行。所以我觉得，随着我们的计算代理也变得超越人类水平、能做更多事情，控制能力会变得越来越重要。adding on saying like I think another thought is like how do we give customers you know control over the safety reliability autonomy versus speed right like if you have like you know when the model is slow you can kind of watch it do stuff but then you're limited to that to the task that you can do because you can't just sit here watching the thing doing it like it's actually less effective right because it's slower than you but once you can get to a point where you can no longer observe and audit everything it's doing how do we give especially enterprises and businesses the ability to control the amount of human the loop there is and then the auditability right like I think a few months ago it's part of access a few months ago it's part of Astra you know we kind of launched a set of enterprise controls that let businesses choose one like how much human in the loop is there like do they want the agent to automatically approve everything we have this really awesome system called guardian right or auto I think auto approve in the app that is a really smart model looking at every single action that the computer use agent is taking to decide if it's safe it's if it's aligned so there's no intent a consumer might be like I just want it to do everything and I don't want it be asked for approval whereas an enterprise is able to kind of decide which type of actions actually go through and so I think the control part becomes more and more important as our computing agents become also superhuman and doing things

开放网络上的智能体与广告问题Agents on the open web and the ad problem

参与者：完全同意。我觉得，随着你们尝试推出更多企业级应用场景，雇主们对效率提升感到兴奋，但他们也会担心究竟会释放出什么。Participant: totally I think as you try to you know roll out more enterprise use cases I think employers They're excited about the efficiency, but they're also, you know, wary of like what is what are what's sort of being unleashed.

参与者：也许甚至在另一边，Participant: Maybe even on the other side,

参与者：你们有没有看到应用软件方面的回应？大概是说，现在有计算机使用智能体在访问我的旧版工作流软件。你们对此怎么看？你们是不是更关注那些面向消费者或企业用户的智能体？嗯，我觉得这个问题在开放网络上会变得非常有意思。我觉得，如果是在公司批准并使用的软件内部，事情其实就更简单明了，因为我们当然百分之百希望大家能在这里自动化处理工作，不必再自己繁琐地一点点完成。Participant: have you guys seen the response from application software around okay, now there's computer using agents accessing my legacy workflow software and how do you guys think at all about that? Are you more focused on just the agent that's being used by the you know consumer enterprise user? M yeah I think there's like I think this problem becomes really interesting on the open web actually. I think when it's like within software that is like approved and used by the company it's actually more straightforward because it's like oh yeah 100% we want to have people automate and do work in here that they don't have to do tediously themselves.

但说到互联网，我觉得大家大致可以分成几个不同阵营，对吧？其中一种是：我们的商业模式并不适合让智能代理使用我们的产品，对吧？But when it comes to the internet, I think you kind of have a few different camps of folks, right? One is we run a our business model is not in line with having agents use our product, right?

你会想到广告商，以及商业模式完全依赖广告的出版商。如果这些代理代表用户使用网站，那么广告收入从哪里来？而且我认为，在大多数地方，代理产生的广告展示实际上并不计入真正的收入。所以这是一个很有意思的问题，我们一直在思考，也一直在和合作团队一起探讨。另一个相关的问题是，对于那些确实希望拥抱计算机使用代理访问其网站的公司来说，很多公司正在采用某种标准，让人类或用户和代理能够一起使用这些网站。You think about advertisers, publishers whose business model is entirely on advertisement. If the agents using the website for them, then where's the ad revenue coming from? And I think most places a agent's ad impression does not actually count towards real revenue. and so that's like an interesting question that we're always working through and working with our partnerships team on. there's another question kind of around for the companies who do really want to embrace having computer use agents using their sites. A lot of them are adopting kind of standards that let humans or people and agents to be able to use it together.

比如说，WebMCP 是一项让我们非常兴奋的标准。我们已经在应用内浏览器中加入了对它的支持。你可以想象这样的场景：智能体提升了用户使用网站的体验，但并没有完全取代网站本身。So like WebMCP is a standard that we're really excited about. We added support for in the app browser. And so you can kind of imagine these scenarios where the agent is elevating the experience of using the site, but it's not entirely replacing it.

所以我觉得，在我看来，浏览互联网还是很有意思的，对吧？我还是喜欢购物，也喜欢浏览各种酒店、航班和目的地。你可以想象这样的场景：智能体包揽所有繁琐的工作，同时也设计出一种用户体验，让你亲自浏览；等你觉得“好了，现在我要完成交易，输入信用卡信息，做完这些操作”时，再把事情交给智能体。因此，Web MCP 是一种挺有意思的方式：你仍然是通过网站本身而不是 API 与网站交互，同时也由人来配合使用。So like I think browsing the internet is still very fun in my opinion, right? I still like doing shopping. I still like looking at all the hotels and flights and places. And so, you know, you can imagine places where the agent does all the tedious work, but it kind of carves out an user experience where you're doing the browsing be like, "Okay, now I want to finish the transaction and then put in my credit card and do all this stuff, then you can hand it back to the agent." And so, Web MCP is kind of an interesting way where your interface with the website is still the site itself versus an API, but then the person is kind of using it together.

以上就是我的一些想法。And so, those are some thoughts.

哪些任务适合交给智能体处理？Which tasks to hand over to an agent

是的。换个话题，我们想稍微深入探讨一下，完成这项工作意味着什么，或者说如何端到端地把工作完成。我们很想听听你们对此的一些看法，Yeah. Shifting gears, we want to double click a little bit on what it means to do the job or for get the end to end job done. we'd love to hear a little bit from you guys around,

参与者：你知道，哪些事情效果不错。你们如何界定合适的、范围狭窄且可重复的任务？你们认为哪些任务已经可以交给智能体处理，哪些任务还需要用户这边进行检查和纠正？Participant: you know, what works well. How do you guys define the right narrow repeatable task ver you know and what sorts of tasks do you think are ready to hand over to the agent versus the ones like okay it still requires some checking and correcting as the on the user side.

M 我的意思是，总的来说，我发现这个模型相当能干，基本上现在你在电脑上能做的任何事，它都能做。你知道，我会努力让自己做的一件事是：每当我有一项任务，首先就想，我能不能直接让智能体来做？当然，有些情况比较敏感，比如我要给别人发消息时，我希望对方感觉这是我亲自发的。但总的来说，我的想法是，现在有了智能体，电脑……因为 Astra 现在已经能做到人能做的任何事了。我觉得以前大家会有些疑问，比如在具体程度和需求方面存在一些模糊之处。但我总体上发现，当你让模型做某件事，而要求又有些含糊时，它其实已经做得相当不错了。比如我觉得我们推出 Astra 时，加入了这个“灵活输入”功能，对吧？所以它会问你：“嘿，我还需要一些细节。”如果你没及时回复，它就会继续做下去，然后提取所需的上下文。我觉得，对于那些可重复的工作流程，最好能提前明确需求，有点像撰写产品规格说明。M I mean generally I found the model to be pretty capable to do anything that you can do on a computer now like and you know one exercise that I try to force myself to do is that whenever I have a task the first thing I think about it is just can I just have the agent do it now there are obviously some sensitivities around you know if I'm sending a message to someone I want to feel like it's coming from me but in general I think the way I think about it is the computer using agents now since Astra is now capable of doing anything that a person can do and there are questions around I think before of like oh there are some like squishiness around like specificity and requirements I' I've generally found that the model is getting pretty good at whenever you're asking it to do something and there's some vagueness it will actually you know with Astra I think we launched this like flexible input right and so it will actually ask you about hey like I want some details and if you don't answer in time it'll just keep going and then pull the context that's necessary I think there are some things around you know if you have repeatable workflows it's nice to be able to kind of define, you know, the requirements early, similar to like writing a product spec.

但总的来说，我觉得除非有特别敏感、需要你参与其中的事情，否则我现在做什么都会直接找电脑模型帮忙。我想知道你是不是也一样。but in general, I generally feel like unless there is something incredibly sensitive that you want to be in the loop for, I just reach for the computer model for everything I do now. I'm curious if the same for you.

参与者：我……是的，确实如此。我在努力培养伸手去用它的冲动，因为你知道，它真的很好用，而且有时候人很容易陷入旧的做事方式。我觉得，计算机使用这项技术真正很酷的一点是它的普适性。你知道，它适用于所有应用程序。Participant: I Yeah, it is. I'm trying to you know, build that impulse to reach for it because, you know, it's so good and sometimes it's so easy to get locked into the old way of doing things. I think the really cool thing about computer use as a technology is the universality of it. You know it works with every application.

这和我们构建的其他任何东西都不一样：其他东西需要开发者做一些工作才能采用。有人必须构建一个连接器 MCP 服务器。而 computer use 可以与所有东西配合使用，其工作方式是像人类一样使用应用程序。我认为，如果你没有花太多时间了解这项技术，这一点值得理解。It's unlike any other thing that we've built where you know the developer has to do some work to adopt it. Someone has to build a connector MCP server. You know computer use works with everything. And it works by using the application the same way a human uses it. which I think is something worth understanding if you haven't spent much time with this technology.

它就像是……它通过计算机操作，像人类一样使用计算机。它让智能体拥有你所具备的所有能力。It like it computer use uses computers the same way a human does. It makes the agent have all the capabilities you do.

我认为这非常强大。which I think is so powerful.

参与者：是的。我想补充一下 Ari 的说法，其中一部分是，我觉得随着时间推移，我已经不再把它想成“我是在计算机使用任务还是非计算机任务中使用它们”这种问题了。它们经常是相互关联的，对吧？我们对计算机使用的看法是，计算机使用并不是一种你拿起来就说“我想用它做点什么”的东西。Participant: Yeah. And I guess like adding on to Ari saying like part of it is like I think over time I've stopped thinking about it as like am I using them on like a computer use task or a non-computers task. Frequently they're like connected, right? Like the way we think about computer use is that like computer use is not like a thing you pick up and just say I want to do something with it.

它确实很像一种通用 API，模型可以借助它尽其所能完成任务。所以，如果你有一个雄心勃勃的任务，希望智能体来完成，那么任务越复杂，它在执行过程中某个时候用到计算机操作的可能性就越大。也就是说，只有在某些任务特别简单、我自己点两下就能搞定的时候，我可能才不会选择用计算机操作。It's really like the universal kind of API that the model has to do whatever it takes. And so if you have an ambitious task and you want the agent to do it likely the more complex it becomes the more likely it will probably touch computer use at some point in that journey. and so like maybe the only time that I don't reach for computer use for some cases is if the task is so simple that then it's like easier for me to just go and like click two buttons and do it.

但你知道，随着使用电脑的速度越来越快，问题就变成了：也许我不需要那么做，我只要用语音来操控，它就能和我一起使用电脑。嗯，这些想法也挺有意思的。是啊。But you know as computer use gets faster even faster then the question becomes maybe I don't need to do that I just you know drive voice and it just uses the computer with me. so those are some interesting thoughts there too. Yeah.

让企业信任计算机使用功能Getting enterprises to trust computer use

你觉得有什么需要跟企业用户分享，才能让他们对此感到放心吗？显然，你们已经彻底迷上了计算机操作，但我觉得企业用户有时显然会有所顾虑：好吧，是的，人类会点击操作，而且可能做得更糟；但他们还是会觉得，至少出了问题有人负责，而且你知道，那个人受过培训，诸如此类——都是那些经典的反驳理由。我挺好奇的。Do you think there's anything you have to share with like an enterprise user to get them comfortable with that? Obviously, you guys are totally pilled on computer use, but you know, I think there's obviously like hesitance sometimes on enterprise user of like, okay, well, yes, the human would click through it and probably do it worse, but there is that feeling of like there's accountability and you know, that person's trained and all you know, all the classic kind of rebuttals. I'm curious

参与者：必须说服周围的人。Participant: have to convince people around.

参与者：当然。最近这段时间，我们和企业进行了很多交流。我认为最重要的是，我们把它看作两个层面。第一层是：我们如何创建确定性控制？我认为，这确实能让企业感到很安心。虽然我们的模型是有史以来对齐程度最高的，而且我认为它在防范提示注入之类的问题上也比大多数人做得更好，因为它的智能确实非常强大，但我认为，当企业管理员能够明确规定，某些任务或某些数据源敏感到不能被使用时，他们就会有一种安全感。因此，我们看到企业方面的需求……所以我们推出了两项功能，对吧？一项是能够以确定性的方式为你的智能体创建一个沙箱。Participant: For sure. lots of conversation with enterprises kind of over the last little bit and I think the main thing is we think about it as two layers right the first layer is like how do we create deterministic controls I think that's really what gives you know enterprises a lot of feeling of safety is when they can even though our model is the most aligned ever and it's I think better you know better than most people at you know protecting against prompt injections and things because it's like just you know the intelligence is so smart but I think there is just a feeling of safety when the enterprise admins can basically say for certain task or for certain data sources just too sensitive to be used. and so we see kind of enterprise so we launched was two things right one is the ability to be able to deterministically create a sandbox around your agent.

因此，要能够明确说明它可以使用哪些应用、不能使用哪些应用，可以访问哪些网站、不能访问哪些网站。第二点是，这涉及某种人在环路中的机制：哪些类型的操作需要人工介入？比如我们可以规定，任何时候发生下载或上传，都必须由人批准；任何时候它访问新网站，都必须由人批准。So being able to say it can use certain apps it can't use other apps it can use certain websites and not other websites. And then two is you know this kind of part of a human in the loop like what type of actions does it require a human to be interveneed? And so we might say, okay, for anytime there's like a download and upload, a person has to kind of approve it. Anytime it goes to a new website, a person has to approve it.

再说一次，这些都像是企业可以逐项调整的设置。所以这算第一点。第二点就是向企业管理员展示我们内部做过的所有评估，让他们了解这些模型的对齐程度，包括与内部评估以及第三方评估相比的情况。我认为，我们还会在各方面做一些基础工作，帮助人们建立对这类产品的信任。Now, again, these are all like dials an enterprise can kind of go through. And so it's really like that's one. And then two is really just showing the enterprise admins just how you know all the evals that we've run internally around how aligned these models are even compared to like you know both internal and kind of third party evals. I think there's also some baseline stuff that we do across the board to just try to help people build trust with these kinds of products.

你知道，这对大家来说还是非常新鲜的体验，既神奇，有时又有些陌生。所以我们在产品设计上投入了很多精力。比如，如果你想看它执行任务，可以随时看到光标，而且光标会以有趣的方式动起来，这样看着就不会无聊。然后，就像 Jame 刚才说的，这款产品对企业用户和普通用户来说都非常可定制。You know, we know it's so new. It's such a magical and sometimes alien experience for people. And so, you know, we put a lot of work into the design of the product. So, you can always see the cursor if you want to watch it perform a task and it sort of animates in a fun way so it's not boring to watch. and then like Jame was saying, this is very, customizable for enterprises, but also for a regular user.

你知道，根据你所处的模式不同，我们会在使用新的应用或网站之前征求你的许可。我觉得这能让人不至于觉得它会在自己的电脑上失控、乱来。你会希望有一个逐步建立信任的阶段，先了解它是什么感觉、希望它为你做些什么，之后随着时间推移，你就会觉得更放心。You know, we ask you for permission before using a new application or website. depending on which sort of modes you're in. And, I think that helps people sort of not feel like it's going to go do some runaway stuff on their computer. You kind of want to have this trust building mode moment where you figure out what it feels like and what you want it to do for you and then you sort of feel more comfortable over time.

Codex 光标真可爱。The Codex cursor is so cute.

最佳企业级应用场景：看似平淡，实则蕴含机遇The best enterprise use cases: boring is opportunity

我爱I love

参与者：这好像是个——我知道这只是个小细节，但看着它使用电脑会让你更有信心，也让体验变得更好。所以我觉得从用户体验的角度来看，这一点值得投入，而且绝对值得投入。我还想从更广泛的角度了解一下：好，现在我们有了能力很强的模型……Participant: it's like such a I know it's a small detail, but like watching it use the computer gives you like confidence and it makes the experience nice. So I think there's a kind of like a user experience point. that's worth that's definitely worth investing on. And I guess I'm curious like in a broader sense like okay now we have like capable models

参与者：它们的成本正在变得足够低，可以大规模运行。在你看来，各类公司应该考虑自动化哪些事情？你会怎么说？你觉得企业里最有前景的应用场景是什么？有没有一些工作流程？也许是围绕 ERP 的，或者是关于人类和智能体如何协作完成任务的。但如果让你选，你最期待看到你的模型在哪些领域发挥作用？Participant: they're getting like cheap enough to run at scale. In your view, what should kind of companies think of automating? Like what would you what where do you see the most promising kind of use cases within enterprises? Are there some workflows? Maybe something around the ERP, maybe something around kind of u how like again humans and agents can work together to get tasks done. But like if you had to like pick where are you most excited to see your models perform in the world like

参与者：是啊，我想从我的角度来说，我很想听听我们这边的想法。我觉得可以分成两大类。第一类就是自动化那些繁琐的工作流程。这类比较无聊，我举几个无聊的例子吧，比如……Participant: Yeah, I guess like from my perspective, I'd love to hear ours. It's like two I think of it as in two big categories. The first category is like automating you know tedious workflows. That's like the more boring one and I'll give us some boring examples like

参与者：无聊也是一种机会。Participant: boring as opportunity.

参与者：机会。对，没错。比如 ERP 就是一个很大的例子。你能想到的所有缩写，比如 ERP、CRM，Participant: Opportunity. Yeah, exactly. like ERPs is a really big one. and so all the acronyms you can think of ERPs, CRM,

参与者：所有权威记录系统。Participant: all the systems of record.

参与者：是的。每一个记录系统，对吧？我们知道，我们见过一些人，比如使用采购软件的公司。你知道，他们每年都要向供应商订购一大堆实验室设备。过去他们会用一个有上千行的巨大电子表格，然后由运营团队处理。通常，公司会让各个团队自己填写这个巨大的电子表格。然后运营团队把它导入内部的采购软件，整理成一条条最终的采购项目。接着销售团队——然后是采购团队——再拿着软件生成的那一大堆内容，去联系供应商。电脑可以把这一整套流程自动化，对吧？Participant: Yeah. Every single system of record, right? And so we know we've seen folks you know procurement software like you know you're going to vendors and every year you're ordering a bunch of lab equipment instead of having this giant spreadsheet with like a thousand rows that then the ops team so it's like you know work companies like normally what they have done is you they have this giant spreadsheet that gets filled out by the teams themselves. Then an ops team takes that giant spreadsheet puts it into an internal you know procurement software and then that gets like you know finalized as a bunch of lines and then a sales team then goes procurement team then takes that giant software output and then goes out and reaches out to vendors. computers can automate all of that, right?

所以，采购算是我们常见的一大项。And so like procurement is a big one we see.

金融是我们发现的另一个非常有趣的类别。你知道，一家公司里有很多不同的账目，对吧？比如费用账目以及各种其他账目。因此，到了月底，就需要进行大量对账，而许多对账财务软件都没有 API，无法快速提取数据。这样一来，财务团队就可以利用计算机操作功能查看所有这些账目、进行对账、找出任何值得关注的内容，标记给相关人员，或者进一步调查。这也是一种很枯燥但很实用的工作流程。至于我们的销售团队，很多销售团队会使用各种不同的软件来处理通话转录、获取数据之类的事情。finance is another really interesting category we've seen and so you know there's a lot of different ledgers within a company right and so you have like ledgers for expenses ledgers for all these things and so at the end of the month there's a lot of reconciliation and a lot of these recon financial software don't have APIs to be able to quickly pull things. and so the finance team being able to have computer use, look through all these ledgers, reconcile, pull out anything that's like interesting, to flag either to a person or then do some investigation. Also a very boring but useful workflow. for our sales team, a lot of sales teams uses a lot of different software for like transcripts, for getting data, all these kind of things.

所以，让计算机操作去挖掘潜在客户，然后把信息录入 CRM 或某种外联软件之类的，再处理后续所有事情。And so having computer use go and do prospecting and then going and putting that into a CRM or some kind of like you know outreach software and then all that.

所以，所有系统记录都是比较枯燥的部分。然后，市场调研也是我们经常看到的另一类事情。你知道，我空闲时间一直在为 Bmon 开发一个视频标注软件。我从 Bmon 里导出内容，比如我会拍下自己玩游戏的过程，然后让智能体去查看里面的东西。我当时就在想，怎么才能让这个东西真正有用？市面上有一些针对匹克球、壁球和网球的软件，所以我就让电脑操作功能登录这些软件、创建账号、试用它们、截图并给我反馈，然后再想办法把这些功能构建回应用里。这些都算是可以自动完成的繁琐工作。然后我觉得，更有意思、能让人完成更具雄心的工作的部分是，你知道，我们之前聊过，现在模型在视觉设计方面已经做得非常好了。So all the system records is the boring part. and then like market research is another one that we see a lot. and so you know in my free time I've been building this like video annotation software for Bmon. I pull out of Bmon and like I like to film myself playing the game and then having the agent kind of go and look at stuff and I was like okay how do I make this thing really useful there's some software out there for like pickle ball for squash for tennis and so I just had computer use go log in create an account on these soft on these softwares try it take screenshots give me feedback and then try to build it back into the app and so these are all kind of like tedious workflows that could be done and then I think the some of the more like interesting creative work of like enabling more ambitious work is like, you know, we talked about the model now being really good with like visual design.

所以像电路设计、建筑设计、硬件工程、游戏设计这些领域，我认为也是 Astro 大有可为、让我们非常兴奋的方向。现在企业不太会考虑这些，因为他们关注的是那些比较枯燥的应用场景。不过我觉得，随着大家越来越了解模型的能力，接下来我们会看到更多这类应用。想听听大家有什么想法。抱歉。And so circuit design, you know, being able like architecture work, hardware engineering, game design, these are also areas that I think we're really excited about where Astro can take. less, you know, less common for enterprises to think about those right now because they're thinking about the boring opportunities. But I think as folks understand the capabilities of model more, that's where, I think we'll kind of see more usage next. But curious if any thoughts. Sorry.

参与者：是的，我觉得在消费者这一侧也有类似的情况，有很多事情是在自动化处理繁琐事务，或者只是帮你做那些你不想做的事。比如替你填写机动车管理局的表格，帮你退订收件箱里一大堆邮件。 参与者：然后我觉得还有一类事情更像是，我听很多人讲过，他们做了一些原本没时间做、没空做，或者根本没想到要去做的事。比如说，就我个人而言，我在维护财务电子表格和膳食计划电子表格；我有个朋友是私人教练，他在为自己的客户制作定制内容。你知道，这些事情在日常生活中做起来会非常耗时。但有了这个，做起来就容易多了。Participant: Yeah, I think kind of similarly on the consumer side, there's a lot of things where it's sort of automating drudgery or just doing things you don't want to do. It's filling out DMV forms for you. It's helping you unsubscribe from a bunch of emails in your inbox. Participant: and then I think there's also things that are more like I'm hearing from a lot of people stories of people doing things that they just wouldn't have time they wouldn't have had time or wouldn't have thought to do otherwise. like I don't know for me personally I'm maintaining spreadsheets about my finance and spreadsheets about meal planning and I have a friend who's a personal trainer who's making customized things for his clients and you know some of these things would just be so time prohibitive to do in your in your personal life. but this makes it so easy.

我用的是 Apple Numbers，这是一款比较冷门的电子表格软件，但这个模型还是能轻松搞定。I use Apple Numbers which is like a obscure spreadsheet program but the model still knocks it out.

参与者：我想我从没听过有人用 Apple Numbers。Participant: I don't think I've ever heard someone use Apple Numbers.

参与者：这是一款出乎意料地优秀的电子表格软件。然后我觉得还有第三类事情，是我最兴奋的——它才刚刚起步，我认为你知道，它就在眼前。说到底，就是以一种不同的方式使用整台电脑，或与软件交互。我想你在我们的 Astra 广告中看到了很酷的例子：人们只需用语音与电脑对话。Participant: It's a surprisingly great spreadsheet software. and then I think there's a third category of thing which I'm kind of the most excited about which is just starting and I think you know on the horizon you know which is really just having a different way to use your entire computer or interface with software. I think you saw in our Astra ad really cool examples of people just talk to their computer with their voice.

智能体会运行旧版软件，还是取而代之？Will agents run legacy software or replace it?

这是一种非常自然的电脑使用方式。我觉得，随着时间推移，这项技术会改变软件本身的形态，对此我真的很期待。 参与者：说到这一点，好。软件形态的变化。想象一下，如果你是企业用户，或者你已经习惯使用那些……你知道的，30年前那种笨重的 ERP 系统。现在有一种可能是，你可以部署一组智能体，让它们直接操作那些软件，而你其实并不在意软件本身。还有另一种可能是，有人作为新兴的 AI 原生厂商，使用电脑操作智能体。我很好奇，你们觉得未来会更倾向于哪一种：保留那些老牌厂商，只是在上面加一层；还是说，使用电脑的智能体完全可以成为新软件厂商的一部分？这是我经常思考的问题，所以很想听听你们的看法。It's such a natural way to use your computer and I think over time we're going to see the form factor of software itself change due to this technology and I'm really excited about that Participant: on that point. Okay, form factor of software changing. if imagine if you're an enterprise user or you've used you're used to using whatever clunky ERP from you know 30 years ago. there's a world in which you now, you know, unleash a set of agents that just operate that software and you don't really care. there's another world in which folks are a new like AI native player is using computer using agent and I'm curious if do you think the world is going to go more in the direction of like keep the legacy folks and just have this layer on top or do you think you know computer using agents can absolutely be part of new software players and this is something I think a lot about so I'm curious to get your guys' thoughts.

如今正是一个非常有意思的时刻：与此同时，开发新软件的成本和复杂度都大幅降低了。It is such an interesting moment where simultaneously the cost and complexity of creating new software is so much lower.

所以，你很难想象软件会发生如此巨大的变化，但与此同时，世界上也存在着大量的遗留软件。通过计算机操作非常适合与这些软件交互。因此，我预计这两种情况都会持续很长时间。你看，如今投入生产使用的软件中，有很多是 10 年、20 年、30 年甚至 40 年前编写的，我不认为这些软件会一夜之间消失。So you can't imagine software changing so much but also there exists so much legacy software in the world. The computer use is so well suited to interact with. So I expect both of those things to continue for a long time. You know if you look at the software that's in production today a lot of it was written 10 20 30 40 years ago and I don't think that stuff's going to go away overnight.

我确实认为，在某种程度上，计算机使用能力可能会给公司带来更大压力，促使他们让软件更好地与智能体协同工作，因为所有软件默认都会与智能体协同工作。我认为 WebC 就是一个很好的例子，而且我们会看到更多类似的东西。I do think that in some way the existence of computer use puts may put more pressure on companies to make software work better with agents because all the software is going to work with agents by default and I think webc is a great example of this and I think we'll see more stuff like that.

参与者：所以，是的，我觉得这两者都非常重要。不过我确实认为，随着时间推移，它们的重要性会有所变化。你怎么看，詹姆斯？Participant: so yeah, I sort of expect both to be super important. but I do think it'll like kind of shift over time. what do you think, James?

参与者：是的，我同意你的看法。我觉得这就像……你看，有很多艺术和工艺在里面，你应该能理解这一点，因为 Ari 和他的团队打造出了很棒的产品。Cursor 全身心投入了。所以我觉得，打造那种人们可以拿在手里的优秀产品，确实需要很多艺术和巧思。我不知道，对我来说，如果互联网变成那种毫无个性的东西，只剩下智能体彼此交谈，人们再也不使用任何东西了，那我会非常难过。我不觉得事情会变成那样，因为手工打造的软件、带有匠心的艺术作品，总有一种特别之处，用起来就是让人觉得舒服，因为设计它的人真正理解客户想要获得怎样的体验。Participant: Yeah, I agree with you. I think it's like the there the way there's like a lot of arts and craft and you understand this because Ari built beautiful product with his team. the cursor was all in. and so the I think there's like a lot of art and craft into building like great products that people can hold in their hand. And I don't know like for me if we go if the internet becomes just this like faceless, you know, thing that is just agents talking to each other and people are not using anything anymore. I would be very sad. and I don't think it will be like that because like there's like something about craftsmanship and art artisan software that's like just feels nice to use because the person who designed it really understands like the experience that the customer wants to have.

就是，我妻子最近特别喜欢玩一种桌游。你知道的，我忘了它叫什么。就是把五颗小石子连起来就能赢的那种。我想它好像叫 Omach。So like my wife loves is recently got into playing this like you know board game. I forgot the name of it. It's where you connect like five pebbles to like win. I think it's like Omach is the name of it.

反正，她想把这个练好，因为她在网上和朋友一起玩时总是被打得很惨，所以我就说：“好吧，我给你做一个战术训练工具，让你能练习各种不同的弃兵开局，这样你就能变得更厉害。”比如说，这就是我做出来的第一个版本。我当时想：“哦，我知道她喜欢用某种特定的方式来学。比起其他方式，她更喜欢这几种。”于是我就能把软件设计成她用起来感觉特别顺手的样子。And so anyway, so she's trying to get good at it because she's playing online with friends and it's getting her ass kicked and so I was like, "All right, I'm going to build you like a tactical trainer for you know, going through different gambits so you can like get better at it." Like that's for example something that like I you know created a first version. I was like, "Oh, I know like she likes to do it in a certain way. She likes these type of ways to learn more than others." And so I was able to like, you know, craft the software to be something that feels really good for her.

还有像这种个性化软件，我知道这可能听起来仍然像是只有真正懂技术的人才会做的事。但我觉得，如果你喜欢……And like these kind of personalized software, I know it's like a thing maybe today still feels like only really tech people will do. But I think if you like

参与者：想想几年之后的情形，如果任何人都能打造任何东西，而且能在很短的时间内，针对一些小众用途打造出非常精心设计、贴合用户需求的软件，我觉得那会是一个美好的世界。再说，你知道，在她训练结束时，她学到的经验和犯过的错误，可以由一个智能体总结出来，并以文本形式提供给她；但当她真正和 AI 一起练习、努力进步时，这件事还是得由她本人来做。所以我确实认为——我更看好 UI 不会消失。我想这有点偏离你最初的问题，不过我确实认为，廉价软件带来的压力会促使软件变得更好。Participant: think about this a few years out, it's like if anybody can build anything and they can build these really craft software in a very short amount of time for like niche use cases that are like so well tailored for people, I think that's a beautiful world. and again, you know, at the end of her training, the lessons and mistakes she made, that could be an agent that summarizes it and gives her in text format, but when she's actually playing with the AI and trying to get better, that's something that the person has to do. So I do think it's I'm more I'm bullish that I don't think UI is going away and I think kind of a little tangent to your original question, but I do think that the pressure of cheap software will make it so that software has to be better.

但我不觉得事情会只是这样，比如说，嗯，所有东西都会有一个 MCP 和一个 API，而你知道，comput 永远不会有用。but I don't think it will just be like yeah everything will have an MCP and an API and you know comput will never be useful.

参与者：我非常喜欢这一点。我觉得人类天生就很擅长直接操控。你真的会想要亲自动手，移动东西、做各种事情。我觉得，回想过去的软件总是千篇一律、面向所有人而设计，这件事很令人兴奋。我们每天使用的许多软件，往往都是为数百万、数千万、数亿甚至数十亿人设计的。我认为，我们将进入一个新时代，任何一款软件都可以专为一百个人，甚至一个人而设计。Participant: I love this point so much. I think that like humans are so hardwired for direct manipulation. you really want to be able to, you know, move and do things with your own hands. and I think that it's really exciting to think about how software in the past was so oneizefits-all and had to be, you know, often a lot of the software we interact with every day is designed for millions, tens of millions, hundreds of millions, billions of people to use. And I think we will enter an era where any piece of software could be designed for a hundred people or one person

参与者：而且这类软件会变得多么个性化、独特。Participant: and just how much more personal and unique that kind of software will be.

参与者：是的。我——我的心智模型是，我觉得大概分成两类。一类是在消费者这边，你得到的东西要么能让你感到愉悦、让你喜欢使用，要么能让你学到东西。这里说的不是它的实用性。Participant: Yeah. I my mental model is I think there's probably two buckets. One is on the consumer side you've got either something you just get you get joy out of you enjoy using or you learn from. There is this it's not the utility of it.

无论是互动、学习还是别的什么，其中的乐趣就在于此，而你想把时间花在 B2B 业务上。你可能希望让你的智能体完成一大堆繁琐的工作，但与此同时，还有审计日志这样的概念，而且你可以按照自己想要的方式对其进行个性化设置。不过，产品的核心其实是由我这个人类用户来设定目标，然后能够理解其中的逻辑，但不一定需要弄清事情发生的整个指挥链条。所以，任何 B2B 用户仍然需要弄明白：好吧，这笔款为什么付出去了？然后回过头来查看，获取一份事情经过的摘要，并且能够说：好，在这个时间向这个人付款；到了 30 天时，应当应用这么多折扣，等等。It's the joy of interacting or learning or whatever it is and you want to spend time on the B2B side. You may want your agent to complete a bunch of the tedious work, but you also there's this concept of like the audit log like and you that could be personalized in how you want to interact, but the product really is me as the human user setting the goal and then being able to like understand the logic but not necessarily need to run through like the entire chain of command of how it happened. And so like any B2B user still needs to figure out okay like why did that payment get made and go back in and get a summary of what happened and be able to say okay pay this person at this time yeah at 30 days this much discount should be applied etc.

他们希望代理来负责执行，但又希望在某种程度上仍然保留一些控制权。我觉得这确实需要某种形式的 UI。而且我觉得，如果我们越来越相信 AI 会承担更多这类工作，那么这部分也得让人用起来愉快，对吧？为什么不呢？我们为什么要满足于难用的软件？They want the agent to take care of like the execution of it but they want to be able to at some level like some level of control still needs to be put in there and I think that's like something that does require a UI of some sort. and it has to be I think if our it if we're moving more towards believing that AI will do more of this that part has to be enjoyable to use too, right? Like why not? Like why should we settle for shitty software?

参与者：你知道，这是】【。Participant: You know, it's

参与者：我确实觉得，看看我们是否会看到新型软件会很有意思，而且我对此很乐观：这类软件既有传统用户界面，也会在底层使用智能体。也许有时这些智能体本身还会通过操作计算机来自动化旧式软件，为你提供更友好、更贴合特定用途的体验。我对这一类软件真的非常期待。Participant: I do think it'll be really interesting to see if we and I'm optimistic that we will see new kinds of software where it has traditional user interfaces but is also using agents under the hood. And then maybe sometimes those agents are themselves using computer use to automate legacy software by giving you some much nicer more purpose-built experience. I'm like really excited for that class.

参与者：是的，这有点像我们内部常说的那个共享智能层，对吧？比如，你的智能体在别处使用旧版软件，也会在其他地方执行各种操作。如果你在使用某个应用，这个应用就希望能访问这些上下文，从而提供更好的体验。我觉得，如果每个网站都有自己的智能体，却不跟你的智能体交流，那整个世界就会非常碎片化。Participant: Yeah, it's kind of like the one where we like to use internally like the shared intelligence layer, right? Like you have your agent using legacy software somewhere else and doing things other places. You like if you're using an application, the application wants to get access to that context so it's able to deliver a better experience. It's like I think if we had every website has its own agent that doesn't talk to your agent, it's a very fragmented world.

所以，弄清楚这一切如何协同运作，让智能体能够接入并使用你的其他软件，我觉得接下来几个月的发展会非常有意思。And so figuring out like how that plays together where the agent can plug in and use your other software, I think it's going to be like a really interesting thing to play out over the next few months.

为基于计算机使用功能构建产品的创始人提供建议Advice for founders building on computer use

参与者：我很喜欢你们谈这个。对。我觉得人与 AI 的交互重要性太容易被低估了，尤其是你希望产品取得成功的时候。所以要认真考虑人们如何协作。你希望任何智能体都能融入团队。我很好奇，如果把这个思路应用到一家新创公司上会怎样？比如说，你明天要创办一家公司，并且想用计算机操作来解决企业内部的一项工作流程问题。Participant: I love that you guys Yeah. talk about this because I think it's so underappreciated the importance of like human AI interaction, especially if you want like a product to be successful. So like being thoughtful of like how can I kind of like humans are used to collaborate. So you want any agent to be able to like kind of fit into the team. I'm just curious like if you apply that to a new startup let's say you're like I don't know starting a company tomorrow and you want to use computer use to like solve a workflow within the enterprise.

你会怎么考虑这件事？还有，你觉得哪些事情会最先浮现在脑海里？比如说：“哦，我想知道它什么时候会出错”；或者“我希望它能和人交流”；又或者“我想让它像个……我不知道，黑箱一样”。如果你现在要和创业者交流，在他们尝试利用你的模型和能力为企业解决问题时，你会给他们什么建议？How would you think about it? And would you be kind of what would be the things that jump to mind in terms of like oh I want to know when it fails well. I want it to be able to communicate with people or I want it to be like a I don't know a black box just like maybe if you had to talk to founders right now what advice would you give them as they try to use your models and capabilities to just solve problems for enterprises.

在我看来，这取决于……这很大程度上取决于模态，你知道的。In my mind, it depends on it depends so much on the modality of you know

参与者：你们是在尝试……你们是一个小团队，希望团队中的某个成员能够使用你们团队正在使用的某款软件，只是用得更高效一些吗？或者你们可能是在构建某种本身就需要与现有软件交互的东西？这种情况下，你们或许会在云端某处有一台机器，在后台执行计算机操作。我认为这会很大程度上影响你们希望人类与智能体如何协作，对吧？而且，嗯，在前一种情况下，我认为我们的很多产品都在服务这种使用场景：你坐下来打开 chatbt，就能进行很多很棒的协作。而在另一种情况下，嗯，你们可能是在用 API 构建某种东西。Participant: are you trying to are you a small team and you want sort of individual on the team to be able to use some piece of software that you're that your team is using but in a more efficient way. or are you maybe building something that itself wants to interact with existing software in which case maybe you have some you know machine in the cloud somewhere that's doing computer use in the background I think that will really affect the modality of how you want the human agent collaboration to be right and yeah you know in the former case I think a lot of our products are serving this use case you know where you sit down at chatbt and you know there's a lot of great you collaboration you can have there. and in the other case, you know, maybe you're building something with the API.

你们在实现自己的计算机操作功能。然后，嗯，真正有意思的是，什么时候需要让人介入？智能体会告诉你出了什么问题吗？会告诉你哪些事情进展顺利吗？你知道，会不会有东西帮你总结一下，这样你就不会被“嘿，发生了这个。发生了那个。又发生了这个。”之类的通知轰炸？不然你肯定会直接忽略。我也说不准。詹姆斯，你怎么看？you're doing your own computer use implementation. And then yeah, then it really is interesting to figure out when do you want the human to be in the loop? Is the agent telling you when something's wrong? Is it telling you when something went right? you know, is something summarizing that so that you don't get bombarded with notifications saying, "Hey, this happened. This happened. This happened." Which actually you're going to ignore. I don't know. What do you think, James?

参与者：是的，我是说，你知道，我和 Ari 在加入 OpenAI 之前都创办过初创公司，所以你知道，我现在仍然非常看好初创公司。你知道，我想问题归根结底是：现在还有什么机会，对吧？当实验室在做这些事情时，初创公司还能解决些什么？总的来说，我觉得有两点。第一，我觉得还有太多东西有待开发，对吧？尽管我们现在规模已经相当大，员工有几千人，但我们内部仍然感到人手非常紧张。你知道，面对所有这些问题，各个团队的规模都不算大。所以，如果你深入了解某个问题，拥有行业专长，了解监管如何运作、如何推进市场，以及如何服务这些客户，我认为这仍然是一片充满机会的蓝海。而且，正如 Ari 提到的，通过我们的 API，你可以使用各种工具，比如 computer use。Participant: Yeah, I mean I you know, both Ari and I were startup founders before we joined OpenAI and so you know I'm still very bullish on startups. You know I think the question right kind of comes down to like what is there still a mo right? Is there something that startups can still tackle while labs are doing things? I mean in general I just think there's like two things like one is like I think there's so many things to build right and like even though we're quite big now thousands of people we are still feeling very stretched thin internally and like you know it's like the teams are not very big for all the problems and so if you have a problem that you really deeply understand you have industry expertise you understand how regulatory works the go to market motion how to service the those customers it is still I think a very green field out there and the tools that you have through our APIs like Ari mentioned with computer use.

我觉得我们还是会让你加快速度、提升效率。说到底，我认为速度和付费大概就是当下存在的这种模式。我经常想到的另一件事是，在我看来，仍然有很多垂直行业，AI 的普及速度要慢得多。前几天我和一家公司聊过，他们用 computer use 来帮助餐饮集团拍摄后仓的照片，然后通过视觉识别，根据对明天客流量的预测，判断哪些东西会短缺。接着，它会通过 computer use 进入 Shopee 或某种采购软件，订购第二天需要的食材。之后，它还会进入 ERP，调整员工的排班——比如说，我们可能需要安排更多人手来帮忙，因为去年这个时候，一天中的这个时段客流量要大得多，可能是因为有什么节日之类的。I think we'll still let you accelerate and speed really I think at the end of the day speed and pay is kind of the mode that exists today. I think the other thing I think about a lot is like there are in my opinion still a lot of like vertical industries where the diffusion of AI is like much slower. And so you know I was talking to a company the other day who uses computer use to help restaurant groups to you know take pictures of like their back storage you know and then using that the vision looks at kind of like what is going to be missing based on a projection of tomorrow's traffic and then it takes that to then use computer use to go to like a shopppee or some kind of like procurement software to then order your groceries for the next day. and then based on that it will go into your ERP to adjust like the timetable for your like employees who are who we might need more people to help because in the last year at this time of the day there was a lot more traffic coming in from like a holiday or something like that.

所以你知道，那些餐饮集团可能不像科技公司那样，对各种可用工具都特别熟悉。And so like you know those restaurant groups are probably not like tech companies that are super you know familiar with all the tools are available.

计算机使用的下一步是什么What’s next for computer use

所以我觉得，你知道，住在旧金山，我们经常会看到有人想：“哇，实验室能做任何事，Codex 也是。那为什么大家不干脆都装上 Codex，让它帮忙做事呢？”但对普通美国人、普通小型、中型企业主，甚至普通企业来说，光是理解这些工具能做什么就是第一步；以合理的方式把工具连接起来是第二步；部署是第三步；维护是第四步。所以在我看来，如果有一些复杂工作流需要用到许多彼此独立的软件，使用计算机的智能体就很适合把它们连接起来，完成整个预期流程；然后再以一种让客户用起来感觉顺手的形式和界面构建出来。这仍然是一个非常大的机会，我觉得现在依然有待开拓。跟我们聊聊接下来有什么吧：在计算机使用智能体这个前沿领域，你有什么可以分享、又让你感到兴奋的新进展？And so I think like you know living in San Francisco we see a lot of thinking wow like you know labs can do everything the codecs and like you know why doesn't everybody just install codecs and just have it do stuff but like for the average American and the average small mid medium business owner the average enterprise even understanding what the tools are capable is one connecting tools in a way that makes sense is two deploying it is three servicing is four and so there's still a lot in my opinion where if there are complex workflows that require a lot of disparate software computer use is a great way to join them complete the entire intend flow and then building it in a format and UI that is feels good to hold for those customers feels still like a really big opportunity that you know we I think is still open right tell us about what's next what's what are you excited about that's on the frontier that you can share around computer using agents

参与者：嗯，我已经大致提到过这一点了，但我们真的很期待速度的提升，也很期待让计算机使用达到超人水平，还很期待各种不同类型的界面融为一体、整合多模态输入和输出的产品。这样你就能和计算机进行更加流畅的交互。我觉得现在我们的很多计算机使用产品都非常注重后台运行。它很像是“我给智能体布置一项任务，让它去做，这样我就能同时做别的事”——这实际上是产品的一大优势，对吧？Participant: well I' I've sort of gestured towards it already but we're really excited about speed we're really excited about making computer use superhuman and we're really excited about things that blend together different kinds of interfaces that bring together sort of multimodal inputs and outputs. so you can have more kind of fluid experiences with your computer. I think right now a lot of our computer use product is very background focused. It's very much like let me give the agent a task. So it can do that while I do something else, which is actually a tremendous strength, right, of the product.

但我认为，未来我们还会看到更多这种你和智能体之间的实时协作。所以我对这个领域真的很兴奋。而且除此之外，整体也会变得更好、更便宜、更容易获取、更易于使用，But I think that in the future we'll also see more of this like real time collaboration between you and the agent. so I'm really excited about this area. and other otherwise just sort of better, cheaper, more available, easier to use,

参与者：更多，是的，没错。能够真正访问所有内容。是的。我觉得有，Participant: more Yeah, exactly. Being able to actually access everything. Yeah. I think there is,

参与者：你知道，我们用 DOTS，对吧，我们给大家提供这种云端电脑。Tragic work 也有云端浏览器。我觉得说实话，对我来说，我很期待的一件事就是让整个体验更连贯一些。你知道，现在你可以安装 Chrome 扩展，它会使用你在 Chrome 里的密码管理器以及其他所有东西；然后桌面应用里还有一个应用内浏览器，它使用的是另一套 Cookie 和密码。Participant: you know, we with DOTS, right, we're giving folks these like computer in the cloud. Tragic work also has a browser in the cloud. I think honestly like for me like one thing I'm excited about is just making the experience feel a little bit more coherent. Like you know today you can install the Chrome extension and it that uses your like password manager and everything in Chrome and then we have an inapp browser in the desktop app which has a different set of cookies and different passwords.

云端电脑也有各种各样不同的东西，对普通消费者来说很让人困惑。就像，为什么我有四个彼此都不互通的不同浏览器？我在一个浏览器里打开一个标签页，它会不会也显示在另一个浏览器里？所以，我想我特别期待的一件事，就是我们怎样才能让电脑的使用从一种只有企业和那些真正走在最前沿的硬核用户才会用的产品，变成普通人也能真正理解的东西。Your computer in the cloud also has different things and for the average consumer it's confusing. It's like why do I have these four different browsers that all don't talk to each other? If I open up a tab in one does it like show up in the other? And so like I think a lot of what I'm excited about is how do we make computer use go from this product that is used by like enterprises and like really hardcore tip of the spear kind of users to something that's actually understandable for people.

这有点像一种矛盾说法，因为我觉得电脑操作其实是最容易理解的 AI 产品，对吧？It's kind of like an oxymoron because I think computer use is actually the most easy to understand AI product because right

参与者：这是一个使用你电脑的 AI，但由于你需要管理各种复杂的状态，而且会在不同地方使用它，实施细节有时会渗透出来，结果就变得有点难以理解。所以，能够把它打磨成面向消费者的产品，让它更容易理解、更加连贯，是我对下个季度特别期待的事情。Participant: it's an AI using your computer but then because of the like complexities of like the different like states you have to manage and the different places you're using it the implementation details sometimes seep through and then it becomes kind of hard to gro and so being able to kind of make that consumer grade and easier to understand more coherent is something I'm really excited about next quarter

参与者：是的，我觉得这又回到了那个问题：毕竟还是有人在触发智能体，对吧？而且界面和控制功能越好，用起来越简单、越愉快，他们就越能从中受益。然后就是……Participant: yeah and I think it comes back to the point that there is still a human that's setting off the agent right and they have to the better the UI and controls and the easier and more enjoyable it is to use, the more they're going to get out of it. And that's

参与者：当然。Participant: absolutely

参与者：原则。关于这一点，我还想提一件事：我认为，当下这款产品的另一个挑战是，你得花心思琢磨怎么用它，还得记得去用它，并且得说服自己去做一些不同于以往习惯的事情。我确实认为，正如詹姆斯刚才提到的，Dots 有一种非常强大的主动性，它们可以主动找到你、识别情境并帮助你解决问题。我觉得我们还会看到更多主动性。我非常期待这样一个世界：智能体能够适应你，而不是你去适应它。我认为这种情况会越来越多，也会让人们更容易与这项技术建立联系、理解它并将其融入生活，同时仍由人们来设定和引导目标——如果我说清楚了的话。Participant: principle. On that point, one other thing I want to gesture towards is I think another challenge of the product today is you have to put some thought into how you're going to use it and you have to remember to use it and you have to convince yourself to do things that are different from how you're used to doing them. and I do think, dots, like James was talking about earlier, have this really powerful proactivity where they can sort of come to you and identify situations and help you solve problems. I think we'll also see a lot more proactivity. I'm really excited about a world where the agent can sort of adapt to you instead of you adapting to it. and I think that we're going to see a lot of that and I think it will make it a lot easier for people to actually like relate to and u and understand and incorporate this technology while also still being the ones sort of setting and searing the goals if that makes sense.

参与者：我们还没谈到的一件事，是计算机操作领域的发展势头，以及这方面发生了多大的变化。我和 Fitzi 在我们的文章里稍微聊过一点。想请你大致谈谈它目前的发展轨迹，以及接下来会走向哪里。Participant: One of the things we haven't talked about yet is the momentum behind computer use and how much has changed and Fitzi and I chatted a little bit about this in our piece. would love for you to just kind of talk about where it's, you know, how it's the trajectories so far and where things are headed.

参与者：是的，我是说，我和詹姆斯已经做了一段时间的计算机使用研究了。Participant: Yeah, I mean James and I have been working on computer use for some time now

参与者：在 OpenAI，过去大概六到九个月真的特别令人兴奋，对吧？因为我们开始开发这些计算机操控类产品，而且……Participant: at OpenAI and it's been an incredibly exciting past like six to nine months, right? Because we started working on these computer use products and

参与者：当时使用它们的人非常少。今年早些时候，我感觉模型突然就变得很擅长这类任务了，这要归功于研究团队的出色工作。之后，我们和内部许多团队都非常努力地合作，思考如何围绕这项技术打造真正出色的产品体验。而就在过去几个月里，产品突然开始飞速增长，大家也确实看到了它的价值。但我觉得现在还处于非常早期的阶段；目前仍然只有我们用户群体中的一部分人在使用 computer use，我认为接下来使用人数还会大幅增长。Participant: very few people were using them. the models all of a sudden it felt to me became great at these kinds of tasks earlier this year because of amazing work the research team did and then we've been working super hard with a bunch of teams internally on how to create actually good product experiences around this and then just in the past couple of months all of a sudden you know the product's growing like crazy and people are really seeing the value in it but I think it's still so early you know it's still the case that only some you know portion of our user face is using computer use and I think we're going to see it grow quite a bit.

参与者：是的，完全同意。我是说，你知道，我一月份加入 OpenAI，当时我加入后就在做 Atlas 产品。我很喜欢 Atlas，Atlas 安息吧。不过我想，你知道，我们意识到的一件事是，在 Atlas 产品中，智能体体验出于几个原因，并没有真正成为一个重要卖点。用的人不多，部分原因是我觉得大多数人把它当成普通浏览器来用。而过去几个月里，随着模型能力变得更强，我们现在已经把浏览器和浏览器使用功能整合进了几乎所有原本就有智能体使用场景的现有产品中。我们看到的增长简直达到了 YC 级别，周复一周地增长，而且我们总是对它发展得这么快感到惊讶。Participant: Yeah, totally. I mean, you know, I joined OpenAI in January and I was I joined working on the Atlas product and love Atlas, RIP Atlas, but the I think you know, one of the things that we realized was in the Atlas product, the agentic experience, I think for a few reasons, wasn't actually like a big selling point. Not many people used it, partly because I think most people treated it like a normal browser. and over kind of the last few months because the model capabilities have gotten better. We've now incorporated browser and browser use into kind of all of our existing products where there was already a gentic usage, we've just seen, you know, it's like YC level of growth week over week and then always surprised by how fast it can go.

而我觉得，接下来这个重大的转折点就是云计算，对吧？And I think this next like big inflection point is like cloud, right?

就像，桌面端对我们来说仍然很大程度上是高级用户才会用的东西。我知道我们可能一直都在用桌面端，但大多数消费者都在用手机，对吧？那么，怎样才能让每一位聊天用户，让每一个日常生活中只用手机的人，都拥有一台功能强大的电脑，而且他们甚至不需要知道怎么使用它，它就能自然而然地帮他们把事情办好呢？所以我真的很期待，这能如何把它带给十亿用户。Like how do we like desktop is still very much a power user thing. I know we probably use our desktop all the time, but like most consumers are on their phones, right? And so how do you give every single chat user, how do you give everybody who's just using their phone for their day-to-day life a powerful computer that they don't even need to understand how to use that it just kind of makes things work for them. And so really excited about how that kind of brings it to, you know, billion users.

参与者：感觉计算机操作代理的发展势头一直很猛，而现在正是该思考“好吧，我们怎样才能让更多人用上它”的时候了？让他们信任它，觉得自己真的能用它。你之前提到的连贯体验也很重要，我觉得一旦这方面有了很大改善，消费者就会更放心地使用它，整个过程也会显得更自然。Participant: It feels like computer using agents have had a ton of momentum behind them and now it's the moment of like, okay, how do we get it into more people's hands? they trust it, they feel like they can really use it. And your experience your point earlier about coherent experiences like that's a real I think once as that gets a lot better, I think consumers will feel more comfortable using it and it'll feel more natural.

参与者：我觉得确实如此。我认为，无论是在消费领域还是企业领域，计算机的使用都正处于规模化发展的关键时刻。是的。同时，我也非常期待看到它会发展到什么程度，以及人们会用它创造出什么。Participant: I think that's right. I think we're at the scaling moment of computer use both for consumer and enterprise. Yeah. At the same time. and I'm Yeah. really excited to see where it goes and what people build with it.

参与者：不错。我们也是。Participant: Nice. Us too.

参与者：好的。Participant: All right.

参与者：非常感谢你们加入我们。这太棒了。我们很喜欢聊这个话题，能直接听你们分享也特别有意思。Participant: Well, thank you so much for joining us. This was awesome. we love talking about this topic and it was really fun to hear it from you guys directly.

参与者：谢谢。也很有趣。Participant: Thank you. A lot of fun as well.

参与者：有意思。嗯。谢谢。Participant: Fun. Yeah. Thanks.

参与者：谢谢。Participant: Thank you.

参与者：谢谢你们。Participant: Thank you guys.