用 dot 收集反馈、调查故障并准备修复 PR
原题Hunt bugs with your dot
推荐理由OpenAI 工程师演示让 dot 从产品反馈进入故障调查与 PR 准备,给团队一条可参考的任务委托路径,也展示智能体何时需要承认缺少数据。
看前速览
OpenAI 工程师 Rennie Song 演示用 dot 整理产品反馈,通过语音讨论优先级,并在云端调查故障、准备修复 PR。
共享页面把反馈接入调查流程
分类表汇总外部与内部反馈,用户可在具体问题下留言,让 dot 启动云端调查。
语音讨论可以确定优先级,同时保留统计边界
Benny 认为静默停滞最伤害信任,并明确承认缺少过去两天的准确问题数量。
PR 和调查记录让智能体的工作可供复核
演示将模型权限导致的问候失败与回退方案联系起来,并提供调查时间线,没有声称修复已发布。
完整摘要
OpenAI 工程师 Rennie Song 展示了她的 dot Benny 如何收集和分拣来自 Twitter、Reddit、Slack 及产品运营团队的 Dots 反馈。ChatGPT Space 页面用分类、颜色和表格组织信息;她既能让 Benny 更新外部反馈,也能在具体问题下留言,要求启动云端调查。语音通话中,Benny 将问题归为静默停滞、连接故障和上下文连续性三类。它承认缺少过去两天的准确统计,同时认为静默停滞更伤害信任,应优先处理。随后,它提出修复新 dot 选用用户无权访问模型时的问候失败,让系统回退到可用模型;Song 要求创建云端任务并准备 PR。最后的记录显示,追踪表新增 15 项,总计 210 项,静默停滞调查找到一种具体故障并附有时间线。演示体现了从反馈到调查的流程,但没有证明修复已经合入,也没有给出可靠性改善指标或可复现的权限配置。
章节
文字稿
简要描述Brief Description
Rennie Song 展示了她如何利用自己的点伙伴 Benny 收集反馈、调查故障,并为 Dots 准备修复方案。Rennie Song demonstrates how she uses her dot, Benny, to collect feedback, investigate failures, and prepare fixes for Dots.
目录Table of Contents
- 收集并整理反馈Collecting and organizing feedback
- 与 Benny 一起审查问题Reviewing issues with Benny
- 调查和跟踪器结果Investigations and tracker results
收集和整理反馈Collecting and organizing feedback
大家好,我叫 Rennie Song,是一名负责 Dots 的工程师。今天我想聊聊我是如何用我的 dot 来收集、分类处理并整理我们收到的所有关于 Dots 的反馈,无论是来自 Twitter、Reddit 等外部平台,还是来自 Slack 和产品运营团队等内部渠道。我已经有了自己的 dot。Hey everyone, my name is Rennie Song and I'm an engineer working on Dots. And today I wanna talk about how I use my dot to collect and triage and process all of the feedback that we have on Dots, whether it be external from like Twitter and Reddit or internal from Slack and our product ops teams. So I already have my dot.
它为我所有与 dot 相关的内容创建了一个专属空间,其中还有一个专门收纳我所有反馈的页面。使用页面搭配你的 dot,有一点特别棒:你可以在这里使用分屏,还能完全按照自己的喜好进行个性化设置,决定如何查看 dot 为你完成的所有工作。It's created a space just for all of my dot stuff. And within that a page just for all of my feedback. What's really nice about using pages with your dot is that you have this split screen here and you can kind of totally personalize how you want to visualize all of the work that your dot is doing for you.
所以我希望所有内容都分类整理并用颜色区分,还希望全部放在表格里。那么,你能把 Reddit 和 Twitter 上的最新反馈更新到外部反馈跟踪表中吗?And so I wanted everything categorized, color-coded. I wanted everything in a table. So can you update the external feedback tracker with the latest feedback from Reddit and Twitter?
我来选一个问题。也许是电脑连接故障和登录时出现的 Bad Gateway 错误。这样我就可以在这里留言并@Benny,我的 Dot,问它:“你能在云端线程里帮我调查一下,并找出根本原因吗?”好,它处理这件事的时候,我想给你看看 Benny 正在自己的电脑上做什么。好,它现在好像在 Twitter 上,正在搜索今天发布、包含 Dots 和 OpenAI 的推文。现在我要呼叫我的 Dot。Let me choose one of these issues. Maybe this broken computer connections and Bad Gateway login errors. So I can just comment here and @Benny, my dot, can you look into this in a cloud thread for me and help me identify root cause? Okay, so while it's working on that, I wanna show you what Benny's doing on its own computer. Okay, it looks like it's on Twitter right now. It's searching for tweets from today that contain Dots and OpenAI. So I'm going to call my dot now.
与 Benny 一起审查问题Reviewing issues with Benny
Benny:嘿,Rennie。又来了。Benny: Hey, Rennie. Back again.
伦尼:嘿,本尼。Rennie: Hey, Benny.
Benny:嗨。这次我们要处理什么?Benny: Hi. What are we tackling this time?
Rennie:这次我想和你一起梳理一下你一直在看的那些外部反馈。我想请你总结一下其中的主要主题,以及你看到问题最多的是哪些类别,尤其是过去两天的情况。Rennie: This time I want to talk through some of the external feedback that you've been looking through. I just want you to kind of call out what the major themes are and like, I guess like what categories you're seeing the most issues in, especially in the past two days.
Benny:是的。好。我来跟你一起梳理一下。让我整理一下。好。大致可以分为几类:无声卡住、连接问题,以及连续性问题。过去几天里,出现过的情况包括:任务点显示正在处理中,却始终无法完成;笔记本电脑连接断开;以及语音聊天和 Codex 之间丢失上下文。连接和集成问题是最大的一类,但我没有过去两天的精确统计,所以不会假装知道确切的排名。即便如此,我还是会优先处理无声卡住的问题。它要是突然没了动静,人们就不会信任它。Benny: Yeah. Okay. I'm going to walk through that with you. Let me just pull it together. Okay. So here are the big buckets, silent stalls, connection problems, and continuity stuff. And in the last couple of days, it's been things like dots saying they're working, but never finishing and laptop connections dropping and context getting lost between voice chat and Codex. Connections and integrations are the largest category, but I don't have a clean tally for just the last two days. So I'm not going to pretend I know the exact rank. Even so, silent stalls are what I jump on first. People don't trust it if it just goes quiet.
Rennie:好的。那么无声卡顿听起来是最紧急的问题。你能帮我开个云端线程,试着找出根本原因,看看能不能找到修复办法吗?Rennie: Okay. Silent stalls sounds like the most urgent issue then. Can you kick off a cloud thread for me and just try to root cause that and see if you can find a fix?
Benny:好。我看看能做些什么。等一下。Benny: Yeah. Let me see what I can do. One second.
Rennie:好的。你不必为所有问题都找到解决办法,但如果能找出导致静默卡顿的几个具体原因,那也很棒。另外,在排查的过程中,你觉得现在最容易修复的问题是什么?Rennie: Okay. You don't have to find a fix for everything, but if you find a few individual causes for the silent stalls, that would be great too. And while you're doing that, what would you say is kind of the simplest issue that you can fix right now?
Benny:嗯。让我想想。目前最简单的一个是问候模型失败的问题。新建一个 dot 时,如果它请求的模型是用户无权使用的,就会直接失败,导致无法创建。其实可以改为自动回退到用户可以使用的模型。Benny: Hmm. Let me think. Easiest one right now is the greeting model failure. When a new dot asks for a model the user can't access, it just fails and blocks creation. Instead, it could just fall back to a model they can use.
Rennie:好的。听起来非常简单明了。继续吧,也为那个创建一个云线程,准备好后就发起一个 PR。Rennie: Okay. That sounds super straightforward. Go ahead and also create a cloud thread for that one and just put out a PR when you're ready.
Benny:好的,启动它。Benny: Okay. Starting that up.
Rennie:好的。太好了。谢谢。你完成反馈后告诉我一声。Rennie: Okay. Great. Thank you. And, you know, let me know when you're done with the feedback.
Benny:当然,我马上处理。Benny: Sure. I'm on it.
调查与跟踪器结果Investigations and tracker results
这件事特别棒的一点是,Benny 相当于带着我走过了整个调查过程。这里有它留下的所有这些评论,然后看起来它最终的代码调查确认了一个后端问候语故障,这实际上就是我们刚才在电话里讨论的那个故障。它还发现我们已经在着手提交一个 PR 来修复这个问题,这太好了。What's really nice about this is Benny's kind of walking me through its entire investigation process. So it has all these comments here, and then here it looks like its final code investigation found a confirmed backend greeting failure, which is actually the same failure that we were just talking about on the phone. And it realized that we're already working on a PR to fix it, which is great.
看起来 Benny 已经完成了反馈更新。新增了 15 个条目,使追踪表中的条目总数达到 210 个。针对无声停滞问题的调查发现了一种具体的故障模式,并附上了调查时间线,供我审阅。今天早上我能把这么多工作交给我的 Dot 来处理,真是太棒了——只要在页面里标记它,或者打电话跟它说就行。And it looks like Benny has already finished updating the feedback. It's added 15 new items, bringing the tracker to 210. The silent stall investigation found one concrete failure mode, and it's attached the investigation timeline here for me to review. It's really amazing how much work I was able to offload to my dot this morning, just either tagging it in a page or by talking to it on the phone.
我们在这里看到的进展速度非常快,仅仅是能够使用 Dots 就带来了很大帮助。希望今后你们能看到更多进步和改善。And we're seeing so much acceleration here, just being able to have access to Dots, and hopefully you'll see so much progress and improvements moving forward.