软件开发走向智能体化后，如何判断质量？Emily Bache 与 Sam Newman 对谈

简要说明Brief Description

Emily Bache 和 Sam Newman 探讨了软件的内部与外部质量、领域语言、智能体开发循环，以及经验丰富的工程师和测试人员为何仍不可或缺。Emily Bache and Sam Newman discuss internal and external software quality, domain language, agent development loops, and why experienced engineers and testers remain essential.

目录Table of Contents

• 内部和外部质量
  Internal and external quality
• 领域语言和验收测试
  Domain language and acceptance tests
• TDD 与开发循环
  TDD and development loops
• 明确提出有益的改动
  Making good changes explicit
• 探索性测试与人类专业知识
  Exploratory testing and human expertise
• 扩充与结语反思
  Augmentation and closing reflections

内部和外部质量Internal and external quality

欢迎收听《一个大问题》，我是 Emily Bache，和我一起主持的是 Sam Newman。今天我们要讨论的问题是：当软件开发变得具有智能体特性时，质量是什么样的？Welcome to One Big Question with me, Emily Bache, and with Sam Newman. This question we're discussing today is what does quality look like when software development becomes agentic?

Sam Newman：这算是个有意思的问题。我的……好吧，这是我对这个问题的第一反应。我们可以继续聊，你也能看出来我们意见不一致，或者我们可以进一步探讨。我觉得终端用户对质量的定义完全没有改变，只是我们评估质量的方式变了。我的意思是，除非我错了，或者人们使用软件时，现在对它有了不同的看法，因为软件变得更具智能体能力了。我也不确定。Sam Newman: It's kind of It's an interesting question, like my Okay, this is my knee-jerk reaction to the question, which we can kind of you can see we disagree or we can go further, which is I don't think the end user definition of quality has changed at all. I just think it's how we assess it has got different, like I mean unless I'm wrong or maybe people use software, they've got a different view of it now it's agentic. I don't know.

Emily Bache：也许他们的期望比较低。Emily Bache: Maybe they've got lower expectations.

Sam Newman：是啊。哦，我估计它会很差劲，而且特别贵。我不知道，但感觉这就是那些基本要素。比如说，当有人问“这东西质量好不好”时，他们其实是在说：它在功能上好用吗？速度够快、响应够灵敏吗？Sam Newman: Yeah. Oh, I expect it to be rubbish and really expensive. I don't know, but it kind of feels like those fundamentals, like when I think about you know, the when someone says is this quality or not, it's like they're kind of talking about does it functionally work? Is it kind of quick and responsive enough?

我的数据安全吗？这些是不是都……我不知道，这些情况已经变了。但真正变得非常困难的，似乎是我们这些软件开发人员得想办法搞清楚，究竟怎样才能确保我们的软件足够好。这似乎才是真正棘手的部分。Is my data safe? Is all these are kind of I don't know that has changed. But what seems to have got really difficult is us as software developers working out how the hell to make sure our software is good enough. That seems to be the really tricky part.

Emily Bache：是啊，我的意思是，关于如何区分内部质量和外部质量，也有过这样的讨论。我是说，你在这里谈论的是外部质量，也就是用户感知到的质量；而内部质量则是软件开发人员打开代码并查看代码时感知到的质量。Emily Bache: Yeah, I mean there's also that discussion about distinguishing internal and external quality. Cuz I mean what you've been talking about here is the external quality, what the user perceives. and the internal quality is what the software developers perceive when they open up and look in the code.

Emily Bache：你知道，现在代码大部分都是由这些智能体编写的。人们实际阅读代码的程度各不相同。不过，只要人们会读代码，我想这时候就需要关注代码的内部质量了。Emily Bache: so you know, and now the code is largely being written by these agents. and it varies how much of the code people actually reading. but to the extent that people are reading the code, then that's where the paying attention to internal quality comes in, I guess.

Sam Newman：是的，我觉得这么说挺合理的。我的意思是，我们甚至可以把它拆开。我猜我之前不久和 Kevlin 聊过这个。我们大概还可以把内部质量分成两部分：一部分是运营质量，可能涉及韧性、可用性之类的；另一部分就是开发者体验，对吧？Sam Newman: Yeah, I think that's fair. I mean, we could even break I guess I had this conversation with Kevlin a while ago. We could probably even break that internal quality into two parts, which is kind of operational quality, which would might be about resiliency and you know, availability and stuff. And then that developer experience, right?

真正花时间接触这段代码，或者使用这种架构，是什么感觉？所以我觉得，是的，从这种体验的角度来看，我觉得这完全合理。这里有一场大讨论。What's it like to actually spend time in this code or with this architecture? And so I think like yeah, looking at through that kind of yeah, experiential lens. I think that's absolutely fair. There's this big conversation.

我刚参加完一个会议。昨天我在伦敦参加了 FastFlow Conf。活动很棒，但有很多人都在讨论这个问题——今年我参加的所有活动上，大家也都在聊这个问题：我还需要看代码吗？I was just at a conference. I was at FastFlow Conf yesterday in London. It was a great event, but lots of people having this conversation as has been the case for all the events I've been at this year, which is do I need to look at the code anymore?

这重要吗？我猜，如果你所在的公司会主动查看代码，并且每天都与代码打交道，那么你们对代码是否易于使用，可能会有一套截然不同的要求；或者说，你们对这一点的重视程度可能也会不同于那些处于不同抽象层次的组织。后者可能更在意运营层面和外部的质量特征，而如果不查看代码，代码本身是什么样子就没那么重要了。And does that matter? And I suspect where whether or not you're a company where you actively look at the code and engage with it on a day-to-day basis, you probably have a very different set of requirements about how or a different you might pay to different degree of importance on what that code is like to work with than maybe an organization which is maybe operating at a different level of abstraction where they're they're really caring a bit more about maybe there's operational and external character you know, quality characteristics and what the code looks like is less important if you don't look at it.

Emily Bache：没错。即使人类不看代码，智能体也会看代码。而且似乎是这样：如果代码质量很差，智能体就会消耗更多 token，因此成本更高、耗时更长，也更容易出错。Emily Bache: Right. Well, even if humans don't look at the code, agents are looking at the code and it seems to be the case that the if the code quality is poor that the agents take a lot more tokens. And so it costs more, it takes longer, and they're more likely to make mistakes.

Sam Newman：我觉得这对我来说真的很有意思。所以我想，接下来的问题是：AI 编码智能体在意的质量属性，和人类在意的有什么不同吗？两者有重合之处，我觉得这很好，但这两类群体看重的特性是否有所不同？Sam Newman: I think that for me is like really interesting. So I think like the question then is are there different quality attributes that AI coding agents care about than humans? There's an overlap, which I think is good, but are there different characteristics of importance to those two groups?

Emily Bache：目前，我觉得我们还不知道。我的意思是，这里有太多事情我们并不了解。我们只是在推测，而且研究跟不上技术前沿的发展。因此，很难对此做出确切的判断，但就我合作的这类团队而言，他们仍然在阅读代码。Emily Bache: At the moment, it I don't think we know. I mean, there's so much here we don't know. We're kind of speculating. and the research is not keeping up with the with the state of the art. So it's hard to make definitive claims there, but for the kinds of teams I'm working with, they are still reading the code.

因此，对人类有效的方法似乎也有助于智能体，这是一个不错的起点。So it what works for humans seem to also help agents and that's a good starting point.

领域语言和验收测试Domain language and acceptance tests

Sam Newman：嗯，有一件事我觉得马上就能理解，而且对人类和智能体都同样适用，那就是我们在代码中使用的词语。比如，我们如何命名函数、类或其他东西。因为即使你是在较高的抽象层次上工作，你也会说类似这样的话：“发送订单时，确保也触发一封邮件”，或者类似的表达。你其实是在用领域中的术语和它交流。如果这些词没有出现在代码里，作为人类，我们会……我敢说，你肯定也遇到过这样的代码库：领域本身和代码的样子毫无关系，但我们会在脑子里完成这种转换，对吧？Sam Newman: Well, that's one thing that immediately makes sense to me that would be applied equally for humans and for agents is the words we use in the code. So, what we call a function, a class, or whatever else because say you are even are operating at that high level of abstraction, you're going to say something like when we send an order, make sure we also trigger an email or whatever you might say. You're you're pretty communicating with it with terms from the domain. And if those words aren't in the code, like as a human being, we will I bet you know I'm sure we've worked in code bases where what the domain is bears no relation to what the code looks like, but we do that translation in our heads, right?

但 AI 做不到。所以，如果代码里没有这种含义，这对查看代码的人来说是好事，对 AI 来说也一样，对吧？But AI can't. So, if that meaning isn't in the code, that's great for the humans looking at it, but that's also great for AI, right?

因为我们可以用这些术语来交流。而我觉得这一点仍然非常重要。Because we can talk in those terms. And that feels that feels really important still.

Emily Bache：没错。所以，把领域语言融入代码，取决于你如何设定需求，或者你如何与智能体沟通，告诉它应该构建什么、做什么。因此，这其中仍有很大一部分属于通过示例进行规格说明，以及编写良好的验收测试。我认为，领域语言以及提前设定这些内容，都可以从这里着手。Emily Bache: Right. And so, those getting that domain language into the code comes from the way you've set up the requirements or the conversation with the agent about what it should build and what it should do. So, there's still a big part of this which is specification by example and good acceptance tests and all of that's where the domain language and setting that up in advance, I think, can come from.

Sam Newman：实际上我和几个团队聊过这件事。我的意思是，我觉得这进一步凸显了领域驱动设计整体上的重要性，即便你不打算全盘采用，对吧？我身后就是那本书。书挺厚的，对吧？但我觉得，把语言提炼出来这一点真的很重要。因为当人直接与代码交互时，我们可以依赖脑海中掌握的许多隐性知识，但现在我们需要把这些知识明确表达出来。所以在我看来，这进一步凸显了做一些类似事件风暴的练习、以此来建立起一部分共同语言的重要性。Sam Newman: I've spoken to a few teams actually where they've you know, I mean I think I think this like reinforces the importance of domain-driven design in general, even if you're not going to buy the whole enchilada, right? The book's behind me. It's a thick book, right? But that aspect of getting the language out, I think, is really important because I think when we've got humans directly interacting with code, we can rely on a lot of tacit knowledge we've got in our heads, but we need to get it out now. and so, that for me kind of reinforces like the importance of doing something like an event storming exercise to bootstrap some of that language.

我甚至和一些团队聊过，他们就是没法让所有人聚到一个房间里来讨论，或者他们的团队更加分散，于是他们一直在用诸如“Grill Me”技能之类的东西，从大家那里收集他们使用的语言和术语方面的信息。我觉得这些事情真的就是……反正看起来都是些不言自明、顺手就该做的事，对吧？I've even spoken to some teams where they just can't get everyone in the room to have the conversation or they've got more disparate teams and they've been using things like the grill me skills to get information from people about how the language and the terms they use and I think those things are like really kind of just they they seem like no-brainer stuff to do anyway, right?

因为我觉得，从这个角度来看，它们对人类和 AI 都有帮助，而且我想，这样也可能更容易按照这些标准来验证软件的行为，对吧？比如说，做验收测试时，产品负责人在编写规格说明时，就可以用这些语言来明确验收标准。是这样吗？不过你比我更了解验收测试。Because I think they help humans and they help the AI from that point of view and it also probably I'd imagine makes it easier to verify the behavior of your software in those terms as well, right? When you're doing say an acceptance test cuz then you could have a product owner like when you're working on a spec decide they could probably specify the acceptance criteria as an example in those languages as well. Is that You know more about acceptance testing than I do though.

Emily Bache：我觉得，是的，我觉得你说到点子上了。我听到的也是这个意思：它是在定义这些测试、开展领域驱动设计方面的讨论，这真的很有帮助。我还发现大家在谈论内循环。你知道，按照传统做法，人们会用那些粒度非常细的单元测试来进行 TDD。Emily Bache: I think it yeah, I think you're on that's what I'm hearing as well that it's defining these tests having this domain driven design conversations and that's that's really helpful. I'm also finding that people are talking about the inner loop you know, where traditionally you'd have done TDD with the those very fine grained unit tests.

Sam Newman：是的。Sam Newman: Yeah.

Emily Bache：当然，那部分现在很大程度上是由智能体来完成的。这是一个非常大的变化，而且……Emily Bache: That that part of course to a large extent the agent is doing that part now. which is very big change and

TDD 与开发循环TDD and development loops

Sam Newman：是的。我对 TDD 有个颇具争议的看法。我并不是故意要引发争议，但我很喜欢 TDD。不过对我来说，TDD 一直是为了帮助设计代码，让人以那些小单元来思考之类的。我不知道 TDD 现在是否还有同样的价值，因为正如你所说，那个内循环已经交给智能体了。Sam Newman: Yeah. I have a controversial view about TDD. I don't like being controversial by purpose but I love TDD but for me it was always about helping design the code and thinking is those small units and sort of I don't know that TDD has the same value now because we to your point that inner loop has gone to the agent.

Emily Bache：是的，最近我在频道上专门做了一个视频谈这个，因为有些知名人士表示：“哦，我现在不再做 TDD 了。”但我和很多人聊过，他们仍然觉得 TDD 很有价值，因为最底层的内部代码质量依然很重要，而 TDD 是确保这一点的一种方法。如果你每隔几分钟就跑一轮这种紧密的开发循环——不，真正这么做的人并不是在用那种非常天真的、只用一段话描述的 TDD。他们为此搭建了相应的流程，可能会让子代理分别处理其中的部分工作，并使用非常具体的提示，以及习惯钩子或传感器，真正确保代理频繁重构。所以我觉得，关于如何确保底层代码质量，仍然有值得讨论的地方。Emily Bache: Yeah, and there's there's been I've made a video about this on the channel recently cuz there have been some high profile people saying, "Oh, I no longer do TDD." But I've been talking to a lot of people who are still finding that valuable because the internal code quality at the lowest level still seems valuable and TDD is one way that you can ensure that. If you're if you've got the these tight development loops going around every few minutes with No, it's it's the people who are doing this are not using a very naive kind of one paragraph TDD description. They they've got setups for this with perhaps sub subagents doing parts of it and very specific prompting and habit hooks or sensors that really ensure that the agent is refactoring frequently. So there is I think there's a still a discussion to be had about how to ensure that very low-level code quality

Emily Bache：也许 TDD 是解决这个问题的一部分。不过没错，现在情况确实不同了。Emily Bache: and perhaps TDD is part of the answer there. But yes, it is different now.

明确记录有益的变更Making good changes explicit

Sam Newman：我最近看到一个挺有意思的例子：我当时接触过一个来自爱尔兰 Finn 的团队，他们后来被 Salesforce 收购了。上周他们在 Honeycomb 的活动上做了一场演讲，谈到他们为避免需要人工审查代码所采取的做法。他们说：“现在我们很多代码都不再经过人工审查了。但要做到这一点，我们团队必须先明确‘一次好的变更应该是什么样的’，而且变更必须满足这些标准。其中一项标准是，变更集的代码行数必须低于某个数值，比如 500 行。”Sam Newman: There's something I seen quite a I was team from Finn in Ireland who's been purchased by Salesforce actually did a talk at Honeycomb's event last week and they were talking about the things that they do to avoid the requirement for a human to look at code. And they say, "A lot of our code doesn't have a human review now, but for us to get there we as a team had to say, 'This is what a good change looks like.' And you have to have passed those criteria. One of those criteria is the change set has to be below a certain number of lines of code. It might be 500 lines of code.

我可能是在瞎编这个数字，但他们说：“所以你得通过所有这些关卡，其中大多数都不是 AI 参与的自动化测试。”比如要跑 lint 检查，你知道的，还有 linter、各种其他检查，以及静态检查之类的。I might be making that number up, but they said, 'So you've got to pass all these gates, most of which are not AI involved automation test.' Like this like going through your lint you know, your linter and everything else and your static checks and things.

但如果这项改动超过 500 行代码，就必须由人工审核。所以你会受到激励，尝试和 AI 合作，把改动控制得更小。他们是在试图鼓励正确的行为。不过只要改动足够小，就可以直接合并。But like if this change is over 500 lines of code you have to have a human review. So you're incentivized to try and find a way to work with your AI to get it to be smaller changes. So they're trying to incentivize the right behavior. But if you do, then it can go straight in.

他们的理由当然是，你做的改动比较小。所以，如果那个未经人工审查的改动出了问题，处理起来也比较简单，对吧？不过，我觉得这是一种不错而且细致的折中：不必逐行检查代码、连分号都亲手一个个敲上；也不必干脆整件事都 YOLO 一把。Their rationale being of course is you've made a smaller change. So if that no human review change has had a problem it can it's a small change to deal with, right? But and I kind of I think that was a nice and nuanced approach to I've got to look at every single line of code and I've got to artisanally created and type my own semicolons too. I'm just going to yolo the whole thing.

而这更多的是再次把我们脑海中关于什么样的变更才算好的那些默会知识变得更加明确——也许可以说，这是回到这一点。Whereas this is much more making again maybe it's coming back to making that tacit knowledge in our heads about what a good change looks like making that more explicit.

这也再次帮助 AI 验证：如果它是在按照你的规范执行，那么它写代码的方式就是正确的。不过我也想到，这也能让新加入团队的人更清楚地了解该做什么，对吧？你需要做的就是这些。我觉得这挺有意思的。Which again helps the AI verify that it's writing code in the right way if it's executing your specs. But it also occurred to me that it also makes things more explicit for new people joining the team, right? This is what you've got to do. And I thought that was kind of interesting.

我的意思是，他们不得不在内部手动完成其中很多工作，不过……I mean they had to hand roll a lot of this internally but

艾米莉·巴什：我觉得这里有个、有个主题：你得知道什么样才算好。Emily Bache: I think there's there's a there's a theme here about you need to know what good looks like.

Sam Newman：是的。Sam Newman: Yeah.

Emily Bache：要保证智能体软件的质量，你知道，在某种程度上，这一切都是为了增强人类的专业能力。对我来说，反复出现的主题是，所有真正成功进行智能体开发的人，本来就已经非常娴熟了。没错，他们很清楚什么才算好，也擅长工程实践。现在，他们有了新的工具来开展工程工作。Emily Bache: For quality in agentic software you there's a certain amount of that this is all about augmenting human expertise. And it's the recurring theme for me is that all the people who are really doing agentic development successfully were already really skilled. Yeah, and really knew what good looked like and were good at engineering. And now they've got new tools to do that engineering with.

它们可以发挥更大的作用。所以我……我也从另一方面看到了这种情况：你把这些工具交给那些一辈子都没写过一个像样的单元测试的人，他们就会生成一大堆测试，而据我所知，这些测试基本毫无价值。现在要做到这一点比以往任何时候都容易。所以我认为，专业知识确实有其用武之地。They can work even more effectively. So I I've also on the on the flip side you can see this where you've give these tools to people who have never written a good unit test in their life. And they're generating oodles of tests and they're basically worthless as far as I can tell. and that is easier than ever to do. So I think there's a real place for expertise.

Sam Newman：当然。我的意思是，我看过几场演讲，也和一些人聊过他们是怎么使用规格说明的。稍微深入了解一下就会发现，他们基本上做的就是迷你版瀑布式开发，对吧？他们把规格说明当成一次性定稿的东西，比如说：“我们会花几周时间，团队一起编写规格说明，然后再实现这些功能。”好吧，各位，这种做法我们以前见过，对吧？Sam Newman: Absolutely. I mean I've seen a few presentations and spoken to a few people about how they're using specs and you don't have to dig very far to realize what they're basically doing is mini waterfall. Right? They're using the spec as as a sort of one shotting a spec that's say oh we'll spend a few weeks working on a spec as a team and then we'll implement the features like Okay, people we've done this before, right?

时间尺度不同，但结果、结果很可能会非常相似。我觉得你刚才说的那点——知道什么才算好——非常重要。几周前我参加了一个，怎么说呢，写作相关的活动。有好几个人发言，他们做的都是某种软件工厂式的工作，有些相当放手，或者说非常……而有些团队则非常依赖规格说明。Yossi Matias 当时也在场，那时他还没加入 Anthropic。The time scales are different but the but the results are the results are very much going to be the same. And I think that thing you said though about knowing what good looks like is so important. I did a I don't know writing thing on a few weeks ago. I had a number of people talking who were doing sort of software factory stuff where they're quite hands-off or certainly heavily and some teams have been quite heavy specs based off and I had Yossi Matias there as well just before he joined Anthropic.

而且，他们每个人都提到，如果你不能明确说明什么才算做得好，就没法完成这件事。因为如果做不到这一点，智能体就不会产出你想要的结果。因此，如果你正在创建某些东西，却不知道什么才算好，或者你对这个问题的理解要模糊得多，那你就必须采用另一种工作模式。否则，这根本行不通。And every single one of them talked about you can't do this unless you can be specific about what good looks like. Because if you can't do that, the agent's not going to produce what you want. And so if you are creating things where you don't know what good looks like, where your understanding of this problem is a lot more vague, you're going to have to have a different kind of working model. Otherwise, this just isn't going to work.

而且我觉得这挺有道理的，我……And that kind of makes a lot of sense to me and I

探索性测试与人类专业知识Exploratory testing and human expertise

Emily Bache：是的。我们能不能谈谈外部质量？我们刚才稍微聊了聊这件事的其他方面。传统上，我们一直进行大量手动测试来确保质量，尤其是在可用性方面。现在，出现了一些能够进行探索性测试的工具。Emily Bache: Yeah. Can we can we talk about the the external quality? so we talked a little bit about the kind of other parts of this. The external quality is traditionally we've had a lot of manual testing for ensuring that, particularly around usability, you know. And now there are tools coming out that are able to do exploratory testing.

事实上，本期节目由 Bear Queue 赞助，它就是这类工具之一。本期节目由 SmartBear 推出的 Bear Queue 赞助。Bear Queue 是 SmartBear 的智能体 QA 系统，旨在帮助测试跟上 AI 加速软件开发的步伐。它的 QA 智能体会探索并绘制应用程序地图，创建和维护测试，验证真实用户旅程，并发现传统脚本可能遗漏的覆盖缺口。And in fact this episode is sponsored by Bear Queue, which is one of those tools. This episode is sponsored by Bear Queue from SmartBear. Bear Queue is SmartBear's agentic QA system built to help testing keep pace as AI accelerates software development. Its QA agents explore and map your application, create and maintain tests, validate real user journeys, and uncover coverage gaps that traditional scripts can miss.

随着产品不断变化，Bear Queue 会根据产品当前的运行方式调整测试，帮助团队扩大测试覆盖范围、减少脚本维护，并在掌控整个流程的同时发现实际问题。As your product changes, Bear Queue adapts testing around how it works today, helping your team expand coverage, reduce script maintenance, and find real issues while maintaining control over the process.

今天就通过简介中的链接开始免费试用 Bear Queue。我还想指出，我觉得这可能会非常棒。不过我还是想回到那个关于专业知识的问题上。如果我不是一名专业的探索式测试人员，怎么能判断这些东西是否真的有用呢？想让一个什么都不懂的人留下深刻印象，是很容易的。Start your free Bear Queue trial today via the link in the description. And I just wanted to point out that this I think this could be amazing. but again, I'm I'm wanting to come back to that thing with expertise. If I'm not an expert exploratory tester, how can I tell if these things are actually doing it any doing any good? It's very easy to impress somebody who doesn't know anything.

Sam Newman：嗯，是的，当然。我记得早期那些视觉测试工具刚出来的时候，得由专家告诉你它们有多糟糕，对吧？因为它们是基于像素级精确匹配之类的定义。所以，我觉得……我的意思是，我们也可以从 QA 或测试领域专家的角度来看这件事：我共事过的最优秀的 QA 人员，总能审视你围绕某个系统编写的一系列测试，然后指出系统的哪些部分需要更多或更少的覆盖，甚至会说：“我们在这部分的覆盖已经太多了。”Sam Newman: well, yeah, absolutely. And you know, I remember the early visual testing tools that came out, and it took experts to tell you that they were junk, right? Because they're based on pixel-perfect definitions and things like that. So, I am I think it's I mean, but also, let's look at this from a from an expert in the QA field or testing field, it's like the best QAs I've always worked with were able to look at like a portfolio of tests that you have around a system to say which parts of your system need more or less coverage. and even saying, "We've got too much coverage in this area.

这没什么帮助。它拖慢了我们的进度。”转而从更全面的角度看正在发生的事情。比如说：“好吧，在生产环境中，我们在这方面遇到了一些问题。仔细看下来，我们或许可以在这方面增加一些测试覆盖。It's not helpful. It's slowing us down." looking more holistically about what's happening. Like saying, "Okay, in production, we've had some issues in this area. Looking at it, we maybe could do with some more coverage around that."

所以，这些工具正在变得越来越好，我当然也能看到它们的价值，尤其是在视觉元素方面，它们能提供帮助，甚至可能减轻一部分探索性测试的负担，而探索性测试往往更多地围绕一些可视化元素展开。不过，你仍然需要理解其中的平衡。顺便说一句，别忘了，我们讨论的还只是本质上主要具有确定性的软件，对吧？一旦我们开始将具备智能体能力的软件投入生产，事情就会变得更加复杂，对吧？因为到那时，我们就得面对一整套问题。是的，你仍然需要过去使用新工具时一直需要的那些专业知识，而且人们必须懂得如何使用这些新工具。哦，顺便说一下，我们现在还得开始讨论评估（Evals）以及人们的情感分析，并试着弄清楚：用户给聊天机器人的回答点了踩，是因为聊天机器人出了故障、做错了什么，还是因为聊天机器人运行正常，只是给了客户一个他们不喜欢的答案？所以这简直就是打开了一整罐麻烦。So, the tools getting better, and certainly I can see the value, especially around some of the visual elements in helping and maybe even reducing some of the exploratory testing burden, which was often a bit more around some of the visualized elements potentially. but you still need that understanding of the balance of all of this. bear in mind, we're just talking about software, by the way, that is deterministic primarily in nature, right? That gets even more crazy once we start putting agentic software into production, right? Cuz at that point, then we've got the whole Yes, you need all that expertise that you've always needed with new tools, and they've got to understand how to use these new tools. Oh, and by the way, now we've got to start talking about Evals and sentiment analysis from people and trying to understand was that thumbs down on the response from your chatbot because the chatbot was broken and did something wrong, or was it because the chatbot was functioning correctly, it just gave the customer an answer they didn't like and so it's a whole like can of worms.

Emily Bache：一旦你开始在产品中引入具有自主行动能力的 AI，那绝对会引出一大堆棘手问题。但即便没有这个问题，我想强调的是，如果你把这类工具交给真正优秀的工程师，把它视为增强他们能力的方式，让一个人能完成更多工作，你就会取得最好的效果；而不是指望用这种工具取代整个 QA 团队。我认为，至少目前这样做可能还行不通。Emily Bache: that's definitely a huge can of worms when you start having agentic AI in your product. But even without that I'd guess I just want to deemphasize that I think you're going to get the best results with these kind of tools if you give them to really good engineers and see it as a way to augment them and give one person do more than if you're going to replace your whole QA with this with a tool. I think that's that's probably not going to work yet at least.

增强与结语思考Augmentation and closing reflections

萨姆·纽曼：我觉得这也、这也在某种程度上关乎人在系统中的位置，还有某种思维方式。我大约十个月前听科里·多克托罗谈过这件事。他当时是在放射科医生的语境下谈的。Sam Newman: I think it's also it's also kind of where the position of the human is in the system to an extent. Some of the kind of the mindset. I saw Cory Doctorow talk about this stuff a about 10 months ago. He was talking like in the context of radiologist.

你有一位放射科医生，知道如何查看扫描图像并发现问题。在这种情况下，你可以从两个角度来看待 AI 的价值，对吧？第一种是：我们有六位放射科医生，那就裁掉其中五位，让 AI 先初步查看所有扫描图像，再由放射科医生进行检查。这样就能降低我们的成本。You've got a radiologist who knows how to look at a scan and spot things. And there are two ways you could approach the value of AI in that world, right? The first is let's get rid of we've got six radiologists, let's get rid of five of them and then we'll have AI do the first pass of all the scans and then have the radiologist check them. And that's going to reduce our costs.

对吧？但这样基本上就是让某个人给计算机批改作业。另一种用法则是：不，我们要把这种能力加到现有的放射科医生的工作中，让它像坐在医生肩膀上一样，在医生阅片时提醒他：“顺便看看这些点”或者“这里有些区域是我为你标出来的。”Right? But then you've basically got someone marking the homework of a computer. Or the other way to use it is no, we're going to add this capability to the radiologist we've already got and they're going to be like sitting on their shoulder going and you'll radiologist going through oh by the way, look at these spots or here's some areas I've highlighted for you.

我们一起合作，弄清楚这件事的方向，结果可能会更好。不过回到你刚才提到的，如果你把这些新工具视为裁员的手段，我认为你会逐渐失去有效使用这些工具所必需的专业知识。所以我一直更愿意这样看待这些工具：它们能帮我们做更多事情，也能帮我们处理一些比较简单的工作，让我们腾出时间专注于更有意思的事情。Let's work together and work out where this is and that's probably going to lead to better outcomes. But back to your point like if you're seeing these new tools as a way of getting rid of people I think you're going to start losing the expertise you need to use those tools well. So I think it's like I would always like to position this more as these tools can help us do more things, can help us do some of the easier things so we can free up time to focus on the more interesting things.

Emily Bache：是的，确实一直如此。这始终是测试人员关注的一个主题：自动化是有成本的，也就是要花费我的时间。而这实际上是一种机会成本，因为我把时间花在了自动化上，而不是探索上。Emily Bache: And that's that's always yeah, definitely. That's that's always been a theme in for testers looking at well, there's this cost to doing automation in terms of my time. And that's actually an opportunity cost cuz I'm spending time automating rather than exploring.

Sam Newman：是的。Sam Newman: Yeah.

Emily Bache：那么，怎样才是恰当的平衡呢？我认为，这些工具带来的机会是，自动化所需的时间也许会减少，从而让你有更多时间有效地运用自己的专业知识。这就是我希望这类工具能做到的。Emily Bache: And what is the right balance? And the opportunity I think you get with these tools is that the automation time maybe goes down and gives you more time for that using your expertise effectively. So, that's what I would be looking for in these kinds of tools.

Sam Newman：当然，不仅如此，如果你打算把 AI 智能体工作流融入产品，就还需要额外的带宽。因为我们突然得腾出精力，去弄清楚该如何验证这些行为方式与我们传统上对代码的预期截然不同的东西。Sam Newman: And of course not only that, but you're going to need that additional bandwidth if you then plan to put AI agentic workflows into your products because suddenly I need to free up bandwidth to get my head around how we now verify these things that don't behave in the way that we, you know, traditionally expect, you know, code to have code to behave.

Emily Bache：我们确实生活在一个有趣的时代。Emily Bache: We definitely live in interesting times.

Sam Newman：是的。Sam Newman: Yeah.

Emily Bache：或许现在正是结束讨论的时候。Emily Bache: Maybe this is the point to round off the discussion.

Sam Newman：是的，我觉得……我真的很喜欢把内部质量和外部质量区分开来。我觉得这样来理解这件事也很不错。所以我挺喜欢这个说法的。不过，希望这次讨论……我觉得我们之前争论得很激烈。我不知道这会不会让讨论更有意思，但也许我们在大方向上是达成共识的。Sam Newman: Yeah, I think it's it's I think it's I really like the separation between the in the you know, the internal versus external quality. I think that's that's a nice framing around this as well. So, I I kind of like that, but yeah, hopefully this has been I think we were in violent disagreement. I don't know how interesting this makes this, but maybe we're in, you know, broad agreement about this.

Emily Bache：是啊，也许我们对 TDD 本身的看法不太一致。Emily Bache: Yeah, maybe we're not quite aligned on TDD specifically.

Sam Newman：不，我喜欢这个。我想我还在琢磨这件事。比如，我喜欢对自己说一些有争议的话，看看自己是不是真的还相信那些观点。所以，我想在和更多团队聊过之前，先暂缓发布那篇会引起轩然大波的博客文章。Sam Newman: No, I like that. I I think I'm still working that through. Like, I like saying controversial things to myself to see if I really still believe things or not. So, we I you know, I'm just I'm going to hold fire on the explosive blog post on this until I've spoken to a few more teams, I think.

Emily Bache：不管怎样，Sam，很高兴和你聊这个话题。谢谢。Emily Bache: It was great talking to you about this topic though anyway, Sam. Thank you.

Sam Newman：是的，谢谢你，Emily。也感谢 Bear Queue 赞助本场活动。如果你有兴趣了解市面上的一些新工具，可以看看 SmartBear 提供了哪些产品，了解这些新工具如何帮助 QA 把更多时间花在那些棘手的事情上。Sam Newman: Yeah, thank you, Emily. And thank you also to Bear Queue for sponsoring the session. If you're interested in looking at some of the kind of new tools out there, go look at what SmartBear have got to offer and see how these new tools might help your QA spend more time on the hard things in life.

Emily Bache：当然。Emily Bache: Absolutely.