更好的 AI 模型救不了工程团队：Nicholas Arcolano 谈真正影响产出的因素

简要描述Brief Description

Jellyfish 的 AI 与研究负责人 Nicholas Arcolano 探讨了 AI 工具、技能、令牌消耗、公司和个人工程师与已合并拉取请求吞吐量之间的关系。他借助高尔夫类比和工程观察数据，说明领导者能够影响哪些因素，同时指出吞吐量并非质量指标。Nicholas Arcolano, head of AI and research at Jellyfish, examines how AI tools, skills, token consumption, companies, and individual engineers relate to merged pull request throughput. He uses a golf analogy and observational engineering data to explain which factors leaders can influence, while noting that throughput is not a quality metric.

目录Table of Contents

• 简介
  Intro
• 一个高尔夫类比
  A golf analogy
• 模型不断变得更好。
  Models keep getting better
• 一项自然实验：Grok 开关切换
  A natural experiment: the Grok switch
• 究竟是什么驱动输出
  What actually drives output
• 该工具
  The tool
• 你对它的运用程度：技能
  How well you use it: skills
• 使用量：token 数量
  How much you use it: tokens
• 你的工作地点
  Where you work
• 您，作为个人
  You, the individual
• AI 正在重新洗牌顶尖工程师的排名吗？
  Is AI reshuffling the top engineers?
• 领导者能够掌控的方面
  What leaders can control

简介Intro

我是 Nicholas Arcolano，Jellyfish 的 AI 与研究负责人。很高兴再次来到 AI Engineer。今天我想谈谈，为什么如果你没能通过 AI 获得想要的成果，问题很可能不在模型，而在你自己。既然我们今天在这里，而且领导力专题有财务主题，那我们就先从一个高尔夫的比喻说起，应该挺合适的。现场有人打高尔夫吗？I'm Nicholas Arcolano. I'm the head of AI and research at Jellyfish. It's great to be back here at AI Engineer. And today I want to talk about why if you're not getting the outcomes that you want with AI, chances are it's actually not the model, the problem is you. since we're here, you know, we're doing this finance theme thing in the leadership track, we're going to start with a golf analogy. Seems appropriate. is anybody here a golfer?

一个高尔夫球类比A golf analogy

有打高尔夫的吗？有人吗？如果你……好吧。是这样，如果你不打高尔夫，请多包涵。我很抱歉。如果你打高尔夫，我也很抱歉。这是一项非常折磨人的运动。我喜欢高尔夫，但我打得很差。说自己打得差算是这项运动文化的一部分，但我是真的打得差。你在球场上的时候，只想让这种痛苦消失。你会想到其中一件事，比如说：“哦，我是不是该买一套更好的球杆？这样打起来就容易些了，对吧？”然后你去找教练，他就会说：“不，问题出在你身上。你不需要更好的球杆，对吧？关键在于基本功。”不过你知道，我是个数据控，所以我会想：“我不知道。这真的是这样吗？”如果你看看数据，就会发现球杆确实很重要，球杆也确实越来越好了。所以，你们现在看到的是从1980年至今，PGA巡回赛的平均开球距离。Any golfers? Anybody? If you, okay. so yeah, I if you don't golf, bear with me. I apologize. If you do golf, I also apologize. It's a very painful sport. I love golf. I'm a bad golfer. it's part of the culture to say you're a bad golfer, but I genuinely am a bad golfer. And when you're out there, you just want to make the pain go away. And one of the things you think of is like, oh, good should I just buy better clubs, right? That would make this easier. and then you get an instructor and he's like, no, like you're the problem. You don't need better clubs, right? it's about the fundamentals. But, you know, I'm a I'm a data guy, so I'm like, I don't I don't know. Is that really true? if you look at the data, like clubs matter. Clubs have gotten better. So, what you're looking at here, this is the average driving distance for the PGA tour from 1980 to the present.

1980年时，我很震惊。我确实不知道这一点。平均距离是257码。我自己也打出过这么远。正如我提到的，我是个很差劲的高尔夫球手。所以自1980年以来，距离增加了大约20%。你知道，有一条证据表明球杆确实有影响：就是中间这段实心球时代，到2000年左右。那时发生了一系列技术变革，距离也恰好一下增加了大约15码。人们不可能在那三年里突然变得更厉害；这完全是技术进步带来的。In 1980, I was shocked. I actually didn't know this. The average was 257 yd. Like I've hit the ball that far. And as I mentioned, I'm a terrible golfer. so it's gone up about 20% since 1980. And you know, one bit of evidence that the clubs matter is this solid core ball era thing in the middle by 2000. A bunch of technological changes happened, and there was like a 15-yard bump right at that time. People didn't magically get better over that 3-year span, like that was all technology-driven.

模型不断变得更好Models keep getting better

这张图让我想起了我们在模型前沿看到的很多情况，对吧？我们看到模型在稳步进步，变得越来越好。这是来自 Artificial Analysis 的一张图。我想大家都看过一张或很多张这样的图，每次有新模型发布时都会查看，对吧？所以，模型变得越来越好。我觉得这点没有争议。我们现在能做一些一年前根本做不到的事情，对吧？所以，模型非常重要。对吧？是吧？嗯，我……所以，我也不确定。呃，这很有意思，对吧？所以，我在一家叫 Jellyfish 的公司工作。我们将 AI 编码工具和智能体的遥测数据，与工程工作流数据和工程成果结合起来。我们拥有同类数据集中规模最大的一个，涵盖数十万名开发者和大约 1,300 家公司。This chart reminded me a lot of what we're seeing with the model frontier, right? We're seeing the steady advance of models getting better and better. This is a chart from artificial analysis. I assume everyone has seen one or many of these and checks them every time a new model comes out, right? So, the models have gotten better and better. I think that's uncontroversial. We can do things that we just could not do a year ago, right? So, the models matter a lot. Right? Yes? Well, I So, I don't know. The So, this is interesting, right? So, I work I work at this company called Jellyfish. we join AI coding tool and agent telemetry with engineering workflow data, engineering outcomes. we have the largest data set of its kind, hundreds of thousands of developers, about 1,300 companies.

一项自然实验：Grok 开关的切换A natural experiment: the Grok switch

我们一直在追踪他们在开发领域使用 AI 所做的各种事情。我在数据中注意到一件事：这些数据来自 SpaceX 宣布收购 Cursor 前后那段时间。我们发现 Grok 的使用量大幅上升，模型切换幅度也是我们见过最大的，速度远远快于工程师通常只是改变偏好、寻找最适合当前任务的模型时的情况。深入调查后我们发现，Cursor 在界面上做了很多调整，比如在界面中提升 Grok 的位置，以及改变自动模式的工作方式。因此，Grok 的使用量很快就飙升了。作为一名数据科学家，这件事令人兴奋，因为这就是我们所说的一种“自然实验”：出现了这样一个外部冲击，自然会想问，结果如何？And we're tracking sort of all the things they're doing with AI in the development space. And one thing that I noticed in our data is this is data from around the time when SpaceX announced its acquisition of Cursor, and we noticed a huge bump in Grok usage and the biggest model change that we've ever seen, far faster than just normal engineers sort of changing their preferences and trying to seek out the best model for the job. And when we dug into it, what happened was Cursor was doing a lot of things with the UI, sort of promoting Grok up in the up in the UI, changing how auto mode worked. And so, you know, it shot up very fast, and what was exciting about this as being a data scientist is this is what we call kind of a natural experiment. There was this external shock, and a natural question to ask would be, well, what happened?

对吧？比如说，产出有没有变化？你们在这张图上看到的是每位工程师合并的 PR 数量变化。在我刚才展示的那段时间里，这里的每个点都代表一位工程师。横轴表示他们使用模型的比例从不使用 Grok 到使用 Grok 的变化幅度，范围是 0 到 100%。纵轴表示他们的 PR 产出相对于同一家公司同事的变化幅度。这里的结果其实没什么故事可讲。数据点大致聚集在零附近，没有统计相关性。产出没有变化。可以说，把所有人都推去用 Grok，实际上没什么影响。所以，这个实验让我感到困惑，也让我想知道模型到底有多重要。如果它没那么重要，那还有哪些因素确实会产生影响？接下来这场演讲剩下的部分，我们就来聊聊这些问题。Right? Did output change, for instance? So, what you're looking at here on this chart, this is change in merged PRs per engineer. And so, over that time span I showed you, every dot here is an engineer. And the x-axis is how much of their model share went from not using Grok to using Grok on a scale from 0 to 100%. The y-axis is the relative change in their PR throughput relative to their colleagues at the same company. And the story here is a non-story. There's the cloud sort of center around zero. There's no statistical correlation. Output didn't change. Sort of shunting everyone to Grok didn't really matter. So, I left this experiment confused and wanting to know how much does the model matter. And if it doesn't matter much, what are the other things that do actually matter? So, that's what we're going to spend the rest of the talk talking about.

真正驱动输出的因素What actually drives output

那么，接下来我们要做的是：继续关注这个产出指标——每位工程师每周合并的 PR 数量。我们会使用一个经过筛选的数据集。这是我们数据中的一个子集，包含了开展这项实验所需的所有信息。数据涵盖 447 家公司的约 5 万名工程师，是今年 Cursor 和 Claude Code 用户的数据，还有一些重要因素。首先是你使用的工具。我们会把 harness 和模型放在一起考察，因为它们彼此交织得很紧密。这就像你的球杆，好吗？关键在于你能多熟练地使用它。具体来说，我指的是技能的使用、Cursor 规则、harness 的定制，以及我们这些 AI 工程师都在摸索的各种技巧，目的是让我们更好地使用这些工具。打个高尔夫的比方，这就像是知道怎么从沙坑里脱困——这可不是自然而然就会的。So, here's what we're going to do. We're going to continue to focus on this output metric of merged PRs per engineer per week. We're going to use a curated data set here. This is a subset of our data where we've got all the pieces we need to do this experiment. About 50,000 engineers at 447 companies. Data from this year for Cursor and Claude code users. And some of the stuff that matters. So, you've got the tool. We're going to put the harness and the model together because they're very intertwined. This is like your clubs, okay? There's how well you use it. By that, I mean things like your skill use, cursor rules, the harness customization, the tricks of the trade that we're all learning as AI engineers to make the way that we use these tools better. Golf equivalent would be like this is knowing how to get out of a bunker, which is not like a natural thing.

这是你必须学习的一项技能。还有你使用它的程度。撇开这些因素不谈，单看 token 消耗的绝对量。就像你在球场上打高尔夫的时间，对吧？然后还有一些因素，比如你在哪里工作、所处的环境。你的公司和团队就像球场本身，是你作为高尔夫球手无法掌控的。然后还有你这个个体，作为工程师你自身具备的所有特质；在其他因素都保持不变的情况下，这些特质仍然让你与众不同。It's a thing you have to learn. There's how much you use it. So, separate from these things, just raw amount of token spend. So, golf time on the course, right? and then there's factors like where you work, your environment. So, your company and your team, this is the course itself, something you don't have control over as a golfer. And then there's you, the individual, all the things that you carry with yourself as an engineer that, holding all these other factors constant, that make you still distinct.

所以，这就是我们接下来要讲的内容的路线图。把这些东西都整合起来之后，你会进行一大堆复杂的统计分析；我很想在这里深入讲讲其中的细节，但除了我这样的统计学家，其他人听了都会觉得无聊。不过，这就是路线图。PR 吞吐量的变化中，受工具影响（也就是能由工具解释）的部分不到 1%。你用得有多好，也就是技能这一部分，大约占 3%。你用得有多频繁，影响大约是工具的 10 倍，占 9%。你在哪里工作，占 12%。占比最大的是你自己，也就是个人因素。我们会逐一讲讲这些因素，并解释一下为什么会是这样的。嗯。不是，这里没有质量指标，不过质量确实是我们经常关注的一个方面。So, this is the map for the rest of our talk. You put all this stuff together, you do a bunch of big statistical analyses, the details of which I would love to get into here, but would bore everyone except for statisticians like me. But, this is the map. You've got about less than 1% the in terms of how your how much your PR throughput changes is controlled, explained by the tool. How well you use it, the skills part, that's about 3%. how much you use it, that's like 10x the tool, that's 9%. Where you work is 12%. The biggest piece is you, the individual. We'll go through each of these and explain a little bit about why this looks like this. Mhm. No, there's no quality metric here, but quality's a thing we look at a lot.

我很乐意之后再详细聊聊这个，不过问得很好。I'm happy to talk more about that later, but good question.

该工具The tool

那么，我们先来看工具。我们会快速过一下这部分。正如你们从我演讲的标题中可能猜到的，这是最没意思的部分。不过，工程师切换工具时会发生什么？我们看到了他们在某种程度上受到鼓励去切换工具后会发生什么；但工程师一直在切换自己使用的模型，也会切换 harness。你可以做一些分析，比如观察每周合并的 PR 数量变化。你可以把一位工程师和他自己做比较，看看他们全力转用 Opus 而不是 Sonnet 时会发生什么，或者转用 ChatGPT 会怎样。如果他们从 Cursor 切换到 Claude Code 又会怎样？我们有很多关于做出这种切换的用户的数据。so, let's do the tool first. So, we'll kind of speed through this. This is the least interesting, as you might guess from the title of my talk. But, what happens when engineers switch tools? We saw what happened when they were kind of encouraged, let's say, to switch tools, but engineers switch the models they use all the time, they switch harnesses. So, you can do things like look at change in the weekly merge PRs. You can compare an engineer to themselves and look at what happens when they go all in on Opus versus Sonnet, switch to ChatGPT. what happens if they switch from Cursor to Claude Code, which we have a lot of data around people who've made that switch.

从完全使用自动补全模式工作切换到代理模式，会发生什么？结论大致相同。从统计上看，这基本没什么值得一提。就解释 PR 吞吐量是否有任何实质变化而言，这里并没有太多内容。What happens when you switch from working totally in an autocomplete mode to an agentic mode. The story's kind of the same. Statistically, it's kind of a nothingburger. There's not a lot here in terms of explaining any meaningful change in the amount of PR throughput that you get.

你对它的使用水平：技能How well you use it: skills

那你使用它的效果如何呢？这部分占 3%，在整个饼图里大得多。我们来聊聊技能的使用。这一点很有意思，因为技能本该是个更重要的因素。我们发现，技能确实很重要。它之所以没有那么重要，其中一个原因是——我觉得这可能会让一些人感到意外——在现实世界中，除了我们现在身处的这个美妙的 AI 工程会议圈子，大家其实并不怎么使用技能。这张图展示了技能使用情况随时间的变化，按工程师周数统计。也就是说，工程工作周中，技能使用为零、使用程度较低、中等和较高的比例。What about how well you use it? This is 3%, so much bigger piece of the pie. Let's talk about skill use. This one's interesting because skills should be a bigger factor. We find that skills matter a lot. One of the reasons it doesn't matter more is and I don't This might surprise some folks. People do not use skills that much in the real world outside of this wonderful land of AI engineering conferences that we're inhabiting. this chart shows you know, skill use engineer weeks over time. So, proportion of engineering weeks that have, you know, zero skill use, a low level, a medium level, and a high level of skill use.

截至 9 月，你知道，这里展示的周数中，几乎有一半基本没有使用技能。只有 12% 的周数，技能使用水平达到了我认为现代 AI 工程应该达到的程度。之所以这点如此重要，是因为当技能得到普遍使用时，它对产出的影响远大于模型选择。我这里再次展示之前提到的结果，把更换模型系列作为参照。下面展示的是技能使用水平，以及在其他因素保持不变的情况下，它对 PR 吞吐量的影响程度。你可以看到，当每位工程师每周使用技能的次数达到 30 次以上时，仅这一因素就能带来 30% 的提升，其他因素则保持不变。And as of September, you know, almost half of the weeks here shown had essentially no skill use. and only 12% were using skills at a level that I think we would consider as like what you should be doing in terms of modern AI engineering. And the reason that this is so important is because skill use when it's prevalent moves output way more than model choice. So, I'm showing here again those results I flashed from earlier about changing model family as kind of a reference. But on the bottom here is level of skill use and the amount of effect it has in terms of changing the level of PR throughput holding these other factors constant. And you can see you get you know, when you get up to 30 plus runs of using skills per engineer per week, that's a 30% bump just from that factor alone holding other things constant.

所以，这是个很大的影响。So, it's a big effect.

使用量：令牌数How much you use it: tokens

好，我们来聊聊你用了多少。这就是用量，也就是 token 数量，以及活跃使用天数。这个话题我本来可以完整讲一场。事实上，我确实在 AI Engineer World's Fair 上专门讲过这个话题。所以，如果你觉得有意思，我鼓励你去看看那场演讲。不过，我先给你讲一个一页幻灯片版。你现在看到的数据就来自那场演讲。这些数据涵盖了大约 50 万个开发者周的行为。每个点代表一个 2 个百分点的分组，也就是该周用量水平相同的一群人；它显示的是这群人每位开发者每周平均消耗的 token 数，以及合并的 PR 数。基本结论是：token 用得越多，产出就越高。但边际收益会递减，因此曲线会逐渐趋于平缓。在那场演讲里，我把它比作火箭方程。All right, let's talk about how much you use it. This is consumption. This is tokens. This is active use days. this one I could do a whole talk about this. In fact, I did a whole talk about this at the AI Engineer World's Fair. So, I encourage you if you find this interesting to take a look at that. but I'll give you the one slide version. So, the data you're looking at here is from that talk. So, this is about 500,000 developer weeks of behavior. Each dot is a two percentile bin of folks that use the same level of consumption in that week, and it's their average token spend and their merged PRs per developer per week. And the basic story is more tokens, more throughput. And there's diminishing returns. So, there's sort of a saturating curve. I likened it to the rocket equation in that talk.

这就像火箭燃料。结果是，如果你比较前 10% 和中位数，前 10% 的产出基本上可以达到中位数的两倍。代价是要消耗大约 10 倍的 token。但 token 越多，产出也越多。现在我们来看看更大的影响因素。这些挺有意思的。比如你工作的地方、你的公司、你的团队。It's like rocket fuel. And what you get is if you look at the top 10% versus the median, you can basically double your output between the median and the top 10%. It costs you about 10x the tokens. But more tokens mean more output. Now we get to the bigger pieces of the pie. These are interesting. So, there's the factor of you know, where you work, your company, your team.

你的工作地点Where you work

实际上，如果把这些因素拆开来看，大约三分之一来自团队，三分之二来自公司。你知道，比如代码库、评审流程和企业文化。那么，一旦你把消耗量固定下来，你知道，其他这些因素还重要吗？一种看待这个问题的方式是：如果把这些因素固定住，只看差异性，也就是看在其他条件都不变的情况下，各家公司相对于平均水平的分布范围，结果就是这样的。对此有一种解读方式是，我刚才给你们看了那条 token 曲线。所以，你可以花 10 倍的 token，把表现从中位数提升到前 10%，从而获得两倍的产出。Actually, if you tease out the factors, about a third of it's the team, two-thirds is the company. You know, your code base, your review process, your culture. So, once you sort of fix the amount of consumption, you know, do these other things still matter? One way to look at this is, you know, if you fix these factors and just look at the variability, you look at the spread of companies relative to the average company holding these other things fixed. this is what it looks like. And one way to interpret this is, I just showed you that token curve. So, you could spend 10x the tokens to move from the median up to the top 10 percentile and get double the output.

在这个分布中，即使消费量保持不变，排名前 10% 的公司和排名后 10% 的公司之间也相差一倍。换句话说，即使 AI 支出固定，单是文化、团队和公司这些因素，也会造成很大的差异。There's also a 2x spread between the top 10% company and the bottom 10% company in this distribution holding your consumption fixed. So, in other words, your culture, your team, your company, there's a lot of variation just in those things even for fixed AI spend.

这非常重要，对于这里参加领导力课程的这个小组来说也很有意思，因为这是领导者能够掌控的事情。比如说，掌控这些事情就是我们的职责，即使我们的 CFO 可能会告诉我们，不能再增加 token 了——希望他们不会这么说。This matters a lot and it's interesting for this group in the leadership track here because this is what leaders can control. Like, this is our job is to control these things even if maybe our CFO tells us we can't have any more tokens than we have. which hopefully they don't.

你，作为个人You, the individual

那最重要的部分呢？就是你，作为个体的你，对吧？这占了这个统计模型的四分之三。所以，这一点非常重要。What about the biggest piece? You, the individual, right? This is this three quarters of this statistical model. So, it matters a lot.

一种思考这个问题的方式是：很多人都问过，AI 是否意味着我们需要不同的工程师？我们需要不同的人才吗？对吧？它是否会重新排列排名？它是否改变了优秀工程师的定义？你们刚才看到了我展示的公司分布。如果用同样的方式来看工程师，分布范围还要更宽。因此，差异也更大，这也是为什么在我们构建的统计图谱中，它是一个如此重要的因素——这张图谱用来解释我们可以采取的不同杠杆。分布范围确实更宽。你知道，排名前 10% 和后 10% 之间相差 3.5 倍，而公司之间的差距是 2 倍。这里有很多值得探讨的内容。要把 AI 工程师之间的差异、以及为什么有些人比其他人更优秀的所有因素都讲清楚，还得另开一场演讲。one way of thinking about this is this question that a lot of people have asked is, you know, does AI mean that you need different engineers? Do we need different people, right? Is it resorting the rankings? Is it changing what it means to be a good engineer? So, you saw that distribution I just showed you with companies. If you do the same sort of distribution for engineers, it's even wider. So, there's even more variability and that's why it's such a big factor in this statistical map we've we've laid out for ourselves to explain the different levers available to us. it's even wider. It's, you know, 3.5x spread between the bottom 10% and the top 10% instead of 2x as for companies. And there's a lot to unpack here. This will be a whole 'nother talk to unpack all the things that make different AI engineers better than others.

AI 正在重塑顶尖工程师的格局吗？Is AI reshuffling the top engineers?

我能在剩下的几分钟里稍微回答的一个问题是：AI 正在改变哪些工程师最具生产力。来看一个角度：这里有一项针对 10,000 名工程师的小型研究，对他们进行了排名。我们有长期的纵向数据，时间跨度很长。Jellyfish 已经运营了 10 年，所以我们掌握了大量关于工程师行为的历史数据。如果你看看顶尖工程师的排名，比如一个季度接一个季度、半年接半年地比较，我们就能了解顶尖工程师在生产力排名中的流动情况。这里说的仍然是 PR 吞吐量：他们有多少人会从前四分位中流出。相比之下，大约每半年就有 44% 的人会离开前四分位。如果比较 AI 出现前后，这个比例实际上略微收窄了。One thing that we can answer a little bit in the few minutes I've got left is AI changing who the most productive engineers are. So, one way to look at this, here's a small study ranking 10,000 engineers. We have long you know, longitudinal data, long timeline. Jellyfish has been around 10 years. So, we have, you know, tons of history about how engineers behave. And if you look at, you know, how top engineers rank you know, quarter over quarter, 6 months over 6 months, we can look at, you know, how much do your top engineers cycle in terms of productivity here. PR throughput we're still talking about. How much they cycle out of the top quartile. And if you compare that so, you know, about 44% cycle out of the top quartile every half year. if you look at before AI versus after, it actually tightens a little.

人们的流动性略有下降，降幅是41%。我们还做过其他类似的研究。我们并没有看到世界秩序出现大规模重新洗牌。总体而言，AI出现之前最优秀的工程师，现在仍然是最优秀的工程师。尽管有一种说法认为AI正在拉平竞争环境，但在很多方面，技艺依然重要，这也是这个因素影响如此之大的部分原因。People get a little less mobile. It's 41%. So, and we've done other studies like this. We're not seeing you know, a big re-sorting of the world order. Your best engineers before AI by and large are still your best engineers. Craft still matters for a lot despite the narrative that AI is leveling the playing field, and that's why part of why this factor is as big as it is.

领导者可以控制的事项What leaders can control

回到这张我一直在给你们看的图。综合来看，我希望你们记住的是，左边这些东西，尤其是工具，确实很值得谈论。它们是很酷的新技术，占据了房间里几乎所有的话题，也占满了 Twitter 上的讨论。这很令人兴奋，我也很喜欢这些东西。但我们对它们没有太多掌控权。我不在基金会实验室工作，也没法让这些工具变得更好。而随着成本压力越来越大，我们能自由购买心仪工具的空间可能也会越来越小。但作为领导者，我们能够掌控的，是等式的另一边。Going back to this picture that I've I've been showing you. When you add it all up, you know, what I want you to take away is these things on the left, you know, the tool especially, are like really exciting to talk about. It's the cool new tech. It takes up kind of all the air in the room, all the air on Twitter. And like that's exciting, and I love that stuff. but we don't have a lot of control over that. I don't work at a foundation lab. I can't make the tools better. and as cost tighten, we have less and less freedom maybe to buy the tools that we want. But as leaders, the things we can control are on the other side of the equation.

是人，是组织。我想，回到我们打高尔夫的比喻，这件事真正令人兴奋的地方在于，不像高尔夫球场，你可以改变球场，也可以改变你的环境。比如，你可以把水障碍填平，或者做一些在高尔夫球场上尝试的话会被人喝止的事。但这些都是我们作为领导者可以采取的做法。最后，如果你没有取得想要的结果，我想鼓励你思考一下：也许问题不在于这个模式，也许在于你自己。就是这样。It's it's the people, it's the organization. And I think what's, you know, going back to our golf analogy, what's really exciting about that is unlike golf, you can change the course. You can change your environment. So, you can like fill in water hazards or do things that they yell at you if you try to do on a golf course. But those are things as leaders that are available to us. So, in closing, you know, if you're not getting the outcomes that you want, I would encourage you to think, you know, maybe it's not the model, you know, maybe it's you. So.

谢谢。非常感谢。我的内容就这些。如果你感兴趣，我们每月都会发布基准报告，涵盖这些数据以及更多内容。请访问 Jellyfish AI 工程趋势。我们的许多研究成果都发布在 Substack 上。谢谢。Thank you. Appreciate that. That's all I've got. If you're interested, we publish monthly benchmarks with all of this data and more. Go to Jellyfish AI engineering trends. a lot of our research is published on our Substack. thank you.