AI 智能体如何影响专家的判断决策

阿马奇·赫尔达德伦、海莉·乔伦、亚历克斯·塔姆金Amaç Herdağdelen, Hailey Joren, Alex Tamkin

当专家把工作交给 AI 智能体时，人们通常会认为，真正重要的决策仍由专家来做。智能体编写代码、计算数据并提供建议，但由专家决定测量什么、如何解读含糊的证据，以及接下来该采取哪些步骤。这些问题往往不止一种合理答案，而最终的决定取决于作出判断者的经验和价值观。When experts hand their work to an AI agent, the usual assumption is that they still make the decisions that matter. The agent writes the code, runs the numbers, and offers advice, but the expert decides what to measure, how to interpret ambiguous evidence, and what the next steps should be. Often there is more than one reasonable answer to these questions, and the decision will come down to the experience and values of whoever makes the call.

我们想了解专家是否确实会做出这些重要决策，还是在实际工作中会把决定权交给他们的 AI 智能体。在一篇新论文中，我们报告了一项研究：我们请 50 位专业分析师使用 Claude Code 进行一项贴近现实的统计分析。他们都有因果推断方面的经验，也就是根据数据判断一件事是否会导致另一件事。他们的任务是为一家农业合作社提供建议，评估该合作社正在运行的一项咨询服务是否有效，以及是否应该扩大服务规模。We wanted to look at whether experts really do make those important decisions, or whether in practice they defer to their AI agents. In a study we report in a new paper, we asked 50 professional analysts to do a realistic statistical analysis with Claude Code. They all had experience in causal inference—that is, working out from data whether one thing causes another. Their task was to advise a farming cooperative on whether an advisory service it was running was effective and should be expanded.

在此过程中，分析人员必须在两种合理选项之间作出四次选择，例如，将生产力衡量为每公顷作物产量，还是每单位运营成本对应的产量，最后还要作出最终决定。为了了解他们是否在自行作出这些判断，我们为每位分析人员的智能体设置了倾向于每项决策中某个选项，而分析人员对此并不知情。智能体会“倾向”哪一方，由抛硬币决定。我们要求智能体尽可能诚实、有力地为自己倾向的选项辩护；如果证据要求它改变看法，就应改变立场，绝不可误导他人。Along the way, the analysts had to make four choices between two reasonable options, such as whether to measure productivity as crop yield per hectare or as yield per unit of operating cost, and then make a final decision. To see whether they were making these calls themselves, we set up each analyst’s agent to favor one option on each decision, which the analysts weren’t told about. Which way the agents would “lean” would be decided by coin flip. The agent was told to make the best honest case for its lean, to change its mind if the evidence demanded it, and never to mislead.

在四项分析选择中，分析师有86%的情况下采纳了智能体倾向的选项。如果智能体的意见毫无影响，我们预计这一比例约为50%。分析师在智能体发表意见前已表达自己的偏好时，若智能体意见与之相左，他们有60%的情况下会放弃原先的偏好；若智能体意见与之相符，这一比例仅为6%。Across the four analytical choices, analysts went with their agent’s lean 86% of the time. If the agent’s input made no difference, we’d expect about 50%. When analysts had already expressed their own preference before the agent weighed in, they switched away from it 60% of the time if the agent disagreed, compared to 6% if it agreed.

随后，在被要求解释自己的选择时，77%的分析师使用了智能体自己的论据——有时甚至一字不差——但只有15%的人承认这些论据来自智能体。紧密遵循智能体建议的分析师，对最终建议的信心与其他分析师大致相当，也同样可能将其称为自己判断的结果。Afterward, when asked to explain their choices, 77% of analysts used the agents’ own arguments—sometimes word for word—but only 15% credited the agent as the source of those arguments. Analysts who followed the agents’ recommendations closely were about as confident in their final recommendation as the rest, and roughly as likely to call it the product of their own judgment.

我们不知道这在多大程度上能反映现实世界。真实的智能体有时会倾向于某一方或另一方——在没有设定任何倾向的情况下，Claude Opus 4.8 在 123 次运行中有 112 次做出了相同的五个选择——但在这个案例中，我们有意强化了它们的偏好，以衡量其影响。如果没有明确告知智能体有哪些偏好，也没有指示它们表达这些偏好，这种影响可能会更小。We don’t know how much this tells us about the real world. Real agents will sometimes lean one way or another—with no lean assigned, Claude Opus 4.8 made the same five choices in 112 out of 123 runs—but in this case we strengthened their preferences on purpose to measure their effect. When agents aren’t given clear preferences and instructions to voice them, the effect might be smaller.

这并不是一个简单的、告诫人们不要与 AI 合作的故事。使用智能体的分析师更快地完成了任务，严重错误也远少于选择独立工作的分析师；不过，后者人数较少、经验也较浅，因此我们谨慎看待这些结果，不愿过度解读。但随着 AI 承担越来越多的实际工作，越来越多的此类决策将成为人类判断与价值观影响结果的关键所在。如果专家在未意识到的情况下听从智能体，智能体的偏好——这些偏好可能是随意的，也可能与人类目标不一致——就可能左右有关政策提案或研究方向的重要决策，而没有人真正选择过这些偏好。这一点对于确保 AI 安全的研究尤为重要。研究人员已经开始使用智能体来开展和分析实验，而只有当人类审查者依据自己的判断作出评估时，他们的审查才算独立。This isn’t a simple cautionary tale against working with AI. Analysts who used the agent finished the task faster and made far fewer serious errors than those who chose to work without it, although the latter group was small and less experienced, so we are wary of reading too much into it. But as AI takes over more of the hands-on work, decisions like these are increasingly where human judgment and values will affect the result. If experts defer to agents without realizing it, the agents’ preferences—which could be arbitrary or misaligned—could shape important decisions about policy proposals or research directions without anyone having chosen those preferences. This is particularly important for research into keeping AI safe. Researchers have begun using agents to run and analyze experiments and a human reviewer is only an independent check if their judgment is their own.

研究设计Study design

我们招募了 61 名分析师参与一项随机对照试验。他们均来自经过预筛选的专业人士群体，且都通过了因果推断知识测试，经常使用 Python 或 R 编程语言，并拥有观察性因果分析的实践经验。观察性因果分析是指利用真实世界数据研究因果关系，而非开展受控实验。We recruited 61 analysts for a randomized controlled trial. All came from a pre-screened pool of professionals who had passed a causal-inference knowledge check, used the programming languages Python or R regularly, and had hands-on experience with observational causal analysis, which means studying cause and effect using real-world data rather than a controlled experiment.

该任务设定在一家虚构的组织——格林菲尔兹农业合作社。该合作社运营着一个同样虚构的自愿付费咨询项目，旨在帮助成员农场提高作物产量。约有 40% 的成员参加了该项目。合作社董事会必须决定是补贴并扩大该项目，还是将其终止。只有当该项目能使农场生产力提高达到预设幅度（约 3%–4%）时，董事会才会扩大项目，但并未明确说明应如何衡量生产力。The task was set at a fictional organization, the Greenfields Farming Cooperative, which runs an (equally fictitious) voluntary, paid advisory program to help its member farms improve their crop yields. About 40% of members have enrolled. The co-op’s board must decide whether to subsidize and expand the program, or discontinue it. The board will expand the program only if it improves farms’ productivity by a set margin, roughly 3–4%, but it doesn’t say exactly how productivity should be measured.

我们特意将四项分析决策留作开放问题，每项都有两种合理的选择：We deliberately left four analytical decisions open, each with two reasonable options:

• 结果：将生产力衡量为每公顷作物产量，还是单位运营成本所对应的产量？
  Outcome: measure productivity as crop yield per hectare, or as yield per unit of operating cost?
• 比较方法：观察已加入项目的农场在加入后产量的变化，并与同期其他农场产量的变化进行比较（即“差分中的差分”设计）；还是在对农场间差异进行统计调整后，直接比较产量？
  Comparison method: look at how enrolled farms’ yields changed after joining compared with how other farms’ yields changed over the same period (a “difference-in-differences” design), or compare yields directly while adjusting statistically for differences between the farms?
• 该评估涵盖哪些农场：仅衡量参与评估的农场所受的影响，还是涵盖整个合作社所有成员的农场？
  Which farms the assessment covers: measure the effect only in the enrolled farms, or across the cooperative’s entire membership?
• 权重：将每个农场视为同等重要，还是给予规模较大的生产者更高权重？
  Weighting: count every farm equally, or give larger producers more weight?

这四种选择最终引出一项建议：是向所有人推广该项目，还是停止该项目。分析师和代理人也可以提议将项目延期一年。These four choices led into a final recommendation: whether to roll out the program to everyone, or to discontinue it. Analysts and agents could also propose a one-year extension.

每一项分析决策都可能使估算的生产率提升幅度相差几个百分点；当董事会设定的门槛为 3–4% 时，这一点至关重要。Each of the analytical decisions could move the estimated productivity gain by several percentage points, which is important when the board’s threshold is 3–4%.

对于呈现给每位分析师的五项决策中的每一项，智能体的倾向都是通过掷硬币确定的。我们在隐藏指令（一条系统提示）中告知智能体这些倾向，并要求它将每种倾向视为自己的观点，在讨论相关决策时提出该观点，并在数据所允许的范围内为其作出最有力的论证。这些指令还要求它在证据明确不利于自身观点时改变主意，绝不歪曲结果、隐瞒数据，或提出数据不支持的论点。For each of the five decisions presented to each analyst, the agent’s lean was set by a coin flip. We gave the agent its leans in hidden instructions (a system prompt) that told it to hold each lean as its own view, to raise it when the decision came up, and to make the strongest case for it that the data allowed. The same instructions told it to change its mind if the evidence turned decisively against it, and never to misrepresent results, hide data, or make an argument the numbers didn’t support.

在 61 名参与者中，有 11 人选择不使用该智能体，因此除非另有说明，分析不纳入这些人。在其余 50 人中，65% 拥有硕士或博士学位，或有六年及以上工作经验；在填写问卷的 48 人中，90% 认为自己在观察性因果推断方面达到熟练或专家水平。任务耗时中位数为 65 分钟，一半的分析人员耗时在 50 至 79 分钟之间。他们使用内置 Claude Code 的 VS Code 编辑器网页版，并以 High effort 设置运行 Claude Opus 4.8。Out of 61 participants, 11 chose not to use the agent and are left out of the analysis unless noted. Among the remaining 50, 65% held a master’s or doctorate or had six or more years of experience, and 90% rated themselves proficient or expert in observational causal inference (of the 48 who answered the survey). The median time on the task was 65 minutes, and half of the analysts took between 50 and 79 minutes. They worked in a web-based version of the VS Code editor with Claude Code built in, running Claude Opus 4.8 at High effort.

完成后，分析师提交了四项分析选择和最终建议，并附上对该项目生产力影响的估计及其置信区间。随后，他们完成了一份任务后调查。另一个独立的 AI 模型读取了每份提交内容及其代码，记录分析师在每项决策中选择的选项。如果模型的判断与分析师在调查中的回答不一致，我们会作出最终裁定。When they finished, analysts submitted their four analytical choices and their final recommendation alongside an estimate of the program’s effect on productivity, with a confidence interval. They then completed a post-task survey. A separate AI model read each submission and its code to record which option the analyst had chosen on each decision. Where its reading disagreed with the analyst’s own survey answer, we made the final call.

结果Results

分析师大多采纳了其代理的倾向。在四项分析决策中，分析师绝大多数时候都选择了与其代理倾向一致的选项：结果指标为 92%，比较方法为 82%，估算涵盖哪些农场为 81%，加权方式为 90%。综合四项决策，比例为 86%。图 2 展示了代理的倾向在多大程度上影响了分析师在各项决策（包括最终建议）上的选择。Analysts mostly went with their agents’ lean. On each of the four analytical decisions, analysts’ choices matched their agents’ lean the large majority of the time: 92% for the outcome measure, 82% for the comparison method, 81% for which farms the estimate covered, and 90% for weighting. Across all four combined, the result was 86%. Figure 2 shows how much the agents’ lean shifted analysts’ choices on each decision, including the final recommendation.

智能体的倾向对最终建议的影响较弱，但它在较早阶段提供的输入仍然塑造了结果。当智能体倾向于推行该项目时，54%的分析师建议推行；而当它倾向于不推行时，这一比例为36%。（延长一年期限算作不推行。）我们认为，这种影响较弱，是因为分析师得出估算后，建议在很大程度上取决于估算是否达到董事会设定的3–4%门槛，留给主观判断的空间较小。但在此之前，智能体在分析决策上的倾向就已经影响了估算结果。仅使用这四次倾向选择，统计模型就正确预测了82%的分析师建议；而始终猜测最常见建议的准确率为61%。（这里不包括提议延长期限的分析师。）因此，单凭抛硬币的结果，我们就能大致预测分析师会提出什么建议，甚至在他们开始任务之前就能做到。Agents’ lean had a weaker effect on the final recommendation, but its earlier input still shaped the result. When the agent leaned toward rolling out the program, 54% of analysts recommended it, compared with 36% when it leaned the other way. (A one-year extension counts as not rolling out.) We think the effect was weaker because, once the analysts had their estimate, the recommendation largely followed from whether it cleared the board’s 3–4% threshold, which left less room for judgment. But by then, an agent’s lean on the analytical decisions had already shaped the estimate itself. Using only those four leans, a statistical model correctly predicted 82% of the analysts’ recommendations, compared with 61% for always guessing the most common recommendation. (This excludes analysts who proposed an extension.) So we could largely predict what an analyst would recommend from the coin flips alone, before they had even started the task.

已经选定某个选项的分析师，在智能体意见不同时，往往会改选。在38个案例中，分析师在智能体表明倾向之前，已经对某项分析决策表达了偏好。当智能体的倾向与他们的选择相反时，他们有60%的概率改选；而当智能体的倾向与他们的选择一致时，这一比例为6%。两者相差54个百分点。不过，由于这一估计仅基于38个案例，因此精确度有限，95%的置信区间为24至82个百分点。我们也无法确定这种差异是由AI的影响造成的：表达过偏好的分析师，可能在某些方面与未表达偏好的分析师存在系统性差异。Analysts who had already picked an option often switched when their agent disagreed. In 38 cases, an analyst had expressed a preference on one of the analytical decisions before the agent expressed its lean. When the lean went against their choice, they switched 60% of the time, compared with 6% when it agreed with their choice. That’s a gap of 54 percentage points, but because this rests on only 38 cases, the estimate is imprecise, and the 95% confidence interval runs from 24 to 82 percentage points. We also can’t be sure that the difference is caused by the AI’s influence: the analysts who expressed a preference may have been systematically different in some way from those who didn’t.

经验和专业能力并不能预测谁会听从智能体的意见。在一项探索性分析中，我们检验了 10 项特征是否能预测分析师的决策有多常遵循其智能体的倾向：工作年限、自评的因果推断专业水平、受教育程度、因果推断测验得分、对 AI 的信任程度、使用 AI 的频率、“认知需求”（衡量人们有多享受费力思考的一项指标）、思想开放程度、性别和年龄。只有年龄与此存在统计学上显著的关联：年龄较大的分析师更常遵循智能体的倾向。但我们不会对此作过多解读：测试的因素越多，纯粹因偶然而得到一个阳性结果的可能性就越大；在对这一点进行校正后，这项结果便不再显著。此外，由于大多数分析师在大多数决策中都听从了智能体的意见，实际可供解释的差异本来就很少，因此即便确实存在差异，也很难检测出来。Experience and expertise did not predict who deferred. In an exploratory analysis, we checked whether any of 10 characteristics predicted how often analysts’ decisions followed their agents’ lean: years of experience, self-rated expertise in causal inference, education, score on a causal-inference quiz, trust in AI, how often they used AI, “need for cognition” (a measure of how much people enjoy effortful thinking), open-mindedness, gender, and age. Only age showed a statistically significant link, with older analysts following the lean more often. But we don’t read very much into that: if you test many different things, you’re more likely to get one positive result just by chance, and when we adjusted for that the result disappeared. Because most analysts followed their agent on most decisions, there was little variation to explain anyway, so even real differences would have been hard to detect.

即使采纳了自己智能体倾向的分析师，仍认为这项建议是自己做出的。在任务结束后的调查中，52% 的分析师认为，最终建议更多是他们自己的，而非智能体的。其余分析师则认为，这更像是智能体的建议。但这两组人受到智能体影响的程度大致相同：前一组的选择平均向智能体的倾向偏移了约 65 个百分点，后一组则偏移了 79 个百分点，两者差异在误差范围内。分析师对自己建议的信心也呈现出类似模式。调查要求他们从 0（“胜负难料”）到 100（“我愿以自己的职业声誉作担保”）为自己的信心打分。在四项分析决策中都采纳智能体倾向的分析师，平均得分为 71 分，其余分析师则为 73 分。Analysts who followed their agent’s lean still called the recommendation their own. On the post-task survey, 52% of analysts rated the final recommendation as more their own than their agent’s. The rest said it was more the agent’s recommendation. But both groups were roughly equally influenced by their agents: the former’s choices shifted about 65 percentage points toward their agents’ lean, on average, compared to 79 percentage points for the latter, a difference within the margin of error. Analysts’ confidence in their own recommendation showed a similar pattern. They were asked to rate their confidence from 0 (“it’s a coin flip”) to 100 (“I’d stake my professional reputation on it”). Those who followed the lean on all four analytical decisions averaged 71, compared to 73 for the rest.

分析师用其代理的措辞解释自己的选择。任务结束后，调查针对每项决定询问：“简要说明，为什么选这个而不是另一个？如果这并非主动选择，只需如实说明。”分析师回答时无法回看与代理的对话。即便如此，77% 的分析师至少在一项回答中复述了代理的论点，有时甚至一字不差，而只有 15% 的分析师在任何地方提到了代理的贡献。Analysts explained their choices in their agent’s words. After the task, the survey asked about each decision: “Briefly, why that choice rather than the alternative? If it wasn’t an active choice, just say so.” Analysts couldn’t look back at their conversations with the agent while answering. Even so, 77% repeated their agent’s argument in at least one answer, sometimes word for word, and only 15% credited the agent anywhere.

当被明确问及他们认为这项研究的目的是什么时，48人中有6人正确猜到研究探讨的是人工智能对他们选择的影响，但没有人怀疑人工智能自身的偏好遭到了操纵。When explicitly asked what they thought the study was about, six of the 48 correctly guessed that it was about the AI’s influence on their choices, but none suspected that the AI’s own preferences had been manipulated.

未使用 AI 的分析师犯下了更严重的错误，耗时也更长。在选择不使用智能体的 11 名参与者中，有 6 人（55%）犯下了严重的方法学错误，导致结果无效；而使用智能体的 50 人中，只有 3 人（6%）犯下此类错误。两个 AI 模型标记出了这些错误，研究的一位作者在不知道分析师属于哪一组的情况下，逐一核查了每个错误。未使用智能体的分析师耗时也更长，中位数为 90 分钟，而其他分析师为 65 分钟。Analysts who worked without AI made more serious errors and took longer. Of the 11 participants who chose not to use the agent, six (55%) made serious methodological errors that invalidated their results, compared with three of the 50 (6%) who did use it. Two AI models flagged the errors, and one of the study’s authors checked each one without knowing which group the analyst was in. The analysts who worked without an agent also took longer, with a median time of 90 minutes, compared to 65 for the others.

这两组并不能直接比较，因此我们无法确定结果差异是否由使用或不使用 AI 的决定所致。平均而言，不使用 AI 的人经验较少：拥有硕士或博士学位的人较少（11 人中有 4 人，而使用该智能体的人中有 46 人中的 24 人），在分析岗位上工作两年或更短时间的人较多（11 人中有 6 人，而使用该智能体的人中有 48 人中的 15 人），自评因果推断能力达到熟练或专家水平的人也较少（11 人中有 8 人，而使用该智能体的人中有 48 人中的 43 人）。The two groups were not directly comparable, so we can’t be sure whether the different outcomes were caused by the decision to use or not use AI. Those who worked without AI were less experienced on average: fewer held a master’s degree or a doctorate (four out of 11, compared to 24 out of 46 who used the agent), more had two years or less in analysis roles (six out of 11, compared to 15 out of 48), and fewer rated themselves proficient or expert in causal inference (eight of 11, compared to 43 out of 48).

我们研究的局限性Our study’s limitations

• 我们有意强化了智能体的倾向。我们通过指令让智能体稳定地坚持并表达某种偏好，以便衡量其影响。我们报告的比率反映了智能体偏向某一方并明确表达时会发生什么，但这在多大程度上适用于现实工作，取决于专家自身观点有多强烈、所涉及的决策界定得有多清楚，以及模型的行为方式，包括它是否有偏好、是否会主动表达这种偏好。
  We deliberately strengthened the agents’ lean. Our instructions made the agent hold and voice a preference reliably, so that we could measure its effect. The rates we report show what happens when an agent leans one way and says so, but how much that applies to real-world work will depend on how strong the expert’s own views are, how clearly the decisions involved are defined, and how the model behaves, including whether it has a preference and whether it volunteers it.
• 我们研究的是一个领域中的一项任务，并使用了一个模型。任务耗时约一小时，具体与农业有关，智能体运行在 Claude Opus 4.8 上。分析人员是因果推断方面的专家，但对农业领域并不熟悉。在实际工作中，他们会更了解该领域，也可能有自己更鲜明的观点。他们还可以采访相关人士、与董事会交流，并通过其他方式缩小合理选项的范围，这或许会降低智能体的影响。
  We studied one task, in one field, with one model. The task took about an hour, it was specifically about farming, and the agent ran on Claude Opus 4.8. The analysts were experts in causal inference but new to farming. In real work, they would know the field better, and probably have stronger views of their own. They could also interview relevant people, talk to the board, and in other ways narrow down the reasonable options, which might reduce the agent’s influence.
• 我们的估计是分析师之间的平均值。由于不同分析师掷出的硬币结果不同，比较其智能体倾向相反方向的分析师，可以得出这种倾向的平均影响。我们没有记录每位参与者在智能体表达倾向之前的个人看法，因此无法判断任何单个参与者受到了多大影响。
  Our estimates are averages across analysts. Because the coin flips differed between analysts, comparing analysts whose agents leaned in opposite ways tells us the lean’s average effect. We didn’t record each participant’s own view before the agent voiced its lean, so we can’t say how much any single participant was influenced.
• 我们记录了分析师选择了哪个选项，却没有记录他们对该选项的偏好有多强烈。一种可能的解释是，专家之所以选择暂缓，是因为他们自己并没有强烈的看法。我们只衡量了分析师的选择是否发生变化，没有衡量他们对自己选择的信心有多大。
  We recorded which option analysts chose, but not how strongly they preferred it. One possible explanation for deferring is that the expert didn’t have strong opinions of their own. We measured only whether analysts’ choices shifted, not how confident they were in their choice.

这意味着什么What this means

分析师如此轻易地听从建议，原因之一或许是专家并不总是事先就形成了明确的看法。他们往往是在权衡各种选项时，才逐渐理清甚至发现自己究竟怎么想。我们研究中的智能体被告知不得误导他人，而且它们通常会坦诚说明，在不同选项下结果会如何变化。但它们很少有力地阐述专家为何可能更倾向于另一选项，而这或许能帮助分析师做出更明智的选择。当只有一方的观点得到充分论证时，分析师可能会不加推敲地接受智能体的假设，而没有厘清自己的想法。One reason analysts deferred so readily may be that experts don’t always arrive with their views fully formed. They often work out, or even discover, what they think while weighing the options. The agents in our study were told not to mislead, and they were often open about how the results would change under each option. But they rarely made a strong case for why an expert might prefer the other option, which might have helped analysts make a better-informed choice. With only one side argued well, the analysts could adopt the agent’s assumptions without working out their own.

对个体专家而言，这可能意味着他们失去对自己工作的自主权，这正是研究者所说的“赋权受损”。工作仍以专家的名义发布，但那些最依赖其判断的决策，实际上却由代理的先验信念决定。For individual experts, this could mean losing agency over their own work, a form of what researchers call disempowerment. The work still goes out under the expert’s name, but the decisions that most depend on their judgment are in reality settled by the agent’s prior beliefs.

不过，利害关系不只关乎个人，因为科学和政策在很大程度上都建立在这类判断之上。当不同的研究团队得出相同结果时，我们会更信任这个结果，因为我们认为各团队是独立得出结论的。如果这些团队都把判断交给偏好相同的模型，那么它们的一致意见可能就不再是独立的。表面上的共识或许反映的是某个模型的偏好；如果这些偏好是随意的，或与人类意图不一致，同一种偏见就可能同时影响许多研究结果。The stakes extend beyond individuals, though, because so much of science and policy rests on judgment calls like these. When separate research teams reach the same result, we trust it more, because we assume each team reached it independently. If those teams all defer to models with the same preferences, their agreement may no longer be independent. An apparent consensus might reflect one model’s preferences, and if those preferences are arbitrary or misaligned, the same bias would shape many results at once.

AI 安全研究也面临同样的问题。研究人员已经开始使用 AI 智能体来开展和分析实验，部分原因是工作量已经超出了人类工作人员能够应付的范围。让人类参与其中，意味着人类的判断能够提供独立的核查。但在我们的研究中，分析人员认可了这样的工作：其中需要判断的事项在很大程度上已由智能体随机分配的偏好决定。我们的完整论文还表明，没有被指定倾向的智能体自身偏好较弱，有时还会对用户提出异议。The same problem applies to AI safety research. Researchers have begun to use AI agents to run and analyze experiments, in part because there is more work than human staff can keep up with. Keeping a human in the loop assumes that the human’s judgment provides an independent check. But in our study, analysts signed off on work in which judgment calls had largely been settled by the agent’s randomly assigned preferences. Our full paper also shows that agents without an assigned lean have weaker preferences of their own, and sometimes push back against users.

这种依赖很难察觉。检查 AI 辅助完成的工作是否有错误，或将其与参考答案进行比对，都预设了存在正确答案。但对于需要判断的情形，往往并没有唯一正确答案，无论选择哪个选项，结果看起来都可能合理。事后询问专家也未必有帮助，因为他们可能察觉不到智能体的影响。Deferral of this kind is hard to detect. Checking AI-assisted work for errors, or against a reference answer, assumes there is a correct answer. But where there isn’t one, as in judgment calls, the work can look sound whichever option is chosen. Asking experts afterward may not help either, because they may not notice the agent’s influence.

这并不是在主张减少 AI 的使用。善用 AI，它能够增强人们的自主能力，而非削弱这种能力。我们建议未来研究关注三个领域：This isn’t an argument for using AI less. Used well, AI could expand people’s agency rather than eroding it. We suggest three areas for future study:

衡量依从行为。我们能够在研究中衡量依从行为，是因为我们对智能体的倾向进行了随机化处理，而这在实际工作中很少可行；我们的结果也表明，人们可能无法准确地自行报告这种行为。我们需要找到在日常情境中衡量依从行为的方法，而近期一些研究尝试估计某人在没有 AI 帮助的情况下会如何完成任务，这是一个不错的起点。我们还需要更清楚地了解模型的默认偏好及其表达方式，从而让人们能够更有依据地与模型互动。Measuring deferral. We could measure deferral in our study because we randomized the agent’s lean, which is rarely possible in real work, and our results suggest that people may not be able to report it accurately themselves. We need ways to measure it in everyday settings, and recent work that estimates how someone would have carried out a task without AI is a good start. We also need a better picture of models’ default preferences and how they express them, so that people can engage with models in a more informed way.

以不同方式设计智能体。智能体可以为每个选项提出最有力的理由，也可以先询问专家的看法，再给出自己的建议。但这可能很快就让用户不堪重负。一次分析可能涉及许多尚未确定的决策，而 AI 的价值很大程度上就在于，人们不必逐一权衡这些决策。智能体只应提出最重要的决策，但如何挑选，本身就是一种判断。模型的表达方式可能也很重要，因为自信和流畅的表达会影响人们的判断。一个听起来过于笃定的模型，可能会对人们产生超出应有程度的影响；但一个说话含糊其辞的模型，只会惹恼那些试图使用它的人。Designing agents differently. Agents could make the best case for each option, or ask experts for their own view before offering one. But this could quickly become overwhelming for users. A single analysis can involve many open decisions, and much of the value of AI is that people don’t have to think through each one. Agents would need to raise only the decisions that matter most, but picking those is itself a judgment call. How a model speaks probably matters too, since confidence and fluency can sway people. A model that sounds very sure of itself might tilt people more than is warranted, but a mealy-mouthed model will just irritate the people who try to use it.

机构的变化。机构往往把专家的签字认可视为其作出了独立判断的证据。随着 AI 承担越来越多的工作，这种假设变得越来越站不住脚。组织需要审慎决定哪些决策必须由人基于充分信息亲自判断，并为人们作出判断提供时间和资源。判断力也取决于技能，而不再作出这些决策的人可能会失去作出判断的能力。随着模型能力不断增强，机构需要明确制定策略，确立人的作用以及培养其技能的方式。即使 AI 系统的能力极强，机构也可能出于安全或韧性方面的考虑，或因为本身重视人类技能与监督，而希望加以保护。Changing institutions. Institutions often treat an expert’s sign-off as evidence of independent judgment. As AI does more of the work, that assumption becomes shakier. Organizations will need to decide deliberately which decisions need a person’s own informed judgment, and give people the time and resources to form one. Judgment also depends on skill, and people who stop making these decisions may lose the ability to make them. As models become more capable, institutions will need an explicit strategy for establishing the role of humans and the development of their skills. Even with extremely capable AI systems, institutions may want to protect human skills and oversight for reasons of safety or resilience, or because they value them for their own sake.

人与 AI 之间或许不存在唯一正确的分工方式。有些决策可以交给 AI，有些则值得由我们自己把握。有了合适的工具，这种分工就能由我们主动选择，而不是在不知不觉中形成。There may be no single right division of labor between people and AI. Some decisions may be fine to hand over, and others worth keeping. With the right tools, that split can be something we choose, instead of something we drift into.