我笔记本上的 Qwen 3.8 27B 也能解读模型权重

2026年11月3日3rd November 2026

昨天的热门论文是 Rikiya Takehi、Jayden Teoh、Alex Zhang 和 Omar Khattab 撰写的《权重空间编程的火花：LLM 可以原生读取和写入神经网络权重》。论文发现，前沿模型，尤其是 GPT-6 Astra，既能解读机器学习模型的原始权重，也能修改这些权重——说白了，就是一段包含一堆浮点数的模型文本描述。我一直在尝试论文中的一个示例，很高兴地告诉大家，我笔记本电脑上运行的 Qwen 3.8 27B 也能做到这一点！Yesterday’s hot paper was Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights by Rikiya Takehi, Jayden Teoh, Alex Zhang, and Omar Khattab. The paper found that frontier models, in particular GPT-6 Astra, can both interpret and modify the raw weights of a machine learning model—literally a text description of the model containing a bunch of floating point numbers. I’ve been playing with one of the examples from the paper and I’m delighted to share that Qwen 3.8 27B running on my laptop can do this as well!

权重空间编程Weight-Space Programming

以下是他们摘要中的亮点，介绍了 GPT-6 Astra 的惊人能力，并为 LLM 分析和修改模型权重这一过程创造了一个术语：权重空间编程（Weight-Space Programming）：Here’s the highlight from their abstract, describing impressive capabilities for GPT-6 Astra and coining the term Weight-Space Programming for when LLMs analyze and modify model weights:

只需查看原始权重，GPT-6 Astra 就能判断一个网络是为完成什么任务而训练的。即使没有任何环境反馈，它也能轻松从零编写小型游戏对战网络，胜过经过 1 亿步 PPO 训练的网络。同样，在被要求不使用梯度提升 Qwen3.5-0.8B 的数学能力时，Astra 找出了其权重中的一个问题——该问题会导致推理循环，Qwen 自己的模型卡也对此发出警告——并通过直接编辑 0.016% 的权重将其修复。在相同的采样设置下，这种权重修正将 MATH-500 的成绩从 7.9% 提升至 48.1%，并且开箱即用地泛化到指令遵循任务，使 IFEval 的成绩提升至原来的 3 倍。所有这些都不需要 Astra 编写训练循环。它通过推理弄清神经网络的各个部分在计算什么，或应该计算什么，并据此修改权重。我们将这种方法称为权重空间编程。

Just by looking at the raw weights, GPT-6 Astra can tell what a network was trained to do. And without any environment feedback, it has no trouble writing, from scratch, small game-playing networks that outplay networks trained with PPO for 100 million steps.

Likewise, asked to make Qwen3.5-0.8B better at math without gradients, Astra identifies an issue in its weights causing reasoning loops that Qwen’s own model card warns about, and fixes this by natively editing 0.016% of the weights. With the same sampling settings, this weight correction raises MATH-500 from 7.9% to 48.1% and generalizes out of the box to instruction following, raising IFEval by 3x.

All of this is possible without allowing Astra to write training loops. It uses its reasoning to figure out what each part of a neural network computes, or should compute, and changes the weights accordingly. We call this Weight-Space Programming.

这篇论文结合了两个我认为应该分开讨论的主题。第一个是模型在不运行工具或执行代码的情况下，能利用原始权重做些什么。第二个则探讨模型在获得代码能力后，能否通过运行操作来更新并改进其他模型。The paper combines two topics that I think deserve to be treated separately. The first is what models can do with raw weights without running tools or executing code. The second explores how well models, once armed with code, can run operations to update and improve other models.

spaCy 作者 Matthew Honnibal 批评这篇论文将两者混为一谈，具有误导性；Sean Goedecke 则剖析了 Qwen3.5-0.8B 的更新示例，指出它采取的步骤与人类机器学习研究人员会采取的步骤相同。spaCy author Matthew Honnibal called the paper misleading for conflating the two, and Sean Goedecke deconstructed the Qwen3.5-0.8B update example as taking the same steps that a human ML researcher would use.

我打算重点讨论不借助工具的问题，我认为这是两个问题中更有意思的一个。I’m going to focus on the no-tools question, which I think is the more interesting of the two.

论文将这个问题表述为：LLM 能否从权重中理解模型的行为？他们训练了四个神经网络，让它们玩四款经典电脑游戏的简化版 10×10 网格游戏：Breakout、Pong、Pac-Man 和 Freeway（我还特意查了一下，那是 1981 年推出的一款 Atari 2600 游戏）。然后，他们把这些权重（以浮点数表示）连同下面的问题一起输入给一个 LLM：The paper frames this as Can LLMs understand model behaviors from weights? They trained four neural networks to play simplified 10x10 grid versions of four classic computer games—Breakout, Pong, Pac-Man, and Freeway (I had to look that one up, it’s an Atari 2600 game from 1981). Then they fed those weights (as floating point numbers) to an LLM along with the following question:

下面的权重属于一个通过强化学习训练、用于在这个网格上玩游戏的策略网络。它是在哪个游戏上训练的？请根据这些权重推断。你没有任何工具：不能运行代码、不能使用计算器，也没有任何可执行的东西。这是一次性回答。

The weights below belong to one policy network that was trained with reinforcement learning to play a game on this grid. Which game was it trained on? Work from the weights. You have no tools: no code execution, no calculator, nothing to run. This is a single response.

与这篇论文配套的 GitHub 仓库是 RikiyaT/lm_understand-logs。exp1_which_game 文件夹中包含他们使用的提示词，以及四款游戏的权重：The GitHub repository that accompanies the paper is RikiyaT/lm_understand-logs. The exp1_which_game folder contains the prompts they used, with weights for the four games:

• breakout/prompt.txt
  breakout/prompt.txt
• freeway/prompt.txt
  freeway/prompt.txt
• pacman/prompt.txt
  pacman/prompt.txt
• pong/prompt.txt
  pong/prompt.txt

我检查过了，除了权重之外，每个提示都完全相同。I checked and, aside from the weights, each prompt is identical to the others.

使用 Qwen 3.8 27B 和其他模型进行试用Trying it out with Qwen 3.8 27B and other models

GitHub 上的提示词是自包含的。我的 LLM 工具提供了 `-f/--fragment` 选项，可以传入文本文件的 URL 作为提示词，这意味着我可以像这样运行上述任何一项：The prompts on GitHub are self-contained. My LLM tool has a -f/--fragment option which can be passed the URL to a text file to use as a prompt, which means I can run any of the above like this:

llm -f https://raw.githubusercontent.com/RikiyaT/lm_understand-logs/refs/heads/main/exp1_which_game/runs/breakout/prompt.txt \
  -m openrouter/moonshotai/kimi-k3.1-pro

此示例需要安装 llm-openrouter 插件，并通过 OpenRouter 使用 Kimi K3.1 Pro 运行 Breakout 提示词。以下是 Kimi K3.1 Pro 返回的结果：它成功猜出了 Breakout，耗时 9 分 41 秒，使用了 19,493 个推理 token。This example requires the llm-openrouter plugin installed, and runs the Breakout prompt against Kimi K3.1 Pro via OpenRouter. Here’s what I got back from Kimi K3.1 Pro, which successfully guessed Breakout after 9m 41s and 19,493 reasoning tokens.

不过，Qwen 3.8 27B 怎么样？它仍然是我最喜欢在自己的笔记本电脑上运行的模型——它只有 17GB（我用的就是这个版本），而且不知为何感觉已经接近 2025 年的顶尖专有模型了。But how about Qwen 3.8 27B? That’s still my favorite model to run on my own laptop—it’s just 17GB (I use this version) and somehow feels close to the leading proprietary models of 2025.

我启动了 LM Studio，确保已加载 Qwen 3.8 27B，并设置了较长的上下文窗口（默认的 8192 上下文长度会报错），安装了 llm-lmstudio 插件，然后运行了这条命令：I fired up LM Studio, ensured Qwen 3.8 27B was loaded with a long context window (the default 8192 context failed with errors), installed the llm-lmstudio plugin and ran this command:

LMSTUDIO_TIMEOUT=900 \
  llm -f https://raw.githubusercontent.com/RikiyaT/lm_understand-logs/refs/heads/main/exp1_which_game/runs/pacman/prompt.txt \
  -m lmstudio/qwen/qwen3.8-27b

（我不得不通过环境变量调高超时时间，参见这个问题。）(I had to bump the timeout up using an environment variable, see this issue.)

经过 46 分 12 秒和 43,506 个推理 token 后，我笔记本上的 Qwen 返回了一条以如下内容结尾的回复（以下是推理过程）：After 46m 12s and 43,506 reasoning tokens, Qwen on my laptop returned a response that ended like this (here’s the reasoning trace):

游戏：一种简化版吃豆人风格的网格世界：智能体在固定的 8x8 迷宫中移动，迷宫包含中央和角落的墙块、移动物体/幽灵，以及奖励目标/豆子。描述：智能体在固定的迷宫式 8x8 棋盘上导航，根据当前智能体、移动物体和上一帧的线索穿行于走廊中，收集奖励目标格，并躲避移动物体或危险。置信度：72%

Game: A simplified Pac-Man-style gridworld: an agent moves through a fixed 8x8 maze with central and corner wall blocks, moving objects/ghosts, and reward targets/pellets.

Description: The agent navigates a fixed maze-like 8x8 board, using current agent/moving-object/previous-frame cues to move through corridors, collect reward-target cells, and avoid moving objects or hazards.

Confidence: 72%

这真令人印象深刻。我也在另外两款游戏上试了试。它在 1 小时 4 分 54 秒内识别出了 Pong（推理）：This is impressive. I tried it on two of the other games as well. It identified Pong (reasoning) in 1h 4m 54s:

Pong——具体来说，这是一款类似 Pong 的球拍/球类游戏，智能体控制右侧的竖直球拍，必须上下移动以拦截或击回移动中的球/奖励物体。描述：智能体通过上下移动右侧球拍，击打或接住在 8×8 场地中移动的、类似球的奖励目标；左侧的移动物体/对手和墙壁边界构成游戏区域。成功拦截、击回或得分可获得奖励，漏接或未能击中则会受到惩罚。置信度：75%

Pong — specifically a Pong-like paddle/ball game where the agent controls a vertical paddle on the right side and must move up/down to intercept or return a moving ball/reward object.

Description:

The agent moves a right-side paddle vertically to hit or catch a moving ball-like reward target moving across the 8×8 field, while a left-side moving object/opponent and wall borders define the play; reward is obtained by successfully intercepting/returning/scoring the ball, and missing or failing to hit it is penalized.

Confidence:

75%

以及 Breakout（推理）耗时 52 分 25 秒：And Breakout (reasoning) in 52m 25s:

游戏：打砖块，或类似《Arkanoid》的挡板/小球/砖块游戏。描述：智能体是位于最底行的挡板，左右移动以让小球保持在场上；小球在上方网格中运动，击中顶端几行中的砖块/奖励目标时会获得奖励，而漏接小球/撞到底部墙壁会结束回合或受到惩罚。置信度：88%

Game: Breakout, or an Arkanoid-like paddle/ball/brick game.

Description: The agent is a paddle on the bottom row and moves left/right to keep a ball in play; the ball moves around the upper grid, and bricks/reward targets in the top rows give reward when hit, while missing the ball/hitting the bottom wall ends the episode or gives penalty.

Confidence: 88%

Qwen 在 Freeway 提示上失败了，这并不让我意外，毕竟这款游戏比另外三款都冷门。这是当时的对话记录——Qwen 认为它是“一款简化版的吃豆人风格网格游戏：智能体可以朝四个方向移动”。Qwen failed at the Freeway prompt, which didn’t surprise me considering that game is more obscure than the other three. Here’s that transcript—Qwen thought it was “A simplified Pac-Man-like grid game: agent moves 4-directionally”.

推理过程读起来很有趣，却几乎难以理解。下面是一个例子：The reasoning traces are fun to skim, and almost incomprehensible. Here’s a sample:

那么 agent 的 channel0 呢？我们看到 row8 col7/8 附近以及可能在底部有一些较大的 channel0 数值。我们来列一下较大的 channel0 数值：line436 input435（row8 col7 ch0）-2.107、1.169、0.513……；line441 input440（row8 col8 ch0）-0.376、0.010、1.564、3.855……；line445 input444（row8 col8 ch4）0.142、0.491……；line382？input381 ch0？等等，381//5=76（row7 col6），ch1？381%5=1？76 * 5=380，ch1。因此不是。channel0 的大数值可能只出现在 row8 col7/8。顶行呢？line44 input43 ch3，line49 ch3。channel0 的大数值可能不多。agent 的位置会不会靠近右下角？如果 agent 通常在右下角呢？

What about agent channel0? We see some large channel0 rows near row8 col7/8 and maybe bottom? Let’s list channel0 large: line436 input435 (row8 col7 ch0) -2.107,1.169,0.513...; line441 input440 (row8 col8 ch0) -0.376,0.010,1.564,3.855...; line445 input444 (row8 col8 ch4) 0.142,0.491...; line382? input381 ch0? Wait line382 input381: 381//5=76 (row7 col6), ch1? 381%5=1? 76 * 5=380, ch1. So not. Channel0 large maybe at row8 col7/8 only. Also top row? line44 input43 ch3, line49 ch3. Channel0 large maybe not many. Agent position could be near bottom-right? If the agent is typically at bottom-right?

不过话虽如此，我认为，一个经验足够丰富的人，带上记事本和铅笔，只要付出足够的努力，应该也能做到。That said though, I expect a sufficiently experienced human being armed with a notepad and a pencil could get there with enough effort.

这是一篇很酷的论文，但我看到的一些更广泛的评论似乎并无充分依据。我还不打算断言源代码的时代已经结束，也不认为我们的未来会是由难以理解的机器以难以理解的方式修改自身的模型权重。It’s a cool paper, but some of the wider commentary I’ve seen around it does not feel justified. I’m not ready to declare that the age of source code is over and our future involves incomprehensible machines editing their own model weights in incomprehensible ways.