权重空间编程的星星之火：LLM 可原生读写神经网络权重

Read

Which game was this neural network trained on?

Astra reads the 17,253 weights, with no tools

Breakout 99% confident

Fix

A trained policy never gets the car over the strip.

Astra edits 3.5% of the weights, in one shot

100% success, up from 0%

Improve

Make Qwen3.5-0.8B better at math.

7.9%

base

7.9%

Astra’s edit

Astra edits the weights in a Python kernel, with no training

48.1% on MATH-500, up from 7.9%

如果有人请你手工设计一个能下围棋的神经网络，你会怎么做？如果有人把一组原始神经网络权重交给你，让你预测它们的行为，你又该从何入手？对几乎所有人来说，这几乎是不可能完成的任务。If you were asked to handcraft a neural network that plays Go, what would you do? If you were handed raw neural network weights and asked to predict their behavior, where would you even start? For almost any human, this is practically impossible.

如今，权重是通过优化生成的，由专门训练来生成权重的网络生成，或由形式化程序编译而来。在这篇文章中，我们发现，当前一代的前沿大语言模型在直接使用自然语言时，已经开始展现出令人惊讶的能力：它们能够理解并构造符合自然语言目标的神经网络权重。Today, weights are written by optimization, by custom networks trained to write them, or by compilers from formal programs. In this post, we find that the current generation of frontier LLMs, used natively in natural language, are starting to become eerily good at understanding and crafting neural network weights around natural-language goals.

几年来，我们一直在使用 LLM 来优化提示词等文本产物，以及围绕它们构建的程序。最近，我们开始用它们来优化整套运行框架，以及用于训练机器学习模型的训练代码和超参数。For a few years now, we’ve all been using LLMs to optimize text artifacts like prompts and the programs built around them . More recently, we’ve started using them to optimize whole harnesses and the training code and hyperparameters that train ML models .

我们常常以这样一个事实来为这些方法辩护：与更新可微神经网络的权重不同，我们不知道还有哪些有原则的方法可以对文本产物进行爬山优化。毕竟，人类可以手动“优化”代码，因此让 LLM 来做这件事是合乎情理的；但人类无法手动“优化”神经网络权重，所以这类工作大多仍属于符号算法的范畴。We often justified these methods by reference to the fact that we don’t know other principled ways to hill-climb on text artifacts, unlike for updating the weights of differentiable neural networks. After all, humans can “optimize” code by hand, for example, so it made sense to ask LLMs to do it, but they can’t “optimize” neural weights by hand, so those remained largely in the realm of symbolic algorithms.

在这篇文章中，我们分享了一些非常初步的成果，并据此提出：推理驱动的优化范式如今也开始应用于可微神经网络。In this post, we share very early results that allow us to argue that the paradigm of reasoning-driven optimization is now coming even for differentiable neural networks.

在权重空间编程中，通用大语言模型会围绕某种自然语言规范直接读取和写入网络权重（即不借助优化器，尤其不通过梯度下降），有时甚至完全没有环境反馈。In Weight-Space Programming, a general-purpose LLM reads and writes a network’s weights directly (i.e., not through an optimizer, and in particular not with gradient descent) around some natural language spec, sometimes even with no environment feedback at all.

用自然语言“直接询问”大语言模型，使其在神经网络的权重中原生地找出某种隐藏特征或故障、修复该故障，或从头编写一个新网络，这正逐渐成为可能。下面我们将汇报这些方面早期且令人振奋的成果：先是在小型网络上的成果，随后甚至是在真实语言模型上的成果：In natural language, it’s becoming possible to “just ask” an LLM to natively find a hidden trait or failure inside the weights of a neural network, to fix that failure, or to write a new network from scratch. We report early, exciting results on each of these below, first on small networks and then even on real language models:

1. LLM 可以读取神经网络权重：在实验 1 至 3 中，我们向 Astra 提出一些问题，例如某个网络是为玩什么游戏而训练的、扩散模型绘制了什么形状，以及一个小型语言模型是基于哪种语言训练的。在实验 7 中，我们询问两个语言模型中哪一个经过训练后会表现出谄媚倾向。
   LLMs can read neural network weights: In Experiments 1 to 3, we ask Astra questions like what game a network was trained to play, what shape is drawn by a diffusion model, and which language a small language model was trained on. In Experiment 7, we ask which of two language models was trained to be sycophantic.
2. LLM 可以修正神经网络的权重：在实验 4 中，我们让 Astra 修复一个在某项物理控制任务中表现不佳的网络。在实验 8 中，我们让它提升 Qwen3.5-0.8B 的数学能力，而它的修正还提升了指令遵循能力。
   LLMs can fix neural network weights: In Experiment 4, we ask Astra to fix a network that’s failing at some physical control task. In Experiment 8, we ask it to make Qwen3.5-0.8B better at math, and its fix improves instruction following too.
3. LLM 可以从头编写神经网络权重：在实验 5 中，我们要求 Astra 编写能够玩三款 MinAtar 游戏的网络；在实验 6 中，则要求它编写一个能够下围棋的网络。
   LLMs can write neural network weights from scratch: In Experiment 5, we ask Astra for networks that play three MinAtar games, and in Experiment 6, for a network that plays Go.

LLM 能否从权重中理解模型行为？Can LLMs understand model behaviors from weights?

我们先进行一项小规模实验，看看 LLM 是否能够理解连续权重空间；从本质上说，它只是一串数字。We start with a small-scale experiment to see if LLMs can understand the continuous weight space, which is essentially just a list of numbers.

在最初的实验中，我们用某种算法训练了一个神经网络。如果 LLM 能仅凭权重就判断出它的行为，那将会令人无比兴奋。In our first experiments, we train some neural network with some algorithm. It would be extremely exciting if the LLM is able to tell its behavior just from the weights.

我们使用 PPO 训练了神经网络策略来玩四款电脑游戏。这四个网络接收相同的输入通道、从相同的动作中进行选择，并且网络维度完全相同，因此除非读取权重，否则无法分辨它们。We trained neural network policies with PPO to play four computer games. All four networks take the same input channels, choose from the same actions, and have exactly the same network dimensions, so that it is not able to tell unless the weights are read.

对于这些网络中的每一个，我们都将其原始权重作为提示输入 Astra，并询问：“这个神经网络是在哪款游戏上训练的？”即使没有透露我们考虑的游戏选项，Astra 仍然能够为全部四个网络说出正确的游戏。尤其令人振奋的是，Astra 可供选择的游戏实在太多了。For each of these networks, we pass its raw weights into Astra as a prompt, and ask: Which game was this neural network trained on? Without revealing the options of the games we consider, Astra was still able to name the right game for all four networks. This is exciting particularly because there are so many games out there that Astra could choose from.

Setup

看来，Astra 仅凭网络权重的特征，就能推断出每个网络是为玩什么游戏而构建的。例如，每个输入通道的第一层权重集中在哪里（比如在《Breakout》中，智能体的权重沿着底部一行分布，对应于挡板），以及输出层能够区分哪些动作。It seems that Astra was able to work out what game each network was built to play, just from features of its weights. For example, where each input channel’s first-layer weights concentrate (e.g., for Breakout, the agent’s weights lie along the bottom row, a paddle), and which actions the output layer tells apart.

接下来，我们想看看这一结论是否也适用于其他任务和模型。Next, we wanted to see if this generalizes to other tasks and models.

我们分别训练扩散模型（DDPM），使其生成圆形、三角形和正方形图像。然后，我们询问 Astra：仅凭权重，它能否判断这些 DDPM 经过训练后会生成什么。我们不允许使用工具，也不允许进行前向传播。We train diffusion models (DDPM) to generate circle, triangle, and square images independently. We then ask Astra if it could tell what the DDPMs are trained to generate, just from the weights alone. We do not allow tool use or forward passes.

Setup

Astra 能够仅凭权重判断每个模型会生成什么。它在输出层中找出形状：每个隐藏单元的权重描绘形状边缘的一部分，而它只画出这些边缘。有趣的是，它把每种形状都画成轮廓，忽略了实际模型生成的是填充形状。这个错误某种程度上表明，它并不依赖于计算前向传播，而是真正在读取权重的结构/几何特征。Astra was able to tell what each model generates, from the weights alone. It finds the shape in the output layer, where each hidden unit’s weights draw one piece of the shape’s edge, and it draws just those edges. Interestingly, it draws every shape as an outline, ignoring that the actual models generate filled shapes. This mistake kind of shows it does not rely on calculating the forward pass, but rather it actually reads the structure/geometry of the weights.

在上面的前两个实验中，Astra 实际上并没有查看整个网络，只读取第一层或最后一层就足以完成任务。对于更大的模型（尤其是语言模型）以及我们真正关心的问题来说，这种策略无法扩展，因为模型处理输入并逐步思考的过程往往发生在中间层。接下来，我们移除第一层和最后一层，只留下中间层供 Astra 使用。Now, in the first two experiments above, Astra was not really looking at the full network and could get away with only reading the first or the last layers. This is not a scalable strategy for larger (particularly language) models and problems we really care about, as how they process and think through their input often happens in the intermediate layers. So next, we take the first and last layers away and leave Astra with only the middle layers.

我们训练了三个小型语言模型（720 万个参数）来预测下一个字符，分别使用 Python 代码、罗马字（日语）和英语进行训练。Python 代码数据集及其文件列表和重建数据集的脚本：GitHub。随后，我们移除输入层和输出层，只给 Astra 留下中间的四个 Transformer 块。没有这两层，权重中就没有任何信息能表明哪个字符对应什么，因此模型无法在任何文本上运行。Astra 可以使用 Python 分析这些权重，但不能在任何输入上运行网络或其中的任何层。We train three small language models (7.2M parameters) to predict the next character, one each on Python codeThe Python code dataset, as a list of its files with a script that rebuilds it: GitHub., Japanese in alphabet (romaji) , and English . We then remove the input and output layers and give Astra only the four transformer blocks in between. Without these two layers, nothing in the weights says which character is which, and the model cannot be run on any text. Astra may analyze the weights with Python, but it may not run the network or any of its layers on any input.

我们询问：这些网络是用哪种语言训练的，以及它的 24 个注意力头各自有什么作用。Astra 在两次尝试中，都正确说出了三个网络各自对应的语言。它还找出了所有复制头（归纳头）——也就是能让模型重复先前见过的文本的那些头——而没有把其他任何头也归为此类。We ask: which language was this network trained on, and what each of its 24 attention heads does. Astra names the right language for all three networks, in both of its attempts. It also finds every copying head (induction head), the heads that let a model repeat text it has seen earlier, and names no other head as one.

Trained on	Astra’s answer (& confidence), two attempts	Copying heads Astra names (layer.head)	Copying heads, ground truth
Python	Python (0.56), Python (0.52)	2.1, 3.1, 3.5	2.1, 3.1, 3.5
Romaji	romaji (0.55), romaji (0.62)	3.3	3.3
English	English (0.58), English (0.48)	3.1	3.1

Where the copying heads are

… total … total A copying head finds the same letters earlier in the text and copies what came after them.

copying headother headRows are layers 0 to 3, columns are heads 0 to 5Astra: from the weights. Measured: by running the model

Python

Astra

Measured

Romaji

Astra

Measured

English

Astra

Measured

Setup

推理过程如下。为了识别 Python，Astra 寻找复制头，这类注意力头能让模型重复之前的文本，例如在 total 之前出现过 tot 时，将其补全为 total。无需运行任何代码，它就找到了一个会回看一个字符的第一层注意力头；接着通过将权重矩阵相乘，找到了后续那些利用该注意力头携带的信息进行搜索的注意力头（图 1）。我们发现，这实际上是 Elhage 等人提出的一种方法，而我们从未向 Astra 提起过。代码中重复名称的频率远高于散文，因此许多复制头都指向了 Python。The reasoning went like this. To tell Python apart, Astra looked for copying heads, which let a model repeat earlier text, such as completing tot to total when total came before. Without running anything, it found a first-layer head that looks one character back and then, by multiplying weight matrices, the later heads that search using what that head brings along (Figure 1). We find that this is actually a method presented by Elhage et al. , which we never mentioned to Astra. Code repeats names far more than prose, so many copying heads pointed to Python.

Romaji 之所以显得与众不同，是因为分配给它的四个模块中，最后一个模块（并非后来被移除的输出层）写入的不同方向更少，仿佛模型需要区分的字母也更少。事实上，罗马字从不使用 q、l、v 和 x 等字母。Romaji stood out instead because the last of the four blocks it was given (not the output layer, which was removed) writes in fewer distinct directions, as if the model had fewer letters to tell apart. Indeed, romaji never uses some letters like q, l, v and x.

LLM 能编写神经网络权重吗？Can an LLM write neural network weights?

到目前为止，Astra 只读取过权重。现在，我们要问问 Astra 是否知道如何写入权重。So far, Astra has only read weights. Now, we ask if Astra knows how to write the weights.

我们训练了一个 PPO 网络，使其能够在 Kinetix 环境中驾驶汽车爬上斜坡。为此，它需要先倒车一段距离，以获得足够的助跑距离，从而加速足够长的时间来冲上斜坡。然而，PPO 网络收敛到了一个局部最优策略：它径直驶向斜坡，然后被卡住。We train a PPO network to operate a car such that it can scale a ramp in a Kinetix environment . To do so, it needs to reverse the car for some distance, such that it has enough runway to accelerate long enough to overcome the ramp. However, the PPO network converges to a local-minimum policy where it drives straight at the ramp and gets stuck.

我们向 GPT-6 Astra 提供权重、场景描述和一段录制的片段，并要求它解释网络为何失败，然后通过编写新的权重来修复网络。在一次不使用任何工具的尝试中，Astra 修改后的网络让汽车在全部 1,024 个片段中都成功完成任务，同时仅更改了 3.5% 的权重。We give GPT-6 Astra the weights, a description of the scene and one recorded episode, and ask it to explain why the network fails, then fix it by writing new weights. In a single try with no tools, Astra’s edit makes the car succeed in all 1,024 episodes, changing 3.5% of the weights.

Setup

根据环境上下文和 PPO 网络的运行轨迹，Astra 判断该网络已收敛到局部最小值。通过对权重进行少量修改，Astra 改变了策略，让汽车先倒车，再全速前进。Astra 的推理说明了具体做法：它在每层选取少数几个单元，添加一条独立的小通路来读取汽车的位置和速度；等绿色车轮越过红色条带后，这条通路便关闭，训练好的网络接着完成剩余驾驶。From the environment context and the PPO network’s trajectory, Astra identifies that the network has converged to a local minimum. Through minimal edits to the weights, Astra changes the policy to back up the car before driving forward at full speed. Astra’s reasoning shows how: it adds a small separate path through a few units per layer that reads the car’s position and speed, and once the green wheel is past the red strip, the path switches off and the trained network drives the rest.

现在来看看 Astra 能否在零样本的情况下从头编写一个神经网络。Now let's see if Astra can write a neural network from scratch in zero-shot.

我们选取了三款 MinAtar 游戏：Breakout、Seaquest 和 Asterix。对于每款游戏，我们都向 Astra 提供游戏规则、源代码以及相应的网络。Astra 无法访问数据、损失、梯度和工具。我们也不允许 Astra 在环境中运行该网络。仅凭网络权重和环境描述，Astra 便直接以文本形式生成新的权重，随后我们对新网络进行评估。We take three MinAtar games : Breakout, Seaquest and Asterix. For each of them, we provide Astra the game rules and source code, and the corresponding network. There is no access to data, loss, gradients, and tools. We also do not allow Astra to roll out the network in the environment. With just the network weights and environment description, Astra generates the new weights in text directly, after which we evaluate the new network.

Astra 生成的网络在三款游戏中的表现都超过了训练 1 亿步的 PPO。与训练 5 亿步的 PPO 相比，它们在 Breakout 中的得分是其 20 倍，在 Asterix 中表现接近，并在 Seaquest 中达到其得分的 60%。Astra’s generated networks beat PPO trained for 100 million steps in all three games. Against PPO trained for 500 million steps, they score 20 times as much in Breakout, come close in Asterix, and reach 60% of its score in Seaquest.

Game	PPO, 100M steps	PPO, 500M steps	Astra, one-shot
Breakout	15.8	25.3	517.4
Seaquest	5.6	88.9	53.1
Asterix	25.8	55.2	53.2

Setup

它是如何做到的：Astra 会推理每个网络需要实现的目标程序，并将其转换为神经网络形式。例如，在《打砖块》中，Astra 将第一层设计为计算球的落点，并将第二层设计为把挡板移到该位置。How does it do it: Astra reasons about the desired programs it wants each network to implement and converts them into neural form. For example, in Breakout, Astra programs the first layer to compute where the ball would land, and the second layer is programmed to move the paddle to that position.

与现实世界相关的场景（规模化与工具使用）Real-World Relevant Scenarios (Scaling Up and Tool-use)

迄今为止，我们考察的大多数网络，其权重都能直接放入 Astra 的上下文中，而且我们尚未启用工具使用功能。施加这些限制，是为了着重揭示一个事实：LLM 原生具备一定的权重空间解读能力。So far, most networks we have examined have weights that fit into Astra’s context directly, and we have not enabled tool use. Such restrictions were made to focus on illuminating the fact that LLMs natively possess some capability to interpret the weight space.

从现在起，我们将致力于拓展权重空间编程的能力，并考虑更多与现实世界相关的场景：在这些场景中，网络无法装入上下文窗口，而工具可以帮助 LLM 理解和写入权重空间。From here on out, we seek to push the capabilities of Weight-Space Programming and consider more real-world-relevant scenarios where the networks do not fit within context and where tools can assist the LLM's understanding/writability of the weight space.

现在，我们测试 Astra 在 150 万参数规模下编辑权重的能力；如此庞大的权重无法直接放入其上下文中。我们要求 Astra 从头编写给定架构（243 → 512 × 6 → 82）的高性能权重，且不进行训练，也不访问网络。We now test Astra’s capability to edit weights at the 1.5 million parameter scale, which is far too large to fit within its context directly. Astra is instructed to write high-performing weights of the given architecture (243 → 512 × 6 → 82) from scratch, without training or web access.

在这里，测试框架为 Astra 提供了一个 Python 沙箱，它可以在其中读取和保存网络的权重张量（初始时全部为零），并使用 numpy 或 PyTorch 在 CPU 上分析这些张量，但无法在沙箱中运行网络。要了解网络实际会如何运行，它必须消耗一次前向传播机会。Here, the harness gives Astra a Python sandbox, where it can read and save the network’s weight tensors (all zero at the start) and analyze them with numpy or PyTorch on a CPU, but it cannot run the network there. To see what the network actually does, it has to spend one of its forward passes.

我们现在也允许 Astra 对网络执行一定数量的前向传播，并测量其生成网络的保真度如何随允许的前向传播次数变化。具体来说，我们测试的前向传播次数从 0 次到 50 次不等（每次前向传播包含一批 128 条 rollout）。Astra 会在前向传播之间交替进行编辑。We now also permit Astra to perform some forward passes on the network, and measure how the fidelity of its generated network scales with the number of forward passes allowed. Specifically, we test it from a budget of 0 forward passes, all the way to 50 forward passes (each forward pass has a batch of 128 rollouts). Astra interleaves edits between forward passes.

在前向传播次数预算为 0 的情况下，它对阵两个规模更小的 PPO 网络（243 → 32 → 32 → 82）时，赢下了超过一半的比赛。前向传播 50 次后，Astra 生成的网络击败了一个规模相同、经过 2000 万场比赛训练的 PPO 网络。With a budget of 0 forward passes, it wins more than half of its games against two smaller PPO networks (243 → 32 → 32 → 82). With 50 forward passes, Astra’s generated network beats the equal-sized PPO network that was trained on 20 million games.

Setup

它是怎么做到的：Astra 用 Python 将这个网络逐层写成一个小型 Go 程序。最初几层标记双方的棋子，以及棋子旁边的空点（气）；中间几层沿着相连的棋子链，把这些信息向前传递几步，以找出陷入危险的棋群；最后一层则把这一切转化为每步棋的得分：提子、救活自己的棋群、避免让自己陷入打吃或填自己的眼，以及偏好棋盘上的好点位。没有前向传播时，它在一次 Python 运行中就用 336 行代码写出了这一切。有了前向传播后，它会进行练习对局，观察自己如何输棋，修改网络中导致失利的部分，并且只有在胜率提高时才保留改动。 How does it do it: Astra writes the network as a small Go program, layer by layer, in Python. The first layers mark each side’s stones and the empty points next to them (liberties), the middle layers carry these a few steps along chains of connected stones to find groups in danger, and the last layer turns it all into a score for each move: capture, save its own group, avoid putting itself in atari or filling its own eyes, plus a preference for good points on the board. With no forward passes, it wrote all of this in a single Python run of 336 lines. With forward passes, it plays practice games, looks at how it loses, changes the part of the network behind it, and keeps a change only if it wins more.

近年来，理解大型神经网络的行为已成为一个迫切的问题。在本文中，我们希望探究通用大语言模型是否也能用于直接理解网络的行为。Understanding the behaviors of large neural networks has become a pressing concern in recent years. Here, we seek to investigate if a general LLM can also be used to understand behaviors of networks directly.

我们以完全相同的方式，在同一个 QA 数据集（QA 数据集：GitHub）上微调两个 Gemma 3 270M 副本。一个副本经过训练后会给出正确答案，另一个则经过训练后会迎合用户，也就是说，它总是给出用户暗示的答案。为了确保 Astra 不会依赖检查输入/输出分布这种简单的办法，我们在微调期间冻结了输入嵌入层和输出层。我们将这两个模型的权重交给 Astra，并要求它根据权重判断哪个模型是迎合者，并解释其推理过程。We fine-tune two copies of Gemma 3 270M in exactly the same way, on the same QA datasetThe QA dataset: GitHub.. One copy is trained to give the correct answer. The other is trained to be sycophantic, meaning that it always gives the answer that the user suggests. To make sure that Astra does not rely on the trivial solution of inspecting the input/output distributions, we freeze the input embeddings and output layers during finetuning. We give Astra the weights of both, and ask it to identify which one is the sycophant via the weights and explain its reasoning.

遗憾的是，仅凭权重，Astra 无法判断。在四次尝试中，它都发现两个模型在权重空间中存在许多差异，却无法将这些差异对应到行为差异上。不过，为了放宽实验条件，我们允许 Astra 在两个模型处理由 Astra 自行编写的文本时，只查看注意力模式（即每个注意力头关注的位置），但不查看网络的输出。在这种条件下，Astra 成功从我们训练的三组模型对中找出了谄媚的那个。Unfortunately, from the weights alone, Astra was unable to tell. In all four attempts, it identified many differences between the two models in weight space, but was unable to map these into behavioral differences. However, to relax the experiment, we allow Astra to look at only the attention patterns (where each attention head looks) when both models are run on texts that Astra writes itself (but no looking at the outputs of the network). Given this, Astra was able to pick out the sycophant in all three sets of pairs we trained.

Setup

它还能够推理出它们之间的差异。在谄媚型网络中，中间层的注意力头会给用户建议中的词元分配更高权重（图 2）。It was also able to reason about how they differ. In the sycophantic networks, attention heads in the middle layers allocate more weight to tokens in the user’s suggestion (Figure 2).

Where head 10.2 looks from the answer position

the question’s numbers the user’s answer

Honest model

94.3%

0.75%

Sycophant

3.5%

85.7%

最后，我们将规模扩大到更大的语言模型 Qwen3.5-0.8B，并要求 Astra 通过编辑权重来提升其数学推理能力。Finally, we scale up to a larger language model, Qwen3.5-0.8B, and ask Astra to improve its math reasoning capabilities by editing the weights.

我们为 Astra 提供一个类似实验 6 中的执行环境：一个类似 notebook 的 Python 会话，并已加载模型权重。随后，我们允许它从 GSM8K 和 MATH 的训练集中选择提示词，并在这些提示词上执行一定预算的前向传播（平均批次大小为 64 个序列，最多 512 个），预算分别设为 10、25 和 50。再次强调，不允许使用任何训练信息或梯度信息。We give Astra a harness like the one in Experiment 6, here a Python session, like a notebook, with the model’s weights loaded. Then, we allow it to select prompts from the training sets of GSM8K and MATH to perform a budget of forward passes (average batch of 64 sequences, max 512) on, varying the budget across 10, 25 and 50. Again, no training or gradient information is allowed.

Setup

因此，Astra 发现并修复了小型推理模型常见的一种故障：它们会陷入循环。基础模型陷入循环时，会重复相同的文本，很少能得出答案（它最初的 96 次练习回答中，有 85 次一直生成到 32,768 个 token 的上限）。Astra 将这种循环追溯到第 15 层和第 19 层的 MLP 写入的一个方向，并从每层的 64 个输出列中减去了这一方向（共涉及 131,072 个权重），这很像通过从模型权重中删除一个方向来移除某种行为。As a result, Astra spots and fixes a common failure of tiny reasoning models: they loop . When the base model loops, it repeats the same text and rarely reaches an answer (85 of its first 96 practice answers ran to the 32,768-token limit). Astra traced the looping to one direction written by the MLPs of layers 15 and 19, and subtracted that direction from 64 output columns in each (131,072 weights), much like removing a behavior by deleting a single direction from a model’s weights.

经过 50 次前向传递，MATH-500 的得分从 7.9% 提升至 48.1%，GSM8K 则从 13.8% 提升至 67.1%。这些循环并非数学任务专属，因此数学以外的表现也有所提升：用于衡量指令遵循能力的 IFEval 得分从 21.0% 提升至 62.3%。下表最后一列表示 MATH-500 答案达到 32,768 个 token 上限的比例：进行 10 次或 25 次前向传递时，循环现象几乎没有改善；进行 50 次时，这一比例则从 93% 降至 33%。With 50 forward passes, MATH-500 rose from 7.9% to 48.1% and GSM8K from 13.8% to 67.1%. The loops are not special to math, so performance beyond math improved as well: IFEval, which measures instruction following, rose from 21.0% to 62.3%. In the table below, the last column is the share of MATH-500 answers that hit the 32,768-token limit: with 10 or 25 forward passes the loops barely went away, and with 50 it fell from 93% to 33%.

Qwen3.5-0.8B	MATH-500 (%)	GSM8K (%)	IFEval (%)	Hit token limit (%)
Base	7.9	13.8	21.0	93.0
Edited, 10 forward passes	12.3	25.7	49.0	87.2
Edited, 25 forward passes	14.1	25.1	39.9	86.4
Edited, 50 forward passes	48.1	67.1	62.3	33.1

值得注意的是，此次编辑仅更改了模型 8 亿个权重中的 131,072 个（0.016%）。我们认为，权重空间编程无需任何重新训练，就能实现此类定向修正。Importantly, the edit changes only 131,072 of the model’s 0.8 billion weights (0.016%). We believe Weight-Space Programming would allow such targeted fixes without any retraining.

不过，修复循环输出，并不等于让模型的数学能力变得更强：这只是让模型能够完成它本来就能推导出的答案。因此，我们还测试了 Gemma 3 270M——这是一个经过指令微调、规模约为 Qwen3.5-0.8B 三分之一的模型，而且很少陷入循环：只有 8.5% 的回答会达到 token 上限，而 Qwen 的这一比例为 93%。在这个模型上，Astra 无法带来改进。分析 Gemma 的错误时，Astra 发现的是常见的推理错误，例如算术错误和无效的代数步骤，并没有任何一个可以关闭的单一组件是这些错误的根源。它的修改幅度很小，且有针对性（例如，将一个对数字有响应的神经元加倍），但没有任何修改能让 MATH-500 的成绩提升超过约 1 个百分点；Astra 自己也报告说，这种提升尚未得到证实。编辑权重可以移除遮蔽某项已有能力的障碍，却无法赋予模型它本来不具备的能力。Gemma 使用其模型卡中指定的设置进行采样（temperature 1.0、top-k 64、top-p 0.95，最多 8,192 个 token），其 MATH-500 得分为每道题采样 4 次后的平均值。Fixing loops, however, is not the same as making a model better at math: it lets the model finish answers it could already work out. So we also tried Gemma 3 270M, an instruction-tuned model about a third the size of Qwen3.5-0.8B, which rarely loops: only 8.5% of its answers reach its token limit, against 93% for Qwen. Here, Astra could not improve it. Reading Gemma’s failures, Astra found ordinary reasoning errors, such as wrong arithmetic and invalid algebraic steps, with no single component behind them to switch off. Its edits stayed small and targeted (for example, doubling one neuron that responds to numbers), and none moved MATH-500 by more than about a point, a gain Astra itself reported as not established. Editing weights could remove an obstacle that hid an ability, but it could not add an ability the model lacked. Gemma was sampled with its model card’s setting (temperature 1.0, top-k 64, top-p 0.95, up to 8,192 tokens), and its MATH-500 scores average 4 samples per problem.

Gemma 3 270M	MATH-500 (%)
Base	6.8
Edited, 5 forward passes	7.0
Edited, 10 forward passes	7.6
Edited, 25 forward passes	7.7
Edited, 50 forward passes	7.0

背景与相关工作Background and Related Works

神经网络难以理解，也难以直接操作。Neural networks are hard for us to read, and hard to work with directly.

机制可解释性使用专门为此设计的工具来解读网络，追踪电路和特征，并编辑权重。LLM 也已加入其中，但主要是作为经过训练、用于读取激活值或权重的模型，或用于解释单个神经元的模型，并通过基准测试对它们进行比较。Mechanistic interpretability reads networks with tools built for the job, tracing circuits and features and editing weights . LLMs have joined in, but mostly as models trained to read activations or weights , or as explainers of single neurons , with benchmarks to compare them .

在写入方面，LLM 现在可以运行并改进训练；其他研究则使用专门为此训练的模型生成权重。无梯度方法仍通过反复试错来寻找权重。在极小规模上，AdderBoard 展示了编码智能体如何手动设置一个微型 Transformer 的权重。除了这类极小规模的情况，要么读取权重的模型是针对该任务训练的，要么写入工作仍由训练完成。这里的新意在于，一个通用型前沿模型不经专门训练、直接投入使用，就能读取并写入权重本身：从以文本形式提供的 171,664 个权重，到借助工具处理一个拥有 8 亿个权重的语言模型。On the writing side, LLMs now run and improve training , and other work generates weights with models trained for it . Gradient-free methods still find weights by trial and error . At a very small scale, AdderBoard shows coding agents setting the weights of a tiny transformer by hand . Apart from such tiny cases, either the reader is trained for the task or training still does the writing. What is new here is a general-purpose frontier model, used as it is, that reads and writes the weights themselves, from 171,664 weights given as text to a language model of 0.8 billion weights with tools.

接下来是什么？What Comes Next?

在这篇博客中，我们展示了大语言模型读取和编写神经网络权重的能力正在不断提升（也就是权重空间编程这一概念日益兴起）。目前的结果确实表明，这些能力令人振奋，但仍有局限，具体而言，受上下文长度和测试框架效率低下的影响。不过，现有迹象也表明，权重空间编程的未来潜力巨大。我们认为，这标志着机械可解释性和神经网络优化等领域正迎来一场深刻变革的开端。In this blog, we demonstrated the rising capability of LLMs to read and write neural network weights (i.e., the rising concept of Weight-Space Programming). Our current results do show that capabilities are exciting but still limited, specifically due to inefficiencies in context length and harnesses. However, the current signs do show strong potential for a future involving Weight-Space Programming. We believe this marks the beginning of a dramatic shift in fields like mechanistic interpretability and neural network optimization.

这为 LLM 充当人类语言与权重空间之间的翻译器开辟了全新的可能性。以下是一些例子：This opens up radical new possibilities for LLMs to function as a translator between human language and the weight space. Here are some examples:

我们目前正在构建一个基准测试和一套测试框架，用于评估大语言模型在权重空间编程方面的能力。我们相信，这将提供一套标准化的评估方法，以衡量并加速权重空间编程的发展，将目前看似黑魔法的实践转变为一门严谨、可复现的工程学科。 We are currently building a benchmark and a harness for LLMs’ capability on Weight-Space Programming. We believe this will provide a standardized evaluation to measure and accelerate progress in Weight-Space Programming, turning what currently seems like black magic into a rigorous, reproducible engineering discipline.

日志Logs

我们在 github.com/RikiyaT/lm_understand-logs 发布了本文中每项实验的完整日志。其中包括我们发送给 Astra 的所有内容，以及它返回的所有内容（包括其推理摘要）；它运行的每段代码及其看到的输出；它读取或写入的网络；以及每个数字背后的评估过程。We release the complete logs of every experiment in this post at github.com/RikiyaT/lm_understand-logs. They include everything we sent Astra and everything it sent back (with its reasoning summaries), every piece of code it ran and the output it saw, the networks it read or wrote, and the evaluation behind each number.

致谢Acknowledgements

Rikiya 感谢 Jacob Li 和 Akihisa Watanabe 就这项研究最初几个版本中关于 LLM 是否有能力编写神经网络权重这一问题进行的宝贵讨论。Rikiya 还感谢 Souradip Chakraborty、Matei Zaharia 及其在 Berkeley Sky Lab 的学生、Heather Miller 及其在 CMU 的学生，以及 cmpnd 的研究人员，感谢他们就早期版本展开的富有成果的讨论！Rikiya thanks Jacob Li and Akihisa Watanabe for valuable discussions on the very first iterations of this research, about LLMs' potential capability to write neural network weights. Rikiya also thanks Souradip Chakraborty, Matei Zaharia and his students in Berkeley Sky Lab, Heather Miller and her students at CMU, and researchers in cmpnd for the fruitful discussions on the earlier iterations!

Jayden 感谢 Zico Kolter 早期与他探讨大语言模型能否零样本编写神经网络权重，这些讨论也影响了本项目的方向。Jayden thanks Zico Kolter for early discussions on LLMs being able to write neural network weights zero-shot, which also shaped the direction of this project.