欢迎在 Diffusers 中接入 FLUX.3 Image Klein

Back to Articles

Update on GitHub

Black Forest Labs FLUX.3 Image Klein is here, and it runs in Diffusers from day one. One model does text-to-image and multi-reference image-to-image, and it adds something new, the layout control. You write bounding boxes into the prompt to say where each part of the image should go, or which parts of a reference image to move.

这两个开放检查点都属于 FLUX.3 Image Klein 系列：Both open checkpoints are in the FLUX.3 Image Klein collection:

现在就可以前往 Hugging Face Space 生成一些图片！You can head on to the Hugging Face space and generate some images right now!

简而言之 TL;DR

• 一个模型，支持文本生成图像、多参考图像到图像转换，以及边界框布局控制。
  One model for text-to-image, multi-reference image-to-image, and bounding-box layout control.
• 该蒸馏检查点在 H100 上生成一张 1024 × 1024 的图像约需 2 秒。
  The distilled checkpoint makes a 1024 × 1024 image in about 2 seconds on an H100.
• NVFP4 和 FP8 权重可通过 from_single_file 加载，适用于显存较小的 GPU。
  NVFP4 and FP8 weights load with from_single_file, for smaller GPUs.
• 用于文本生成图像和图像编辑的 LoRA 训练脚本。
  LoRA training scripts for both text-to-image and image editing.
• 以模块化 Diffusers 流水线的形式提供，因此你可以移除其中的模块，也可以添加自己的模块。
  Ships as a Modular Diffusers pipeline, so you can pop blocks out or add your own.

目录 Contents

• 入门指南
  Getting started
• FLUX.3 能做什么
  What FLUX.3 can do
• 基础模型及其量化版本
  Base model and quantized variants
• LoRA 微调
  LoRA finetuning
• 运行成本
  What it costs to run
• 架构
  The architecture
• 模块化扩散器
  Modular Diffusers
• 试试看
  Try it

快速入门 Getting started

安装最新版本的 diffusers 和 transformers（用于 Qwen3-VL 文本编码器）：Install the latest diffusers and transformers (for the Qwen3-VL text encoder):

pip install -U diffusers transformers

加载蒸馏后的检查点并生成图像：Load the distilled checkpoint and generate an image:

import torch
from diffusers import ModularPipeline

pipe = ModularPipeline.from_pretrained("black-forest-labs/flux-3-image-klein")
pipe.load_components(dtype=torch.bfloat16)
pipe.to("cuda")

pipe(prompt="a red fox", output="images")[0].save("fox.png")

FLUX.3 能做什么 What FLUX.3 can do

文本生成图像 Text-to-image

FLUX.3 最适合输入包含主体、场景、光线、构图和风格的完整场景描述。Black Forest Labs 的官方提示指南提供了更多建议。默认情况下，蒸馏检查点会通过 16 个步骤生成一张 1024 × 1024 的图像。传入 height、width 或 num_inference_steps 即可更改这些设置。FLUX.3 responds best to a full scene description with the subject, the setting, the light, the composition, and the style. The official prompting guide from Black Forest Labs has more tips. The distilled checkpoint generates a 1024 × 1024 image in 16 steps by default. Pass height, width, or num_inference_steps to change them.

prompt = (
    "A wide-angle wildlife photograph of a red fox standing alert in a snow-covered forest clearing at golden hour. "
    "The fox is centered in the frame, its thick orange winter coat contrasting with the fresh white snow, breath "
    "visible in the cold air. Snow-laden pine trees fill the softly blurred background, and low warm sunlight rakes "
    "across the scene from the left, casting long blue shadows."
)

fox = pipe(
    prompt=prompt,
    generator=torch.Generator().manual_seed(0),
    output="images"
)[0]
fox.save("fox.png")

图生图 Image-to-image

通过 image 参数传入一张或多张参考图像。将提示词写成你想要的图像的描述，并包含所需的修改。Pass a reference image (or more) with the image argument. Write the prompt as a caption of the image you want, with the change included.

prompt = (
    "The same red fox in the same snow-covered forest clearing, now at night under a full moon: cool blue "
    "moonlight, deep shadows, stars visible through the pine branches, the fox's eyes catching the light."
)
night = pipe(
    prompt=prompt,
    image=fox, # from the previous text to image example
    generator=torch.Generator().manual_seed(0),
    output="images",
)[0]
night.save("night.png")

如果有多个参考图，请在提示词中按照传递给 image 的顺序将它们称为 <ref_image_0>、<ref_image_1>……。With several references, refer to them in the prompt as <ref_image_0>, <ref_image_1>, ... in the order they are passed to image:

red_fox = pipe(prompt="a red fox", generator=torch.Generator().manual_seed(0), output="images")[0]
bench = pipe(prompt="a wooden bench", generator=torch.Generator().manual_seed(0), output="images")[0]

combined = pipe(
    prompt="The fox from <ref_image_0> sitting on the wooden bench from <ref_image_1> in a snowy setting.",
    image=[red_fox, bench],
    output="images",
)[0]

combined.save("combined.png")

布局控制 Layout control

在图注后添加一个 JSON 区域列表。每个区域包含 id、bbox 和 desc，图注中会以 <id> 的形式提及该区域。Add a JSON list of regions after the caption. Each region has an id, a bbox, and a desc, and the caption mentions it as <id>.

边界框按 [y_min, x_min, y_max, x_max] 的顺序书写，先 y 后 x，取值范围为 0–1000。由于该尺度经过归一化，同一个边界框适用于任意图像尺寸。

Boxes are written as [y_min, x_min, y_max, x_max], y first, on a 0–1000 scale. Because the scale is normalized, the same box works at any image size.

在新图像中放置区域 Placing regions in a new image

layout_prompt = (
    "Minimalist graphic illustration featuring a black silhouette of a person <silhouette_1> centered against a "
    "solid, vibrant chartreuse background <background_1>. The figure is captured in a dynamic, mid-stride running "
    "pose, facing toward the left side of the frame. "
    '[{"id": "background_1", "bbox": [0, 0, 1000, 1000], "desc": "A flat field of neon yellow-green with a subtle '
    'paper texture and fine, organic grain."}, {"id": "silhouette_1", "bbox": [150, 150, 850, 850], "desc": "A '
    "black silhouette of a person in motion, made of dense stippled ink. The leading edges are solid; the trailing "
    'arm and leg dissolve into coarse square pixels and scattered dots."}]'
)

runner = pipe(prompt=layout_prompt, generator=torch.Generator().manual_seed(0), output="images")[0]

在参考图像中移动区域 Moving regions within a reference image

对于编辑，每个区域都会说明其内容的来源位置（参考图中的 src_bbox）和目标位置（输出中的 tgt_bbox）。两个框相同的区域会保持原位。For edits, each region says where its content comes from (src_bbox in the reference) and where it goes (tgt_bbox in the output). A region with the same two boxes stays where it is.

此示例将针织骑士向左上方移动，并固定背景、悬崖、针、龙和火焰：This example moves the knitted knight up and to the left, and pins the backdrop, the cliff, the needle, the dragon, and the fire:

from diffusers.utils import load_image

knight_image = load_image("knight_orig.png")
edit_prompt = (
    "In <ref_image_0>, move the miniature grey amigurumi knight figure <knight_1> upwards and to the left along the "
    "yarn cliff <cliff_1>. Keep the rest of the image completely unchanged, preserving the blurred yarn backdrop "
    "<backdrop_1>, the large knitted dragon <dragon_1>, and the fiery orange thread <fire_1>. "
    '[{"id": "knight_1", "from": "ref_image_0", "src_bbox": [500, 150, 850, 350], "tgt_bbox": [194, 55, 544, 255], '
    '"desc": "A miniature amigurumi knight crocheted from thick grey woolen yarn."}, '
    '{"id": "backdrop_1", "from": "ref_image_0", "src_bbox": [0, 0, 1000, 1000], "tgt_bbox": [0, 0, 1000, 1000], '
    '"desc": "A soft, out-of-focus backdrop of cream and indigo knitted yarn."}, '
    '{"id": "cliff_1", "from": "ref_image_0", "src_bbox": [300, 0, 1000, 450], "tgt_bbox": [300, 0, 1000, 450], '
    '"desc": "A steep cliff of stacked yarn skeins in mustard, terracotta, and indigo."}, '
    '{"id": "needle_1", "from": "ref_image_0", "src_bbox": [420, 250, 550, 450], "tgt_bbox": [420, 250, 550, 450], '
    '"desc": "An oversized polished steel tapestry needle."}, '
    '{"id": "dragon_1", "from": "ref_image_0", "src_bbox": [100, 600, 950, 1000], "tgt_bbox": [100, 600, 950, 1000], '
    '"desc": "A large, fluffy knitted dragon in terracotta and mustard yarn."}, '
    '{"id": "fire_1", "from": "ref_image_0", "src_bbox": [350, 400, 650, 650], "tgt_bbox": [350, 400, 650, 650], '
    '"desc": "Curled puffs of fiery orange angora thread suspended in mid-air."}]'
)

moved = pipe(prompt=edit_prompt, image=knight_image, generator=torch.Generator().manual_seed(0), output="images")[0]

基础模型和量化变体 Base model and quantized variants

基础模型 Base model

以上内容均使用蒸馏检查点。基础检查点在分类器无关引导下运行 50 步，它才是需要微调的版本（参见 LoRA 微调）。Everything above uses the distilled checkpoint. The base checkpoint runs 50 steps with classifier-free guidance, and it's the one to fine-tune (see LoRA finetuning).

pipe = ModularPipeline.from_pretrained("black-forest-labs/flux-3-image-klein-base")
pipe.load_components(dtype={"vae": torch.float32, "default": torch.bfloat16})
pipe.to("cuda")

pipe.guider = pipe.guider.new(guidance_scale=5.0)  # optional
image = pipe(prompt=prompt, output="images")[0]

单文件格式检查点 Single-file format checkpoints

Black Forest Labs 还提供单文件形式的 transformer，包括 bf16 版本和两种量化格式：fp8r（按行量化的 FP8）和 nvfp4。使用 from_single_file 加载其中任意一种，然后按名称加载其他组件，这样就不会额外下载 bf16 transformer。Black Forest Labs also ships the transformer as a single file, in bf16 and in two quantized formats: fp8r (rowwise FP8) and nvfp4. Load any of them with from_single_file, then load the other components by name so the bf16 transformer isn't downloaded too.

from diffusers import BFLQuantizationConfig, Flux3Transformer2DModel, ModularPipeline

model_id = "black-forest-labs/flux-3-image-klein-base"
variant = "fp8r"  # or "nvfp4"

transformer = Flux3Transformer2DModel.from_single_file(
    f"https://huggingface.co/{model_id}/blob/main/variants/{variant}/model.safetensors",
    config=model_id,
    subfolder="transformer",
    quantization_config=BFLQuantizationConfig(compute_dtype=torch.bfloat16),
    dtype=torch.bfloat16,
)

pipe = ModularPipeline.from_pretrained(model_id)
pipe.load_components(dtype=torch.bfloat16)
pipe.update_components(transformer=transformer)
pipe.to("cuda")

For the unquantized single file, use https://huggingface.co/{model_id}/blob/main/model.safetensors and drop quantization_config.

LoRA 微调 LoRA finetuning

FLUX.3 同时支持文生图和图生图，因此非常适合作为定制 LoRA 的基础。diffusers 提供了适用于这两种任务的 DreamBooth LoRA 训练脚本（你也可以使用 Ostris 的 AI Toolkit 进行训练）。FLUX.3 works for both text-to-image and image-to-image, which makes it a great base for custom LoRAs. diffusers ships DreamBooth LoRA training scripts for both tasks (you can also train with Ostris' AI Toolkit).

以下是一个文本生成图像示例，它从 multimodalart/1920-raider-waite-tarot-public-domain 数据集中学习塔罗牌风格，并通过 trtcrd 触发：Here is a text-to-image example that learns a tarot-card style from the multimodalart/1920-raider-waite-tarot-public-domain dataset, triggered with trtcrd:

accelerate launch train_dreambooth_lora_flux3_klein.py \
  --pretrained_model_name_or_path="black-forest-labs/flux-3-image-klein-base" \
  --dataset_name="multimodalart/1920-raider-waite-tarot-public-domain" \
  --caption_column="caption" \
  --instance_prompt="trtcrd" \
  --output_dir="trained-flux3-tarot-lora" \
  --mixed_precision="bf16" \
  --resolution=1024 \
  --use_aspect_ratio_buckets \
  --train_batch_size=1 \
  --rank=16 \
  --lora_alpha=16 \
  --learning_rate=1e-4 \
  --lr_scheduler="constant" \
  --max_train_steps=1500 \
  --validation_prompt="a trtcrd of a cat, tarot style" \
  --validation_epochs=25 \
  --seed=0 \
  --push_to_hub

To reduce memory usage, add a few optimization flags:

  --gradient_checkpointing \
  --cache_latents \
  --use_8bit_adam

--gradient_checkpointing recomputes activations in the backward pass (lower VRAM, slightly slower), --cache_latents pre-encodes the images and frees the VAE, and --use_8bit_adam keeps the optimizer states in 8-bit.

有几个参数值得尝试。下方的网格使用不同的 LoRA 秩和优化器训练相同的塔罗风格；适配器托管在 Hub 上，地址为 diffusers-modular/flux3-tarot-lora。更多训练功能和配置请参阅 DreamBooth FLUX.3 README。A couple of knobs are worth experimenting with. The grid below trains the same tarot style with different LoRA ranks and optimizers; the adapters are on the Hub at diffusers-modular/flux3-tarot-lora. More training features and configurations are documented in the DreamBooth FLUX.3 README.

对于基于指令的编辑，图生图脚本（train_dreambooth_lora_flux3_klein_img2img.py）使用（参考图像、目标图像、指令）三元组进行训练，其中提示词通过 <ref_image_0> 指代参考图像。For instruction-based editing, the image-to-image script (train_dreambooth_lora_flux3_klein_img2img.py) trains on (reference image, target image, instruction) triplets, where the reference is addressed in the prompt as <ref_image_0>.

有关所有选项和完整的图生图示例，请参阅 DreamBooth FLUX.3 README。See the DreamBooth FLUX.3 README for all options and the full image-to-image example.

运行成本 What it costs to run

在本节中，我们将讨论模型的内存消耗和生成速度。In this section, we will talk about the memory consumption and the speed of generation of the models.

内存 Memory

如果你的内存较少：If you have less memory:

• 加载一个量化后的 Transformer。
  Load a quantized transformer.
• 单独运行文本编码器，然后释放它。它在去噪之前，每张图像只运行一次（参见“编码一次，生成多次”）。
  Run the text encoder separately and free it. It only runs once per image, before denoising (see Encode once, generate many).
• 开启自动 CPU 卸载功能，仅在运行时将各个组件移至 GPU（我们未对此进行基准测试）
  Turn on automatic CPU offloading, which moves each component to the GPU only while it runs (we didn't benchmark this)

from diffusers import ComponentsManager, ModularPipeline

manager = ComponentsManager()
manager.enable_auto_cpu_offload(device="cuda")

pipe = ModularPipeline.from_pretrained("black-forest-labs/flux-3-image-klein", components_manager=manager)
pipe.load_components(dtype={"vae": torch.float32, "default": torch.bfloat16})
# no pipe.to("cuda"): the manager handles device placement

速度 Speed

蒸馏检查点（16 步）的端到端耗时，在预热后进行三次运行并取平均值：End-to-end timings for the distilled checkpoint (16 steps), averaged over three runs after a warmup:

单张 NVIDIA H100 80GB，不启用卸载，使用默认注意力后端。

Single NVIDIA H100 80GB, no offloading, default attention backend.

参考图大致会使耗时翻倍。参考图 token 会与文本和正在生成的图像一起加入同一个序列，因此一张 1024 × 1024 的参考图会为每次 Transformer 调用增加 4,096 个 token。A reference image roughly doubles the time. Reference tokens join the same sequence as the text and the image being generated, so a 1024 × 1024 reference adds 4,096 tokens to every transformer call

架构 The architecture

我们来逐一拆解这个模型。模型接收三种类型的 token：Let's open the model up, one piece at a time. The model takes three kinds of tokens:

1. 文本
   text
2. 正在生成的噪声图像
   the noisy image being generated
3. 以及可选的参考图像
   and optional reference images

从像素到词元 From pixels to tokens

FLUX.3 是一种用于图像生成的潜空间流模型。这意味着去噪器以潜变量为输入，并输出潜变量。我们使用 VAE 将潜变量解码回像素。FLUX.3 使用与 Flux.2 相同的 VAE 架构。FLUX.3 is a latent-space flow model for image generation. This means that the denoiser takes latents as inputs and produces latents as outputs. We use a VAE to decode the latents back to pixels. FLUX.3 uses the same VAE architecture as Flux.2.

FLUX.2 VAE 会将图像在两个方向上各压缩 8 倍，生成 32 个通道；随后，潜变量中的每个 2 × 2 图块都会被打包成一个包含 128 个通道的 token。The FLUX.2 VAE compresses an image 8× in each direction into 32 channels, and then every 2 × 2 patch of that latent is packed into a single token with 128 channels.

因此，一个 token 对应一个 16 × 16 像素的图像块，而一张 1024 × 1024 的图像会变成一个 64 × 64、共 4,096 个 token 的网格。带噪图像一开始是形状完全相同的随机噪声。参考图像经过相同的 VAE 编码器和打包处理，并保留自身尺寸。So one token covers a 16 × 16 pixel patch, and a 1024 × 1024 image becomes a 64 × 64 grid of 4,096 tokens. The noisy image starts as random noise of exactly that shape. A reference image goes through the same VAE encoder and packing, and keeps its own size.

从提示词到令牌 From prompt to tokens

文本编码器是 Qwen3-VL，这是一种视觉语言模型。FLUX.3 并非只使用它的最后一层，而是读取其 36 层中的 8 层（第 4、8、……、32 层）的隐藏状态并将它们堆叠起来，因此每个文本 token 在经过线性层投影到 Transformer 的 3072 维宽度之前，会变成一个 8 × 2560 = 20,480 维向量。The text encoder is Qwen3-VL, a vision-language model. FLUX.3 doesn't just take its last layer. It reads the hidden states of eight of its 36 layers (layers 4, 8, …, 32) and stacks them, so each text token becomes a 8 × 2560 = 20,480-dimensional vector before a linear layer projects it to the transformer's width of 3072.

这会提取语言模型多个深度层的 Transformer 特征，而不仅仅是最后一层。提示词会填充到 80 个 token 的整数倍，Transformer 也会关注这些填充内容，因此该流水线每次调用只处理一个提示词。This gives the transformer features from several depths of the language model, not just its final layer. The prompt is padded to a multiple of 80 tokens, and the transformer attends to the padding too, which is why the pipeline takes one prompt per call.

两个阶段 Two stages

Transformer 分两个阶段处理这些 token：The transformer processes the tokens in two stages:

1. 阶段 1：分模态块。每个流都会经过 5 个各自拥有独立权重的块。文本有自己的块，噪声图像有自己的块，参考图像也有自己的块。
   Stage 1: per-mode blocks. Each stream goes through 5 blocks with its own weights. Text has its own blocks, the noisy image has its own, and the reference images have their own.
2. 阶段 2：联合块。各个流被拼接成一个序列 [文本 | 图像 | 参考]，并经过 28 个共享同一组权重的块。
   Stage 2: joint blocks. The streams are concatenated into one sequence, [text | image | reference], which goes through 28 blocks that share one set of weights.

这两个阶段的区别在于谁可以照看谁。The two stages differ in who can attend to whom.

在阶段 1 中，每个流只关注自身，因此每种 token 都会先按照自身的特点进行细化。在阶段 2 中，所有内容相互关注，图像从文本和参考内容中学习。作为对比，FLUX.2 的双流模块也为文本和图像分别设置了权重，但从一开始就允许它们相互关注。FLUX.3 则先让各个流完全分离，然后再将它们合并。In stage 1, each stream only attends to itself, so each kind of token gets refined on its own terms first. In stage 2, everything attends to everything, where the image learns from the text and from the references. For comparison, FLUX.2's double-stream blocks also give text and image separate weights, but let them attend to each other from the start. FLUX.3 keeps the streams fully apart first, then merges them.

最后，文本 token 和参考 token 会被直接丢弃。只有图像 token 会经过输出层，由输出层预测如何让它们向干净图像迈进一步。At the end, the text and reference tokens are simply dropped. Only the image tokens go through the output layers, which predict how to move them one step toward a clean image.

在一个块内 Inside one block

两个阶段中的每个块都具有相同的结构。注意力模块和 MLP 并行处理同一个归一化后的输入，然后将它们的输出相加并加回残差流。Every block, in either stage, has the same shape. Attention and the MLP run in parallel on the same normalized input, and their sum is added back to the residual stream.

每个块都以一个 LayerNorm 开始，它本身没有可学习的缩放或偏移参数。缩放、偏移和门控则来自时间步嵌入（这称为调制）。在注意力机制内部，查询和键经过 RMS 归一化，并加入旋转位置嵌入（RoPE），我们将在下文中介绍。Each block starts with a LayerNorm that has no learned scale or shift of its own. The scale, shift, and gate come from a timestep embedding instead (this is called modulation). Inside the attention, the queries and keys are RMS-normalized and get rotary position embeddings (RoPE), which we look at below.

在联合块中，所有令牌共享相同的权重，但文本、图像和参考令牌各自拥有独立的缩放、偏移和门控。In a joint block, all tokens share the same weights, but text, image, and reference tokens each get their own scale, shift, and gate.

def block(x, shift, scale, gate):      # shift, scale, gate come from the token's timestep embedding
    h = layer_norm(x)                  # no learned scale or shift
    h = (1 + scale) * h + shift
    q, k, v = to_q(h), to_k(h), to_v(h)
    q, k = rope(rms_norm(q)), rope(rms_norm(k))
    out = attention(q, k, v) + mlp(h)  # parallel, MLP is SwiGLU
    return x + gate * out

每个 token 都有自己的时间步 Every token gets its own timestep

在 FLUX.1 和 FLUX.2 中，一个时间步嵌入会调制每个 token。在 FLUX.3 中，调制方式取决于 token 的类型。In FLUX.1 and FLUX.2, one timestep embedding modulates every token. In FLUX.3, the modulation depends on the kind of token.

带噪图像 token 会根据当前时间步 t 进行调制。文本和参考 token 始终由时间步 0 调制，这意味着“干净，无需在此去噪”。因此，即使某个参考图像看起来几乎与输出相同，它也会被标记为给定的上下文，而不是需要生成的内容。The noisy image tokens are modulated by the current timestep t. The text and reference tokens are always modulated by timestep 0, which means "clean, nothing to denoise here". So even a reference that looks almost the same as the output is marked as given context, not as something to generate.

蒸馏后的检查点就是一个流映射。它还会获知每一步要跳到的时间步。这个第二时间步也有自己的嵌入，并会加到每个 token 的调制中，因此它的 16 个步骤都可以实现更大的跳跃。The distilled checkpoint is a flow map. It is also told the timestep each step jumps to. That second timestep gets its own embedding, which is added to every token's modulation, so each of its 16 steps can make a larger jump.

四维空间中的位置 Positions in four dimensions

注意力机制本身并不知道 token 位于何处（这种现象称为置换等变性）。RoPE 通过根据 token 的位置旋转每个 query 和 key 来解决这个问题。FLUX.3 使用四个位置轴 (t, h, w, l)（时间步、高度、宽度和序列长度），并为每个 128 维注意力头中的每个轴分配四分之一的维度（32 维）：Attention on its own doesn't know where a token is (this phenomenon is called permutation equivariance). RoPE fixes that by rotating each query and key by an amount that depends on the token's position. FLUX.3 uses four position axes, (t, h, w, l) (timestep, height, width, length of sequence), and gives each one a quarter (32 dimensions) of every 128-dimensional attention head:

文本 token 沿着 l 排列，就像句子中的单词一样。图像和参考图是位于 (h, w) 平面上的二维网格，并沿 l 轴彼此分开。由于参考图与输出使用相同的 (h, w) 坐标，因此“参考图中的同一位置”对模型来说有明确的含义，这正是 src_bbox / tgt_bbox 编辑所依赖的。Text tokens line up along l, like words in a sentence. The image and the references are 2D grids in (h, w), separated from each other along l. Because a reference uses the same (h, w) coordinates as the output, "the same spot in the reference" has a precise meaning to the model, which is what the src_bbox / tgt_bbox editing relies on.

去噪循环 The denoising loop

FLUX.3 是一种流匹配模型。生成过程从纯噪声开始，在 50 个步骤中的每一步，Transformer 都会预测一个速度向量，表示从噪声指向清晰图像的方向。调度器沿着这个方向迈出一步。FLUX.3 is a flow-matching model. Generation starts from pure noise, and at each of the 50 steps the transformer predicts a velocity. This is the direction from noise toward a clean image. The scheduler takes one step along it.

在每一步，你还可以直接查看模型对最终图像的当前猜测。在几个步骤中解码这些猜测，就能看到图像逐渐成形：At every step you can also read off the model's current guess of the final image. Decoding that guess at a few steps shows how the image takes shape:

到第 10 步时，狐狸、它的姿势和光线都已就位，而噪声水平仍为 0.97。积雪的枝条和毛皮的纹理只在最后几步才显现。FLUX.3 的调度器采用较大的时间偏移（6.93），因此在大部分生成过程中噪声水平都保持较高：50 步中有 44 步的 σ 大于 0.5。我们只需向流水线中添加一个小模块，就构建出了这幅图；这也引出了 Modular Diffusers。The fox, its pose, and the light are already in place by step 10, while the noise level is still 0.97. The snow-laden branches and the texture of the fur only appear in the last few steps. FLUX.3's scheduler uses a large time shift (6.93), which keeps the noise level high for most of the run: 44 of the 50 steps happen above σ = 0.5. We built this figure by adding one small block to the pipeline, which brings us to Modular Diffusers.

如果你刚接触流匹配，我们为你准备了一个入门教程和一段视频，帮助你了解它的工作原理。

If you are new to flow matching, we have a beginner friendly tutorial and a video for you to understand how it works.

模块化 Diffusers Modular Diffusers

FLUX.3 以模块化 Diffusers pipeline 的形式发布。FLUX.3 ships as a Modular Diffusers pipeline.

查看工作流 See the workflows

你可以列出这些工作流，并打印出每个工作流的确切代码块，而无需加载任何权重。You can list the workflows and print the exact blocks of each one, without loading any weights.

from diffusers import Flux3AutoBlocks

blocks = Flux3AutoBlocks()
print(blocks.available_workflows)
# ['text2image', 'image_conditioned']

print(list(blocks.get_workflow("text2image").sub_blocks))
# ['text_encoder', 'denoise.input', 'denoise.prepare_latents', 'denoise.set_timesteps',
#  'denoise.prepare_rope_inputs', 'denoise.denoise', 'denoise.after_denoise', 'decode']

get_workflow() 会解析自动分支，并提供将要运行的步骤序列。print(blocks.get_workflow("image_conditioned").doc) 会打印该工作流的组件、输入、默认值和输出。get_workflow() resolves the auto branches and gives you the plain sequence of steps that will run. print(blocks.get_workflow("image_conditioned").doc) prints the components, inputs, defaults, and outputs of that workflow.

编码一次，生成多份 Encode once, generate many

卸载文本编码器，只对提示词编码一次，然后释放它，并使用缓存的嵌入进行生成。Pop the text encoder out, encode the prompt once, free it, and generate from the cached embeddings.

import torch
from diffusers import ModularPipeline

model_id = "black-forest-labs/flux-3-image-klein-base"

blocks = ModularPipeline.from_pretrained(model_id).blocks
text_blocks = blocks.sub_blocks.pop("text_encoder")  # `blocks` now starts at the VAE encoder

text_pipe = text_blocks.init_pipeline(model_id)
text_pipe.load_components(dtype=torch.bfloat16)
text_pipe.to("cuda")

state = text_pipe(prompt=prompt, negative_prompt="")  # no guider here, so ask for the negative embeddings

prompt_embeds = state.get("prompt_embeds")
negative_prompt_embeds = state.get("negative_prompt_embeds")

del text_pipe
torch.cuda.empty_cache()

image_pipe = blocks.init_pipeline(model_id)  # no text encoder to load
image_pipe.load_components(dtype={"vae": torch.float32, "default": torch.bfloat16})
image_pipe.to("cuda")

for seed in range(3):
    image = image_pipe(
        prompt_embeds=prompt_embeds,
        negative_prompt_embeds=negative_prompt_embeds,
        generator=torch.Generator().manual_seed(seed),
        output="images",
    )[0]
    image.save(f"{seed}.png")

在循环中添加一个步骤 Add a step inside the loop

上方的去噪示意图来自一个额外模块，该模块会在每一步记录模型对干净图像的预测：The denoising figure above came from one extra block that records the model's guess of the clean image at every step:

from diffusers.modular_pipelines import Flux3AutoBlocks, InputParam, ModularPipelineBlocks, OutputParam


class Flux3PreviewStep(ModularPipelineBlocks):
    model_name = "flux3"

    @property
    def inputs(self):
        return [InputParam("latents"), InputParam("noise_pred")]

    @property
    def intermediate_outputs(self):
        return [OutputParam("x0_previews", type_hint=list)]

    @torch.no_grad()
    def __call__(self, components, block_state, i, t):
        if i == 0:
            block_state.x0_previews = []
        sigma = t / 1000  # timesteps are sigmas scaled to [0, 1000]
        block_state.x0_previews.append(block_state.latents - sigma * block_state.noise_pred)  # x0 = x_t - sigma * v
        return components, block_state


blocks = Flux3AutoBlocks()
denoise_loop = blocks.sub_blocks["denoise"].sub_blocks["text2image"].sub_blocks["denoise"]
denoise_loop.sub_blocks.insert("preview", Flux3PreviewStep(), 1)  # between "denoiser" and "after_denoiser"

pipe = blocks.init_pipeline(model_id)
pipe.load_components(dtype={"vae": torch.float32, "default": torch.bfloat16})
pipe.to("cuda")

out = pipe(prompt=prompt, generator=torch.Generator().manual_seed(0), output=["x0_previews", "images"])

试试看 Try it

• Models: FLUX.3 Image Klein collection
• Docs: FLUX.3 in Diffusers
• Fine-tune: DreamBooth LoRA scripts
• 提示词：Black Forest Labs 提示词指南
  Prompting: Black Forest Labs prompting guide

用 FLUX.3 做了什么作品吗？欢迎在 Hub 上分享你的 LoRA 和工作流。我们很期待看到它们！😎Built something with FLUX.3? Share your LoRAs and pipelines on the Hub. We'd love to see them! 😎