Reflection Beam 在 Google Cloud 上实现 Day-0 支持与 5 倍性能提升

今天，Reflection 发布了 Beam，这是其首个开放权重模型，旨在为开发者、研究人员和企业提供开放的控制与定制能力，以应对前沿软件工程和长时程智能体任务。Beam 采用宽松的 Apache 2.0 许可证发布，并在发布时提供 BF16、FP8 和 NVFP4 检查点。它通过解决复杂的推理和编程问题，在 5000 亿参数级别树立了推理效率的新标准：与同类开放模型相比，其推理计算效率高出 3 至 4 倍。Today, Reflection released Beam, its first open-weight model built to give developers, researchers, and enterprises open control and customization for frontier software engineering and long-horizon agentic tasks. Released under a permissive Apache 2.0 license with BF16, FP8, and NVFP4 checkpoints available at launch, Beam sets a new standard for inference efficiency in the 500B-parameter class by solving complex reasoning and coding problems with 3x to 4x higher reasoning compute efficiency than comparable open models.

伴随今日版本发布，Google Cloud 宣布从首日起支持在 Google Cloud AI Hypercomputer 和 NVIDIA AI Infrastructure 上提供 Beam 服务，并推出可直接运行的部署方案。Alongside today’s release, Google Cloud is announcing day 0 support and ready-to-run deployment recipes for serving Beam on Google Cloud AI Hypercomputer and NVIDIA AI Infrastructure.

为了帮助开发者在生产环境中充分发挥 Beam 的 token 效率优势，我们与 Reflection 合作，在 Google Cloud 上使用 llm-d（搭配 vLLM）和 NVIDIA Dynamo（搭配 SGLang）优化底层服务栈的性能：To help developers get the full benefit of Beam’s token efficiency in production, we worked with Reflection to optimize performance of the underlying serving stack using llm-d (with vLLM) and NVIDIA Dynamo (with SGLang) on Google Cloud:

“通过在 Google Cloud 上部署我们的 Beam 模型，我们实现了最高 5 倍的性能提升和 8 倍的延迟降低，从而优化了智能体工作负载的服务。”——Joe Spisak，Reflection 产品副总裁兼开源负责人"By deploying our Beam model on Google Cloud, we achieved up to 5x higher performance and 8x lower latency to optimize serving of agentic workloads." - Joe Spisak, VP of Product & Head of Open Source, Reflection

随着 Google Cloud 今日发布 Day 0 版本，开发者将获得：With today’s day 0 release on Google Cloud, developers get:

• 复合模型与系统效率：将 Beam 3 至 4 倍的推理计算效率与 A4X Max 实例（NVIDIA GB300 NVL72）上的系统级优化相结合，可实现最高 5 倍的吞吐量和 8 倍的低延迟；*而 G4 虚拟机（NVIDIA RTX PRO 6000）则可实现最高 2 倍的吞吐量和 8 倍的低延迟。*

  Compounding model and system efficiency: Combining Beam’s 3x to 4x reasoning compute efficiency with system-level optimizations on A4X Max instances (NVIDIA GB300 NVL72) for up to 5x higher throughput and 8x lower latency,* and G4 VMs (NVIDIA RTX PRO 6000) for up to 2x higher throughput and 8x lower latency.*

• 高效多轮服务：KV 缓存感知路由、预填充与解码分离、推测解码，以及将分层 KV 缓存卸载到主机 DRAM 和本地 SSD，可减少长时间编码会话中的提示词重复计算。

  High-efficiency multi-turn serving: KV-cache-aware routing, prefill/decode disaggregation, speculative decoding and hierarchical KV cache offloading to Host DRAM and Local SSD reduce redundant prompt recomputation across long coding sessions.

• 开箱即用的优化部署：我们为 agentic 工作负载推荐的性能优化方案，现已发布在 AI-Hypercomputer/gpu-recipes、llm-d 仓库和 Gemini 平台的 Model Garden 中，只需点击几下即可使用。

  Turnkey optimized deployment: Our recommended performance optimized recipes for agentic workloads with llm-d and NVIDIA Dynamo are available in AI-Hypercomputer/gpu-recipes, llm-d repo and Model Garden in Gemini platform with a few clicks. 

*基于客户基准测试*Based on customer benchmarking

扩展推理规模需要进行系统级优化Scaling inference requires system-level optimization

在评估模型的智能体式编程和工具使用能力时，推理效率体现在两个不同层面：模型推理的效率，以及服务系统执行多轮循环的效率。When evaluating models for agentic coding and tool use, inference efficiency happens at two distinct layers: how efficiently the model reasons and how efficiently the serving system executes multi-turn loops.

模型层级效率：Reflection 的 BeamModel-level efficiency: Reflection’s Beam

Reflection 专门为文本原生软件工程和多步骤工具调用设计了 Beam（总参数量约 501B，每个 token 激活约 23B 参数，上下文窗口约 100 万 token）。它可直接接入开发者已在使用的编码框架，包括 Reflection 的 Mirror CLI、Claude Code、Hermes、OpenCode，以及 ，从而处理 GitHub issue、运行 CLI 和 DevOps 工作流，并完成长期任务。Reflection designed Beam (~501B total parameters with ~23B active parameters per token, and an approximately 1M-token context window) specifically for text-native software engineering and multi-step tool use. It plugs directly into the coding harnesses developers already use, including Reflection’s Mirror CLI, Claude Code, Hermes, OpenCode, and , to resolve GitHub issues, run CLI and DevOps workflows, and complete long-horizon tasks.

由于 Beam 每个 token 仅激活 230 亿个参数，并且相比其他同类开放权重模型，达到解决方案所需的推理 token 数减少至三分之一到四分之一，因此它大幅降低了每个推理步骤所需的推理时计算量。Because Beam activates only 23B parameters per token and reaches solutions using 3x to 4x fewer reasoning tokens than other comparable open weight models, it dramatically reduces the inference-time compute required for each reasoning step.

系统级效率：针对 AI Hypercomputer 优化的服务部署System-level efficiency: AI Hypercomputer optimized serving

开发者将智能体部署到生产环境后，服务层可能会出现另一组瓶颈。在智能体编码循环中，模型在单次会话里会被调用数十次。每一轮中，编码框架都会将新的文件读取内容、代码差异、编译器日志和工具输出追加到共享的对话历史中，而这段历史很快就会膨胀到数万甚至数十万个 token。Once developers deploy an agent into production, a second set of bottlenecks could appear at the serving layer. In an agentic coding loop, the model is called dozens of times in a single session. With every turn, the coding harness appends new file reads, code diffs, compiler logs, and tool outputs to a shared conversation history that quickly grows to tens or hundreds of thousands of tokens.

即使模型本身每轮生成的推理 token 减少了 3 到 4 倍，跨多轮累积的上下文窗口仍会带来两大基础设施挑战：Even when the model itself generates 3x to 4x fewer reasoning tokens on each turn, the accumulated context window across turns creates two major infrastructure challenges:

首先，预填充计算会成为瓶颈。标准 Kubernetes 负载均衡器使用无状态轮询路由来分发请求。当编码执行环境发送会话中的第 15 轮请求时，该请求往往会被分配到与第 14 轮不同的副本上。这个副本不得不从头重新计算整个历史提示词，浪费 GPU 算力，并造成漫长的预填充队列，进而延误其他用户的请求。First, prefill compute becomes a bottleneck. Standard Kubernetes load balancers distribute requests using stateless round-robin routing. When a coding harness sends Turn 15 of a session, it often lands on a different replica than Turn 14. That replica is forced to recompute the entire historical prompt from scratch, wasting GPU compute and creating long prefill queues that delay other users.

其次，解码阶段会受 KV 缓存容量限制。为数十个并发运行的智能体会话保留长上下文 KV 缓存，很快就会占满 GPU 高带宽内存（HBM）。HBM 一旦占满，服务引擎就不得不淘汰较早的 KV 块。块一经淘汰，缓存命中率便会下降，GPU 吞吐量也会停滞。Second, decode becomes bottlenecked by KV cache capacity. Holding long-context KV caches for dozens of concurrent agent sessions quickly fills up GPU High-Bandwidth Memory (HBM). Once HBM fills up, the serving engine has to evict older KV blocks. As soon as those blocks are evicted, cache hit rates drop and GPU throughput stalls.

为了在集群规模下保留 Beam 在模型层面的高效性，服务栈必须按照分布式系统的方式进行优化。因此，我们的方案将 Beam 分别与 llm-d（搭配 vLLM）和 NVIDIA Dynamo（搭配 SGLang）结合使用。To preserve Beam’s model-level efficiency at cluster scale, the serving stack has to be optimized as a distributed system. That is why our recipes pair Beam with llm-d (with vLLM) and NVIDIA Dynamo (with SGLang).

在 AI Hypercomputer 推理架构内部Inside the AI Hypercomputer inference architecture

为了在 Day 0 以最高效率运行 Beam 的 BF16、FP8 和 NVFP4 检查点，我们在 AI Hypercomputer 上设计了垂直整合的架构，其中每一层都负责智能体服务生命周期中的特定环节。To run Beam’s BF16, FP8, and NVFP4 checkpoints at peak efficiency on Day 0, we designed a vertically integrated architecture on AI Hypercomputer where each layer handles a specific part of the agentic serving lifecycle.

不是将推理 Pod 视为无状态端点，而是围绕多轮智能体的生命周期对技术栈的每一层进行协同设计：Rather than treating inference pods as stateless endpoints, each layer of the stack is co-designed around the multi-turn agent lifecycle:

• 前缀感知与负载感知路由（GKE + llm-d EPP / Dynamo 路由器）：跟踪各工作 Pod 中缓存的前缀和队列深度，将编码框架发起的每个后续回合引导至持有其热 KV 前缀的副本，同时通过回合优先级流控防止队列过载。

  Prefix- and load-aware routing (GKE + llm-d EPP / Dynamo router): Tracks cached prefixes and queue depth across worker pods, steering each follow-up turn from a coding harness to the replica holding its warm KV prefix while preventing queue saturation via turn-priority flow control.

• 解耦式计算与推测解码（A4X Max 和 G4 实例）：运行 vLLM 和 SGLang，支持原生 Blackwell NVFP4、针对 Beam 的 23B 活跃专家的专家并行（EP）、数据并行注意力，以及 DFlash2 推测解码。将 Prefill 和 Decode（P/D）资源池分离，确保大量提示词突发请求不会阻塞正在进行的 token 生成。

  Disaggregated compute and speculative decoding (A4X Max & G4 instance): Runs vLLM and SGLang with native Blackwell NVFP4, Expert Parallelism (EP) for Beam’s 23B active experts, Data-Parallel Attention, and DFlash2 speculative decoding. Separating Prefill and Decode (P/D) pools ensures heavy prompt bursts never stall active token generation.

• 分层 KV 缓存（GPU 内存 -> DRAM 和本地 SSD）：活跃的 KV 块保留在高带宽 GPU 内存（第 1 层）中，而长时段会话期间被逐出的温热 KV 块则通过 vLLM 的 OffloadingConnector 或 SGLang 的 HiCache 溢出到主机 CPU DRAM 和本地 NVMe SSD（第 2 层），从而可以在毫秒内取回历史上下文，而不必从头重新计算。

  Tiered KV cache (GPU Memory -> DRAM & Local SSD) Active KV blocks stay in high-bandwidth GPU memory (Tier 1), while warm KV blocks evicted during long-horizon sessions overflow into host CPU DRAM and local NVMe SSDs (Tier 2) via vLLM’s OffloadingConnector or SGLang's HiCache, allowing historical context to be fetched in milliseconds instead of recomputed from scratch.

多轮智能体工作负载的系统优化方案System optimizations for multi-turn agentic workloads

在逐步提高并发级别的情况下，为 AgentX 轨迹数据集这类多轮编码和工具调用工作负载提供服务时，服务栈中的每一层都能提升智能体工作负载中多轮、长上下文流量模式的性能并降低延迟：When serving multi-turn coding and tool-use workloads like the AgentX trajectory dataset across rising concurrency levels, each layer in the serving stack contributes to the performance and latency improvements for multi-turn, long context traffic patterns of agentic workloads:

关于如何解读这些图表的说明：这两种配置彼此不可比较。NVIDIA Dynamo 运行在 A4X Max 实例上，使用 32 块 NVIDIA GB300 NVL72 GPU；llm-d 运行在 G4 虚拟机上，使用 8 块 NVIDIA RTX PRO 6000 GPU。每张图表都显示了相对于同一硬件基线的性能提升。副本越多，路由器可命中的预热 KV 缓存就越多，也有更多空间将预填充与解码拆分开来，因此随着部署规模扩大，性能提升也会随之增加。llm-d 和 NVIDIA Dynamo 都会根据底层基础设施进行扩缩容。A note on reading these charts. The two configurations are not comparable to each other. NVIDIA Dynamo ran on A4X Max instances with 32 x NVIDIA GB300 NVL72GPUs; llm-d ran on G4 VMs with 8 x NVIDIA RTX PRO 6000 GPUs. Each chart shows the gain over a baseline on the same hardware. More replicas give the router more warm KV caches to hit and more room to split prefill from decode, so the uplift grows with the deployment. Both llm-d and NVIDIA Dynamo scale up and down to match the underlying infrastructure.

在这两个平台上，层层叠加这些优化可推动帕累托前沿向外扩展，同时提升整体输出吞吐量并加快每位用户的交互响应速度：Across both platforms, layering these optimizations shifts the Pareto frontier to deliver higher output throughput and faster per-user interactivity simultaneously:

• 基线部署（无状态轮询路由）。在不具备前缀感知能力的标准 Kubernetes 部署中，同一编程会话的连续轮次会被分配到不同副本，导致历史上下文不断重新进行预填充计算，并使吞吐量随着并发数增加而趋于停滞。

  Baseline deployment (Stateless round-robin routing). In a standard Kubernetes deployment without prefix awareness, consecutive turns from the same coding session land on different replicas, forcing constant prefill recomputation on historical context and causing throughput to plateau as concurrency scales.

• 支持 KV-cache 感知与流量控制的路由。在 NVIDIA Dynamo 或 llm-d 的 Sticky EPP 中启用前缀感知路由，并配合按轮次优先的流量控制，可让多轮智能体轨迹固定路由到持有其预热 KV 缓存的副本，从而立即降低首个 Token 延迟（TTFT）、释放 GPU 算力，并显著提升 Token 吞吐量。

  KV-cache-aware & flow control routing. Enabling prefix-aware routing in NVIDIA Dynamo or llm-d’s Sticky EPP with turn-priority flow control keeps multi-turn agent trajectories pinned to the replica holding their warm KV cache, immediately cutting Time-to-First-Token (TTFT) latency and freeing up GPU compute, resulting in a significant increase in token throughput.

• 主机 DRAM 和本地 SSD（LSSD）KV 卸载。将逐出的 KV 块从 GPU 内存卸载到主机 DRAM 和本地 NVMe SSD，可避免数十个长上下文编码会话超出设备 GPU 内存容量时发生缓存抖动。

  Host DRAM and Local SSD (LSSD) KV offloading. Offloading evicted KV blocks from GPU memory into host DRAM and local NVMe SSDs prevents cache thrashing when dozens of long-context coding sessions exceed on-device GPU memory capacity.

• 分离式服务、推测解码和 KV 池调优。通过调优混合 KV 缓存的内存分配、启用推测解码，并将预填充与解码工作进程解耦，可以将令牌生成与繁重的提示词突发负载隔离开来；同时，草稿模型可大幅提升输出吞吐量，让每个智能体都能快速进行令牌交互。

  Disaggregated serving, speculative decoding, and KV pool tuning. Tuning hybrid KV cache memory allocation, enabling speculative decoding, and disaggregating prefill and decode workers isolates token generation from heavy prompt bursts while the draft model multiplies output throughput to keep per-agent token interactivity fast.

立即开始，并在 PyTorchCon 和 GTC Berlin 与我们见面Get started and meet us at PyTorchCon and GTC Berlin 

这篇博客探讨了如何利用系统级优化来提升性能，从而为智能体工作负载提供 Reflection 的 Beam 系列模型服务。本文中的所有 Kubernetes 清单、路由器配置和基准测试脚本均已开源，可直接在你自己的 GKE 集群上运行。This blog explored how you can leverage system level optimizations to scale performance to serve Reflection's Beam family of models for agentic workloads. All of the Kubernetes manifests, router configurations, and benchmarking scripts from this post are open-source and ready to run on your own GKE cluster.

• Deploy the optimized stack: Explore the day 0 guides for running Reflection’s Beam with llm-d and NVIDIA Dynamo in the AI-Hypercomputer/gpu-recipes repository on A4X Max and G4, and Model Garden in Gemini platform.
• 下载权重：前往 Reflection 的 Hugging Face 页面获取 Apache 2.0 许可的 BF16、FP8 和 NVFP4 检查点。
  Download the weights: Grab the Apache 2.0 BF16, FP8, and NVFP4 checkpoints on Reflection’s Hugging Face page.
• 欢迎在圣何塞参加 llm-d 和 PyTorch Conference：如果您今年 10 月将参加在圣何塞举办的 PyTorch Conference North America，欢迎前往 llm-d 社区专题活动和 Google Cloud 展位 #。我们将现场演示适配 KV 缓存的路由、P/D 解耦，以及面向智能体工作负载的 KV 缓存分层。
  Visit us at the llm-d and PyTorch Conference in San Jose: If you are attending PyTorch Conference North America in San Jose this October, stop by the llm-d community sessions and the Google Cloud booth #. We will be walking through live demos of KV-cache-aware routing, P/D disaggregation, and KV cache tiering for agentic workloads.
• 欢迎在 NVIDIA GTC Berlin 与我们见面：如果您将参加 NVIDIA GTC Berlin，欢迎莅临我们的 3026 号展位，或前往位于 w3.hub 的 Google Cloud Hub 与我们交流，那里距离会场仅需步行 5 分钟。了解更多关于我们专题活动的信息。
  Come see us at NVIDIA GTC Berlin: If you are attending NVIDIA GTC Berlin, come connect with us at our booth #3026, or our Google Cloud Hub at w3.hub: just a short 5 minute walk from the venue. Learn more about our sessions.