Visko 如何在 Google Cloud TPU 上流式传输实时世界模型

编者按：生成式 AI 初创公司正迅速超越静态片段生成，转而打造交互式实时世界。在这篇技术深度解析中，我们将探讨 AI 初创公司 Visko 如何与 Google Cloud TPU 工程师合作，将其持续流式传输的 Live World Model——Orbis 部署到 Cloud TPU 上，从项目启动到移交生产环境仅用了 30 天。Editor's note: Generative AI startups are moving rapidly beyond static clip generation to build interactive, real-time worlds. In this technical deep dive, we explore how AI startup Visko collaborated with Google Cloud TPU engineers to deploy Orbis — its continuously streaming Live World Model—on Cloud TPUs, going from initial kickoff to production handover in just 30 days.

是什么让实时视频世界模型在生产环境中的服务如此具有挑战性？瓶颈并非平均吞吐量，而是必须不断赶上无情的实时节奏。与用户提交提示词后排队等待的离线视频生成器不同，实时世界模型会持续进行流式生成。每一段新视频及其同步音频都必须在观众看完上一段之前渲染完成，否则整个世界就会卡住。What makes a live video world model so challenging to serve in production? The bottleneck isn't average throughput — it's beating a relentless real-time clock. Unlike offline video generators where a user submits a prompt and waits in a queue, a live world model streams continuously. Every new chunk of video and synchronized audio must finish rendering before the viewer finishes watching the previous one, or the world freezes.

今天，我们很高兴与大家分享，专注于交互式世界模型的 AI 初创公司 Visko 如何与 Google Cloud 携手解决这一挑战。Visko 与 Google Cloud 的工程师并肩合作，将 Visko 端到端的实时推理流水线——融合视频生成、超分辨率放大和同步音频——部署到单个 Cloud TPU 切片上。双方团队将端到端生成速度提升至实时播放速度的 1.07 倍。Today, we're excited to share how Visko, an AI startup pioneering interactive world models, teamed up with Google Cloud to solve this exact challenge. Working side by side, Visko and Google Cloud engineers deployed Visko's end-to-end live inference pipeline — combining video generation, super-resolution upscaling, and synchronized audio—onto a single Cloud TPU slice. Together, the teams brought end-to-end generation down, rendering at 1.07x playback speed.

下面将深入介绍我们如何通过工程化打造这条流水线，在短短一个月内达成这一里程碑；哪些架构优化带来了显著成效；以及其他初创公司在 Google Cloud 上构建超低延迟生成式 AI 时可以借鉴的实用经验。Here is an inside look at how we engineered the pipeline to hit that milestone in a single month, the architectural optimizations that moved the needle, and practical lessons for other startups building ultra-low-latency generative AI on Google Cloud.

用实时世界模型重新定义交互式媒体Redefining interactive media with Live World Models

虽然大多数生成式视频平台专注于渲染预先制作好的固定片段，但 Visko 正在构建其所谓的“实时模型”：能够持续生成并即时响应的视频世界模型。While most generative video platforms focus on rendering fixed, pre-baked clips, Visko is building what it calls Live Models: video world models that generate continuously and respond on the fly.

Visko 的旗舰模型 Orbis 可根据单条文本或图像提示实时生成视频流。视频播放期间，观众可以实时输入新的引导提示，动态演变场景，无需重新开始，也不会破坏连贯性。例如，在一次测试中，直播以一只猫在噼啪作响的壁炉旁打盹开场；观众输入提示，在场景中加入一碗食物后，碗便自然地显现出来，猫也随之伸展身体、站起身来并喵喵叫。Visko's flagship model, Orbis, streams live video starting from a single text or image prompt. As the stream plays, viewers can enter new steering prompts in real time, dynamically evolving the scene without restarting or breaking continuity. For instance, in one test session, the livestream opens on a cat napping beside a crackling fireplace; when the viewer steers the scene to introduce a bowl of food, the bowl materializes seamlessly, and the cat stretches, stands up, and meows.

自 2026 年 9 月 1 日推出公开演示以来，Orbis 为互动娱乐、直播电商、AI 伴侣、沉浸式教育和机器人仿真开辟了新的可能性。为了可靠地扩展这些互动体验，Visko 于 2026 年 8 月与 Google Cloud TPU 工程师合作，将 Orbis 实时流水线移植到 Google 的张量处理单元（TPU）上。不到 30 天，完整的多模态流水线便已在 TPU 上实现实时运行。Since launching its public demo on September 1, 2026, Orbis has unlocked new possibilities across interactive entertainment, live commerce, AI companions, immersive education, and robotics simulation. To scale these interactive sessions reliably, Visko partnered with Google Cloud TPU engineers in August 2026 to port the Orbis live pipeline to Google's Tensor Processing Unit (TPU). Within 30 days, the full multimodal pipeline was running in real time on TPUs.

Visko 的愿景是为新一代互动世界提供动力，让人们能够置身其中，并实时塑造世界。要在严格的时序约束下持续传输视频和音频，与离线片段生成相比，是一种本质上不同的系统挑战。我们与 Google Cloud 的 TPU 工程师并肩合作，仅用 30 天就将完整的 Orbis 流水线迁移到了 TPU 上，为我们提供了所需的低延迟计算能力和内存带宽，从而能够大规模地将实时模型带给创作者和开发者。Visko’s ambition is to power a new generation of interactive worlds that people can step into and shape in real time. Streaming continuous video and audio under a strict clock is a fundamentally different system challenge than offline clip generation. Working side by side with Google Cloud's TPU engineers, we brought the full Orbis pipeline to TPUs in just 30 days — giving us the low-latency compute and memory bandwidth required to bring Live Models to creators and developers at scale.

“Visko 正在为 Cloud TPU 生态系统带来一类令人振奋的全新工作负载：实时且可操控。我们与他们共同在单个 TPU 切片上，使用 JAX、torchax 和自定义 Pallas 内核构建实时多模态流水线，证明了一支精干的初创团队能够多快地从 PyTorch 原型迈向以 1.07 倍播放速度进行渲染。”——Dave Elliott，Google Cloud AI 开发者倡导负责人"Visko is bringing an exciting new class of real-time, steerable workloads to the Cloud TPU ecosystem. By co-engineering their live multimodal pipeline on a singleTPU slice using JAX, torchax, and custom Pallas kernels, we proved how quickly a lean startup team can go from PyTorch prototype to render at 1.07x playback speed on Google Cloud's AI Hypercomputer."
– Dave Elliott, Head of Developer Advocacy for AI, Google Cloud 

实时视频是一个硬截止期限问题Live video is a hard-deadline problem

当你提供离线片段生成器时，延迟可以宽容一些——用户只需等待一次，让后端处理帧即可。而实时世界模型没有等待的余地。When you serve an offline clip generator, latency is forgiving — the user waits once while the backend processes frames. A live world model has no waiting room.

在整个会话期间，每个分块都必须经过降噪、从潜变量解码、放大至 1664×936（18 fps），并配上生成的音频，处理速度还必须快于实时播放速度。Every single chunk must be denoised, decoded from latents, upscaled to 1664x936 (at 18 fps), and paired with generated audio faster than realtime playback speed, for as long as the session runs.

两项结构性约束使实时世界模型的要求远高于标准批量推理：Two structural constraints make live world models far more demanding than standard batch inference:

• 批量大小为一：每条实时流都代表一位观众的独立交互会话。你无法通过将几十个并发请求合并成一个庞大批次来掩盖延迟——而这正是让 AI 加速器保持满负载运行的常规做法。

  Batch size of one: Each live stream represents an individual viewer's interactive session. You cannot mask latency by grouping dozens of concurrent requests into a massive batch — the standard playbook for keeping AI accelerators saturated.

• 严格的顺序依赖：每个分块都直接依赖前一个分块。由于视频模型会滚动保留近期视频潜变量的记忆，使分块 N+1 的动作能够从分块 N 平滑延续，因此不能乱序预渲染分块，也不能并行处理未来时间戳。

  Strict sequential dependency: Every chunk depends directly on the one before it. Because the video model maintains a rolling memory of recent video latents so that chunk N+1 smoothly continues the action from chunk N, you cannot pre-render chunks out of order or parallelize across future timestamps.

对于从事 LLM 服务的初创公司基础设施工程师来说，不妨将实时世界建模视为一项严苛的单步延迟 SLO，而不是 tokens-per-second 吞吐量基准。For startup infrastructure engineers coming from LLM serving, it helps to think of live world modeling not as a tokens-per-second throughput benchmark, but as an unforgiving per-step latency SLO.

在单个 TPU 切片上设计多模态流水线架构Architecting the multimodal pipeline on a single TPU slice

为了在 Visko 扩展的同时保持基础设施精简且具备成本效益，团队设计了实时流水线，在单个以 BF16 精度运行的 Cloud TPU 切片上串联三个不同的生成式模型。To keep infrastructure lean and cost-effective as Visko scales, the team designed the live pipeline to chain three distinct generative models on a single Cloud TPU slice running in BF16 precision.

当观众提交初始提示词或实时引导更新时，三个生成模型——核心视频扩散模型、超分辨率放大器和音频生成器——会共用同一批芯片，在处理每个片段时轮流使用硬件。When a viewer submits an initial prompt or a live steering update, all three generative models — the core video diffusion model, the super-resolution upscaler, and the audio generator — share the same chips, taking turns on the hardware for each chunk.

为了将快速的启动迭代与裸机加速器的速度结合起来，该软件栈基于三大开源支柱构建：To combine rapid startup iteration with bare-metal accelerator speed, the software stack is built on three open-source pillars:

• JAX：Google 面向加速器的高性能 Python 数组库，可由 XLA 编译器编译为优化的机器码。

  JAX: Google's high-performance Python array library for accelerators, compiled into optimized machine code by the XLA compiler.

• torchax：一个通过 JAX 直接在 Cloud TPU 上运行 PyTorch 模型的互操作层，使初创团队无需预先从头重写，就能迁移现有的 PyTorch 组件。

  torchax: An interoperability layer that runs PyTorch models directly on Cloud TPUs via JAX, enabling startup teams to bring existing PyTorch components over without an upfront, ground-up rewrite.

• Pallas：JAX 中用于 TPU 的自定义内核编程语言——相当于在 GPU 上编写自定义 Triton 或 CUDA 内核。

  Pallas: The custom kernel programming language for TPUs in JAX — the TPU counterpart to writing custom Triton or CUDA kernels on GPUs.

实时性能细分：精准把握时机Real-time performance breakdown: Hitting the clock 

通过在 BF16 下对单个切片的各个阶段进行性能剖析和优化，工程团队将每个数据块的总延迟控制在目标值以下，同时保留约 7% 的余量，以应对实时流传输期间的波动：By profiling and optimizing each stage on a single slice in BF16, the engineering team achieved a total per-chunk latency of under the target while — maintaining roughly 7% headroom to absorb variance during live streaming:

由于视频去噪占总计算预算的近三分之二，因此该阶段成为我们优化内核和并行性的重点。Because video denoising accounts for nearly two-thirds of the total compute budget, that stage became the primary focus of our kernel and parallelism optimizations.

深入技术栈：实现实时流式传输的三项优化Inside the stack: Three optimizations that unlocked real-time streaming

1. 在所有 TPU 芯片上进行 Ulysses 上下文并行1. Ulysses context parallelism across all TPU chips

一个视频块在视频 Transformer 内展开为大量时空 token。为了在 batch size 为 1 时将这项工作负载分布到各个 TPU 芯片上，团队实现了 Ulysses 上下文并行。A video chunk expands into a massive sequence of spatiotemporal tokens inside the video transformer. To distribute that workload across the TPU chips at batch size 1, the team implemented Ulysses context parallelism.

Ulysses 不会拆分模型权重，而是将 token 序列分配到不同芯片上。在注意力层之前，高速全互连交换会重新组合张量，使每块芯片都持有完整的视频序列，但只负责四分之一的注意力头。每块芯片针对所分配的注意力头，在完整序列上计算注意力后，第二次全互连交换会恢复按序列拆分的布局，以供前馈层使用。Instead of splitting model weights, Ulysses splits the token sequence across chips. Immediately before the attention layer, a high-speed all-to-all interconnect exchange regroups the tensors so that each chip holds the entire video sequence, but only for one-quarter of the attention heads. After each chip computes attention over the full sequence for its assigned heads, a second all-to-all exchange restores the sequence-split layout for the feed-forward layers.

2. Custom Pallas splash attention from MaxDiffusion

为了加速注意力计算本身，团队集成了来自 MaxDiffusion 的自定义 Pallas Splash Attention 内核。MaxDiffusion 是 Google Cloud 面向 Cloud TPU 扩散模型的开源参考库。To accelerate the attention computation itself, the team integrated a custom Pallas splash attention kernel from MaxDiffusion, Google Cloud's open-source reference library for diffusion models on Cloud TPU.

Splash Attention 将 FlashAttention 式分块引入 TPU 架构：它不会在高带宽内存（HBM）中构建完整的二次方注意力矩阵，而是在紧凑的分块中计算注意力，并将这些分块保留在超高速片上 VMEM 中。初创团队可以通过 custom_splash_attention.py，将此移植版本使用的精确内核直接集成到自己的流水线中。Splash attention brings FlashAttention-style tiling to TPU architecture: rather than materializing the full quadratic attention matrix in High Bandwidth Memory (HBM), it computes attention in compact tiles that remain inside ultra-fast on-chip VMEM. Startup teams can drop the exact kernel used in this port straight into their own pipelines via custom_splash_attention.py.

3. 从 torchax 迁移到纯 JAX，音频速度提升 5.8 倍3. A 5.8x audio speedup by moving from torchax to pure JAX

我们首先通过 *torchax* 将全部三个 Orbis 模型迁移到 Cloud TPU 上。*torchax* 借助 JAX 和 XLA 在 TPU 上运行 PyTorch 代码，无需重写。对于视频和超分辨率模型，`torchax` 搭配 Pallas 注意力内核后，直接达到了我们的延迟目标；在最终流水线中，这两个模型也都继续使用 `torchax`。We first brought all three Orbis models onto Cloud TPU with *torchax*, which runs PyTorch code on TPU through JAX and XLA without a rewrite. For the video and upscaling models, `torchax` paired with Pallas attention kernels met our latency target directly, and both stay in `torchax` in the final pipeline.

只有音频模型需要重写。将音频阶段从 `torchax` 迁移到*原生 JAX*，使其速度提升了 *5.8 倍*，并让整个流水线能够在播放时限内完成。Only the audio model needed a rewrite. Moving the audio stage from `torchax` to *native JAX* made it *5.8x faster* and brought the full pipeline under the playback deadline.

初创公司构建技巧：有策略地移植。重点不是“跳过 torchax”，而是把它用作快速启动的桥梁。torchax 让我们的 PyTorch 模型在几天而非几周内就能在 TPU 上运行，为我们提供了端到端遥测数据，从而精准定位真正的瓶颈。先测量完整流水线，再投入工程资源，只重写那些占据延迟预算大头的组件。Startup Builder Tip: Port strategically.
The takeaway isn't "skip torchax" — it's to use it as a fast bring-up bridge. torchax got our PyTorch models running on TPU in days rather than weeks, giving us the end-to-end telemetry needed to pinpoint true bottlenecks. Measure the full pipeline first, and invest engineering cycles rewriting only the components that dominate your latency budget.

给 AI 构建者的实地笔记：Cloud TPU 经验教训Field notes for AI builders: Cloud TPU lessons

在 Cloud TPU 上调优实时多模态推理的要点：Takeaways from tuning live multimodal inference on Cloud TPU:

- *单路视频的 Ulysses 上下文并行：* 在批量大小为 1 时，通过 `all-to-all` 互连交换将 token 序列拆分到多个芯片上，速度比张量并行和环形注意力都快。- *Ulysses context parallelism for single stream video:* At batch size 1, splitting the token sequence across chips with `all-to-all` interconnect exchanges was faster than both tensor parallelism and ring attention.

- *连续流的原生 BF16：* FP8 量化可降低内存带宽压力，但在连续流中，微小的量化误差会随着数据块不断累积。原生 BF16 在满足延迟预算的同时，保持了视觉质量。- *Native BF16 for continuous streams:* FP8 quantization cuts memory bandwidth pressure, but small quantization errors build up across chunks in a continuous stream. Native BF16 kept visual quality intact within the latency budget.

- *选择性移植到 JAX：* `torchax` 保留了用 PyTorch 实现的视频和超分辨率模型，因此团队只将音频瓶颈部分重写为原生 JAX。- *Selective JAX ports:* `torchax` kept the video and upscaling models in PyTorch, so the team only rewrote the audio bottleneck in native JAX.

- *XLA 编译缓存：*首次启动时需要花几分钟编译 XLA 计算图。缓存编译产物后，工作进程重启时间可缩短至一分钟以内。- *XLA compilation caching:* The first boot spends a few minutes compiling XLA graphs. Caching compiled artifacts brings worker restarts under a minute.

在 Cloud TPU 上构建实时生成模型？从这里开始Building real-time generative models on Cloud TPU? Start here

如果你的初创公司正在突破实时视频、世界模型或超低延迟多模态推理的边界，你可以使用同一套工具进行构建——它曾为 Visko 的移植提供支持：If your startup is pushing the boundaries of live video, world models, or ultra-low-latency multimodal inference, you can build with the same toolkit that powered Visko's port:

• 开箱即用地加速扩散模型的注意力计算：从 MaxDiffusion 获取此次合作中使用的开源 Splash Attention 内核，并查阅 Pallas 文档以编写自定义 TPU 内核。

  Accelerate diffusion attention out of the box: Grab the open-source splash attention kernel used in this collaboration from MaxDiffusion, and explore the Pallas documentation to write custom TPU kernels.

• 快速迁移 PyTorch，再有针对性地优化：使用 torchax 快速将现有 PyTorch 模型迁移到 TPU，通过 Cloud TPU Profiler 精准定位瓶颈，并将对延迟敏感的阶段移植到原生 JAX。

  Migrate PyTorch fast, then optimize selectively: Bring your existing PyTorch models onto TPU quickly with torchax, pinpoint bottlenecks with the Cloud TPU Profiler, and port latency-critical stages to native JAX.

• 按需启动 TPU：使用 TPU Flex-start 虚拟机为短期开发和基准测试任务配置 TPU 容量，无需长期预留。

  Spin up TPUs on demand: Use TPU Flex-start VMs to provision TPU capacity for short development and benchmarking runs without a long-term reservation.

• 借助 Google for Startups 实现规模化：寻求云额度、技术指导以及使用 Google Cloud AI Hypercomputer 技术栈的早期 AI 初创公司，可以申请 Google for Startups Cloud Program。

  Scale with Google for Startups: Early-stage AI startups looking for cloud credits, technical mentorship, and access to Google Cloud's AI Hypercomputer stack can apply to the Google for Startups Cloud Program.

特别感谢 Visko.ai 团队的紧密技术合作，也感谢主导该项目的 Google Cloud TPU 工程师，包括：Yixin Shi、James Huang 和 Elisa Tsai。^{Special thanks to the Visko.ai team for the close technical partnership, and to the Google Cloud TPU engineers who lead the project including: Yixin Shi, James Huang, and Elisa Tsai.}