Granite Switch 4.1：单一端点的对话、决策与语音

Back to Articles

Introduced last spring, the Granite Switch project enables a modular approach to model development. Now Granite Switch 4.1 models, armed with a Decision adapter and the fastest ASR model on the OpenASR leaderboard, can move efficiently between natural-language responses, general and specialized decisions, and speech-to-text tasks—all from one hosted endpoint.

具体而言，Granite-Switch-4.1-8B 和 -30B 模型配置为包含：Specifically, Granite-Switch-4.1-8B and -30B models are configured to include:

1. 一个面向通用决策（例如 v1/systemone）的新 Decision aLoRA，
   A new Decision aLoRA for general-purpose decision-making (e.g. v1/systemone),
2. A specialized Guardian aLoRA that, on average, outperforms Jev 1.13, Microsoft-Decision-1, GPT-Luna-Decisions, and Clef-Flash on safety-related decision tasks, and
3. Granite-Speech-5.0-Turbo-CTC 是 OpenASR 排行榜上速度最快的 ASR 模型，已嵌入系统以用于转录任务。
   Granite-Speech-5.0-Turbo-CTC, the fastest ASR model on the OpenASR Leaderboard, embedded for use in transcription tasks.

这三项技术（连同底层基础模型）被整合到一个模型中，占用的 GPU 显存大约与基础模型单独运行时相同。这意味着用更少的 GPU 实现更多功能。All three of these technologies (along with the underlying base model) are composed into one model, taking up approximately the same amount of VRAM on a GPU as the base model would by itself. That means more capabilities with fewer GPUs.

而且，由于 Switch 适配器是以缓存高效的 Activated LoRA 构建的，因此基于基础模型生成内容时创建的 KV 缓存可以复用于之后任何基于适配器的决策。结合团队近期为加速低秩适配器推理而对 vLLM 做出的贡献，以及同机部署的 ASR 编码器，Granite Switch 4.1 模型能够以极快的速度在聊天请求、决策请求和语音请求之间切换。And because the Switch adapters were built as cache-efficient Activated LoRAs, the KV cache created from a base model generation can be reused for any subsequent adapter-based decisions. Combined with our team's recent vLLM contributions to speed up low rank adapter inferencing, as well as a co-located ASR encoder, Granite Switch 4.1 models can switch between a chat request and a decision request and a spoken request, with blazing fast speeds.

Granite-switch-4.1-8b 可转写音频、生成摘要、筛查有害内容，并测试 15 种以上通用标记——所有功能都能在单个端点上实时运行。Granite-switch-4.1-8b can transcribe audio, generate summaries, screen for harmful content, and test 15+ generic flags—all running on a single endpoint in real time.

Granite 库：最初的决策模型 Granite Libraries: The OG Decision Models

决策模型并不是什么新事物。我们此前发布的 Granite Libraries 中，包含一个专用“决策模型”适配器及其他专用任务适配器的仓库，这些适配器可以组合成自定义的 Granite Switch 模型。Jev 等闭源决策模型侧重通用性，而我们发现，在专门的分类任务中，许多专用的 granitelib 适配器表现优于通用模型。Decision models aren’t new. Our previous release of Granite Libraries features a repository of specialized “decision model” adapters and other specialized task adapters that can be composed into custom Granite Switch models. And while Jev and other closed decision models focus on generality, we’ve found that many of our specialized granitelib adapters can outperform generalists on specialized classification tasks.

比较 granitelib 适配器与通用型决策模型在专业分类任务上的表现。Compare granitelib adapters with generalist decision models when evaluated on specialized classification tasks.

由于适配器共享基础模型的 KV 缓存，因此在通用型决策适配器旁添加专用适配器几乎不增加成本。通用型适配器可提供广泛的任务覆盖，而专用适配器则能在你关注的任务上实现更高的准确率。Because adapters share the base model's KV cache, adding specialists alongside a generalist decision adapter costs almost nothing. You get broad coverage from the generalist and higher accuracy from specialists on the tasks you care about.

在 Granite-Switch-4.1-30B（冻结的 Granite 基础模型 + 4 个专用 aLoRA）与 Jev-27B（冻结的 Qwen 基础模型 + 1 个通用决策 LoRA）的“竞赛”中，Granite-Switch-4.1-30B 完成一个真实场景下的五步 RAG 工作流，速度远超对手。In a “Race” between Granite-Switch-4.1-30B (frozen base Granite + 4 specialized aLoRAs) and Jev-27B (frozen base Qwen model + 1 generic decision LoRA), Granite-Switch-4.1-30B was able to complete a real world, 5-step RAG workflow much faster than its competitor.

组合并扩展你自己的 Granite Switch 模型 Compose and Extend Your Own Granite Switch Model

Granite-Switch-4.1-8B 和 -30B 发布时就“配齐了所需组件”。它们是我们预先搭配了一组实用适配器编译而成的 Granite Switch 模型。不过，你不必使用我们的版本。任何人都可以从 Granite Libraries 中挑选并组合适配器，自行组装 Granite Switch 模型，以最契合自己的需求。Granite-Switch-4.1-8B and -30B are released with “batteries included”. They are Granite Switch models that we compiled with a predefined set of helpful adapters. But you don’t need to use our version. Anyone can compose their own Granite Switch model, mixing and matching adapters from Granite Libraries to best meet their needs.

展望未来，我们认为社区通过自定义适配器和第三方适配器来组合和扩展 Granite Switch 模型，拥有巨大的发展机遇。我们计划发布 Adapter SDK，以便在 Mellea 中训练适配器，同时还会提供其他教程和 Granite Switch 支持。这些适配器可以是专用分类适配器，用于增强基于决策的工作流，也可以用于特定语言任务。你甚至可以使用自定义适配器，教一个拥有 3B 参数的 Granite 版本玩经典游戏《毁灭战士》（Doom）。Moving forward, we see tremendous opportunity for the community to compose and extend Granite Switch models with custom and third party adapters. We plan to release an Adapter SDK for training adapters in Mellea, along side other tutorials and Granite Switch support. These adapters could be specialized classification adapters to augment decision-based workflows or adapters for specialized language tasks. You could even use custom adapters to teach a 3B parameter version of Granite to play the classic game Doom.

与 Jev 类似，我们定制的 30 亿参数 Granite Switch 模型也能在《毁灭战士》游戏中瞬间做出决策并进行导航，但它每秒能做出 60 至 100 次决策，而 Jev 报告的速度是每秒 10 次。1 与 Jev 不同的是，这款 Granite Switch 模型还可以在游戏过程中进行解说，并回答你的问题。Similarly to Jev, our custom 3B parameter Granite Switch model made split second decisions to navigate the Doom game, but was able to do so at 60 - 100 decisions / second, compared to Jev’s reported 10 decisions / second.¹ Unlike Jev, this Granite Switch model can also narrate and respond to your questions while it is playing.

We look forward to releasing more details and data on Granite Switch in the upcoming weeks, but until then, we encourage you to try out our latest Switch models and these other helpful resources for getting started.

入门资源 Resources for Getting Started

• Granite Switch 4.1 Model Cards
• 语音转决策演示
  Speech-to-Decisions Demo
• Granite Switch GitHub Repo
• Original Granite Switch and Granite Libraries Release Blog
• 在 Colab 上开展你自己的 aLoRA 与 LoRA 竞速对比。
  Create your own aLoRA v LoRA race on Colab.

---

脚注Footnotes

¹ https://typesafe.ai/blog/introducing-system-one-models-and-jev