ReplaySSM: 重新思考状态存储以加速混合模型解码

混合模型将状态空间模型（SSM）层与少量注意力层交错排列。NVIDIA Nemotron 3、Qwen3.5 到 Qwen3.8、Kimi-K3 以及 GLM-5.3-Flash 都采用了这种架构，其中 SSM 层使用 Mamba-2、门控 DeltaNet（GDN）或 Kimi Delta Attention（KDA）。Hybrid models interleave state-space model (SSM) layers with a few attention layers. NVIDIA Nemotron 3, Qwen3.5 through Qwen3.8, Kimi-K3, and GLM-5.3-Flash are all built this way, with Mamba-2, Gated DeltaNet (GDN), or Kimi Delta Attention (KDA) as the SSM layer.

SSM 层旨在加快解码速度：它们用固定大小的状态取代了会随上下文长度增长的键值（KV）缓存。然而，在服务端批量大小下，直到上下文长度达到 100K 个 token，延迟瓶颈都可能出在 SSM 层，而非注意力层。SSM 层还使两种能够可靠加速注意力解码的技术更难实现：推测解码和量化。SSM layers are designed to make decoding fast: they use a fixed-size state in place of a key-value (KV) cache that grows with context length. At serving batch sizes, however, the SSM layers, not attention, can be the latency bottleneck up to 100K tokens of context. SSM layers also complicate two techniques that reliably speed up attention decoding: speculative decoding and quantization.

ReplaySSM 让 SSM 层再次成为快速环节。它将 SSM 状态表示为一个检查点和一个近期输入缓冲区。这将输出计算与状态更新分离，并解决当前 SSM 解码中的三个挑战：状态数据传输、推测解码的回滚，以及量化状态中的误差累积。ReplaySSM makes SSM layers the fast part again. It represents the SSM state as a checkpoint plus a buffer of recent inputs. This separates output computation from state updates and addresses three challenges in current SSM decoding: state traffic, rollback for speculative decoding, and error accumulation in a quantized state.

ReplaySSM 是 NVIDIA 与普林斯顿大学 Dao AI Lab 的合作成果。论文《ReplaySSM：缓存 SSM 输入，而非状态》详细介绍了这三项优势，并展示了完整的评估结果。论文已发布在 arXiv 上，代码托管于 GitHub。ReplaySSM 已集成到 NVIDIA TensorRT-LLM、vLLM 和 FlashInfer 中，并已用于 GLM-5.3-Flash 和 Kimi-K3 的生产环境。ReplaySSM is joint work between NVIDIA and the Dao AI Lab at Princeton. The paper, ReplaySSM: Cache SSM Inputs, Not State, details all three benefits and presents the full evaluation. It is available on arXiv, with code on GitHub. It is integrated into NVIDIA TensorRT-LLM, vLLM, and FlashInfer, and also runs in production in GLM-5.3-Flash and Kimi-K3.

为什么 SSM 层可能成为解码瓶颈Why SSM layers can become a decoding bottleneck

以 NVIDIA Nemotron 3 Ultra 中使用的 Mamba-2 层为例。它通过以下递推关系解码一个 token：Take the Mamba-2 layer used in NVIDIA Nemotron 3 Ultra as an example. It decodes one token with the following recurrence:

其中，St S_t 是状态，αt \alpha_t 是依赖于数据的衰减项，而 vt v_t、kt k_t 和 qt q_t 是该 token 的输入。每一步都会从高带宽内存（HBM）中加载 St−1 S_{t-1}，应用衰减和秩为 1 的更新，使用 qt q_t 计算输出，然后将 St S_t 写回。GDN 和 KDA 会添加一个 delta-rule 修正项，但它们的更新也可以写成相同的外积形式，因此下文同样适用于它们。where St S_t is the state, αt \alpha_t is a data-dependent decay, and vt v_t , kt k_t , and qt q_t are the token’s inputs. Each step loads St−1 S_{t-1} from high-bandwidth memory (HBM), applies the decay and a rank-1 update, computes the output with qt q_t , and stores St S_t back. GDN and KDA add a delta-rule correction, but their updates can be written in the same outer-product form, so what follows applies to them too.

在解码过程中，状态承担两项任务：它既用于生成当前 token 的输出，也会更新为下一个 token 的状态。传统解码会同时完成这两项工作：每一步都会更新状态，从中读取输出，并将其保存起来供下一步使用。所有这三项挑战都源于这两项工作的融合。During decoding, the state does two jobs. It is read to produce this token’s output, and it is updated to become the next token’s state. Conventional decoding does both at once: each step updates the state, reads the output from it, and stores it for the next step. All three challenges come from this fusion.

状态内存流量State memory traffic

SSM 解码步骤会加载并存储完整状态，而对其进行的算术运算很少，因此该内核受限于 HBM 带宽，而非计算能力。在服务场景的批量大小下，在单张 NVIDIA Blackwell Ultra B300 GPU 上以批量大小 256 运行 Nemotron-3-Super-NVFP4 时，直到 100K 个 token，SSM 层都是解码步骤中最大的组成部分，而非注意力层（图 2，左）。An SSM decoding step loads and stores the full state while performing little arithmetic on it, so the kernel is bound by HBM bandwidth, not compute. At serving batch sizes, that makes the SSM layers, not attention, the largest component of the decode step up to 100K tokens on Nemotron-3-Super-NVFP4 at batch size 256 on one NVIDIA Blackwell Ultra B300 GPU (Figure 2, left).

这由两个因素驱动：Two factors drive this:

• 层数：该模型的 SSM 层数量是注意力层数量的五倍。
  Layer count: The model has five times as many SSM layers as attention layers.
• 状态大小：每个序列中一层的 FP32 状态为 4 MiB，而每个 token 的 FP8 KV 条目为 0.5 KiB，因此，一个 SSM 层的状态大小相当于一个注意力层中 8K 个 token 的 KV 缓存。
  State size: One layer’s FP32 state is 4 MiB per sequence, while an FP8 KV entry is 0.5 KiB per token, so an SSM layer’s state is the size of an attention layer’s KV cache for 8K tokens.

投机解码的回滚Rollback for speculative decoding

推测解码会让草稿模型提出 T 个 token，再通过目标模型一次性验证，并回滚到最后一个被接受的 token。注意力机制只需移动 KV 缓存指针即可轻松回滚。但 SSM 将草稿汇总到其状态后，更新就难以逆转，因此回滚的代价很高。vLLM 等服务引擎通过在每个草稿 token 后存储一个状态快照来绕过这一问题，然后选取与已接受前缀匹配的快照。这会在受内存带宽限制的路径上，每步产生一次读取和 T 次写入，而且这些快照会限制 GPU 内存能够容纳的请求数量。因此，推测解码虽是 Transformer 的可靠加速手段，在服务批次大小下却几乎无法提升 SSM 的性能。Speculative decoding has a drafter propose T tokens, verifies them in one pass through the target model, and rolls back to the last accepted token. Attention naturally supports rollback by moving the KV cache pointer. But once an SSM has summarized the drafts into its state, the update has no easy inverse, so rollback becomes expensive. Serving engines such as vLLM work around this by storing a state snapshot after every draft token and picking the one that matches the accepted prefix. That is one read and T writes per step on a memory-bound path, and the snapshots cap how many requests fit in GPU memory. As a result, speculative decoding, a reliable speedup for Transformers, barely helps SSMs at serving batch sizes.

状态量化State quantization

量化后的状态会生成输出，并作为下一次更新的输入，因此其舍入误差会在每一步重新进入递归过程。KV 缓存不存在这样的循环：量化后的条目会被读取多次，但不会再写回自身，这就是 KV 缓存能够容忍比 SSM 状态更低精度的原因。例如，Qwen3.5-122B-A10B-NVFP4 的权重采用 NVIDIA NVFP4 4 位浮点格式，同时使用 FP8 KV 缓存和 FP32 SSM 状态。The quantized state produces the output and is the input to the next update, so its rounding error re-enters the recurrence at every step. A KV cache has no such loop: a quantized entry is read many times but never written back into itself, which is why KV caches tolerate lower precision than SSM states. Qwen3.5-122B-A10B-NVFP4, for example, uses the NVIDIA NVFP4 4-bit floating-point format for its weights, with an FP8 KV cache and an FP32 SSM state.

ReplaySSM 如何表示状态How ReplaySSM represents the state

SSM 递归可以在长度为 h 个 token 的窗口上展开：The SSM recurrence can be unrolled over a window of h tokens:

其中，S ckpt S_{\mathrm{ckpt}} 表示窗口之前的状态，γ\gamma 表示累积衰减。常规解码逐步沿左侧进行。ReplaySSM 使用右侧：对于 Mamba-2，它将状态表示为一个检查点，加上一个存储近期输入记录（αj \alpha_j、vj v_j、kj k_j）的小型缓冲区。where S ckpt S_{\mathrm{ckpt}} is the state before the window and γ\gamma is the cumulative decay. Conventional decoding follows the left side step by step. ReplaySSM uses the right side: it represents the state as a checkpoint plus a small buffer of recent input records ( αj \alpha_j , vj v_j , kj k_j ) for Mamba-2.

在大多数步骤中，也就是我们所说的非刷新步骤，系统会将当前输入追加到缓冲区，并根据检查点和缓冲区计算输出，而不存储状态。当包含 L 条记录的缓冲区填满时，刷新步骤会将缓冲的输入折叠到检查点中，将新的检查点一次性写入 HBM，然后清空缓冲区。输出与传统解码的结果一致，只有浮点数差异。ReplaySSM 的方法可以概括为三个问题：On most steps, which we call non-flush steps, it appends the current inputs to the buffer and computes the output from the checkpoint and the buffer without storing a state. When the buffer of L records fills, a flush step folds the buffered inputs into the checkpoint, writes the new checkpoint to HBM once, and clears the buffer. The outputs match conventional decoding up to floating-point differences. ReplaySSM’s approach can be summarized in three questions:

我们如何评估 ReplaySSMHow we evaluated ReplaySSM

我们评估了来自两个系列的四种混合模型：采用 Mamba-2 层的 Nemotron 3 和采用 GDN 层的 Qwen3.5，覆盖范围从单块 NVIDIA H100 Tensor Core GPU 上的 4B 稠密模型，到单块 NVIDIA Blackwell Ultra B300 GPU 上的 120B 级混合专家（MoE）模型（表 1）。我们在 vLLM 中实现了 ReplaySSM，并将其与 vLLM 自带内核在各自最佳启动配置下进行了比较。We evaluate four hybrid models from two families, Nemotron 3 with Mamba-2 layers and Qwen3.5 with GDN layers, from 4B dense models on one NVIDIA H100 Tensor Core GPU to 120B-class mixture-of-experts (MoE) models on one NVIDIA Blackwell Ultra B300 GPU (Table 1). We implemented ReplaySSM in vLLM and compared it with vLLM’s own kernels at their best launch configuration.

性能测试将 SSM 状态保持为 FP32（这些模型的默认精度），并使用 L = 16 的窗口，这是四个模型中表现最佳的窗口大小。推测解码测试使用每个模型的多令牌预测（MTP）头来生成草稿令牌。Performance runs keep the SSM state in FP32, the default precision for these models, and use a window of L = 16, the best-performing window size for all four models. Speculative decoding tests use each model’s multi-token prediction (MTP) head to generate draft tokens.

优势 1：减少状态内存流量Benefit 1: Reduce state memory traffic

在长度为 L 个 token 的窗口内，传统解码会读取状态 L 次，也会将其写入 L 次。ReplaySSM 仍会在每一步读取检查点，因为输出需要用到它，但只在刷新时写入一次。非刷新步骤只负责一件事——计算输出，并且只写入新的输入记录，即少量向量，而不是完整矩阵。这大致将主要状态流量减半。Over a window of L tokens, conventional decoding reads the state L times and writes it L times. ReplaySSM still reads the checkpoint on every step, because the output needs it, but writes it only once, during the flush. A non-flush step has one job—computing the output—and writes only the new input record, a few vectors instead of a full matrix. This roughly halves the dominant state traffic.

在服务端批次大小下，内存带宽限制了传统内核的性能，因此减少状态内存流量可以加快解码。在批次大小为 1 时，延迟主要由其他因素决定，ReplaySSM 在这些测试中没有带来加速。在批次大小为 256 时，内核的运行速度提升了 1.46 至 1.80 倍，四个模型的解码步骤速度提升了 1.20 至 1.47 倍（图 3）。随着批次大小增大，端到端收益也随之增加，因为 SSM 在每个步骤中所占的比重变大。At serving batch sizes, memory bandwidth limits the conventional kernel’s performance, so reducing state memory traffic speeds up decoding. At batch size 1, other sources of latency dominate, and ReplaySSM provides no speedup in these tests. At batch size 256, the kernel runs 1.46× to 1.80× faster, and the decode step runs 1.20× to 1.47× faster across the four models (Figure 3). The end-to-end gain grows with batch size because the SSM share of the step grows.

优势 2：简化回滚并加快验证速度Benefit 2: Simplify rollback and speed up verification

在推测解码中，ReplaySSM 会先将全部 T 个草稿 token 的输入追加到输入缓冲区，再进行验证；验证后，通过推进缓冲区指针来提交已接受的前缀。与 KV 缓存一样，指针之后的记录会被忽略，后续追加操作会覆盖这些记录。每个草稿 token 都无需保存状态快照，也无需恢复任何状态（图 2，中间）。刷新操作只会将已提交的记录合并到检查点中，因此无需等待已接受的长度。In speculative decoding, ReplaySSM appends the inputs of all T draft tokens to the input buffer before verification and commits the accepted prefix afterward by advancing the buffer pointer. As with a KV cache, records beyond the pointer are ignored, and the next append overwrites them. There is no state snapshot to save for each draft token and no state to restore (Figure 2, middle). A flush folds only committed records into the checkpoint, so it never waits for the accepted length.

这种分离还带来了另一个好处。传统解码每个 token 都必须生成两样东西：输出和更新后的状态，因为下一个 token 会用到该状态。ReplaySSM 的非 flush 步骤只生成输出，而只计算输出也让另一种计算方法成为可能。对于从零状态开始的一次更新，o=(vk⊤)q=v(k⊤q)o = (v k^\top)q = v(k^\top q)。左侧的表达式会构造一个 d_v × d_k 状态并读取它。右侧的表达式则会计算一个标量并用它缩放一个向量，无需构造状态。对于针对 h 条已提交记录的 T 个草稿，同样的思路可得以下表达式：This separation offers another benefit. Conventional decoding must produce two things per token, the output and the updated state, because the next token consumes the state. A non-flush step in ReplaySSM produces only the output, and computing only the output opens up a different computation method. For one update from a zero state, o=(vk⊤)q=v(k⊤q)o = (v k^\top)q = v(k^\top q) . The expression on the left builds a dv×dk d_v \times d_k state and reads it. The expression on the right forms a scalar and scales a vector without building the state. For T drafts against h committed records, the same idea gives the following expression:

其中，D 保存草稿位置的累积衰减，W 是一个带因果掩码的衰减矩阵。我们称这种方式为仅输出求值。检查点会用于全部 T 个查询一次，其他操作则是在缓存的键、缓存的值和草稿查询之间进行两次小型矩阵乘法。整个过程中不会生成中间状态，因此可以节省片上内存和 FLOP，而且 Tensor Core 能加速这些计算。此外，K⊤QK^\top Q 在同一组中的所有值头之间共享。这与预填充内核已经采用的分块计算相同，只不过此处以缓冲区和草稿作为数据块。GDN 也可以采用这种形式。仅输出求值也适用于标准解码，但在验证阶段最为重要，因为此时每步的计算量会增加到 T 倍。with D holding the cumulative decays of the draft positions and W a causally masked matrix of decays. We call this output-only evaluation. The checkpoint is used once for all T queries, and everything else is two small matrix products over the cached keys, the cached values, and the draft queries. No intermediate state is formed, which saves on-chip memory and FLOPs, and Tensor Cores can speed up these computations. Also, K⊤QK^\top Q is shared by all value heads in a group. This is the same chunkwise computation that prefill kernels already use, with the buffer and the drafts as the chunk. GDN can also use this form. Output-only evaluation applies to standard decoding too, but it matters most in verification, where T multiplies the arithmetic per step.

图 4 展示了两种 120B 级模型在 GSM8K 提示词上的推测式服务吞吐量（T = 4）。在批量大小为 512 时，ReplaySSM 的吞吐量分别达到 vLLM 推测解码的 1.85 倍和 2.10 倍，且接受长度相匹配。Figure 4 shows speculative serving throughput on GSM8K prompts with T = 4 on the two 120B-class models. At batch size 512, ReplaySSM delivers 1.85× and 2.10× the throughput of vLLM’s speculative decoding, with matched acceptance lengths.

收益有两个来源。The gain has two sources.

• 更快的验证：vLLM 的内核会为每个 token 保存一个快照，因此延迟会随 T 增长；当 T = 8 时，延迟几乎是 vLLM 标准解码步骤的五倍。ReplaySSM 在大多数步骤中都不会写入状态，因此验证八个 token 所需的时间约等于一个标准步骤（包括刷新操作；见图 5）。
  Faster verification: vLLM’s kernel stores one snapshot per token, so its latency grows with T and at T = 8 is almost five times that of vLLM’s standard decoding step. ReplaySSM writes no state on most steps, so it verifies eight tokens in about the time of one standard step, flushes included (Figure 5).
• 更高的并发度：草稿状态快照会占用 GPU 内存并限制并发度。常规推测解码在批大小超过 128 后不再带来收益；在 Qwen3.5-122B-A10B-NVFP4 上，批大小为 512 时，其表现低于标准解码，而 ReplaySSM 则能随着批大小增大持续扩展。
  Higher concurrency: Draft state snapshots consume GPU memory and cap concurrency. Conventional speculation stops gaining beyond batch size 128, and on Qwen3.5-122B-A10B-NVFP4, it falls below standard decoding at batch size 512, while ReplaySSM keeps scaling with batch size.

优势 3：通过使用不同精度限制误差累积Benefit 3: Limit error accumulation with separate precisions

较低精度的状态占用的字节数更少，因此在内存带宽限制内核性能时，可以降低延迟。在 Mamba-2 上，读取 NVFP4 状态时，非刷新内核的运行速度比读取 FP32 状态时快 3.8 倍。A lower-precision state requires fewer bytes, which reduces latency when memory bandwidth limits kernel performance. On Mamba-2, the non-flush kernel runs 3.8× faster when reading NVFP4 state than when reading FP32 state.

ReplaySSM 通过两种方式改变量化。首先，检查点不再每步写入一次，而是每个窗口写入一次，因此单个低精度检查点每隔 L 步才会舍入一次。Nemotron 3 Ultra 报告探讨了这种周期性检查点方法。它降低了舍入频率，但舍入误差仍会进入后续更新。ReplaySSM changes quantization in two ways. First, the checkpoint is written once per window instead of once per step, so a single low-precision checkpoint is rounded once every L steps. The Nemotron 3 Ultra report explores this periodic checkpointing approach. It reduces rounding frequency, but rounding error still enters later updates.

其次，非刷新步骤仅读取状态以计算输出，而刷新步骤会更新后续状态更新所用的检查点。这两项工作可以采用不同的精度。我们为递推保留一个高精度检查点，并为非刷新读取保留一个低精度副本。刷新操作从高精度检查点开始，合并缓冲的输入，并分别将结果转换为两种格式。低精度副本绝不会被读回递推中，因此其舍入误差只会影响当前窗口的输出，不会沿递推传播。在刷新时写入额外的低精度副本，可减少每个非刷新步骤读取的数据量。接受、回滚和缓冲区管理均保持不变（图 2，右侧）。Second, non-flush steps read the state only to compute outputs, while flush steps update the checkpoint used in subsequent state updates. The two jobs can use different precisions. We keep a high-precision checkpoint for the recurrence and a low-precision copy for non-flush reads. A flush starts from the high-precision checkpoint, incorporates the buffered inputs, and casts the result independently into both formats. The low-precision copy is never read back into the recurrence, so its rounding error affects only the outputs of the current window and never propagates across the recurrence. Writing the additional low-precision copy during a flush reduces the amount of data read on every non-flush step. Acceptance, rollback, and buffer management are unchanged (Figure 2, right).

图 6 衡量了在 PG-19 文本上采用 teacher forcing、最多生成 128K 个 token 时的输出误差，指标为相对于 FP32 状态解码的 Kullback–Leibler（KL）散度。共享的 FP8 或 NVFP4 检查点会随着序列长度增加而累积误差。将递推过程保持在 FP32，可消除大部分误差增长：在 Qwen3.5-4B 上生成 128K 个 token 时，采用 FP32 递推的 NVFP4 读取将 KL 散度从 0.136 降至 0.0038。剩余误差来自读取过程，且不会随序列长度增加。Figure 6 measures the output error as the Kullback–Leibler (KL) divergence from FP32-state decoding under teacher forcing on PG-19 text, out to 128K generated tokens. A shared FP8 or NVFP4 checkpoint accumulates error with length. Keeping the recurrence in FP32 removes most of the growth: at 128K tokens on Qwen3.5-4B, NVFP4 reads with an FP32 recurrence cut the KL from 0.136 to 0.0038. What remains is the read error, which does not grow with length.

采用随机舍入（SR）的 FP16 递归计算与 FP32 非常接近，因此使用半精度递归检查点就足够了。对读取副本进行更好的量化，例如沿状态维度和通道维度分别设置缩放因子，可以进一步降低剩余的读取误差。An FP16 recurrence with stochastic rounding (SR) tracks FP32 closely, so a half-precision recurrence checkpoint is enough. Better quantization of the read copy, such as separate scales along the state and channel dimensions, could cut the remaining read error further.

我们还检查了 KL 差距是否会影响下游准确率。表 2 展示了 GPQA-Diamond，这是我们运行的三个基准测试中生成文本最长的一个。共享的 NVFP4 检查点在四个模型上的准确率损失为 3 到 31 个百分点。使用 FP16 或 FP8 递归计算的 NVFP4 推理结果弥补了大部分损失，并使四个模型中的三个与 FP32 的差距保持在约 2 个百分点以内。We also checked whether the KL gap shows up in downstream accuracy. Table 2 shows GPQA-Diamond, the benchmark with the longest generations of the three we ran. A shared NVFP4 checkpoint loses 3 to 31 percentage points of accuracy across the four models. NVFP4 reads with an FP16 or FP8 recurrence recover most of that and keep three out of four models within about 2 percentage points of FP32.

图 7 展示了 GPQA-Diamond 上准确率损失随生成长度的变化情况。在三个基准测试中：Figure 7 shows how accuracy loss varies with generation length on GPQA-Diamond. Across the three benchmarks:

• GPQA-Diamond：在 Nemotron-3-Super-NVFP4 上，共享检查点在较短生成时与 FP32 的准确率相当，但随着生成长度增加，二者逐渐出现差异。分离精度配置则始终与 FP32 保持接近。
  GPQA-Diamond: On Nemotron-3-Super-NVFP4, the shared checkpoint tracks FP32 accuracy for shorter generations but diverges from it as generation length grows. The separate-precision configuration stays close to FP32 throughout.
• MMLU-Pro 和 Math-L5：生成内容通常比 GPQA-Diamond 中的更短。四个模型中有三个使用共享 NVFP4 检查点时准确率下降不超过 3 个百分点，而单独配置的检查点平均准确率与 FP32 相差不到半个百分点。
  MMLU-Pro and Math-L5: Generations are generally shorter than those in GPQA-Diamond. Three out of four models lose at most 3 percentage points of accuracy with a shared NVFP4 checkpoint, while the separate checkpoint configurations stay within half a percentage point of FP32 on average.

论文包含了全部三个基准测试和一项长输入 RULER 测试的结果。The paper includes results for all three benchmarks and a long-input RULER test.

开始使用 ReplaySSMGet started with ReplaySSM

ReplaySSM 已投入生产环境。GLM-5.3-Flash 使用 ReplaySSM 提供服务，Z.ai 的基础设施报告则介绍了如何将其集成到 KDA 解码内核中。Kimi-K3 会在其 KDA 层中缓存草稿 token 的 SSM 输入以供验证，并将 ReplaySSM 列为一种同期方案。ReplaySSM 还集成到了 TensorRT-LLM、vLLM、SGLang 和 FlashInfer 等开源推理框架中。ReplaySSM is in production. GLM-5.3-Flash serves with ReplaySSM, and Z.ai’s infrastructure report describes adding it to the KDA decode kernel. Kimi-K3 caches the SSM inputs of draft tokens for verification in its KDA layers and cites ReplaySSM as a concurrent approach. ReplaySSM is also integrated into open-source inference frameworks such as TensorRT-LLM, vLLM, SGLang, and FlashInfer.

如需了解更多信息，请查看以下资源：To learn more, check out these resources:

• 阅读论文：ReplaySSM：缓存 SSM 输入，而非状态
  Read the paper: ReplaySSM: Cache SSM Inputs, Not State
• Try the code: github.com/Johnny-Liou/ReplaySSM
• Serve hybrid models with ReplaySSM in TensorRT-LLM: PR 14203, PR 16464
• 在 vLLM 中使用 ReplaySSM 为混合模型提供服务：PR 48018、PR 52506、PR 52928
  Serve hybrid models with ReplaySSM in vLLM: PR 48018, PR 52506, PR 52928
• Use the ReplaySSM kernels in FlashInfer: PR 3975, PR 4081
• 阅读 Dao AI Lab 之前发布的文章，了解标准解码和推测解码的详细介绍。
  Read the earlier post from the Dao AI Lab for a longer walkthrough of standard and speculative decoding