今年早些时候，我在 Relational AI 的同事帮助 AT&T 构建了专注于电信领域的 LLM 模型，名为 Otel，并将它推上了排行榜榜首。我真希望能把这一切都告诉你，但我并不在那个团队里。不过没关系，你可以通过上面的链接了解详情。接下来，我要讲讲如何训练一个专注于巴西柔术的 LLM。我们的想法是向它提出各种愚蠢的新手问题，让它给出出色的回答。整个过程中，它不会不耐烦，也不会在你尝试飞身十字固之前叫你先专注于基本功。说真的，我真希望能直接把 Claude 报名参加我们当地的柔术馆，时光快进十年，这事就搞定了，但 LLM 没法这样学习。大型语言模型需要语言，也需要文本。Earlier this year, my colleagues at Relational AI helped AT&T build LLM models focused on Telecom called Otel and push it to the top of the leaderboard. I wish I could tell you all about it, but I wasn’t on the team. No worries though, you can read all about on the links above. Instead I’m going to tell you about training an LLM focused on Brazilian Jiu Jitsu. The idea is to ask it all kinds of dumb beginner questions and have it provides great answers. All without getting frustrated or telling you to focus on the fundamentals before trying flying armbars. Now, I wish I could just sign up Claude to our local school, fast forward 10 years and be done, but LLMs can’t learn that way. Large Language Models need Language, they need Text.

幸运的是，有数以千计的“教学视频”（讲解某个主题或技巧的视频课程），通常质量很高；此外，YouTube 上也有数以千计、质量参差不齐的视频，可以作为我们的起点。儿童课程虽然是当地学校的收入来源，但事实证明，中年男性才是教学视频市场的目标受众。我已经在 BJJ Fanatics 上买了 50 多套教学视频，但这还远远不够，于是我采用了 Jozef Chen 的方法，又找到了不少。我还从 YouTube 上找了一些配有解说的比赛片段，并用 yt-dlp 下载了下来。Luckily there are thousands of “Instructionals” (video classes teaching a topic or technique) of usually high quality and thousands of YouTube videos of varying degrees of quality we can use as our starting point. While kid classes pay the bills for local schools, it turns out middle aged men are the target audience of the instructional market. I had already purchased over 50 instructionals on BJJ Fanatics, but that wasn’t nearly enough, I had to use the Jozef Chen method and got a lot more. I also got some competition match clips that were narrated from youtube and downloaded them using yt-dlp.

如今大家的注意力都很有限，所以这是 AI:DR 版本，方便你在自己的视频或文档库中复现这一做法：Everyone is short on attention these days, so this is the AI:DR version for you to replicate this for your own video or document library:

1. 清理音标转录错误，并在嵌入前添加上下文检索标头（[上下文：作者 — 来源 | 类别 | 标签]）。
   Clean phonetic transcription errors and prepend Contextual Retrieval headers ([Context: Author — Source | Category | Tags]) before embedding.
2. 构建一个显式知识图谱，映射规范标签、跨学校同义词、父子标签层级以及互斥规则。
   Build an explicit Knowledge Graph mapping canonical tags, cross-school synonyms, parent-child tag hierarchy, and mutual-exclusivity rules.
3. 使用 CLIP + 视觉语言模型审核视频关键帧，让无声的实物演示成为可搜索的[视觉演示]片段。
   Audit video keyframes with CLIP + a Vision-Language Model so silent physical demonstrations become searchable [Visual Demonstration] chunks.
4. 从现有文档中挖掘带时间戳的链接，使用 5 路列表式损失微调 ONNX Cross-Encoder 重排序器（bge-reranker-base）。
   Mine timestamped links from your existing docs to fine-tune an ONNX Cross-Encoder Re-Ranker (bge-reranker-base) with 5-way listwise loss.
5. 合成一个三轨 SFT 数据集（包含干扰项的 6 段 RAFT 摘录 + 诊断性 Go/No-Go 门控 + 知识图谱溯源/规则），并使用自动化的 <0.35% 7-gram 检查器对其进行门控。
   Synthesize a 3-Track SFT dataset (6-excerpt RAFT with distractors + diagnostic Go/No-Go gates + Knowledge Graph lineage/rules) and gate it with an automated <0.35% 7-gram linter.
6. 在单块 GPU 上运行 2.5 倍决策加权 QLoRA SFT，随后运行零额外显存的 PEFT DPO（使用 model.disable_adapter(): + 0.15 NLL 锚定项）。
   Run 2.5x Decision-Weighted QLoRA SFT followed by Zero-Extra-VRAM PEFT DPO (with model.disable_adapter(): + 0.15 NLL anchor) on a single GPU.
7. 使用按评分标准评估的领域基准进行验证，并部署到由持久化卷支持的 vLLM 无服务器环境中。
   Verify against a rubric-scored domain benchmark and deploy on volume-backed vLLM serverless.

A Recipe for Building a Domain-Expert AI Coach

假设你拥有一个规模庞大的专业领域知识库——就我们而言，其中包括约 30,000 段教学视频片段、14,000 段比赛录像，以及 79 份巴西柔术技术指南。你的领域可能是外科手术、航空维修、工业机器人或税法，但面临的问题都一样。Suppose you have a massive library of specialized domain knowledge—in our case, about 30,000 instructional video segments, 14,000 competition match clips, and 79 written technical guides on Brazilian Jiu-Jitsu. Your domain might be surgical procedures, aviation maintenance, industrial robotics, or tax law, but the problem is the same.

当用户提出问题时，你希望 AI 助手能够：When a user asks a question, you want an AI assistant that:

1. 真正熟悉这个领域的语言（知道不同学派的五种不同术语其实说的是同一回事）。
   Speaks the real language of the domain (knowing when five different terms from different schools mean the exact same thing).
2. 精准提取口头讲解和无声视觉演示中最重要的时刻。
   Retrieves the exact moments that matter across both spoken explanations and silent visual demonstrations.
3. 撰写一份清晰的多段式回答，包含可将视频播放器跳转到准确时间戳的行内引用 [1]–[6]，同时忽略搜索结果中与目标稍有偏差的内容。
   Writes a clear, multi-paragraph answer with inline [1]–[6] citations that jump the video player to the exact timestamp, while ignoring near-miss search results.
4. 在一个 70 亿参数的开放权重模型上运行，速度快、成本低，无需支付前沿 API 的高昂费用。
   Runs fast and cheap on a 7-billion-parameter open-weights model without frontier API bills.

如果只是给现成的 7B 模型接上基础向量检索，你得到的只会是含糊得像算命签语的建议、凭空编造的规则，以及错误的身体动作指导（比如告诉学生从脚跟勾锁中朝错误的方向挣脱）。If you just wire up basic vector search to an off-the-shelf 7B model, you will get vague fortune-cookie advice, hallucinated rules, and backwards physical mechanics (like telling a student to spin the wrong way out of a heel hook).

我们将逐步介绍我们构建 maxdemarzi/bjj-coach-7b-v3 和 bjj-reranker-v3 的完整配方：在我们的领域基准测试（BJJ-Bench-250）中，这套方案使运动学精确度达到 5.00 / 5.0，同时将干扰项引用率降至 0.0%。We are going to walk through the complete, step-by-step recipe we used to build maxdemarzi/bjj-coach-7b-v3 and bjj-reranker-v3 taking our domain benchmark (BJJ-Bench-250) to 5.00 / 5.0 on Kinematic Precision with a 0.0% distractor citation rate.

我们开始吧。Let’s jump in.

组成部分（架构概览）The Ingredients (Architecture Overview)

在我们开始烹饪之前，先来看看系统由哪些部分组成：Before we start cooking, here is what goes into the system:

训练 cross-encoder 重排序器和 7B 语言模型，只需要一块 GPU（在 RunPod 上使用 NVIDIA L40S、A40 或 A100，约 30 分钟）。You only need a single GPU (an NVIDIA L40S, A40, or A100 on RunPod for ~30 minutes) to train both the cross-encoder re-ranker and the 7B language model.

步骤 1：清理“略有错误”的数据并添加上下文标题Step 1: Clean the “Slightly Wrong” Data and Add Contextual Headers

现实世界中的转录文本往往杂乱无章。教练在垫子上气喘吁吁地说话，还会使用来自日语和葡萄牙语的领域借词。Whisper 这类语音转文字模型会把“kimura”听成“camera”，把“De La Riva”听成“dealer river”，把“Danaher”听成“donna hair”，把“ashi garami”听成“oshi garami”，还会把“inside sankaku”听成“inside sinkhole”。Real-world transcripts are messy. Instructors are out of breath on a mat, using domain loanwords from Japanese and Portuguese. Speech-to-text models like Whisper hear “kimura” as "camera", “De La Riva” as "dealer river", “Danaher” as "donna hair", “ashi garami” as "oshi garami", and “inside sankaku” as "inside sinkhole".

1a. 首先修正语音识别错误1a. Repair Phonetic Mishearings First

如果可以在数据摄取阶段确定性地修正语音转写错误，就别浪费模型容量去教 LLM 猜这些错误。建立一个领域专用替换词典，并在嵌入或训练原始转录文本之前，先用它们进行处理：Don’t waste model capacity teaching an LLM to guess phonetic typos if you can fix them deterministically at ingest time. Build a domain replacement dictionary and run it over your raw transcripts before you embed or train on them:

const PHONETIC_REPAIRS = [
  [/\bcamera\b/gi, 'kimura'],
  [/\bdealer river\b/gi, 'de la riva'],
  [/\bdonna (?:her|hair)\b/gi, 'Danaher'],
  [/\boshi garami\b/gi, 'ashi garami'],
  [/\binside sinkhole\b/gi, 'inside sankaku'],
  [/\bhonor roll\b/gi, 'honey hole'],
  [/\bMarcelo team\b/gi, 'marcelotine'],
  [/\b(?:uma|homo) plata\b/gi, 'omoplata'],
];

1b. 在嵌入前添加上下文检索标头1b. Prepend Contextual Retrieval Headers Before Embedding

一段 60 秒的视频转录片段里充满了代词：“他推这里时，我就把脚移到那里，然后把这个锁住。”单看这一段，完全无法判断当时是什么体位。A 60-second video transcript window is full of pronouns: “When he pushes here, I move my foot there and lock this up.” By itself, that chunk has no idea what position it is in.

在计算密集型嵌入（BAAI/bge-small-en-v1.5）或全文搜索向量之前，为每个分块添加一个结构化的上下文检索头：Before computing dense embeddings (BAAI/bge-small-en-v1.5) or full-text search vectors, prepend a structured Contextual Retrieval Header to every chunk:

[BJJ Context: Gordon Ryan — Systematically Attacking The Guard (NOGI) | Chapter: Half Butterfly Sweeps | Tags: half-butterfly, butterfly-hook, half-guard]
<spoken transcript body>

现在，每个独立分块的向量表示中都直接包含教练、课程标题、规则集/着装（GI 与 NOGI）、章节以及规范标签。Now every single chunk carries the instructor, course title, ruleset/attire (GI vs. NOGI), chapter, and canonical tags right inside its vector representation.

步骤 2：构建领域知识图谱Step 2: Build a Domain Knowledge Graph

向量相似度擅长处理模糊语义，但在严格的领域逻辑方面表现很差。在训练模型之前，应将领域分类体系编码为显式的知识图谱。Vector similarity is great at fuzzy semantics, but it is terrible at strict domain logic. Before training your model, encode your domain’s taxonomy into an explicit Knowledge Graph.

我们的图有三项任务：Our graph has three jobs:

1. 跨流派同义词等价（153 个规范标签，388 个同义词）：不同流派会用不同名称指代同一种技巧。在 Sankaku 中，The Saddle、4/11、Honey Hole 和 Top Rock 都指向规范的鞍位节点。当用户询问“honey hole”时，图谱会将搜索查询扩展到所有同义词，并在 LLM 提示中附上流派说明，让模型知道它们指的是完全相同的位置。
   Cross-Lineage Synonym Equivalence (153 canonical tags, 388 synonyms): Different schools name the same mechanic differently. Inside Sankaku, The Saddle, 4/11, Honey Hole, and Top Rock all point to the canonical saddle node. When a user asks about "honey hole", the graph expands the search query across all synonyms and passes a lineage note into the LLM prompt so it knows they are the exact same position.
2. 宽泛到具体的标签剪枝：如果某个视频片段或比赛片段被标记为特定防守（half-butterfly 或 de-la-riva-guard）或特定腿部缠绕（saddle 或 outside-ashi），图谱会在自动标记时自动抑制宽泛的父级标签（open-guard、ashi-garami）。这样可以让混合搜索中的逆文档频率（IDF）标签权重保持精确。
   Broad-to-Specific Tag Pruning: If a video segment or competition clip is tagged with a specific guard (half-butterfly or de-la-riva-guard) or a specific leg entanglement (saddle or outside-ashi), the graph automatically suppresses broad parent tags (open-guard, ashi-garami) during auto-tagging. That keeps Inverse Document Frequency (IDF) tag weights sharp during hybrid search.
3. 运动学互斥性（scoreKinematicCompatibility）：两名运动员不可能同时处于闭合防守和骑乘位。在图中编码互斥的位置后，我们的重排序器便能对起始位置在物理上不兼容的近似匹配搜索结果，确定性地施加 -0.25 的惩罚；当 [GI] 与 [NOGI] 的服装不匹配时，则施加 -0.20 的惩罚。
   Kinematic Mutual Exclusivity (scoreKinematicCompatibility): Two athletes cannot be in closed-guard and in the saddle at the same time. Encoding mutually exclusive positions in the graph lets our re-ranker apply a deterministic -0.25 penalty to near-miss search results that start from a physically incompatible position, and a -0.20 penalty when [GI] vs. [NOGI] attire mismatches.

第 3 步：为检索器装上“眼睛”（多模态视觉定位）Step 3: Give Your Retriever “Eyes” (Multimodal Visual Grounding)

在任何以视频为主的领域，专家都会用手演示一些从不会说出口的细节：“看我这里的握法——如果我把膝盖这样转个角度，他就没法把我压平。”如果你只索引音频转录文本，这些肢体动作的细节就无从察觉。In any video-heavy domain, experts show things with their hands that they never say out loud: “Look at my grip right here—if I angle my knee like this, he can’t flatten me.” If you only index audio transcripts, those physical details are invisible.

下面介绍如何将无声的视觉演示转化为可搜索的文本片段：Here is how to turn silent visual demonstrations into searchable text chunks:

1. 提取候选关键帧：在指南或章节提及演示的时间戳处提取视频帧。
   Extract Candidate Keyframes: Pull video frames at the timestamps where your guides or chapters reference a demonstration.
2. 使用对比式 CLIP 评分筛选 Talking Heads：对每一帧运行 512 维 CLIP 视觉模型（Qdrant/clip-ViT-B-32-vision），并将其与正向领域提示词（“两名缠斗者在垫子上积极演示一项柔术技巧”）的匹配分数减去负向提示词（“静态标题卡，或一名单人对着镜头讲话”）的匹配分数。
   Filter Talking Heads with Contrastive CLIP Scoring: Run a 512-dim CLIP vision model (Qdrant/clip-ViT-B-32-vision) over every frame and score it against positive domain prompts (“two grapplers actively demonstrating a jiu-jitsu technique on the mat”) minus negative prompts (“a static title card or a single person talking to the camera”).
3. 使用视觉-语言模型为物理力学动作添加说明：对于经过验证的演示关键帧（资料库中共 861 个），融合口头引述、指导者的生物力学论断，以及视频视觉-语言模型（Qwen2.5-VL-7B）对画面中可见的握法、勾挂动作和肢体角度的运动学描述。
   Caption the Physical Mechanics with a Vision-Language Model: For verified demonstration keyframes (861 in our library), fuse the spoken quote, the guide’s biomechanical claim, and a Video-VLM (Qwen2.5-VL-7B) kinematic description of the grips, hooks, and limb angles visible on screen.
4. 为 chunk_embeddings 建立索引：将这些描述作为 [视觉演示 — <主题>] ... 插入向量表，并将这些块固定到精确的演示时间戳 (timestamp_sec)。
   Index into chunk_embeddings: Insert those descriptions into your vector table as [Visual Demonstration — <Topic>] ... chunks pinned to the exact demonstration timestamp (timestamp_sec).

现在，当用户询问如何设置一个仓位时，混合搜索会同时检索口头讲解和[视觉演示]片段。Now, when a user asks how a position is set up, hybrid search retrieves both the spoken explanation and the [Visual Demonstration] chunk.

步骤 4：基于内部引用训练自定义交叉编码器重排序模型Step 4: Train a Custom Cross-Encoder Re-Ranker from Internal Citations

双编码器会分别将用户的问题和视频片段嵌入为两个 384 维向量，然后计算它们的点积。因此，它的速度足以通过 HNSW 在 PostgreSQL 中扫描 265,000 个片段，但它会把整段 60 秒视频的转录内容压缩成一个向量。细微的技术差异（“从下位半防守位防守膝切过腿”与“从上位总部位完成膝切过腿”）会变得模糊不清，这确实是个问题。A bi-encoder embeds the user’s question and the video chunk separately into two 384-dimensional vectors and takes their dot product. That makes it fast enough to scan 265,000 chunks in PostgreSQL via HNSW, but it compresses an entire 60-second video transcript into a single vector. Subtle mechanical differences (“defending the knee cut from bottom half guard” vs. “finishing the knee cut from top headquarters”) blur together and that’s a real problem.

第一阶段的混合检索（结合 pgvector HNSW 密集搜索、PostgreSQL tsvector BM25 搜索，以及通过倒数排名融合进行的 IDF 加权标签匹配）可快速筛出约 30 个候选片段。为了从中选出最适合交给 LLM 的 6 个片段，你需要一个交叉编码器重排序器，能够通过完整的双向注意力机制同时读取（问题，候选段落）。First-stage hybrid retrieval (combining pgvector HNSW dense search, PostgreSQL tsvector BM25 search, and IDF-weighted tag matching via Reciprocal Rank Fusion) gives you a fast pool of about 30 candidate moments. To pick the best 6 moments for the LLM, you want a cross-encoder re-ranker that reads (question, candidate_passage) together with full bidirectional attention.

现成的重排序模型（BAAI/bge-reranker-base）不懂领域术语。如何获取训练数据来微调模型，而不必手动标注数千组样本？Off-the-shelf re-rankers (BAAI/bge-reranker-base) don’t know domain jargon. How do you get training data to fine-tune one without hand-labeling thousands of pairs?

挖掘现有文档中的链接Mine Your Existing Documentation Links

我们的 79 份 Markdown 指南中包含约 3500 个带时间戳的视频链接（/watch/<segment_uuid>?t=<seconds>）。每个链接都是经过验证的真实数据对，将一项技术声明与对应的精确视频片段关联起来！Our 79 markdown guides contained about 3500 timestamped video links (/watch/<segment_uuid>?t=<seconds>). Every link is a verified ground-truth pair connecting a technical claim to an exact video chunk!

我们将这些指南引用与我们的 [视觉演示] 分块以及新标注的分类体系分块相结合，构建了约 4000 组三元组训练数据，每一行包含：We combined those guide citations with our [Visual Demonstration] chunks and newly tagged taxonomy chunks to build about 4000 training triplets, where each row contains:

• 查询：指南部分中的技术问题或生物力学主张。
  query: The technical question or biomechanical claim from the guide section.
• 正例：时间戳 t 处的确切转录文本或视觉演示片段（带有其上下文检索标题）。
  positive: The exact transcript or visual-demo chunk at timestamp t (with its Contextual Retrieval header).
• hard_negatives (4 per query):
  1. Two chunks from different segments in the same instructional volume (same instructor, same general topic, wrong specific detail).
  2. One chunk from the same guide topic with high lexical word overlap.
  3. One cross-instructional chunk sharing a canonical tag.

使用 5 路列表式损失进行微调并导出为 ONNXFine-Tune with 5-Way Listwise Loss and Export to ONNX

我们使用组合式的 5 路列表排序交叉熵损失 + 成对边际损失，对 BAAI/bge-reranker-base 进行了 3 个 epoch 的微调（lr=2e-5，单 GPU 上耗时约 2.5 分钟），随后将模型直接导出为 ONNX，使其能够在我们的嵌入微服务中本地运行，且无需占用 GPU：We fine-tuned BAAI/bge-reranker-base for 3 epochs (lr=2e-5, ~2.5 minutes on a single GPU) using a combined 5-way listwise Cross-Entropy + pairwise margin loss, then exported the model directly to ONNX so it runs locally inside our embedding microservice with zero GPU overhead:

跨编码器（BAAI/bge-reranker-base，一种拥有 2.78 亿个参数的 RoBERTa 模型）将（查询、段落）对拼接在一起——<s> query </s></s> passage </s>——然后输出一个表示相关性的标量 logit s(q,p) s(q,p)。A cross-encoder (BAAI/bge-reranker-base, a 278M-parameter RoBERTa model) takes a (query, passage) pair concatenated together—<s> query </s></s> passage </s>—and outputs a single scalar logit s(q,p) s(q,p) representing relevance.

如果你使用简单的二元交叉熵来训练重排序器（每次对一段文本打分，标签为 0 或 1），模型就不会直接将正确片段与同一卷中的相似但不匹配的片段进行比较。相反，对于一个批次中的每个查询，我们会交错排列 5 段文本（[pos, neg1, neg2, neg3, neg4]），将它们输入交叉编码器，再将输出 logits 重塑为 (batch_size, 5)，其中第 0 列是正确的正例，第 1 到第 4 列是四个难负例，并同时优化以下两个损失：If you train a re-ranker with simple binary cross-entropy (scoring one passage at a time as 0 or 1), the model never directly compares the gold chunk against its near-miss sibling chunks from the same volume. Instead, for every query in a batch, we interleave 5 passages ([pos, neg1, neg2, neg3, neg4]), run them through the cross-encoder, reshape the output logits to (batch_size, 5) where column 0 is the gold positive and columns 1..4 are the four hard negatives, and optimize two losses together:

1. 5 路 Listwise InfoNCE（温度缩放交叉熵）：将 5 段文本视为一个 5 分类问题，其中类别 0 为获胜类别，并按温度 τ=0.7 进行缩放：Llistwise=−log⁡ exp⁡(s(q,p+)/0.7)exp⁡(s(q,p+)/0.7)+ ∑j=14 exp⁡(s(q,pj−)/0.7)
   5-Way Listwise InfoNCE (Temperature-Scaled Cross-Entropy): Treats the 5 passages as a 5-class classification problem where class 0 is the winner, scaled by temperature τ=0.7 : Llistwise=−log⁡ exp⁡(s(q,p+)/0.7)exp⁡(s(q,p+)/0.7)+ ∑j=14 exp⁡(s(q,pj−)/0.7)
2. 成对边际排序损失（边际 = 1.25）：要求正样本 logit 值 s(q,p+) 至少比每个单独的困难负样本 s(q,pj−) 高 1.25 个 logit 单位，以便之后与知识图谱兼容性调整相结合时，各分数仍能保持清晰区分： Lmargin = ∑j=1⁴ max(0, 1.25 − (s(q,p+) − s(q,pj−)))
   Pairwise Margin Ranking Loss (margin = 1.25): Forces the positive logit s(q,p+) to beat every individual hard negative s(q,pj−) by at least 1.25 logit units so scores stay well-separated when we later combine them with Knowledge Graph compatibility adjustments: Lmargin= ∑j=14 max⁡(0,  1.25−(s(q,p+)−s(q,pj−)))

以下是核心训练循环：Here is the core training loop:

for step, (qs, pos_list, neg1_list, neg2_list, neg3_list, neg4_list) in enumerate(loader, 1):
    bsz = len(qs)
    # Interleave [pos, neg1, neg2, neg3, neg4] per query -> (bsz * 5) pairs
    flat_q, flat_p = [], []
    for i in range(bsz):
        flat_q.extend([qs[i]] * 5)
        flat_p.extend([pos_list[i], neg1_list[i], neg2_list[i], neg3_list[i], neg4_list[i]])
    enc = tokenizer(flat_q, flat_p, padding=True, truncation=True, max_length=384, return_tensors="pt").to(device)
    optimizer.zero_grad()
    with torch.cuda.amp.autocast(enabled=use_amp):
        logits = model(**enc).logits.view(bsz, 5)  # col 0 = positive, cols 1..4 = hard negatives
        targets = torch.zeros(bsz, dtype=torch.long, device=device)
        # 1. 5-Way Listwise InfoNCE Loss (temperature = 0.7)
        loss_ce = ce_loss_fn(logits / 0.7, targets)
        # 2. Pairwise Margin Ranking Loss (margin = 1.25)
        margins = [torch.clamp(1.25 - (logits[:, 0] - logits[:, j]), min=0.0).mean() for j in range(1, 5)]
        loss = loss_ce + 0.25 * sum(margins)
    scaler.scale(loss).backward()
    scaler.step(optimizer)
    scaler.update()
    scheduler.step()

导出为 ONNX，用于无 GPU 的生产环境服务Exporting to ONNX for Zero-GPU Production Serving

在生产环境中，你不会想仅仅为了重新排序 20 个候选文本块，就启动一个 PyTorch GPU 工作进程。将微调后的交叉编码器导出为 ONNX 图，并设置动态 batch_size 和序列轴后，我们的轻量级 Python 嵌入服务便能通过 onnxruntime + Rust tokenizers 在 CPU 上运行它，每个查询耗时约 40 毫秒：In production, you don’t want to spin up a PyTorch GPU worker just to re-rank 20 candidate chunks. By exporting the fine-tuned cross-encoder to an ONNX graph with dynamic batch_size and sequence axes, our lightweight Python embedding service runs it on CPU via onnxruntime + Rust tokenizers in about 40ms per query:

# 1. Export PyTorch cross-encoder to ONNX
torch.onnx.export(
    model,
    (dummy["input_ids"], dummy["attention_mask"]),
    "models/bjj-reranker-v3-onnx/model.onnx",
    input_names=["input_ids", "attention_mask"],
    output_names=["logits"],
    dynamic_axes={
        "input_ids": {0: "batch_size", 1: "sequence"},
        "attention_mask": {0: "batch_size", 1: "sequence"},
        "logits": {0: "batch_size"},
    },
    opset_version=14,
    do_constant_folding=True,
)

而在请求处理时，OnnxCrossEncoder 会在启动时加载一次 model.onnx 和 tokenizer.json，完全不依赖 PyTorch：And at request time, OnnxCrossEncoder loads model.onnx and tokenizer.json once at boot with zero PyTorch dependency:

class OnnxCrossEncoder:
    def __init__(self, model_dir: Path):
        self.tokenizer = Tokenizer.from_file(str(model_dir / "tokenizer.json"))
        self.tokenizer.enable_truncation(max_length=384)
        self.tokenizer.enable_padding(pad_id=1, pad_token="<pad>")
        self.session = ort.InferenceSession(
            str(model_dir / "model.onnx"),
            providers=["CPUExecutionProvider"],
        )
    def rerank(self, query: str, documents: list[str]) -> list[float]:
        encodings = self.tokenizer.encode_batch([(query, doc) for doc in documents])
        feed = {
            "input_ids": np.array([e.ids for e in encodings], dtype=np.int64),
            "attention_mask": np.array([e.attention_mask for e in encodings], dtype=np.int64),
        }
        logits = self.session.run(["logits"], feed)[0]
        return [float(x) for x in logits.reshape(-1).tolist()]

微调 3 个 epoch（lr=2e-5，在单块 GPU 上耗时约 2.5 分钟）使 Recall@1 提升了 +10.0%，MRR@10 达到 0.8749：Fine-tuning for 3 epochs (lr=2e-5, ~2.5 minutes on a single GPU) jumped Recall@1 by +10.0% and MRR@10 to 0.8749:

第 5 步：构建三轨 SFT 数据集（并自动进行静态检查）Step 5: Build a Three-Track SFT Dataset (and Lint It Automatically)

现在，让我们为 7B 教练模型准备训练数据。Now let’s prepare the training data for the 7B coach model.

如果你想构建一个既能充当 RAG 综合器（提供视频片段时），又能作为直接的领域专家（回答生物力学、血统和规则体系相关问题）的模型，可以构建一个三轨道 ChatML 数据集（约 3,000 段对话）：If you want a model that works both as a RAG synthesizer (when video excerpts are provided) and as a direct domain expert (answering biomechanics, lineage, and ruleset questions), build a Three-Track ChatML Dataset (about 3,000 conversations):

A 组：多文档 RAFT——检索增强微调（1,200 段对话）Track A: Multi-Document RAFT — Retrieval-Augmented Fine-Tuning (1,200 conversations)

在生产环境中，/ask 会发送从多位讲师那里检索到的 6 段摘录（[1]–[6]）。通常，其中 4 段会直接回答问题（“答案依据”），另外 2 段则是看似相关但并未真正回答问题的相近内容（“干扰项”）。In production, /ask sends 6 retrieved excerpts ([1]–[6]) from multiple instructors. Usually 4 of those excerpts directly answer the question (“oracles”) and 2 are near-miss neighbors (“distractors”).

如果你只用干净的单段摘录提示来训练模型，它要么总结[1]并忽略[2]–[6]，要么会虚构对干扰项的引用。If you only train your model on clean, single-excerpt prompts, it will either summarize [1] and ignore [2]–[6], or hallucinate citations to the distractors.

使用 RAFT 时，每个 Track A 训练提示都会将 4 个 oracle 片段和 2 个难例干扰片段打乱后放入 [1]–[6]。助手的完成内容是一篇 3 到 4 段的综合论述，将多位指导者的观点融为一体，在有依据的句子末尾标注 [1]–[6] 引文，并且绝不引用干扰项的索引。Using RAFT, every Track A training prompt packs 4 oracle chunks + 2 hard distractor chunks shuffled into [1]–[6]. The assistant completion is a 3-to-4 paragraph synthesis that weaves the multiple instructors together, places [1]–[6] citations at the end of supported sentences, and never cites the distractor indices.

B轨道：深入生物力学与诊断性“继续/停止”关卡（1,680次对话）Track B: Deep Biomechanics & Diagnostic “Go / No-Go” Gates (1,680 conversations)

真正的领域专长建立在诊断性体格检查之上——这些二元测试能告诉你是该继续采用某种技术，还是该及时收手：Real domain expertise is built around diagnostic physical checks—binary tests that tell you whether to advance a technique or bail out:

• 外侧脚跟勾逃脱：“你能把脚尖绷直（跖屈）吗？”如果可以，你还有旋转余量——把膝盖移出对方控制的膝线。如果你的脚被固定在背屈状态，旋转会撕伤膝盖——立即拍垫认输。
  Outside Heel Hook Escape: “Can you point your toes (plantarflexion)?” If yes, you still have rotational slack—clear your knee line. If your foot is pinned in dorsiflexion, spinning will tear your knee—tap immediately.
• 膝切过腿：“你的脚跟能碰到自己的臀部吗？”如果可以，说明膝盖楔入后方的空间已经封闭——朝对方髋部支点斜向下、向后切入。
  Knee Cut Pass: “Can your heel touch your backside?” If yes, the space behind your knee wedge is closed—slice down-and-back toward their hip fulcrum.
• 半蝴蝶防守与蝴蝶防守：半蝴蝶防守（蝴蝶半防守）是指下方腿缠住对手的一条腿，同时内侧腿插入一个蝴蝶钩；而蝴蝶防守（双蝴蝶钩）则是双钩都置于内侧的坐姿开放式防守。
  Half-Butterfly vs. Butterfly Guard: Distinguishing half-butterfly (butterfly half guard, where your bottom leg traps one of the passer’s legs while your inside leg inserts a single butterfly-hook wedge) from butterfly-guard (double butterfly, seated open guard with both hooks inside).

确保你的训练回答第一段中出现这些具体的机械杠杆（肘部、膝部、髋部、框架、楔形、支点、向量、角度、手腕、脚跟、钩、支撑点），让模型开门见山地讲解可操作的力学原理，而不是先来一段铺垫。Make sure the concrete mechanical levers (elbow, knee, hip, frame, wedge, fulcrum, vector, angle, wrist, heel, hook, post) appear in the first paragraph of your training answers so the model leads with actionable mechanics rather than throat-clearing preamble.

赛道 C：组合知识图谱谱系与规则集（210 次对话）Track C: Combinatorial Knowledge Graph Lineage & Rulesets (210 conversations)

直接从你的知识图谱和各项赛事规则手册（IBJJF、ADCC、EBI）中生成问答对，涵盖所有同义词族、按腰带级别划分的合法性规则（脚跟勾、膝部切入、摔砸、跳入防守，以及袖口相关规则）和战术比赛策略。Generate Q&A pairs directly from your Knowledge Graph and competition rulebooks (IBJJF, ADCC, EBI) covering every synonym family, belt-level legality rule (heel hooks, knee reaping, slams, jumping guard, sleeve cuffs), and tactical match strategy.

使用自动化 Linter 和评审器为数据集把关Gate Your Dataset with an Automated Linter & Critic

启动训练前，务必先对数据集进行 lint 检查。我们的 linter 会强制执行三项硬性门槛：Never launch a training run without linting the dataset first. Our linter enforces three hard gates:

1. <0.35% 7-gram 多样性门槛：统计全部 3,090 条助手回复中的每个 7 词序列。如果某个 7 词短语在超过 0.35% 的回复中重复出现，linter 就会拒绝该数据集，从而确保 LoRA 适配器永远无法记住重复的套话后缀。
   The <0.35% 7-Gram Diversity Gate: Counts every 7-word sequence across all 3,090 assistant responses. If any 7-word phrase repeats in more than 0.35% of responses, the linter rejects the dataset so the LoRA adapter can never memorize a repetitive boilerplate suffix.
2. 生物力学与角色颠倒审查器：扫描每条回复，查找领域内的矛盾（例如，将脚尖绷直等同于背屈、让上位传球手轻拍，或声称两个同义姿势有所不同）。
   The Biomechanical & Role-Inversion Critic: Scans every response for domain contradictions (e.g., equating pointing toes with dorsiflexion, telling the top passer to tap, or claiming two synonymous positions are different).
3. 引用与杠杆密度门槛：验证 Track A 的回答仅引用 oracle 索引，且平均包含至少 8.7 个具体的解剖学杠杆点。
   Citation & Lever Density Gate: Verifies that Track A responses cite only oracle indices and that responses average 8.7+ concrete anatomical levers.

第 6 步：训练 7B 模型——决策加权 QLoRA SFT + 零额外显存 DPOStep 6: Train the 7B Model — Decision-Weighted QLoRA SFT + Zero-Extra-VRAM DPO

准备好包含 3,090 个样本的 SFT 数据集和包含 680 对样本的 DPO 数据集后，我们就可以在单张 GPU 上通过两个快速阶段训练 Qwen/Qwen2.5-7B-Instruct。With our 3,090-sample SFT dataset and 680-pair DPO dataset ready, training Qwen/Qwen2.5-7B-Instruct happens in two fast stages on a single GPU.

阶段 6a：采用 2.5 倍决策 Token 加权的 QLoRA SFTStage 6a: QLoRA SFT with 2.5x Decision-Token Weighting

我们以 4 位 NF4 加载 Qwen/Qwen2.5-7B-Instruct，并在全部 7 个线性投影层（q_proj、k_proj、v_proj、o_proj、gate_proj、up_proj、down_proj）上挂载 LoRA 适配器（r=64，α=128）。We load Qwen/Qwen2.5-7B-Instruct in 4-bit NF4 and attach a LoRA adapter ( r=64,α=128 ) across all 7 linear projections (q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj).

有两个实现细节会带来巨大差异：Two implementation details make a huge difference here:

1. 连续的仅补全损失屏蔽，决策权重提高至 2.5 倍：我们屏蔽系统和用户提示词的 token（标签 = -100），因此交叉熵损失仅针对助手的指导回复计算。在该回复中，我们对影响重大的决策 token 应用 2.5 倍损失权重：引用括号（[1]–[6]）、诊断关卡（跖屈、背屈、轻敲）以及规则体系/策略术语（IBJJF、ADCC、EBI、合法、非法、绞膝、策略、位置、传承）。
   Contiguous Completion-Only Loss Masking with 2.5x Decision Weights: We mask the system and user prompt tokens (labels = -100) so cross-entropy loss is only computed on the assistant’s coaching response. Within that response, we apply a 2.5x loss weight to high-leverage decision tokens: citation brackets ([1]–[6]), diagnostic gates (plantarflexion, dorsiflexion, tap), and ruleset/strategy terms (IBJJF, ADCC, EBI, legal, illegal, reaping, strategy, position, lineage).
2. 在 Softmax 前切片活跃 Token（显存减少超过 15 倍）：由于 Qwen2.5 的词表很大（152,064 个 Token），在完整的填充批次上计算 CrossEntropyLoss(reduction="none")，会为提示词和填充 Token 分配数 GB 的中间 logits，而这些 Token 最终反正都会被屏蔽掉。对每个序列，只在调用 cross_entropy 前切出未屏蔽的助手 Token（mask = seq_labels != -100），在数学上完全等价，并且能将 logits 占用的显存减少 15 倍以上：
   Slice Active Tokens Before Softmax (>15x Less VRAM): Because Qwen2.5 has a large vocabulary (152,064 tokens), computing CrossEntropyLoss(reduction="none") across a full padded batch allocates gigabytes of intermediate logits for prompt and padding tokens that just get masked out anyway. Slicing only the unmasked assistant tokens (mask = seq_labels != -100) per sequence before calling cross_entropy is mathematically identical and cuts logit VRAM by more than 15x:

def compute_weighted_causal_lm_loss(logits, labels, token_weights):
    loss_fct = torch.nn.CrossEntropyLoss(reduction="none")
    total_weighted_loss = logits.new_zeros((), dtype=torch.float32)
    total_weight = logits.new_zeros((), dtype=torch.float32)
    for i in range(logits.size(0)):
        seq_labels = labels[i, 1:]
        mask = seq_labels != -100
        if not mask.any():
            continue
        seq_logits = logits[i, :-1, :][mask]
        active_labels = seq_labels[mask]
        active_weights = token_weights[i, 1:][mask].float()
        per_tok_loss = loss_fct(seq_logits.float(), active_labels)
        total_weighted_loss += (per_tok_loss * active_weights).sum()
        total_weight += active_weights.sum()
    return total_weighted_loss / total_weight.clamp(min=1.0)

在我们的分层优先级池（1,314 个序列，对旗舰诊断门控、规则集、战术和谱系进行过采样）上完成一个训练轮次需要 10.1 分钟，使完成损失降至 1.7644（验证损失为 1.4705）。One epoch over our stratified priority pool (1,314 sequences, oversampling flagship diagnostic gates, rulesets, tactics, and lineage) finishes in 10.1 minutes, bringing completion loss down to 1.7644 (1.4705 validation loss).

第 6b 阶段：零额外 VRAM 的 PEFT DPO + +0.15 NLL 锚点Stage 6b: Zero-Extra-VRAM PEFT DPO + +0.15 NLL Anchor

SFT 教会模型如何像教练一样写作。直接偏好优化（DPO）则教会模型哪些错误绝不能犯。SFT teaches the model how to write like a coach. Direct Preference Optimization (DPO) teaches it what mistakes never to make.

我们在五个类别中构建了 680 组困难负例偏好对（选中 vs. 拒绝）：We built 680 hard-negative preference pairs (chosen vs. rejected) across five categories:

1. 诊断性“通过/不通过”关卡（例如跖屈与背屈）。
   Diagnostic Go/No-Go Gates (e.g., plantarflexion vs. dorsiflexion).
2. 运动学矢量（例如，让膝切动作向下并向后，而不是向前并向外）。
   Kinematic Vectors (e.g., driving the knee cut down-and-back vs. forward-and-out).
3. 赛事规则体系与战术策略（例如 ADCC 与 IBJJF 对脚跟钩和摔砸的规则；面对柔韧性强的防守选手时，选择胸贴胸的身体锁定过防，而不是松散的托雷安多过防）。
   Competition Rulesets & Tactical Strategy (e.g., ADCC vs. IBJJF heel hook/slam rules; choosing a chest-to-chest body lock pass over a loose toreando pass against a flexible guard player).
4. 谱系与新分类法的对应关系（半蝴蝶式 vs. 蝴蝶防守、鞍式 vs. 内三角）
   Lineage & New Taxonomy Equivalence (half-butterfly vs. butterfly-guard, saddle vs. inside sankaku).
5. 多摘录 RAFT 抗干扰项配对（选中项引用跨越 3 段的 3–4 个 oracle 摘录；拒绝项引用一个干扰项摘录）。
   Multi-Excerpt RAFT Anti-Distractor Pairs (chosen cites 3–4 oracle excerpts across 3 paragraphs; rejected cites a distractor excerpt).

标准 DPO 训练器会将第二个冻结的 7B 参考模型加载到 GPU 内存中。如果你正在训练 PEFT LoRA 适配器，就完全不需要第二个模型——只需用 `model.disable_adapter()` 包裹参考模型的前向传播，并加上一个 `+0.15 * chosen_nll` 锚定项，这样 DPO 就能强化决策边界，同时不损害文本流畅度：Standard DPO trainers load a second frozen 7B reference model into GPU memory. When you are training a PEFT LoRA adapter, you don’t need a second model at all—just wrap the reference forward pass in with model.disable_adapter(): and add a +0.15 * chosen_nll anchor so DPO sharpens decision boundaries without hurting prose fluency:

# Exact reference log-probabilities with zero extra VRAM
with model.disable_adapter(), torch.no_grad(), torch.cuda.amp.autocast(dtype=compute_dtype):
    ref_c_logp, _ = compute_completion_logprob_and_nll(model, c_ids, c_lbls)
    ref_r_logp, _ = compute_completion_logprob_and_nll(model, r_ids, r_lbls)
with torch.cuda.amp.autocast(dtype=compute_dtype):
    pol_c_logp, c_nll = compute_completion_logprob_and_nll(model, c_ids, c_lbls)
    pol_r_logp, _ = compute_completion_logprob_and_nll(model, r_ids, r_lbls)
    logits_diff = beta * ((pol_c_logp - pol_r_logp) - (ref_c_logp - ref_r_logp))
    dpo_loss = -F.logsigmoid(logits_diff).mean()
    loss = (dpo_loss + 0.15 * c_nll) / grad_accum

在单张 GPU 上对 532 组均衡配对运行 DPO（beta=0.1, lr=5e-6）耗时 8.8 分钟，并达到 97.2% 的偏好边际胜率（dpo_loss = 0.0297）。Running DPO (beta=0.1, lr=5e-6) over 532 balanced pairs takes 8.8 minutes on a single GPU and reaches a 97.2% Preference Margin Win Rate (dpo_loss = 0.0297).

步骤 7：使用领域基准测试进行验证，并部署基于卷存储的无服务器服务Step 7: Verify with a Domain Benchmark & Deploy with Volume-Backed Serverless

绝不要凭感觉就发布模型。构建确定性的领域测试套件，并制定明确的评分标准（必须提及领域概念、对禁忌误区扣分、统计具体策略杠杆的数量，以及检查留出的多摘录 RAFT 引用）。Never ship a model on vibes. Build a deterministic domain test suite with explicit rubrics (must_mention domain concepts, penalize forbidden myths, concrete lever counts, and held-out multi-excerpt RAFT citation checks).

在推理时，我们还会将模型与一个轻量级、无参考的可验证奖励（RLVR）和运动学共识过滤器配合使用，该过滤器会检查 6 段检索到的摘录之间的标签共识，从而确保离群的干扰文本块绝不会被引用。At inference time, we also pair the model with a lightweight, reference-free Verifiable Reward (RLVR) & Kinematic Consensus Filter that checks tag consensus across the 6 retrieved excerpts so an outlier distractor chunk can never be cited.

以下是最终配方（maxdemarzi/bjj-coach-7b-v3）在 BJJ-Bench-250 全部 250 道问题上的得分情况：Here is how the finished recipe (maxdemarzi/bjj-coach-7b-v3) scores across all 250 questions of BJJ-Bench-250:

而在我们的 50 个留出型六摘录 RAFT 提示中（每个提示包含 4 个 oracle 摘录 + 2 个困难干扰项）：And on our 50 Held-Out 6-Excerpt RAFT Prompts (4 oracle excerpts + 2 hard distractors per prompt):

• 3至4段多讲师综合率：100.0%
  3-to-4 Paragraph Multi-Instructor Synthesis Rate: 100.0%
• 多预言机引用率（引用的预言机数量≥2≥2）：100.0%
  Multi-Oracle Citation Rate (≥2≥2 Oracles Cited): 100.0%
• 干扰项引用幻觉率：0.0%
  Distractor Citation Hallucination Rate: 0.0%

生产环境提示：基于存储卷的无服务器部署Production Tip: Volume-Backed Serverless Deployment

训练完成后，将 LoRA 适配器合并到 16 位基础模型中（PeftModel.from_pretrained(...).merge_and_unload()），并将 16 位 safetensors 推送到 Hugging Face（maxdemarzi/bjj-coach-7b-v3）。Once training finishes, merge the LoRA adapter into the 16-bit base model (PeftModel.from_pretrained(...).merge_and_unload()) and push the 16-bit safetensors to Hugging Face (maxdemarzi/bjj-coach-7b-v3).

要在 RunPod Serverless 上提供服务（使用 runpod/worker-v1-vllm:v2.30.1，并设置 QUANTIZATION=bitsandbytes），又不必为 24/7 闲置的 GPU 付费，也不用在冷启动时等待 worker 从 Hugging Face 下载 15 GB 的权重、耗时 3 分钟，可以挂载一个持久化的 RunPod Network Volume 到 /runpod-volume，并将 Hugging Face 和 vLLM 的缓存都指向该卷：To serve it on RunPod Serverless (runpod/worker-v1-vllm:v2.30.1 with QUANTIZATION=bitsandbytes) without paying for an idle 24/7 GPU—and without waiting 3 minutes on cold start for a worker to download 15 GB of weights from Hugging Face—attach a persistent RunPod Network Volume mounted at /runpod-volume and point both Hugging Face and vLLM caches to the volume:

MODEL_NAME=maxdemarzi/bjj-coach-7b-v3
QUANTIZATION=bitsandbytes
LOAD_FORMAT=bitsandbytes
BASE_PATH=/runpod-volume
HF_HOME=/runpod-volume/huggingface-cache/hub
HUGGINGFACE_HUB_CACHE=/runpod-volume/huggingface-cache/hub
VLLM_CACHE_ROOT=/runpod-volume/vllm-cache

首次预热请求将数据写入 /runpod-volume 后，后续每个无服务器工作进程都能直接从本地 NVMe 存储启动，只需几秒钟。After the first warmup request populates /runpod-volume, every subsequent serverless worker boots straight from local NVMe storage in seconds.