PyTorch 2.15 发布博客

PyTorch 2.15 版本包含以下变更：The PyTorch 2.15 release features the following changes:

• 重写后的 NCCL 后端成为默认选项，并配备飞行记录器，可在发生卡顿时转储；回滚方式包括设置 TORCH_DIST_USE_NCCL2=0 或使用 nccl-legacy。
  Rewritten NCCL backend becomes the default, with a flight recorder that dumps on hangs and TORCH_DIST_USE_NCCL2=0 or nccl-legacy as rollback paths
• 用于 torch.distributed 的单边传输，采用 NIXL 后端，并支持可被 CUDA 图捕获的传输
  One-sided transports for torch.distributed, with a NIXL backend and CUDA-graph-capturable transfers
• FSDP2 和流水线并行带来了更精细的内存与通信控制，包括对精度敏感的参数使用 fp32、通过 CUDA Graph 捕获梯度累积，以及将梯度归约与计算重叠执行。
  FSDP2 and pipelining gain finer memory and communication control, with fp32 for precision-sensitive parameters, CUDA-graph-captured gradient accumulation, and gradient reduction overlapped with compute 
• 通过减少内核启动次数并提升启动间的重叠，在 NVIDIA Blackwell GPU 上借助 NVGEMM 实现更快的 4 位（NVFP4）LLM 解码
   Faster 4-bit (NVFP4) LLM decoding on NVIDIA Blackwell GPUs through fewer, better-overlapped kernel launches in NVGEMM
• Apple Silicon 新增 FP8 和更快的解码支持，包括 float8_e4m3fn 和 _scaled_mm、带有解码快速路径的新 grouped_mm（MoE 解码时 tokens/s 提升至 1.32–1.34 倍），以及公开的 torch.mps.Stream API。
  Apple Silicon gains FP8 and faster decode, with float8_e4m3fn and _scaled_mm, a new grouped_mm with a decode fast path (1.32–1.34x tokens/s on MoE decode), and public torch.mps.Stream APIs
• 更广泛的平台支持：AMD gfx950 上的 FlyDSL 内核、gfx1250（MI450）的 Tensor Data Mover 支持、ROCm 10.0 wheel 包、Intel GPU 上更简便的 SYCL 内核编写和多 GPU 性能分析、默认提供 CUDA 13.2 wheel 包并同时提供 CUDA 13.4 wheel 包，以及支持在 Python 3.15 上使用 torch.compile。
  Broader platform support with FlyDSL kernels on AMD gfx950, Tensor Data Mover support for gfx1250 (MI450), ROCm 10.0 wheels, simpler SYCL kernel authoring and multi-GPU profiling on Intel GPUs, CUDA 13.2 as the default wheel with CUDA 13.4 alongside it, and torch.compile on Python 3.15

自 PyTorch 2.14 发布以来，本次版本共包含 583 位贡献者提交的 3,117 次 commit。我们衷心感谢社区成员的热心贡献。一如既往，我们鼓励大家试用这些更新，并在我们持续改进 2.15 的过程中报告遇到的问题。有关如何开始使用 PyTorch 2 系列的更多信息，请访问我们的入门页面。This release is composed of 3,117 commits from 583 contributors since PyTorch 2.14. We want to sincerely thank our dedicated community for your contributions. As always, we encourage you to try these out and report any issues as we improve 2.15. More information about how to get started with the PyTorch 2-series can be found at our Getting Started page.

欢迎带上您对 PyTorch 2.15 的任何疑问，参加我们于 2026 年 11 月 5 日（星期四）太平洋时间上午 11 点举办的问答网络研讨会。Tristan Rice、Natalia Gimelshein、Huy Do 和 Jane Xu（均来自 Meta）将介绍 2.15 版本的概况，并回答社区关于 PyTorch 及此版本新增功能的问题。Chris Gottbrath（Gottbrath Tech）将担任主持人。立即报名。Bring any questions you may have about PyTorch 2.15 to our Q&A webinar on Thursday, November 5, 2026, at 11 AM PT. Tristan Rice, Natalia Gimelshein, Huy Do, and Jane Xu (all Meta) will share an overview of the 2.15 release and answer community questions about PyTorch and the new capabilities in this release. Chris Gottbrath (Gottbrath Tech) will moderate. Register today.

在整个 2.x 系列的发展过程中，PyTorch 正从一个以研究为先的框架，演进为统一且与硬件无关的平台，支持大规模生产训练和推理。PyTorch 2.13 将 FlexAttention 引入 Apple Silicon，为 Inductor 增加了 CuTeDSL 代码路径，并推出了用于大规模集群训练的 torchcomms。PyTorch 2.14 则将这一路径发展为 NVGEMM，预览了重写后的 NCCL 后端，并为 Apple Silicon 带来了原生线性代数支持。Throughout the 2.x series, PyTorch has been evolving from a research-first framework into a unified, hardware-agnostic platform for production training and inference at scale. PyTorch 2.13 landed FlexAttention on Apple Silicon, gave Inductor a CuTeDSL code path, and introduced torchcomms for large-cluster training. PyTorch 2.14 matured that path into NVGEMM, previewed the rewritten NCCL backend, and brought native linear algebra to Apple Silicon.

PyTorch 2.15 在此基础上继续发展。重写后的 NCCL 后端如今为所有 NCCL 进程组提供底层支持；NVGEMM 针对低精度解码进行了调优；Apple Silicon 新增了 FP8 和专用于解码的内核；AMD 支持扩展至新的编译器 DSL，并为下一代 Instinct 提供预览版支持；Intel GPU 则获得了更简单的内核编程模型和更深入的性能分析功能。PyTorch 2.15 builds on that work. The rewritten NCCL backend now backs every nccl process group, NVGEMM is tuned for low-precision decode, Apple Silicon adds FP8 and decode-specific kernels, AMD support extends to new compiler DSLs and (preview support) for next Instinct generation, and Intel GPUs gain a simpler kernel programming model and deeper profiling.

分布式训练Distributed Training

重写后的 NCCL 后端成为默认后端The Rewritten NCCL Backend Becomes the Default

当模型在多块 GPU 上训练时，这些 GPU 会通过通信后端持续交换数据；在 NVIDIA 硬件上，这个后端就是 NCCL。在 PyTorch 2.15 中，所有使用 nccl 的作业默认都运行在我们重写的后端上。PyTorch 2.14 预览了我们从 torchcomms 移植而来的重写版 NCCL 后端，并将其作为可选的 nccl2 后端。在 2.15 中，它成为所有 nccl 进程组的默认后端；TORCH_DIST_USE_NCCL2=0 和 nccl-legacy 后端名称则可用于回退。现在会在创建进程组时立即创建通信器；重新配置时还可以选择启用收缩/扩容快速路径（NCCL 2.29 或更高版本），让存活的秩跳过完整的引导流程。借助 RCCL 2.30.7，ROCm 上也启用了用于容错的重新配置。When a model trains across many GPUs, those GPUs constantly exchange data through a communication backend, and on NVIDIA hardware that backend is NCCL. In PyTorch 2.15, every job that uses nccl runs on our rewritten backend by default.

PyTorch 2.14 previewed our rewritten NCCL backend, ported from torchcomms, as the opt-in nccl2 backend. In 2.15 it is the default for every nccl process group, with TORCH_DIST_USE_NCCL2=0 and the nccl-legacy backend name as rollback paths. Communicators are now created eagerly with the process group, and reconfiguration can opt into a shrink/grow fast path (NCCL 2.29 or newer) so surviving ranks skip a full bootstrap. Reconfigure for fault tolerance is also enabled on ROCm with RCCL 2.30.7.

为了使后端在生产环境中可调试，一个与后端无关的飞行记录器会在集合操作挂起时将事后分析信息写入磁盘；同时，监视程序的健康状态会发布到调试服务器，因此任何一个 rank 发生超时，都会触发所有 rank 导出飞行记录。对于集群规模的任务，单个 GPU 停滞就可能导致整个训练过程停摆，而查明原因往往意味着必须手动重现故障。飞行记录器会捕获挂起发生时每个 rank 正在执行的操作。To make the backend debuggable in production, a backend-agnostic flight recorder writes a post-mortem to disk when a collective hangs, and watchdog health is published to the debug server so a timeout on any rank triggers flight-recorder dumps from every rank. At cluster scale, a single stalled GPU can halt an entire training run, and finding the cause has often meant reproducing the failure by hand. The flight recorder captures what each rank was doing at the moment of the hang.

API 不稳定API Unstable

(PRs #192281, #192895, #199240, #198865 and #198339 by Tristan Rice, Meta; #194774 by Chinmay Kuchinad, AMD; #192233 and #199092 by Tushar Jain, Meta; #192075 by Atharva Kshirsagar, Red Hat)

使用 NIXL 后端的单边传输One-Sided Transports with a NIXL Backend

torch.distributed 新增了用于单边数据传输的 Python 传输 API，首个后端为 NIXL。传输层通过 store 连接到对等端，无需进程组或集合通信；Transport.read_stream/write_stream 会在 CUDA 流上加入传输操作队列，因此可以使用 torch.cuda.graph 捕获。torch.distributed gains a Python transport API for one-sided data movement, with NIXL as the first backend. Transports connect to a peer through a store, without needing a process group or collectives, and Transport.read_stream/write_stream enqueue transfers on a CUDA stream so they can be captured with torch.cuda.graph.

API 不稳定API Unstable

(PRs #195643, #197032, #198889 and #198918 by Tristan Rice, Meta)

FSDP2 与流水线并行：内存与通信控制FSDP2 and Pipeline Parallelism: Memory and Communication Control

FSDP2 现在可以更精细地控制精度和捕获。对精度敏感的参数（例如 MoE 路由门控和归一化权重）可以保持为 fp32，而模型其余部分仍以 bf16 训练，同时每个 fully_shard 仍只执行一次 all-gather（MixedPrecisionPolicy(param_dtype_override_fn=...)）。现在可以将完整的梯度累积步骤捕获到单个 CUDA 图中。还可以为每个参数单独设置梯度精度。梯度会直接累积到 .grad 中，并遵循各参数的 grad_dtype；当 reduce_dtype=None 时，这会改变默认行为（参见“弃用和不兼容变更”）。FSDP2 now gives finer control over precision and capture. Precision-sensitive parameters, such as MoE router gates and norm weights, can stay in fp32 while the rest of the model trains in bf16, still with one all-gather per fully_shard (MixedPrecisionPolicy(param_dtype_override_fn=...)). A full gradient-accumulation step can now be captured in a single CUDA graph. Gradient precision can also be set per parameter. Gradients accumulate directly in .grad and follow each parameter’s grad_dtype, which changes the default behavior when reduce_dtype=None (see Deprecations and Backwards-Incompatible Changes).

流水线并行现在会在每次接收前才分配接收缓冲区，而不是预先为每个微批次分配；并会在前向输出发送完成后释放这些输出。调度方案现在支持 max_outstanding_sends 预算；FSDP 的 unshard 操作也会分散到各个 rank 的启动气泡期间执行，而不是在一个 step 开始时全部发出。现在，根据调度方案确定的流水线并行 rank 之间，每条唯一边都会获得专用的 P2P 通信器。FSDP 的梯度归约现在可以与后续计算重叠执行。这些改进使 PyTorch 流水线支持 CUDA Graph 捕获且具有确定性，并优化了内存占用，同时提供了性能调优参数，以尽可能减少未被计算隐藏的通信开销。Pipeline parallelism now allocates receive buffers just before each receive instead of for every microbatch up front, and releases forward outputs once their sends complete. Schedules accept a max_outstanding_sends budget, and FSDP unshards are staggered across each rank’s startup bubble instead of all being issued at the start of a step. Each unique edge between pipeline-parallel ranks, as determined by the schedule, now gets a dedicated P2P communicator. FSDP reduce grads can now be overlapped with subsequent compute. These improvements make PyTorch pipelining cudagraphable, deterministic, with an optimized memory profile, and expose performance tuning knobs for minimizing exposed communication.

API 不稳定API Unstable

(PRs #196640, #196724, #196727, #196641 and #196963 by Chien-Chin Huang, Meta; #196626 by Anshul Sinha, Meta; #198417 by Andrew Or, Apache Spark PMC); #194114 by Animesh Jain, Meta; #196463, #196464 and #196706 by Sanket Purandare, Meta)

性能改进Performance Improvements

用于解码和线性代数的更快 MPS 内核Faster MPS Kernels for Decode and Linear Algebra

在 2.14 迁移的基础上，更多 MPS 算子从 MPSGraph 转向原生 Metal 内核。对含有 128,256 个词条的词表进行单行 softmax，耗时缩短至原来的八分之一；上采样反向计算最高提速 21.7 倍；归约现在共用一套内核，使 norm 全量归约的速度提升超过 100 倍。Building on the 2.14 migration, more MPS operators move from MPSGraph to native Metal kernels. A single-row softmax over a 128,256-entry vocabulary drops by 8x, upsample backward is up to 21.7x faster, and reductions now share one set of kernels, making norm full reductions more than 100x faster.

对于解码，F.linear 现在使用现有的 gemv 内核（几何平均加速 1.23 倍，小型投影最高可达 5 倍）；新的 MPS grouped_mm 增加了一条 gemv 路径，使 M3 Ultra 上的 MoE 解码吞吐量提高了 1.32–1.34 倍。多个线性代数求解器也经过了重写：对 512 × 512 矩阵进行 QR 分解的耗时从约 7 秒降至 1.5 毫秒，三角方程组求解现在也采用了分块处理。For decode, F.linear now uses the existing gemv kernels (1.23x geometric-mean speedup, up to 5x on small projections), and the new MPS grouped_mm has a gemv path that gives 1.32–1.34x higher MoE decode throughput on an M3 Ultra. Several linear algebra solvers were also rewritten: QR of a 512 x 512 matrix falls from about 7 seconds to 1.5 ms, and triangular solves are now blocked.

(PRs #196071, #199687, #198611, #198495, #194933, #198721, #198725, #198732, #195944 and #196900 by Irakli Salia, Hugging Face; #196922, #196241 and #195369 by Nikita Shulga, Thinking Machines Lab; #193153 by Sunghyun Choi, Tech University of Korea; #188473 by Shrivas Shankar, University of Wisconsin-Madison)

核心功能Core Features

默认使用 CUDA 13.2，并提供 CUDA 13.4 轮子包CUDA 13.2 by Default and CUDA 13.4 Wheels

默认的 PyTorch wheel 将迁移至 CUDA 13.2（13.2.2 工具包）；面向 Linux x86_64、Linux aarch64 和 Windows 的 CUDA 13.4 GA 版 wheel 也将同步发布。CUDA 13.4 wheel 新增了对 NVIDIA Rubin GPU 的实验性支持。The default PyTorch wheel moves to CUDA 13.2 (13.2.2 toolkit), with CUDA 13.4 GA wheels shipping alongside it for Linux x86_64, Linux aarch64 and Windows. The CUDA 13.4 wheels add experimental support for NVIDIA Rubin GPUs.

(PRs #198913 and #196014 by Andrey Talman, Meta; #194996 and #198843 by Ashish Karale, NVIDIA; #196560, #199269 and #196351 by Ting Lu, NVIDIA; #192653 by Wei Wang, NVIDIA; #186245 by Lakshay Garg, NVIDIA)

ROCm 10.0 与 TheRock 轮子包ROCm 10.0 and TheRock Wheels

PyTorch 2.15 在 2.14 引入的 ROCm 7.14 wheel 之外，还新增了 ROCm 10.0 wheel。现在，所有 ROCm wheel 均基于 AMD 的 TheRock 软件包构建，并将 ROCm 作为 pip 依赖项（对应的 rocm 软件包）安装，而不再捆绑 ROCm 库。PyTorch 2.15 adds ROCm 10.0 wheels alongside the ROCm 7.14 wheels introduced in 2.14. All ROCm wheels are now built on AMD’s TheRock packages, and they install ROCm as a pip dependency (the matching rocm package) instead of bundling the ROCm libraries.

(PRs #194919, #198895 and #196610 by Ethan Wee, AMD; #195672 by Andrey Talman, Meta)

Python 3.11 至 3.15，以及在 Python 3.15 上使用 torch.compilePython 3.11 to 3.15, and torch.compile on Python 3.15

PyTorch 2.15 支持 Python 3.11 至 3.15，包括自由线程版 3.14t 和 3.15t。在 2.15 中，Dynamo 新增了对 Python 3.15 的支持。PyTorch 2.15 supports Python 3.11 through 3.15, including the free-threaded 3.14t and 3.15t builds. In 2.15, Dynamo adds Python 3.15 support.

(PRs #178393, #194117 and #198018 by Rob Timpe, OpenTeams; #198233, #199398 and #199356 by Andrey Talman, Meta)

平台功能与更新Platform Features and Updates

CUDACUDA

用于 NVFP4 解码的 NVGEMMNVGEMM for NVFP4 Decode

NVGEMM 是 2.14 中新增的 CuTeDSL GEMM 后端，现在会将 ModelOpt NVFP4 线性层的标量反量化因子融合进 GEMM，从而在解码期间为每层省去一次独立的逐点启动。有界且感知形状的候选策略覆盖了 nvMatmulHeuristics 未涵盖的 Blackwell NVFP4 解码形状；NVGEMM 内核还可以使用程序化依赖启动，使 Triton 消费者与生产者 GEMM 的尾部执行重叠。NVGEMM 也迁移到了 CUTLASS Operators。NVGEMM, the CuTeDSL GEMM backend added in 2.14, now folds the scalar dequantization factor of ModelOpt NVFP4 linear layers into the GEMM, removing a separate pointwise launch per layer during decode. A bounded, shape-aware candidate policy covers Blackwell NVFP4 decode shapes that nvMatmulHeuristics misses, and NVGEMM kernels can use programmatic dependent launch so a Triton consumer overlaps with the tail of the producer GEMM. NVGEMM also moves onto CUTLASS Operators.

API 不稳定API Unstable

(PRs #194651, #194652, #197485, #194792 and #194793 by Michael Lazos, Meta; #194364 by Masaki Kozuki, NVIDIA)

ROCmROCm

gfx950 上的 FlyDSL 内核与 gfx1250 上的张量数据搬运器FlyDSL Kernels on gfx950 and Tensor Data Mover on gfx1250

FlyDSL 加入 CuTeDSL、Triton 和 Helion，成为 PyTorch 可使用的内核 DSL。在 gfx950 上，Inductor 会为 GEMM、不规则批次的 F.grouped_mm 以及 MXFP4/MXFP8 scaled_mm 自动调优 FlyDSL 模板，并与 Triton 和 ATen 一同使用；在 eager 模式下，FlyDSL 还可以覆盖 RMSNorm 和 topk。FlyDSL 是可选组件，需要 FlyDSL 0.3.x。FlyDSL joins CuTeDSL, Triton and Helion as a kernel DSL that PyTorch can use. On gfx950, Inductor autotunes FlyDSL templates for GEMM, ragged F.grouped_mm and MXFP4/MXFP8 scaled_mm alongside Triton and ATen, and in eager mode FlyDSL can override RMSNorm and topk. FlyDSL is optional and requires FlyDSL 0.3.x.

在 gfx1250（MI450）上，Inductor 在 dense mm/addmm、逐点运算和归约的代码生成，以及 FlexAttention 前向和解码中使用 Tensor Data Mover。gfx1250 已添加到 nightly wheel 的架构列表中。On gfx1250 (MI450), Inductor uses the Tensor Data Mover for dense mm/addmm, pointwise and reduction codegen, and FlexAttention forward and decode. gfx1250 is added to the nightly wheel architecture list.

API 不稳定API Unstable

（PR #192877、#190903、#194032、#191446 和 #191447 由 AMD 的张小兵提交；#194981 由 AMD 的徐玉涛提交；#196719 由 AMD 的易恒提交；#191166、#196525 和 #197329 由 AMD 的曹格伦提交；#196610 由 AMD 的 Ethan Wee 提交）(PRs #192877, #190903, #194032, #191446 and #191447 by Xiaobing Zhang, AMD; #194981 by Yutao Xu, AMD; #196719 by Heng Yi, AMD; #191166, #196525 and #197329 by Glen Cao, AMD; #196610 by Ethan Wee, AMD)

MPS（Apple Silicon）MPS (Apple Silicon)

FP8、流以及 Apple Silicon 上更快的 MoE 推理FP8, Streams and faster MoE inference on Apple Silicon

MPS 现已支持 torch.float8_e4m3fn，以及支持张量级或行级缩放的 _scaled_mm，并为解码规模的形状提供 GEMV 快速路径。MPS 还新增了一个仿照 CUDA 的公开流 API：torch.mps.Stream、current_stream、set_stream、torch.mps.stream 上下文管理器，以及 Tensor.record_stream。torch.float8_e4m3fn is now supported on MPS, along with _scaled_mm with tensorwise or rowwise scaling and a gemv fast path for decode-sized shapes. MPS also gains a public stream API modeled on CUDA’s: torch.mps.Stream, current_stream, set_stream, the torch.mps.stream context manager, and Tensor.record_stream.

API 不稳定API Unstable

(PRs #194074, #196916, #197862, #196945, #196918 and #196955 by Irakli Salia, Hugging Face; #191415, #193464 and #195867 by Kurt Mohler, OpenTeams; #196395 by Aditya Venkataraman, Meta)

XPU（英特尔 GPU）XPU (Intel GPUs)

SYCL 内核编程简明指南SYCL Kernel Programming Simplified

此版本为 XPU 后端引入了对 SYCL 自由函数内核的支持，大多数现有 SYCL 内核已迁移至新的编程模型。开发者可以用显式的自由函数内核取代冗长的仿函数实现，以更直观的方式编写设备代码，从而降低为 Intel XPU 开发和移植自定义 GPU 内核的门槛。This release introduces SYCL free-function kernel support for the XPU backend, with most existing SYCL kernels already migrated to the new programming model. By replacing verbose functor-based implementations with explicit free-function kernels, developers can write device code in a more intuitive way, lowering the barrier to developing and porting custom GPU kernels for Intel XPUs.

(PRs #196387, #196932, #198200, #198720, and #200223 by Yifeng Cui, Intel)

增强的 XPU 运行时能力Enhanced XPU Runtime Capabilities

此版本扩展了 XPU 运行时功能，新增对 torch.xpu.Event 的 IPC 支持，可在进程间共享事件，并与 torch.multiprocessing 集成。此外，torch.xpu.MemPool 现在支持 use_on_oom 和 non_split 分配策略，为需要处理内存不足情况或利用分布式对称内存工作流的应用提供了更大的灵活性。This release expands XPU runtime capabilities with IPC support for torch.xpu.Event, enabling event sharing across processes and integration with torch.multiprocessing. In addition, torch.xpu.MemPool now supports use_on_oom and non_split allocation policies, providing greater flexibility for applications that need to handle out-of-memory conditions or leverage distributed symmetric-memory workflows.

API 不稳定API Unstable

(PRs #191182, #192495, #192514, #192038 and #192033 by Guangye Yu, Intel)

改进的 XPU 性能分析体验Improved XPU Profiling Experience

此版本新增了对 XPU Graph 重放的内核级可见性、XCCL 集合通信的跨 rank 关联，以及多 GPU 工作负载中所有 Intel GPU 的逐内核硬件计数器采集。这些增强功能可对基于图的、多 GPU 以及分布式训练和推理进行更深入的性能分析。This release adds kernel-level visibility into XPU Graph replay, cross-rank correlation for XCCL collectives, and per-kernel hardware counter collection across all Intel GPUs in multi-GPU workloads. These enhancements enable deeper performance analysis for graph-based, multi-GPU, and distributed training and inference. 

(PR #195128 by Chuanqi Wang, Intel; PR #5486 and #1479 by Adrian Ostrowski, Intel) 

C++ ABIC++ ABI

扩展的稳定 ABI 与仅头文件接口层Expanded Stable ABI and Header-Only Surface

`torch::stable::sort` 加入稳定 ABI；Python 互操作垫片在现有张量转换功能的基础上，新增了 dtype/device 转换和 `is_tensor` 探测功能。更多 `c10/util` 内容（`irange`、`load`、数学常量、类型转换和溢出检查）迁移到 `torch/headeronly`，旧路径则保留为转发包含。现在，如果 `TORCH_TARGET_VERSION` 高于扩展所使用的 libtorch 头文件版本，扩展会在编译时出错；`load_inline` 也可以生成能够安全导入到自由线程 Python 中的 pybind11 模块。torch::stable::sort joins the stable ABI, the Python interop shims gain dtype/device conversions and an is_tensor probe alongside the existing tensor conversions. More of c10/util (irange, load, math constants, type casting and overflow checks) moves into torch/headeronly, with the old paths kept as forwarding includes. Extensions now get a compile-time error if TORCH_TARGET_VERSION is newer than their libtorch headers, and load_inline can generate pybind11 modules that are safe to import on free-threaded Python.

(PRs #198038 by Adrian Abeyta, Red Hat; #194673, #194781 and #193962 by Chris Leonard, Red Hat; #192774 by Paweł Gadziński, NVIDIA; #196802 by Alex Malyshev, Meta)

性能分析与调试Profiling and Debugging

使用 Cuspy 进行 CUDA Graph 内核归因CUDA Graph Kernel Attribution with Cuspy

从 CUDA 图回放的内核现在可以追溯归因到捕获它们的代码。内核注释通过 CUPTI 节点创建回调记录，并由 export_chrome_trace 内联写入；启用 record_py_stacks 后，CUDA 图会捕获每个节点的 Python 启动调用栈。CUPTI 监视器已更名为 Cuspy，并通过 CuspyConfig 启用。Kernels replayed from a CUDA graph can now be attributed back to the code that captured them. Kernel annotations are recorded from CUPTI node-creation callbacks and written inline by export_chrome_trace, and with record_py_stacks CUDA graphs capture the Python launch stack for each node. The CUPTI monitor is renamed Cuspy and is enabled with a CuspyConfig.

API 不稳定API Unstable

(PRs #191999, #194854, #196528, #198419 and #196529 by Natalia Gimelshein, Meta; #196066 by Ryan Zhang, Meta)

弃用项和不兼容的变更Deprecations and Backwards-Incompatible Changes

• 不再支持 Python 3.10。PyTorch 2.15 要求使用 Python 3.11 或更高版本：不再构建适用于 3.10 的 wheel，也不再在 CI 中测试 3.10。不支持在 3.10 上从源代码构建，后续版本还将移除剩余的 3.10 代码路径，包括 torch.compile (Dynamo) 中的相关路径。请参阅 #198233 和 #199417。
  Python 3.10 is no longer supported. PyTorch 2.15 requires Python 3.11 or newer: 3.10 wheels are no longer built, and 3.10 is no longer tested in CI. Building from source on 3.10 is not supported, and upcoming releases will remove the remaining 3.10 code paths, including in torch.compile (Dynamo). See #198233 and #199417.
• nccl 现在会选择重写后的 NCCL 后端。将 TORCH_DIST_USE_NCCL2 设置为 0，或使用 nccl-legacy 后端，即可恢复旧版 ProcessGroupNCCL。参见 #192281。
  nccl now selects the rewritten NCCL backend. Set TORCH_DIST_USE_NCCL2=0 or use the nccl-legacy backend to restore the legacy ProcessGroupNCCL. See #192281.
• torch.ops 和自定义算子的 C++ 警告现在会传递到 Python。参见 #193451。
  C++ warnings from torch.ops and custom ops now propagate to Python. See #193451.
• macOS 14 已不再受支持。PyTorch 2.15 要求使用 macOS 15 或更高版本，才能启用 MPS（Apple Silicon GPU）加速。macOS 14 已停止维护，针对它的 MPS 变通方案也已移除。PyTorch 可能仍可在 macOS 14 上运行，但未经测试，相关问题也不会得到修复。请参阅 #199261。
  macOS 14 is no longer supported. PyTorch 2.15 requires macOS 15 or newer for MPS (Apple Silicon GPU) acceleration. macOS 14 has reached end of life, and the MPS workarounds for it have been removed. PyTorch may still run on macOS 14, but it isn’t tested there and issues on it won’t be fixed. See #199261.
• FSDP2 将分片梯度和未分片梯度的数据类型分开处理。参见 #194434。
  FSDP2 separates sharded and unsharded gradient dtypes. See #194434.