PyTorch 2.10 is now available with a series of optimizations aimed at improving performance and simplifying numerical debugging. This release includes the work of 536 contributors, with over 4160 commits since version 2.9.

Key Features

  • Python 3.14 Support: torch.compile() now supports Python 3.14, including the freethreaded build (experimental).
  • Combo-kernels: Reduced latency thanks to horizontal kernel fusion in TorchInductor.
  • varlen_attn(): New operation for handling ragged and packed sequences.
  • DnXgeev: Efficient eigenvalue decomposition execution on NVIDIA GPUs.
  • use_deterministic_mode: torch.compile() now respects deterministic mode.
  • DebugMode: Tool for tracking calls and facilitating the debugging of numerical divergences.

Numerical Debugging

Determining behavior across multiple runs is crucial for debugging. PyTorch 2.10 enables this functionality via torch.use_deterministic_algorithms(True), ensuring consistency of operations even with torch.compile().

DebugMode offers advanced features such as runtime logging, tensor hashing, and dispatch hooks to isolate and analyze numerical divergences.

Other News

  • Torchscript Deprecated: Torchscript has been deprecated and replaced by torch.export.
  • tlparse & TORCH_TRACE: Tools to simplify reporting compiler-related bugs.
  • Release Cadence: Starting in 2026, the release cadence will increase from quarterly to bi-monthly.