Optimizations at the serving layer are improving throughput and utilization: Helion's autotuned GEMM in vLLM, TLX attention on Blackwell, FluidPD prefill-decode elasticity, and BIG-IP Next network balancing all reduce GPU waste.
FluidPD addresses prefill-decode imbalance in LLM serving without adding workers. FluidToken offloads part of prefill work to decode workers when slack is available; FluidRole reassigns running workers between roles without reloading the model. On Az...
The PyTorch team has consolidated image, video and audio decoding and encoding in TorchCodec. TorchVision and TorchAudio now focus on transforms, while models and datasets move to the Hugging Face ecosystem. The change simplifies dependencies and bui...
Helion's linear backend in vLLM shows a single GEMM kernel with an ahead-of-time autotuner can cover specialized variants and improve inference by up to 10% on Qwen3. For self-hosted LLM teams, the signal is twofold: less dependence on deep CUDA skil...
Red Hat and Meta integrated Helion into vLLM's linear backend. A single GEMM implementation covers Standard, Split-K, and Swap-AB variants, with per-shape autotuning on NVIDIA Hopper GPUs. On H100 tests, quantized kernels outperform CUTLASS, FlashInf...
F5's test on BIG-IP Next for Kubernetes shows that the same GPU cluster can deliver more than triple the performance by tuning traffic balancing and routing. For on-premise deployments, node-level efficiency becomes central to TCO; the risk is coupli...
Meta's GEM attention kernel, rewritten in TLX, beats FlashAttention-4 on jagged Blackwell workloads: +13% forward and +50% backward on average, with about 3,200 lines versus 10,000. The result shows explicit hardware control no longer requires hand-w...
In F5's lab, using BIG-IP Next for Kubernetes showed more than triple performance on the same GPU cluster. The result shifts focus from raw compute to network and orchestration efficiency: for on-premise LLM deployments, data traffic can be the real ...
