A return to origins: direct patch against extensibility

The developer known for scx_flow, a sched_ext-based scheduler, has chosen not to continue down that path. For his new project, Infinity Scheduler, he has gone back to the traditional approach: a set of patches that directly modify the Linux kernel's Completely Fair Scheduler (CFS) and Real-Time (RT) behavior. This is not the first time the community has attempted to improve CPU scheduling, but the decision to forgo the extensible BPF framework and surgically modify the kernel reopens concrete questions about maintainability, performance, and adoption.

What changes in CFS and RT

CFS has been Linux's default scheduler since 2007, designed to fairly balance CPU allocation among processes. RT handles tasks with strict timing deadlines. Modifying them means acting on the operating system's backbone, influencing latency, throughput, and responsiveness for every workload. Unlike sched_ext-based solutions — which load scheduling logic into a BPF program without touching the kernel — Infinity Scheduler integrates directly into the source code, potentially offering finer control but requiring continuous rebasing with each new release. Technical details haven't been thoroughly disclosed yet, but the approach suggests a focus on reducing latency and managing NUMA bottlenecks, critical for multi-socket servers.

Why it matters for those running LLMs locally

AI workloads, especially Large Language Model inference on CPUs, are highly sensitive to scheduling quality. Tools like llama.cpp use all available cores and demand efficient allocation to prevent threads from haphazardly competing for resources, thereby increasing token generation latency. In on-premise environments, where Total Cost of Ownership also depends on machine saturation, a scheduler capable of cutting wait times and improving parallelism on NUMA architectures can make the difference between a sustainable deployment and one that wastes resources. Infinity Scheduler, by intervening on CFS and RT, could offer gains on these fronts, albeit with the uncertainty of not being upstream.

Outside the official tree: the maintainers' chess game

The decision not to use sched_ext comes at a cost: out-of-tree patches require constant effort to remain compatible with the evolving kernel. BPF-based schedulers, by contrast, can be updated as userspace programs, simplifying distribution and maintenance. Yet those chasing top performance often favor direct modification, aware that upstream integration may never arrive. In a local, self-hosted context, where administrators control their infrastructure and do not have to adhere to cloud standards, such patches find fertile ground. The open question is whether, with an expanding ecosystem of extensible schedulers, Infinity Scheduler's path represents a viable alternative or a dead end.

This news lands at a time when infrastructure efficiency is under intense scrutiny. Those managing dedicated on-premise AI servers will watch the first metrics with interest, knowing that every millisecond saved in scheduling translates into faster token generation and, ultimately, a better return on hardware investment.