Prefill-decode disaggregation has become a common architecture for LLM serving because it separates two phases with different execution profiles and SLO objectives. The difficulty is that most systems combine this separation with a rigid ratio between workers dedicated to prefill and workers dedicated to decode. That works as long as real traffic matches the planned sizing. When it does not, latency breaches SLOs even if idle capacity exists elsewhere. FluidPD starts from this mismatch.

The system introduces two mechanisms. FluidToken handles transient imbalance by offloading a bounded portion of prefill computation to decode workers when decode-side slack is available. FluidRole addresses sustained changes by reassigning already-running workers between prefill and decode roles without reloading the model or restarting the inference engine. Both are guided by lightweight pressure indices designed to expose resource pressure before it turns into an SLO violation.

Reported results on production Azure traces are striking: up to 94.6 percentage points higher overall SLO attainment compared with static SGLang, without adding new workers. This is not an isolated benchmark; it is a structural signal. Prefill-decode imbalance is not a problem to be solved by buying more GPUs. It is a problem of orchestration and timing.

That shifts the debate. For teams running fixed on-premise GPU fleets, where adding a unit is a budget decision, the ability to reassign roles without stopping the engine has immediate value. It means less time lost to model reloads, less reserve capacity sitting idle, and faster response to short bursts. This is not just operational efficiency; it is TCO. Total cost falls when the same hardware serves more load without breaking service levels.

There are less visible implications. If in-place elasticity becomes standard, the quality of orchestration software becomes a stronger competitive factor than the number of installed accelerators. Operations teams will need to look beyond average utilization and watch pressure indices that anticipate bottlenecks. Serving providers will be judged not on how much capacity they can add, but on how well they use the capacity already present. In that sense, the real opponent of FluidPD is not another serving system but the assumption that elasticity always requires new resources.

Azure traces have specific characteristics, and the 94.6 percentage point gain should not be projected automatically onto every context. But the direction is clear: prefill-decode disaggregation needs fine-grained adaptation, not a fixed ratio. For teams evaluating on-premise deployment, AI-RADAR offers analytical frameworks at /llm-onpremise to compare trade-offs among fixed capacity, orchestration, and SLO compliance. FluidPD is a concrete case of serving infrastructure maturing without waiting for the next hardware generation.