Owners of a single AMD Radeon R9700 now have a concrete reason to revisit local inference. The updated radiance-vllm-mxfp4 repositories bring a reported doubling of performance through software optimizations aimed at that single-card configuration. The measurements were made using Unsloth's Qwen3.8 27B in NVFP4 format.

The numbers show a consumer card that is no longer a fallback option: in decode, structured workloads reach 153.1 tokens/s on JSON output, 141.7 on summarization, 138.0 on file editing, and 120.5 on code. Chat and prose stay below 70 tokens/s, a gap that reflects different token distributions. Median update latency is around 42 milliseconds for chat and about 34 for summarization and reasoning.

Prefill holds above 3,500 tokens/s up to 16,000 tokens of context, peaking at 3,619 tokens/s, and remains above 3,100 tokens/s at 64,000. With concurrent requests, aggregate throughput scales from 120 tokens/s with a single stream to 471 tokens/s with eight.

The less obvious reading is not the peak figure but the shift in bottleneck. NVFP4 is a 4-bit format that reduces weight footprint and memory bandwidth pressure; the MXFP4 kernels in this vLLM fork take advantage of that compression. On a single card, this moves the constraint from raw silicon to software quality. For local serving, the default answer used to be multi-GPU boxes or cloud APIs; now a single slot with the right runtime can handle a 27-billion-parameter quantized model.

The second-order effect is economic and organizational. A lab working on proprietary code, a law firm with confidentiality constraints, or an IT team with data residency obligations can consider a smaller hardware estate. Upfront costs drop, but complexity shifts to quantization pipelines, runtime updates, and latency monitoring. Vendors that treated multi-GPU servers as the default for self-hosting now face a cheaper alternative that does not necessarily sacrifice tokens/s.

The broader signal is the widening of AMD's role in local inference. Custom optimizations for 4-bit formats reduce the advantage that depends purely on NVIDIA hardware. This does not erase memory or compatibility gaps, but it moves competition onto frameworks and runtimes. For on-premise planners, TCO can no longer be calculated by looking only at the GPU; it includes serving stack maturity, specialized kernels, and the ability to update models without rebuilding infrastructure.

For those evaluating on-premise deployments, there are trade-offs between hardware cost, operational complexity, and data control; AI-RADAR's /llm-onpremise section covers these analytical frameworks.

The next question is not whether a single Radeon R9700 can handle real workloads, but how far the same approach can go with larger models and wider context windows. For now, this release makes concrete an option that until recently seemed limited to lab environments.