Forty-eight cents a day. That’s the worst-case energy cost of running an AMD Strix Halo system for LLM inference around the clock, with CPU, GPU, and NPU all under simultaneous load for model serving and compilation. The figure, shared by a Reddit user, flips a common narrative: that consumer-grade hardware for self-hosted LLMs is always inadequate because it’s too slow.

The user describes a setup running a Qwen 35 billion parameter model with Q8_XL quantization, achieving 50 tokens per second. That’s not a record compared to an NVIDIA A6000 or other enterprise cards. But the context makes the difference: the device is the size of a small router, silent, and draws less than half the power of a single A6000 (300W for the card alone). The Strix Halo APU, with integrated graphics and NPU, keeps the whole system well under 150W. And when inference isn’t saturating the resources, the CPU and NPU remain available for other services, turning a development box into a versatile on-premise node.

The discussion hits a sensitive spot for anyone evaluating self-hosted Large Language Models. Standard metrics – VRAM, memory bandwidth, raw throughput – tell only part of the story. A system that runs 24/7 with minimal thermal and acoustic footprint opens concrete scenarios for offices, labs, and edge environments where data sovereignty and operational continuity matter more than cloud latency. This isn’t about replacing discrete GPUs in data centers; it’s about redefining the scale of “good enough”. Fifty tokens per second can suffice for summarizing documents, querying internal knowledge bases, or prototyping agents without sending a single prompt to external servers.

The comparison with Apple Silicon Macs, mentioned in the post, is no coincidence. The Apple ecosystem has shown that a unified architecture with fast memory and dedicated accelerators can run large models at efficiency levels unattainable for a discrete GPU at similar power. Strix Halo brings that philosophy to x86, integrating an NPU that AMD rates for up to 50 TOPS and a GPU with memory bandwidth potentially around 500 GB/s (as simulated on comparable models). The key is the ability to leverage aggressive quantization without unacceptable performance degradation, as the user’s Q8_XL experience demonstrates.

For those designing on-premise deployments, the message is clear: total cost of ownership isn’t measured only in dollars per token. Energy use, heat dissipation, physical footprint, and the ability to repurpose spare compute cycles become decisive factors when the system is meant to run continuously. That $0.48 daily figure, though based on an unspecified electricity rate, is a tangible sign: with mixed and sustained workloads, power consumption can fall below the radar, making local inference not just feasible but economically justifiable even in non-enterprise settings.

Of course, limitations remain. A 35 billion parameter model with aggressive quantization can’t match the accuracy of larger, less compressed versions. And 50 tokens per second won’t handle low-latency interactive applications with many users. But the point isn’t winning the speed race. It’s proving that LLM self-hosting is moving past the experimental phase, into a space where overall system efficiency – not just raw GPU power – drives architectural choices.