A nearly 2.7-billion-parameter LLM running on a smartphone without a GPU, at 17 tokens per second, is no longer a niche experiment: it signals that on-device inference is becoming a concrete alternative to the cloud for a certain class of workloads. A developer demonstrated this by getting LFM2.5-2.6B running on a OnePlus 13, using only the phone’s CPU.

The model – built for multi-step agent workflows and sporting a 128K-token context window – was executed in the Q4_K_M GGUF quantization. This 4-bit compression drastically reduces memory footprint while maintaining acceptable quality for many tasks. The inference engine, built from scratch, weighs just 450 KB and communicates with the device over ADB through a text-based UI that doubles as a system probe. Despite its minimal size, the runtime supports several architectures: Qwen, Gemma, Bonsai, and others.

The current 17 tok/s is already useful for conversational interactions, but the declared target is to reach 30 tok/s – a threshold that would make the experience indistinguishable from many cloud solutions. Hitting it would shift the conversation around local AI from technical feasibility to everyday convenience and integration.

What stands out is not just the raw number, but the ecosystem it hints at. A 450 KB engine means a reduced attack surface, nimble updates, and the ability to be embedded into consumer apps without bloat. Support for heterogeneous models breaks vendor lock-in, a critical advantage for anyone prioritizing data sovereignty or operating in air-gapped environments.

For organizations evaluating on-premise or edge deployments, the OnePlus 13 test offers tangible proof: no dedicated GPU or bulky server is required to run LLMs with multi-step reasoning capabilities. There are of course memory constraints (the quantized model on a phone takes around 1.3 GB, manageable even on mid-range devices), compute power, and energy efficiency to consider. But the signal is clear: the race toward compact models and optimized runtimes is lowering the hardware floor for local inference, tipping the balance from cloud to device.

This isn’t the only experiment of its kind, but the pairing of an agentic-workflow LLM and an ultra-lean engine offers a glimpse of what could become routine: truly autonomous personal assistants, industrial applications that never transmit sensitive data beyond the corporate perimeter, and a new competitive lever for software developers unwilling to depend on third-party APIs.

The leap to 30 tok/s, if confirmed, could accelerate an already unfolding trend. In the meantime, demonstrations like this remind us that AI innovation doesn’t only flow through data centers – it also springs from finely tuned code for hardware we already carry in our pockets.