One morning in 2019, in a Cape Town hotel room, Adebayo Alonge was about to demonstrate his startup’s answer to a deadly African healthcare problem: counterfeit medication, which kills thousands every year. The Rxscanner, a handheld spectrometer, shone infrared light on a pill, sent its molecular profile to an AI model hosted in a U.S. data center, and waited for a verdict. That day the connection was painfully slow: “It took over five minutes for a single scan,” Alonge recalls. The demo was on the brink of failure.

His engineering team reacted within two hours: they shrunk the AI model enough to run entirely on Alonge’s Android phone, removing any reliance on the cloud. It worked, and that emergency fix gave birth to a version of the device that now authenticates pills anywhere in the world, even without broadband, computers, or reliable electricity. What we now call “small AI” was born, in practice.

The Power of “Just a Few Watts”

It’s not an isolated case. In India, a drone-mounted system photographs cashew plants and identifies disease blotches: all processing happens on board, with zero remote servers. In Uruguay, a compact model detects ant infestations in vineyards. In Brazil, an electrocardiogram runs on an Arduino in communities where a hospital is a distant dream. “This is the most important area in AI today,” says Marcelo José Rovai, a professor at the Federal University of Itajubá who has contributed to several of these projects. “It’s growing very fast.”

At the heart of small AI is a paradigm shift: models with at most a few billion parameters, obtained by pruning, distillation or quantization of larger models, capable of running on devices that consume just a few watts. The new Arduino UNO-Q, featuring a Qualcomm chipset and costing about 50 dollars, can run a language model that analyzes sensor data to detect tiny pools of water where mosquitoes might breed – all on 3 watts.

This isn’t technological charity. It’s a necessity driven by real constraints: according to the World Bank, only 0.7% of internet users in the poorest countries have ever used ChatGPT, compared with a quarter in high-income nations. “We discuss LLMs and generative AI, but those require computing power, electricity, massive data, and skilled people,” World Bank President Ajay Banga reminded the audience at Davos. “Outside the developed world, very few countries have that combination.”

Beyond the Gap: When Local Makes Sense

Yet the phenomenon goes well beyond access in developing nations. The push for models that run entirely on-device – never leaving the machine they operate on – speaks the language of data sovereignty, granular control, and total cost of ownership (TCO). For a company managing sensitive health, legal, or industrial data, being able to perform inference locally without touching an external cloud means shrinking the attack surface, staying compliant with regulations (from GDPR upward), and slashing the operational expenses of large-scale API calls.

Hardware is doing its part. In 2025, a third of smartphones shipped worldwide were already capable of running generative AI thanks to integrated neural processing units (NPUs); by year’s end that share will rise to 45 percent, according to Counterpoint, and next year the majority of phones will be “AI-ready.” No hyperscale data center is required. And open-weight models like Google DeepMind’s Gemma 4 or Alibaba’s Qwen 3.5 allow users to retrain and adapt the model on specific data – say, from the dairy industry – creating vertical tools that run on modest hardware.

This shift doesn’t erase giant models. They are still needed to distill compact “students” and to push research forward. But it signals a structural bifurcation: on one side, the AI of tech giants – centralized, extremely costly, accessible to a few; on the other, an ecosystem of specialized, affordable, and private models growing at the edge of the network. “The future of AI isn’t a giant model at the center, but millions of small, precise models deployed everywhere, each solving a specific problem,” Alonge argues.

The sticking point remains basic infrastructure: stable electricity, working supply chains, trained talent. Even the drug scanner, autonomous for days, must periodically sync to update molecular signatures. “Even when you use batteries, reliable power is essential. That phone battery won’t last forever,” Alonge reminds us. Politics will have to decide whether to invest to sustain this ecosystem long term. But the direction is set: the AI that touches the most lives will be the one that can operate silently, on device, far from the spotlight of mega-data centers.