Measuring the return on AI investments has always been opaque. Until recently, cost-per-token was the most common proxy: the fewer you spent per token generated, the more efficient the system. The advent of autonomous agents — software that chains actions, tools, and multiple model calls to complete a task — makes that metric dangerously misleading. An agent can consume thousands of tokens for a result that a single well-crafted prompt could have achieved with a tenth of the computational effort. Counting tokens becomes like measuring office productivity by pages printed: an indicator of activity, not of value.

The source highlights three levers: measuring useful work per dollar, improving efficiency, and scaling high-value workflows. It’s a paradigm shift that moves focus from purely technical metrics (throughput, latency) to a composite economic indicator: what is the real cost of completing a business task? This means integrating the entire pipeline into the calculation — orchestration, retrieval, validation, possible human intervention — not just LLM inference.

Organizations running on-premise infrastructure have a head start in this race. In a private data center, the marginal cost of an additional token is almost zero once hardware is amortized. Useful work per dollar rises when GPUs are saturated with continuous loads, models are reused after fine-tuning on proprietary data, and the cost multipliers of cloud providers at high volumes are avoided. Conversely, a pay-per-use cloud architecture can quickly become uneconomical as soon as an agent multiplies API calls without a corresponding increase in generated value.

This doesn’t mean cloud is doomed, but that the on-premise versus cloud decision must now be made with different analytical tools. The useful-work-per-dollar metric rewards those who control the stack: the ability to choose quantized models to reduce VRAM footprint, balance multiple models on the same node, optimize inference queues with frameworks like vLLM or TensorRT-LLM, and decide which workloads to keep local for latency or data sovereignty. The second-order implications are deep. Hardware vendors will see more stable demand for GPUs with high-bandwidth memory (long-term contracts), while cloud providers may be forced to offer task-based pricing instead of token-based if they want to compete with on-premise TCO.

There is also a third-order effect on research and the open-source ecosystem. If useful work per dollar becomes the yardstick, smaller, highly specialized models — perhaps obtained through fine-tuning on vertical datasets — become more attractive than giant general-purpose ones. The incentive to publish benchmarks measuring real operational effectiveness (task completion) rather than academic scores strengthens. For those building local stacks, this means being able to transparently demonstrate the cost per completed task, a decisive argument in investment committees weighing high initial CapEx against unpredictable variable OpEx.

Ultimately, the agentic era forces enterprises to rethink not only how much they spend on AI, but what they get in return. And for those betting on on-premise deployment, the new metric could be their strongest ally.