Token is no longer the universal unit of measurement

With the LFM3 family, Liquid AI has introduced a distinction most benchmarks do not capture: a model can be designed to produce text token by token, or to return an already calibrated decision. The two decision models, d1-Omni-2B and d1-2B, do not generate tokens: they perform a single forward pass and return structured answers with declared latencies of 5.8 ms on NVIDIA H100 and 3.7 ms on AMD MI325X for the first, and 5.7 ms and 3.8 ms for the second.

This architectural choice has a deep consequence. Computational cost does not scale with the length of a generated sequence, because there is no autoregressive generation pipeline. For tasks such as extracting text, numbers, dates, or estimating quantities with an interval, the bottleneck is no longer token throughput, but model activation time and memory management. Teams designing on-premise systems can size hardware around latency and footprint, not around billable token volumes.

The text model LFM3-1.7B shares the same overall approach but generates tokens. Its 24-layer tri-hybrid architecture combines 13 short gated convolutions, 5 sliding-window attention layers, and 6 global attention layers. It is not a classic transformer, and this partly explains the 256K token context and the small footprint in 4-bit quantization. For AI-Radar, the point is not the architectural detail in itself, but the fact that the design constraint has shifted: device resources come first, then model metrics.

One gigabyte of VRAM changes the deployment map

The 4-bit model occupies about 1 GB of VRAM. That single figure reopens deployment scenarios that large cloud generalist models had made marginal. A 1.7 billion parameter LLM in 4-bit quantization can run on common phones and laptops, but also on self-hosted microservices without dedicated GPUs. For organizations with data sovereignty constraints, this means keeping inference within their own boundaries without bearing the cost of high-end accelerated infrastructure.

llama.cpp support has already been integrated. This is not a minor technical detail. Teams planning local deployments on consumer or edge hardware can use established runtimes, without depending on proprietary stacks or cloud services. This reduces integration friction and makes it easier to distribute models across heterogeneous fleets, with update cycles managed internally.

Liquid AI notes that fine-tuning the text model requires a single GPU. This shifts the center of gravity from centralized training to distributed customization. A team operating in a regulated context can train local variants on proprietary data, maintaining control over the data flow and reducing the exposure surface. The TCO of such an approach is not limited to hardware cost: it includes the ability to avoid API contracts, transfer limits, and operational dependencies.

Decision models and TCO: the marginal cost that disappears

The decision models hit an exposed nerve in the cloud API provider business. If a billing service is based on generated token volume, a model that responds in milliseconds without generating tokens drastically reduces the revenue unit. This is not just about efficiency: it is a category of workload that can leave the cloud perimeter and move to edge or on-premise without losing speed.

For the declared decision models, latencies on server hardware are in the range of a few milliseconds. On edge devices the picture changes, but the principle remains: when the answer is a calibrated value and not a textual sequence, marginal cost does not grow with the syntactic complexity of the request. This makes it possible to automate repetitive decisions on images, audio, or structured data with a predictable computational budget.

A trade-off must be observed. The d1 models are not designed for agentic coding or multi-step reasoning, and Liquid AI itself advises against using them in those contexts. They are specialized tools for extracting information and producing immediate decisions. Adopters must accept a narrower functional scope in exchange for latency and footprint. In AI-Radar terms, this is a deployment choice, not an absolute quality ranking.

Who pays the bill for specialization

The LFM3 family is not a single model, but a matrix of specializations: vision, audio, omni-modal input, and decision. LFM3-VL-2B accepts text, images, and video; LFM3-Audio-2B handles audio processing with multilingual speech synthesis in 18 languages; LFM3-Omni-2B replaces ASR, VLM, LLM, and TTS in a single checkpoint. This modularity has a management cost: each variant must be evaluated, integrated, and maintained separately.

Who gains? Local agent developers, SMEs that want automation without sending sensitive data to third parties, and robotics teams that need low-latency spatial and temporal reasoning. For them, the availability of compact self-hosted models reduces dependence on APIs and enables more direct control over data and performance.

Who loses? Cloud API providers that monetize token consumption on generalist models. If the decision model responds without generating text, the billable token volume drops sharply. Teams that have built pipelines around generalist LLMs may also need to revise their assumptions: not all workloads require a large model, and not all problems are solved with more parameters.

What to watch in the coming months

Liquid AI's move is part of a broader trend toward architectural efficiency and deep specialization. The signals to monitor are concrete: independent benchmarks on extraction and decision tasks, integrations with self-hosted frameworks, hardware support for 4-bit quantization, and distribution patterns on edge devices. It remains to be seen whether the declared performance holds outside the lab.

A second front concerns the update and maintenance of local models. The ability to fine-tune with a single GPU opens interesting scenarios, but also demands clear versioning, validation, and rollback processes. Production operators must consider lifecycle management costs, not just service costs. Data sovereignty is not achieved with a single deployment, but with repeatable procedures.

Finally, the real test will be adoption by teams building edge and on-premise applications. If the decision models prove robust enough for real tasks, they could reshape the economics of local inference: fewer tokens, more decisions, more modest hardware. If quality remains limited to very narrow use cases, they will remain an interesting but not structural niche.