Mistral has opened a direct channel with developers, and the response was swift. The new survey launched by the French company, designed to gather input on its roadmap, surfaced a dominant theme: the community wants larger, open-weight models, in the 30–120 billion parameter range, built to run on local hardware.

This isn't just a technical footnote—it's a sign that an entire ecosystem is maturing. The message is clear: interest in self-hosted LLMs isn't confined to compact 7B or 13B models, nor to Mixture-of-Expert architectures like Mixtral. There's hunger for generative capability and reasoning power that only mid-sized models currently offer, but without delegating them to cloud infrastructure.

A gap between supply and local demand

The data highlights a disconnect between what Mistral has released so far and what the technical base considers usable on its own infrastructure. On one side, efficient and fast models like Mistral 7B, but with limitations in complex tasks. On the other, Mixtral 8x22B, an MoE architecture that still requires significant VRAM unless heavily quantized. Missing are open weights in the 30B to 120B range—the same sweet spot where models like Llama 3 70B or Qwen 72B are gaining traction and adoption in on-premise environments.

Those pushing for this tier have specific scenarios in mind: enterprise servers, internal GPU clusters, multi-card workstations. With 4-bit quantization, a 70B model needs around 35 GB of VRAM, well within reach of two NVIDIA RTX 4090s or a single A6000 Ada. This makes inference on private hardware not just technically feasible, but economically sensible for organizations that don't want to send sensitive data to the cloud. It's not just about cost: it's a strategic stance on data sovereignty.

Winners and losers

The community pressure on Mistral has implications that go well beyond the company's perimeter. If Mistral were to answer with an open-weight model in this bracket, it would ignite direct competition with Meta and Alibaba in the mid-sized on-premise LLM space. Users would gain performant, permissively licensed alternatives, accelerating the race for local inference optimizations and reducing reliance on proprietary APIs. Hardware makers—NVIDIA with its consumer and professional GPUs, but also AMD and emerging chip designers—would see increased demand for high-bandwidth-memory multi-GPU systems, precisely because models this large are the ideal workload to justify investment in dedicated workstations.

On the opposite side, pure cloud services stand to lose ground: if inference of 30–120B models becomes practically achievable on-premise at contained costs, part of the traffic currently feeding cloud APIs could shift to private infrastructure, moving value from service delivery to model and hardware ownership. For companies offering LLM behind APIs, this means differentiating not just on model scale, but on enterprise features and integration, while commodity hardware captures the bulk of low-latency, high-privacy workloads.

What it signals structurally

The demand that emerged from Mistral's survey isn't just a forum wish: it's confirmation that the on-premise market for generative AI is organizing itself around precise parameter bands, and that the open-source community no longer settles for being the lab for experimental small models. It wants tools that compete with large proprietary models—but deployed in its own data center. And it's telling providers like Mistral that the road to building a loyal ecosystem is not paved only by efficiency or attention techniques, but also by the willingness to release weighty checkpoints, without restrictions, in the size that private machines can digest.