When Dario Amodei, CEO of Anthropic, says that open source models could lead us to a «very dangerous place», he is not talking about science fiction. He is pointing to a concrete, immediate risk — one that for those managing on-premise AI infrastructure translates into a critical question: to what extent does full control over an LLM also guarantee safety?

A statement that shakes the AI community

Amodei, a long-time advocate for rigorous AI safety, voiced his concern at a time when open source Large Language Models are spreading like never before. Llama, Mistral, Falcon and dozens of other freely available LLMs have democratized access to the technology, enabling even non-cloud-native organizations to run inference locally. Yet this very ease of distribution, according to the Anthropic CEO, hides a potential threat: without the safety filters and guardrails developed by companies like Anthropic or OpenAI, an open source LLM can be used to generate harmful content, large-scale disinformation or automated cyberattacks. His warning shifts the debate from the philosophical plane to the practical realm of deployment decisions.

Open source: transparency or risk?

The open model offers undeniable advantages for those choosing self-hosted deployments: full code audits, customization up to fine-tuning on proprietary data, and no contractual lock-in with third-party vendors. However, the transparency that makes these LLMs so attractive in regulated environments — think of GDPR compliance while keeping data within your own infrastructure — is also what makes them harder to contain. A closed model, delivered via API with built-in guardrails, reduces the potential for abuse but introduces vendor dependency and limits technological sovereignty. For on-premise deployments, the trade-off is even starker: the team managing the deployment must take direct responsibility for all mitigation measures, from input/output filtering to continuous conversation monitoring. Having the model running on local GPUs is not enough to be safe.

Safety and sovereignty: the on-premise dilemma

Amodei’s comment strikes at the heart of a growing tension. On one hand, organizations in highly regulated sectors — healthcare, finance, defense — see on-premise deployment as the only way to maintain data confidentiality and meet digital sovereignty requirements. On the other, those same organizations must ask whether an open source LLM, without a robust safety layer, can actually be used in production without exposing them to legal and reputational risks. It is no coincidence that AI-RADAR’s evaluation frameworks extensively address these aspects: for an on-premise deployer, the model choice is only the first step; equally critical is designing a pipeline that includes quantization tools, controlled inference and auditing, avoiding the “gaps” that a completely open model could inadvertently create.

Balancing innovation and responsibility

The industry is already striving to meet this challenge. Collective safety initiatives, such as community red teaming on open source LLMs and the development of moderation-focused libraries (like Guardrails AI or NVIDIA NeMo Guardrails), show that the community is aware of the dangers. Quantization itself — often used to run models on hardware with limited VRAM — can introduce unexpected behaviors, forcing teams to validate each checkpoint meticulously. For those evaluating on-premise deployments, the lesson is clear: the «dangerous place» evoked by Amodei is not an inevitable consequence of open source, but a risk that can only be governed through an equally open and verifiable safety architecture, integrated into the model’s lifecycle.

Beyond the debate: tools for informed decisions

Choosing between an open or closed model cannot be reduced to ideology. Every deployment context — cloud, hybrid or on-premise — carries TCO, compliance and compute capacity constraints. For those moving in the self-hosted space, platforms like AI-RADAR provide analytical frameworks to weigh the factors at play: from inference latency on local hardware to privacy requirements, without falling into oversimplifications. The real «dangerous place» would be making decisions without a map.