What does it take to trust an LLM-based chat when the provider swears no prompts are ever logged? The question, raised in an online discussion, gets to the heart of an unsolved challenge for anyone building their own LLM apps: turning a promise into a verifiable proof.

The naive answer — “just trust me” — no longer holds, especially after years of scandals around hidden data collection. The reference to Proton, the email service that built its reputation on transparency and encryption, is telling: Proton earned trust through independent audits, widespread open source, and verifiable architectures. Duplicating that playbook in the world of language models, however, presents non-trivial hurdles.

The Secure Enclave You Can’t Afford

The most rigorous cryptographic approach is the Trusted Execution Environment (TEE), an isolated area of the processor where code runs inaccessible even to the system administrator. It would be ideal for running inference without the operator being able to intercept data. The problem, as the developer notes in his reflection, is that TEEs require specialized hardware and resources well beyond a hobbyist project. AMD SEV, Intel SGX, or NVIDIA Confidential Computing demand recent servers and a configuration complexity that confines them to the enterprise. For the average self-hoster, the road is closed.

Open Source and Hashing: Transparency, Up to a Point

The next, more accessible step is to open the repository and even the model, accompanied by a hash attesting to the integrity of the code running in production. The idea: if the source is verifiable and the hash matches, users can check that no hidden logging functions exist. Unfortunately, this strategy leaves large margins of uncertainty. The hash proves that the examined binary is identical to the one running, but it does not prevent the underlying environment (from the operating system to the network layer) from intercepting requests. Moreover, open source and hashing do not guarantee that the production service actually uses that code and only that code, without runtime modifications or malicious proxies.

The On-Premise Route and Data Control

For those who push the principle of sovereignty to its limit, the only true certainty comes from on-premise deployment: the user installs and runs the application on their own machine, examining network traffic and processes firsthand. In this scenario, the developer stops being a provider to be trusted and becomes a mere author of code. This is the model adopted by many self-hosted tools like Ollama or LocalAI: the software is inspectable, execution happens locally, and any exfiltration attempt would be detectable by firewalls or monitoring tools. Naturally, not all users have the technical skills to perform a forensic analysis, but the combination of open code and local data control shifts the burden of proof from the one promising to the one verifying.

Beyond a Single Project: Lessons for Sovereign Deployments

The hobby developer’s story highlights a universal trade-off: the more control you want to give the user, the more burdensome deployment becomes for the developer. TEEs promise trust without sacrificing a centralized service, but they are costly and hardware-dependent. Full open source, while essential, is not enough to certify runtime behavior. In the end, the strongest guarantee shifts to the end user, who must be able to run and inspect the system within their own infrastructure.

For those evaluating on-premise deployments of LLMs, there are trade-offs between verifiable governance and simplified management. The AI-RADAR community explores analytical frameworks at /llm-onpremise to navigate these choices. The case of this small project thus becomes a reminder: building trust means moving the proof from the realm of storytelling to that of inspectability. And in doing so, self-hosting remains the most transparent lever.