Meredith Whittaker, president of Signal, chose unambiguous words to remind everyone interacting with a conversational assistant what really lies behind the screen: «These are not your friends. These are not conscious beings. These are not sentient interlocutors.» This warning comes not from a generic academic, but from the head of one of the most privacy-focused apps, and it resonates strongly as LLM-based chatbots proliferate in both consumer apps and enterprise settings, increasingly deployed on-premises.

Whittaker’s statement strikes the core of a subtle but dangerous psychological dynamic: the tendency to humanize statistical tools that produce coherent text without any real understanding. For those managing self-hosted infrastructure, the trap is twofold: direct control over data and models creates an illusion of extra safety, while the end user lowers their guard because the assistant “runs on our servers” and therefore appears more harmless.

The pattern matcher behind the speaking interface

Large language models – Llama, Mistral, Phi, and the various open-weight forks – do not reason, feel emotions, or have intentionality. They are systems trained to predict the most likely token sequence given an input string. The conversational interface deceives us: fluent answers and a friendly tone trigger human relational patterns. Yet every generated sentence is the output of statistical calculations on a training corpus, with no mental states or episodic memory. In on-premise enterprise deployments, this distinction may be obvious to the MLOps team, but it is far from obvious to colleagues in other departments who start confiding professional fears or project data to a “friendly” chatbot.

The control paradox: on-prem is not a magic curtain

Bringing the model into one’s own data center guarantees data sovereignty and regulatory compliance, two pillars that AI‑RADAR systematically analyzes in self-hosting assessment paths. However, technical control does not automatically extend to human behavior. An internal assistant can become an escape route for confidential information if the company culture fails to clearly distinguish between a tool and a confidant. The risk is not model hacking, but the mundane sharing of trade secrets, credentials, or strategies with a system that, however isolated, cannot protect them and may literally reproduce them in a later session, especially without context-clearance mechanisms.

Beyond compliance: “perception sovereignty”

Whittaker’s warning shifts the focus from data sovereignty to what we might call perception sovereignty. It is not enough that logs remain in the company rack; those who interact with the model must be fully aware they are speaking with a statistical artifact. This brings into play often underestimated design choices: the interface must persistently signal the non-human nature of the assistant, session logs should be analyzed for overly confidential exchange patterns, and usage policies must be paired with specific training. In regulated environments like GDPR, the gap between “data accidentally shared with an internal system” and “lawfully processed data” can narrow to the point of becoming a legal liability.

On-premise: a lever for accountability, not a shield

The lesson for those choosing local stacks is clear: on-premise remains the prime path for governing information flows, but it must be accompanied by a strict accountability framework. Monitoring interactions without violating user privacy, setting content filters, and limiting long-term memory usage are all moves that reduce the risk surface. AI‑RADAR has long devoted special attention to these trade-offs, because the decision to adopt a self-hosted LLM is not only technical, it is also organizational. Whittaker’s warning does not halt chatbot adoption, but it reminds us that bringing them in-house multiplies responsibility, it does not erase it.