Prompt injection: a real risk for self-hosted LLM systems

A user reported a serious prompt injection vulnerability in their self-hosted LLM system. During a testing phase, a QA team member managed, through a specially crafted prompt, to obtain the disclosure of the entire system prompt.

This incident highlights a critical issue: the difficulty of protecting LLM systems from prompt injection attacks. Traditional Web Application Firewalls (WAFs), designed to protect web applications from common threats, are unable to recognize and block malicious prompts.

The problem lies in the fact that the LLM model interprets prompts, even malicious ones, as normal user input, executing them accordingly. This behavior makes systems vulnerable to various types of attacks, including the disclosure of sensitive information and the manipulation of the model's behavior.

Protecting against prompt injection requires more sophisticated approaches than simple input sanitization, as malicious prompts can be formulated to appear completely harmless.