LLMs Under Attack: The Evolution of Red-Teaming

Large Language Models (LLMs) are increasingly integrated into high-stakes applications, making safety a top priority. Current research focuses on identifying vulnerabilities through predefined prompt sets. A new study published on arXiv highlights how this approach ignores realistic attack scenarios, where inputs are continuously refined to overcome protections.

Automated Prompt Optimization: A New Attack Vector

The research explores the use of black-box prompt optimization techniques, originally designed to improve performance on benign tasks, to systematically search for security flaws. Using DSPy, the authors applied three optimizers to prompts drawn from HarmfulQA and JailbreakBench, explicitly optimizing toward a continuous danger score provided by an independent evaluator model (GPT-5.1).

Vulnerabilities Amplified in Open-Source Models

The results demonstrate a significant reduction in safety safeguards, with particularly marked effects on smaller open-source language models. For example, the average danger score of Qwen 3 8B increases from 0.09 in its baseline setting to 0.79 after optimization. This suggests that static benchmarks may underestimate residual risk, making automated and adaptive red-teaming necessary for robust safety evaluation.

For those evaluating on-premise deployments, there are trade-offs to consider. AI-RADAR offers analytical frameworks on /llm-onpremise to evaluate these aspects.