To understand whether an aligned LLM really resists jailbreaks, traditional automated methods require the model to produce a full response and then measure whether the attack succeeded. It is a slow and expensive process, and often not very informative: on strongly aligned models, most attempts are rejected with the same negative outcome, leaving few useful signals to guide the search for new variants. The NeuronFuzz framework, described in a new paper, shifts the observation point: instead of waiting for generated text, it reads the activation of internal safety neurons.

The technical core is a SafetyOracle that converts safety neuron activations into a continuous safety alarm score. This score is obtained during prefill, before and without autoregressive response generation. Fuzzing no longer has to wait for full decoding, cutting the computational load per candidate. It is not just about speed: the continuous signal makes it possible to distinguish risk levels even when the model refuses, offering denser guidance for mutating templates.

To build the SafetyOracle, NeuronFuzz selects a compact set of neurons using template-invariant harmful and benign inputs and stability-aware selection. The score is differentiable, so the framework uses its gradients to identify safety-sensitive template positions and a masked language model to generate fluent, context-compatible mutations while preserving the original harmful payload and avoiding additional optimization variables.

The numbers reported by the authors give a sense of the effectiveness. On five white-box source models, NeuronFuzz achieves a jailbreak discovery rate between 76 and 100%, outperforming baselines by up to 48 percentage points. The optimized templates transfer zero-shot to open-weight and six proprietary target models: the average attack success rate (ASR) reaches 69.6%, with a top-5 ensemble ASR of 92.6% on the former; on proprietary models the values drop to 44.1% and 60.0%. The gap between open and closed models is not just a curiosity: it signals that access to weights and internal activations changes the quality of evaluation.

The real structural implication concerns audit asymmetry. NeuronFuzz is a white-box framework: it needs to read internal activations. This makes it directly applicable to open-weight LLMs running locally or on controlled infrastructure, where the security team can instrument the model. For closed models accessible only via API, one can only import templates already optimized elsewhere, with on average lower results. This is not an academic detail: teams managing self-hosted models have access to a class of evaluation tools that proprietary cloud services do not offer with the same depth. Security auditing becomes a factor in deployment decisions, alongside latency, TCO, and data control.

There is also a second-order effect on the cost of continuous security monitoring. Because the alarm score is obtained during prefill, periodic evaluations after model updates, fine-tuning, or template changes can be run without paying the cost of generating thousands of responses. In on-premise environments with limited compute resources, this makes frequent monitoring more practical. At the same time, the fact that gradients guide mutations indicates that the framework does more than look for superficial mistakes: it interrogates the internal representation of the safety concept, a layer that response filters do not cover.

One delicate point remains: the selection of safety neurons depends on known harmful and benign inputs, so coverage depends on the quality of that sample. But the paradigm shift is clear: safety evaluation can exit the expensive generation loop and become a problem of analyzing activations. For those deploying LLMs in sensitive contexts, the question is no longer only whether the model answers badly, but whether the chosen vendor or model allows looking inside deeply enough to find that out in advance.