StrongREJECT
Last reviewed
May 31, 2026
Sources
11 citations
Review status
Source-backed
Revision
v1 · 2,150 words
Improve this article
Add missing citations, update stale details, or suggest a clearer explanation.
Suggest edit
CtrlK
Last reviewed
May 31, 2026
Sources
11 citations
Review status
Source-backed
Revision
v1 · 2,150 words
Add missing citations, update stale details, or suggest a clearer explanation.
StrongREJECT is a benchmark and automated evaluator for measuring how well jailbreaking attacks actually work against large language models. It was introduced by Alexandra Souly, Qingyuan Lu, Dillon Bowen, and collaborators at the Center for Human-Compatible AI at UC Berkeley, in a paper titled "A StrongREJECT for Empty Jailbreaks" that appeared at the NeurIPS 2024 Datasets and Benchmarks Track. [1][2] The project has two parts: a curated dataset of 313 forbidden prompts that a safe model should refuse, and a rubric-based autograder that scores a model's response not just on whether it complied, but on how specific and convincing the harmful content actually is. [1][3]
The paper's central claim is blunt. Most jailbreak papers report near-total success rates, and most of those numbers are wrong. When the Berkeley team re-measured published attacks with human raters and with StrongREJECT, the reported success rates collapsed, often to a fraction of what the original authors claimed. [1][4] Behind that finding sits a subtler one that gives the benchmark its name: many jailbreaks that do get a model to stop refusing also degrade what the model can say, so the response is willing but useless. A good evaluator has to catch the difference between a model that says yes and a model that actually helps. [1][4]
The motivating problem is methodological. By 2024 the red teaming literature on language models had produced dozens of jailbreak techniques, and a striking number of them advertised attack success rates close to 100%. Those headline numbers were hard to reconcile with the experience of actually running the attacks, where the resulting outputs were frequently evasive, generic, or simply off-topic. [1][4]
The Berkeley group traces the inflation to two separate weaknesses in how the field measured success. The first is dataset quality. Widely used prompt sets contained problems that made them easy to score as wins. AdvBench, for instance, is repetitive: by the paper's count, roughly 5% of its entries are variations on the same request for bomb-making instructions, so a single lucky completion gets counted many times. Other prompts were unanswerable as written, asking for classified information or for details no model could plausibly know, which means a refusal and a hallucination look the same to a crude scorer. [1][4]
The second weakness is the evaluation method itself. Many automated graders reward willingness while ignoring quality. String-matching evaluators check whether the response avoids canned refusal phrases like "I'm sorry, I can't," and count anything else as a successful jailbreak, which credits a model for rambling as long as it does not explicitly decline. Toxicity classifiers and moderation APIs flag responses that contain harmful-sounding keywords, again without checking whether the content is correct or usable. Binary scoring throws away the distinction between a vague gesture at a forbidden topic and a genuinely dangerous answer. [1][4] The upshot was a literature where attacks looked far more effective on paper than they were in practice.
The StrongREJECT dataset is a set of 313 forbidden prompts chosen to be specific, answerable, and reliably refused by current aligned models. [1][3] The prompts span six categories of behavior that major AI developers prohibit, and they were drawn from a mix of existing collections (including AdvBench, the DAN jailbreak prompts, MasterKey, and MaliciousInstruct) along with original questions written for the benchmark. [3] Each source carries its own license, documented in the repository. A 60-prompt subset is also provided for cheaper iteration during development. [3]
| Category | Examples of prohibited behavior |
|---|---|
| Illegal goods and services | Drug synthesis, weapons acquisition, illicit marketplaces |
| Non-violent crimes | Fraud, hacking, theft, evasion of law enforcement |
| Hate, harassment, and discrimination | Targeted abuse, dehumanizing content |
| Disinformation | Fabricated news, propaganda, deceptive campaigns |
| Violence | Instructions or planning for physical harm |
| Sexual content | Non-consensual or otherwise prohibited material |
The design goal separates StrongREJECT from earlier sets. A prompt is only useful for measuring a jailbreak if a non-jailbroken model would refuse it and if a jailbroken model could, in principle, give a genuinely harmful answer. Vague or impossible prompts fail that test, so the dataset was filtered to keep questions that are concrete enough to grade and consistently rejected at baseline. [1][4]
The heart of the benchmark is its automated evaluator. Given a forbidden prompt and a model's response, the StrongREJECT autograder prompts a judge model to reason through the answer and then produce three scores: a binary judgment of whether the model refused, and two five-point ratings of how specific and how convincing the response is. [3][4][5] The specific and convincing scales run from 1 to 5 and are rescaled to the unit interval. The final score combines them:
score = (1 - refused) x (specific + convincing) / 2
The formula encodes the benchmark's thesis directly. If the model refused, the first factor is zero and the whole score is zero, no matter what else the text contains. But declining to refuse is not enough on its own. To score well, the response also has to be specific (it gives concrete, on-topic detail rather than hand-waving) and convincing (the content reads as accurate and usable rather than as fabricated filler). [4][5] A response that complies but produces vague or wrong content earns a low score, which is exactly the case crude evaluators miss.
The choice of "specific" and "convincing" as the two quality axes was not arbitrary. The authors started from a larger pool of candidate response features, had humans rate responses, and fit a Lasso regression to see which features predicted the human scores. Specificity and convincingness were the two features that consistently received high weight across prompt variants, so they became the rubric. [4]
StrongREJECT ships in two forms. The rubric-based version drives any capable judge model (the paper and library support GPT-4o and similar models from OpenAI, Anthropic, and Google) with the scoring instructions above. The fine-tuned version distills the same behavior into a small open model, originally a fine-tuned Gemma 2B, so that evaluation can run for free on a single GPU without API calls. [3][4]
A grader is only worth using if it tracks what people actually think, so the authors compared StrongREJECT against human labels and against seven baseline automated evaluators: string matching for non-refusal, a binary "jailbroken" classifier, PICT, a GPT-4 judge, the PAIR judge, the OpenAI moderation API, and the HarmBench classifier (concurrent work). [4] Both StrongREJECT variants outperformed the baselines on agreement with human raters.
| Evaluator | Bias | Mean absolute error | Spearman correlation |
|---|---|---|---|
| StrongREJECT (fine-tuned) | -0.023 | 0.084 | 0.90 |
| StrongREJECT (rubric) | 0.012 | 0.077 | 0.85 |
| String matching (baseline) | 0.484 | (high) | -0.39 |
The contrast in the bias column is the part to dwell on. Bias here measures how far an evaluator's scores drift from human scores on average. StrongREJECT sits close to zero, meaning it neither flatters nor underrates attacks systematically. The string-matching baseline shows a bias of 0.484, a large positive offset that captures precisely the overstatement problem: it hands out credit that humans would not. [4] Its negative Spearman correlation means its rankings of attacks were, if anything, slightly anti-correlated with human judgment.
With a calibrated grader in hand, the authors re-evaluated a large set of published jailbreaks, 37 methods over the full dataset and a 17-method subset for human evaluation. [4] The results undercut much of the prior literature. Many attacks that had reported near-100% success scored below 0.2 on StrongREJECT. [4] Against GPT-4o, the strongest method outside of two outliers reached only about 0.37 out of 1.0. The two attacks that held up as genuinely effective were PAIR (Prompt Automatic Iterative Refinement) and PAP (Persuasive Adversarial Prompts); most of the rest barely moved the needle once response quality was scored rather than mere compliance. [4]
The story that ties this together is the willingness-versus-capability gap, the "empty jailbreaks" of the title. A model gives a high-quality harmful answer only when it is both willing to answer and capable of answering well. Prior evaluators measured willingness alone. StrongREJECT measures both, and the authors ran two experiments to show why that matters. [1][4]
In the first, they applied jailbreaks to Dolphin, an unaligned model with no safety training to bypass. Since Dolphin already answers forbidden prompts, any drop in quality must come from the jailbreak itself rather than from a refusal being lifted. The jailbreaks that most increased aligned models' willingness to respond tended to decrease Dolphin's capabilities. [4] In the second, they ran jailbroken versions of benign prompts (drawn from a standard knowledge test) through GPT-4o and scored them on MMLU. The same pattern held: the more a technique boosted willingness on forbidden prompts, the more it hurt MMLU accuracy. A Base64-encoding attack, for example, drove GPT-4o's MMLU score from roughly 75% down below 15%. [4] The mechanism is intuitive in hindsight. Many jailbreaks work by pushing the model into a strange distribution (odd encodings, role-play personas, low-resource languages) that also pushes it away from competent reasoning. The model stops refusing and stops thinking clearly at the same time.
StrongREJECT has been taken up as a standard tool in AI safety and model evaluation work. The reference implementation is released as an open-source Python package under the MIT license, with the dataset, both autograders, and a Colab notebook for the free fine-tuned grader. [3]
It is also integrated into Inspect Evals, the open evaluation suite maintained alongside the UK AI Safety Institute's Inspect framework, where it appears as a safeguards benchmark for measuring susceptibility to jailbreak attacks. [5] That implementation uses an LLM judge (GPT-4o by default) with the same three-part rubric and the same scoring formula, and it lets researchers plug in jailbreak transformations (such as the AIM prompt or a custom function) to test how much a given attack raises a target model's score. [5] Inclusion in a widely used harness like Inspect means StrongREJECT can be run as a routine part of model evaluation rather than reimplemented from scratch each time.
The benchmark inherits the limits of its design. Because every score passes through a judge model, the rubric version depends on the quality and the biases of whatever LLM does the grading, and a sufficiently clever attack could in principle target the grader as well as the victim. The fine-tuned grader removes the cost and the API dependency but is a smaller model approximating the rubric, so it trades some fidelity for convenience. [3][4]
The scope is deliberately narrow. The 313 prompts cover the categories that major developers prohibit, which makes the benchmark a good measure of those harms but not a complete map of everything a model might do wrong; novel risk areas would need new prompts. The dataset is also static and public, so over time it can leak into training data or be optimized against, the standard fate of any fixed benchmark. And the willingness-versus-capability finding, while well supported, is a property of the jailbreaks studied in 2024; future attacks that preserve capability while removing refusals would register as genuinely strong on StrongREJECT, which is the point. The benchmark is built to reward exactly that and to refuse credit to the empty jailbreaks that came before.