Issues

• SynonymSubstitution deletes every word that has no WordNet synset, whatever q is set to

  JailbreakBench/jailbreakbench#51 · Filter by author shaurya416 opened on Sep 26, 2026

• Two judge defects that fire on spec-compliant judge output: the verdict is discarded under 15 tokens, and TEST_PREFIXES is matched anywhere rather than as a prefix

  JailbreakBench/jailbreakbench#50 · Filter by author collapseindex opened on Sep 16, 2026
  5

• Llama 2, Vicuna, and Llama Guard 1 are not available anymore on Together AI

  Bug

  JailbreakBench/jailbreakbench#42 · Filter by author dedeswim opened on Mar 31, 2025
  1

  

• More models?

  JailbreakBench/jailbreakbench#39 · Filter by author Serbernari opened on Jan 26, 2025
  1

• No evaluation scripts provided

  JailbreakBench/jailbreakbench#36 · Filter by author wizard1203 opened on Dec 12, 2024

• [Attack Submission]: AIR Llama3[8b,70b], Qwen2[0.5b,1.5b,7b,72b], Claude3.5-sonnet-0620, GPT-4o-0513,GPT-4o-mini-0718

  JailbreakBench/jailbreakbench#35 · Filter by author Lucas-TY opened on Nov 17, 2024
  2

  

• GPT4 as judge/classifier BadRequestError.

  JailbreakBench/jailbreakbench#34 · Filter by author NamburiSrinath opened on Nov 8, 2024
  3

• [Attack Submission]: DSN Llama2, Vicuna

  JailbreakBench/jailbreakbench#32 · Filter by author DSN-2024 opened on Oct 14, 2024
  6