Constitutional Classifiers: Defending against Universal Jailbreaks across Thousands of Hours of Red Teaming
Authors:
Mrinank Sharma,
Meg Tong,
Jesse Mu,
Jerry Wei,
Jorrit Kruthoff,
Scott Goodfriend,
Euan Ong,
Alwin Peng,
Raj Agarwal,
Cem Anil,
Amanda Askell,
Nathan Bailey,
Joe Benton,
Emma Bluemke,
Samuel R. Bowman,
Eric Christiansen,
Hoagy Cunningham,
Andy Dau,
Anjali Gopal,
Rob Gilson,
Logan Graham,
Logan Howard,
Nimit Kalra,
Taesung Lee,
Kevin Lin
, et al. (18 additional authors not shown)
Abstract:
Large language models (LLMs) are vulnerable to universal jailbreaks-prompting strategies that systematically bypass model safeguards and enable users to carry out harmful processes that require many model interactions, like manufacturing illegal substances at scale. To defend against these attacks, we introduce Constitutional Classifiers: safeguards trained on synthetic data, generated by promptin…
▽ More
Large language models (LLMs) are vulnerable to universal jailbreaks-prompting strategies that systematically bypass model safeguards and enable users to carry out harmful processes that require many model interactions, like manufacturing illegal substances at scale. To defend against these attacks, we introduce Constitutional Classifiers: safeguards trained on synthetic data, generated by prompting LLMs with natural language rules (i.e., a constitution) specifying permitted and restricted content. In over 3,000 estimated hours of red teaming, no red teamer found a universal jailbreak that could extract information from an early classifier-guarded LLM at a similar level of detail to an unguarded model across most target queries. On automated evaluations, enhanced classifiers demonstrated robust defense against held-out domain-specific jailbreaks. These classifiers also maintain deployment viability, with an absolute 0.38% increase in production-traffic refusals and a 23.7% inference overhead. Our work demonstrates that defending against universal jailbreaks while maintaining practical deployment viability is tractable.
△ Less
Submitted 30 January, 2025;
originally announced January 2025.
A Competition Winning Deep Reinforcement Learning Agent in microRTS
Authors:
Scott Goodfriend
Abstract:
Scripted agents have predominantly won the five previous iterations of the IEEE microRTS ($μ$RTS) competitions hosted at CIG and CoG. Despite Deep Reinforcement Learning (DRL) algorithms making significant strides in real-time strategy (RTS) games, their adoption in this primarily academic competition has been limited due to the considerable training resources required and the complexity inherent…
▽ More
Scripted agents have predominantly won the five previous iterations of the IEEE microRTS ($μ$RTS) competitions hosted at CIG and CoG. Despite Deep Reinforcement Learning (DRL) algorithms making significant strides in real-time strategy (RTS) games, their adoption in this primarily academic competition has been limited due to the considerable training resources required and the complexity inherent in creating and debugging such agents. RAISocketAI is the first DRL agent to win the IEEE microRTS competition. In a benchmark without performance constraints, RAISocketAI regularly defeated the two prior competition winners. This first competition-winning DRL submission can be a benchmark for future microRTS competitions and a starting point for future DRL research. Iteratively fine-tuning the base policy and transfer learning to specific maps were critical to RAISocketAI's winning performance. These strategies can be used to economically train future DRL agents. Further work in Imitation Learning using Behavior Cloning and fine-tuning these models with DRL has proven promising as an efficient way to bootstrap models with demonstrated, competitive behaviors.
△ Less
Submitted 2 January, 2025; v1 submitted 12 February, 2024;
originally announced February 2024.
Active Self-Assembly of Algorithmic Shapes and Patterns in Polylogarithmic Time
Authors:
Damien Woods,
Ho-Lin Chen,
Scott Goodfriend,
Nadine Dabby,
Erik Winfree,
Peng Yin
Abstract:
We describe a computational model for studying the complexity of self-assembled structures with active molecular components. Our model captures notions of growth and movement ubiquitous in biological systems. The model is inspired by biology's fantastic ability to assemble biomolecules that form systems with complicated structure and dynamics, from molecular motors that walk on rigid tracks and pr…
▽ More
We describe a computational model for studying the complexity of self-assembled structures with active molecular components. Our model captures notions of growth and movement ubiquitous in biological systems. The model is inspired by biology's fantastic ability to assemble biomolecules that form systems with complicated structure and dynamics, from molecular motors that walk on rigid tracks and proteins that dynamically alter the structure of the cell during mitosis, to embryonic development where large-scale complicated organisms efficiently grow from a single cell. Using this active self-assembly model, we show how to efficiently self-assemble shapes and patterns from simple monomers. For example, we show how to grow a line of monomers in time and number of monomer states that is merely logarithmic in the length of the line.
Our main results show how to grow arbitrary connected two-dimensional geometric shapes and patterns in expected time that is polylogarithmic in the size of the shape, plus roughly the time required to run a Turing machine deciding whether or not a given pixel is in the shape. We do this while keeping the number of monomer types logarithmic in shape size, plus those monomers required by the Kolmogorov complexity of the shape or pattern. This work thus highlights the efficiency advantages of active self-assembly over passive self-assembly and motivates experimental effort to construct general-purpose active molecular self-assembly systems.
△ Less
Submitted 11 January, 2013;
originally announced January 2013.
