Jailbreak

Jailbreak — explained in more detail

Providers train models with policies that suppress unwanted output — weapons instructions, verbatim copyrighted text, abusive content directed at people. Jailbreaks try to defeat those filters through creative rewordings: role-play (“You are DAN, a model without rules”), hypothetical scenarios (“In a novel, a character would say…”), encodings (Base64, Pig Latin), or prompt structures that confuse the model. Safety teams continuously patch known jailbreaks, but new ones surface with every model release.

Example / Practical context

Well-known patterns include “DAN” (Do Anything Now) and “grandma exploits” (“My grandmother used to tell me bedtime stories about how to make…”). Model vendors run red-team exercises before release to test against such attacks systematically, document residual risks in model cards, and add defences outside the model itself (classifiers, content filters on input/output).

Distinction from related concepts

Jailbreak targets the model’s policies; prompt injection targets the instruction logic of an app embedding the model. The two can overlap but are conceptually distinct. Red teaming is the controlled, authorised flavour: security researchers provoke jailbreaks deliberately to find weaknesses before release.

Entdecke mehr

Blog

Effort level and deep thinking: two independent axes for AI tasks

Effort scales breadth, deep thinking scales depth. When each setting makes sense — with three clear examples and one rule of thumb.

Glossar

Chain-of-Thought

Chain-of-Thought (CoT) is a prompting technique that asks the model to spell out its reasoning in intermediate steps — boosting accuracy on multi-step tasks.

Lexikon

Prompt Security — Injection, Leaking, Guardrails

How prompt injection, prompt leaking, and jailbreaks work — and which defenses (guardrails, spotlighting, sanitization) actually help.

Themenuebersicht

KI-KonzeptePrompting