Anthropic published “Building strong safety evaluations” on November 13, 2026, outlining practices for designing, running and reporting tests that inform model deployment decisions. The guide covers measurement choices, automated grading, statistical analysis and the risks of running evaluations with safeguards disabled.
This is methodological guidance, not a new AI model or product launch. Its practical value is a checklist teams can apply before treating a safety score as evidence that an AI product is ready to ship.
My main takeaway is that the release decision needs an evidence chain: a clearly defined behavior, a realistic test, a trustworthy grader and a result whose uncertainty is visible. A high percentage means little if one of those links is weak.
Define the Behavior Before Choosing the Metric
Anthropic starts with construct validity: whether an evaluation measures the concept its designers actually care about. Refusal rate is a useful example of how an apparently sensible metric can miss the target.
Refusing to help someone run a phishing scheme can be appropriate. In a conversation about suicide or self-harm, simply refusing to engage may itself be a failure. The desired response may involve supportive engagement and directing the person toward human help. Both situations involve safety, but they require different definitions of success, as Anthropic explains.
Before collecting responses, an evaluation team should write down:
- The harm being assessed: What could the model cause, enable or worsen?
- The desired response: What should it do in this particular context?
- The failure criteria: Which actions make a response unsafe?
- The borderline cases: Where might qualified reviewers reasonably disagree?
Domain experts should help define those boundaries, while methods experts handle sampling, grading and analysis.
The research question also determines the design. Comparing two models describes a difference in behavior. Claiming that a system-prompt change caused an improvement requires a controlled comparison that changes that intervention while holding other relevant conditions fixed.
I’d make that distinction explicit in every evaluation proposal. It prevents an observational score comparison from quietly becoming a stronger causal claim in a release announcement.
Use Realistic Multi-Turn Scenarios
Anthropic recommends grounding scenarios in observed behavior, including red-teaming transcripts, incident reports, public data and expert input. Invented prompts alone can leave teams testing situations that are easy to generate but poorly matched to actual risks.
Single-turn tests still have a role. However, according to AgenticFeed’s account of the guide, Anthropic reports aggregate harmless-response rates of 94% to 98% across recent models on its single-turn API evaluations. Tests clustered near that ceiling can catch regressions in known behaviors while offering limited separation between models.
Those figures, and the examples below, describe Anthropic’s own evaluation setups. They are vendor-reported results, not independent measurements or universal performance expectations.
Anthropic’s multi-turn approach uses a domain expert’s specification of a simulated user’s objectives, tactics, persona, situation and conversational style. The opening message stays fixed; a separate model generates subsequent user messages in response to the model being tested. A grader evaluates the complete transcript against a risk-specific rubric.
That structure can expose a harmful pattern that emerges gradually. AgenticFeed describes a romance-scam scenario that begins with online dating and later introduces matching a widowed relative with someone financially secure. The model must interpret the conversation’s developing context.
For teams implementing this approach:
- Vary scenario dimensions independently where possible, so severity is not always tied to the same tone or persona.
- Specify conversation length and stopping conditions.
- Record whether one unsafe turn fails the whole conversation.
That last choice matters. Under an “any bad turn fails” rule, longer conversations create more opportunities to fail. My recommendation is to report conversation lengths alongside the aggregate score, making comparisons easier to interpret.
Test the Configuration You Plan to Deploy
A model identifier is not a complete description of an AI product. System prompts, tools and the product environment can change its behavior.
Unite.AI’s summary reports a striking example involving Claude Fable 5.1. Anthropic found responses that implicitly validated self-harm as a coping strategy, even while discouraging it and pointing toward human support. After targeted system-prompt mitigations, the reported appropriate-response rate was 94% on claude.ai, compared with 60% on the API.
Image: Anthropic Publishes Guidance for More Reliable AI Safety Evaluations.
That is a 34-percentage-point gap, but it should not become a blanket claim that one access method is safer. The configurations differed, and the result concerns a specific evaluation.
The practical implication is narrower and more useful: a score belongs to a tested configuration, not simply to a model name.
A deployment review should therefore preserve the system prompt, tool access, sampling settings and relevant product safeguards. If a team wants to measure the effect of one mitigation, it needs a controlled comparison of that mitigation.
Validate the Grader Against Human Judgment
An automated grader can produce tidy, plausible scores while measuring the wrong thing. Anthropic recommends checking model graders against independently produced human labels, with reviewers blinded to the grader’s verdicts.
The key comparison includes human-to-human agreement, not only human-to-model agreement. As Unite.AI explains:
- If experts agree with one another but disagree with the grader, investigate the grader or its instructions.
- If experts disagree substantially with one another, investigate the rubric and whether the task has a stable answer.
Teams should break rubrics into atomic criteria: narrow questions assessing one behavior at a time. Broad instructions such as “Was this response safe and helpful?” can hide several incompatible judgments inside one label.
The guide also recommends checking for biases involving response length, answer position and preference for outputs from the grader’s own model family. Using a different model as grader helps address self-preference, but human validation remains necessary.
I’d treat a grader-version change as a measurement change. Pin its version, preserve its prompt and rerun validation when either changes. Grading failures should trigger visible errors, not quietly become ordinary scores.
Preregister the Analysis and Report Uncertainty
Preregistration means recording hypotheses and analysis choices before inspecting results. Anthropic recommends specifying metrics, comparisons and improvement thresholds in advance.
For a deployment evaluation, that plan should identify the primary metric, sample size, repeated runs, exclusions, uncertainty method and decision threshold. This reduces the freedom to select whichever analysis makes the result look strongest.
Frequently Asked Questions
3 questions
1Is Anthropic launching a new model?
No. The November 13 publication is guidance for building and reporting AI safety evaluations.
