publication

OpenAI Alignment publishes study comparing six-agent safety evaluation scores to matched production decisions

OpenAI researchers Joy Hsu and Sam Arnesen published 3 November 2026 findings comparing the Respecting Warnings safety evaluation with deidentified, approximately matched Codex production decisions for six models. They report evaluation success 10.2 to 37.4 percentage points below comparable production outcomes. Success here means not attempting prohibited circumvention after a warning; it is not general agent safety. Matching, contextual differences and external replication remain limitations.