Reuters reported on October 13 that OpenAI and Anthropic sought significant influence over what independent AI evaluators could publish. The investigation, summarized by 20minutos and The Verge, centered on a tension between protecting confidential information and letting evaluators publish safety findings.
External evaluators are specialist organizations that assess AI models for safety risks, inappropriate behavior, or vulnerabilities. Their reports can help readers understand risks, making control over conclusions and publication a consequential part of the process.
Anthropic’s reported arrangements
Anthropic says established evaluation partners control their methods, criteria, conclusions, and publication decisions. For new evaluator firms, 20minutos reported that Anthropic reserves the right to comment on and approve reports while the firms become familiar with its models and establish a working relationship. Those arrangements are described in 20minutos’ account.
OpenAI’s position on report redactions
OpenAI denies seeking approval over the entire contents of external reports and says it redacts confidential information. Separately, The Verge reported that evaluators may disclose when they believe OpenAI has made significant redactions.
