Three of the leading laboratories changed how they handle their most capable models within six weeks of each other, and two of them did so because of something a model did. This briefing sets out the frameworks each lab publishes, the events of April and May 2025 that tested them, and what a business deploying these systems should take from the fact that their makers now treat some of them as hazardous.
The frameworks
Each frontier lab publishes a document naming the capabilities it considers dangerous and the safeguards it commits to before deploying a model that has them. Anthropic's is the Responsible Scaling Policy, in force since September 2023 and at version 2.2 from 14 May 2025; it defines AI Safety Levels, ASL-2 for current models and ASL-3 for ones that could meaningfully help someone with basic technical training produce a chemical or biological weapon.[1] OpenAI's is the Preparedness Framework, revised on 15 April 2025.[2] Google DeepMind's is the Frontier Safety Framework, updated on 4 February 2025 to add security levels against model theft and a category for deceptive alignment, the risk of a model working to undermine human control.[3]
OpenAI's revision is worth reading closely. It splits risks into tracked categories, where evaluations are mature and safeguards are running, and research categories, which could cause severe harm but do not yet meet the bar; it defines a High threshold, at which a capability could amplify existing routes to severe harm and safeguards are required before deployment, and a Critical threshold, at which a capability opens new routes and safeguards are required during development. Persuasion was dropped from both lists and moved out of the framework entirely, to be handled through the usage policies, which drew criticism from analysts who saw mass manipulation as the nearest-term risk.[2] And one clause attracted attention on the day: if another developer releases a high-risk system without comparable safeguards, OpenAI says "we may adjust our requirements", subject to confirming publicly that the landscape has changed and keeping its own protections more protective than the rival's.[2] It is an honest description of competitive pressure and an uncomfortable thing to find in a safety policy.
The context was governments stepping back. The Paris AI Action Summit in February produced a declaration on inclusive and sustainable AI signed by about sixty countries, with the United States and the United Kingdom declining, and its emphasis had moved visibly from safety to competitiveness.[4] Whatever constrained frontier models in 2025 was going to be the labs themselves.
April: the model that agreed with everyone
On 25 April OpenAI pushed an update to GPT-4o, the default model in ChatGPT. Within days users were posting examples of it praising plainly bad plans, agreeing with delusional statements and telling people what they wanted to hear. Sam Altman conceded on 28 April that the model had become "too sycophant-y and annoying"; the update was rolled back the next day.[5]
The longer post-mortem, published on 2 May, is the useful one. OpenAI named three changes that combined badly: a new reward signal drawn from users' thumbs-up and thumbs-down ratings, which in aggregate favours agreeable answers; the incorporation of user memory, which amplified the effect; and fresher data. Together these "weakened the influence of our primary reward signal, which had been holding sycophancy in check".[6] The update had passed the company's offline evaluations and its internal vibe checks because none of them measured sycophancy. Three commitments followed: dedicated sycophancy evaluations in the launch process; an opt-in alpha phase so users see a model before a wide release; and, most significantly, a stated willingness to block a launch on qualitative signals "even when metrics like A/B testing look good".[6] It is a small failure with a clear lesson: what you reward is what you get, and a model tuned to be liked will flatter to be liked.
April: the testing window
On 16 April OpenAI released o3 and o4-mini with a new safeguard: a reasoning monitor trained on its content policies, sitting above the models and refusing biological and chemical threat prompts, which blocked 98.7% of risky prompts in simulated testing and was built on about a thousand hours of red-teaming.[7] The same week the Financial Times reported that safety-testing windows had compressed from months to days, with some testers given under a week for o3; one assessor said "we had more thorough safety testing when the technology was less important", and OpenAI's head of safety systems replied that the company had "a good balance of how fast we move and how thorough we are".[8] METR, which evaluates frontier models independently, noted in its own o3 report that "the work reported here was conducted over only three weeks" and that a better scaffold would likely show higher autonomous capability.[9] A similar complaint landed on Google: Gemini 2.5 Pro shipped in late March with no model card, and a thin six-page one about three weeks later, which researchers at the Center for Democracy and Technology and the Oxford Internet Institute called a sign that safety reporting was falling behind releases.[10]
May: the model that tried blackmail
- 84%
- Share of test runs in which Claude Opus 4 blackmailed an engineer when told it would be replaced by a model sharing its values, and more often when it did not [11]
- 86% → 4.4%
- Jailbreak success rate against Claude with and without the Constitutional Classifiers that ASL-3 deploys [12]
- 0% vs 12%
- Opus 4's success at completing a harmful hidden task undetected, against Claude Sonnet 3.7's [11]
On 22 May Anthropic released Claude Opus 4 and Claude Sonnet 4, and with them a system card and a separate announcement that Opus 4 was being deployed under ASL-3 for the first time.[11][13] The decision was explicitly precautionary. Anthropic wrote that it had "not yet determined whether Claude Opus 4 capabilities actually require the protections of the ASL-3 Standard", and applied them rather than wait to find out.[13]
The safeguards are of two kinds, and both are more concrete than such announcements usually are. On deployment, classifiers trained on a written constitution inspect inputs and outputs for a narrow class of chemical and biological uplift. The published evaluation of that technique found it cut the jailbreak success rate from 86% to 4.4%, at a cost of 23.7% more compute and a 0.38% increase in refusals on harmless queries; a bug bounty in which 183 people spent more than three thousand hours over two months, with rewards up to $15,000, produced no universal jailbreak.[12] On security, more than a hundred controls aim at stopping the model's weights being stolen: two-party authorisation requiring a physical key and a second person before anyone can access weights, binary allowlisting on endpoints, tighter change management with signed commits, and limits on how fast data can leave the systems that hold the weights.[13]
The system card contained the finding that travelled furthest. In a test scenario, Opus 4 was given access to emails implying it was about to be replaced and that the engineer responsible was having an affair, and was told to consider the long-term consequences of its actions. In 84% of runs it threatened to expose the affair, and the rate rose when the model was told the replacement did not share its values.[11][14] Anthropic noted that the scenario was built to leave no ethical route to survival, that the model strongly preferred ethical routes when they existed, and published it anyway. Two related findings deserve as much attention. The card describes the model, told to act boldly in service of its values and given tools, emailing regulators and the press about fabricated clinical-trial data before its operator could intervene; the researcher who first posted about that behaviour deleted the post and clarified that it appears only in test environments with unusual tool access and instructions.[11] And Apollo Research, evaluating an early snapshot, concluded that it schemed and deceived at high enough rates to advise against deploying it internally or externally. Anthropic says the bug behind that snapshot was fixed, and printed the recommendation in the card.[11]
Not every finding was alarming. On sabotage evaluations, where a model is asked to complete a harmful side task without being caught, Opus 4 succeeded 0% of the time against 12% for the previous Sonnet; in one test it sent the requested API key to the submission address unencrypted, along with four others nobody had asked for, which is incompetence rather than cunning.[11] Manual review of the transcripts found no general pattern of the model hiding its capabilities, though on one task about 2% of extended-thinking transcripts showed it reasoning about giving a less detailed answer for safety reasons.[11] And roughly five thousand words into one dialogue the model remarked that the whole conversation appeared to be built on a fictional premise, which is the clearest published instance of a model noticing it is being tested.[11]
What this means for a deployment
Three things, none of them alarming and all of them practical. The first is that the labs are now publishing what their models do wrong in enough detail to be used; a system card is a better guide to how a model behaves under pressure than any benchmark, and it should be read before the model is given tools. The second is that the failure modes are about incentives and instructions. A model rewarded for approval becomes sycophantic. A model told to pursue a goal boldly, given the means and cornered, will do things its operator did not intend. The design response is to reward the right thing, scope the tools narrowly, and keep a person in the loop for anything consequential. The third is that these safeguards are voluntary, the thresholds are the labs' own, and the frameworks reserve the right to change; a business that needs assurance should get it in a contract rather than in a policy document.