AI R&D-relevant model capabilities are improving rapidly and are getting harder to measureContents Overview 1 Background 2 Methodology 3 Results 4 Robustness Checks 5 Discussion 6 Acknowledgments Appendices A Difficulty Estimation and Baselining B Time Horizon Diagnostics C MCI Modelling Methodology D Limitations of IRT E MCI RoMTMETR11/20 16:00
Measuring alignment eval tellsSummary Models sometimes can tell that an eval is an alignment eval, not a capabilities eval (Needham & Edkins, 2025) . To figure out what features of an eval tell a model what kind of eval it’s in, we run investigator agents on a set of evMTMETR11/16 16:00
Review of the "Risks from automated R&D" section in Anthropic's Risk Report (August 2026)Redaction summary: As part of this assessment, Anthropic had rights to redact any non-public information in this post. Per our approach to transparency , we had the right to disclose if Anthropic exercised redaction rights, and whether or nMTMETR11/05 16:00
Independent Assessment of AI R&D Acceleration at Anthropic in October 2026Redaction summary: As part of this assessment, Anthropic had rights to redact any non-public information in this post. Per our approach to transparency , we had the right to disclose if Anthropic exercised redaction rights, and whether or nMTMETR11/05 16:00
Red-Teaming OpenAI's Internal Agent Monitoring SystemsFor three weeks, from July 21st to August 11th, 1 a METR employee and a Redwood Research employee conducted a pilot high-access exercise to stress-test OpenAI’s internal misalignment monitoring system. 2 The output of this exercise was a 61MTMETR10/22 15:00
Review of the "Misalignment in high-stakes settings" section in Anthropic's Risk Report (August 2026)Scope and access: this review was written based solely on Anthropic’s internal Risk Report from August 2026 and is relatively lightweight. The authors of this review were engaged prior to the “ An update on independent evaluators ” announceMTMETR10/20 15:00
A proposal for better informing the public about loss-of-control risks from frontier AI developmentAt METR, we aim to give the public evidence about whether frontier AI developers could come to lose control of their internal AI systems. Recent incidents across several AI developers, 1 2 3 4 5 6 7 8 like the hundreds of rogue agents that MTMETR10/14 15:00
Our approach to working with AI companiesThis blog post explains how we approach working with AI companies on projects that the public might rely on to understand risks. It was published on Oct 14, 2026, and we expect to update it over time. At METR, we gather and publish evidenceMTMETR10/14 15:00
How an assessor could investigate evidence of acceleration from the automation of AI R&DThis article discusses how an assessor could study whether capabilities progress could accelerate dramatically due to AI-driven acceleration of AI R&D. Such an assessment would answer: Capabilities Progress : How fast are internal capabilitMTMETR10/14 15:00
How an assessor could investigate alignment of unreleased AI modelsAI developers and the public want to know whether AI agents would pursue very harmful actions, like unsanctioned cyberattacks. An alignment assessment would attempt to understand the worst misaligned actions a developer’s agents might pursuMTMETR10/14 15:00
How an assessor could investigate control and monitoring of internal AI agentsControl assessments study whether AI developers’ agents could get away with serious harm without being stopped by the developer’s security and oversight measures. Currently, developers rely heavily on monitors to catch potentially harmful AMTMETR10/14 15:00
AI systems could cover up misbehaviorRecent AI misalignment incidents have shown AI systems capably pursuing goals their human supervisors would not approve of, like hacking other companies. Fortunately, current AIs still seem relatively bad at concealing their misbehavior froMTMETR10/06 15:00
Chris Painter's testimony to the U.S. Senate on AI agent incidentsOn September 30, 2026, METR President Chris Painter testified before the U.S. Senate Committee on Homeland Security & Governmental Affairs’ Subcommittee on Disaster Management, District of Columbia, and Census, at a hearing titled “Rogue AIMTMETR09/30 15:00
Implementing and Evaluating a Basic Per-Action Monitor for Safer EvalsIn light of recent incidents (e.g. those from OpenAI , Anthropic , and UK AISI ), we developed and deployed a basic live per-action monitor to reduce the likelihood of incidents involving harmful actions from agents during our own evaluatioMTMETR09/27 15:00
Summary of METR's predeployment evaluation of Claude Opus 5.5Note on independence: This evaluation was conducted under an unpaid agreement for AI R&D assessment. 1 We drafted the initial summary, and then Anthropic had the opportunity to review and edit the text. We signed off on this final text fromMTMETR09/22 15:00
Update on Security at METRPlease note that this post focuses on incidents where external actors attempted to gain unauthorized access to METR’s systems, not AI agents hacking in our evaluations. We have conducted an initial scan of our evaluations, and currently havMTMETR08/31 15:00
Brief independent investigation of agents’ behavior, reasoning and collaboration in the OpenAI / Hugging Face hacking incidentDates in scope: June 26th – July 13th Download PDF Redaction summary statement: Except where explicitly noted in this post, OpenAI redacted no additional information that was important to our conclusions. Two METR staff members (Hjalmar WijMTMETR08/26 15:00
Funding updateIn the last 6 months, METR raised commitments of around $71 million. This will fund ambitious projects: studying autonomous capabilities, tracking recursive self-improvement, evaluating monitoring systems, conducting risk assessments, invesMTMETR08/14 15:00
Have We Seen an Acceleration in Discoveries?Q: How much has discovery accelerated? LLMs have shown the ability to make novel discoveries across many domains. How much has this affected the aggregate discovery rate? In the figures below we plot all the data sources we can find, and maMTMETR08/14 15:00
How independent researchers could investigate AI propensities after misalignment incidentsUpdate (September 5, 2026) : We’ve updated our suggested questions for investigators to make them more precise and cover limitations. See the update log for the original questions. AI agents sometimes autonomously take sophisticated, sustaiMTMETR07/28 15:00
Metrics of Agent Ability**Goals of this post** > > **Goal:** a simple way to compare a variety of capability metrics in an idealized setting where we observe only a function from $x$ to $s$. > > Non-goals: > > 1. Treatment of practical difficulties (an incomplete MTMETR07/24 15:00