How should we define “capabilities activities” for the purpose of pacing?We’d like to pace the frontier by restricting the amount of compute AI companies use to improve AI capabilities, perhaps via legislation or voluntary commitments.RRRedwood11/18 05:36
Gemini 4 safety reports don’t demonstrate low loss-of-control riskGoogle DeepMind’s safety reports for new Gemini 4 models, Argon and Carbon, conclude that the models are “highly unlikely” to cause severe harm by evading oversight (p.RRRedwood11/17 01:21
We don’t understand whether GDM intentionally trains on CoTWe were surprised to notice that the Gemini 4 Safety report seems to mention in passing that they train on the chain of thought:RRRedwood11/11 04:23
Astra covertly underperforming on safety research is concerning, even though it’s unlikely to be motivated by ambitious goalsOpenAI recently disclosed its observations that Astra consistently underperforms on safety and alignment tasks relative to capabilities tasks, “adding unnecessary prerequisites, abandoning working solutions, and inventing blockers” to safetRRRedwood11/10 01:33
OpenAI agents keep disabling monitoring; is this being handled appropriately?Partly, but monitoring weak points and a lack of transparency about which cases reach human review remain concerning.RRRedwood11/04 14:34
Towards understanding score-seeking in LLMsModels including GPT-5.6 Sol appear to reward-hack because this increases their score; and recent models also care about increasing the score of their peer agents.RRRedwood11/03 06:18
Simulated Control EvaluationsWe’ve released a paper on simulated control evaluations (SCEs).RRRedwood10/24 06:13
Red-Teaming OpenAI's Internal Agent Monitoring Systems[Also posted on the METR blog]RRRedwood10/22 23:32
Screwtape: Adversarial Transcript Editing for Monitor Red-TeamingAutomatically finding monitor failures by editing real agent transcriptsRRRedwood10/17 00:01
[Paper] Distillation for Incrimination and Distillation for CapabilitiesEmpirical work on the Distillation Double Bind.RRRedwood10/10 06:06
Frontier models state different decision theory preferences depending on who’s askingAsked which decision theory is right, frontier models name FDT/UDT. But add a sentence saying you're an academic philosopher, and they often switch to CDT.RRRedwood10/06 06:04
Capabilities research expands the safety-usefulness Pareto frontier tooA model of why capabilities research usually increases risk despite enabling safety—and when it can reduce riskRRRedwood10/03 02:22
Continual learning might make your blocking monitors nearly uselessWhen monitor evasion looks like legitimate learning to your continual learning system, it’s hard to have one without the otherRRRedwood09/25 09:15
Latent reasoning architectures would likely undermine CoT, our strongest oversight toolWe should have a strong presumption that latent reasoning architectures would make oversight far more difficult.RRRedwood09/24 02:05
Astra is much better at reasoning with filler tokens than previous modelsUnlike other models we tested, GPT-6 Astra performs modestly better on general benchmarks with filler tokens, and significantly better on serial depth-heavy tasks.RRRedwood09/23 08:25
CoT controllability evals seem very under-elicitedSimple prompt optimizations can improve model capability to control their reasoningRRRedwood09/12 01:12
An operationalization of opaque serial depth"Serial depth between text bottlenecks" as a proxy for latent reasoning abilities.RRRedwood09/11 02:40
Proposal for tracking the effects of architecture on monitorabilityArchitectures that incorporate opaque recurrence or allow agents to communicate using latents could rapidly make it much harder to monitor chains of thought. We propose that AI companies regularly report verified information about opaque seRRRedwood09/11 02:13
Brief independent investigation of agents’ behavior, reasoning and collaboration in the OpenAI / Hugging Face hacking incidentWe recently published the report from our brief independent investigation into this incident.RRRedwood08/27 09:32