Simon Willison's Weblog: recursive-self-improvement

Research acceleration: The view inside OpenAI

2026-09-06T23:57:40+00:00

Research acceleration: The view inside OpenAI

Apparently today is RSI day at OpenAI, for Recursive Self-Improvement - I think it's their new AGI. Both this piece and the new essay An Alien Mind (by Chief Scientist Jakub Pachocki) talk about it, and this one doesn't even bother to expand the acronym.

Included are details on how OpenAI's own research team are using coding agents. Like pretty much everyone else 2026 has been the year that agentic engineering really took off at OpenAI, best illustrated by this chart:

I'm intrigued at what caused that significant acceleration in AI spend per researcher in late July - my best guess is that's when internal employees gained access to the model later released as GPT-6 Astra.

Tags: ai, openai, generative-ai, chatgpt, llms, coding-agents, november-2025-inflection, recursive-self-improvement

Quoting Jeremy Howard

2026-06-10T15:23:34+00:00

Easy solution to slow down recursive AI self improvement:

The lab with the top-ranked model must agree THEY must not use it for working on frontier AI

But everyone else should have access to it.

By definition, this means the frontier doesn't advance.

It also has the critical benefit of avoiding a dangerous power imbalance.

Anthropic has chosen the opposite of the safe path: they are allowing themselves, the current top lab, to use their top model for frontier AI research. They've said they'll sabotage others who try.

This means the AI frontier advances, & power imbalance increases.

(To be clear, I don't think we should try to slow down recursive AI self improvement - I think we should open it up and democratize it as much as possible. My point is: if you claim we should slow down, and you have the best model, you should ensure your org can't use it.)

— Jeremy Howard, in a Twitter thread

Tags: ai, generative-ai, llms, jeremy-howard, anthropic, ai-ethics, claude-mythos-fable, recursive-self-improvement

If Claude Fable stops helping you, you'll never know

2026-06-10T00:37:25+00:00

If Claude Fable stops helping you, you'll never know

Jonathon Ready highlights one of the more eyebrow-raising details from the 319 page system card for Fable 5 and Mythos 5. Here's a longer excerpt, highlights mine:

In light of the ability of recent models to accelerate their own development, we’ve implemented new interventions that limit Claude’s effectiveness for requests targeting frontier LLM development (for example, on building pretraining pipelines, distributed training infrastructure, or ML accelerator design). Using Claude to develop competing models already violates our Terms of Service, but enforcing this restriction through our safeguards avoids accelerating the actors most willing to violate these terms.

Unlike our interventions for cybersecurity, biology and chemistry, and distillation attempts, these safeguards will not be visible to the user. Fable 5 will not fall back to a different model. Instead, the safeguards will limit effectiveness through methods such as prompt modification, steering vectors, or parameter-efficient fine-tuning (PEFT). These interventions will not affect the vast majority of coding work. We estimate they will impact ~0.03% of traffic, concentrated in fewer than 0.1% of organizations.

I believe this is the first time Anthropic have announced these kinds of silent interventions. The justification still feels pretty science-fiction to me - the linked article talks about "recursive self-improvement". I'm not at all keen on a model that silently corrupts its replies to questions about "ML accelerator design" purely to slow down research that might conflict with Anthropic's own goals!

Update: Anthropic walked back this policy in the face of widespread outrage from the research community.

Via Hacker News

Tags: ai, generative-ai, llms, anthropic, claude, ai-ethics, claude-mythos-fable, recursive-self-improvement