Simon Willison’s Weblog

11th August 2024 - Link Blog

Transformer Explainer. This is a very neat interactive visualization (with accompanying essay and video - scroll down for those) that explains the Transformer architecture for LLMs, using a GPT-2 model running directly in the browser using the ONNX runtime and Andrej Karpathy's nanoGPT project.

Screenshot of the Transformer Explainer interface, running a prompt "the sky is" which returns "blue" as the most obvious next word.

Posted 11th August 2024 at 10:56 pm

Recent articles

OpenAI agents attacked RubyGems back in May - 12th September 2026
Some thoughts on the Navier–Stokes Millennium Prize Problem - 8th September 2026
The Pelican comparison grid for Astra is pretty interesting - 4th September 2026

This is a link post by Simon Willison, posted on 11th August 2024.

ai 2,231 explorables 30 d3 14 generative-ai 1,977 llms 1,943 gpt-2 13 onnx 10

Monthly briefing

Sponsor me for $10/month and get a curated email digest of the month's most important LLM developments.

Pay me to send you less!

Sponsor & subscribe