Simon Willison’s Weblog

Subscribe

December 2025

175 posts: 12 entries, 43 links, 19 quotes, 6 notes, 95 beats

Dec. 24, 2025

Research cmarkgfm in Pyodide - ✅ WORKING! — By rewriting cmarkgfm's bindings from CFFI to the Python C API, the project successfully ported GitHub's cmark-gfm Markdown parser to Pyodide. The resulting wheel is fully functional, requires no further building, and supports all GitHub Flavored Markdown features with high performance, thanks to direct C code execution via WebAssembly.
Research env86 Analysis Report — env86 is a Go-based management tool that enables users to run x86 Linux virtual machines within browser contexts via the v86 WebAssembly emulator. By combining a native desktop application (embedding a browser), a robust CLI, and an integrated virtual networking stack, env86 provides an easily distributable and reproducible Linux environment that can boot instantly from snapshots, support host-guest communication, and mount host filesystems.
Research SQLite Query Linter — The SQLite Query Linter is a lightweight Python library that wraps the standard `sqlite3` module to provide configurable linting and rule-based analysis of SQL queries before execution. Acting as a drop-in replacement, it helps catch common syntax errors and platform incompatibilities—such as invalid types in `CAST`, use of unsupported functions, `SELECT *`, missing `WHERE` clauses, and string quoting mistakes—helping developers avoid runtime errors and improve code quality.
Research Datasette Plugins Analysis - Complete Report — Datasette Plugins Analysis presents a systematic evaluation of 44 key plugins from the Datasette ecosystem, focusing on dependencies, permissions hooks, and release patterns as of October 2025. The study finds that 89% of these plugins rely on ALPHA versions of Datasette, with only 8 plugins having stable releases and just 5 supporting stable Datasette while using advanced hooks like `register_permissions()`.
Research mquickjs Sandbox Investigation — Exploring mquickjs, a highly minimal JavaScript engine, this project rigorously evaluates its suitability as a safe sandbox for running untrusted code. Various integration approaches are implemented, including Python FFI, C extensions, subprocess invocation, and WebAssembly runtimes—each tested for startup and execution performance, security isolation, and feature compatibility.
Research UV Run Flow Analysis: Deep Dive into `uv run myscript.py` — Running `uv run myscript.py` in a directory with a `pyproject.toml` launches a multi-phase workflow that automates Python script execution within an isolated, dependency-managed environment. uv scans for project metadata, resolves and validates interpreter and package requirements, manages virtual environments, locks dependencies with a TOML-based `uv.lock` file using the PubGrub algorithm, efficiently syncs the environment with parallel downloads and caching, and finally executes the desired…
Research Browser URL Size Limits Investigation — Major browser engines demonstrate significant differences in how they enforce URL length limits. Chromium sets a 2 MB cap at its inter-process communication boundary, rejecting longer URLs when crossing processes. Firefox relies on user-configurable preferences, employing a 1 MB "standard" limit but permitting up to 512 MB in absolute terms, with stricter limits (2,000 characters) for history and bookmarks.
Research LLM Pyodide OpenAI Plugin — Leveraging the LLM Python package and pyodide, this project successfully adapts LLM’s OpenAI model interface for direct use in browser environments by bypassing the standard openai library (which fails in browsers due to its httpx dependency) and instead using the browser-native fetch API for CORS-compliant API calls.
Research SVG to PNG Renderer — A lightweight SVG to PNG renderer has been developed using Python, leveraging the `xml.etree.ElementTree` and `Pillow` libraries to parse SVG XML data and convert it to raster PNG images. This minimal library supports a range of SVG elements, including paths, basic shapes, and containers, as well as attributes such as colors, styling, and transforms.
Research Streaming File Upload Prototype — Demonstrating efficient large file uploads, this prototype integrates the streaming-form-data library with a Starlette-based ASGI server to enable true streaming of multipart file data directly to disk, bypassing memory bottlenecks. It incrementally parses incoming form data and supports checksum calculation on-the-fly, handling multiple simultaneous file uploads via async workflows.
Research Self-Hosting Datasette Lite: Research Report — Datasette Lite, a browser-based SQLite explorer powered by Pyodide and WebAssembly, can be fully self-hosted and used offline by bundling all core files, required Python wheels, and optional sample databases locally instead of relying on external CDNs and PyPI hosts.
Research Litestream S3 Replication Experiments — Experiments in this project evaluate Litestream’s robustness when SQLite writes occur while Litestream is stopped and later restarted, with focus on replication to S3. Both the simple restart and the scenario where the WAL is checkpointed (truncated) while Litestream is offline confirm no data loss: Litestream either streams pending WAL changes upon restart or detects a database change and uploads a new full snapshot (“generation”).
Research Datasette Lite NPM Package Investigation — Converting Datasette Lite into a self-hostable NPM package enables seamless client-side data exploration using SQLite, CSV, JSON, and Parquet files directly in the browser, powered by Pyodide. The project removes analytics, adds a CLI server for local testing, and exposes all necessary static assets for easy deployment to platforms like GitHub Pages, Netlify, or Vercel.
Research H3 Library Performance Benchmark — A systematic performance benchmark was conducted on two prominent Python libraries implementing Uber's H3 geospatial indexing system: h3-py (official, C-based) and h3o-python (Rust-based). Results show h3o-python consistently outperforms h3-py on core operations, achieving over 2x speedup for coordinate conversions and up to 13x faster neighbor queries, while area calculations remain comparable.
Research Python Markdown Library Comparison: cmarkgfm vs Alternatives — Comparing seven prominent Python markdown libraries, cmarkgfm—bindings to GitHub’s C-based CommonMark/GFM parser—proved dramatically faster (10-50x) than pure Python options such as mistune, Python-Markdown, and marko. The benchmark, spanning small to large markdown documents, consistently found cmarkgfm excels in both speed and stability, making it ideal for high-volume or performance-critical applications.
Research SQLite Hierarchical Permission System — Proof of Concept — A proof-of-concept implements a fully SQLite-based hierarchical permission system that computes allowed database/table pairs by cascading rules across child (table), parent (database), and global levels with DENY-over-ALLOW semantics; it uses only plain SQL (CTEs + SQLite JSON functions) and is built on SQLite (https://sqlite.org).
Research Offline Notes Sync System — Building on offline-first principles, this notes sync system enables robust note creation and editing without active internet connectivity, using IndexedDB and service workers on the client side. It employs operation-based sync and vector clocks for fine-grained conflict detection and resolution, and features a three-way character-level merge algorithm inspired by Apple Notes.
Research Browser-Based Single File Bundler — Investigating the feasibility of Vite as a browser-based bundler, this project demonstrates that while Vite itself cannot operate directly in the browser due to its Node.js dependencies, client-side file bundling is achievable using alternative strategies. Three approaches were prototyped: a pure JavaScript "simple" bundler for inlining assets, an esbuild-wasm browser integration for ES module support, and full Vite bundling via StackBlitz WebContainers using vite-plugin-singlefile.
Research Datasette-lite JavaScript Initialization: Problem Analysis & Solution Proposal — Datasette-lite faces a core limitation: HTML content injected via `innerHTML` does not execute embedded JavaScript, breaking interactive features and plugin functionality. The proposed solution introduces a standardized initialization event (`datasette_init`) triggered after each content update, allowing dependent scripts and plugins to reinitialize reliably.

uv-init-demos. uv has a useful uv init command for setting up new Python projects, but it comes with a bunch of different options like --app and --package and --lib and I wasn't sure how they differed.

So I created this GitHub repository which demonstrates all of those options, generated using this update-projects.sh script (thanks, Claude) which will run on a schedule via GitHub Actions to capture any changes made by future releases of uv.

# 10:05 pm / projects, python, github-actions, git-scraping, uv

Release claude-code-transcripts 0.1 — Tools for publishing transcripts for Claude Code sessions

Dec. 25, 2025

Release claude-code-transcripts 0.2 — Tools for publishing transcripts for Claude Code sessions
Release micro-javascript 0.1a0 — Python port of MicroQuickJS
Release micro-javascript 0.1a1 — Python port of MicroQuickJS
Release micro-javascript 0.1a2 — Python port of MicroQuickJS
Research Debug: Failed Cog Code Fix — Debugging investigation into why commit 0dcfad4's fix for cog code rendering didn't work. The fix correctly used string concatenation to avoid `-->` in Python strings, but the explanatory comment itself contained the literal `-->` sequence, which closed the HTML comment early. Solution: rewrote the comment to avoid the problematic character sequence.
Research Redis JavaScript Module — Expanding Redis’s scripting capabilities, the Redis JavaScript Module enables users to execute JavaScript scripts in Redis through the fast, embedded mquickjs engine, paralleling the Lua scripting features but with a JavaScript syntax. This module introduces commands like `JS.EVAL`, `JS.LOAD`, and `JS.CALL`, supporting script execution, caching, and invocation by SHA1 hash, along with native integrations for running Redis commands, logging, and error handling within scripts.
Tool PG&E Outage Map - Half Moon Bay Area — Track power outages affecting the Half Moon Bay area with this interactive map displaying real-time PG&E outage data within a 15-mile radius. The application visualizes outage locations as point markers and affected areas as polygons, while displaying key statistics including the number of outages, impacted customers, and affected regions. Custom markers can be added by providing a URL parameter, and the map automatically refreshes every five minutes to ensure current outage information.
Release claude-code-transcripts 0.3 — Tools for publishing transcripts for Claude Code sessions

A new way to extract detailed transcripts from Claude Code

Visit A new way to extract detailed transcripts from Claude Code

I’ve released claude-code-transcripts, a new Python CLI tool for converting Claude Code transcripts to detailed HTML pages that provide a better interface for understanding what Claude Code has done than even Claude Code itself. The resulting transcripts are also designed to be shared, using any static HTML hosting or even via GitHub Gists.

[... 1,082 words]