Simon Willison on baked-data

10 items tagged “baked-data”

2024

We used this model [periodically transmitting configuration to different hosts] to distribute translations, feature flags, configuration, search indexes, etc at Airbnb. But instead of SQLite we used Sparkey, a KV file format developed by Spotify. In early years there was a Cron job on every box that pulled that service’s thingies; then once we switched to Kubernetes we used a daemonset & host tagging (taints?) to pull a variety of thingies to each host and then ensure the services that use the thingies only ran on the hosts that had the thingies.

— Jake Teton-Landis

# 25th September 2024, 6:08 pm / sqlite, kubernetes, feature-flags, baked-data

2021

Clickhouse on Cloud Run (via) Alex Reid figured out how to run Clickhouse against read-only baked data on Cloud Run last year, and wrote up some comprehensive notes.

# 29th July 2021, 6:07 am / cloudrun, baked-data, clickhouse

The Baked Data architectural pattern

I’ve been exploring an architectural pattern for publishing websites over the past few years that I call the “Baked Data” pattern. It provides many of the advantages of static site generators while avoiding most of their limitations. I think it deserves to be used more widely.

[... 1,896 words]

8:23 pm / 28th July 2021 / sqlite, staticgenerator, datasette, baked-data

2020

Building a search engine for datasette.io

This week I added a search engine to datasette.io, using the search indexing tool I’ve been building for Dogsheep.

[... 1,391 words]

6:12 pm / 19th December 2020 / projects, search, sqlite, datasette, dogsheep, weeknotes, baked-data

datasette.io, an official project website for Datasette

This week I launched datasette.io—the new official project website for Datasette.

[... 1,971 words]

8:34 am / 13th December 2020 / projects, datasette, weeknotes, sqlite-utils, baked-data

datasette-ripgrep: deploy a regular expression search engine for your source code

This week I built datasette-ripgrep—a web application for running regular expression searches against source code, built on top of the amazing ripgrep command-line tool.

[... 1,362 words]

6:51 am / 28th November 2020 / async, css, projects, python, datasette, weeknotes, ripgrep, baked-data

Bedrock: The SQLitening (via) Back in March 2018 www.mozilla.org switched over to running on Django using SQLite! They’re using the same pattern I’ve been exploring with Datasette: their SQLite database is treated as a read-only cache by their frontend servers, and a new SQLite database is built by a separate process and fetched onto the frontend machines every five minutes by a scheduled task. They have a healthcheck page which shows the latest version of the database and when it was fetched, and even lets you download the 25MB SQLite database directly (I’ve been exploring it using Datasette).

# 7th October 2020, 11:47 pm / django, mozilla, sqlite, datasette, baked-data

2019

niche-museums.com, powered by Datasette

I just released a major upgrade to my www.niche-museums.com website (launched last month).

[... 1,154 words]

10:27 pm / 25th November 2019 / museums, projects, yaml, datasette, weeknotes, baked-data

Weeknotes: Niche Museums, Kepler, Trees and Streaks

Every now and then someone will ask “so when are you going to build Museums Near Me then?”, based on my obsession with niche museums and websites like www.owlsnearme.com.

[... 872 words]

10:42 pm / 28th October 2019 / museums, productivity, projects, visualization, weeknotes, baked-data

2018

The interesting ideas in Datasette

Datasette (previously) is my open source tool for exploring and publishing structured data. There are a lot of ideas embedded in Datasette. I realized that I haven’t put many of them into writing.

[... 2,857 words]

2:28 am / 4th October 2018 / projects, sqlite, datasette, baked-data