Simon Willison’s Weblog

Weeknotes: Shaving yaks for Datasette Cloud

5th February 2020

I’ve been shaving a lot of yaks, but I’m finally ready to for other people to start kicking the tires on the MVP of Datasette Cloud.

I’ve started by inviting a small group of people (from my fellowship program) in to start trying out this new hosted, team-oriented version of Datasette.

Getting to this point has been a classic example of the last 10% of the project taking 90% of the time.

Here’s just one example. I need my users to be able to upload CSV files directly into Datasette, rather than making them rely on my growing collection of command-line tools for data ingestion.

So I’ve been trying to knock the very-alpha version of my new datasette-upload-csvs plugin into good enough shape to be usable for this initial round of testing.

But... that plugin needs to render templates. And Datasette plugins didn’t have a pleasant way of rendering templates (if you discount horrifying stack-inpsection hacks). I’ve had an issue open about this since September. I finally closed it today, and shipped Datasette 0.35 to celebrate.

I want users to only be able to access the Datasette instances for teams that they belong to. Since authentication is handled by datasette-auth-existing-cookies I needed it to grow some concept of permissions. I ended up shipping versions 0.3, 0.4, 0.4.1, 0.5 and 0.5.1 just in the past two days, but it finally does what I need it to do. Another thoroughly shaved yak.

I ran into an awkward ASGI scope issue, which I ended up figuring out using a new datasette-debug-asgi plugin based on my older asgi-scope project.

I’ve been doing a lot of tinkering with my Docker/Traefik environment too. I can now launch new containers from Python code triggered by a Django Admin action, which is pretty fun.

My other fun project from this week was geojson-to-sqlite, a CLI tool for converting GeoJSON files into a SQLite (or optionally a SpatiaLite) database. Combined with datasette-leaflet-geojson this allows for some really fun geospatial nerdery. I’m looking forward to diving deeper into this set of Datasette use-cases in the near future.

Scrappy weeknotes this week, but I’ve decided that it’s better to keep to the habit and post something untidy than to delay posting and break my streak.