Simon Willison’s Weblog

Subscribe

May 2021

49 posts: 5 entries, 16 links, 2 quotes, 26 beats

May 18, 2021

TIL Django data migration using a PostgreSQL CTE — I figured out how to use a PostgreSQL CTE as part of an update statement in a Django data migration. The trick here is mainly understanding how to combine CTEs with a PostgreSQL update - here's the pattern for that:

Weeknotes: Velma, more Django SQL Dashboard

Matching locations for Vaccinate The States, fun with GeoJSON and more improvements to Django SQL Dashboard.

[... 555 words]

TIL Switching between gcloud accounts — I have two different Google Cloud accounts active at the moment. Here's how to list them with `gcloud auth list`:

May 19, 2021

Flat Data. New project from the GitHub OCTO (the Office of the CTO, love that backronym) somewhat inspired by my work on Git scraping: I’m really excited to see GitHub embracing git for CSV/JSON data in this way. Flat incorporates a reusable Action for scraping and storing data (using Deno), a VS Code extension for setting up those workflows and a very nicely designed Flat Viewer web app for browsing CSV and JSON data hosted on GitHub.

# 1:05 am / github, deno, git-scraping

May 20, 2021

TIL Finding duplicate records by matching name and nearby distance — I wanted to find potentially duplicate records in my data, based on having the exact same name and being geographically located within 500 meters of each other.
Release geojson-to-sqlite 0.3.1 — CLI tool for converting GeoJSON files to SQLite (with SpatiaLite)

May 23, 2021

Release datasette 0.57a0 — An open source multi-tool for exploring and publishing data

May 24, 2021

How to look at the stack with gdb. Useful short tutorial on gdb from first principles.

# 6:23 pm / c, debugger, julia-evans

HackSoft Django styleguide: services and selectors. HackSoft’s Django styleguide uses the terms “services” and “selectors”. Services are functions that live in services.py and perform business logic operations such as creating new entities that might span multiple Django models. Selectors live in selectors.py and perform more complex database read operations, such as returning objects in a way that respects visibility permissions.

# 7:17 pm / django

May 25, 2021

Release django-sql-dashboard 0.15 — Django app for building dashboards using raw SQL queries
TIL Docker Compose for Django development — I had to get Docker Compose working for a Django project, primarily to make it easier for other developers to get a working development environment.

Weeknotes: Spinning back up on Datasette

I’ve been somewhat distracted from Datasette for the past couple of months, thanks to my work on VIAL and the accompanying open source project django-sql-dashboard. This week I scraped back some time to work on Datasette.

[... 401 words]

TIL Running Docker on an M1 Mac — *TLDR: Use `platform: linux/amd64` in `docker-compose.yml` and your containers will probably work!*

May 26, 2021

M1RACLES: M1ssing Register Access Controls Leak EL0 State. You need to read (or at least scan) all the way to the bottom: this security disclosure is a masterpiece. It not only describes a real flaw in the M1 silicon but also deconstructs the whole culture of over-hyped name-branded vulnerability reports. The TLDR is that you don’t really need to worry about this one, and if you’re writing this kind if thing up for a news article you should read all the way to the end first!

# 3:25 pm / journalism, security

May 27, 2021

Release datasette 0.57a1 — An open source multi-tool for exploring and publishing data

May 28, 2021

explain.dalibo.com (via) By far the best tool I’ve seen for turning the output of PostgreSQL EXPLAIN ANALYZE into something I can actually understand—produces a tree visualization which includes clear explanations of what each step (such as a “Index Only Scan Node”) actually means.

# 5:41 pm / postgresql

May 29, 2021

TIL Finding CSV files that start with a BOM using ripgrep — For [sqlite-utils issue 250](https://github.com/simonw/sqlite-utils/issues/250) I needed to locate some test CSV files that start with a UTF-8 BOM.
Release sqlite-utils 3.7 — Python CLI utility and library for manipulating SQLite databases

2021 » May

MTWTFSS
     12
3456789
10111213141516
17181920212223
24252627282930
31