Simon Willison’s Weblog

Subscribe

August 2021

72 posts: 6 entries, 20 links, 8 quotes, 38 beats

Aug. 2, 2021

TIL Search and replace with regular expressions in VS Code — I wanted to replace all instances of this:
Release sqlite-utils 3.14 — Python CLI utility and library for manipulating SQLite databases
Release sqlite-transform 1.2.1 — Tool for running transformations on columns in a SQLite database

OkCupid had a CSRF vulnerability (via) Good write-up of a (now fixed) CSRF vulnerability on OkCupid. Their site worked by POSTing JSON objects to an API. JSON POSTs are usually protected against CSRF because they can only be sent using fetch() or XMLHttpRequest, which are protected by the same-origin policy. Yan Zhu notes that you can use the enctype="text/plain" attribute on a form (introduced in HTML5) and a crafty hidden input element with name='{"foo":"' value='bar"}' to construct JSON in an off-site form, which enabled CSRF attacks.

# 10:12 pm / csrf, security

Aug. 3, 2021

TIL Check spelling using codespell — Today I discovered [codespell](https://github.com/codespell-project/codespell/) via [this Rich commit](https://github.com/willmcgugan/rich/commit/9c12a4537499797c43725fff5276ef0da62423ef#diff-ce84a1b2c9eb4ab3ea22f610cad7111cb9a2f66365c3b24679901376a2a73ab2). `codespell` is a really simple spell checker that can be run locally or incorporated into a CI flow.

Exploring the SameSite cookie attribute for preventing CSRF

Visit Exploring the SameSite cookie attribute for preventing CSRF

In reading Yan Zhu’s excellent write-up of the JSON CSRF vulnerability she found in OkCupid one thing puzzled me: I was under the impression that browsers these days default to treating cookies as SameSite=Lax, so I would expect attacks like the one Yan described not to work in modern browsers.

[... 2,198 words]

TIL Set a GIF to loop using ImageMagick — I managed to accidentally create a GIF that ran once without looping. I think this is because I created it in [LICEcap](https://www.cockos.com/licecap/) but then deleted some frames and re-saved it using macOS Preview.

Aug. 4, 2021

TIL SQLite aggregate filter clauses — SQLite supports aggregate filter clauses (as of [3.30.0, released 2019-10-04](https://www.sqlite.org/changes.html#version_3_30_0)), as described in this [SQL Pivot in all databases](https://modern-sql.com/use-case/pivot) tutorial.

Aug. 6, 2021

Apply conversion functions to data in SQLite columns with the sqlite-utils CLI tool

Visit Apply conversion functions to data in SQLite columns with the sqlite-utils CLI tool

Earlier this week I released sqlite-utils 3.14 with a powerful new command-line tool: sqlite-utils convert, which applies a conversion function to data stored in a SQLite column.

[... 1,941 words]

Breaking Changes to the Web Platform (via) “Over the years there have been necessary changes to the web platform that caused legacy websites to break.”—this list is thankfully very short, only 11 items so far. Let’s hope it stays that way!

# 6:32 am / web

The thing about semver major version numbers are that they don't mean new stuff, they're a permanent reminder of how many times you got the API wrong. Semver doesn't mean MAJOR.MINOR.PATCH, it means FAILS.FEATURES.BUGS

Will McGugan

# 4:17 pm / versioning, will-mcgugan, semantic-versioning

Running GitHub on Rails 6.0. Back in 2019 Eileen M. Uchitelle explained how GitHub upgraded everything in production to Rails 6.0 within 1.5 weeks of the stable release. There’s a trick in here I really like: they have an automated weekly job which fetches the latest Rails main branch and runs the full GitHub test suite against it, giving them super-early warnings about anything that might break and letting them provide feedback to upstream about unintended regressions.

# 4:30 pm / continuous-integration, github, rails

Aug. 7, 2021

Release datasette 0.59a0 — An open source multi-tool for exploring and publishing data

Aug. 8, 2021

Stanford School Enrollment Project (via) This is Project Pelican: I’ve been working with the Big Local News team at Stanford helping bundle up and release the data they’ve been collecting on school enrollment statistics around the USA. This Datasette instance has data from 33 states for every year since 2015—3.3m rows total. Be sure to check out the accompanying documentation!

# 12:23 am / data-journalism, journalism, datasette

Release datasette-pyinstrument 0.1 — Use pyinstrument to analyze Datasette page performance

Aug. 9, 2021

Release datasette 0.59a1 — An open source multi-tool for exploring and publishing data
Release datasette-query-links 0.1 — Turn SELECT queries returned by a query into links to execute them
Release datasette-query-links 0.1.1 — Turn SELECT queries returned by a query into links to execute them
Release datasette-query-links 0.1.2 — Turn SELECT queries returned by a query into links to execute them

The World of CSS Transforms. Comprehensive, clearly explained tutorial on CSS transforms by Josh W. Comeau, with some very neat interactive demos. I hadn’t understood how useful it is that the translate() transform treats percentages as applying to the dimensions of the element being transformed, not its parent. This means you can use expressions like transform: translateX(calc(100% + 4px)); to shift an element by its entire width plus a few more pixels.

# 2:30 pm / css, josh-comeau

TIL Tailing Google Cloud Run request logs and importing them into SQLite — The `gcloud` CLI tool has [the alpha ability to tail log files](https://cloud.google.com/logging/docs/reference/tools/gcloud-logging#live-tailing) - but it's a bit of a pain to setup.
TIL Find local variables in the traceback for an exception — For [sqlite-utils issue #309](https://github.com/simonw/sqlite-utils/issues/309) I had an error that looked like this:
Release sqlite-utils 3.15 — Python CLI utility and library for manipulating SQLite databases

Aug. 10, 2021

Everything new in Datasette since January, plus Django SQL Dashboard. I sent out the first Datasette newsletter since late January this year, covering everything that’s new in Datasette and sqlite-utils this year and introducing Django SQL Dashboard.

# 1:28 am / datasette, sqlite-utils, django-sql-dashboard

Bare columns in an aggregate queries. This is a really nice SQL tweak implemented in SQLite: If you run a query like “SELECT a, b, max(c) FROM tab1 GROUP BY a” SQLite will find the row with the highest value for c and use the columns of that row as the returned values for the other columns mentioned in the query.

# 1:29 am / sql, sqlite

TIL Adding Sphinx autodoc to a project, and configuring Read The Docs to build it — I built a [new API reference page](https://sqlite-utils.datasette.io/en/latest/reference.html) today for `sqlite-utils`, using the Sphinx [autodoc extension](https://www.sphinx-doc.org/en/master/usage/extensions/autodoc.html) to extract docstrings from the code and use them to build a full class reference.
Release sqlite-utils 3.15.1 — Python CLI utility and library for manipulating SQLite databases

Aug. 11, 2021

sqlite-utils API reference (via) I released sqlite-utils 3.15.1 today with just one change, but it’s a big one: I’ve added docstrings and type annotations to nearly every method in the library, and I’ve started using sphinx-autodoc to generate an API reference page in the documentation directly from those docstrings. I’ve deliberately avoided building this kind of documentation in the past because I so often see projects where the class reference is the ONLY documentation, which I find makes it really hard to figure out how to actually use it. sqlite-utils already has extensive narrative prose documentation so in this case I think it’s a useful enhancement—especially since the docstrings and type hints can help improve the usability of the library in IDEs and Jupyter notebooks.

# 1:03 am / documentation, python, sqlite-utils, sphinx-docs

Adding Sphinx autodoc to a project, and configuring Read The Docs to build it. My TIL notes from figuring out how to use sphinx-autodoc for the sqlite-utils reference documentation today.

# 1:21 am / documentation, sqlite-utils, sphinx-docs, read-the-docs

2021 » August

MTWTFSS
      1
2345678
9101112131415
16171819202122
23242526272829
3031