Simon Willison’s Weblog

Subscribe

December 2020

52 posts: 5 entries, 12 links, 7 quotes, 28 beats

Dec. 1, 2020

New for AWS Lambda – Container Image Support. “You can now package and deploy Lambda functions as container images of up to 10 GB in size”—can’t wait to try this out with Datasette.

# 5:34 pm / aws, lambda, docker, datasette

Scaling Datastores at Slack with Vitess (via) Slack spent three years migrating 99% of their MySQL query load to run against Vitess, the open source MySQL sharding system originally built by YouTube. “Today, we serve 2.3 million QPS at peak. 2M of those queries are reads and 300K are writes. Our median query latency is 2 ms, and our p99 query latency is 11 ms.”

# 9:30 pm / mysql, scaling, sharding, youtube, slack, vitess

Dec. 3, 2020

Release datasette-yaml 0.1.1 — Export Datasette records as YAML
Release datasette 0.52.2 — An open source multi-tool for exploring and publishing data
Release datasette 0.52.3 — An open source multi-tool for exploring and publishing data

The value of a product is the number of problems it can solve divided by the amount of complexity the user needs to keep in their head to use it. Consider an iPhone vs a standard TV remote: an iPhone touchscreen can be used for countless different functions, but there's very little to remember about how it works (tap, drag, swipe, pinch). With a TV remote you have to remember what every button does; the more things you can use the remote for, the more buttons it has. We want to create iPhones, not TV remotes.

Adam Wiggins: Heroku Values

# 9:25 pm / heroku, product-management

Dec. 4, 2020

Command Line Interface Guidelines (via) Aanand Prasad, Ben Firshman, Carl Tashian and Eva Parish provide the missing manual for designing CLI tools in 2020. Deeply researched and clearly presented—I picked up a bunch of useful tips and ideas from reading this, and I’m looking forward to applying them to my own CLI projects.

# 8:44 pm / cli, design, usability

Discoverable CLIs have comprehensive help texts, provide lots of examples, suggest what command to run next, suggest what to do when there is an error. There are lots of ideas that can be stolen from GUIs to make CLIs easier to learn and use, even for power users.

Command Line Interface Guidelines

# 8:48 pm / cli, usability

Release datasette-search-all 1.0.1 — Datasette plugin for searching all searchable tables at once

Dec. 5, 2020

The secrets of Monkey Island’s source code (via) To celebrate the thirty year anniversary of the Secret of Monkey Island the Video Game History Foundation interviewed developer Rod Gilbert and produced this comprehensive collection of cut content and material showing how the game was originally constructed.

# 4:32 pm / game-design

Release datasette 0.52.4 — An open source multi-tool for exploring and publishing data

Dec. 6, 2020

Weeknotes: github-to-sqlite workflows, datasette-ripgrep enhancements, Datasette 0.52

Visit Weeknotes: github-to-sqlite workflows, datasette-ripgrep enhancements, Datasette 0.52

This week: Improvements to datasette-ripgrep, github-to-sqlite and datasette-graphql, plus Datasette 0.52 and a flurry of dot-releases.

[... 1,123 words]

Dec. 7, 2020

TIL Running Python code in a subprocess with a time limit — I figured out how to run a subprocess with a time limit for [datasette-ripgrep](https://github.com/simonw/datasette-ripgrep), using the `asyncio.create_subprocess_exec()` method. The pattern looks like this:
Release datasette-seaborn 0.2a0 — Statistical visualizations for Datasette using Seaborn
TIL Using custom Sphinx templates on Read the Docs — I wanted to make a small customization to one of my documentation templates on Read the Docs.
TIL Controlling the style of dumped YAML using PyYAML — I had a list of Python dictionaries I wanted to output as YAML, but I wanted to control the style of the output.
Release yaml-to-sqlite 0.3.1 — Utility for converting YAML files to SQLite

Dec. 8, 2020

Cameras and Lenses (via) Fabulous explotable interactive essay by Bartosz Ciechanowski explaining how cameras and lenses work.

# 3:38 am / photography, explorables

TIL Escaping a SQL query to use with curl and Datasette — I used this pattern to pass a SQL query to Datasette's CSV export via curl and output the results, stripping off the first row (the header row) using `tail -n +2`.

Dec. 9, 2020

The case against client certificates (via) Colm MacCárthaigh provides a passionately argued Twitter thread about client certificates and why they should be avoided. I tried using them as an extra layer of protection fir my personal Dogsheep server and ended up abandoning them—certificate management across my devices was too fiddly.

# 2:41 pm / certificates, dogsheep

Release datasette 0.52.5 — An open source multi-tool for exploring and publishing data
Release datasette-pretty-json 0.2.1 — Datasette plugin that pretty-prints any column values that are valid JSON objects or arrays
Release datasette-column-inspect 0.2a — Experimental plugin that adds a column inspector

Dec. 10, 2020

Deno 1.6 Release Notes. Two signature features in Deno 1.6 worth paying attention to: a built-in language server for code editors like VS Code, and the “deno compile” command which can build Deno JavaScript/TypeScript projects into standalone binaries. The ability to build binaries has turned out to be a killer feature of both Go and Rust, so seeing it ship as a default capability of a interpreted dynamic language is fascinating. I would love it if Python followed Deno’s example.

# 1:25 am / javascript, deno

Dec. 11, 2020

Release datasette 0.53 — An open source multi-tool for exploring and publishing data

datasette.io (via) Datasette finally has an official project website, three years after the first release of the software. I built it using Datasette, with custom templates to define the various pages. The site includes news, latest releases, example sites and a new searchable plugin directory.

# 4:11 am / projects, datasette

Release datasette-auth-passwords 0.3.3 — Datasette plugin for authentication using passwords

If you are pre-product market fit it's probably too early to think about event based analytics. If you have a small number of users and are able to talk with all of them, you will get much more meaningful data getting to know them than if you were to set up product analytics. You probably don't have enough users to get meaningful data from product analytics anyways.

Michael Malis

# 6:39 am / analytics, startups

Dec. 12, 2020

TIL Skipping CSV rows with odd numbers of quotes using ripgrep — I'm working with several huge CSV files - over 5 million rows total - and I ran into a problem: it turned out there were a few lines in those files that imported incorrectly because they were not correctly escaped.
Release datasette-publish-fly 1.0.1 — Datasette plugin for publishing data using Fly

2020 » December

MTWTFSS
 123456
78910111213
14151617181920
21222324252627
28293031