Simon Willison’s Weblog

Weeknotes: datasette-auth-existing-cookies and datasette-sentry

29th January 2020

Work on Datasette Cloud continues—I’m tantalizingly close to having a MVP I can start to invite people to try out.

I’m trying to get as much work as possible done for it using Datasette Plugins. This week I’ve released two new plugins to assist in the effort.

datasette-auth-existing-cookies

My first attempt at adding authentication to Datasette was datasette-auth-github, which takes advantages of GitHub’s best-in-class OAuth flow to implement sign-in with GitHub and builds a relatively powerful permission system on top of GitHub users, organizations and teams.

For Datasette Cloud I need to go a step further: I’m definitely going to have regular username/password accounts, and I’ll probably implement sign-in-with-Google as well.

I don’t particularly want to implement username/password accounts from scratch. Django (and django-registration) provide robust and very well tested solution for this. How about I use that?

Datasette Cloud teams will each get their own Datasette instance running on a subdomain. If I implement authentication as a Django app running on example.com I can set that as the cookie domain—then Datasette instances running on teamname.example.com will be able to see the resulting authentication cookie.

Given a Django authentication cookie (which may just be a sessionid) how can I tell if it corresponds to a logged in user? That’s where my new datasette-auth-existing-cookies plugin comes in.

The plugin lets you configure Datasette to read in a specified list of cookies and then forward them on as part of an API request to an underlying application. That application then returns JSON showing if the user is signed in or not. The plugin then sets a short-lived signed cookie that persists that information.

Here’s what the configuration looks like:

{
    "plugins": {
        "datasette-auth-existing-cookies": {
            "api_url": "https://www.example.com/user-from-cookies",
            "auth_redirect_url": "https://www.example.com/login",
            "original_cookies": ["sessionid"]
        }
    }
}

Any hits to teamname.example.com will be checked for a sessionid cookie. That cookie is forwarded on to https://www.example.com/user-from-cookies to see if it’s valid.

If the cookie is missing or invalid, the user will be redirected to the following URL:

https://www.example.com/login?next=https://teamname.example.com/

The plugin has a few other options: you can request that the ?next= parameter is itself signed to help protect against unvalidated redirects for example. But it’s a pretty simple piece of code that hopefully means I won’t have to spend much more time thinking about login and registration.

httpx for testing ASGI apps

All of my Datasette plugins ship with unit tests—mainly so that I can implement continuous deployment from them, where new tagged releases are automatically shipped to PyPI provided the tests pass.

For ASGI plugins, this means writing unit tests against the ASGI spec. I’ve mainly been doing this using the ApplicationCommunicator class from asgiref, which provides powerful low-level hooks for interacting with an ASGI application. The tests end up being pretty verbose though!

Here’s the ApplicationCommunicator test I first wrote for datasette-auth-existing-cookies.

I’ve been exploring Tom Christie’s httpx library for asynchronous HTTP calls in Python recently, and I spotted an interesting capability buried deep in the documentation: you can pass it an ASGI app and make requests directly against the app, without round-tripping through HTTP!

This looked ideal for unit testing, so I had a go at rewriting my tests using it. The result was delightful:

auth_app = ExistingCookiesAuthTest(
    hello_world_app,
    ...
)
async with httpx.AsyncClient(app=auth_app) as client:
    response = await client.get(
        "https://demo.example.com/", allow_redirects=False
    )
    assert 302 == response.status_code
    location = response.headers["location"]
    assert "https://www.example.com/login" == location

This is a much nicer way of writing tests for ASGI applications and middleware. I’m going to be using this for all of my projects going forward.

datasette-sentry

In starting to deploy Datasette Cloud I quickly ran into the need to start collecting and analyzing errors thrown in production.

I’ve been enjoing using Sentry for this for several years now, and I was pleased to see that the official Sentry SDK grew support for ASGI last July.

Wrapping it up as a Datasette plugin took less than half an hour: datasette-sentry. It’s configured like this:

{
    "plugins": {
        "datasette-sentry": {
            "dsn": {
                "$env": "SENTRY_DSN"
            }
        }
    }
}

The DSN configuring Sentry will then be read from the SENTRY_DSN environment variable.