Simon Willison’s Weblog

Subscribe
Atom feed for python

1,161 posts tagged “python”

The Python programming language.

2009

easy_install no longer working with SourceForge-hosted projects? Unsurprising, since installation software (which is often run as root) that crawls the web and scrapes HTML pages for download links is a horrible, horrible idea.

# 19th August 2009, 11:38 am / easyinstall, python, sourceforge

Kung Fu People (via) The first site to launch based on the open source Django code from djangopeople.net!

# 19th August 2009, 11:37 am / kungfu, django-people, open-source, django, python, peter-bengtsson

Python logging from multiple processes. Use Python’s socket log handler to send all log messages to a single server—the python-loggingserver project implements such a server as a Twisted application with a handy web interface for viewing the aggregated logs.

# 13th August 2009, 11:55 pm / python, logging, twisted

Scriptlets—Quick web scripts (via) From the prolific Jeff Lindsay, a pastebin-style tool for short server-side scripts written in Python, JavaScript or PHP that executes them within a Google App Engine powered sandbox. The Java code that implements the service is available on GitHub.

# 13th August 2009, 1:51 pm / github, jeff-lindsay, webhooks, scriptlets, python, javascript, php, googleappengine, appengine, open-source, java

minixsv (via) As far as I can tell, this is the only library that can validate XML using pure Python (no C extension required). I’d be extremely happy if someone would write a pure Python library (or one that only depends on ElementTree, which is included in the standard library) for validating XML against a Relax NG Compact syntax schema. Even DTD validation would be better than nothing!

# 12th August 2009, 4:59 pm / relaxng, elementtree, minixsv, python, validation, xml, xmlschema

topia.termextract. Impressive Python term extraction library (similar to the various term extraction web APIs but you can run it on your own hardware), incorporating a Parts-Of-Speech tagging algorithm.

# 10th August 2009, 9:26 pm / python, topia, termextraction, nlp

Richard Jones: Something I’m working on... Python’s with statement appears to provide just enough syntactic sugar to create some really interesting DSL-style APIs—here’s a very promising example for laying out GUI applications.

# 7th August 2009, 3:47 pm / python, richard-jones, with, gui, dsl

Django: Security updates released. A fix for a directory traversal attack in the Django development server (the one with the big “never run this in production” warnings in the documentation). Also reminds that the release of 1.1 means that 0.96, released over two years ago, has reached end of life and will not receive any further bug fixes after the just-released 0.96.4.

# 29th July 2009, 1:45 pm / django, security, python

Django 1.1 release notes (via) Django 1.1 is out! Congratulations everyone who worked on this, it’s a fantastic release. New features include aggregate support in the ORM, proxy models, deferred fields and some really nice admin improvements. Oh, and the testing framework is now up to 10 times thanks to smart use of transactions.

# 29th July 2009, 9:34 am / django, python, releases, open-source, orm, aggregates, django-admin

NASA NEBULA Services (via) NASA’s new NEBULA cloud computing platform appears to be built entirely on open source infrastructure, including Python, Django, Fabric, Eucalyptus, RabbitMQ, Trac and Solr.

# 28th July 2009, 12:10 pm / fabric, eucalyptus, nasa, django, open-source, cloud-computing, nebula, python, rabbitmq, solr, trac

Fabric, Django, Git, Apache, mod_wsgi, virtualenv and pip deployment. I’m slowly working my way through this stack at the moment—next stop, fabric.

# 28th July 2009, 11:56 am / fabric, virtualenv, django, python, git, apache, modwsgi, gareth-rushgrove, pip, deployment, wsgi

xmlwitch. An XML building library for Python that doesn’t suck (I love ElementTree for parsing XML, but I’ve never really liked it for generation). Makes smart use of the with statement.

# 24th July 2009, 12:33 am / withstatement, python, xml, xmlwitch

Django 1.1 release candidate available. If all goes well, the final release will be out next week.

# 22nd July 2009, 12:19 pm / django, releasecandidate, python

Curating conversations. Chris Thorpe has open-sourced the Guardian’s moderated Twitter backchannel app, for displaying back channels at high profile (and hence high potential for abuse) events. It’s a Python application that runs on App Engine.

# 16th July 2009, 7:34 pm / guardian, twitter, chris-thorpe, python, appengine

Tools of the Modern Python Hacker: Virtualenv, Fabric and Pip. Ashamed to say I’m not using any of these yet—for Django projects, my manage.py inserts an “ext” directory at the beginning of the Python path which contains my dependencies for that project.

# 9th July 2009, 11:40 am / python, tools, django, virtualenv, fabric, pip, deployment, pythonpath

Tips on using python’s datetime module. Wow. I’ve run in to problems with datetime and timezones before, but I had no idea how intrinsic those problems were to the design of the library.

# 6th July 2009, 2:20 pm / datetime, python, dates, times

Address Extractor. Running on App Engine, an address extractor web service using code from the EveryBlock open source release.

# 1st July 2009, 8:03 pm / everyblock, python, appengine, addressextractor

EveryBlock source code released. EveryBlock’s Knight Foundation grant required them to release the source code after two years, under the GPL. Lots of neat Django / PostgreSQL / GIS tricks to be found within.

# 1st July 2009, 8:01 pm / gis, postgresql, django, everyblock, python, open-source, gpl

What’s New In Python 3.1. Lots of stuff, but the best bits are an ordered dictionary type (congrats, Armin), a Counter class for counting unique items in an iterable (I do this on an almost daily basis) and a bunch of performance improvements including a rewrite of the Python 3.0 IO system in C.

# 28th June 2009, 3:02 pm / python, python3, python31, performance, releases, armin-ronacher

Towards a Standard for Django Session Messages. I completely agree that Django’s user.message_set (which I helped design) is unfit for purpose, but I don’t think sessions are the right solution for messages sent to users. A signed cookie containing either the full message or a key referencing the message body on the server is a much more generally useful solution as it avoids the need for a round trip to a persistent store entirely.

# 19th June 2009, 9:57 pm / django, sessions, messages, flash, signedcookies, cookies, python

Investigate your MP’s expenses. Launched today, this is the project that has been keeping me ultra-busy for the past week—we’re crowdsourcing the analysis of the 700,000+ scanned MP expenses documents released this morning. It’s the Guardian’s first live Django-powered application, and also the first time we’ve hosted something on EC2.

# 18th June 2009, 11:16 pm / django, ec2, guardian, mpexpenses, projects, python, crowdsourcing

Jython 2.5.0 Final is out! It’s been a long time coming—congratulations to the team.

# 16th June 2009, 11:21 pm / jython, python, java

Installing Django, Solr, Varnish and Supervisord with Buildout. Useful, detailed instructions... but I still think this stuff is Way Too Difficult at the moment. I’m a big fan of the idea of sites that are assembled from multiple smaller web services talking HTTP to each other, but ensuring all the moving parts stay running is massively more painful than just running Apache and MySQL.

# 7th June 2009, 1:54 pm / apache, mysql, sysadmin, operations, rest, django, solr, varnish, supervisord, buildout, python, bertrand-mathieu

Perl 6: The MAIN sub (via) "Calling subs and running a typical Unix program from the command line is visually very similar: you can have positional, optional and named arguments." - that's exactly what I was thinking when I came up with optfunc.

# 28th May 2009, 9:32 pm / perl, optfunc, perl6, python, unix, cli

optfunc. Command line parsing libraries in Python such as optparse frustrate me because I can never remember how to use them without consulting the manual. optfunc is a new experimental interface to optparse which works by introspecting a function definition (including its arguments and their default values) and using that to construct a command line argument parser. Feedback and suggestions welcome!

# 28th May 2009, 7:38 pm / optfunc, github, introspection, optparse, projects, python, cli

geocoders. A fifteen minute project extracted from something else I’m working on—an ultra simple Python API for geocoding a single string against Google, Yahoo! Placemaker, GeoNames and (thanks to Jacob) Yahoo! Geo’s web services.

# 27th May 2009, 10:02 am / geocoders, github, projects, geocoding, placemaker, google, yahoo, geonames, jacob-kaplan-moss, python, web-services

Testing Django Views for Concurrency Issues. Neat decorator for executing a Django view under high concurrency in your unit tests, to help spot errors caused by database race conditions that should be executed inside a transaction.

# 27th May 2009, 10:01 am / django, testing, concurrency, raceconditions, python, threadsafety

uuidd.py. Neat implementation of an ID server from Mike Malone—it serves up incrementing integers over a socket (using Python’s asyncore for fast IO) and records state to a file only after every 10,000 IDs served, so most of the time it’s not reading or writing to disk at all. If the server crashes it doesn’t matter because it can start up again at an integer it’s sure hasn’t been used before.

# 25th May 2009, 9:34 pm / uuid, idserver, python, mike-malone, asyncore, scaling