Simon Willison’s Weblog

Hack Day tools for non-developers

28th July 2009

We’re about to run our second internal hack day at the Guardian. The first was an enormous amount of fun and the second one looks set to be even more productive.

There’s only one rule at hack day: build something you can demonstrate at the end of the event (Powerpoint slides don’t count). Importantly though, our hack days are not restricted to just our development team: anyone from the technology department can get involved, and we extend the invitation to other parts of the organisation as well. At the Guardian, this includes journalists.

For our first hack day, I put together a list of “tools for non-developers”—sites, services and software that could be used for hacking without programming knowledge as a pre-requisite. I’m now updating that list with recommendations from elsewhere. Here’s the list so far:

Freebase

Originally a kind of structured version of Wikipedia, Freebase changed its focus last year towards being a “social database about things you know and love”. In other words, it’s the most powerful OCD-enabler in the history of the world. Create your own “Base” on any subject you like, set up your own types and start gathering together topics from the millions already available in Freebase—or add your own. Examples include the Battlestar Galactica base, the Tall Ships base and the fabulous Database base. If you are a developer the tools in the Make Things with Freebase section are top notch.

Dabble DB

Dabble is a weird combination of a spreadsheet, an online database and a set of visualisation tools. Watch the 8 minute demo to get an idea of how powerful this is—you can start off by loading in an existing spreadsheet and take it from there. You’ll need to sign up for the free 30 day trial.

Google Docs

You can always build a hack in Excel, but Google Spreadsheets is surprisingly powerful and means that you can collaborate with others on your hack (including developers, who can use the Google Docs API to get at the data in your spreadsheet). Check out the following tutorials, which describe ways of using Google Spreadsheets to scrape in data from other webpages and output it in interesting formats:

• Data Scraping Wikipedia with Google Spreadsheets
• Calling Amazon Associates/Ecommerce Web Services from a Google Spreadsheet

There’s also a simple way to create a form that submits data in to a Google Spreadsheet.

Yahoo! Pipes

Visual tools for combining, filtering and modifying RSS feeds. Combine with the large number of full-content feeds on guardian.co.uk for all sorts of interesting possibilities. Here’s a tutorial that incorporates Google Docs as well.

Google My Maps

Google provide a really neat interface for adding your own points, lines and areas to a Google Map. Outputs KML, a handy file format for carting geographic data around between different tools.

If you already have a KML or GeoRSS feed URL from somewhere (e.g. the output of a Yahoo! Pipe), you can paste it directly in to the Google Maps search box to see the points rendered on a map.

Google SketchUp

A simple to use 3D drawing package that lets you create 3D models of real-world buildings and then import them in to Google Earth.

OpenStreetMap

Try your hand at some open source cartography on OpenStreetMap, the geographic world’s answer to Wikipedia. If you have the equipment you can contribute GPS traces, otherwise there’s a clever online editor that will let you trace out roads from satellite photos—or you could just make sure your favourite pub is included on the map. The export tools can provide vector or static maps, and if you export as SVG you can further edit your map in Illustrator or Inkscape.

CloudMade Maps

Commercial tools built on top of OpenStreetMap, the most exciting of which allows you to create your own map theme by setting your preferred colours and line widths for various types of map feature.

Many Eyes

IBM Research’s suite of data visualisation tools, with a wiki-style collaboration platform for publishing data and creating visualisations.

Dapper

Dapper provides a powerful tool for screen scraping websites, without needing to write any code. Output formats include RSS, iCalendar and Google Maps.

TiddlyWiki

TiddlyWiki is a complete wiki in a single HTML file, which you can save locally and use as a notebook, collaboration tool and much more. There’s a large ecosystem of plugins and macros which can be used to extend it with new features—see TiddlyVault for an index.

WolframAlpha

The “computational knowledge engine” with the hubristic search-based interface, potentially useful as a source of data and a tool for processing and visualising that data.

Tumblr

Useful as both an input and an output for feeds processed using other tools, and with a smart bookmarklet for collecting bits and pieces from around the web.

The UCSB Toy Chest

An outstanding list of tools that people “without programming skills (but with basic computer and Internet literacy) can use to create interesting projects”, compiled by the English department at UC Santa Barbara.

Your help needed

There must be dozens, if not hundreds of useful tools missing from the above. Tell me in the comments and I’ll add them to the list.