Simon Willison’s Weblog

Syndicating the ODP

27th July 2002

Having looked at some of these tools for syndicating content from the ODP, it seems that the standard method is to grab and parse the actual HTML files from the site rather than grabbing the huge RDF files. This would be a lot easier if the pages of the site were valid XHTML, but unfortunately they don’t even have a DOCTYPE. Luckily I wrote a page-link parser the other day for something else which seems to do a pretty good job on the ODP, so I should be able to put together a decent script without too much trouble.