Simon Willison’s Weblog

New anti-comment-spam measure

13th October 2003

I’ve added a new anti-comment-spam measure to this site. The majority of comment spam exists for one reason and one reason only to increase the Google PageRank of the site linked from the spam and specifically to increase its ranking for the term used in the link. This is why so many comment spams include links like this: Cheap Viagra.

Cut off the PageRank boost and you cut off the advantage of spamming, simple as that. I’ve altered my comments system to redirect ALL outgoing links through a simple redirect script, and added that script to my robots.txt file. Links still work fine (even the referral information persists across the redirect) but Google will ignore them completely when calculating PageRank.

Will this reduce the floods of comment spam my site receives? Probably not; I’ve added a note about the restriction to my ’add comment’ form but I doubt many spammers bother to read much about the sites they are targetting. What’s really needed is for this technique to become widespread by being integrated in to existing blogging tools—are you listening Moveable Type hackers?

Update: Sencer has pointed out in the comments that PageRank persists over redirects, and Google appears to ignore robots.txt when used to hide a redirecting page. I’ve updated my redirection script to use javascript to power the redirect (with a link for people with javascript disabled) and an extra meta tag to remind Google not to follow the link. This has the unfortunate side effect that referral information no longer persists across the redirect.