you can turn it off, that's what I was going on about there... see robotstxt.org1 and popbitch.com/robots.txt2 and clarkeology.com/robots.txt... I want stuff here to be indexed (though I'd rather it wasn't every day), but popbitch doesn't want stuff cached, in case it was a bit dodgy, not deleted quickly enough, and used against them in court...
Paul Clarke's weblog - I live in Hythe in Kent. Wed + father to two, I am a full-stack web engineer, + I do js / Node, some ruby, python, php etc. I like pubbing, running, eating, home-automation and other diy stuff, history, genealogy, TV, squirrels, pirates, lego, and TIME TRAVEL.