you can turn it off, that's what I was going on about there... see robotstxt.org 1 and popbitch.com/robots.txt 2 and clarkeology.com/robots.txt... I want stuff here to be indexed (though I'd rather it wasn't every day), but popbitch doesn't want stuff cached, in case it was a bit dodgy, not deleted quickly enough, and used against them in court...
⬅️ search engines :: ah i understand ➡️
Paulʼs blog - I live in A sleepy town in Kent. Wed + father to 2, I am a full-stack web engineer, and I do js / node, some ruby, python, php ect ect. My hobbies are pubbing, running, restaurants, home-automation 🤖 and other diy stuff, history, tree stuff, TV, squirrels, pirates ☠️, lego, and TIME TRAVEL.