worked quicker than I thought, thanks for the feedback. Should have passed in both pages as a refererer, this one once (when looking for robots.txt) and then the page of mine that's related to the topic of yours... which site were you then?
⬅️ RE: Crawler - 514 :: Google API's ➡️
Paulʼs blog - I live in A sleepy town in the deep South. Wed + dad to two, I'm a full-stack web engineer, and I do js / node, some ruby, python, php ect ect. I like pubbing, parkrun, restaurants, home automation and other diy jiggery-pokery, history, tree stuff, TV, squirrels, pirates ☠️, lego, + TIME TRAVEL.