Panscient’s Naughty Bot
So I was log rummaging the other day, and came across this. This isn’t the first time the Panscient crawler bot has been caught ignoring robots.txt. There’s absolutely no reason for their bot to be hammering sites like this. None whatsoever. For now I’m going to let them be, but if they return I’ll add [...]