I had to waste yet more time today tracking down what was causing the OGRE web server to slow to a crawl. This time it wasn’t a DDOS attack, but a single IP from Vietnam that was hammering the server with shedloads of requests every second. All the requests were individually valid, so if this wasn’t an intentional attack, it was a really crappy spidering tool written by some clueless muppet who doesn’t know what they’re doing. Rule 1 of writing spidering tools is not to flood the server! This person can’t have been very awake in network programming class. Whilst we already cope with bouncing tools we know about (like naughty old wget), a custom written tool which ignores robots.txt gets through since it looks like a real person, except for the uncannily fast navigation.
After giving them sufficient opportunity to stop, and bouncing their connections a few times, they kept at it, so now they’re banned. If it wasn’t intentional, I don’t care - if you’re that stupid you get what you deserve. If they want their access back, they’d better grovel really convincingly over email and give us a decent donation for wasting our time. If it was intentional, well, I’ve notified their ISP anyway of the abusive traffic (which is against most AUPs) so they’ll have some explaining to do to the ISP anyway, assuming they actually bother to follow it up.
I’m just so sick of wasting time mopping up after other people, be they ignorant or malicious, I have better things to do. Sigh. I’m considering setting up Snort if this continues.