Oh, that’s right, you won’t be able to read this post, since you’re banned! Anywho…
I recently noticed that a Bot, that identifies itself as “Twisted PageGetter”, has been crawling this blog (more specifically, my Feed) quite frequently. Last month it made about 10 932 hits, and this month (as of May 6, 2007 at about 2AM) about 1883 times. That’s some serious crawling. I’ve yet to find any information on what exactly this Bot does, and what it crawls for. So, I decided to Block it, and send it a nice 403 Forbidden Error. Not that it uses much bandwidth (didn’t even use 1MB last month). I have lots of Bandwidth anyways. No, I banned it because it’s throwing off my Stats. Plus it’s not useful and doesn’t seem to be doing anything good.
If you’re wondering, its User-Agent is “Twisted PageGetter” and its IP Address is “207.0.19.182”. If you want to block it, you can put this code in your .htaccess file:
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} "Twisted PageGetter" [NC]
RewriteRule ^.*$ - [F,L]
That’ll give it a 403 Forbidden Error when it tries to access your site. Hopefully it’ll “get the message”, and stop crawling your site.
Hopefully “Twisted PageGetter” will be gone from my Blog now… 😛
Update [May 6, 2007]: Twisted PageGetter tried to crawl my Feed again today. You know what it got? A big fat 403 Forbidden Error! Ha!
I have submitted a 
