On Wed, 24 Sep 2008 19:08:12 +0200 Sagr spamcatcher@suffolk-ancestor-genealogy-research.co.uk allegedly wrote:
I have noticed an increasing number of visits from a web bot called Twiceler (ID String: Mozilla/5.0 Twiceler-0.9 http://www.cuil.com/twiceler/robot.html). It's initial visits seem to come from IP:208.36.144.6 but I have also noticed it appearing from IP:38.99.44.101 et al.
I initialy thought it was a Hacker Bot as it seems to occasionaly cloak it's identity and claim to be the Google Bot. However the multiple IP address hopping and ID cloaking kind of became insignificant when my inbox started to become flooded with Hacker Alert emails from some of my PHP scripts who were reporting they were being probed with an assortment of invalid input parameters by this Bot.
In short this Twiceler Bot appears to behave like a Hacker Bot deliberately designed to probe websites and scripts for vunerabbilities. However looking at it's website it gives the impression of complete innocence... What on earth is going on???
Have any other ALUGers found their website being checked for vunerabilities by Twiceler?
Sagr
I've seen this in my logs too. It may be a fairly aggressive robot, but that is all it is. Cuil was set up recently by a bunch of ex google guys. Their intention is to be bigger and better than google. They already claim to have indexed some silly number of pages.
The site you reference says that the robot will obey robots.txt directives so I'd suggest you block it there if you are concerned. If this has no effect, then you can be pretty sure that the the robot is /not/ from cuil and you may want to take other action.
Your logs should show that the robot is simply following links harvested from your site. If it appears to trigger php script vulnerabilities, then you may want to revisit those scripts.
Mick ---------------------------------------------------------------------
The text file for RFC 854 contains exactly 854 lines. Do you think there is any cosmic significance in this?
Douglas E Comer - Internetworking with TCP/IP Volume 1
http://www.ietf.org/rfc/rfc854.txt ---------------------------------------------------------------------