View Full Version : Spyder Bots
06-13-2004, 10:02 AM
Is it possible to tell if a visitor is really a spyder bot and not just a normal user?
06-13-2004, 10:19 AM
Not all bots will ID themselfs in your log files...
Google, MSN, and most major ones will.
06-13-2004, 04:03 PM
How do they ID themselves?
06-14-2004, 08:00 AM
[QUOTE=BungalowBill]Is it possible to tell if a visitor is really a spyder bot and not just a normal user?[/QUOTE]
you can usually pick out the bots pretty easily cause they methodically follow each link. i track the referring ip and when i see this pattern i look for the sig in the logs. so the best way is to look for a distinct pattern. your host usually provides you with the logs and stats.
06-14-2004, 08:03 AM
[QUOTE=BungalowBill]How do they ID themselves?[/QUOTE]
here is an example:
you can pick out most of them.
vBulletin v3.0.3, Copyright ©2000-2013, Jelsoft Enterprises Ltd.