Spyder Bots

[BungalowBill]

Is it possible to tell if a visitor is really a spyder bot and not just a normal user?

[jocelyn]

Not all bots will ID themselfs in your log files...
Google, MSN, and most major ones will.

[BungalowBill]

How do they ID themselves?

[callmeshoe]

[QUOTE=BungalowBill]Is it possible to tell if a visitor is really a spyder bot and not just a normal user?[/QUOTE]

you can usually pick out the bots pretty easily cause they methodically follow each link. i track the referring ip and when i see this pattern i look for the sig in the logs. so the best way is to look for a distinct pattern. your host usually provides you with the logs and stats.

[callmeshoe]

[QUOTE=BungalowBill]How do they ID themselves?[/QUOTE]

here is an example:

Googlebot/2.1 (+http://www.googlebot.com/bot.html)

you can pick out most of them.