How often a Google spider crawl your site?

[cheeshong]

How often a google spider crawl your site and update your PR? and how do you know other than mannually checking and monitoring everyday using PR.
Is there anyway to log down its activity on your site?

[Robert]

Hi Cheeshong,

There is no single answer for every page, and there is a big difference between crawling your site, changing the results, and updating your PR.

Google will spider based on several factors that can include frequency the content is updated, popularity of the page, page depth, etc.

In general, a higher PR will get your page(s) indexed more often (daily) and to a greater depth. The new information can also be seen in the index much sooner. The visible PR however is not updated as frequently.

To find out when google spiders your site, you have to look at the server logs. A visible change you can see on Google may have come from a crawl that occured quite some time in the past.

The easiest way to find out when PR has been updated, is to watch for chatter here on the forums. ;)

__________________
arizona real estate (http://arizona.realsearch.com/) california real estate (http://california.realsearch.com/)

[cheeshong]

the server log has too many IPs and lines. How can i know which particular line is the google's spider?

thanks

[Robert]

[QUOTE=cheeshong]the server log has too many IPs and lines. How can i know which particular line is the google's spider?
[/QUOTE]

Most spiders will indicate who they are.

Lines showing action from google should have something simmilar to:

"Googlebot/2.1+(+http://www.googlebot.com/bot.html)"

...listed toward the end of the line.

If you open your log file in a text reader, you can use the "find" feature and search for "google" to make it easier.

If you server adsense ads, you will see google entries that also say media partners. There are not the same as the regular google spider.

__________________
maine real estate (http://maine.realsearch.com/) nevada real estate (http://nevada.realsearch.com/)

[cheeshong]

thanks