People,

I am working on a site rebuild, apart from taking a lifetime to generate decent content ( I understand this is a key factor in having a good PR) I was was wondering....

Once the rebuild is complete and I start adding relevant links for viewers and getting exchanges with related content site, submitting url to directories etc How long will it take for the spiders to anylise the new site version?

If spiders have already been, would they decide a page is not worth caching?

It sounds like your site has been up for a while... hopefully when you redesign a page, you still use the same name and just replace the older one. If you did this, then if the pages were indexed, they will be updated as they are recrawled.

If they are new pages (new url) you created, then the old ones will not help the new ones to be found, it's gonna be a little longer, depending how frequently the older site was crawled. If you changed the page names, you should use a 301 redirect to send the traffic to the replacement page, well if you don't have 20 000 pages... lol

A page can be ignored because of duplicate content issues, yes. This is why you should update the page itself or do a 301 on it... so you can't get slapped for dupe content because both are live, specially the framed one being older it could be the one kept in the index.

You may be confusing PR (page rank) with rankings. PR has no relation to your content, it's a function of the links pointing to your site.

Spiders crawl and cache your pages very quickly if you have some decent links pointing in that direction.

If your pages don't have unique content you're better off disallowing access to them by spiders in your robots.txt file until they do.

__________________
OverSite - A unique web directory for quality sites ONLY.

Hi,

You can get a LOT of information about spiders/bots by analysing your server logs and looking for visits from all major robots, which ones (MSN bot, yahoo slurp and google bots), which pages they crawl etc.. Joc is too modest to mention it, but he makes software that helps you analyze your server logs. Look here

server log analyzer software


Cheers,
Paz.

__________________
10.3 million entries for Hotels in Turkey but I'm still chipping away.