Robot.txt - yahoo, google, msn
 

How effective is the robot.txt in all aspects of seo tactics and submission tools?

Matt Miladinovich
www.pricepoint.com

It's quite effective in keeping the search engines from spidering and indexing pages and directories that you don't want indexed.

As for other aspects of seo, I couldn't tell ya.

Just an FYI if you are working with Google Sitemap one of the things the bot looks for is your robots.txt file. To allow or disallow

Great Question.

I dont think its really useful for SEO purposes... more so just to tell the SE's which pages to crawl.