Where is Slurp?

[eitemiller]

We went live with our site 3-1-2004. It has yet to be visited by Slurp, and that is according to our stats provided by Urchin. Out of the 25k pages our website contains, about 700 are indexed by Yahoo, but I am sure that happened when I installed the Y Toolbar on IE, and started surfing our site (I saw results on that day).

My problem is, where is Slurp? I hope I am not missing anything important. I do not have the same problem with Googlebot, which visits daily.

[interactive]

Right on, another Urchin user. You know I think Slurp doesn't show up when your looking under 'Robots'. I see an 'unknown' bot that always has generally the same number of visits as MSNbot. My suspicion is this 'unknown' bot IS sluprp.

Can you check you stats again and see if you have an unknown bot that's visits about the same frequency as MSN?

[jocelyn]

Yeah... maybe they renamed it like they did for googlebot a little while ago.

[wlh]

i think so too...I havent seen a 'slurp' i awhile now that I think about it... I shall look around other forums and see what people have found

[interactive]

The only Yahoo! bot I get is Yahoo MMCrawler - Yahoo-MMCrawler is their multimedia (image) crawler. We have a lot of audio ( voice over MP3's ) so I figure that's why it's visiting. No sign of Slurp, even after a paid directory submission. Answers anyone???

[eitemiller]

You peoples are always so helpful - thanks.

So I am looking at Urchin from March 1st of this year up to today. We took our "under construction" splash down on March 1st. We have had 127 various bots stop by and visit. MSNbot tops the charts with 124k hits. The closest thing I see to that number, and be something I don't reconize, is "Microsoft+URL+Control+ +6.00.8862" with 85k hits. Not sure what that is, but I doubt it is Slurp.

To get to a bot that I do not reconize, I have to go down to "lwp trivial" with 98 hits - no where near the number of the MSNbot. None of the bots in the list say anything like "unknown".

I am concerned about this a bit. I know that our pages are better optimized for Y - and what pages we do have in Y rank high, such as Katherine's Landing (#1) (http://search.yahoo.com/search?p=Katherine%27s+Landing&ei=UTF-8&fr=fp-pull-web-t&n=20&fl=0&x=wrt), Prescott National Forest (#5) (http://search.yahoo.com/search?p=Prescott+National+Forest&ei=UTF-8&fr=fp-pull-web-t&n=20&fl=0&x=wrt), etc. If I can get some Slurp bait, and get Y to reconize us more, our traffic will soar. We sell sponsership to make money, so for us, traffic is the name of the game.

I am half tempted to spend a Saturday drinking beer, and cruising all over our site with the Y Toolbar, to see if I can get more pages indexed. Don't know what else to do. Worked hard increasing backlinks for the last two months. Oh - here's a maybe - when I first started SEO, I made the fatal flaw of getting involved in FFA linkfarms. I haven't done that in the last 2+ months, and most of those links have fallen off, but not all. Maybe Slurp is still looking at us in a negative manner? G doesn't - G (for comparision) visits almost daily. G has about 7000 pages indexed, compared to 700 from Y.

Any ideas, I am all game. Thanks!

[interactive]

[QUOTE=eitemiller]

I am half tempted to spend a Saturday drinking beer, and cruising all over our site with the Y Toolbar, to see if I can get more pages indexed. Don't know what else to do. [/QUOTE]

eitemiller,

Where did this idea come from? Just theory?

Same thing, we've got 6K pages in G and less than 100 in Y! I can't seem to crack Yahoo! either. Many of the pages they have indexed are months old.

Unfortunately, building pages and links just isn't cutting it for me and Yahoo!
(yes, I know that's poor grammar)

[eitemiller]

[QUOTE=interactive]eitemiller,

Where did this idea come from? Just theory?
[/QUOTE]

You talking about the Y Toolbar? More than theory, but can't call it science yet. Here's what I got...

We went live on March 1st, this year. From March first, up to August 10th, Yahoo had indexed 109 pages out of 25,000. (G at that time was up to 4800 just for comparision). I had read in a different forums that it is "very likely" that the Y toolbar does help Y in putting together thier database. This isn't that far-fetched of an idea, because it's pretty common knowledge if you install the Alexa toolbar, you can increase you rankings there. So anyways, on Aug 10th, I install the Y toolbar, and drove all over our site. Four hours later - literally - pages indexed in Y went from 109 up to 574. Since that day in August, our pages indexed by Y vary from a little at 550-700.

I have definatly tried to duplicate the results. I got excited, and starting running all over my site with the Y toolbar, and our pages indexed by Y have not changed much since that Aug 10th day. You might think it's a coincidence, but then you have to tell me, how did I get as many pages indexed by Y as I did WITHOUT Urchin showing Slurp as a bot that visited my site.

[eitemiller]

On that note:

seo-guy.com (http://www.seo-guy.com) has 57,700 pages index in G , and only 862 indexed in Y.

[jocelyn]

I think the new slurp referer description is :

Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp)

So they did change it.

[eitemiller]

[QUOTE=jocelyn]I think the new slurp referer description is :

Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp)

So they did change it.[/QUOTE]

I get "Mozilla Compatible Agent" in Urchin - ya think that's it, eh? It's doesn't hit near as often as "MSNbot", kinda thought it would be closer than it is. Thanks for the tip, jocelyn.

[cpucandy]

[QUOTE=jocelyn]I think the new slurp referer description is :

Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp)

So they did change it.[/QUOTE]

I was looking at my log files and noticed the same thing today. Yahoo slurp only seems to be referencing my robots.txt? Is this normal?

[yogenmaniyar]

SLURP is working fine as my site was spidered within 4 days of going online!