SEO | Link Popularity | Search Engine Consulting | SEO Tutorial | SEO Tools | SEO Forum
Reply
 
Thread Tools Search this Thread Rate Thread Display Modes
  #1  
Old 06-20-2004, 11:42 PM
BungalowBill BungalowBill is offline
SEO Junior
 
Join Date: Jun 2004
Posts: 10 BungalowBill is on a distinguished road
Secured Archeive Pages

I need to submit articles that are protected by a cookie, using .asp. If the cookie is present the users get the article. If the cookie is not present then a seperate page is displayed asking the person to join. What will happen If I submit the article page to the browser. Will it index the target article or stop when it has not cookie?
Reply With Quote
  #2  
Old 06-21-2004, 02:31 PM
Pyrrhonist Pyrrhonist is offline
Uber Mod
 
Join Date: Jan 2004
Posts: 681 Pyrrhonist has a spectacular aura aboutPyrrhonist has a spectacular aura aboutPyrrhonist has a spectacular aura about
Send a message via ICQ to Pyrrhonist
What you're going to need to do is put an exception on the page, so that the search engine doesn't need the cookie to view the article. This is something similar to what www.medbroadcast.com does. You can see that many of their pages are indexed by Google, but can't be viewed until you agree to the terms of service when you visit them. (They use a php flag to indicate that you've agreed, but it's the same type of idea.)
__________________
SEO Tutorial
Bob's quote of the week:
SEO Bob: "world cup of hockey is starting isn't it [, Dilligaf]? Who are you cheering for dilli?"
Dilligaf: "Well... if you promise not to tell my neighbours... "Oh Canada... ""
Reply With Quote
  #3  
Old 06-22-2004, 01:02 PM
BungalowBill BungalowBill is offline
SEO Junior
 
Join Date: Jun 2004
Posts: 10 BungalowBill is on a distinguished road
Spyder Bot

But how do I know if it's a spyder?
Reply With Quote
  #4  
Old 06-22-2004, 01:17 PM
Pyrrhonist Pyrrhonist is offline
Uber Mod
 
Join Date: Jan 2004
Posts: 681 Pyrrhonist has a spectacular aura aboutPyrrhonist has a spectacular aura aboutPyrrhonist has a spectacular aura about
Send a message via ICQ to Pyrrhonist
In order to determine whether it's a spider or not, you want to examine the HTTP_USER_AGENT that is sent with every request to your site. What you have to create is a conditional that says:
Code:
if (spider) 
   ignore cookie requirement
else 
   require cookie


I did a quick search for lists of user agents, and found quite a large list here:

http://www.siteware.ch/webresources/useragents/db.html
and http://www.pgts.com.au/pgtsj/pgtsj0208c.html

I have no idea whether these are the best or not - I just wanted to provide you with an example.

As to how to implement this, there's a couple ways you could do it. You could put a comprehensive list of user agents into a file, and then load the file and check against that. Doing it that way gives you the advantage of being able to update the list as often as required, and makes maintenance really quite easy.

You can also load the user-agents into an array and iterate through the array when checking.

Or the quick and easy hack is to choose 2 or 3 really important ones (googlebot and inktomi slurp come to mind) and create a longer if statement (if (googlebot || inktomi.slurp) and so on).

Oh yeah, you can check to see how this is working for you (and if it is working at all) by setting your script to exempt mozilla and then trying to load the page. If you can get in without the cookie, then you know you're set, and you can get rid of the exemption for Mozilla and off you go on you way.

Does that help at all?
__________________
SEO Tutorial
Bob's quote of the week:
SEO Bob: "world cup of hockey is starting isn't it [, Dilligaf]? Who are you cheering for dilli?"
Dilligaf: "Well... if you promise not to tell my neighbours... "Oh Canada... ""
Reply With Quote
  #5  
Old 06-22-2004, 07:54 PM
BungalowBill BungalowBill is offline
SEO Junior
 
Join Date: Jun 2004
Posts: 10 BungalowBill is on a distinguished road
Thanks SEO Bob

Thank-you, thats just the info I needed.

Claude Wilson (BungalowBill)
Reply With Quote
  #6  
Old 06-22-2004, 08:43 PM
Pyrrhonist Pyrrhonist is offline
Uber Mod
 
Join Date: Jan 2004
Posts: 681 Pyrrhonist has a spectacular aura aboutPyrrhonist has a spectacular aura aboutPyrrhonist has a spectacular aura about
Send a message via ICQ to Pyrrhonist
No problem Claude,

just a hint, put your site in your signature - it will end up giving you a backlink for every post that you make.
__________________
SEO Tutorial
Bob's quote of the week:
SEO Bob: "world cup of hockey is starting isn't it [, Dilligaf]? Who are you cheering for dilli?"
Dilligaf: "Well... if you promise not to tell my neighbours... "Oh Canada... ""
Reply With Quote
  #7  
Old 06-23-2004, 09:09 AM
eCommando eCommando is offline
Premium member
 
Join Date: Apr 2004
Posts: 366 eCommando is on a distinguished road
That's kinda like cloaking...
Reply With Quote
Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump


Login/Register
User Name
Password
Remember Me?

Forum Links
Forum Home
SEO Forum
Internet Marketing Forum
Web Design Forum
Web Hosting Forum
Programming Forum
SEO Chat

Quick Links
Forum Home
New Posts
Mark Forums Read
Open Buddy List
User Control Panel
Edit Avatar
Edit Profile
Edit Options
Miscellaneous
Subscribed Threads
My Profile

Search Forums

Advanced Search
All times are GMT -8. The time now is 05:37 AM.


Powered by: vBulletin Version 3.0.3
Copyright ©2000 - 2013, Jelsoft Enterprises Ltd.