iEntry 10th Anniversary Forum Rules Search
WebProWorld
Register FAQ Calendar Mark Forums Read
Search Engine Optimization Forum SEO is much easier with help from peers and experts! The WebProWorld SEO forum is for the discussion and exploration of various search engine optimization topics. Any non (engine) specific SEO or SEM topics should go here.

Share Thread: & Tags

Share Thread:

Reply
 
LinkBack Thread Tools Display Modes
  #1 (permalink)  
Old 09-06-2005, 03:05 PM
WebProWorld Member
 
Join Date: Mar 2004
Posts: 72
synergytheory RepRank 0
Default ISP's following robots.txt guidelines, possible?

I ask because I have a client with an off-line, marketed domain [short] and an online, search engine optimized domain [longer].

The off-line marketing domain is a "click here to enter" type page, we block bots from the page with a noindex,nofollow and a robots.txt

Our client is saying that some of her visitors can't access the off-line shorter domain [which is blocked using robots.txt etc] but can access the longer domain which isn't using a robots.txt to block the bots.

I was wondering if some smaller ISP's that use a cache, are seeing the robots file and not caching the site?

Just for the record I know a 301 redirect would work better than an entry page and I will use a 301, when I figure out why the above is happening.
Reply With Quote
  #2 (permalink)  
Old 09-06-2005, 04:37 PM
WebProWorld 1,000+ Club
 
Join Date: May 2004
Location: Dallas, Texas USA
Posts: 1,492
bhartzer RepRank 1
Default

Quote:
I was wondering if some smaller ISP's that use a cache, are seeing the robots file and not caching the site?
No, that generally doesn't happen. Without knowing the domain in particular I'm unable to give you any other specific reasons why it's not accessible. You might try looking at it through something like Rex Swain's HTTP Viewer to see what response codes are being given.

Do you have any indication of what those users are getting when they try to access the domain?
__________________
Bill Hartzer's Blog
Reply With Quote
Reply

  WebProWorld > Search Engines > Search Engine Optimization Forum

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On



All times are GMT -4. The time now is 11:02 PM.



Search Engine Optimization by vBSEO 3.3.0