Yahoo is nor crawling all my site.

hrozob · #1 (**permalink**) 04-27-2006, 02:25 PM

Hello.
I'm not an expert so I would like to have some advice from you guys about the following issue.

I administrate a travel site, www.afinetour.com and Yahoo is not catching all the pages in my site.

Example. We have flights to Argentina, flights to Brazil, flights Chile, flights to Peru, etc, and yahoo has only cached the pages of flights to Chile and Bolivia, but all the other pages for flights are not in their index.

This is a URL that has not been cached http://www.afinetour.com/index/eng/f...category/Colom bia-Flights/

Can anyone give me some advice on how to make Yahoo to catch all my pages? And why while we have good rankings on Yahoo, Google ignores our site and we do not rank at all.?

I would like to have a different option than "Search Submit" as I consider that having to pay 0.30 per click for being listed on the regular results is like signing a blank check to Yahoo and unfortunately we do not have the budget to do that.

Thank you for sharing your experience.

Hector Rozo
www.afinetour.com

Chris · #2 (**permalink**) 04-27-2006, 03:56 PM

hey hrozob, you may want to look into a yahoo sitemap (done in text), which, in theory, informs Y of all the pages contained within your site.

once you've developed a Yahoo-compliant sitemap, submit it here. for more about sitemaps, check this guide out.

hrozob · #3 (**permalink**) 04-28-2006, 10:04 AM

I have done the txt file with the list of urls and I have submited it to Yahoo.
How long do they take to crawl again the site?

Thank you for your advice.

Hector

nipplecharms1 · #4 (**permalink**) 05-03-2006, 02:07 PM

Quote:

Originally Posted by Chris

once you've developed a Yahoo-compliant sitemap, submit it here. for more about sitemaps, check this guide out.

Chris, thanks for the link. Question... the link you provided starts "Creating Sitemaps for Google, MSN and Yahoo! - The Easy Way..."

Easy is relative. I know that. I created a sitemap for the "G" that I have in my main folder. Will that work for Yahoo or do I need a separate one for "Y"?

Thanks,

Michael

Chris · #5 (**permalink**) 05-03-2006, 02:39 PM

being that it's a text file, i don't see why it would have to be in another directory... but i'm not sure. the reason i think he calls it easy is because all you are doing is creating a file that has each page in your site's full path address in it.

i've never heard of anyone indicating one or the other, so i'm thinking it's a non-issue. but if anyone knows for certain and my response is incorrect, by all means say so.

as for the frequency of Yahoo's crawls, well, their bots crawl the web almost constantly and i'm sure once your text file is accepted, the bot responsible for your section of the web will be along shortly thereafter. just keep an eye on your site stats and look for increased Yahoo bot activity.

**incrediblehelp** · #6 (**permalink**) 05-03-2006, 11:11 PM

I see more than that:

http://search.yahoo.com/search?p=htt...&cop=&ei=UTF-8

You also going to want to 301 redirect http://afinetour.com/ to http://www.finetour.com/. Yahoo is indexing both, not good.

nipplecharms1 · #7 (**permalink**) 05-04-2006, 11:24 AM

Quote:

Originally Posted by incrediblehelp

You also going to want to 301 redirect http://afinetour.com/ to http://www.finetour.com/. Yahoo is indexing both, not good.

Ok for those of us that are young at heart.. I mean new to some of this... (I am far from young anywhere!) how do I make a 301 redirect?

Thanks

Michael

**incrediblehelp** · #8 (**permalink**) 05-04-2006, 11:57 AM

Please research it. It has been discussed and explained here and other forums many times.

nipplecharms1 · #9 (**permalink**) 05-04-2006, 12:11 PM

Quote:

Originally Posted by incrediblehelp

Please research it. It has been discussed and explained here and other forums many times.

Wow. thanks.

m

hrozob · #10 (**permalink**) 05-17-2006, 03:08 PM

Hello and thank you for all your valuable information.

We have set up the 301-redirect already.

One question. Is there any way to know when Yahoo crawled my site?
When I click on the cache link of the results, they do not show when was visited that page. And the statistics of my site are not showing Yahoo but Inktomi Slurp, are these the same?

Thank you
Hector
www.afinetour.com

**incrediblehelp** · #11 (**permalink**) 05-17-2006, 03:19 PM

Quote:

Originally Posted by hrozob

And the statistics of my site are not showing Yahoo but Inktomi Slurp, are these the same?

Yes

hrozob · #12 (**permalink**) 05-19-2006, 11:59 AM

Hello.

Yahoo has crawled our site and for example our page for Flights to Colombia is in the index now. (Last Inktomi visit was today).

2 weeks ago we were number 2 and today this page is not in the first 300.

The problem was first, that they had not crawled this page, and we submited the .txt file for yahoo to crawled the whole site. Now they indexed it but they do not rank it as they used to do.

Can anyone tell me if you see anything wrong?

Thank you
Hector Eduardo
www.afinetour.com

**incrediblehelp** · #13 (**permalink**) 05-19-2006, 12:42 PM

Hector are you seeing other page or keyword ranking drops? Has your overall traffic dropped? Has sign ups dropped?

Remember rankings change all the time on Yahoo, Google and MSN and watching is a sure way to drive yourself crazy.

hrozob · #14 (**permalink**) 05-19-2006, 05:33 PM

Thank you for replying.

We used to be between number 2 and 6 for keywords
like flights to Argentina, flights to brazil,
flights to Ecuador, flights to Colombia, but these pages were NOT crawled 2 or 3 weeks ago so we lost that traffic.

On the 17 of may Yahoo crawled again these pages
and now they are included on the index but they
completely lost the good ranking they used to have.

However two pages, flights to Bolivia and flights to Chile
were always included on the index and they have not lost their good ranking. (Number 1 and number 3)

It's like they stop crawling all the site to make us submit the url list and now they do not rank those pages as they used to do, in order to force us to buy Pay per click or the paid submission + cost per click.

I’m just guessing, but because of these we lost most of our traffic and we do not see any reason to have disappeared from the ranking.

Thank you

TrafficProducer · #15 (**permalink**) 05-19-2006, 05:48 PM

Does Yahoo have SandBox.? Don't know but if you think about it..

Any big engine that allows submissions must have a buffer, back-log, of sites waiting to get listed.

**incrediblehelp** · #16 (**permalink**) 05-19-2006, 07:34 PM

Quote:

Originally Posted by hrozob

I’m just guessing, but because of these we lost most of our traffic and we do not see any reason to have disappeared from the ranking.

Remember it doesn't have to be anything you have done. Your competition could have simply gotten better while you have done nothing.

**incrediblehelp** · #17 (**permalink**) 05-19-2006, 07:46 PM

Also remember Yahoo is one of the slowest to update its index regardless of the question a sandbox existing or not.

SemAdvance · #18 (**permalink**) 05-21-2006, 07:55 PM

Well beyond the fact your page naming convention is headed towards spamville.....

http://www.afinetour.com/index/eng/f...category/Colom Columbia-Flights/

Simply link from your home page to this page or now that it's posted here you have the deep link needed to get the page indexed.

Do you read your web hosting stats?

Watch the robots and the hits each time they visit your site.