My site has Vanished from google???

[watto]

I awoke thismorning only to find my site Businesses for Sale - Buy or Sell Australian Business for Sale had vanished from google.com.au for my main search terms 'business for sale' and 'businesses for sale'. I have been on page 1 for nearly 2 years and today it is gone???

Not sure what to do? The only thing different that has been done recently is I submitted to about 50 directories including yahoo. I have also increased my link pop with relevant, quality links over the ;ast couple of months.

How do I contact google and ask for advice?

Any suggestions?

regards

watto

[incrediblehelp]

Google recently updated in the last few days. We you dong anything shady? Buying links, selling links? maybe someone reported you as a spammer. What does GWT tools say as far as penalties go?

[watto]

I was doing nothing shady at all. Never bought or sold links. Where do I find GWT tools penalty information??

watto

[watto]

I just remembered that last week I aaded quite a lot of code to my .htaccess file on the advise of webnaughts. see here SPAM Attack!

Could this be the reason?

Also, I noticed that the only page that has been effected is my home page. But when I search site:business-trader.com.au I noticed that the following page is is indexed Businesses for Sale - Buy or Sell Australian Business for Sale /index.php . How can this be?

watto

[Easywebdev]

I rarely post in the seo forums (more likely to find me in the web programming area of this site) but that does not mean that I am oblivious to what goes on in the SEO world.

Two things strike me straight away. You have posted 3 times in this thread and both times you listed your site you could not refrain from trying to add a totally miniscule amount of pr with keyword stuffed links to your site.

I don't have any clue as to how google approach the programming of their spiders but if someone came to me with a brief of programme a spider that crawls the net detecting spammy sites then I would run a first pass test with something similar to the following.

I dont need to go into regular expression syntax to give you an inkling but it goes as follows.

Look for:
Bold text, color red, larger text (in anchor links/styles etc) containing £, € and $ symbols, multiples of either symbol such as "make $$$$$, make ££££" etc.

Ridiculously high keyword ratio for the words "buy, sell, sale, make money" and so on and so forth.

Take a look at your site, I could not even be bothered to read the index. Google have programmers on their payroll that are a lot more seo savvy than myself and have seen all the scams in the book, SEO guys push the boundaries, the spider programmers get wise and what was a great tactic yesterday now get you delisted.

If you look at your own site as a visitor do you really need to ask why? It had to happen sooner or later.

Most of your links on your index double back to your index or to "buyabusiness.php". Time to take another look at your site and add more content to outweigh the amount of spam that is there. Looking at your site with the red $$$ etc it looks like one of those sites from an ebook "make a million dollars in a week".

Thats how I see it and you can be sure thats how the spiders see it.

Go through your site whilst looking at one of those get rich quick sites (yellow background, huge red font monetary symbols etc) and don't do what they do, stuff your site with keyword rich content but dont bold it or color it red, 1 monetary symbol is all that is needed to convey a price. Multiple monetary symbols = red flag to a spider.

Time to smarten up your site.

Another thing about .htaccess why? why bother with redirects, mod rewrite etc, if a little thought goes into the planning of a site then http://www.mysite.com/seo/articles/47 does not outperform http://www.mysite.com/index.php?cont...articles&id=47 both end up at the same location and both will rank similarily, up to 3 parameters on a dynamic url and google does not bat an eyelid. Silly needless .htaccess tricks will get you in the s($t rather quickly.

The rules of the seo game are well publicised. You only get penalties when you don't play by them.

[Peter (IMC)]

I'd search the problem in your backlinks. I don't know much yet about Google's last update, but I do know that links from forums, directories, link pages, etc. don't have that much value. And that's all Google is showing.

and yes,.. it seems that the root and /index.php are both indexed. Could be a duplicate page problem.

That you have to fix first!

[Easywebdev]

I just got through reading the .htaccess file you referred to in your third post. For crying out loud delete that crap from it, in fact delete your .htaccess altogether unless there is some specific rewrite rule you must have.

SetEnvIfNoCase User-Agent "libwww-perl/" bad_bot
SetEnvIfNoCase User-Agent "^libcurl-agent/" bad_bot
SetEnvIfNoCase User-Agent "^Java/" bad_bot
SetEnvIfNoCase User-Agent "^Mozilla/4\.0\ .*Win\ 9x\ 4\.90.*$" bad_bot

Bad bots my arse.

If you have any commercial web based software on your server then most likely banning libcurl is going to render your license invalid. 60% of scripts utilizing a call home licensing system will use libcurl. And banning jre's from detecting if your site supports java, hrrm real smart.

That is a load of crap someone read on a forum populated by clueless paranoids.

Take 10 minutes to read the apache documentation on .htacces and SetEnvIfNoCase and see what your nice shiney new .htaccess file does. It is not doing you any good, believe me.

[watto]

Peter, In regard to the duplicate page issue, the root and index.php have only both been showing as indexed today. I redirected it in my google webmaster tools and I also 301 rdirected it in my .htaccess file a long time ago.

Easywebdev, thanks for the tips. I have made a few changes to my homepage in regards to minimizing spam techniques (bold, red, linking to home page, etc). However, there are sites still ranking well that are far worse than mine (bsale.com.au). I'm pretty sure that my recent serp problem revolves around the recent code I uploaded to my .htaccess file. That was a whole lot of code and a friend of mine advised me to NOT upload it, but I did anyway. That will teach me......

There are some codes that will need to stay, redirects etc, but I have removed all of the rect code I added. I only added this 2 or 3 weeks ago and now this!!!

Now that I have cleaned out my .htaccess file from the currupt code, should I do a 'Request Reconsideration' via google webmaster tools? or should I just sit back and wait?

regards

watto

[watto]

In my webmaster tools I noticed in I have 1 x http error, which is my url. It is detailed as a 4xx error.

After researching I found that this is a server issue not being able to communicate with my site and googlebots. Do I need to contact my hosting company about this or would this be a direct result from the nasty code I had uploaded to my .htacces file (which is now removed)?

Regards

watto

[Webnauts]

In respect to the comments about the .htaccess file and the bad bots, I can only tell that since implemented, I save per month several GBs bandwidth. And I admit that I have paranoia, but I have already very good reasons for that. Anyway, lets stay on topic.

About the sites being de-indexed, what the hell are you talking about? I am just wondering what are these results buddy:

Site Search:
site:business-trader.com.au - Google Search
Results 1 - 10 of about 2,710 from business-trader.com.au. (0.03 seconds)

Cache:
Businesses for Sale - Buy or Sell Australian Business for Sale
This is Google's cache of Businesses for Sale - Buy or Sell Australian Business for Sale. It is a snapshot of the page as it appeared on 23 Jul 2008 09:39:31 GMT. The current page could have changed in the meantime.

and

Site Search:
site:business-sale.com.au - Google Search
Results 1 - 10 of about 206 from business-sale.com.au. (0.12 seconds)

Cache:
This is Google's cache of Online Business for Sale - Australian Businesses for Sale. It is a snapshot of the page as it appeared on 22 Jul 2008 08:17:21 GMT. The current page could have changed in the meantime.

Or is the discussion about another site and I missed something?

From my perspective, I can only tell that the site has other serious issues than just being hurt implementing those rules in the .htaccess file, since those cannot happen over night:

Just one example: Doing a site search site:business-trader.com.au this page comes first: Business for Sale Australia | Buy & Sell Australian Businesses for Sale

After all, it is obvious that you have bigger problems than what you are crying about.

Good luck.

[Webnauts]

I also have a question about that nasty code. I have that in my .htaccess file since April. When will my site get de-indexed too?

In addition, please notice that my crawl cycle is 2.9 days and nothing happen yet! And my rankings have not been affected.

[watto]

Webnauts,

If you remember when I added the first lot of code you recommended, instantly I could not access my site. It said 500 internal error.....you can refresh your memory here SPAM Attack!

Then you modified the code and I added it to my .htaccess file. Big mistake!!! The next thing I know my homepage has been de-indexed. It didn't happen over night as you suggested in your reply, it happend about 2 weeks later, which was yesterday.

Whether you like to admit it or not, this is to much of a coincidence. It is obviously this is a direct result from adding the code. If it was for other reasons as you suggest, why would my google webmaster tools tell me I have a http error 4xx error with my home page only???????????? Are you familiar with http error 4xx error? You tell me what it means???

Just because this code works for your site doesn't mean it will work for my site. May be you should put a lot more thought into your suggestions. But I don't blame you, because ultimately I am the one who took your advise. I blame myself.

Based on your response webnauts, do you think that Easywebdev is giving me bad advise when he suggest removing all of that "crap" you told me to upload to my .htaccess file??????????????

If you read my original post you will see which site I am referring to.

You mentioned that doing a site search site:business-trader.com.au this page comes first: Businesses for Sale - Buy or Sell Australian Business for Sale /blog , well this has only happend since yeaterday, when my homepage was de-indexed.

[watto]

Since removing the code yeaterday, my google webmaster tools now tell me that there are (0) http errors........WOW! what a coincidence, don't you think??????

[Feydakin]

I would wager you had a typo in there somewhere causing you a problem..

@easywebdev, the keyword stuffing you are referring to is an automatic piece of this forum where it grabs the title of the page you are linking to automatically..

[watto]

A typo maybe the case, however I just copied and pasted as suggested. As I mentioned, webnauts code should have been more thorough if a typo was the case.

Thanks for clearing up the keyword stuffing pont that Easywebdev pointed out......I wasn't quite sure what he meant.

watto

[Webnauts]

Originally Posted by watto

A typo maybe the case, however I just copied and pasted as suggested. As I mentioned, webnauts code should have been more thorough if a typo was the case.

That is strange. I copy and pasted the content of my .htaccess file into my post you mentioned, and until today I never had a server error.

Many of my clients and associates are using that without problems. Don't you think it is strange that it screwed up with you?

And again. Your site have not been de-indexed. You problem was a rankings flux. Evidence provided in my previous posts. And at last make something clear here:
When a bot gets an HTTP error does not de-index sites/pages right away. And if other professionals here look closer to my posts above, they will confirm that I am right all the way.

At last your site needs seriously professional optimization if you want to accept that or not.

Good luck Peter.

[watto]

Webnauts, please read my post before replying:

1) I didn't say there was an error in your code
2) Yes I do think it is strange that it had my homepage de-indexed, but not surprising after the '500 internal error' with the original code you gave me
3) I didn't say my site was de-indexed, I said my homepage was de-indexed (business-trader.com.au)
4) I hate to tell you, but if it was a ranking flux why is the url not indexed????????????
5) I didn't say the url was de-indexed right away. If you read my post it say it was de-indexed 2 weeks later, which was yesterday
6) If my site needs professional optimisation, why have I been on page 1 in google.com.au for my main keywords for the last 2 years?????
7) I don't think all of the other 'professinal seo's need you to comment on their behalf......lol

webnauts, can you please answer this, why was my homepage url listed as a http error 4xx error in my webmaster tools and when I remove the code from my .htaccess file it disappeared? I can answer it for you, it is because of the code you gave me. Just accept it and stop looking for other reasons when it is quite obvious.

[watto]

Now that this code has been removed and the 4xx error is now gone from my webmaster tools, would it be safe to say that homepage will be re-indexed the next time googlebots visit my site?

When my homepage is re-indexed will my rankings go back the the way they were on Saturday (which have been the same for the last 2 years, until yesterday)?

All help is greatly appreciated.

regards

watto

[activeco]

Originally Posted by watto

Now that this code has been removed and the 4xx error is now gone from my webmaster tools, would it be safe to say that homepage will be re-indexed the next time googlebots visit my site?

Re-indexed - yes, very soon, however usually you'll have to wait a little longer until your good positions come back.

Next time, evaluate thoroughly any free advice you get on this forum before implementing it.

Originally Posted by Easywebdev

You have posted 3 times in this thread and both times you listed your site you could not refrain from trying to add a totally miniscule amount of pr with keyword stuffed links to your site.

Easyweb, the system on this forum automatically adds the title from a linked page.

Originally Posted by Easywebdev

Bad bots my arse.

If you have any commercial web based software on your server then most likely banning libcurl is going to render your license invalid. 60% of scripts utilizing a call home licensing system will use libcurl. And banning jre's from detecting if your site supports java, hrrm real smart.

That is a load of crap someone read on a forum populated by clueless paranoids.

Take 10 minutes to read the apache documentation on .htacces and SetEnvIfNoCase and see what your nice shiney new .htaccess file does. It is not doing you any good, believe me.

Very nice. Man, you should post more often to the SEO sections. We miss some real quality here.
Many so-called SEO-ers have indeed no clue when it comes to networking, programming or server basics, which all constitutes a huge part of proper SEO.

[watto]

Activeco, thanks for your imput. I'm very happy to hear that I should be re-indexed very soon. Very happy!!! I really have learned a valuable lesson. I was actually advised by my friend to not paste this code inot my .htaccess file because it was quite a lot and he didn't trust it, but I didn't listen and suffered the consequences. I will never do this again.

Hopefully my good positions will be back sooner than later. I'm hoping that because I am in Australia, and competion is weaker here than the US, I will bounce right back. Fingers crossed!!!

Are we talking 'days' to be re-indexed?

Regards

watto

[activeco]

As noted above you're already re-indexed.
When doing similar changes a good habit should be to check accessibility with e.g. Find broken links on your site with Xenu's Link Sleuth (TM).

[watto]

Actually, where did you note above that I am 'already' re-indexed?

My site still hasn't been crawled by the googlebots and as a result my site homepage url hasn't been re-indexed yet. I just noticed that in my webmaster tools it now say in 'error for urls in sitemaps' that my homepage url (wwwbusiness-trader.com.au/) is a 404 (not found).

I'm guessing this wil change when I get crawled and re-indexed.

[activeco]

Originally Posted by watto

Actually, where did you note above that I am 'already' re-indexed?

My site still hasn't been crawled by the googlebots and as a result my site homepage url hasn't been re-indexed yet. I just noticed that in my webmaster tools it now say in 'error for urls in sitemaps' that my homepage url (wwwbusiness-trader.com.au/) is a 404 (not found).

The term "indexed" means that your site exists in a search engine's database and is shown to users.
As Webnauts noted Google indexed your site without problem: site:business-trader.com.au - Google Search
A few pages I clicked were accessible.

Google sees your homepage as http://www.business-trader.com.au/index.php and it is accessible too.

Have you made some changes in sitemaps? E.g. you referred to wwwbusiness-trader.com.au/ which could be a typo, but also a serious problem.

[inertia]

From my experience (and excuse me if this has been said before, i havent go time to read the lot) sites disappear for odd days all the time, in the same way as sites appear for one day all the time. Clients usually ring up within 4 minutes of disappearing giving me hell about changing things and correcting this. I have to remind them that we cant assume they disappeared until a few weeks down the line. If you start changing things and panicing sometime you make things worse. In this case fixing the errors in GWT is a good idea and should be done pronto anytime.

Just a comment on what Easywebdev said about links to the same page, i counted 10? I'd say this is too spammy but i would like other peoples opinions...

[watto]

activeco, it is only my homepage that is not indexed. It is only my homepage that had the 4xx error in my GWT (which has now corrected itself after I removed the code from my .htaccess file). This is the problem. I am aware that every other page in my site is still indexed and ranking well. The home page is still accessable, but not indexed!

Google has never, ever indexed my homepage as Businesses for Sale - Buy or Sell Australian Business for Sale until this problem occured (as mentioned in previous post).

No changes have been made to my sitemap.

I only referrd to my site with the typo so it wasn't a link in this thread.

Inertia, my site has never, ever, ever disappeared from the serps. I am aware that some sites do frequently disappear, but my site has never disappeared in 2 years........Thats a long time.

[watto]

If you visit my site Businesses for Sale - Buy or Sell Australian Business for Sale and take a look at the cache.............You will see exactly what I have been talking about. It NOW says "Your search - cache:Businesses for Sale - Buy or Sell Australian Business for Sale - did not match any documents", which means my homepage is not even cached!!!

As far as I know this doesn't happen with a rankings flux. Right? So, based on this would it be safe to say that my predictions are correct? Which was the code I uploaded has resulted in my current situation?

Based on this, now that I have removed the code, would it be safe to say that the next time my page is cached that my homepage will be re-indexed and my rankings will return??????

It has been 3 days now and my conversion went from 5 per day to zero per day. This exercise has cost me quite a lot of money!!

watto

[activeco]

Watto, Google has chosen your homepage to be "http://www.business-trader.com.au/index.php", instead of "/". Do you think it's wrong?

[watto]

activeco, yes this is wrong. Google has never previously chosen "http://www.business-trader.com.au/index.php", it has always chosen "/", until now.

activeco can you explain why my homepage 'all of a sudden' has not been cached?

[activeco]

Check all your links and find out which ones point to index.php. Change them to "/" if you need to.

[watto]

I have zero links pointing to index.php version. We are getting of track here a little bit. The problem stems around the code I pasted into my .htaccess file which can be viewed at the link provided in my earlier post in this thread.

The de-indexing of my homepage (only), the fact that my homepage all of a sudden is not cached (after 2 years at the top of google.com.au), the http 4xx error in my GWT for "www.business-trader.com.au/" (which now say 404 not found after code was removed) are a direct result from the code I uploaded to my .htaccess file, which I have now removed.

Obviously and based on GWT, there was an error with the url. It had nothing to do with spam techniques, rankings flux etc, otherwise why would GWT tell me there was a 4xx error, and why is my url not cahced?

[Webnauts]

OK Peter,

this is getting seriously ridiculous.

You homepage is duplicated. You have:

Businesses for Sale - Buy or Sell Australian Business for Sale

and this

Businesses for Sale - Buy or Sell Australian Business for Sale

No 301 redirect andboth pages PR are greyed out!!!!!!! (Or is my toolbar wrong?)And do you want to tell us here that a 404 error would be a reason Google to grey out your toolbar? Come on buddy.

Your problem is that Google identified the duplicated homepage and de-indexed the one and kept the other. What the hell does those htaccess rules have to do with that. Do you want to drive us nuts? Sorry buddy.

I think it is time to pay a professional to fix your site, if you are really making your living from it.

Good luck.

[Webnauts]

I just noticed that all pages of the site are greyed. Looks like the nofollow did not really help you after all.

If you remember I warned you a while ago. Anyway...

[watto]

webnauts, I think your toolbar is wrong. I have check with other computers and they all say pr3. Your funny...lol

As previously mentioned I 301 redirected non www to the www version 12 months ago. my url used to be displayed in the serps as "business-trader.com.au", but after I did the redirect they now display as "www.business-trader.com.au"

I hate to tell you webnauts, but this theory is wrong. This is a direct result from the code, and I'll tell you what, when google visits my site next I will come back and let everyone know if it was re-indexed now that the code is removed.

If it does get re-indexed surely you will have to agree with me then?????

Let it go webnauts, I'm sure you didn't set out to ruin my site. As I mentioned, I blame myself.

Read through this thread and you will see which way fellow members are swaying.........lol

[Webnauts]

Originally Posted by watto

Webnauts, please read my post before replying:

1) I didn't say there was an error in your code

OK

Originally Posted by watto

2) Yes I do think it is strange that it had my homepage de-indexed, but not surprising after the '500 internal error' with the original code you gave me

I page does not get de-indexed when an SE gets a Server Error. If the Server error is there longer, then yes. But it was not. Otherwise your site would have been offline.

Originally Posted by watto

3) I didn't say my site was de-indexed, I said my homepage was de-indexed (business-trader.com.au)

I explained above. Google have chosen the homepage version you did not want them to take.

Originally Posted by watto

4) I hate to tell you, but if it was a ranking flux why is the url not indexed????????????

Duplicated content.

Originally Posted by watto

5) I didn't say the url was de-indexed right away. If you read my post it say it was de-indexed 2 weeks later, which was yesterday

You answered the question yourself. It has nothing to do with the server error. Or didn't Google bot visit your site within that timeframe? If they did then or later and the server error was no more there, that could not happen.

Originally Posted by watto

6) If my site needs professional optimisation, why have I been on page 1 in google.com.au for my main keywords for the last 2 years?????

My newest customer told me today: "John, but I was ranking for the term "marketing company" for 5 years now number one and now I am not in the first 100." Looks like your site does not conform anymore to the over 450 algo tweaks of last year and how many of this year. Google is tweaking, but looks like you are sleeping. Don't you think?

Originally Posted by watto

7) I don't think all of the other 'professinal seo's need you to comment on their behalf......lol

I do not mind. It would be nice though.

Originally Posted by watto

webnauts, can you please answer this, why was my homepage url listed as a http error 4xx error in my webmaster tools and when I remove the code from my .htaccess file it disappeared? I can answer it for you, it is because of the code you gave me. Just accept it and stop looking for other reasons when it is quite obvious.

Here I must drop the topic. I feel like I am talking to a wall.

[Webnauts]

Originally Posted by watto

Read through this thread and you will see which way fellow members are swaying.........lol

Can you tell me exactly which member you are talking about? If you are talking about the web developer, re-read his post. If you are not using a licensed software that uses those rules, you should not have a problem. He just doesn't care about bad bots. But Google and the other SE are not blocked with those rules. Or did someone else disagree with me? If yes, please point me out who and which post. I just can't find anything.

AND THIS TO ALL MEMBERS IN THIS THREAD!!!

Does this url Businesses for Sale - Buy or Sell Australian Business for Sale redirect to Businesses for Sale - Buy or Sell Australian Business for Sale ???

After all, I am definitely drawing myself back. I am not willing to help any further. I think others already are helping you out, so I am not necessary here anyway. There are endless of members here who would love to have my horrible support and they sure will get it.

End of story.

P.S. I just want to invite Wige who helped me out completing the .htaccess file, but I assume it is worthless to do so.

[watto]

point 2) the server error was there the whole time the code was uploaded. I just didn't notice it until it was too late

point 3) The redirect from NON www to the www version is for every page in my site, not just the homepage, so why would only the homepage be de-indexed for duplicate content and not the whole site if what you are saying is true??????? I think this only happend when the code was in place. It must have conflicted with my redirect. That was a whole lot of code I upload (see Easywebdev's post).

point 4)

Point 5) googlebot did NOT visit my site in that time frame. So what do you have to say about that????

Point 6) please!!! whay does every other page on my site still rank on page 1 in google.com.au for their main phrases ie new south wales business for sale, queensland business for sale, cafes for sale, transport business for sale, etc, etc, etc.....you get the picture

point 7) I don't think the other members would like you to comment on their behalf. I'm sure they can speak for themselves, and I'm glad they did, because going back through this thread most members have told me what a big mistake it was to add the code.

webnauts, you always try to have the last word. Lets just wait and see if I get re-indexed when google visits next. For my sake I hope I do and if I do get re-indexed, then we can only assume it was the code. Right???

[Webnauts]

Originally Posted by activeco

Many so-called SEO-ers have indeed no clue when it comes to networking, programming or server basics, which all constitutes a huge part of proper SEO.

I do not know about which SEOs you are talking about, but if we are talking here about the networking, programming and server basics which constitute a huge part of proper SEO, here is my stuff:

SEO Workers Search Engine Optimization Consulting Company

HEAD / HTTP/1.1
Host: SEO Workers Search Engine Optimization Consulting Company
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.1) Gecko/2008070208 Firefox/3.0.1
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Referer: SEO Workers Search Engine Optimization Consulting Company
Cookie: __utma=215523349.2906040448828528600.1217192195.12 17287045.1217291733.14; __utmz=215523349.1217231055.6.5.utmcsr=webtalkforu ms.com|utmccn=(referral)|utmcmd=referral|utmcct=/showthread.php; __utmc=215523349; __utmb=215523349.1.10.1217291733
X-lori-time-1: 1217291733333
If-Modified-Since: Sat, 26 Jul 2008 21:03:33 GMT
If-None-Match: "ae34513ae10d65a82a7dfc8fa527df7d"

HTTP/1.x 200 OK
Date: Tue, 29 Jul 2008 00:35:36 GMT
Server: Apache
Etag: "ae34513ae10d65a82a7dfc8fa527df7d"
Last-Modified: Sat, 26 Jul 2008 21:03:33 GMT
Cache-Control: private, must-revalidate
Pragma: no-cache
Content-Description: SEO Workers - Organic Search Engine Optimization Consulting Company
Vary: Negotiate,Accept-Encoding
Content-Encoding: gzip
P3P: policyref="/w3c/p3p.xml"
Pics-label: (pics-1.1 "http://www.icra.org/pics/vocabularyv03/" l gen true for "http://seoworkers.com" r (n 0 s 0 v 0 l 0 oa 0 ob 0 oc 0 od 0 oe 0 of 0 og 0 oh 0 c 3) gen true for "http://www.seoworkers.com" r (n 0 s 0 v 0 l 0 oa 0 ob 0 oc 0 od 0 oe 0 of 0 og 0 oh 0 c 3))
Imagetoolbar: no
Connection: close
Content-Type: text/html;charset=UTF-8
Content-Language: en-us

And about blocking bad bots, we save a hell a lot of bandwidth and are paranoia is reduced to the minimum:
https://seoworkers.com/plesk-stat/we...astrobots.html

Notice we added more bad bots the last few days.

Is there something that can be improved, especially when it comes to SEO needs? We have implemented all necessary HTTP headers, including If-Modified-Since and If-None-Match as Google REQUIRES in their webmaster guidelines? Anybody else here? Any further suggestions? Tips are always very welcome.

[Webnauts]

Are you claiming that your site has a server error for days or weeks and you did not notice?

Originally Posted by watto

webnauts, you always try to have the last word. Lets just wait and see if I get re-indexed when google visits next. For my sake I hope I do and if I do get re-indexed, then we can only assume it was the code. Right???

First you are not saying is not true. You asked me a while ago for a quote. If you remember, we went briefly through the site and I remember damn well that there were no redirects and I told you that we should do some server side work for that. Can you remember now?

And Google does not see the whole site as duplicated in such cases. They only see the homepage. How can they see the other pages as duplicated? Is this a joke or do you want to drive everyone nuts here?

If you do not implement the redirects which were always missing and your page without index.php gets indexed again, then we can look once again if the problem was that God damn code, which I still insist that it was NOT, since it is physically not possible! Period!

[watto]

webnauts, I have just realised that the redirect you are referring to from the ".index.php" to the "/" was not in place. The only direct I have set up is the non www version to the www version. As a result, I believe that the code you supplied me with may not have been the issue after all. If this is the case a truely apologies for my accusations.

And due to the fact that google has indexed the ".index.php" version and dropped the "/" version, this makes me believe even more that the code may have been fine. It may have been a total coincidence that this url was de-indexed around the same time the code was added.

I am truely baffled as to why google would de-index the "/" version as I have no internal links or backlinks including this url. I just don't understand why after all this time google would do that.

regards

watto

[Webnauts]

Originally Posted by watto

I am truely baffled as to why google would de-index the "/" version as I have no internal links or backlinks including this url. I just don't understand why after all this time google would do that.

What does that make for a difference. I edited myself the redirects in your .htaccess file to solve the problem. Isn't that enough? Jesus Christ Peter.

[watto]

I really appreciate you giving your time and knowledge to help with my redirect, but you can't blame me for asking the question why?

With no internal or backlinks pointing to the index.php version, it is very strange.

[Webnauts]

Originally Posted by watto

I really appreciate you giving your time and knowledge to help with my redirect, but you can't blame me for asking the question why?

With no internal or backlinks pointing to the index.php version, it is very strange.

Its OK Peter. I will stay subscribed to this thread and look very much forward to see what the other SEO experts have to say about this. I am damn curious myself.

[Webnauts]

Just thought to add here:

1. Your site map is broken: Google Sitemap Validator

2. You have 704 broken urls on your site.

Are you still wondering why Googlebot messes up when it crawls your site?

3. Are you sure that the mod_rewrites you have in your .htaccess file do not generate that index.php? Can you explain what those redirects are made for and what they do exactly? I think we are most probably closer to the answer. Explain again here what you have explained me over the phone about those redirects. Or why don't you post those mod_rewrites here so we can all have a closer look? Especially the Network experts.

[watto]

1) Thats interesting because GWT tells me that there are no errors.
2) How do I rectified those broken links?
3) Please find detailed below my entire .htaccess file. I'm sure what it all means, but I can explain what the rewrite codes are for:

# -FrontPage-



IndexIgnore .htaccess */.??* *~ *# */HEADER* */README* */_vti*



<Limit GET POST>

#The next line modified by DenyIP
order allow,deny

#The next line modified by DenyIP
#deny from all

allow from all

</Limit>

<Limit PUT DELETE>

order deny,allow

deny from all

</Limit>

AuthName Businesses for Sale - Buy or Sell Australian Business for Sale

AuthUserFile /home/busine31/public_html/_vti_pvt/service.pwd

AuthGroupFile /home/busine31/public_html/_vti_pvt/service.grp



RewriteEngine On

RewriteRule ^buy_sell_business_australia/(.*)/(.*)/(.*)/(.*)/(.*)/index.html addshow.php?bsz_name=$1&adv_id=$2&catid=$3&pno=$4& page=$5

RewriteRule ^buy_sell_business_australia/(.*)/(.*)/index.html addshow.php?bsz_name=$1&pno=$2


# for the search.php

RewriteRule ^buy_sell_business_search/(.*)/(.*)/(.*)/(.*)/(.*)/(.*)/index.html search.php?pno=$1&kw=$2&cat=$3&cont=$4&state=$5&pa ge=$6
RewriteRule ^buy_sell_business_search/(.*)/(.*)/(.*)/(.*)/index.html search.php?kw=$1&cat=$2&cont=$3&state=$4
RewriteRule ^buy_sell_business_search/(.*)/(.*)/(.*)/index.html search.php?cat=$1&cont=$2&state=$3
RewriteRule ^buy_sell_business_search/(.*)/(.*)/index.html search.php?cont=$1&state=$2
RewriteRule ^buy_sell_business_search/(.*)/index.html search.php?cat=$1

#for search_result.php

#RewriteRule ^buy_sell_business_result/(.*)/(.*)/(.*)/(.*)/(.*)/(.*)/(.*)/(.*)/index.html search_result.php?adv_id=$1&pno=$2&page=$3&catid=$ 4&kw=$5&cat=$6&cont=$7&state=$8

#RewriteRule ^buy_sell_business_result/(.*)/(.*)/(.*)/(.*)/(.*)/(.*)/(.*)/index.html search_result.php?adv_id=$1&pno=$2&page=$3&catid=$ 4&cat=$5&cont=$6&state=$7

#RewriteRule ^buy_sell_business_result/(.*)/(.*)/(.*)/(.*)/index.html search_result.php?kw=$1&cat=$2&cont=$3&pno=$4

#RewriteRule ^buy_sell_business_result/(.*)/(.*)/(.*)/index.html search_result.php?cat=$1&cont=$2&pno=$3
RewriteRule ^buy_sell_business_result/(.*)/ search_result.php?adv_id=$1




# for listings.php australian business classifieds

RewriteRule ^australian_business_classifieds.html listings.php
RewriteRule ^australian_business_classifieds_(.[0-9]*)_(.[0-9]*).html listings.php?pno=$1&page=$2

RewriteBase /

RewriteRule wsiinformation\.html wsiinformation.php


RewriteEngine On

RewriteBase /

Options +FollowSymLinks

RewriteCond %{HTTP_HOST} ^businesstrader.biz$ [NC,OR]

RewriteCond %{HTTP_HOST} ^www.businesstrader.biz$ [NC]

RewriteRule ^(.*)$ http://www.business-trader.com.au/$1 [R=301,L]

<Files 403.shtml>
order allow,deny
allow from all
</Files>


suPHP_ConfigPath /home/busine31/public_html

RewriteEngine On
RewriteCond %{HTTP_HOST} ^business-trader.com.au [NC]
RewriteRule (.*) http://www.business-trader.com.au/$1 [R=301,L]

RewriteCond %{HTTP_HOST} !^www\.business-trader\.com.au [NC]
RewriteCond %{HTTP_HOST} !^$
RewriteRule ^/(.*) http://www.business-trader.com.au/$1 [L,R]

RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /.*index\.php?\ HTTP/
RewriteRule ^(.*)index\.php?$ http://www.business-trader.com.au/$1 [R=301,L]




This is my entire .htaccess file.

This page Businesses for Sale - Business Classifieds now looks like Businesses for Sale - Business Classifieds after the rewrite. All of the individual listings on this page used to be dynamic and they all had the same meta tags and could never rank. After the rewrite the meta tags are automatically generated from each advertisement (title, description, keywords) and now each indiviual listing ranks in google.

The same applies for the state pages ie New South Wales Business for Sale - Sydney Businesses for Sale (these used to have the same meta tags as dynamic pages)

Same applies for the category pages ie Cafe Business for Sale - Cafes for Sale Australia (these used to have the same meta tags as dynamic pages)

Befor this rewrite I had less than 10 pages indexed. After the rewrite I had over 3000 pages indexed.

Now, tere still ramians 1 problem that no one can seem to fix. each individual listing has about 3 different urls. example this url is the actual url after the rewrite which ranks in google Michel's Patisserie for Sale . The problem is if you do a search for the same listing here Business for Sale - Buy Sell Businesses for Sale via category and state these are the other urls it will display:

Michel's Patisserie for Sale
Michel's Patisserie for Sale

[Webnauts]

Originally Posted by watto

Befor this rewrite I had less than 10 pages indexed. After the rewrite I had over 3000 pages indexed.

If I was you, I would not have been so proud about that, because:

Supplemental Ratio for business-trader.com.au: 95.16%

* Google has a total of 2730 pages indexed from business-trader.com.au
* 132 are in the main index
* 2598 are in the supplemental index

What are the supplementals about? Read my notes here: Supplemental Pages, Results & Ratio

[watto]

I'm not proud of it. But it is better than what is was. After my rewrite was finished, my traffic almost trippled over night.

[Webnauts]

Originally Posted by watto

I'm not proud of it. But it is better than what is was. After my rewrite was finished, my traffic almost trippled over night.

Well if that traffic is still available, then you got a great deal!

[watto]

Traffic has been consistant for 12 months, until now that is........

[Webnauts]

Originally Posted by watto

Traffic has been consistant for 12 months, until now that is........

I love facts. May I view your stats?

[watto]

I wouldn't lie to impress myself......lol I have emailed you.