odd search results

[chandrika]

I am seeing a strange search result in google

findallsorts site:tohid.jamroll.net - Google Search

findallsorts is my site, but there is this site tohid jamroll, that has some cgi script that seems to show my pages. Actually it is not just my site, it displays many other sites this way and I just wondered what it is about and whether it will affect my seo.

I first spotted it in copyscape, so wondered if them using this script would be a dupe content problem for my site?

[wige]

They are a proxy service. This could cause you to have a duplicate content issue. I know of a method that will break their display of your content, however, I am trying to figure out the best implementation that will benefit your site...

First, to make sure this would work in the first place, is your site on Apache, and are you able to edit your .htaccess file, and create custom error documents? Also, do you have PHP?

[dozzn]

Hey! ,
I would like to add a little comment to the question. ./ comment...

The good new is... every time I search on google I see 'odd search results' ahaha.


Have you submitted your site to any free directories or ezines.
One time I had a website and I submitted it this free ezine site.... a week later my site could be found on google 30 times in a row but when you clicked on the result it would take me some bogus page that had nothing to do with the website I submitted ( and I couldn't find my site there either). I don't even have that website anymore but I am still upset about that ahahah!

[chandrika]

Originally Posted by wige

They are a proxy service. This could cause you to have a duplicate content issue. I know of a method that will break their display of your content, however, I am trying to figure out the best implementation that will benefit your site...

First, to make sure this would work in the first place, is your site on Apache, and are you able to edit your .htaccess file, and create custom error documents? Also, do you have PHP?

Hi, yes it is on a dedicated apache server with php and I can edit the htaccess and create custom error pages so I am very interested to know of your method to break their display.

Thanks

[chandrika]

Originally Posted by dozzn

Hey! ,
I would like to add a little comment to the question. ./ comment...

The good new is... every time I search on google I see 'odd search results' ahaha.


Have you submitted your site to any free directories or ezines.
One time I had a website and I submitted it this free ezine site.... a week later my site could be found on google 30 times in a row but when you clicked on the result it would take me some bogus page that had nothing to do with the website I submitted ( and I couldn't find my site there either). I don't even have that website anymore but I am still upset about that ahahah!

I have yes, my sitemaps get scraped for this kind of thing quite alot. This one was somewhat unusual as it was actually displaying my content under their url, as opposed to redirecting to some other page. Wige says it is a proxy service, I had not come across this before.

[wige]

Removing your site from the proxy duplicators will require a few steps, and may take some time.

The first step is to create a new page on your site that only shows the user agent of the visitor's browser. Use the proxy web site to access that page, and note what the user agent is. In most cases, it will not be the same as a regular browser user agent. Lets say the user agent is "bad proxy 2.0"

Next, you need to create a custom error page. The proxy server will change any links on your site to keep the user in the proxy system, so any link you put in the page would not work. The page should simply contain a message that the page is being accessed with an unauthorized proxy, and please visit the site directly at www.domain.tld. This page should be located in a subdirectory of the site root, so that it can be made accessible even after everything else is blocked.

Next, since you have a dedicated server, open up your server configuration, and add the following line:

BrowserMatch "bad proxy 2" bad_browser

This should be added right before the virtual hosts section.

Finally, add the following to your .htaccess file:

<Directory />
Order Allow,Deny
Deny from env=bad_browser
Allow from All
</Directory>
<Directory /errordocs/>
Order Deny,Allow
Allow from All
</Directory>

The first directory node needs to point to the web root, and the second directory node needs to point to the directory wherein your error document is located. This will ensure that the proxy system can only see the error messages. Also, the error document you created needs to be specified for 403 (access denied) errors.

Note, you can expand on this a bit further by adding the user agents of spambots after the BrowserMatch directive in the main configuration I mentioned, like this:

BrowserMatch "spambot" bad_browser

This will cut down massively on your bandwidth, block many spambots once you identify them, and cut down on proxy duplication as you find it.

Note, if the proxy system does not use a distinguished user agent, there is a slightly different approach to try. Let me know how this works though.

PS: This will take a while to actually take effect, since the proxy system needs time to see the change and start removing your pages. Your URLs will still show up, but you should not have an issue with new pages being added, or any risk of duplicate content penalties once the changes propagate.

[chandrika]

Thanks that is great I am going to try and implement that as soon as possible.

It is very timely advice, as a few weeks back I had a problem with the way I was trying to identify and stop bad bots, too many redirects meant it somehow stopped cron working, so i needed to do something else.

My server actually went down this afternoon, first time in ages that has happened, overloads on sql and was thinking that has been caused because since removing the old code i had to avoid too many bad bots, that they were again accessing the site making too many requests. So thanks and I will let you know the results of implementing this after some time.

[dozzn]

Originally Posted by chandrika

Thanks that is great I am going to try and implement that as soon as possible.

It is very timely advice, as a few weeks back I had a problem with the way I was trying to identify and stop bad bots, too many redirects meant it somehow stopped cron working, so i needed to do something else.

My server actually went down this afternoon, first time in ages that has happened, overloads on sql and was thinking that has been caused because since removing the old code i had to avoid too many bad bots, that they were again accessing the site making too many requests. So thanks and I will let you know the results of implementing this after some time.

I was thinking posting about this earlier...
if they are using frames to display your website maybe you could add a 'break out of frames' coding...

[chandrika]

Originally Posted by wige

Note, if the proxy system does not use a distinguished user agent, there is a slightly different approach to try. Let me know how this works though.

The proxy did not use a distinguished user agent. I was wondering what your other method was. In the meantime I checked the ip address that showed from the proxy which was jamroll.nets ip, and so I have blocked that ip from the server completely.

I am still going to implement what you said above.

[admonharr]

Although very advance thread but it really helps to read expert views and even expert level questions cheers guys

[wige]

If the proxy mimics a normal browser, you would need to block all IP addresses that they crawl from. You should be able to determine the range of IP addresses by visiting several sites that display the visitor's IP address, and putting those URLs into the proxy. You can block these IP addresses using the following line (added to the same place as the BrowserMatch directives above):

SetEnvIf Remote_Addr "^1\.2\.3\." bad_browser

This example will block every IP address starting with 1.2.3.x.

[full house]

is that part of algorithm?

[seosean]

Wow, there is some great content on here about removing your site from a proxy, thanks wige! I've actually been looking around for some content about this, not that I have this issue going on with my sites but just for future reference. Thanks again.

[ronwits]

Wee, I need to check our site also. Sometimes, I am experiencing the same issue