PDA

View Full Version : google found our dev server



obuwebco
06-15-2004, 12:40 PM
Hey there, any know how to unspider a site..?

Our development server that we use for testing and the like of websites and email campaigns was spider by google.. so our testing pages are showing up in google, which is bad.. we have just created a new robots.txt file, to limit spiders, but is the damage already done?

we do not want our testing server to show up on the search engines.. any ideas?

thanks

bhartzer
06-15-2004, 12:46 PM
You can use the Google Removal Tool to remove the site from the index, usually within a few minutes:

http://www.google.com/remove.html

Mel
06-16-2004, 03:08 AM
If your robots.txt bans all robots you should be dropped from all search engines within one or two spidering cycles.

If you want faster try bhartzers suggestion.

obuwebco
06-16-2004, 11:46 PM
Thanks for the help