How to remove single pages from the Google Index

[Dubbya]

"How do I remove a deleted page or an outdated URL from the Google index?"

I've seen this question posted in several forums and thought I'd take the opportunity to post an answer.

Whenever I've made some sweeping changes to my site, I use Google Webmaster tools to check for crawl errors.

While a broken hyperlink can be a simple fix, errors caused by missing files (deleted pages) will cause your placement in search results to drop drastically. Seemingly overnight, 10-20 missing pages can cost you over a hundred places and effectively bury your listing.

If you're unable to find a method to right the wrongs, your listing will continue to drop until you either recreate the pages or tell Google to stop indexing the missing pages.

There's good news though, Google has provided several methods, one of which will allow you to delete the problem file references from their index and it'll prevent Googlebot from trying to spider those pages again.

Log in to the Google URL Console and select the method you'd like to use to remove the outdated link.

While there have been reports of entire directories being removed from the index, it appears that the tool is ready for the masses. (I removed 22 separate references without incident.)

*Make sure you specify only the precise URL's to the pages you want to remove and not entire directories (unless that's your intention).*

Using the link deletion page, you'll have to specify each page individually and it'll take between 3-5 days to complete the process. You'll receive an email notification when the pages have been removed but you'll have to wait until the site is spidered again for the errors to clear from your Web Crawl Errors list in Webmaster tools.

Now, go clean up those outdated links!

[incrediblehelp]

Good advice. Have you found anyway to remove lots of URLs using wildcards with Google? For example:

User-agent: Googlebot
Disallow: *jal_no_js*

[bhartzer]

While Google's "tool" is supposed to be used for this purpose, I'm still finding it difficult to actually get Google to remove a URL from their index.

I actually still prefer the "old faithful" method of serving up a 404 error or redirecting the URL with a 301 Permanent Redirect.

It's also helpful to remove any internal links to the page(s) also.

Why do we need to go out of our way to notify Google that we've removed a page from a website when all we really need to do is serve up a proper 404 error? Am I missing something here?

[incrediblehelp]

Bill not sure if you missed this post about a issue I was having in Google with my blog, but that was what I was referring to.

[Dubbya]

There are a couple of other methods to do a bulk "delisting", one of them being via a robots.txt file.

It's my understanding that this is not the best method. Apparently, using the "noindex" may not work as intended since, from what I can tell, Googlebot keeps looking for the page to see if the "noindex" is there, thus perpetuating the file not found error.

I have not tried this, so can't say with absolute certainty that this is in fact the case, but wouldn't be at all surprised. We're talking about Google here.

I'd appreciate knowing if the robots.txt method proves successful, so if anyone tries it, please give us the "heads up".

[MetroMark]

Using the webmaster tools, I've seen not found errors show up in the crawler stats for pages that were 3 years old (and three years gone). They appear a half-dozen or so at a time, and then are replaced by a new set of old pages. One of my sites is an event calendar, and the daily calendar url's expire, well, daily, yet googlebot is still looking for them three years later. I have a sitemap.xml file that drops the old urls daily, yet they still have ancient urls tucked away somewhere.
I get the sense that Google is so distracted with all their 20% projects and gizmo's that they are not keeping up with search. I mean, getting sued by Viacom for a billion bucks has got to be pretty distracting, I presume.
Also, based on the original post, has anyone else seen their site dropped hundreds of placements in G due to old broken links on their site? Given the economic impact of placement, it's kind of sickening if that's true, and it's not the webmaster's fault.

[Dubbya]

Anyone?? Yes, ME!

I went from number 7 to 244 for the search phrase "toner cartridges" and "ink cartridges" dropped from number 11 to 310.

You're correct in that this is sickening but it was my fault.. I updated my site and changed some file names... live and learn.

It's not the end of the world though. I've removed the outdated links and my site just got spidered today. They're already coming back up, so I'd expect things should even out within a few days or weeks.