Need help to re-index complete website

[vikashony]

Hi,

I have an website that contain many spamming pages or links that has been crawled by search engine and coming in search engine result. I have removed many them from my website but still Google showing its in search engine result.

I know I can remove links from Google toolbar but the link is approx 1000 may be more then it in quantity.

I just want to crawl my website again. Please help me is there anyway that I can re index my complete website.

One more thing would all pages may removed if I remove home page from Google toolbar? I need help on it

[deepsand]

Join Google Webmaster Tools and to Site Configuration --> Crawler access and click on remove URL's, add the links which you want to remove from Google Search
Then open the robots.txt file and disallow the links to prevent again indexing

The robots.txt cannot be used for doing anything other than blocking crawling of specified files and/or sub-directories.

To block indexing of on-page links requires the use of <META NAME="ROBOTS" CONTENT="NOFOLLOW"> on the pages that contain such links. However, this will also suppress references to internal links as well as external ones.

If a page contains both links to be and not to be followed, one must instead individually add rel="nofollow" to the latter.

[Shadders]

I just want to crawl my website again. Please help me is there anyway that I can re index my complete website.

You can re-submit your sitemap within Google web master tools. The updates to your web pages should filter through to the search results soon enough.

[deepsand]

You can re-submit your sitemap within Google web master tools.

For some time now, Google has been recommending that one use their Fetch as Googlebot rather than submitting a sitemap.

[Lynn32]

Google's cache will probably still show some of the 'spam' results for searches relating to your website. Re-indexing your site will not remove these cached pages.

[bzforum]

To block indexing of on-page links requires the use of <META NAME="ROBOTS" CONTENT="NOFOLLOW"> on the pages that contain such links. However, this will also suppress references to internal links as well as external ones.
If a page contains both links to be and not to be followed, one must instead individually add rel="nofollow" to the latter.

If the OP is saying there are many Spamming Pages...I am sure that the website is either a Forum or some other CMS which allows Guest or Member posting and then the pages are created...and in such cases it would be difficult to manually keep on adding Meta tags or making the links no follow...

If you have deleted them from your website them google will remove them from index after some time if it finds 404....or redirects

[deepsand]

Google's cache will probably still show some of the 'spam' results for searches relating to your website. Re-indexing your site will not remove these cached pages.

Google does not cache and display search results, but only extant pages that have been indexed.

Therefore, if pages are removed from the site or altered, once re-crawled and re-indexed, no caches will be displayed for those pages which were remove, and those of altered pages will change accordingly.

[deepsand]

If the OP is saying there are many Spamming Pages...I am sure that the website is either a Forum or some other CMS which allows Guest or Member posting and then the pages are created...and in such cases it would be difficult to manually keep on adding Meta tags or making the links no follow.

Doesn't change the fact that robots.txt cannot be used to bar discovery and indexing of links.

If he has links that he does not want indexed on pages that he wants indexed then he'll need to use one of the mentioned alternatives.

[deepsand]

Use <META NAME="ROBOTS" CONTENT="NOINDEX" instead of "META NAME="ROBOTS" CONTENT="NOFOLLOW">", ...

As <META ROBOTS> directives are not discovered until the page has been fetched and parsed, that will not prevent the page from being crawled, and the URL itself from being indexed, albeit without its attendant content. And, if memory serves me well, any on-page links will still be evaluated unless CONTENT="NOFOLLOW" is included.

If it is desired that an entire page be ignored, exclude it via robots.txt.

...and what other fellow members have said update sitemap.xml and resubmit it, remove URLs from GWT.

Did you too miss post #4 re. Google's advice to use Fetch as Googlebot instead of submitting a sitemap?

[deepsand]

Yes, content="NoIndex" would hault all indexing ...

Not quite so. See post directly above this one for details.