How to prevent Google from indexing a site?

Question

My client asked me to prevent google bot from indexing the website, so I have added the following meta to the head tag of my main layer which (in theory) is the head of all pages:

<meta name="googlebot" content="noindex">

This should prevent Google from indexing any pages, however, it is not preventing it. Apparently my client observed somehow that Google bots are still indexing the site (I do not know how he knows that Google is still indexing the site, because, as usual, clients are not too descriptive) so it seems that this is not solving the problem.

In fact I wanted to prevent google from indexing the site by adding a meta in the header of my template used at all the pages. Why is this still not enough for google to prevent it from indexing the site? How should I fix the issue?

Thank you very much.

Answer 1

TL;DR: Google might accidentally ignore the meta tag that you added to the website. If you recently added the meta tag it will follow it when the bot recrawls the website. If you want to get rid of all search bots, and not just google use <meta name="robots" content="noindex">

Here is the official answer from Google -

"Note that because we have to crawl your page in order to see the noindex meta tag, there's a small chance that Googlebot won't see and respect the noindex meta tag. If your page is still appearing in results, it's probably because we haven't crawled your site since you added the tag. (Also, if you've used your robots.txt file to block this page, we won't be able to see the tag either.)

If the content is currently in our index, we will remove it after the next time we crawl it. To expedite removal, use the URL removal request tool in Google Webmaster Tools."

Also, please note that your client may be confusing the Google bot with another webcrawler bot. In that case, I recommend adding:

<meta name="robots" content="noindex"> 

To the HTML document so that no webcrawlers can index the site, not just googlebot.

Answer 2

I've always done <meta name="robots" content="noindex">.

You can see https://support.google.com/webmasters/answer/93710?hl=en and https://support.google.com/webmasters/answer/156449?hl=en&ref_topic=2370588 for a little more info.

The reason your tags might not work is if you have a robots.txt file blocking Google from re-indexing the site.