November 14, 2019
November 14, 2019
Site owners will do anything to get their websites indexed. However, you might not want search engines to crawl through your website if it’s still in development. In a case like this, it’s recommended to discourage search engines from indexing your site. Stick with us if you want to learn more about this topic!
There are some cases where people want to discourage search engines from indexing their sites:
So how do you block search engines from indexing your site? Well, take a look at several options below and try it yourself.
The simplest way to stop search engines from indexing your website is by preventing them from crawling it. To do it, you need to edit your website directory’s robots.txt file. Here are a few ways to achieve that:
Editing WordPress robots.txt is quite easy as you only need to use a WordPress built-in feature. Here’s how:
If you prefer the manual option, you can use File Manager or an FTP client to edit the robots.txt file.
In this article, we’ll show you how to do it through the hPanel’s File Manager:
User-agent: * Disallow: /
The code above will prevent search engines from indexing your whole site. If you want to apply the disallow rule to a specific page, write the page’s subdirectory and slug. For example: Disallow /blog/food-review-2019.
The syntaxes in robots.txt files are case sensitive, so be careful when editing.
Search engines and web crawlers don’t have access to password-protected files. Here are a few methods to password protect your WordPress site:
If you are a Hostinger client, you can password protect your website using hPanel’s Password Protect Directories tool:
If your root directory is public_html, leave the directory column blank
The process in cPanel is also quite similar:
There are tons of plugins that can help to password protect your site. However, the Password Protected plugin might just be the best one out there. It’s been tested with the new WordPress update, and it’s pretty straightforward to use.
After installing and activating the plugin, head to Settings -> Password Protected and configure the settings to match your needs.
Don’t worry if Google has indexed your site. You can remove it from SERPs by following these steps:
And that’s it! Google will temporarily remove your site from search results. Make sure to apply the previous methods to prevent Google from indexing your site again.
There you have it! Quick and easy ways to discourage search engines from indexing your sites. Here’s a quick recap of the methods we’ve learned today:
If you have any other methods, or if you have any questions, please do let us know in the comments. Good luck!