problem

The WordPress blog does not appear in search engines like Google.

cause

If your WordPress blog does not appear in Google, this can have various causes.

• Your blog is your position in the Google index, but are too low.
• You have to function in WordPress “I would like to block search engines, but allow normal visitors.”
• Your blog is blocked by your robots.txt for search engines.

Crawlers or robots of the search engines are about the robots.txt file, asked not to index certain pages or directories.

Using the robots.txt file

The robots.txt file is designed primarily for crawlers of search engines to convey the wish to refrain from accessing certain pages or directories of a web or perform. In addition, there are some specific instructions, eg to delay access by crawlers of search engines.

It is not mandatory to use a robots.txt file. Many spider search for it but have no problem if it does not exist. No means an existing robots.txt, there are no restrictions on the acquisition of the contents of the website. Without exception, all contents must be recorded.

Robots.txt not available by calling an HTTP status code is returned 404, which is recorded in the log files. Web statistics in the 404 HTTP status code is commonly referred to as an error message. The error messages are no problem because it is a mistake. The 404 message is to appear as soon as the robots.txt file is no longer available.