Discourage Search Engines Left On in WordPress: How to Fix It

What does the WordPress Discourage Search Engines From Indexing This Site setting do?

It is a checkbox on the Reading screen under Settings in WordPress that asks search engines not to index your site. When it is on, WordPress adds a noindex and nofollow directive to your pages. Also, it never locks visitors out. It only asks bots to stay away, so a site launched with it on will not appear in Google.

The label above is the English interface text from the official WordPress documentation. We could not verify the exact Turkish or German labels, so look for a similar phrase on your own screen. Also, menu names can vary by version and language.

The discourage search engines setting is useful during development. In practice, the trouble starts when you forget to turn it off at launch. This article covers only that one situation: what the setting does, how it shows up, how to switch it off, and how to protect a staging copy properly.

What exactly happens while the setting is on?

According to the WordPress.org documentation, several things happen at once. First, WordPress adds a robots directive that asks bots to ignore the site. Second, it stops sending pings to update services. Third, normal visitors still see the site as usual.

One note in the documentation matters a lot: the setting does not block access to the site. Therefore, it is up to search engines to honor the request. In other words, the setting is a sign on the door, not a lock.

That difference is critical for staging sites, because they hold private work. You should never rely on this checkbox alone to hide a private copy. We explain the right method further down, so keep reading.

Ping behavior is a handy clue, too. With the setting on, WordPress hides the update services field on the Writing screen and shows a message about your privacy settings. If that field is missing on your site, the discourage setting may be on.

Why does the behavior differ between WordPress versions?

The official documentation separates two eras. Up to version 5.2, the setting made robots.txt requests return a rule that disallowed all bots. From version 5.3 onward, it adds a robots meta directive with noindex and nofollow to the page source.

The documentation also says the older behavior only worked when WordPress sits in the site root and no physical robots.txt file exists. So if you inherit an older site, check the robots.txt file as well as the page source.

However, details may have changed in the current version. For exact information, read the WordPress.org Reading Settings documentation. So we only repeat the behavior described there.

In practice, this means you search for the problem in different places. For example, on an old install you look at robots.txt. On a modern one, you look at the page source. Also remember that an SEO plugin may generate robots.txt for you, so you would manage that file from the plugin settings.

Why is the setting so often left on at launch?

This mistake is more common than you might think, because it is a side effect of a natural workflow. Teams build the site in a development environment, switch the option on there, and move everything to the live server when the work is done. The database travels with the files, so the setting travels too.

Here is an example scenario. For instance, an agency prepares a client's new site on a development copy. Then, on launch day, it moves the files and the database to the live server. Meanwhile, everyone checks the design, the speed, and the forms. Nobody opens the Reading screen.

A few other causes show up often:

• The launch checklist has no line for search engine visibility.
• Someone copies the setting while moving an old site to a fresh install.
• A maintenance mode plugin or a hosting panel turns the option on automatically.
• Different people manage staging and live, so nobody owns the check.

Can restoring a backup or migrating turn the setting back on?

Yes, it can. The discourage search engines setting lives in the database. So if you restore a backup taken from staging onto the live site, the old value comes back with it. As a result, you may lose a fix you made earlier.

The same applies to rolling back, too. Suppose a plugin update breaks the site and you restore a backup from three days ago. If that backup dates from a period when the setting was on, then the problem returns quietly.

Therefore, repeat the same short check after every major operation:

• After a migration or a clone.
• After restoring from a backup.
• After changing hosting providers.
• Whenever a developer rebuilds the site.

What symptoms show up when the setting stays on?

The clearest symptom is that your site never appears in Google. For example, when you search your brand name, you find only your social profiles or mentions on other sites. New posts stay invisible for days, even weeks.

You will also notice other signs:

• Organic traffic sits near zero, and only direct and paid visits remain.
• The page source shows noindex and nofollow as the robots directive.
• Search Console lists pages as excluded.
• Pages that used to rank slowly drop out of results.

One more detail deserves attention. The discourage search engines setting affects pages that were already indexed, not just new ones. Because Google revisits them, it sees the directive and may drop them. So even an established site can lose traffic if someone switches the option on.

Still, none of these signs is proof on its own. A brand new site can also take time to appear. So checking the setting and the page source directly is the most reliable route.

What do you see in Search Console when Discourage Search Engines is on?

The typical picture is pages sitting outside the index because of a noindex directive in the page indexing report. Report names change over time, so expect a similar wording in your language. We explain the general meaning of that status in our guide to excluded by noindex tag.

Likewise, when you test a page with the URL Inspection tool, you get a similar result saying indexing is not allowed. The result also points to the robots meta directive as the reason.

Here is the key distinction: this is not an error, it is the result of an instruction. Google is following your request. Additionally, the fix lives in WordPress, not in Google.

If you want a wider view of pages missing from the index, read how to find unindexed pages in Search Console.

How do you confirm that the setting is on?

Confirmation has two layers. First you check the setting, and then you check the real output. That order matters because another plugin may add noindex even when the setting looks off.

1. Open the Reading screen under Settings in the WordPress admin.
2. Look at the search engine visibility section and see whether the box is checked.
3. Open your home page in a browser and view the page source.
4. Search the source for the word robots and see whether noindex appears.
5. Test the same address with the URL Inspection tool in Search Console.

Steps three and four are important because caches can mislead you. If a caching plugin serves an old copy, the source you see may differ from reality. Then look again in a private window and after clearing the cache.

Check more than one page, too. Moreover, the home page may look clean while a post template outputs noindex. For example, pick a home page, a post, a category page, and a service page. That way you learn whether the problem covers the whole site or one template.

What should you do first after turning the setting off?

Uncheck the discourage search engines box and save your changes. Then clear your caches, because cached pages may keep serving the noindex directive. If you skip this step, Google can therefore still see the wrong version.

The next steps look like this:

1. Turn the setting off and save.
2. Clear the page cache, any CDN cache, and your browser cache.
3. Confirm that the home page source no longer contains noindex.
4. Check that robots.txt has no rule that blocks the whole site.
5. Submit your sitemap in Search Console.
6. Request recrawling for key pages with the URL Inspection tool.

Do not change this order. A sitemap or a crawl request sent before the fix is wasted. Also, it can even make Google look at the page in the wrong state.

When and how should you submit the sitemap?

Submit it after you confirm that the directive is gone, because an early submission achieves nothing. Google's recrawling documentation recommends a sitemap when you have many URLs. It calls this especially valuable after launching a new site or a migration.

The built-in WordPress sitemap or the one from your SEO plugin will do the job. Also, make sure the map lists only the addresses you want indexed. If you need to build a new one, try our XML sitemap generator.

After submitting, then, watch the status in Search Console. A successful read means that the map is reachable. However, it does not mean every address will be indexed. Google decides which pages to add, and the sitemap is only a discovery hint.

If you see a fetch warning, read our guide to the Search Console sitemap couldn't fetch error. For a refresher on the basics, see what an XML sitemap is.

How do you request a recrawl with URL Inspection?

Paste the address of an important page into the inspection box in Search Console, wait for the result, and request indexing. Google's documentation says you need owner or full user access to the property. If you lack it, solve the access problem first.

A few limits apply:

• There is a quota for submitting individual URLs, so check the current value in the official help page.
• Requesting a recrawl several times for the same URL does not speed it up.
• A crawl request does not guarantee that the page will appear in results.

So pick your priority pages, such as the home page, the main service pages, and your best articles. Instead of submitting the whole site one by one, trust the sitemap. That is enough for the rest. In practice, you can read the details in the Google recrawl documentation.

How long until results come back after the fix?

We cannot give an exact time, because there is none. Google's documentation says crawling can take anywhere from a few days to a few weeks. For pages dropped because of noindex, a revisit may take months depending on the importance of the page.

Your site size, authority, and crawl frequency set the pace. For example, a home page you update often returns quickly. An old article buried deep may return later.

While you wait, you can do the following:

• Watch the page indexing report every week.
• Check whether the number of excluded pages is falling.
• Strengthen internal links so bots reach key pages easily.
• Stay patient and avoid repeated requests for the same URL.

However, a longer wait is not a reason to panic. First confirm that the setting is off, that no noindex remains in the source, and that robots.txt does not block anything. Once those three hold, you only need to wait for Google to return. If crawling seems slow for other reasons, read why Googlebot crawls less.

Why does noindex remain after you turn Discourage Search Engines off?

If you switch the setting off and still see noindex, the source lies elsewhere. WordPress has several layers that can output a robots directive, and they work independently.

The most common causes are these:

• The SEO plugin has noindex on for the whole site or for some content types.
• A single page or post still carries a page level noindex flag.
• The theme or a maintenance mode plugin adds its own robots directive.
• A cache or CDN layer keeps serving the old page.
• The server response headers contain a robots directive.

Testing each layer in turn takes time, but it is therefore the safest way. Check the page source first, then the response headers. After fixing any layer, also clear the cache again.

To separate the cache layer, open the page with a query parameter or in a private window and compare the results. If they differ, then the problem sits in the cache, not in the setting.

How does the discourage setting clash with SEO plugins and themes?

Clashes usually happen because two settings do the same job from different places. For example, you turn the WordPress setting off, but your SEO plugin keeps the whole site on noindex. Or the reverse: you fix the plugin, but the WordPress box stays checked.

The table below shows where to look for each source:

Plugin menu names change between versions. Please consult the plugin's own help documentation for the exact path.

What is the difference between this setting and a page level noindex?

The discourage search engines setting is a single switch for the whole site. A page level noindex excludes only the post or page you choose. Additionally, both send a similar instruction to search engines, but their scope is very different.

So choose by purpose. Use the page level method for single addresses such as thank-you pages, internal search results, or test pages. Use the global setting together with a password when the whole site must stay hidden during development.

Our guide on noindex, nofollow, and robots.txt compares them in depth. Here we only stress why this setting is risky: one checkbox affects hundreds of pages at once.

Why does a blocked robots.txt make the problem twice as bad?

This distinction is easy to miss, and it matters. Google's documentation says that for a noindex rule to work, the page must not be blocked by robots.txt and must be accessible to the crawler. Googlebot cannot visit a blocked page, so it cannot see the noindex either.

In practice, even after you switch the setting off, a robots.txt that blocks everything keeps Google from learning the new state. The pages are not crawled, so their index records may stay as they are or never update.

To check your file, read what robots.txt is and how to create it. If you need to prepare a new one, use our robots.txt generator. In practice, a line that blocks the entire site has no place on a live site.

How do you protect a staging site properly?

For a copy that must stay private, the only correct method is restricting access. Password protection is the strongest choice because nobody gets through without it. Bots and visitors cannot log in, so the copy is neither crawled nor indexed. Moreover, the WordPress discourage setting remains a polite request.

Our recommended approach looks like this:

• Protect the staging site with a password at the server level, together with your hosting provider.
• Keep discourage search engines on as an extra layer, but never rely on it alone.
• Also, do not share the staging address in public places.
• Decide in advance which settings you will copy when you move to live.

However, how you set up the password depends on your hosting setup. We do not run servers, so follow your provider's official documentation for that step.

Separating the copy this way has another benefit. Clients and teammates review the draft safely, and unfinished content never appears in search results. You can also add a visible label to the staging admin bar to reduce the risk of mixing up the two environments.

Which protection method is enough in which case?

Each method has strengths and weaknesses, so compare them. The comparison below shows which one fits which situation. For example, a development copy needs a password, while a single thank-you page on the live site only needs a page level noindex.

Google also lists restricting access as one of the ways to keep content out of results. You can read the details in the Google block indexing documentation.

What does a fix look like in an example scenario?

Here is an example scenario. A small service company launches a new WordPress site. Two weeks later, however, the owner searches the brand name and finds nothing. Instead of panicking, the owner works through a sequence.

First, the owner opens the Reading screen, sees the checked box, and clears it. Next, the owner clears the cache and confirms that the home page source no longer contains noindex. Then the owner opens robots.txt and finds no blanket block.

Finally, the owner submits the sitemap in Search Console and requests inspection for the home page and three service pages. Over the following weeks, the owner also watches the page indexing report. Also, this is only an example scenario. Real timing and results vary by site, and no step guarantees an outcome.

Notice that there is no magic move here. In practice, the process consists of calm, ordered, verifiable steps.

Which pre-launch checklist should you use?

The permanent fix for a forgotten discourage search engines box is to turn the check into a habit. First, add one line to your launch list and give it to one person. That way you remove the "someone probably checked" assumption.

Here is a simple order for launch day:

1. Confirm the discourage setting is off on the Reading screen.
2. View the source of the home page and one inner page, and confirm there is no noindex.
3. Check the site wide visibility setting in your SEO plugin.
4. Confirm robots.txt has no rule that blocks the whole site.
5. Submit the sitemap in Search Console.
6. Look at the page indexing report one week after launch.

For a broader list, use our website launch checklist. If you are also changing domains or design, the website migration SEO checklist helps too.

Also, keep the list in writing and use it in the same order at every launch. That way knowledge survives team changes. For monitoring after launch, see website metrics to track after launch.

What should you watch for with Search Console access and data?

After the fix, data may not appear in Search Console right away. If you added a new property, for example, processing takes time. We covered that case in Search Console shows no data. So wait calmly and make sure the setting is right.

Access is another matter. If a former agency or employee still has owner access, that is both a security and a management problem. Additionally, you need to know who can request recrawls. For details, see how to remove an old owner or user from Search Console.

For general use, our guide to what Google Search Console is is a good starting point.

Will rankings and traffic return after the fix?

We cannot guarantee that. Once your pages return to the index, they can rank again; however, nothing is certain. However, where and how fast they appear depends on content quality, competition, and your site's overall condition. No method restores old rankings with certainty.

With discourage search engines off, the realistic expectation is this: pages that ranked before get re-evaluated once the block is gone. A brand new site simply starts the normal process.

Here is what you can focus on after the fix:

• Track the number of indexed pages.
• Strengthen key pages with internal links.
• Review the site's technical health with the topics in what technical SEO is.
• Investigate the cause if you see an unexpected drop.

Which common mistakes should you avoid?

In our experience, the typical mistakes come mostly from missing process, not missing technical knowledge. Read the list below as a warning.

• Assuming you switched the setting off without checking the source.
• Sending a crawl request before clearing the cache.
• Forgetting a block in robots.txt.
• Protecting staging with only the discourage setting.
• Requesting a recrawl for the same URL again and again.
• Deleting and rebuilding old pages to solve the problem.
• Opening a new domain to get around the rules.

The last one is especially wrong, because it never works. Trying to bypass systems with a new domain does not fix the issue. It only creates confusion, and it goes against the rules. So if the problem is a setting, fixing the setting is enough.

Another frequent mistake is testing only the home page. A clean home page can hide a template level noindex that keeps hundreds of inner pages out. After every fix, therefore, recheck a few different page types at the source level.

When should you get expert help?

Suppose you switched the setting off, cleared the cache, and the pages still do not enter the index. Then the problem lives in another layer. It could be, for example, a plugin clash, a server header, or a canonical error. A technical audit speeds things up in such cases.

At Talha Aslan and team, we run these checks as part of our SEO consulting work. We focus on finding the source and then clarifying the next steps. We do not promise results, because Google makes the final indexing decision.

Before you reach out, gather a few facts: when the setting was switched on, the date of your last migration or backup, your SEO plugin, and the exclusion details from Search Console. Moreover, these records make life easier for everyone.

If you are building a new site, our web design service can also help with pre-launch checks. This article is not hosting advice. For your own infrastructure, follow your provider's documentation.