If a WordPress page or your whole site is missing from Google, the fastest fix is Settings → Reading and unchecking “Discourage search engines from indexing this site.” That single option, backed by the
blog_publicdatabase value, forces a site-widenoindextag and overrides any per-post SEO plugin setting. If that box is already unchecked, the next place to check is the individual post’s SEO plugin panel, not your robots.txt file, since a robots.txt block actually stops Google from ever seeing a noindex tag at all.
Why it happens: WordPress has two separate places to say “don’t index this”
WordPress controls indexing through a core function called wp_robots_noindex(). It checks one option, blog_public, and if that option is off, it adds noindex to the page’s robots meta tag through the wp_robots filter. That single option is the database value sitting behind the checkbox in Settings → Reading labeled “Discourage search engines from indexing this site.” Nothing exotic is happening under the hood; it’s one option controlling one filter.
The confusion comes from how that setting used to work. Before WordPress 5.3, checking that box made the virtual robots.txt file output Disallow: /, blocking crawlers outright. That approach had a real flaw: a robots.txt block stops a crawler from fetching the page at all, so it also stops the crawler from ever reading a noindex tag placed on that page. WordPress 5.3 switched to the current behavior, outputting a noindex,nofollow meta tag instead, specifically to close that gap.
That history matters because the same conflict still shows up constantly, just from different sources. Google’s own documentation is explicit about it: for a noindex rule to work, the page must not be blocked by robots.txt. If it is blocked, Googlebot never crawls it, never sees the noindex tag, and the page can still appear in search results. Combine a stray robots.txt disallow rule with a page you meant to noindex, and you get the opposite of what you wanted.
Layered on top of WordPress core’s setting is whatever your SEO plugin adds. Rank Math, for example, gives every individual post a Robots Meta control, plus a site-wide default per post type under Titles & Meta. That per-post control only wins if nothing upstream is overriding it. WordPress’s own “discourage search engines” checkbox sits above all of it: if that box is checked, it forces noindex site-wide regardless of what any individual Rank Math toggle says. Most “my noindex setting won’t stick” support threads trace back to exactly this: someone is fighting the wrong control.
Step-by-step fix
Work through these in order, checking each one against the live, rendered page rather than a settings screen, since that’s the only place you can see what’s actually being sent to Google.
1. Rule out the site-wide switch first. Go to Settings → Reading and confirm “Discourage search engines from indexing this site” is unchecked. This is the single most common cause of an entire site vanishing from search results, and it overrides everything else on this list. It’s also the setting most likely to get left on by accident, which the next section covers.
2. Check the post’s own SEO plugin setting. In Rank Math, open the post, switch to the Advanced tab in the metabox (turn on Advanced Mode from Rank Math SEO → Dashboard if you don’t see it), and look at Robots Meta. If it’s set to No Index, that’s a deliberate per-post override, either one you set intentionally or one left behind from an earlier draft state. The admin bar shortcut “Rank Math SEO → Mark this page → As NoIndex” sets the same field, so check there too if you’re not sure how it got set.
3. Check the post-type default. If individual posts look correct but an entire content type is missing, look at Titles & Meta → [Post Type] → Robots Meta in Rank Math. This sets the default for every post of that type and is easy to change once by accident while adjusting an unrelated setting.
4. Check robots.txt separately from noindex. These are two different mechanisms and fixing one doesn’t touch the other. If robots.txt disallows a path, no noindex tag on pages under that path will ever be read by Google, because the crawler stops at the robots.txt block. Load /robots.txt directly and look for a stray Disallow rule covering the page in question.
5. Save and move to verification. Once you’ve corrected whichever layer was wrong, save the change and don’t assume it worked from the dashboard alone.
How to verify it worked
Load the actual published URL in a browser, view source, and search for name="robots". That tag, as rendered on the live page, is the only reliable confirmation; a settings panel saying “Index” doesn’t guarantee the output matches, especially if a caching layer is serving a stale version of the page. This is the same discipline this site applies after any programmatic change made through the WordPress REST API: a successful API response confirms the database was updated, not what a crawler actually receives.
Separately, check the response headers for an X-Robots-Tag value, since some hosting configurations or CDN rules set this independently of the meta tag in the page’s HTML. Both the meta tag and the header are subject to the same robots.txt caveat: if the page is blocked, neither one gets read.
Confirm your XML sitemap doesn’t list any URL you intend to keep out of the index. A noindexed page sitting in your sitemap sends Google a contradictory signal and can slow down how quickly the removal actually takes effect once you fix it. After correcting a stuck noindex tag, expect the change to take days to weeks to show up in Google’s index; Google doesn’t publish a fixed recrawl schedule, so treat any faster claim with skepticism.
What to do when the standard fix fails
If you’ve cleared the site-wide setting, the per-post override, and the post-type default, and the page is still noindexed, the most common remaining cause is a staging-to-production migration that carried the “discourage search engines” setting along with it. Sites are routinely built on staging with that box checked on purpose, to keep the unfinished copy out of search results. When the staging database gets migrated to the live domain, that setting comes with it, and unless someone remembers to uncheck it after launch, the live site stays invisible to Google, sometimes for weeks before anyone notices the traffic never showed up. If your site recently moved from staging, or a migration or restore ran recently, this is the first thing to check, not the last.
Also worth remembering: “discourage search engines” is a request WordPress sends to crawlers, not an enforcement mechanism. Well-behaved crawlers like Googlebot respect it, but it doesn’t block access to the content itself, so it’s not a substitute for authentication or access control if the actual goal is keeping a page private rather than out of search results.
If none of the WordPress-side checks explain it, treat noindex verification as one line item in a broader routine rather than a one-off fix, alongside checks like confirming your canonical URLs point where you expect. Noindex and canonical tags solve different problems and are easy to confuse, but both are worth confirming together. Running through both as part of a standing WordPress SEO checklist, rather than only after Search Console flags a drop, catches this kind of drift before it costs weeks of missing impressions.
Frequently asked questions
Does unchecking “discourage search engines” index my site immediately?
No. Unchecking it removes the noindex signal, but Google still has to recrawl the page and decide to index it, which is not instant and has no published fixed timeframe. It generally takes days to weeks, not hours.
Can a page be noindexed and still show up in Google search results?
Yes, if robots.txt is blocking the page. Google’s documentation is explicit that a noindex tag only works if the crawler can actually reach the page; a robots.txt block prevents that, so the page can keep appearing in results even with a noindex tag sitting unread on it.
Is noindex the same as a canonical tag?
No. A canonical tag tells Google which URL is the authoritative version among duplicates; noindex tells Google not to show a page in results at all. Using one where you meant the other is a common source of confusion covered in more detail in the canonical URL guide.
Why does my SEO plugin’s noindex toggle not seem to work?
Check WordPress’s own Settings → Reading page first. The core “discourage search engines” setting overrides any per-post plugin toggle, so a plugin set to “Index” will still render as noindex if that core box is checked.
Leave a Reply