Sitemap URLs Not Indexed: How to Diagnose and Fix the Problem

Submitting an XML sitemap and then watching those URLs sit permanently in the "Not indexed" column of Google Search Console is one of the more frustrating experiences in technical SEO. The sitemap is supposed to make discovery easier, yet pages that you have published, optimised, and linked internally still fail to appear in search results. The problem is rarely the sitemap file itself. It is almost always a signal — sent elsewhere on the site — that tells Google the page is not worth indexing.

Sitemap URLs not indexed is a diagnostic question, not a content question. The exclusion reasons Google surfaces in Search Console each point to a different fix, and conflating them wastes weeks of effort on the wrong remedy. This article walks through every major exclusion category, explains what Google is actually seeing, and gives you a sequenced plan for resolving each one.

If you're looking for expert help in this area, explore how our technical SEO services can drive measurable results for your business.

Why Google Skips URLs Even When They Are in Your Sitemap

A common misconception is that adding a URL to a sitemap forces Google to index it. Google's own documentation is explicit that sitemaps are hints, not directives. Googlebot decides independently whether a page merits a place in the index, based on signals gathered from crawling, rendering, and cross-referencing with other pages on the web.

This means a URL can appear in your sitemap and still be excluded for any of the following reasons:

• A conflicting signal elsewhere on the site tells Google not to index it.
• Google has crawled it and judged the content low-quality or duplicate.
• The page has never been crawled because crawl budget was spent on other URLs.
• A technical error (redirect chain, server error, soft 404) prevents Googlebot from accessing it cleanly.

Understanding which of these applies to your URLs is the entire diagnostic exercise. Guessing and making changes in parallel only obscures what actually caused the fix.

Reading the Coverage and Page Indexing Reports Correctly

Google Search Console's Page Indexing report (formerly the Coverage report) is the primary diagnostic tool. Navigate to Indexing > Pages in the left-hand menu. The report divides URLs into four states: Error, Valid with warnings, Valid, and Excluded. The Excluded tab is where most sitemap investigations begin.

Exclusion reasons that signal a misconfiguration you control

• Excluded by 'noindex' tag — A noindex directive exists on the page itself, in an HTTP header, or in a meta robots tag. Google is honouring your instruction. The fix is to remove the noindex if the page should be indexed, or remove the URL from the sitemap if it genuinely should not be.
• Blocked by robots.txt — Googlebot cannot crawl the page because your robots.txt file disallows the path. Note that Google can still index a URL it cannot crawl if it discovers the URL elsewhere; it will just index it without content. Remove the disallow rule, or remove the URL from the sitemap.
• Redirect — The sitemap URL redirects to another destination. Sitemaps should only contain canonical, final-destination URLs. Update the sitemap entry to the redirect's target URL.
• Not found (404) — The page returns a 404 response. Either restore the page, create a proper redirect, or remove the URL from the sitemap.
• Soft 404 — The server returns a 200 status but the content signals "no meaningful content here" to Google (empty search results pages, thin placeholder pages). Fix the content or remove the URL.

Exclusion reasons that reflect Google's editorial judgement

• Duplicate without user-selected canonical — Google has found near-identical content elsewhere on your site and chosen a different URL as the canonical version. You have not specified a canonical tag, so Google made its own decision.
• Alternate page with proper canonical tag — You have set a canonical pointing to a different URL, and Google is respecting it. This is expected and correct behaviour if intentional.
• Crawled — currently not indexed — Google crawled the page but decided not to index it. This is the most ambiguous state and typically points to thin content, near-duplicate content, or low perceived authority.
• Discovered — currently not indexed — Google knows the URL exists but has not yet crawled it. On large sites this is a crawl budget signal.

Canonical Mismatches: The Silent Killer of Sitemap Submissions

Canonical conflicts account for a disproportionate share of sitemap URL indexing failures on e-commerce and CMS-heavy sites, and they are frequently overlooked because they require cross-referencing multiple signals simultaneously.

The scenario plays out like this: your sitemap includes https://example.com/products/blue-widget/, but the page's canonical tag points to https://example.com/products/blue-widget (without the trailing slash). Google treats these as different URLs. It follows the canonical, indexes the version the canonical points to, and marks the sitemap URL as "Alternate page with proper canonical tag" or "Duplicate without user-selected canonical."

To audit this systematically, crawl your site with a tool such as Screaming Frog SEO Spider and compare the canonical tag on each page against the URL listed in your sitemap. Any mismatch means the sitemap entry is sending a contradictory signal. The sitemap should always reflect the canonical URL — not the URL that happens to be in your CMS database.

Paginated and filtered URLs

Sites that generate URLs through filtering (faceted navigation, sorting parameters, session IDs) commonly populate sitemaps with non-canonical variants. A product listing page with ?sort=price-asc appended is almost never the page you want indexed. Audit your sitemap generation logic to ensure it pulls only from your defined canonical set, not from raw URL logs or crawl outputs.

Crawl Budget and the "Discovered — Currently Not Indexed" State

On sites with thousands or tens of thousands of URLs, a significant proportion of sitemap URLs may sit in the "Discovered — currently not indexed" state for weeks or months. This is a crawl budget problem, not a content problem, and treating it as a content problem produces no results.

Crawl budget — the number of URLs Googlebot crawls within a given period — is influenced by two factors: crawl capacity (how many requests Google's infrastructure allows before slowing down to avoid overloading your server) and crawl demand (how much Google wants to revisit or discover URLs based on their perceived value and rate of change). Google's crawl budget guidance specifically notes that sites with a large number of low-quality or duplicate pages waste budget that could go to high-value content.

The practical implication: if you have 50,000 URLs in your sitemap and a meaningful proportion return thin content, near-duplicates, or auto-generated pages with minimal differentiation, Googlebot will deprioritise the entire domain. Your genuinely valuable pages suffer because the low-quality ones dilute the signal.

How to reclaim crawl budget

• Audit your sitemap for URLs that serve no unique search purpose and either noindex them (with care) or consolidate them into canonical versions.
• Ensure your robots.txt correctly blocks sections you never want crawled (admin areas, internal search results, duplicate parameter variations).
• Check your server logs or Google Search Console's crawl stats report to see which URL types Googlebot is spending the most time on — and whether those match your priorities.
• Submit a sitemap that reflects only your canonical, indexable, high-quality URLs. A sitemap of 500 strong pages is more valuable than a sitemap of 10,000 mixed-quality ones.

"Crawled — Currently Not Indexed": When the Problem Is the Page Itself

This state is the one that most site owners find hardest to accept because the surface reading — Google crawled it and still said no — feels like a rejection with no appeal process. In practice, it is Google's way of saying: "I found this page, I could read it, and it does not add enough to my index to be worth including."

This typically affects:

• Thin or templated pages where the majority of content is boilerplate (standard terms and conditions, near-identical product variants that differ only by colour).
• Pages that duplicate content from a stronger, more authoritative source — either elsewhere on your own site or on a third-party domain.
• Pages with very low engagement signals inferred from historical crawl data.
• New pages on low-authority domains where Google has not yet built enough trust to include speculative content.

The remedy is editorial, not technical. Improve the content substantially, ensure each page answers a distinct query that no other page on your site addresses, and build internal links that signal to Googlebot where editorial effort is concentrated. A page that earns backlinks from external sources is also far more likely to be indexed than one that exists only in your sitemap.

What to Do This Week

Diagnosing sitemap exclusions is iterative and requires working through each category methodically rather than applying a single fix to all excluded URLs.

Day one: export and categorise

In Google Search Console, go to Indexing > Pages > Not indexed. Export all exclusion reasons as a CSV. Open the file and group URLs by exclusion reason. This single step tells you whether your problem is structural (misconfigurations you can fix today) or editorial (quality signals that require more sustained effort). Do not attempt to fix all categories simultaneously — you will lose track of what caused which improvement.

Fix misconfigurations first

Prioritise in this order: noindex removals on pages that should be indexed, robots.txt unblocking, redirect corrections in your sitemap, and 404 restorations. These are binary — the signal is wrong and you can make it right. After fixing, use the URL Inspection tool in Search Console to request re-indexing for a sample of corrected URLs, then monitor the Page Indexing report over the following two to three weeks.

Audit your sitemap for non-indexable URLs

Run Screaming Frog with the "Crawl Linked XML Sitemaps" setting enabled, then apply the "Non-Indexable URLs in Sitemap" filter under the Sitemaps tab. Export this list. Every URL on it is sending a contradictory signal: you are telling Google to index a page while simultaneously instructing it not to. Resolve each entry — either fix the indexability or remove the URL from the sitemap.

Address crawl budget if you have a large site

If more than 20% of your submitted sitemap URLs are in the "Discovered — currently not indexed" state, treat that as a crawl budget signal. Review your sitemap to remove low-value URL types, check your crawl stats in Search Console for unexpected URL patterns consuming budget, and consolidate thin pages where possible. A leaner, higher-quality sitemap consistently outperforms a comprehensive but indiscriminate one.

Tackle "Crawled — currently not indexed" pages editorially

For pages Google has visited and declined to index, audit the content against competing pages that are indexed. If the page does not clearly answer a query better than any other page — on your site or elsewhere — consider whether it should exist as a standalone URL at all. Consolidation, noindexing thin variants, and redirecting near-duplicates to a stronger canonical are often more effective than attempting to pad word counts on weak pages.

FAQ

How long does it take for Google to index a URL after I submit my sitemap?

There is no fixed timeline. New URLs on established, high-authority domains are sometimes crawled within hours of sitemap submission. URLs on newer or lower-authority sites may wait days or weeks. The "Discovered — currently not indexed" state in Search Console confirms Google knows about the URL; the wait is determined by crawl priority, not by whether the sitemap was accepted.

Should I remove non-indexable URLs from my sitemap even if they have canonical tags?

Yes. A URL with a canonical pointing to a different page is, by definition, not the version you want indexed. Including it in your sitemap alongside its canonical counterpart creates a contradiction that Googlebot must resolve. Sitemaps should contain only canonical, indexable URLs — removing non-canonical variants reduces noise and keeps your sitemap a clean signal.

Can a sitemap be too large?

A single sitemap file supports up to 50,000 URLs and must not exceed 50MB uncompressed, per Google's sitemap specification. For larger sites, a sitemap index file references multiple individual sitemaps. The real risk is not file size but content quality — a sitemap containing large numbers of low-value URLs signals poor site quality regardless of how many URLs it lists.

Does requesting indexing via the URL Inspection tool speed things up?

It can accelerate crawling of individual URLs, particularly for time-sensitive content. However, it is not a substitute for fixing the underlying issue. If a URL is excluded because of a noindex tag or a canonical mismatch, requesting indexing through the URL Inspection tool will not override those signals — Googlebot will crawl it, encounter the same instruction, and leave it excluded.

Related Reading

• What Are Common SEO Mistakes?
• Why Does SEO Still Matter in 2026?
• What Is Topical Authority and How Do You Build It?
• How AI Search Engines Decide What to Cite
• Entity SEO: How to Build Knowledge Graph Authority