Crawled, Currently Not Indexed: Technical or Quality?

Every technical SEO has opened the Page Indexing report, found a few thousand URLs sitting under "Crawled, currently not indexed", and gone hunting for the bug. We've done it too. You check robots.txt, you check the canonical, you check the response code, everything looks fine, and the pages still aren't indexed.

Here's the thing that took us a while to accept: a lot of the time there is no bug. Google is telling you something about the pages, not about your server. But some of the time there really is a technical cause, and you can't tell which situation you're in by staring at the report. So this is the triage order we use: rule out the mechanical causes fast, and only then have the harder conversation about quality.

What the status actually means

Start with Google's own definition, because half the confusion comes from skipping it:

The page was crawled by Google but not indexed. It may or may not be indexed in the future; no need to resubmit this URL for crawling.

Two things worth pulling out of that. Googlebot got the page, so this is not a fetch failure. And Google explicitly says not to resubmit, which is exactly what most people do first.

It's easy to confuse this with its sibling status, "Discovered, currently not indexed", which Google defines as:

The page was found by Google, but not crawled yet. Typically, Google wanted to crawl the URL but this was expected to overload the site; therefore Google rescheduled the crawl.

Those are different problems with different fixes. Discovered means Googlebot hasn't spent the fetch yet, which points at crawl capacity, server response times, or discovery. Crawled means it spent the fetch, looked at the page, and passed on it. Split those two buckets before you do anything else.

Google's own advice is to stop looking for the bug

On the July 2026 episode of Search Off the Record, Martin Splitt asked John Mueller whether "crawled, currently not indexed" is often or only sometimes a sign of a quality issue. Mueller's answer was "Sometimes", and then he explained the mechanism:

So it's definitely the case if our systems are seriously worried about the quality of a website, that they will reduce the number of pages that they index.

He went further, and this is the part worth sitting with:

It's not that you need to fix this technical issue that Google is not indexing this page at the moment.

Instead he suggested that when you see a broad pattern with no technical explanation, you step back and assess the site's overall quality. He was careful about AI content specifically, saying "That's not to say that all AI-generated content is bad", but describing the failure mode as pages where you think "Anyone could have written this. This tells me nothing."

That's useful guidance, but on its own it's hard to act on. Nobody wants to conclude "it's a quality problem" while a real redirect loop sits undiagnosed. So earn the right to that conclusion by clearing the technical list first.

The technical causes, in the order we check them

1. Confirm the page is genuinely servable

Fetch the URL the way Googlebot would, not the way your browser does. Look for a 200 that's actually a 200, not a soft 404 dressed up as a success page. Check that the response isn't gated behind a cookie, a geo redirect, or a bot rule that fires only for non-browser user agents. Firewalls and bot mitigation are an underrated cause here, because they fail silently and only for crawlers.

Also check the directive layer. A noindex in the HTML head or in an X-Robots-Tag header will produce a page Google crawls and then declines to index, and you will hunt for weeks if you only ever view source in a logged-in browser session. Our robots.txt tester is a quick way to confirm the crawl rules do what you think they do.

2. Check whether the page got clustered as a duplicate

This is the cause people miss most often. If Google decided your page is a near-duplicate of another URL, it may show up as not indexed rather than under an explicit duplicate status. Thin template pages, filtered listing URLs, and location or variant pages that differ by a few words are the usual suspects.

Google updated its canonicalization troubleshooting guide on 10 July 2026 with clearer expectations on timing, and the headline number is worth knowing:

Even after fixing content issues, Google might hold pages in a duplicate cluster for up to two weeks.

So if you differentiated the content last Tuesday and nothing has moved, that's normal. Google also notes that pages split out of a cluster faster when the difference between them is clear and significant. A rewritten intro paragraph usually isn't enough. If the only thing separating two pages is a city name, they're going to keep getting clustered no matter how many canonicals you add.

3. Verify the page renders

If your content arrives via client-side JavaScript, the version Googlebot indexes may be close to empty. A page that renders beautifully for you and returns a bare shell to a crawler is a classic route into this bucket, and it looks exactly like a quality problem from the report's point of view. Compare the raw HTML against the rendered DOM and see whether the main content, the internal links, and the canonical tag all survive. We wrote up the full diagnostic path in our guide to JavaScript SEO and rendering.

4. Look at how the page is linked

Orphan pages and pages buried eight clicks deep get crawled occasionally, then deprioritised. If a URL only exists in your sitemap and nothing on the site links to it, you've told Google the page exists but not that it matters. Sitemap inclusion is a weak canonical and discovery signal, and it doesn't substitute for internal links. Pulling those pages up in the architecture is often the single highest-leverage fix, and it's cheap. Our internal linking guide covers how we prioritise which pages to pull up.

5. Check whether you're simply asking for too much

If you've published 400,000 URLs and Google indexes 30,000 of them, the question isn't why those specific pages missed out. It's whether the other 370,000 should exist at all. Faceted URLs, session parameters, and paginated archives quietly inflate the crawlable surface and dilute everything else. Server logs will tell you where Googlebot is actually spending its time, which is usually not where you'd hope. Both our log file analysis guide and our crawl budget guide go deeper on this.

When the list comes back clean

If the pages are servable, unique, rendering, linked, and within a sane crawl footprint, you're in the situation Mueller described. At that point the honest read is that Google looked at these pages and didn't think they added anything.

That's not a satisfying answer, and it doesn't come with a checklist. But it does come with a direction. Look at a random sample of twenty affected URLs and ask what a reader gets from each one that they couldn't get from the ten other pages on the internet covering the same thing. If the answer is "nothing", consolidating those pages into fewer, better ones usually beats trying to get all of them indexed. Fewer strong pages is a real strategy, not a retreat.

One more thing worth saying plainly: this status is not a penalty. Nothing has been applied to your site. Google is allocating attention, and attention follows demonstrated value.

How we approach it in CrawlX

The reason we built issue triage the way we did is that this exact workflow is tedious by hand. A crawl surfaces the mechanical causes in one pass: soft 404s, noindex directives, redirect chains, canonical conflicts, orphan pages, thin or duplicated content clusters, and pages whose rendered output differs from their raw HTML. Then issues get ranked by how many URLs they affect, so you're working the cause that explains three thousand pages rather than the one that explains four.

If the crawl comes back clean on all of it, that's genuinely useful information. It means you've ruled out the technical explanations and you can stop looking for a bug that isn't there. You can see how we rank and group findings on our issue triage page.

Quick answers

Should I request indexing for these URLs? No. Google's documentation explicitly says there's no need to resubmit. If you fixed a duplicate-content problem, requesting indexing can prompt a re-evaluation, but it won't help a page that was passed over on merit.

How long should I wait after a fix? For duplicate clustering, Google says up to two weeks. For other fixes, give it a full crawl cycle for that section of the site, which your server logs can tell you more accurately than any general rule.

Is a big "crawled, currently not indexed" bucket always bad? No. If it's full of parameter URLs, filtered views, and pages you never wanted indexed anyway, Google is doing you a favour. Check what's actually in the bucket before you treat the number as a problem.

Does this mean my site was penalised? No. It's a resource allocation decision, and it reverses when the underlying assessment changes.