Short answer: A page that is not in a search index cannot be cited from it, and the usual causes are mundane: an accidental noindex, a sitemap that is missing pages, a canonical that points at a dead URL, or a redirect chain.
Fix the plumbing in that order, check each engine on its own terms, and be careful with 'Crawled, currently not indexed', which can be a quality decision rather than a fault.
Why is my page not indexed?
There are two families of cause. The first is plumbing: something told the engine not to index the page, or sent it to the wrong URL, or never told it the page exists. Those are fixable and mechanical. The second is a judgment by the engine that the page is not worth keeping, which no technical fix addresses.
It pays to separate them early. Reading 'Discovered, currently not indexed' or 'Crawled, currently not indexed' in Search Console as a technical fault is a common mistake. Those statuses can reflect quality or priority decisions, and resubmitting the URL does not change them.
Some of this is deliberate, so ask before fixing
A noindex on a login page, checkout, thank-you page, internal search or staging site is correct. A cross-domain canonical is a legitimate way to consolidate. A single-language site has no hreflang to validate. Our audit already treats a noindex page that your sitemap does not list as deliberate. Do not undo a choice you made on purpose.
What are the common plumbing causes, in order of damage?
Work from the worst to the least. The thresholds in the right-hand column are the ones our own audit uses, so you know what 'fails' means when a report says so.
| Problem | Why it hurts | Audit fails when |
|---|---|---|
| HTTP not redirecting to HTTPS, or a redirect loop | A loop makes a page unreachable; plain HTTP splits signals | The site was not reached over HTTPS, or a redirect chain answers twice from the same URL |
| Accidental noindex | The page is told not to be in the index | noindex (meta tag or X-Robots-Tag header) on the home page, on a page the sitemap lists, or on any page when there is no sitemap |
| Broken pages and bad canonicals | Signals point at URLs that return errors | Any crawled URL returns 4xx or 5xx, or a canonical is relative, malformed or points at a 4xx/5xx URL |
| Sitemap missing pages | Engines are not told the pages exist | Under 90% of crawled indexable pages are listed (a page counts as covered if its canonical is listed) |
| Redirect chains | Extra hops waste crawl effort and dilute signals | A chain has more than 2 hops |
| Invalid hreflang | Language targeting is ignored | A value is not a valid language tag or x-default, or a same-host target was not reached |
Accidental noindex is worth special attention because it is invisible in the browser. A framework default or a staging setting that shipped to production is the usual cause. A staging X-Robots-Tag set in a shared CDN rule that also covers production is a classic.
How should my sitemap work?
Generate it from the same content list that builds the site, so it cannot drift. List only canonical, indexable pages that return 200, and reference it from robots.txt. A sitemap that lists redirected, noindex or 404 URLs tells the engine two things at once.
Set lastmod from real content edits, not from the build time. Google ignores priority and changefreq, and uses lastmod only if it is consistently accurate. A build-time lastmod on every entry teaches crawlers to ignore the field.
A single sitemap file can hold up to 50,000 URLs or 50MB uncompressed. Beyond that, use a sitemap index. Google also says submitting a sitemap is only a hint, with no guarantee it will be downloaded or used. (Google Search Central)
If your sitemap is missing pages, fix the generator, not the file. A hand-written sitemap or one built from a stale list will drift again next month.
How do I tell Google and Bing about my pages?
The channels differ by engine, and using the wrong one is a common time sink.
| Engine | What to use | What not to expect |
|---|---|---|
| Submit the sitemap in Search Console and keep it in robots.txt. Use Request indexing in the Search Console interface for single URLs. | Sitemap submission is only a hint. Google does not take IndexNow. The sitemap ping endpoint was announced as going away in June 2023. The Indexing API is only for pages with JobPosting or BroadcastEvent markup. | |
| Bing | Verify the site in Bing Webmaster Tools, submit the sitemap, and use IndexNow for changes. | IndexNow says a URL changed. It does not guarantee indexing. |
| Yandex, Naver, Seznam, Yep, Amazon | IndexNow | A single submission is shared with the participating engines. |
IndexNow works by hosting a key file you control and sending the changed URLs to one endpoint. Only submit URLs that actually changed, and submit deletions too. Submitting the same URL to every channel repeatedly does nothing extra and can hit quotas.
How do I find and fix what is keeping pages out?
Get definitive status from each engine
For Google, use the URL Inspection tool in Search Console, and compare the canonical Google chose against the one you declared. A mismatch is the finding. For Bing, use Bing Webmaster Tools.
Fix HTTPS and redirect loops first
A loop makes the page unreachable. Redirect all HTTP to HTTPS at the host or CDN.
Remove accidental noindex
Ask of each page whether it is meant to be in search. If yes, remove the noindex. Do not combine a robots.txt Disallow with the hope of deindexing: the crawler never fetches the page, so it never sees the noindex.
Repair broken pages and canonicals
Decide per URL: restore it, 301 it to the closest replacement, or remove links to it. Never 301 everything to the homepage, since engines treat irrelevant redirects like soft 404s.
Fix the sitemap generator and shorten chains
Make every old URL reach its final URL in one hop, and update internal links to the final URL.
Wait, then verify
Recrawling takes days to weeks, not minutes. Search Console lags, so 'URL is on Google' can differ from 'page was recrawled after your fix'.
This work suits a coding agent, because most of it is reading headers and templates in your own repository. The skill behind this article can be installed into Claude Code and other agents. It runs the checks, reads your canonicals and sitemap, and proposes the changes.
Full implementation guide for indexabilityInstall instructions for the agent skills live on one page.
Install the AEO agent skillsThe skill source is open on GitHub.
aeo-indexability on GitHubWhat will fixing this not do?
Being indexed makes a page eligible. It does not make it rank, and it does not make an AI assistant cite it. If a page is crawled but not indexed because of quality, no sitemap tweak helps, and the answer is a better page.
Also be careful about what your tools can prove. Google's URL Inspection is read-only, since Google offers no API to request indexing. And our AI visibility tools check Perplexity only, so a cleaner index does not show up there as proof that ChatGPT, Claude or Gemini now name you.
Common questions
Why is my page not indexed by Google?
The mundane causes are an accidental noindex, a canonical pointing at a different or dead URL, a redirect loop, or no sitemap or links telling Google the page exists. If Search Console says 'Crawled, currently not indexed', that can be a quality or priority decision rather than a technical fault, and resubmitting will not change it.
Does Google support IndexNow?
No. Google is not among the participating engines, which are Amazon, Bing, Naver, Seznam.cz, Yandex and Yep. For Google, submit your sitemap in Search Console, keep it in robots.txt, and use Request indexing for single URLs. Google calls sitemap submission only a hint.
Can I use Google's Indexing API to get ordinary pages indexed?
No. The Indexing API is only for pages with JobPosting or BroadcastEvent (in a VideoObject) markup. Using it for ordinary pages is outside its stated purpose. For ordinary pages, rely on a clean sitemap, internal links and Search Console.
Should I fix every noindex my audit reports?
No. A noindex on a login, checkout, thank-you page, internal search or staging site is correct. Our audit treats a noindex page that the sitemap does not list as deliberate. Fix it only if you ask of the page, is this meant to be in search, and the answer is yes.
Sources
Related guides
Want it done for you
Want this done and re-checked every month? Our monthly maintenance re-runs your buying questions in ChatGPT, Perplexity and Google, ships the fixes that keep you out of the answers, and reports what moved. Or run the free check on your site first at advancelabs.dev/tools/audit.
See how it works