What is the difference between crawling, indexing and ranking?

Answers · More Than Normal, Canterbury

The short answer

Crawling is Googlebot discovering and fetching your pages, indexing is Google analysing and storing them in its database, and ranking is ordering those stored pages for a specific search. Usually a page must be crawled before Google can index and rank it, though a URL blocked from crawling can still appear in results without a description.

Crawling, indexing and ranking are three separate steps in Google’s guide to how Search works. Small business owners often hear all three used interchangeably, which makes it harder to diagnose why a page is not bringing in enquiries.

StageWhat Google is doingWhat can go wrong
CrawlingGooglebot follows links and sitemaps to fetch pagesBlocked by robots.txt, server errors or no links pointing to the page
IndexingGoogle renders the page, analyses content and stores a canonical URLMarked noindex, treated as a duplicate or left out for low quality
RankingGoogle orders indexed pages for a user’s search queryBetter-matched or stronger pages answer the query more clearly

Crawling is finding and fetching the page

There is no central directory of every page on the internet, so Google uses automated crawlers, mainly Googlebot, to look for new and updated URLs. It finds them by following links from pages it already knows and by reading XML sitemaps.

Being crawled only means Googlebot visited the address and downloaded the code. It does not guarantee the page will appear in search results. Conversely, Google notes that a URL blocked from crawling by robots.txt can still occasionally appear in search results without a description if other pages link to it, though Google cannot read the page’s content.

Indexing is understanding and storing the page

Once Googlebot fetches a page, Google tries to understand what it is about. During indexing, Google renders the page to see how it looks to a visitor, analyses text, images and video, and groups duplicate or near-duplicate URLs under one chosen canonical address in the Google index.

If a page carries a noindex tag, fails to render properly, or looks too similar to another URL, Google may crawl it and still leave it out of the index.

When someone types a query into Google, Google searches its index and orders the matching pages based on factors such as relevance, content quality, usability, freshness and context. Google’s ranking systems are automated, and Google does not accept payment to rank sites higher.

Because ranking only draws from what is already in the index, tweaking headings or keywords will not move a page that has never been indexed.

How to check which stage a page is in

If one of your service pages is not showing up, you do not have to guess which step failed. Google Search Console’s URL Inspection tool shows the exact status of any URL on your site, including when Googlebot last crawled it, whether it is in the index, and which canonical URL Google chose.

You can also run a live test in the same tool to see how Google renders the page right now and, after fixing a problem, request indexing so Googlebot queues it for another crawl. As the SEO Starter Guide notes, clear links and readable page structure help Google move pages smoothly through all three stages. If you would like a structured check across your whole site, our SEO audit looks at crawling, indexing and page-level relevance together.

Sources

Where a claim depends on how Google or another company says something works, we link to their own words so you can check it.

  1. In-depth guide to how Google Search works, Google Search Central. Supports the three stages of crawling, indexing and serving/ranking, how Googlebot discovers pages, canonical selection, and that Google does not accept payment to rank higher.
  2. URL Inspection tool, Google Search Console Help. Supports checking a page's index status, last crawl date, Google-selected canonical, live rendering and requesting indexing.
  3. SEO Starter Guide, Google Search Central. Supports how clear content, links and technical basics help Google crawl, index and understand web pages.

About this answer. We are an independent studio and this is our own research, experience and opinion, shared as general guidance. It is not legal, financial or other professional advice, and what works for one business may not work for another. Where the law or a regulator is involved, check the official guidance for your situation or ask a qualified adviser. Our sources are listed above.

Get our new answers by email

One short email a week with the answers we've published. Unsubscribe with one click.

We email you a link to confirm first. See our privacy notice.

Want to know where your website stands

A written SEO audit shows what to fix first, in order of priority, for £450.

Get an SEO auditBook a discovery call

No obligation, and we reply by email.