Technical SEO Checklist: What to Check Before You Blame the Content

A technical SEO checklist exists to answer one question before anyone rewrites a word: can search engines reach, render and index the pages you want found? When a site stops growing, teams often blame the content. In our audits, the cause is just as often technical: a stray noindex tag, a canonical pointing the wrong way, a JavaScript template that hides the main text, or a firewall rule quietly blocking a crawler.

This is the list we work through, grouped in the order problems tend to block each other. Crawl issues hide indexing issues, and indexing issues hide everything else, so start at the top.

1. Crawling: can search engines reach your pages?

Nothing else on a technical SEO checklist matters if crawlers cannot request your URLs. Start with the file every crawler reads first.

• robots.txt loads at the root with a 200 status and does not disallow key sections by accident.
• Blocking is not deindexing. Google notes robots.txt is not a mechanism for keeping a page out of Google; a blocked page can still be indexed from links, just without its content.
• CSS and JavaScript files are crawlable, so pages can be rendered the way users see them.
• Server errors are rare. Check Search Console's crawl stats for spikes in 5xx responses or timeouts.
• CDN and firewall rules are not challenging or blocking legitimate crawlers, including the search bots of AI assistants you want to appear in.

2. Indexing: are the right pages in, and the wrong ones out?

• Page indexing report reviewed in Search Console; every "excluded" reason is understood, not ignored.
• No stray noindex on templates, staging leftovers or paginated pages that should rank.
• Thin and utility pages kept out: internal search results, filtered duplicates, thank-you pages.
• XML sitemap lists only canonical, indexable URLs that return 200, and is referenced in robots.txt or submitted in Search Console.
• Status codes are honest: missing pages return 404 or 410, not a 200 "soft 404".

3. Canonicals and duplicate URLs

Most sites generate duplicates without meaning to: http and https, www and non-www, trailing slashes, tracking parameters. Google lists the canonicalization methods in order of strength: redirects and rel="canonical" are strong signals, while sitemap inclusion is a weak one.

• One preferred host and protocol, with every other version 301-redirecting to it.
• Every indexable page has a self-referencing canonical with an absolute URL.
• Canonicals, internal links, sitemaps and hreflang all point at the same version of each URL.
• Parameter and filter URLs canonicalize to the clean version or are kept out of the index.

4. JavaScript and rendering

Google explains that it processes JavaScript pages in three main phases: crawling, rendering and indexing, and that pages wait in a queue for rendering. Content that only appears after scripts run can be indexed late, or not at all by crawlers that do not render JavaScript.

• Main content, headings and internal links are present in the initial HTML response.
• Links use real anchor tags with href attributes, not click handlers.
• Titles, meta descriptions and canonicals are rendered on the server, not injected late.
• Lazy-loaded content is loaded without needing a scroll or click that crawlers will not perform.

This is one reason we default to server rendering for marketing sites; our comparison of Next.js vs React goes into the trade-offs.

5. Speed, Core Web Vitals and mobile

Google's Core Web Vitals guidance sets clear targets, measured at the 75th percentile of page loads across mobile and desktop:

Mobile deserves its own line. Google states it uses the mobile version of a site's content for indexing and ranking, and that only content shown on the mobile site is used for indexing. If a tab, table or FAQ is hidden or removed on mobile to save space, check that it still exists in the mobile HTML.

6. Site structure and internal links

• Important pages are reachable within a few clicks from the homepage.
• No orphan pages: every page you want ranked has at least one internal link pointing to it.
• Anchor text describes the target page; avoid "click here" and avoid the same exact phrase on every link.
• Redirect chains are collapsed to a single hop, and internal links point at final URLs.
• Clean, readable URLs that stay stable; when they must change, a permanent redirect goes in the same release.

Site migrations and rebuilds are where structure breaks most often. If you are planning one, map every old URL to its new home before launch, not after traffic drops.

7. Structured data, metadata and AI crawler access

• Unique title and meta description on every indexable page.
• Structured data (Organization, Article, Product, FAQ and so on) matches what is visible on the page and validates without errors.
• Open Graph tags and a share image set for key pages.
• robots.txt rules for AI assistant crawlers are deliberate: allow the search bots you want to appear in, decide separately on training bots.
• An llms.txt file, if useful for your site, is generated from real content; our guide on what llms.txt is explains when it is worth adding.

Prioritizing what the checklist finds

A first pass through a technical SEO checklist on an established site usually produces a long list of findings. Audit tools add to the noise by scoring every warning as if it mattered equally. We sort findings by two questions: how many valuable pages does this affect, and does it stop those pages being crawled or indexed?

Resist fixing the easy low-impact items first just because they are easy. One wrong canonical on a pricing page can cost more than a hundred missing image descriptions.

How to run this technical SEO checklist

1. 01Verify Search Console for the domain property and review the page indexing, crawl stats and Core Web Vitals reports.
2. 02Crawl the site with a desktop crawler to collect status codes, canonicals, titles, redirects and internal links.
3. 03Compare three lists: URLs in the sitemap, URLs found by the crawl, and URLs Google reports as indexed. The gaps between them are your findings.
4. 04Fix in order: crawling, then indexing, then canonicals, then rendering, then speed and structure.
5. 05Re-check after every release, because most technical regressions ship with ordinary code changes.

If you are new to the subject, our SEO basics guide for founders covers the non-technical foundations that sit alongside this list.

Technical SEO is where engineering and search meet, and it is the part of our SEO and AEO service that engineers run, not marketers alone. If your content is good and rankings still stall, tell us what you're building and we will reply within one business day.