A technical SEO checklist is a structured list of the crawlability, indexation, performance, and structured-data checks that determine whether search engines and AI answer engines can find, render, and trust your pages. The short answer is that you don't need to run all 200+ checks by hand — the high-impact ones cluster into a handful of categories, and most can be automated.
This guide organizes the full checklist into ten categories, gives you the specific checks inside each one, and is honest about which parts still need a human. Where a number matters, like Core Web Vitals thresholds, we give you the exact figures rather than a vague "make it fast" instruction.
Which crawlability checks matter most?
Crawlability is the foundation. If a search engine or AI crawler cannot reach a page, nothing else on this list matters for that page.
- HTTP status codes on every page (200 expected; flag 4xx and 5xx)
- Redirect chains longer than one hop
- Redirect loops
- Server response time under crawl load
- Robots.txt syntax errors
- Robots.txt accidentally blocking important sections
- Crawl budget waste on low-value URL parameters
- Orphan pages with no internal links pointing to them
- Crawl depth (pages more than 4 clicks from the homepage)
- Broken internal links returning 404
- Soft 404s (pages returning 200 but showing "not found" content)
- DNS resolution errors
- Server uptime during crawl windows
- Firewall or bot-protection rules blocking legitimate crawlers
- Excessive redirect hops from HTTP to HTTPS to www to final URL
How do I check indexation?
A page can be perfectly crawlable and still never show up in search or get pulled into an AI answer if indexation signals are wrong.
- Noindex tags present on pages that should rank
- Noindex tags accidentally left on after a staging-to-production push
- Canonical tags pointing to the wrong URL
- Pages excluded from the XML sitemap that should be included
- Pages included in the XML sitemap that are noindexed (a direct contradiction)
- Index coverage errors in Search Console
- Duplicate content competing for the same query
- Parameter-based URL duplicates (sorting, filtering, session IDs)
- Pagination pages indexed when they shouldn't be
- Thin or near-empty pages diluting index quality
- Blocked resources (CSS/JS) preventing proper rendering before indexing
- Pages indexed under the wrong protocol (HTTP instead of HTTPS)
- Indexed staging or dev subdomains
- Discrepancy between "indexed" count in Search Console and actual page count
What site-structure checks belong on the list?
Structure determines how link equity and crawl priority flow through a site, and it's one of the easiest things to get wrong at scale.
- URL structure consistency (trailing slashes, casing, parameter order)
- Flat vs. deep architecture for priority content
- Breadcrumb presence and accuracy
- Category and tag page bloat
- Pagination implementation (rel next/prev deprecated but sequencing still matters)
- Faceted navigation generating near-infinite URL combinations
- Duplicate URLs from case-sensitivity (e.g.,
/Pagevs/page) - Trailing-slash inconsistency creating duplicate URLs
- Internal search results pages being crawlable and indexable
- URL length and readability
- Consistent use of hyphens vs. underscores in slugs
Which Core Web Vitals and speed checks?
Core Web Vitals are measured, not guessed at, and the thresholds are public. Here's the table most audits reference:
| Metric | Good | Needs Improvement | Poor |
|---|---|---|---|
| LCP (Largest Contentful Paint) | ≤2.5s | 2.5s–4.0s | >4.0s |
| INP (Interaction to Next Paint) | ≤200ms | 200ms–500ms | >500ms |
| CLS (Cumulative Layout Shift) | ≤0.1 | 0.1–0.25 | >0.25 |
Checks that determine where a page lands on that table:
- LCP element identification (what's actually the largest content element)
- Render-blocking CSS and JavaScript
- Unoptimized or oversized images
- Missing
width/heightattributes causing layout shift - Web fonts causing flash of invisible or unstyled text
- Third-party scripts delaying interactivity
- Server response time (TTFB)
- Lack of lazy loading on below-the-fold images
- Missing image compression (WebP/AVIF vs. legacy formats)
- Unused CSS and JavaScript shipped on every page load
- CDN usage and cache-control headers
- Total page weight trending upward over time
What mobile checks?
Most search engines index the mobile version of a page first, and mobile usability issues affect both rankings and AI-crawler rendering.
- Viewport meta tag present and correctly configured
- Tap targets sized and spaced for touch
- Font sizes readable without zooming
- Content parity between mobile and desktop versions
- Intrusive interstitials or pop-ups blocking content on load
- Mobile page speed measured separately from desktop
- Responsive image serving (correct sizes for device)
- Horizontal scrolling caused by fixed-width elements
- Mobile-specific redirect errors (m-dot domains, if still in use)
Which on-page/metadata checks?
On-page metadata is the layer most people know about, which is exactly why it's worth auditing systematically instead of assuming it's fine.
- Missing title tags
- Duplicate title tags across pages
- Title tags truncated in search results (length/pixel width)
- Missing meta descriptions
- Duplicate meta descriptions
- Missing or duplicate H1 tags
- Heading hierarchy skipping levels (H1 to H3 with no H2)
- Missing alt text on meaningful images
- Keyword cannibalization between similar pages
- Thin content pages (word count far below topic norms)
- Outdated content with stale dates or dead references
- Missing or incorrect Open Graph tags
- Missing or incorrect Twitter Card tags
- Language and hreflang declarations for multi-region sites
Canonicals, sitemaps and robots — what to verify?
These three mechanisms tell search engines and AI crawlers which URL is authoritative and where to look first, so conflicts between them cause real damage.
- Canonical tag present on every indexable page
- Self-referencing canonicals on pages that should be canonical
- Canonical tags pointing cross-domain by mistake
- Canonical vs. noindex conflicts
- XML sitemap validity (well-formed XML, no errors)
- XML sitemap freshness (updated when content changes)
- XML sitemap size limits (50,000 URLs / 50MB per file)
- Sitemap index file referencing all sub-sitemaps correctly
- URLs in sitemap that 404 or redirect
- Sitemap submitted in Search Console and Bing Webmaster Tools
- Robots.txt referencing the sitemap location
- Robots.txt disallow rules tested against real URL patterns
- Robots.txt accessible at the root domain (not blocked itself)
- Consistency between robots.txt, meta robots tags, and X-Robots-Tag headers
Which structured-data checks?
Structured data is increasingly a two-audience concern: it feeds rich results in classic search and it gives AI answer engines a machine-readable summary of what a page is about.
- Schema markup present on eligible page types (Article, Product, FAQ, HowTo, etc.)
- Schema validates without errors (Rich Results Test / Schema.org validator)
- Schema properties match visible on-page content (no mismatched data)
- Organization and website schema on the homepage
- Breadcrumb schema matching actual site navigation
- FAQPage schema matching actual Q&A content on the page
- Duplicate or conflicting schema blocks on the same page
- Missing required properties for the schema type used
- JSON-LD implementation (preferred over microdata for maintainability)
Internal linking checks?
Internal links distribute authority and help both crawlers and AI systems understand which pages you consider important.
- Orphan pages (already covered under crawlability, worth a second pass here)
- Broken internal links
- Anchor text variety and relevance (not all "click here")
- Over-optimized exact-match anchor text patterns
- Deep pages lacking sufficient internal links pointing to them
- Navigation links vs. contextual body links (both matter, differently)
- Internal link count trending down on previously well-linked pages
- Redirected internal links (linking to a URL that then redirects)
- Related-content or "you might also like" modules functioning correctly
JavaScript-rendering checks?
JavaScript-heavy sites introduce a rendering step that many audits skip, and it's one of the more common reasons content doesn't get indexed or extracted correctly.
- Content visible in rendered HTML, not just raw source
- Critical content not dependent on client-side JavaScript to appear
- Rendered vs. raw HTML comparison for key pages
- JavaScript errors blocking rendering entirely
- Lazy-loaded content that never triggers for crawlers
- Infinite scroll content inaccessible without pagination fallback
- Client-side redirects instead of server-side (slower to process, less reliable)
- Hydration mismatches between server-rendered and client-rendered content
How much of this can you automate?
Honestly, most of it. A crawler that checks 200+ ranking factors — SPIDrSEO is built to do exactly this — can automate status codes, redirect chains, duplicate titles and meta descriptions, canonical conflicts, sitemap and robots.txt parsing, Core Web Vitals pulls, schema detection and validation, broken internal links, orphan-page detection, and crawl-depth mapping. That covers the large majority of the checklist above, and it's the part that scales badly if done manually across a site with more than a few hundred pages.
Some checks still need human judgment. A crawler can tell you a page is noindexed; it can't tell you whether that noindex was intentional (a thank-you page) or a mistake (a category page that should be ranking). It can flag thin content by word count, but it can't fully judge whether that content actually answers the searcher's intent. It can detect duplicate titles, but deciding which page should "win" a cannibalization conflict is a strategic call, not a technical one. Treat automated output as a prioritized list of things to verify, not a final verdict.
Screaming Frog, Sitebulb, Ahrefs Site Audit, and Semrush Site Audit all automate large parts of this same checklist, each with different strengths in crawl depth, JavaScript rendering, and reporting format. SPIDrSEO's crawler runs in the cloud, checks 200+ ranking factors in the same pass, and layers in AI-visibility scoring — how extractable and citable your content is to systems like ChatGPT and Perplexity — alongside the classic technical checks. For a broader comparison of how these tools differ on price and depth, see our full SEO tools comparison for 2026. If you want to understand the AI-visibility layer specifically, our guide to AI visibility scoring covers how that scoring works.
How often should this checklist actually run?
Full manual audits once a quarter are common practice, but the point of automating this checklist is that it doesn't have to wait for a quarterly calendar slot. A crawler can re-run the high-impact categories — status codes, canonicals, sitemap validity, Core Web Vitals — on a schedule, and flag regressions the same week they happen instead of three months later. Sites that ship code frequently benefit from tighter cycles; a site that rarely changes can run less often without losing much.
What does a realistic rollout look like?
Start with the categories most likely to have silent failures: crawlability and indexation. A single misplaced noindex tag or a robots.txt rule blocking a critical directory can erase visibility for an entire section without any visible symptom until traffic drops. From there, move to Core Web Vitals and mobile checks, since both directly affect ranking signals and user experience. Structured data and internal linking checks come next — lower risk of catastrophic failure, but steady compounding value when kept clean. Treat the checklist as a rotation, not a one-time project: run it, fix what surfaces, and re-run on the categories most likely to drift.
Pricing note for teams evaluating automation
If you're weighing whether to automate this checklist with a paid tool, see our pricing page for tier breakdowns across the free and paid options available. Free tiers are generally enough to spot-check a handful of pages; sites with more than a few hundred URLs typically need a paid crawl tier to get full-site coverage in a single pass.
Frequently Asked Questions
How many of the 200+ checks actually matter for most sites?
For most sites, a subset of roughly 20 to 30 checks across crawlability, indexation, Core Web Vitals, and canonicalization drive the majority of visible impact. The remaining checks matter more for large or complex sites (faceted navigation, multi-region content, heavy JavaScript rendering) where edge cases compound at scale.
Can a technical SEO audit be fully automated?
No, not fully. A crawler can automate the large majority of the mechanical checks — status codes, duplicates, canonicals, sitemap and robots.txt parsing, Core Web Vitals, schema validation, and broken-link detection. Judgment calls like whether a noindex is intentional, whether content actually matches search intent, or which page should win a cannibalization conflict still need a human to decide.
How often should I run a technical SEO audit?
A full manual review on a quarterly cadence is common, but automated tools can re-check the highest-impact categories (crawlability, indexation, Core Web Vitals) far more frequently, including weekly or after every deploy. Sites that ship code changes often benefit from tighter cycles so regressions get caught before they affect rankings.
Are free tools enough to run this checklist, or do I need a paid one?
Free tiers are generally enough to spot-check a small number of pages or verify a specific fix. Sites with more than a few hundred URLs typically need a paid crawl tier to get full-site coverage, crawl history, and export options in a single pass, since free tiers are usually capped on pages per crawl and crawl frequency.
Do AI answer engines like ChatGPT and Perplexity care about technical SEO?
Yes, in a related but distinct way. AI systems still need to crawl and render a page to extract its content, so crawlability, clean HTML rendering, and structured data all affect whether a page can be cited. Beyond that baseline, AI-visibility scoring looks at extractability and citability specifically, which is a related but separate metric from classic ranking factors.
What's the single highest-impact check on this list?
Indexation status is a strong candidate: a page with a stray noindex tag, a canonical pointing elsewhere, or a robots.txt block is invisible to search and AI crawlers regardless of how good the content is. Checking that indexation signals match intent, page by page, catches the failures most likely to silently erase visibility.
What's the difference between technical SEO and on-page SEO?
Technical SEO covers whether a page can be crawled, rendered, and indexed correctly — server responses, sitemaps, robots directives, Core Web Vitals, structured data. On-page SEO covers what's on the page once it's accessible — titles, headings, content quality, keyword targeting. The two overlap (metadata checks touch both), but technical issues can block a page from ranking regardless of how strong the on-page content is.
Does SPIDrSEO cover all 200+ of these checks?
SPIDrSEO checks 200+ ranking factors in a single cloud crawl, covering the large majority of the categories in this checklist: crawlability, indexation, Core Web Vitals, on-page metadata, canonicals, sitemaps, structured data, and internal linking, plus AI-visibility scoring layered on top. Some judgment-based checks — like whether thin content actually matches intent — still require manual review regardless of which crawler you use.