Crawling Audit
Crawling comes first because nothing else matters if Googlebot can’t reach your pages in the first place. This is where most technical SEO audit checklists should genuinely begin, and where a lot of avoidable damage tends to hide.
Audit Robots.txt and Crawl Directives
Open your robots.txt file and check what’s actually being blocked. It’s shockingly common to find an entire section, sometimes the whole site, accidentally disallowed after a dev push. Verify it manually; don’t assume it’s fine just because nobody complained.
Audit XML Sitemaps
Your XML sitemap should only list canonical, indexable URLs, not redirects, not noindexed pages, not 404s. Check submission status in Google Search Console and compare the sitemap count against what’s actually indexed.
Find Crawl Errors and Broken Internal Links
Crawl errors pile up quietly over time. Broken internal links waste crawl budget and frustrate users equally. Pull the list from Search Console’s coverage report and cross-check it against a fresh Screaming Frog crawl.
Check Redirect Chains and Redirect Loops
A single redirect is fine. Three or four in a row, or worse, a loop that goes nowhere- that’s a crawl budget being burned for no reason. Map every redirect chain and flatten it down to one hop wherever possible.
Identify Crawl Traps and Crawl Waste
Faceted navigation, endless filter combinations, session IDs in URLs- these generate near-infinite URL variations. Identify the patterns and block or canonicalize them.
Analyze Googlebot Activity Through Log Files
Server logs show you exactly what Googlebot is crawling, how often, and where it’s wasting time. This is the one step most audits skip, and it’s often the most revealing; you see the real crawl behavior, not an estimate of it.
Tools for Crawling Audit
Google Search Console, Screaming Frog, and Semrush cover most of what you need here. Log file analyzers fill in the rest.
Rendering Audit
Rendering is where a lot of modern sites quietly fail SEO without anyone noticing for months. It’s the step right after crawling, and it answers a very specific question: does Google actually see what your visitors see?
Compare Raw HTML With Rendered HTML
Pull the raw HTML Google first receives, then compare it against the fully rendered version after JavaScript runs. If your main content only shows up after rendering, that’s a risk worth flagging immediately.
Audit JavaScript-Generated Content and Links
Content and links injected purely through JavaScript can get missed or delayed in indexing. Check whether critical navigation and content exist in the rendered DOM, not just in theory.
Check Blocked JavaScript and CSS Resources
If robots.txt or a server rule is blocking the JS or CSS files needed to render the page properly, Googlebot sees a broken or incomplete version. This happens more often than you would expect, usually by accident.
Identify Client-Side Rendering Issues
Heavily client-rendered pages can render inconsistently for Googlebot compared to a real browser. Test actual pages, not just the homepage, since rendering issues often hide deeper in the site.
Verify How Google Renders Important Pages
Use the URL Inspection tool in Search Console to see Google’s own rendered screenshot of a page. It’s the closest thing to seeing exactly what Googlebot sees.
Tools for Rendering Audit
Google Search Console’s URL Inspection tool, Chrome DevTools, and Screaming Frog’s JavaScript rendering mode all help here.
Indexing Audit
Getting crawled doesn’t guarantee getting indexed; that’s a separate hurdle entirely, and one that trips up a surprising number of sites. This section answers whether your important pages are actually showing up in Google’s index at all.
Check Whether Important Pages Are Indexed
Use the site: search operator alongside Search Console’s coverage report to confirm your key pages are indexed. Don’t assume; verify each important page individually if the site is small enough to do so.
Investigate Crawled and Discovered Currently Not Indexed Pages
These two Search Console statuses mean Google saw the page but chose not to index it. That’s often a quality signal, thin content, or a duplication issue; dig into a sample of these pages to find the pattern.
Find Accidental Noindex Directives
A noindex tag left over from staging, or added by a plugin update, can quietly deindex pages nobody’s watching. Check meta robots tags and HTTP headers across templates, not just individual pages.
Audit Canonical Tags
Every page should have a self-referencing or intentional canonical tag. Conflicting canonicals, where the tag points somewhere Google disagrees with, cause real indexing confusion.
Identify Duplicate URLs and Indexing Bloat
Parameter URLs and case variations often create duplicate versions of the same page. This bloats the index with near-identical content.
Find Soft 404s and Indexing Errors
A soft 404 is a page that returns a 200 status but has essentially no content; Google treats it like an error page anyway. Review these in Search Console and either fix the content or return a proper 404.
Tools for Indexing Audit
Google Search Console does most of the heavy lifting here, with Screaming Frog useful for bulk canonical and status code checks.
If you’ve noticed a sudden website traffic drop, indexing issues are often the first place to look, and they connect closely with Google algorithm update recovery work too.
Site Structure & Internal Linking Audit
Site structure decides how easily both users and crawlers move through your content. A logical structure spreads authority efficiently; a messy one hides your best pages under layers of navigation nobody follows.
Review Website Hierarchy and URL Structure
URLs should reflect a logical hierarchy: category, then subcategory, then page. Deep, inconsistent URL structures confuse both users and crawlers alike.
Check Click Depth and Orphan Pages
Any page more than three or four clicks from the homepage risks being under-crawled. Orphan pages, ones with zero internal links pointing to them, are effectively invisible unless they’re in the sitemap.
Audit Internal Links and Anchor Text
Internal linking should not be accidental. Check that anchor text actually describes the destination.
Review Breadcrumbs and Content Clusters
Breadcrumbs help both users and search engines understand hierarchy at a glance. Content clusters group related pages around a core topic. This structure can strengthen internal linking and improve topical relevance, which is especially useful when investigating a website traffic drop.
Check HTTPS and URL Version Consistency
Make sure only one version of each URL is live. Use HTTPS instead of HTTP. Choose either www or non-www and use it consistently across the site.
Tools for Site Structure Audit
Screaming Frog’s visualization features and Semrush’s site audit both map hierarchy and click depth well. This all ties directly into website structure and Google rankings, and increasingly into programmatic SEO for larger sites managing thousands of pages.
Speed and usability directly affect both rankings and real user experience, so this section covers both together. Core Web Vitals aren’t just a scoring exercise; they reflect how a real visitor experiences your site on a real device.
Audit LCP, INP and CLS
Largest Contentful Paint, Interaction to Next Paint, and Cumulative Layout Shift are the three core metrics. Check each against Google’s thresholds and identify which specific elements are dragging scores down.
Compare Lab Data With Field Data
Lighthouse gives you lab data, a controlled test. The Chrome UX Report gives you field data, what real users actually experienced. They can disagree, and when they do, field data reflects reality more closely.
Check Server Response and Page Loading Resources
Slow server response time (TTFB) drags every other metric down with it. Audit hosting, caching, and how heavy your page’s loading resources actually are before blaming the frontend. If these issues are rooted in the website’s code, architecture, or server configuration, working with an experienced website development company in Jaipur can help resolve them at the development level instead of relying on temporary SEO fixes.
Audit Image Performance
Oversized, unoptimized images are still one of the most common speed killers. Check formats, compression, and whether images are properly sized for their display dimensions.
Verify Mobile and Desktop Parity
Content, links, and structured data should match between mobile and desktop versions. Google indexes mobile-first, so anything missing on mobile is effectively missing altogether.
Check Mobile Usability
Tap targets too close together, text too small to read, viewport issues – these show up clearly in Search Console’s mobile usability report and are worth fixing early.
Tools for Performance Audit
Google PageSpeed Insights, Lighthouse, and the Chrome UX Report cover this well together. This connects directly to Core Web Vitals as a ranking factor, and to website user experience and SEO more broadly; the two aren’t really separate conversations.