A technical SEO audit doesn’t start with an open tool or an 80-page PDF that nobody reads. It starts with an uncomfortable question: what is preventing Google from properly crawling, rendering, and indexing your corporate website? If you can’t answer that with data, what you have is a to-do list, not an audit.
This article describes the exact configuration sequence I apply in B2B projects with hundreds or thousands of URLs, ordered by dependencies: each step produces the data that feeds the next. If you skip the order, you’ll end up collecting metrics you can’t interpret.
Step 0: define the scope before touching a tool
In a company with multiple business lines, the most costly mistake is auditing “the entire domain.” Before configuring anything, define:
- Subdomains included: blog, store, customer area, help center. Decide whether the private area is included (usually not, but it’s worth documenting).
- Languages and regional versions: if you have /es/, /mx/, and /ar/, each one needs its own hreflang review.
- Stakeholders: whoever approves template changes is usually in IT, not marketing. Identify them at this stage or the rest of the process gets blocked.
A real case: an industrial services company with 4 languages discovered during the scope phase that its help center lived on a subdomain with a different CMS and no access to logs. That completely changed the methodology for the rest of the audit: without logs, the crawl analysis had to rely on Search Console samples and crawls with a Googlebot user-agent.
Step 1: set up access to source data
Without this foundation, any audit is speculative. You need, at a minimum:
Search Console
Verify property by URL prefix whenever you can (it allows segmenting by subdirectory) and keep the domain property as a backup. Check that the user you’ll be using has full permission, not restricted: restricted permission hides performance and index data that you’ll need later.
Analytics
If you use GA4, make sure the web data stream has been active for at least 90 days. With less history, any comparison of organic traffic before/after a technical change is noise. Set up an organic traffic segment that excludes internal referral domains and campaign parameters that muddy attribution.
Server logs
Request access to the access files (Apache or Nginx) with a minimum of 30 days. According to Google’s official documentation on crawling, log analysis is the only source that shows with certainty which URLs Googlebot actually visits, versus the ones you think it visits.
Access to the CMS and the repository
You’ll need to be able to read the header templates, the redirect rules, and the robots.txt file. If you don’t have access to the repository, ask the technical team to export the .htaccess or redirect middleware configuration.
Step 2: prepare the crawler with precision
The crawler configuration determines the quality of all subsequent findings. Settings that make a difference:
- User-agent: use Googlebot’s to simulate mobile crawling (Googlebot Smartphone). Many content differences only appear with that agent.
- JavaScript rendering: enable it only if your website needs it. On websites with a lot of dynamic content, rendering drives up crawl time and can exhaust the tool’s credits. In a project with 12,000 URLs, enabling full rendering forced the crawl to be split into segments by section.
- Speed limit: don’t exceed 2-3 requests per second in production without notifying IT. An aggressive crawl can trigger the WAF and skew results with 403 errors.
- Authentication: if there’s content behind a login that’s relevant for SEO (for example, product pages with hidden prices), configure cookies or authentication headers.
Save the crawl configuration. You’ll need it identical to compare results after the changes.
Step 3: validate crawl health with robots.txt and sitemaps
Before reviewing anything else, confirm that Google can reach your URLs:
- Review robots.txt line by line. Look for blocks on CSS/JS resources (they break rendering) and on parameters that do generate unique content.
- Check that the XML sitemap only contains canonical URLs, with a 200 status and not blocked. A sitemap with redirects or 404s is a sign of neglect that Google progressively ignores.
- Verify in Search Console → Sitemaps that the file is processed without errors and that the discovered URLs match the ones you expect.
- Use the URL inspection tool for the 10 most important pages per business line and confirm “URL is on Google” and that the crawled version is the canonical one.
Honest limitation: if your website has more than 50,000 URLs, manual inspection doesn’t scale. In that case, prioritize inspection by template (one representative URL per type) and supplement it with the “Pages” report in Search Console.
Step 4: map status codes and redirects
Export from the crawl all URLs with a status other than 200 and classify them:
- 301/302: verify that redirect chains don’t exceed one hop and identify loops. A common pattern in companies after migrations: /product → /product/ → /es/product/ → 404. Each hop loses signal and adds latency.
- 404/410: cross-reference the list with Search Console to detect URLs that still receive external links. Those are the ones that need a redirect or recovery, not the ones that simply exist.
- 5xx: if they appear in the crawl, stop. An unstable server invalidates any other analysis.
Document each redirect with its source, destination, and reason. That record is what will allow IT to execute the changes without interpretation.
Step 5: resolve canonicals and duplicates
Set up the review in three layers:
- Declared canonicals: compare the rel=canonical tag with the final URL after redirects. A canonical that points to a redirected URL is a frequent error in CMSs with misconfigured SEO plugins.
- Parameters: identify the ones that generate duplicates (sorting, filters, pagination) and decide between canonical, noindex, or blocking. Note: blocking a URL with a canonical in robots.txt prevents Google from reading that canonical.
- URL variants: uppercase, trailing slash, http/https, www/non-www. Force a single version with server-level redirects, not from the CMS.
This doesn’t work the same in every case: in ecommerce with faceted filters, the canonical alone doesn’t solve the problem if the filters receive internal links. There you also need internal linking control and, sometimes, conditional rendering of the filter links.
Step 6: audit rendering and JavaScript
Compare the initial HTML with the rendered DOM on the key templates. Look for:
- Main content that only appears after executing JavaScript (high risk if there’s no server-side rendering).
- Internal links generated by JS that the crawler doesn’t discover without rendering.
- Metadata (title, canonical, hreflang) injected by JS that might not be reflected in the served HTML.
According to the official documentation on rendering, Google can execute JavaScript, but with a limited crawl budget and no guarantee that all scripts will run. The practical conclusion: if a link or a canonical only exists after JS, treat it as unreliable.
Step 7: measure Core Web Vitals with field data
Don’t settle for Lighthouse. Set up the review with real data:
- Export the Core Web Vitals report from Search Console by device type and URL grouping.
- Identify which templates fail LCP, INP, or CLS, not which individual pages.
- Prioritize by impact: a product page template with thousands of URLs and poor LCP matters more than the homepage.
Important nuance: field data has latency (28 days in the Search Console report). If you’ve just optimized images, don’t expect to see the change in the same week. And if your traffic is low, there may not be enough samples to evaluate a specific template; in that case, combine lab and field data, and say so in the report.
Step 8: consolidate the report and prioritize by dependencies
The deliverable isn’t a list of errors. It’s an execution sequence with dependencies:
- Blockers: 5xx, robots.txt blocking resources, broken redirects. Without this, the rest doesn’t matter.
- Structural: canonicals, duplicates, internal link architecture.
- Performance: Core Web Vitals by template.
- Improvements: structured data, hreflang, image optimization.
Each task must indicate the affected URL or template, the responsible team, and how the change will be verified (for example, “recrawl of the 200 URLs in template X and comparison of status codes”).
Configuration mistakes I see repeated
- Auditing with the desktop user-agent on websites where mobile serves different content. The findings don’t match what Google indexes.
- Trusting Search Console coverage without cross-referencing with logs. There are crawled URLs that never appear in the reports and others that appear as “excluded” for no apparent reason.
- Changing robots.txt and canonicals at the same time. If something gets worse, you won’t know what caused it. One change per iteration, with the date recorded.
- Ignoring the customer area in robots.txt and discovering months later that Googlebot is consuming crawl budget on session URLs.
What this methodology doesn’t cover
A technical audit doesn’t fix a search intent or content quality problem. If your website crawls and indexes correctly but the pages don’t respond to what the user is looking for, the technical data will be clean and traffic will stay flat. Nor does it replace a review of toxic links or domain authority: they’re different disciplines that complement each other, not overlap.
And a final warning: no audit guarantees ranking improvements. What it guarantees is that you eliminate measurable obstacles. The rest depends on factors you don’t control from the server.