Most audit playbooks assume you can crawl the whole site, open the pages that look wrong, and hand engineering a list of fixes. On a site with a few million URLs that export can get unwieldy very fast and generally gets stack ranked. You may get a few fixes at the top but the long tail of oddities may stick around for months, or forever.
We've scoped audits at that size, and can say with some authority that the method has to change with the scale. An enterprise SEO audit measures defect rates per template instead of reading pages, reads server logs to see what Googlebot actually fetched, and delivers a scored backlog mapped to the templates that earn revenue, each item with an owner and a test.
Here's what that looks like phase by phase, and where you're going to see a standard audit break down.
What is an enterprise SEO audit?
An enterprise SEO audit covers a site large enough that crawl budget is a live constraint, which Google's crawl budget guidance defines as roughly a million unique pages that change weekly, or ten thousand or more that change daily. Below that, Google says, you can skip the topic. Above it the site usually spans several hostnames (each with its own crawl budget), several page templates generated by different systems, and product, engineering, content, and regional teams that each own part of the stack.
The strategic job is to find the handful of template-level defects that account for most of the lost traffic, price them against revenue, and hand engineering a backlog it will schedule. One template decision can generate more URLs than every other problem combined, so the audit works template by template.
How it differs from a standard audit
A standard audit crawls every URL, opens the ones that look wrong, and lists fixes per page. On a 2,000-page marketing site that's an afternoon in your favorite tool. On a five-million-URL site a full crawl takes weeks and produces an export that, let's be honest, no one reads.
There are a few things that change on large audits:
- Findings become rates. You report the defect rate in a sampled template, multiply it by the template's URL count, and attach a margin of error.
- Every segment has an owner. Country, subfolder, template, and business division each map to a team that ships in sprints, so a finding without an owner falls out of the backlog no matter how severe it is.
- The stack hides things from crawlers. Multiple CMSs, JavaScript hydration, and pre-rendering layers mean you verify what Googlebot receives per template rather than trust one crawler's view.
The seven phases of an enterprise SEO audit
Each phase feeds the next. Scope defines the segments, the crawl measures defects per segment, and the logs show where Googlebot spends its time.
| Phase | Output |
|---|---|
| 0. Scope | Segments with URL count, revenue share, and owner |
| 1. Segmented crawl | Defect rates per template, extrapolated |
| 2. Log analysis | Crawl waste share per directory (the baseline) |
| 3. Indexation and facets | Facet crawl rules, sitemap and canonical fixes |
| 4. Speed and rendering | Failing templates with a root cause |
| 5. Content and link equity | Consolidation and linking rules per group |
| 6. International and multi-domain | hreflang errors per cluster, domain structure call |
| Delivery | Scored backlog, owners, tickets with acceptance criteria |
Delivery runs alongside the phases from Phase 1 onward, so engineering can start on crawl blockers while you're still measuring page speed.
How to scope an enterprise SEO audit
Plenty of enterprise audits fail before anything gets crawled, because the auditor never gets the URL inventory, the raw logs, or a named engineer. Discovery is the cheapest phase, so, frankly, it gets rushed. It produces four outputs:
- URL inventory by template. Pull the sitemap index, the Search Console Pages counts, and a CMS export, then bucket URLs by template. Five to seven buckets is usually enough, and the content inventory guide explains the merge in detail.
- Revenue per template. Overlay revenue or conversions per template from analytics. Traffic concentrates in a small slice of pages on almost every large site, and every later phase reports against that slice.
- Owners and dev capacity. Name who approves and who ships for each segment, and ask how many sprints the next quarter has open for SEO work. That number caps the backlog.
- Data access. Request Search Console for every hostname, analytics, CrUX, and raw server or CDN logs. Most sites don't retain logs for long, so request a 30-day export on day one.
Build the segment-to-revenue table before anyone opens a crawler. Crawl-first audits skip it and produce thousands of findings nobody can rank.
How to crawl a site you cannot fully crawl
No industry body publishes a crawl-percentage standard for enterprise audits. One widely cited sampling table recommends 10% to 25% for sites of 10,000 to 500,000 URLs and 1% to 5% above that, while others recommend far larger or smaller samples for the same site. The disagreement tells you the percentage is the wrong unit. Defects on templated sites repeat by template, so sample by template.
The harder question is how many URLs per template. Rules of thumb run from a handful of examples to a couple dozen per template, and none are statistically derived. The audit sampling math that does exist (written for accessibility audits, but the statistics transfer) gives about 385 observations per population for a five-point margin at 95% confidence, and about 1,068 for a three-point margin. That only holds when the sample is random, and when you stratify by template you size each stratum separately.
That gives us a workable protocol:
- Baseline crawl. One full or very large crawl annually or semi-annually, with targeted test crawls on top.
- Per-template samples. Roughly 400 random URLs per template, drawn from sitemaps or logs rather than a crawler's link discovery, which oversamples well-linked pages and hides the orphans.
- Defect rates. The rate per template for each check (status, canonical, noindex, hreflang return link, near-duplicate body, render failure).
- Extrapolation. Each rate times the template's URL count, with the margin attached. If 12% of a 400-URL sample from a 900,000-URL product template canonicalizes elsewhere, report an estimated 108,000 affected URLs, plus or minus about 45,000, and put that range in the ticket. Re-sample after fixes ship.
Log file analysis for enterprise sites
A crawler export shows what your site can serve. Server logs show what Googlebot chose to fetch, and on a large site the two lists rarely match. Google defines crawl budget as crawl capacity combined with crawl demand, and says the demand factor you control most is perceived inventory, because duplicate and unimportant URLs waste crawling time that could go to pages that change.
Verify Googlebot before you count anything, because user-agent strings are trivial to spoof. Google's verification documentation gives two methods, a reverse-then-forward DNS check per IP or a match against Google's published IP ranges, and dedicated log analyzers do the second for you. Segment by hostname, since crawl budget is per host.
Then compute crawl waste per directory and per template. Waste is the share of verified Googlebot requests that land on URLs you don't want indexed: parameter and facet variants, redirect chains, soft 404s, noindexed pages, and URLs canonicalized elsewhere. Overlay organic performance from Search Console on the same segments, because that's what ties waste to money. If /search/ consumes a third of verified hits and earns nothing while your product template gets crawled every three weeks, you have the first P1 ticket.
One publisher case study across 13 country domains shows the before and after once auto-generated pages were noindexed or removed, blocked in robots.txt, and re-canonicalized:
| Metric | Before | After |
|---|---|---|
| Crawled, not indexed pages (one market) | 513,369 | About 220,000 |
| Soft 404 errors (same market) | 1,193 | 370 |
| Daily crawl requests (another market) | 60,000 to 70,000 | 20,000 to 30,000 |
| Total clicks per day | About 8,000 | 12,000 to 15,000 |
Report the same four rows for each revenue template, then re-run the log window 30 days after the fix ships. Segment AI crawlers while you're in there. Their hits show which engines can reach each template, not whether answers cite you. Measure that separately against a panel of buyer questions per engine, as in AI search visibility metrics.
Facets, canonicals, and index bloat
Index bloat on an enterprise site is a template problem, and faceted navigation causes most of it. A filter UI that writes every combination of color, size, and price into a crawlable URL can put far more facet URLs in the index than the site has real pages. Each of Google's controls does one thing, and the audit checks that the site uses the right one per URL class:
- robots.txt stops crawling and nothing else. Google's faceted navigation guidance says there's usually no good reason to let filtered URLs be crawled, and that canonical and nofollow are less effective long term than robots.txt or URL fragments. A blocked URL can still sit in the index as a bare URL if other pages link to it.
- noindex removes from the index and costs crawl budget. Google has to fetch the page to read the tag, and a URL blocked in robots.txt never gets its noindex read. Use noindex on facet URLs already indexed, then block them once they've dropped out.
- rel=canonical is a hint. Canonicals alone rarely stop a facet explosion, because Google is free to ignore the hint when the pages look distinct enough.
- URL fragments have no crawl impact. Filters that update the page via # keep the UX and generate no crawlable URLs.
Our default is a disallow rule on multi-refinement facet URLs, with single-facet listing pages left crawlable. XML sitemaps are the second half of this phase. Split them by template, keep only indexable, self-canonical URLs in them, and the Search Console Sitemaps report gives you an indexed-to-submitted ratio per template. That ratio, against the index size reported for the same directory, is your index bloat number.
Speed and rendering by template
Core Web Vitals fail by template, because the template ships the JavaScript bundle and the assets that affect loading and layout. The thresholds are LCP at or under 2.5 seconds, INP at or under 200 milliseconds, and CLS at or under 0.1, and a page passes only when it meets all three at the 75th percentile of page loads, split by mobile and desktop. Rank templates on field data, because Lighthouse reports Total Blocking Time in place of INP and lab CLS only captures shifts above the fold during load.
Find the segments CrUX marks Needs Improvement or Poor, group them by template, and put a per-template budget in Lighthouse CI so a fixed template stays fixed.
Rendering gets the same per-template treatment. Googlebot renders pages in headless Chromium and re-parses the result for links, so the check is a diff between raw and rendered HTML for each template's sample. Do links, canonical, hreflang, and the main content exist before JavaScript runs? Server-side rendering with rehydration helps first paint but can hurt INP, because the page looks ready before its event handlers attach.
Log these findings as technical debt sized in sprints, with a priority level, rather than as a slide.
Content and link equity by page group
Content defects at enterprise scale come from templates too. A category template that generates near-identical H1s and intro copy across every category cannibalizes at the template's scale from one line of code. Run the checks per page group, using the segments from Phase 0:
- Duplication. Run the crawler's near-duplicate detection within each template sample, then extrapolate the rate as in Phase 1. Cross-template duplication is usually a data-feed issue, so name the feed.
- Cannibalization. Export Search Console performance by query and page, then flag queries where two or more URLs from the same template earn impressions. Word count alone is a poor thin-content signal, so set the threshold per template.
- Gaps. Per group, list the buyer questions the template should answer, mark which ones have a page that answers them, and only then pull the queries with impressions but no top-10 URL as evidence for the uncovered questions. Rank trackers cap the keywords you can follow, so track by template rather than by page.
Link equity gets the same treatment. Measure click depth per template, find orphans by checking verified Googlebot hits and sitemap URLs against the crawl's link map, and aggregate referring domains by template. Deep pages earn the citations, but only if internal links get them crawled.
International and multi-domain checks
hreflang error rates on real sites are high enough that skipping this phase is negligent. A 2023 analysis of 18,786 sites with hreflang code found 31% with conflicting directives, 16% of clusters missing self-referencing tags, and 9% with at least one invalid language code.
The checks follow Google's localized-versions documentation directly. Every version must list itself and every alternate, return links must be reciprocal or Google may ignore the pair, language codes must be ISO 639-1 with optional ISO 3166-1 Alpha 2 regions, and the canonical must be in the same language as the page. For large sites, put the annotations in XML sitemaps rather than the head, which gets too heavy to maintain across many markets.
Domain structure decides where crawl budget sits. Google allocates it per hostname, so ccTLDs and subdomains isolate a facet explosion to one market while subdirectories pool it. Give each host its own Search Console property, sitemaps, and log segment so every number above can be reported per market.
How to prioritize enterprise SEO audit findings
A finding without a score, an owner, and an acceptance criterion doesn't ship, and an enterprise audit produces hundreds of findings. Generic ICE scoring (Impact, Confidence, Ease) leans on gut feel, and SEO can do better because impact is estimable from templates and revenue data. We define Impact as estimated affected URLs from Phase 1 multiplied by the segment's revenue share from Phase 0, and we double-weight it.
Severity needs a separate rubric from priority, because crawler labels (Errors, Warnings, Notices) don't map to business tiers. We use four levels with a response window attached, from P1 (active ranking loss or crawl failure, fixed within days) down to P4 (refinements, batched quarterly).
Ownership goes in a RACI, with the SEO lead accountable for scoring and recrawl verification, the dev team responsible for template and robots.txt changes, and the product owner approving template code. Each ticket carries the evidence and a testable exit condition:
Title: Block crawling of /search/ facet URLs with 3+ refinements (Category template)
Segment: /search/ facets, est. 1.4M URLs (400-URL sample, 97% non-self-canonical, +/-5 pts)
Evidence: 31% of verified Googlebot hits in 30-day log window, 0.2% of organic revenue
Fix: Disallow rule for 3+ refinements, keep single-facet listing pages crawlable
Acceptance: Googlebot hits on /search/ under 8% of total in next 30-day window
Severity: P2 | Owner: Platform team | Sprint estimate: SmallThe numbers are illustrative. Copy the structure, especially the measurement the engineer can run without you.
Tooling, cost, and cadence
Tooling splits by job: a desktop crawler (Screaming Frog) or a crawl platform (Botify, Lumar), a log analyzer or BigQuery, Ahrefs or Semrush for rank and links, Search Console plus the CrUX and PageSpeed Insights APIs for field data, and Looker Studio to join it all per segment. The crawl platforms sell on quote with multi-year contracts, so budget a negotiation cycle rather than a line item.
Budget in hours rather than a price band, because no published survey isolates enterprise audits. Single-agency estimates put an enterprise audit at 60 to 200 hours over four to eight weeks, with log analysis alone taking 15 to 30 hours on a high-traffic site. Commission an agency for a one-off baseline when you lack log tooling, and bring the work in-house when the log and index phases need to rerun monthly, because the segment definitions and the revenue map leave with the agency. The in-house versus outsourcing math is its own topic.
The full crawl is an annual or semi-annual baseline, a speed fix shows up in CrUX only after its 28-day window rolls over, and a log window needs 30 days to settle. A point-in-time audit expires quickly, and the Phase 1 defect rates are only useful if something re-measures them.
What AI can and can't do on a crawl export
One documented run shows how feeding a crawl export to a chat interface goes wrong. Claude read a 136-page Screaming Frog export and marked 85 pages as too thin at high priority with nothing behind it, claimed 28 problem types against a detail sheet with 30 rows, and miscounted its own severity totals. The author concluded the model captured the signal but failed at cause diagnosis, priority, and the fix.
Better prompts don't fix this, because the causes are structural. Chat tools cap spreadsheet uploads and sample large CSVs, and models lose accuracy well inside their advertised context windows. In the NoLiMa benchmark, 11 of 13 models scored below half their short-context baseline at 32,000 tokens. So use LLMs for the parts that aren't aggregation or judgment. Have one write the pandas or SQL that joins logs to the crawl, then run that code yourself, and have it draft tickets from your scored sheet.
Most teams never get to the re-measurement. The defect rates, the log baseline, and the per-template checks all need to run again next month, and that rarely happens across separate tools. The GrowthX platform holds that standing baseline. Every page inside the zone it crawls carries a Health score and a Quality score, read side by side and grouped by page type, so a regression shows up before the next annual crawl. For the AI-answer side, CheckThat, which powers AI visibility in the GrowthX platform, supplies the category baseline from more than 5,800 brands and 2.6 million AI responses across nearly 200 B2B software categories as of September 2026. If keeping the audit alive is your bottleneck, book a demo and we'll show you how it runs in one workspace.
