Technical SEO Audit: A Step-by-Step Process for Agencies

What a Technical SEO Audit Covers

A technical SEO audit answers four questions about a website, in order: can search engines crawl it, can they render it, will they index it, and do they understand it? Content quality and backlinks matter just as much for rankings, but they are separate reviews. The technical audit covers the infrastructure that decides whether good content gets a chance to rank at all.

The order matters because each layer depends on the one before it. A well-written title does nothing on a page that robots.txt blocks, and structured data on a page marked noindex will never produce a rich result. Working top-down keeps the report from filling up with minor findings while a sitewide problem goes unnoticed.

For an agency, the audit is also a deliverable with a client on the other end. A useful one ends in a prioritized list of changes, each with an owner, rather than an export of every warning a crawler produced. The last two sections cover how to get there.

Set Up Access and a Baseline

Before crawling anything, collect the access and data the audit will be measured against.

• Google Search Console, verified as a Domain property where possible so the data covers every protocol and subdomain. The Page indexing, Sitemaps, Core Web Vitals and Crawl stats reports are the primary sources for this audit.
• Analytics, for organic landing pages and conversions, so each finding can be weighted by the traffic and revenue of the pages it affects.
• A full site crawl, with JavaScript rendering enabled and a smartphone user agent, because Google indexes the mobile version of each page.
• Access to the source: the CMS, the repository or the hosting panel. A finding is only useful if someone can change the thing it describes.

Export the Search Console performance data at the start as well. It is the baseline the client will compare results against later, and the Performance report only reaches back 16 months.

Crawlability: robots.txt, Status Codes and Links

Crawlability is whether a search engine can reach a URL and fetch it. Three things control it: the robots.txt file, the HTTP status code the server returns, and the links that lead to the page.

robots.txt

robots.txt tells crawlers which paths they may request. It controls crawling, not indexing. Google’s documentation states that “a page that’s disallowed in robots.txt can still be indexed if linked to from other sites”. It then appears in results without a description, because Google could not read the content. In the live file, look for broad Disallow rules left over from a staging site, blocked CSS or JavaScript files that pages need in order to render, and a Sitemap line that points at the current sitemap.

Status codes

Every URL in the crawl should return the status code its purpose calls for. These are the ones to review.

Google’s crawlers follow up to 10 redirect hops, but every hop slows crawling, so internal links should point at the final URL. Repeated server errors matter beyond the pages that return them: Google slows its crawl of the whole site, and indexed URLs that keep failing are eventually dropped.

Internal links

Search engines discover most pages by following links, and Google can only follow them reliably when they are <a> elements with an href attribute. Navigation built from click handlers on other elements may not be followed. In the crawl, look for orphan pages (URLs in the sitemap or analytics that no internal link points to), important pages buried many clicks from the home page, and internal links that point at redirects instead of the final URL.

Indexability: noindex, Canonicals and Page Indexing

A page that can be crawled still has to be chosen for the index. The robots meta tag, the X-Robots-Tag HTTP header and the canonical link all influence that choice.

noindex

A noindex rule keeps a page out of Google’s results, but only if Google can see it. Google’s documentation is explicit that “for the noindex rule to be effective, the page or resource must not be blocked by a robots.txt file”. A robots.txt block on a noindexed page is one of the most common reasons an unwanted URL stays in search results. The other common failure runs the opposite way: a noindex left in the templates after a staging site goes live.

Canonical tags

The rel="canonical" link tells Google which URL you prefer when several URLs show the same content: parameters, trailing slashes, print versions, HTTP and HTTPS. Google describes it as a strong signal rather than a rule. It is weighed together with redirects, internal links and sitemap entries, and Google may choose a different URL. The URL Inspection tool shows both the canonical you declared and the one Google selected. When they differ on important pages, the site’s signals contradict each other, and the fix is to make them agree.

Check that each canonical is an absolute URL, appears once per page, and points at a page that returns 200 and is neither redirected nor noindexed.

The Page indexing report

Search Console’s Page indexing report lists why known URLs are not indexed. These are the reasons you will see most often in an audit, and where to start with each.

Not every excluded URL is a problem. The audit question is whether the pages the business depends on are indexed, and whether everything excluded was excluded on purpose.

XML Sitemaps

An XML sitemap lists the URLs you want search engines to crawl and index. It does not guarantee indexing, but it is the clearest statement a site makes about which of its URLs are canonical. Check that:

• it lists only canonical, indexable URLs that return 200: no redirects, 404s, noindexed pages or parameter variants;
• each file stays within the limit of 50,000 URLs or 50 MB uncompressed, with a sitemap index file above that;
• <lastmod> reflects the date the content actually changed. Google uses it only when it is consistently accurate, and ignores <priority> and <changefreq> altogether;
• it is submitted in Search Console and referenced in robots.txt, and the Sitemaps report shows it was read without errors.

Then compare the sitemap with the crawl. Sitemap URLs that the crawl could not reach through links are likely orphan pages. Indexable URLs the crawl found that are missing from the sitemap usually mean the sitemap is generated from a different source than the site’s actual pages.

JavaScript Rendering

Google Search runs JavaScript with an evergreen version of Chromium, so content built with JavaScript can be indexed. Rendering still adds a dependency. Pages wait in a rendering queue, anything that appears only after a click or scroll may not be seen, and a noindex rule in the initial HTML can stop rendering altogether, so JavaScript cannot reliably remove one.

Compare the raw HTML with the rendered HTML for each template. The live test in the URL Inspection tool shows the HTML Google rendered and lists any resources it could not load. Confirm that the title, robots meta tag, canonical, main content, internal links and structured data all appear in the rendered output. Ideally they are already in the initial HTML, which removes the dependency entirely; server-side rendering or static generation is the most reliable way to guarantee that on content-heavy sites.

Check the size of the HTML too. Googlebot crawls the first 2 MB of an HTML file, and content beyond that point is not considered. Pages built with heavy inline styles, scripts or embedded data from a page builder can approach that limit.

Titles, Descriptions, Headings and Images

These elements are usually filed under on-page SEO, but at audit scale they are a technical matter: the problems come from templates, not individual pages.

• Title elements. Every indexable page needs a unique, descriptive title. Google sets no length limit and truncates titles to fit the device. It may build the title link from headings, anchor text or other page text when the title is missing, repeated or boilerplate. Titles duplicated across a template are the first thing to fix.
• Meta descriptions. Google creates snippets mainly from page content, and uses the meta description when it describes the page more accurately. A unique, specific description per page is still the best way to influence the snippet. Missing and repeated descriptions across templates are the usual findings.
• Headings. One clear main heading that states the page’s topic, and subheadings that follow the structure of the content. Headings used for styling, such as a logo wrapped in an h1 on every page, blur that structure.
• Images. Descriptive alt text on meaningful images, short descriptive file names, and <img> elements for any image that should appear in image search, because Google doesn’t index CSS images. Alt text should describe the image, not repeat keywords.
• Language versions. On multilingual sites, hreflang annotations must be reciprocal. Google’s documentation says that “if two pages don’t both point to each other, the tags will be ignored”.

Structured Data

Structured data describes a page’s content in a vocabulary search engines read directly, usually schema.org types written in JSON-LD, which is the format Google recommends. It makes pages eligible for rich results and helps search engines connect entities such as a business, its locations and its services. For most agency clients the useful types are Organization or LocalBusiness on the home and contact pages, BreadcrumbList across the site, Article or BlogPosting on posts, and Product where a page genuinely sells one.

Two rules apply. The markup must describe content that is visible on the page (Google’s policy is “don’t mark up content that is not visible to readers of the page”), and it must be valid. Use two validators, because they answer different questions: Google’s Rich Results Test checks eligibility for the rich results Google supports, and the Schema Markup Validator checks schema.org markup in general.

Mobile, HTTPS and Core Web Vitals

Google uses the mobile version of a site’s content, crawled with its smartphone agent, for indexing and ranking. The mobile rendering is therefore the version to audit. Confirm that it carries the same content, headings, structured data, internal links and meta tags as the desktop version, because content removed from a mobile layout is content Google does not see.

Every page should load over HTTPS, with HTTP URLs permanently redirected to their HTTPS equivalents and no mixed content. Then review Search Console’s Core Web Vitals report. It uses field data from real Chrome users and groups similar URLs together, which makes it a good signal at the template level. What each metric measures and how to improve it is covered in Core Web Vitals Explained.

Prioritizing the Findings

A crawler will report hundreds of issues on almost any site, so the value of the audit lies in the order of the list. Score each finding on how many important pages it affects and how directly it blocks crawling, indexing or understanding. Group findings that share a cause: one template fix often clears hundreds of rows.

Each finding in the report should name the affected URLs or template, the evidence (a URL Inspection result, a crawl export), the change required, and who will make it. The last field decides whether the audit produces results. Findings that wait months for someone to implement them are a common way SEO engagements stall, a problem covered in more detail in the piece on white-label SEO.

How Often to Run an Audit

Run a full audit at the start of an engagement, before and after any redesign or migration, and once or twice a year after that. Between audits, monitoring catches regressions sooner: Search Console’s email alerts for new indexing issues, a scheduled crawl compared against the previous one, and a look at the Page indexing report after any release that touches templates, routing or robots rules.

A redesign carries more search risk than routine changes, and it has its own process: how to relaunch a website without losing rankings. For the audit itself, this is the summary to work from.