How I Do a Full SEO Site Audit: The Complete Pipeline

TL;DR

I do site audits myself, manually, using my own scripts and tools – AI only gets involved in two narrow areas: helping write code for data collection scripts and piecing together my scattered notes into a readable draft report. All analysis, metric interpretation, and deciding what’s critical vs. what can wait is my job. Step by step: reconnaissance, three mandatory blocks for any site, deep-dive checks, a wave of additional checks tailored to the specific business type, and final report compilation.

After my last post in my Telegram channel (subscribe, by the way), people started asking: do I really do the audit myself, manually, and not just run some automated script and copy the result into my report? It’s a fair question – automation in SEO is talked about everywhere, and the word “audit” has become associated with a “launch” button and a five-minute wait. My process is different, and since the question came up, I’ll break down the entire process step by step – including script names, logic, and what I actually get at each stage.

Let me address the AI topic right away, since that’s the core of the question. I use it in two places. First – when writing the Python scripts themselves that collect data: code gets written faster if the model drafts a rough version and I edit and adapt it to the specific task. The second place is at the very end, when I have dozens of notes, numbers, and observations collected manually during the work. It’s a mess of thoughts, a stream of consciousness that needs to be turned into a readable report – here the assistant condenses the text into a structure, and I verify every fact and refine the wording. The actual site analysis, deciding what to check deeper, prioritizing findings – all that is done by me, with my eyes and brain, looking at the numbers my tools produce.

Below I’ll outline step by step what my audit consists of. I’ll give the script names, but not the code itself – that, sorry, is my working tool that puts food on the table. You could try to build a similar pipeline yourself, but without analysis experience, a specific set of scripts and raw numbers won’t be very useful. Numbers without interpretation are just numbers.

I’ll keep it brief where I could dive into details. A detailed technical breakdown of each script would turn the article into a reference book that maybe one and a half people would finish reading. So I’ll give the process logic and what I get at each step, without dissecting internals.

Step 0. Reconnaissance

The first step of any audit is to understand what I’m dealing with. Without this stage, you could spend half a day doing an in-depth check of structured data on a site that doesn’t even have a proper robots.txt.

The goal of reconnaissance is to gather facts and decide which additional checks in Step 3 to even run. If the site isn’t a local business, why run local checks? If the site has no blog, why analyze keyword cannibalization?

Here’s what I run at this stage and why:

Script / Tool	What I check	What I get
sitemap_discovery	Is there a sitemap.xml, robots.txt, how many maps and of what type	Understanding whether important site sections are blocked from indexing in robots, whether there’s a sitemap index (map of maps for large sites)
render_page --mode auto --json	HTML code, rendering type, extracted text, publication date	SPA (a site that builds in the user’s browser rather than serving ready HTML) or classic SSR; whether text is truncated – a signal to check via curl
curl on an array of URLs (/about, /pricing, /blog, /contacts, and a deliberately non-existent address)	Response codes, headers, time to first byte, redirects	200 on live pages, 404 on non-existent, correct redirect from http to https, whether compression and caching are enabled
adv_crawl with a limit of a few hundred pages	title, meta, h1, status, redirects across a sample of pages	Distribution of response statuses, title and meta duplicates, pages without h1, canonical mismatch
adv_urls	Structure of paths and parameters	Breakdown by site sections, UTM junk in the index, localization, potential cannibalization
yandex_webmaster_list_hosts	Is the site added to Yandex.Webmaster	If not – first recommendation in the report: register
yandex_metrika_list_counters	Is the Metrika counter installed	If yes – then pull top pages and traffic from there for text analytics
search query for the domain excluding the domain itself	External mentions, competitors	Who writes about the site, who competes for attention in search results

At the end of this step, I have a note with basic signals about the site – what kind of project it is, what CMS, whether there are obvious technical issues on the surface, where to look deeper. Often even here I can see the direction: if curl shows that half the site returns 404 on pages that actually exist – that note becomes priority number one for the report.

Step 1. Core: Technical Audit, Content, and Structured Data

After reconnaissance, I move to three blocks that I run almost always, regardless of site type. This is the foundation without which any other analysis would be hanging in the air.

Technical Audit

Here I look at everything that prevents search bots from properly crawling and understanding the site. I use the same render_page and sitemap_discovery from the previous step, plus adv_urls and URL security checks. If a quick crawl was already done in reconnaissance, I pull its results to avoid running requests again.

I analyze page accessibility for crawling – whether important sections are blocked in robots.txt or via the noindex tag. I check indexability: whether canonical links match the actual page URL, whether there are content duplicates, whether there are thin pages with minimal text. I look at server security headers, URL structure and redirect logic, mobile responsiveness, loading speed potential, JavaScript rendering dependency, and IndexNow support.

The output is a rating per category – whether the site passed the check or not, an overall technical score from 0 to 100, and a list of findings sorted by priority: critical, high, medium, low.

Most often, technical problems hide not in obvious places like robots.txt, but in the details: canonical pointing to a non-existent page version after a domain change, or a redirect chain of three to four hops where there should be a single direct jump.

Content and E-E-A-T

Next, I look at the content itself. Here I always use a separate script for assessing Russian-language text quality – universal content analysis tools often simply don’t see Cyrillic and return empty results where there’s plenty of text.

I assess using the E-E-A-T model (Experience, Expertise, Authoritativeness, Trust), with weights: Experience – 20%, Expertise – 25%, Authoritativeness – 25%, Trust – 30%.

I look at text volume relative to page type (product card vs. blog article require different amounts), readability, signs of AI-written text without editing, YMYL signals (Your Money Your Life – pages that affect people’s money or health; requirements are stricter: presence of legal details, disclaimers), and the potential for the text to be cited in AI service responses.

The output – a content score, breakdown by E-E-A-T, citation readiness assessment, and flags where text is thin, filler, or written using template AI patterns without editing.

Structured Data

The third block – schema.org markup (the standard is very relevant in 2026. It helps search engines and AI understand page structure: where the price is, where the review is, where the author is).

I check validity of JSON-LD and microdata, presence of outdated markup types – FAQPage and HowTo, for example, search engines no longer show in rich snippets, although the markup is still technically valid. I check required fields and what markup opportunities the site isn’t using but could.

The output – a breakdown by markup blocks and ready-made JSON-LD snippets that can be handed directly to a developer for implementation.

Step 2. Deep Dive

If Step 1 is the foundation, Step 2 is a more focused and technical check. Here there are four directions, each with its own specifics.

Loading Speed and Core Web Vitals

Core Web Vitals (CWV) are a set of Google metrics that evaluate the real user experience on a page: speed of main content appearance, responsiveness to actions, and layout stability during loading.

I check using PageSpeed Insights and CrUX historical data (Chrome User Experience Report – real user data from Chrome, not lab measurements). Additionally, I break down the LCP metric components (time to render the largest visible element) to understand what exactly is slowing things down: server, fonts, images, or scripts.

If PageSpeed hits the request limit and there isn’t enough CrUX data for the site, I switch to manual analysis: a timing matrix via curl plus lab measurements in the browser. It’s slower, but it’s important to get the data regardless.

I look at LCP, INP (interaction delay) and CLS (how much elements shift during loading) at the 75th percentile of field data (I’ll explain what that is below) – this is the metric Google uses for ranking, not averages. Then time to first byte, render-blocking resources, image optimization, and load from third-party scripts.

What the 75th Percentile Means

The arithmetic mean distorts the picture: rare visits with gigabit internet or old smartphones shift the value. The percentile cuts off anomalies.

The 75th percentile shows: 75% of real visitor sessions load at or below the given threshold. If the 75th percentile LCP is 2.2 seconds, three-quarters of all visits to the site fall within that range.

The output – a performance score, status for each metric (passed or failed threshold), and a list of specific bottlenecks with expected impact from fixing them.

Sitemap

I analyze the sitemap.xml separately. For some reason, people increasingly forget about it, which is a shame. The file is very important.

I check XML structure validity, currency of last-modified dates (often the date is the same on all pages – a sure sign the sitemap is generated automatically and no one checks it), coverage of live pages by the sitemap and vice versa, and the 50,000 URL limit per file.

For sites with locations (e.g., branch networks), I apply stricter thresholds: if more than 30 location pages need attention – warning; more than 50 – hard stop and analysis before continuing the audit, because at that scale a template error multiplies across hundreds of pages.

The output – a validation report, a sitemap score, and a list of pages missing from the map or erroneously included.

Visual and Mobile Check

Here I go to the site manually and check both desktop and mobile versions – and I check the mobile version specifically by spoofing the User-Agent to iPhone; some sites serve different layouts depending on who’s asking, and without spoofing you might not see the real mobile picture.

I check the above-the-fold area (no scroll): whether the H1 heading is visible, whether there’s a call-to-action button, whether anything is covering the main visual block. I look at font size – smaller than 12–14 pixels is hard to read on a phone. I check touch target sizes for buttons – less than 44 pixels is an error. I look for horizontal scroll on mobile, images without set dimensions (causing page layout shifts during loading), and overlapping elements.

The output – a verdict on mobile responsiveness based on concrete facts, not general impressions, and a visual score.

Server Logs

This block I only do if the client provides access to server logs – without it there’s nothing to analyze. But if access exists, it’s arguably the most valuable part of the audit because it shows things invisible in Search Console or Metrika.

I look at real crawl budget usage – what percentage of requests to the server come from Googlebot and YandexBot, which pages bots visit more often, which they ignore. I look for 404 errors in logs that don’t appear in search engine reports – often these are old links people still visit. I check load from transitions and unusual requests to paths like /.env or /login – that’s already a security issue, not just SEO.

Step 3. Wave by Indicators

Step 0 determines which additional checks even make sense to run. Analyzing keyword cannibalization on a five-page business card site is a waste of time. So Step 3 is a set of modules that are triggered selectively based on reconnaissance signals.

Readiness for Citation in AI Services

I run this module almost always, because almost any site owner in 2026 should want their content mentioned in ChatGPT responses, Perplexity, and AI search results. I check robots.txt to see whether access is allowed for bots of these services – GPTBot, OAI-SearchBot, ClaudeBot, PerplexityBot.

I look at the llms.txt file – important to note, this file is not a lever of influence because major search engines simply ignore it when generating responses. More weight is given to how citable the page text is in structure – short, self-contained fragments in the range of 130–170 words are better extracted by the model in pure form than long paragraphs with multiple nested thoughts. Plus I look at brand authority signals – mentions on Wikipedia, Reddit, VC.ru, Habr, Dvach (funny as it sounds, for Russian content it’s almost like Reddit for the rest of the world), Pikabu, and of course YouTube.

The output – a readiness score across several dimensions and individual ratings per platform: how the site is likely to be represented in Google overviews, ChatGPT responses, and Perplexity.

Backlink Profile

I check backlinks using a cascade of sources – first the quick free CommonCrawl, and if it doesn’t provide enough data, then the paid ahrefs API. Plus data from Webmaster and Search Console.

Then I cross-reference data from different sources and check link liveness with a separate script – a dead link to a site that’s long gone provides no benefit; it’s dead weight and ballast.

I look at the number of unique domains linking to the site, distribution of these domains by authority, naturalness of anchor text (if 80% of links use the same phrase – that’s suspicious), share of toxic links, ratio of follow to nofollow links, and geographic relevance of donors.

Cannibalization and Content Structure

This block is enabled if the site has a blog or a pillar article structure. I collect the semantic core, analyze texts, look for keyword cannibalization, pages in the “near top” zone (positions 4–20), gaps between a page’s title and the keywords it’s actually ranking for, and share of voice via CTR curve by position.

The output – a plan for query clusters, a “hub and spokes” structure, and an analysis of specific cannibalization cases with a recommendation on which page to prioritize.

E-Commerce

For sites with product cards, I run separate checks. I validate product markup correctness, card image dimensions, search for hidden 404 errors (when a page returns 200 but the content is template-like, as if the product doesn’t exist), double H1 on a page, template meta tags without uniqueness, absence of a related products block, and checking the price.xml file – it may be allowed in robots.txt but not physically exist on the server.

The output – an e-commerce score and breakdown of specific cards with findings.

Multilingual and GSC/GA4 Data

If the site operates in multiple languages or locales – I check hreflang (the attribute that tells search engines which language version of a page is for which audience). If the client has access to Google Search Console and Google Analytics 4: I look at queries, check indexing of specific URLs, pull traffic reports. Without access to these services, this part simply doesn’t run – guessing based on other people’s data is pointless.

Step 3.5. Text Analytics: Only for Traffic Pages

I’ve separated out text analytics because I have a strict rule here – don’t run it on the entire site. There’s no point: if a page doesn’t get traffic and isn’t in the top queries, analyzing word frequency on it won’t help with prioritization.

I take a list of pages from the Google Search Console search queries report or from the top visited pages in Metrika data. For these pages, I calculate word frequency and bigrams – pairs of consecutive words.

The point of this step is simple: by the frequency profile of a page, you can see which semantic clusters it’s actually written for, whether there’s a bias toward keyword insertion without organic context, and whether there’s obvious filler – placeholder words that inflate text volume without benefit to the reader.

The output – a frequency profile for each top page, which feeds into the content block of the audit from Step 1. Often this is where a mismatch surfaces: a page is formally written “for the query,” but the actual focus is skewed to a related topic, and the target phrase is mentioned once in the first paragraph and never again.

Step 4. Synthesis: How Dozens of Reports Become One Document

This is where the main part of my work begins – not running scripts, but assembling the result into a single document that can be read and acted upon.

I read the full versions of all intermediate reports from scripts. I compile the final document: a summary table of scores from 0 to 100 for each audit area and an action plan divided into three levels – critical, warning, passed successfully.

Separately, I verify any mentions of recent changes in algorithms and search engine requirements against an up-to-date list of primary sources that I keep current – every fact in the report is backed by a link to the specific page it came from.

The finished report is saved in a roughly structured format and run through an LLM.

How Long It Takes and What the Client Gets

The entire pipeline from reconnaissance to final report takes, based on experience, from a few hours to a couple of days – depending on site size, number of modules triggered in Step 0, and whether there’s access to server logs and analytics services. A business card site of a dozen pages goes through all three mandatory blocks quickly. A large e-commerce site with a blog, pickup points, and a history of previous audits – that’s a completely different time investment.

At the end, the client receives not a list of twenty “make it better” items copy-pasted from Lighthouse, but a document with scores for each area, a prioritized action plan, and for each item – the rationale, dependencies, failure criterion, and monitoring metric. That’s the difference between “your site is slow” and “LCP exceeds 4 seconds due to compressed images in the header; expected effect after compression – return to the green zone at the 75th percentile within a few weeks.”

If after this breakdown you still have doubts – I can run the entire scenario live on a specific domain; the tools are at hand. Send me the site address, and the analysis will be with real numbers, not abstract examples from the article.