Technical SEO competitor research: build a weighted scoring rubric first

What to take away

Decide first whether you are scoring rivals to win new work or to improve your own site. The rubric differs, and so does the evidence you need.
Build a weighted scoring table before you open any crawl tool. It keeps the research comparable and defensible when a client or director challenges a finding.
Verify that the crawler hitting your test pages is genuinely Google's before you treat log data as evidence.
Check whether a rival's content sits inside online safety duties and advertising sanctions, because those shape what you can safely recommend copying.
Pair crawl data with what buyers actually say. The technical SEO audience research guide for England sets out the wider method for that.

Start from the decision, not the tool

Competitor research fails when it begins with a crawl export and works backwards. Start with the choice you must make. Are you pitching against two named agencies, or trying to beat a rival's category page, where the ecommerce audit checks apply?

A pitch needs evidence of what the rival does well and where their delivery looks thin. A category-page fight needs indexation, internal linking and page-speed comparisons. Different questions, different data.

Write the decision at the top of the document. Every row in your rubric should help answer it, or be deleted.

Name the person who will act on the findings, and the date they need them. A rubric built for a pitch in a fortnight looks different from one built for a quarterly refresh.

Build the scoring rubric

Weight each factor before you collect anything, as the tools and providers guide does for supplier choice, so the weights cannot drift towards whichever rival you already prefer. Score each factor from 1 to 5 and multiply by the weight.

Weighted scoring rubric factors

Crawlability and indexationweight 3
Site architectureweight 2
Core Web Vitalsweight 2
Structured dataweight 1
Content governanceweight 2
Regulatory exposureweight 1

Agree those weights with whoever owns the decision before scoring starts, and apply the same set to every rival. Changing weights mid-way turns a comparison into a preference.

Weight

Crawlability and indexation: 3
Site architecture: 2
Core Web Vitals: 2
Structured data: 1
Content governance: 2
Regulatory exposure: 1

What you score

Crawlability and indexation: Indexed URLs, canonical use, robots directives
Site architecture: Click depth to money pages, orphan pages
Core Web Vitals: Field data for key templates
Structured data: Valid markup on product and article templates
Content governance: Named owners, review dates, update history
Regulatory exposure: Advertising claims, user-generated content duties

Record the date and tool for every score. A finding without a date is an opinion.

Define what each score means

Score against written anchors for the bottom (1), middle (3) and top (5) of each factor, so every reviewer reads the scale the same way.

Crawlability and indexation:1 is key templates blocked or noindexed; 3 is indexed with duplicate URLs; 5 is all indexed with consistent canonicals.
Site architecture:1 is money pages over five clicks deep or orphaned; 5 is within three with no orphans.
Core Web Vitals:1 is all three metrics failing at the 75th percentile; 3 is one failing; 5 is all passing.
Structured data:1 is none or invalid; 5 is valid on all product and article templates.
Content governance:1 is no owners or dates; 5 is owner, review date and history on all.
Regulatory exposure:a higher score means lower exposure. 1 is unsubstantiated claims or unmoderated user content; 5 is substantiated claims and a moderation route.

Worked example with weighted totals

These scores are invented to show the arithmetic. The weights sum to 11, so the maximum total is 55. In table order, Rival A scores 4, 3, 5, 2, 3 and 4, and Rival B scores 3, 5, 2, 4, 4 and 3.

Rival A:(4 × 3) + (3 × 2) + (5 × 2) + (2 × 1) + (3 × 2) + (4 × 1) = 40, or 73% of the maximum.
Rival B:(3 × 3) + (5 × 2) + (2 × 2) + (4 × 1) + (4 × 2) + (3 × 1) = 38, or 69%.

The two-point gap is inside the three-point tie band described below. Report a tie, with Core Web Vitals (5 against 2) as the largest gap.

Verify your crawl evidence

Log files and crawl data only count if you can show who generated them. Google publishes a method for verifying requests from Google crawlers using reverse DNS and IP ranges. Run that check before you cite a single hit.

Verify Google crawler requests

Run reverse DNS check on the crawler IP
Confirm IP falls in Google's published ranges
Keep verification output with the report
Date any third-party index export you cite
Note templates your own crawler could not reach

Spoofed crawlers inflate crawl-budget findings and make a rival look worse than they are. A five-minute verification step protects the whole report.

Keep the verification output with the report so a reviewer can repeat it. Where you lean on a third-party index rather than your own crawl, say so and date the export.

You should also note which templates your own crawler could not reach. Missing data is a finding, not a gap to hide.

Check the governance layer

Technical findings often imply a content recommendation, and content carries duties. The Online Safety Act 2023 sets duties for user-to-user and search services, and the GOV.UK Online Safety Act collection explains them. Cite the Act when a rival's community features are part of your comparison.

The Advertising Standards Authority publishes sanctions it can apply, which matters when a competitor's claims are the reason they outrank you.

Flag exposure rather than pretending it is a technical bug. It changes who owns the fix.

Test your rubric against real buyers

A rubric scores websites. It does not tell you why a buyer chose one. Run short calls with people who recently picked a supplier, and the Technical SEO customer interviews in England piece shows how to structure those conversations.

Use the interviews to reweight your factors. If buyers mention page speed and ignore structured data, your weights were wrong.

What to do when scores conflict

When two weighted totals fall within three points of each other, about 5% of the 55-point maximum, do not force a winner. Report the tie and say which factor would break it, then go and collect that data.

How to present the table

Show raw scores and weighted totals on separate lines. Readers trust a table they can recompute.

Common questions

How many competitors should I score?

Three to five is workable for a single decision. More than five spreads your evidence too thinly, and the rubric stops discriminating between sites.

Can I use only automated tools?

No. Tools give you crawl and speed data, but governance, ownership and buyer reasoning need manual checks and conversations.

How often should the rubric be refreshed?

Refresh weights when the decision changes, and rescore at least quarterly. Templates and rivals move, and stale scores mislead.

What if a rival blocks my crawler?

Record the block as a finding, note the date, and fall back to public data. Do not score what you could not observe.