Claire did the responsible thing. She runs a homeware store, she'd read enough about AI visibility to know it mattered, and she didn't have twenty spare hours — so she delegated. An agency took the brief, and six weeks later the report came back with the number every deck loves: AI Visibility Score: 87/100, up from 74.
Thirteen points. She paid the invoice.
Then she ran the category question test herself — three customer questions, two engines. Absent before, absent after. She pulled a product page the way a crawler does: same hardcoded InStock as always. Whatever went from 74 to 87, it wasn't anything a machine reading her store could detect.
The score wasn't lying, exactly. It was doing something worse: it was unfalsifiable. No way to see what was counted, no way to re-run it, no way to disagree with it. And a number you can't disagree with isn't measurement — it's decoration.
This is the file about what a real score looks like. Not because scores are the goal — citations and transactions are the goal — but because you cannot delegate, prioritize, or track what you cannot measure, and almost everything sold as "AI visibility scoring" today fails the three tests that make a measurement worth having.
The vanity score problem
A single number summarizing "AI visibility" hides exactly the things you'd need to act.
It hides which judge. The four pillars are four different machines asking four different questions — rankable, quotable, retrievable, operable. One blended number can't tell you whether you're failing the ranking system or the agent, which means it can't tell you what work to do. A store at "87" might be superb for search and inoperable for agents; another "87" might be the reverse. Same number, opposite priorities.
It hides which layer. Access, delivery, and legibility fail independently, and the fix for each lives in a different part of your stack. A number that doesn't localize the failure delegates nothing.
And it hides the grader's incentives. Tools tend to score what they can see and — less charitably — what they can sell the fix for. When the entity measuring you also invoices for improving the measurement, grade inflation isn't a risk, it's the business model. This is Goodhart's law with a dashboard: the moment the score becomes the target, the score stops measuring the thing.
None of this means scoring is hopeless. It means scoring has requirements.
What deserves a number
Three principles separate measurement from decoration.
Checks derive from documented machine requirements. Every check should trace to something a machine actually demands — a crawler's published documentation, a structured-data specification, a feed requirement, a protocol schema. "Your description should be more engaging" is an opinion. "Availability must be a valid ItemAvailability value bound to inventory" is a requirement with a source you can read. If a check can't cite its spec, it's taste wearing a lab coat.
Checks are binary at the check level. Each individual check passes or fails — the identifier ships in the raw response or it doesn't; the sold-out product declares OutOfStock or it doesn't. Nuance belongs in which checks exist and how they're weighted, never in fuzzy partial credit that makes results unarguable. Binary checks are what make a score re-runnable by someone who doesn't trust it — which is the whole point.
Checks verify at the wire. Measured against what machines receive, not what dashboards declare. Every earlier file in this series is secretly this principle: the app said schema was installed and the response said otherwise. A score built on declared-layer evidence inherits every gap class we've catalogued. curl is the auditor; the dashboard is a witness with motives.
Eligible, implemented, weighted
Now the shape of the score itself — three facts that most reports collapse into one, and that must stay separate.
Eligible: does this check apply here? Not every check applies to every store or product. A GTIN check behaves differently for a reseller of manufactured goods than for a maker of one-off ceramics with no barcodes in the universe; category-specific schema requirements apply to some catalogs and not others. Scoring a store against inapplicable checks corrupts the number in both directions — penalizing what couldn't apply, or padding the denominator with free passes. Eligibility is a finding about your store, and an honest report states it before scoring anything.
Implemented: of the eligible checks, which pass — at the wire. This is the score's factual core, and because every check is binary and spec-derived, "implemented" comes with proof: here's the request, here's the response, here's the requirement it satisfies or violates. You can re-run it. You can disagree with it and be shown the bytes.
Weighted: not all passing is equal. A missing priceCurrency and a missing breadcrumb are not the same size of problem, and any blended score silently asserts weights whether it admits to them or not. The honest version admits it — and derives the weights from the same documentary evidence the checks came from: what engine documentation emphasizes, what feed specifications mandate versus recommend, what agent protocols require as fields versus suggest as enrichment. Weight derivation is slow, unglamorous work, which is exactly why it's rare — and why we'd rather hold a number back than publish one whose weights we can't show receipts for. A score's weights are its editorial opinion; the receipts are what make the opinion citable.
Three facts, three different sentences in a report: these checks apply to you; these pass, with proof; here's how much each matters, and why. Collapse them and you get Claire's 87 — a number with no argument inside it.
FIG.02 — Three facts, not one number. Most reports collapse these; an honest score keeps them separate sentences.
The four things a real score lets you do
The test of a measurement is what it enables. A score built on the triad gives you four capabilities a vanity number can't:
Falsify it. Re-run any check yourself and get the same verdict, or a documented reason the verdict changed. Claire couldn't disagree with 87. You should always be able to disagree with a score — and lose the argument on evidence.
Localize with it. Every failing check names its pillar and its layer. "You're failing" is anxiety; "your availability field is a template literal, which fails an AIO-critical check at the delivery layer" is a work order.
Delegate with it. This is the article's title earning its keep. A binary, wire-verified check is a completable task with a built-in definition of done: hand it to a developer, an agency, or a VA, and done means the check flips on re-run — not that a report says effort happened. Claire's thirteen points were undelegatable precisely because nothing underneath them was checkable.
Track with it. The same ruler, applied the same way, over time. Which requires the ruler itself to be versioned: when checks or weights change — and they will, because the machine requirements they derive from change — the change gets dated and disclosed, so a moving score means your store moved, not the goalposts.
FIG.03 — What a real score lets you do. The test of a measurement is what it enables, not what it summarises.
Build your own micro-scorecard: eight checks, fifteen minutes
Everything needed for a first honest score is already in this series. Eight binary checks, each verifiable at the wire, each traceable to an earlier file. Mark eligible, then pass/fail:
- robots.txt permits OAI-SearchBot, Claude-SearchBot, and PerplexityBot — FILE 01
- A product URL returns 200 when fetched with a search-bot user-agent (no WAF override) — FILE 01
application/ld+jsoncount ≥ 1 in the raw response of your best seller — FILE 02- Markup price matches the price a buyer pays, with
priceCurrencypresent — FILE 02 - A sold-out product's delivered availability says OutOfStock — FILE 04
- Identifiers (gtin/mpn + brand) ship in the response where they exist in admin — FILE 04
- Offer count equals variant count on your most-varianted product — FILE 04
- At least one of your three category questions cites you in at least one engine — FILE 03
FIG.04 — The eight-check micro-scorecard. Binary, wire-verified, each traceable to a file — scored as implemented over eligible.
Score it as implemented-over-eligible — "6 of 7 eligible" tells the truth in a way "86" never could. No weights on purpose: eight checks don't need them, and unweighted honesty beats weighted hand-waving. What this can't do is scale — eight checks on one product is an afternoon; every check on every product, continuously, re-verified as themes and apps and specs change, is a system. Building that system, on exactly the triad above, is our actual job — and the standard this article holds us to is the one you should hold anyone to: every check cites its spec, every verdict shows its bytes, every weight shows its derivation.
One loose end, and it's the next file's whole subject: real reports don't just score problems — they claim some are fixable. That word is doing enormous work. What must be true before a tool is allowed to say it?
Sources & further reading
- Google — Structured data general guidelines (requirement vs. recommendation language): https://developers.google.com/search/docs/appearance/structured-data/sd-policies
- schema.org — Product, Offer, ItemAvailability: https://schema.org/Offer
- OpenAI — Crawler documentation: https://platform.openai.com/docs/bots
- Anthropic — Crawler documentation: https://support.claude.com
- Rank Sniper Field Notes — FILE 01: /blog/robots-txt-ai-crawlers-shopify · FILE 02: /blog/ai-agent-declared-vs-delivered-product-page · FILE 03: /blog/shopify-store-invisible-ai-buyers · FILE 04: /blog/product-attributes-ai-systems-refuse-to-guess · FILE 05: /blog/seo-aeo-geo-aio-four-pillars
Rank Sniper — Field Notes. Catalog legibility and AI-visibility verification for Shopify. We verify what AI agents actually receive from your store; we don't generate content and hope. Verifier, not generator.
