How we calculate your AI readiness score
A 100-point starting score with defined deductions for each signal we can detect from public data. No hidden math, no rank inflation. This page is the authoritative source for our methodology.
The scoring model in one paragraph
Every site starts at 100. We deduct points for each check that fails, weighted by how much that check actually affects whether an AI crawler can access and understand your page. The final score is capped at 0 and 100.
High-severity checks (crawlers blocked, JS-only homepage, indexing blocks) can take 20–30 points each. Low-severity checks (missing llms.txt, agents.json, Content-Signal) take 2–5 each because they're emerging conventions with limited adoption.
Every check, every weight
The table below shows the current version of every check we run, its point weight, and its severity. These values live in lib/checker.js and change only in tagged releases.
What we check: We fetch your robots.txt and test it against 50+ known AI crawler user-agents (GPTBot, ClaudeBot, PerplexityBot, Google-Extended, etc.). If more than half are Disallowed, this fires.
Why it matters: This is the single biggest thing that stops AI from seeing your site. If ChatGPT-User is blocked, ChatGPT literally cannot fetch your pages when a user asks about you.
What we check: We fetch your homepage HTML without executing JavaScript and measure the text-only content length. If it's below thresholds (300 chars, or 600 with SPA signals like <div id="root">), this fires.
Why it matters: Most AI crawlers don't execute JavaScript. A single-page app that renders content client-side looks like an empty shell to them.
What we check: If between 20% and 50% of AI crawlers are blocked, weight scales with the ratio. Below 20%, we deduct 5 points per blocked crawler.
Why it matters: You may have deliberately blocked one training-only bot (like GPTBot) while allowing others (like ChatGPT-User). We still surface this so you can confirm the intent.
What we check: We parse the homepage <meta name="robots"> and X-Robots-Tag HTTP header for noindex, noai, or nosnippet directives.
Why it matters: Page-level and header-level blocks override a permissive robots.txt. This is a common accidental self-inflicted wound.
What we check: We scan the homepage for <script type="application/ld+json"> blocks and validate they parse as JSON.
Why it matters: Structured data gives AI systems clean facts about your site (name, type, products, reviews) instead of forcing them to infer from raw HTML.
What we check: We check for a <meta name="description"> tag on the homepage with at least a few characters of content.
Why it matters: AI systems often use the meta description as the first summary of what a page is about. Missing it means AI picks a random sentence.
What we check: We look for a Sitemap: directive in robots.txt pointing to an XML sitemap.
Why it matters: Crawlers discover pages via internal links + sitemaps. Without one they may miss pages that aren't linked from the homepage.
What we check: We fetch /llms.txt and check that it's present and parseable Markdown.
Why it matters: Emerging convention — absence does not mean AI can't read your site. Google's official guidance says no special optimizations are needed for AI Overviews. But Perplexity, Anthropic, and Cloudflare do honor llms.txt when present, and it's cheap to add.
What we check: We fetch /.well-known/agents.json to check for AI-friendly endpoint declarations.
Why it matters: Emerging standard — Cloudflare and a few others publish one. Low weight because adoption is still small.
What we check: We parse robots.txt for a Content-Signal line declaring AI training / search / input preferences.
Why it matters: New robots.txt directive that removes ambiguity about your AI content preferences. Very low weight — adoption is still tiny.
What we deliberately don't measure
Being honest about what the score doesn't tell you is as important as showing what it does.
Whether AI actually cites you
The check tells you whether AI can technically read your site. It does NOT tell you whether ChatGPT, Claude, or Perplexity actually mentions you when customers ask relevant questions. That's what our LLM Monitor measures — it queries real AI providers with real prompts and tracks mention rate + brand position over time.
Ranking or authority
AI systems weigh domain authority, backlinks, freshness, and query-specific relevance. The check does not measure those. A perfect 100 doesn't guarantee AI citations — it just removes the technical obstacles.
Content quality
We don't judge whether your content is good, useful, or accurate. We check whether it's readable by machines.
What the paid tiers add — and why
The free check tells you if AI can read your site. What paid tiers add is ongoing measurement: whether AI actually cites you, tracked over time, across multiple providers. That's the difference between "are the doors unlocked" and "are customers walking in."
Free
- All 6 free web tools (llms.txt, schema, robots.txt — generators and validators)
- Chrome extension (unlimited)
- Public shareable report card
Best for: Trying the tool, checking one or two sites you own
Pro
- Weekly OR daily auto-schedule (Monitor runs on its own)
- Per-provider trend charts + share-of-voice history
- 5 API keys for programmatic checks
- Full downloadable reports
- Priority support
Best for: Solo consultants and indie hackers who monitor 1–5 client or their own site's AI visibility
Agency
- Everything in Pro
- 20 competitors tracked per project (share-of-voice vs specific rivals)
- Bulk CSV export for client reports
- Higher API rate limits
Best for: Marketing / SEO agencies managing 10–30 client sites with recurring AI-visibility reporting
Known limitations
- We test robots.txt directives, not enforcement. A crawler could ignore your robots.txt (well-behaved ones don't).
- We fetch as an anonymous browser, then re-fetch as each AI bot. Some sites serve different content based on IP range or fingerprinting — we can't detect that from a single vantage point.
- The homepage is one page. Structural signals (robots.txt, sitemap, schema types) apply site-wide, but content signals (readable text, meta) are checked on the homepage only.
- Google AI Overviews query rendering is emulated via SerpAPI in the Monitor. We can't query Google's AI Mode directly; SerpAPI is the closest proxy available.
- Sentiment classification uses Claude Haiku per response. If our Anthropic API key hits its quota we fall back to keyword-based sentiment (marked as such in the UI).
Data handling
What data do you store when I run a check?
The URL you submitted, the score, per-check results, and (if you're signed in) a link to your account. We do NOT store the full HTML content of your pages — only extracted signals (robots.txt presence, meta tags, structured-data blocks).
How long do you retain check data?
Anonymous checks: retained indefinitely as public report cards (URL is public). Signed-in checks: stored in your history and deletable from your dashboard. Monitor query results: retained per your subscription; deleted 30 days after account cancellation.
Are shareable reports public?
Yes — the /report/[id] URL is public and can be shared. If you don't want a public report for a domain you own, email crawlreadiness@gmail.com and we'll remove it within 5 business days.
Where is the service hosted?
Vercel (application) + Supabase (database + auth). Tundrastone is an Ontario, Canada business.
How do I report a security issue?
Email crawlreadiness@gmail.com with 'security' in the subject. We aim to acknowledge within 48 hours.
Changes to this methodology
When we change how the score is calculated, we'll note it here with the date and the reason. The scoring model version is currently v1.2 (September 2026).
Questions or think we're weighing something wrong? Email crawlreadiness@gmail.com.