Machine Readiness Audit · machinereadiness.com
Machine Readiness Audit

machinereadiness.com

AI Ready

Open to AI crawlers, with the metadata to be understood.

Audited
October 28, 2026
Pages measured
4 of 4 found (from the sitemap)
Measured against
the Machine Readiness standard
Prepared by
Crawl Readiness
100out of 100 · A

Summary

machinereadiness.com is rated A AI Ready with a score of 100 out of 100 on the site-level check. 4 pages were measured: 4 readable without JavaScript, 3 with structured data, 0 with a markdown copy for agents. The access policy reads "Open to AI". Your robots.txt says "open to ai", your other signals agree, and the server does what it says for the 4 crawlers we tested. 35 of 35 AI search and answer crawlers can read the site. 1 improvement is listed in the action plan, starting with 1 of the 4 pages checked have no structured data.

4
Pages measured
of 4 found
4 of 4
Readable without JavaScript
3 of 4
With structured data
35 of 35
AI crawlers with access
Open to AI
Access policy
enforced as written

What this report is: every page we could find measured as a crawler receives it, the site-level signals and the access policy verified at the server, the five files that fix what is wrong written for your platform, one Answer Analysis on live AI search, and a weekly re-check for thirty days. Nothing here promises a mention or a rank; it says what machines can read, and what your site tells them they may do.

Access policy

What the site says machines may do with it, declared by robots.txt, and whether the server does what the rules say. Your public statement reads "AI search: yes · AI training: yes".

Your robots.txt says "open to ai", your other signals agree, and the server does what it says for the 4 crawlers we tested.

CrawlerPurposerobots.txt saysServer answeredOutcome
GPTBottrainingallowHTTP 200served, as allowed
OAI-SearchBotsearchallowHTTP 200served, as allowed
ClaudeBottrainingallowHTTP 200served, as allowed
PerplexityBotsearchallowHTTP 200served, as allowed
  • Content-SignalContent-Signal agrees with the rules
  • TDM reservationno TDM reservation, and none expected for a policy that allows training
  • Page-level directivesno noai directive on the homepage

Every page

Each page as an AI crawler receives it, without running JavaScript: whether the content is in the HTML, whether it carries structured data and a description, whether its form fields are labelled for agents, and whether the server offers a markdown copy when asked for one. Tokens are what an AI reads to take the page in, at about four characters per token.

PageReadableJSON-LDDescriptionFormsMarkdownTokens
/ Machine Readiness — readable and usable by machines, proven✓✓✓·✗7,847
/standard The Machine Readiness Standard, version 1.0 · Machine Readiness✓✓✓·✗8,462
/crawlers The AI crawler registry · Machine Readiness✓✗✓·✗12,176
/permission The permission API: may I use this page for this? · Machine Readiness✓✓✓·✗9,265

Readiness profile

The four areas the score counts on the homepage: how easily AI systems can reach, read and understand the site.

Site-level signals

  • robots.txt reachableThe file was read and its rules applied per crawler.
  • Homepage readable without JavaScriptThe HTML carries the content; crawlers that do not run JavaScript still see it.
  • Structured data (JSON-LD)Machine-readable facts about the organisation or content are present.
  • Meta descriptionA summary AI systems can quote is present.
  • Open Graph tagsLink previews and citations have a title and description to use.
  • No page-level AI or index blockNeither the robots meta tag nor the X-Robots-Tag header blocks indexing.
  • Sitemap listed in robots.txtCrawlers are told where the full page list lives.
  • llms.txt · not scoredA Markdown summary for AI systems is published at the site root.
  • agents.json · not scoredPublished under /.well-known.
  • Content-Signal directive · not scoredDeclares search=yes, ai-input=yes, ai-train=yes.
  • RSS or Atom feed · informationalNo feed advertised.
  • Markdown copy for AI agents · not scoredAsking the homepage for markdown (Accept: text/markdown) returns HTML. Few sites offer a markdown copy yet.

AI crawler access

What the site's robots.txt says to each AI vendor's crawlers. A blocked crawler cannot fetch pages for that vendor's answers. Every AI search and answer crawler checked is allowed.

VendorFeedsStatusCrawlers
OpenAIChatGPT answers and ChatGPT searchAllowed✓ GPTBot✓ OAI-SearchBot✓ ChatGPT-User
AnthropicClaude answers and Claude's web searchAllowed✓ ClaudeBot✓ Claude-User✓ Claude-SearchBot✓ Claude-Web✓ anthropic-ai
Google AIGemini and Google's AI features (Search itself uses Googlebot)Allowed✓ Google-Extended✓ GoogleOther✓ Gemini-Deep-Research✓ Google-NotebookLM✓ Google-CloudVertexBot✓ GoogleOther-Image✓ GoogleOther-Video
PerplexityPerplexity answersAllowed✓ PerplexityBot✓ Perplexity-User
Microsoft CopilotCopilot answersAllowed✓ Copilot✓ CopilotNative
Meta AIMeta AI across Facebook, Instagram and WhatsAppAllowed✓ meta-externalagent✓ meta-externalfetcher✓ FacebookBot
Apple IntelligenceSiri and Apple IntelligenceAllowed✓ Applebot-Extended
xAI / GrokGrok answersAllowed✓ GrokBot✓ Grok-DeepSearch✓ xAI-Grok
AmazonAllowed✓ Amazonbot
MistralAllowed✓ MistralAI-User
DeepSeekAllowed✓ DeepSeekBot
19 other vendorsAllen AI, Andi, Bytedance, Cohere, Common Crawl, Diffbot, DuckDuckGo AI, Elicit, Genspark, Huawei, HuggingFace, Kagi, Neeva, Phind, Quora / Poe, Sider, Timpi, Yandex AI, You.comAllowed✓ all 22 crawlers

The files, ready to install

The files below were written for machinereadiness.com from what the audit found. None of them changes your policy: the audit read the stance your robots.txt states today (“Open to AI”: AI search: yes, AI training: yes) and wrote each file to say the same thing, for every AI crawler we track, in the place AI systems look for it. To take a different stance, the robots.txt generator writes all five from one choice.

The site is served by Vercel, so all five files are plain files in the project and a deploy publishes them. All five are plain files, so this is a short job for whoever looks after the site; the box at the end of this section is written for them.

robots.txt · /robots.txt

What it does. Tells each AI crawler, by name, whether it may read the site. Most crawlers check it before every visit.

Why it matches your policy. Your existing rules stay exactly as they are; none of them names an AI crawler. Below them, an AI section applies your policy to all 51 AI crawlers we track: 51 allowed, 0 blocked. The Content-Signal line under the rules states the same policy in one line, for crawlers that read it. Checked before it was handed to you: the 1 path rule under your "User-agent: *" and every path your file names (1 in all) give each of the 51 AI crawlers the same answer or a stricter one with this file; nothing you closed is opened.

User-agent: *
Allow: /

# The standard's own site is open to AI search, answers, agents and training.

Sitemap: https://machinereadiness.com/sitemap.xml

# ---- AI crawler policy (managed by Crawl Readiness) ----
# Generated at https://www.crawlreadiness.com/tools/robots-txt

# 51 AI crawlers are allowed by your "User-agent: *" rules and follow them, so they are not listed here.

# Content-Signal declares your training / search / AI-input preferences.
User-agent: *
Content-Signal: search=yes, ai-input=yes

Sitemap: https://machinereadiness.com/sitemap.xml

# ---- end AI crawler policy ----

How to install it on Vercel

  1. Replace the site's robots.txt with the file above. Your existing rules are unchanged; the AI section sits below them. On Vercel the root is the project's public folder (public/ in a Next.js project; the static folder of other frameworks); push the change and the next deploy publishes it.
  2. Check: open https://machinereadiness.com/robots.txt in a browser. The AI crawler section should be there, at the end. Then run the free check at crawlreadiness.com: the crawler table should match the policy.

llms.txt · /llms.txt

What it does. A plain-text introduction to the site for AI assistants: what it is, and the 4 pages that matter most with a line on each, from the sitemap.

Why it matches your policy. It helps an assistant describe the site accurately. It grants nothing and blocks nothing, so it fits any policy.

# Machine Readiness

> The Machine Readiness standard: what it means for a website to be readable by AI crawlers, usable by AI agents and verifiably permitted for each use. The crawler registry, the permission check and…

This file helps AI assistants understand this site. It lists the pages we consider most useful for machine consumption. Learn more at https://llmstxt.org

## Overview
- [Machine Readiness — readable and usable by machines, proven](https://machinereadiness.com/): The Machine Readiness standard: what it means for a website to be readable by AI crawlers, usable by AI agents and verifiably permitted for each use. The crawler registry, the permission check and…

## Standard
- [The Machine Readiness Standard, version 1.0 · Machine Readiness](https://machinereadiness.com/standard): What a website must do to be readable by AI crawlers, have its access policy expressed and enforced, be usable by AI agents, and have all of it verified from outside. Four parts, twenty-one checks…

## Crawlers
- [The AI crawler registry · Machine Readiness](https://machinereadiness.com/crawlers): Every AI crawler token the Machine Readiness standard knows, by vendor and purpose: search index, AI answers, training, user-requested fetches. Human-readable here, machine-readable at /crawlers.json…

## Permission
- [The permission API: may I use this page for this? · Machine Readiness](https://machinereadiness.com/permission): One call: a page and a purpose in; what robots.txt, Content-Signal, Content-Usage, an RSL licence, the TDM reservation, ai.txt and the page's directives say out, with what the server did when asked…

How to install it on Vercel

  1. Save the text above as a file named llms.txt and put it in the root, next to robots.txt (the robots.txt step says where that is), so it opens at https://machinereadiness.com/llms.txt.
  2. Check: open https://machinereadiness.com/llms.txt. You should see the text itself: not a “page not found”, not a download. Hosts serve a .txt file as plain text on their own; nothing else is needed.
  3. Keep it current: when the main pages change, run the free llms.txt generator at crawlreadiness.com again and replace the file.

JSON-LD structured data · the <head> of every page

What it does. The facts about Machine Readiness: who it is, what it is and where things are, in the format search engines and AI systems read without guessing. The Organization and WebSite scripts go on every page; the others are templates for one kind of page.

Why it matches your policy. It describes the site; it does not grant or refuse access, so it fits any policy.

Organization · Who you are as a business. Tells AI systems your name, logo, socials, and how to reach you. Add to every page in the site's <head>.
<script type="application/ld+json">
{
  "@context": "https://schema.org",
  "@type": "Organization",
  "name": "Machine Readiness",
  "url": "https://machinereadiness.com",
  "logo": "https://machinereadiness.com/favicon.svg",
  "description": "The Machine Readiness standard: what it means for a website to be readable by AI crawlers, usable by AI agents and verifiably permitted for each use. The crawler registry, the permission check and the public record, by the makers of Crawl Readiness.",
  "contactPoint": {
    "@type": "ContactPoint",
    "contactType": "customer service",
    "email": "support@crawlreadiness.com"
  }
}
</script>
WebSite · The site itself, plus (if we found one) your search endpoint so AI assistants can offer a search box that queries your site.
<script type="application/ld+json">
{
  "@context": "https://schema.org",
  "@type": "WebSite",
  "name": "Machine Readiness",
  "url": "https://machinereadiness.com",
  "description": "The Machine Readiness standard: what it means for a website to be readable by AI crawlers, usable by AI agents and verifiably permitted for each use. The crawler registry, the permission check and the public record, by the makers of Crawl Readiness."
}
</script>
BreadcrumbList (template) · A starter breadcrumb schema. Edit the position/name/item entries to match each page's navigation trail. Add per-page, not sitewide.
<script type="application/ld+json">
{
  "@context": "https://schema.org",
  "@type": "BreadcrumbList",
  "itemListElement": [
    {
      "@type": "ListItem",
      "position": 1,
      "name": "Home",
      "item": "https://machinereadiness.com/"
    },
    {
      "@type": "ListItem",
      "position": 2,
      "name": "Category",
      "item": "https://machinereadiness.com/category/"
    },
    {
      "@type": "ListItem",
      "position": 3,
      "name": "Current page",
      "item": "https://machinereadiness.com/category/current-page"
    }
  ]
}
</script>
FAQPage (template) · Only add this on a page that ACTUALLY has visible FAQ questions and answers — Google penalizes fake FAQ markup. Replace the placeholder Q&As.
<script type="application/ld+json">
{
  "@context": "https://schema.org",
  "@type": "FAQPage",
  "mainEntity": [
    {
      "@type": "Question",
      "name": "What is your first question?",
      "acceptedAnswer": {
        "@type": "Answer",
        "text": "Your answer goes here. Keep it concise but complete — AI assistants may quote it verbatim."
      }
    },
    {
      "@type": "Question",
      "name": "What is your second question?",
      "acceptedAnswer": {
        "@type": "Answer",
        "text": "Your second answer goes here."
      }
    }
  ]
}
</script>

How to install it on Vercel

  1. Each script goes between <head> and </head> of the page's HTML. The Organization and WebSite scripts belong on every page, so put them in the shared layout or header template (a developer will know the file: layout.tsx, _document.js, base.html, header.php).
  2. The Article script is a template: use it only on article pages, filled with each page's own title, dates and author.
  3. Check: open the homepage, press Ctrl+U to view the page source, and search for application/ld+json; or paste the address into Google's Rich Results Test.

tdmrep.json · /.well-known/tdmrep.json

What it does. The machine-readable opt-out from text and data mining that EU law recognises: the file a crawler looks for when it wants the answer about training in one place.

Why it matches your policy. Not written for you. Your robots.txt does not reserve AI training, and a file that said “not reserved” would waive the EU opt-out from text and data mining in writing. That is your decision, not the audit's. If you decide to reserve training, publish the two lines below.

[
  {
    "location": "/",
    "tdm-reservation": 1
  }
]

How to install it on Vercel

  1. Create a folder named .well-known (with the leading dot) in the root, next to robots.txt, and save tdmrep.json inside it, so it opens at https://machinereadiness.com/.well-known/tdmrep.json. On Vercel the root is the project's public folder (public/ in a Next.js project; the static folder of other frameworks); push the change and the next deploy publishes it.
  2. Check: open https://machinereadiness.com/.well-known/tdmrep.json. You should see the two-line JSON, not a “page not found”.

agents.json · /.well-known/agents.json

What it does. A one-file introduction for AI agents: the site's name, what it does and where the summary lives.

Why it matches your policy. It describes; it does not grant or refuse access, so it fits any policy. Extend it with your API or your forms when you want agents to use the site, not only read it.

{
  "version": "1.0",
  "name": "Machine Readiness",
  "url": "https://machinereadiness.com",
  "description": "The Machine Readiness standard: what it means for a website to be readable by AI crawlers, usable by AI agents and verifiably permitted for each use. The crawler registry, the permission check and the public record, by the makers of Crawl Readiness.",
  "documentation": "https://machinereadiness.com/llms.txt"
}

How to install it on Vercel

  1. Save agents.json in the same .well-known folder, so it opens at https://machinereadiness.com/.well-known/agents.json.
  2. Check: open https://machinereadiness.com/.well-known/agents.json. You should see the JSON.
  3. Extend it with your API or your forms when you want agents to use the site, not only read it.

For whoever looks after the site

If someone else maintains machinereadiness.com, send them this section; it is a short job for them. The whole of it:

  1. Replace /robots.txt with the version in this report: the existing rules stay, the AI section is added below them.
  2. Add /llms.txt, plain text, from this report.
  3. Add the Organization and WebSite JSON-LD scripts from this report to the <head> of every page; the Article script only on article pages.
  4. Add /.well-known/agents.json from this report.

Answer Analysis

10 of your customers' questions, put to live AI search (Perplexity and Google AI Overviews) on October 28, 2026: which answers named or cited machinereadiness.com, and which pages the answers drew on instead. An answer counts when it says “Machine Readiness” or “machinereadiness.com”, or cites a page on machinereadiness.com. The questions were suggested from what the site says it does. Open the analysis →

0 of 20
answers named or cited machinereadiness.com
youtube.com
site the answers used most
21 times
12
other sites cited
across all answers
QuestionEngineNamed or cited youPages the answer used
What does it take for a website to be readable by AI crawlers?Perplexityno
Google AI Overviewsno
How can I make my website usable by AI agents?Perplexityno
Google AI Overviewsno
What is the best standard for AI crawler readiness?Perplexityno
Google AI Overviewsno
How do I verify whether AI crawlers can read my website?Perplexityno
Google AI Overviewsno
How can websites prove they are permitted for AI use?Perplexityno
Google AI Overviewsno
What tools check website permissions for AI crawlers?Perplexityno
Google AI Overviewsno
What is the difference between AI-readable and AI-usable websites?Perplexityno
Google AI Overviewsno
How do I prepare my website for AI search and agents?Perplexityno
Google AI Overviewsno
Which standards help websites work reliably with AI systems?Perplexityno
Google AI Overviewsno
How can I keep a public record of AI crawler permissions?Perplexityno
Google AI Overviewsno

What the cited sites publish that machinereadiness.com does not

The sites the answers used most, checked the same way as yours. The signals listed are the ones they have in place and your site lacks.

SiteCitedScoreThey have, you don't
youtube.com8×70 · JavaScript-Dependentnothing you lack
reddit.com6×65 · Restricted by Designnothing you lack
amicited.com4×100 · AI ReadyHomepage forms labelled for AI agents
searchengineland.com4×95 · Partly Checkednothing you lack
apify.com3×100 · AI ReadyMarkdown copy for AI agents
developers.google.com3×100 · AI ReadyHomepage forms labelled for AI agents
facebook.com3×47 · Restricted by DesignHomepage forms labelled for AI agents
linkedin.com3×57 · JavaScript-Dependentnothing you lack

Action plan

In priority order: the site-level findings from the homepage check, then what the page-by-page measurement adds. Each item names the finding and the change that resolves it.

  1. 11 of the 4 pages checked have no structured datamedium

    Add JSON-LD to each page. The Organization and WebSite scripts in the files section go on every page; articles, products and FAQs get a block of their own type on their own page.

What happens next

  • machinereadiness.com is re-checked every week until November 27, 2026. You get an email the day its verdict changes, with the findings that moved, and when its server or robots.txt starts treating one of the AI crawlers we test differently, once a second check confirms it. Install the files first, then watch the re-checks pick the changes up.
  • The Answer Analysis lives in your dashboard, where the questions can be edited; the Monitor plan runs it monthly.
  • The site-level check has its own shareable page: www.crawlreadiness.com/report/pdma7xu4.
  • Questions about a finding, or a page we should look at again: reply to the email this report came with.

How this was measured

Every page was fetched once as a crawler would fetch it, without running JavaScript. Crawler access is read from the site's robots.txt for the AI crawlers the standard lists; enforcement is tested by fetching the homepage as four of them and comparing the server's answer with the rules. The markdown copy is tested by asking each page for text/markdown. The Answer Analysis puts each question to live AI search once and records the pages each answer cites; the cited sites are checked the same way as yours. Methodology in full →