The API

Two calls do the work: the readiness check (is this site set up for AI the way its owner intends?) and the permission check (may this page be used for this purpose?). Both answer JSON and both work without a key at a daily limit; a free key from the dashboard lifts it. The same key works in the MCP server and the Chrome extension.

Keys and limits

Send the key in an x-api-key header. Without one, the readiness check allows 10 calls a day per network and the permission check 25. With one, checks count against the plan's monthly allowance (50 on Free, 1,000 on Monitor, 5,000 on Agency); permission calls are counted but not charged to it during the test period.

The readiness check

GET /api/check?url= runs the same check as the homepage: robots.txt for 51 AI crawler tokens, the access policy and whether the server enforces it, the homepage as a crawler receives it, the AI files, structured data, meta and header directives, and the fixes.

curl "https://www.crawlreadiness.com/api/check?url=example.com" \
  -H "x-api-key: YOUR_KEY"

The response, abridged:

{
  "url": "https://example.com",
  "score": 67,
  "grade": { "letter": "◆", "status": "Restricted by Design", "tone": "info" },
  "robots": { "summary": { "total": 51, "allowed": 34, "blocked": 17 }, "results": [ { "bot": "GPTBot", "allowed": true, "purpose": "training", "scored": false }, "…" ] },
  "accessPolicy": { "label": "Closed to AI", "integrity": "consistent", "enforcement": [ "…" ] },
  "homepage": { "…": "…" },
  "fixes": [ { "severity": "high", "issue": "…", "fix": "…" }, "…" ],
  "shareId": "dju9v8nw"
}

robots.results[] carries every crawler with its purpose and whether it is scored: training-only crawlers are listed but never counted, so blocking them does not lower the score. accessPolicy is what the site says machines may do and whether the server did it for four representative crawlers. Fixes marked extra: true are optional. shareId is the public report at /report/<id>. The scoring is on the methodology page.

The permission check

GET /api/permission?url=&purpose= reads every machine-readable signal a site publishes for one page and one purpose (search, answer, train, agent), requests the page once as a representative crawler, and returns a verdict with a dated receipt signed with Ed25519. Add &agent=GPTBot to read one crawler's own rule, &observe=0 to skip the request to the site. The same fields work as a JSON body on POST.

curl "https://machinereadiness.com/api/permission?url=https://example.com/article&purpose=train" \
  -H "x-api-key: YOUR_KEY"

Verdicts: denied (a signal reserves the use), allowed-under-terms (an RSL licence permits it on conditions), allowed (nothing reserves it, which is not a licence), unknown (robots.txt could not be read). The public key for the receipts is at /api/permission-key. The full reference, the precedence rules and a verification snippet are at machinereadiness.com/permission. It is a reading of published signals, not legal advice.

The same calls, elsewhere

  • The MCP server (npm crawl-readiness-mcp): nine tools for Claude Desktop, Cursor, Cline and Windsurf, the permission check among them.
  • The Postman collection and the RapidAPI listing carry both endpoints with examples.
  • The crawler registry the checks read is public and CC BY 4.0 at machinereadiness.com/crawlers.