Can AI see your website?
Test if ChatGPT, Claude, Perplexity, Google AI, and 46 other AI crawlers can actually access your site — in about 10 seconds. Get a 0-100 score, a radar breakdown, and a list of exact fixes for anything that's blocked.
What we check
Your AI readiness score comes from five signal groups. Each one contributes to the radar chart on your results page, and each one has a specific fix if it's failing.
- Crawler Access. Whether GPTBot, ClaudeBot, PerplexityBot, Google-Extended, and 46 other AI user-agents can pass your robots.txt.
- Structured Data. Whether you have JSON-LD schemas (Organization, WebSite, Article) that AI systems use to extract clean facts about you.
- AI-specific files. Whether you have
llms.txt,llms-full.txt, or aContent-Signaldirective declaring your AI preferences. - Meta quality. Whether your title, meta description, OG tags, and sitemap give AI enough context to describe your page correctly.
- Content accessibility. Whether your homepage renders real content without JavaScript, and whether meta robots or X-Robots-Tag headers are blocking indexing.
How to make your website AI-ready (5 steps)
If your readiness check comes back low, these are the fixes in priority order. Every step here is free and can be done in about 30 minutes end-to-end.
- 1Test if AI crawlers can reach your site
Check your robots.txt against 50+ AI crawler user-agents (GPTBot, ClaudeBot, PerplexityBot, Google-Extended, etc.). Look for Disallow rules that could be blocking them.
- 2Confirm your homepage renders without JavaScript
AI crawlers don't execute JS. View your page's HTML source — if the body is nearly empty and content only loads via JS, AI sees a blank page. Fix by adding server-side rendering or prerendering.
- 3Add JSON-LD structured data
Add Organization, WebSite, and Article schema in <script type="application/ld+json"> tags. AI systems use this to extract clean facts about your business and content.
- 4Add an llms.txt file
Create a plain Markdown file at yourdomain.com/llms.txt listing your most important pages with short descriptions. Growing number of AI systems read this.
- 5Verify with a free AI readiness check
Run your URL through an AI readiness checker to get a 0-100 score and a specific fix list. Re-check after each change.
Why this matters
Every month, more of your customers ask AI assistants their questions instead of Googling. When someone asks ChatGPT “what's the best [X] for [Y]?”, the assistant recommends three to five brands. If your site is invisible to ChatGPT's crawler, you can't be one of those brands — no matter how good your product is.
The tricky part is that being invisible to AI is easy to do accidentally. Cloudflare's “Block AI Scrapers” toggle is a single click that blocks 30+ crawlers. A React app without server-side rendering shows AI a blank page. A copy-pasted robots.txt snippet from a 2024 tutorial can silently remove you from Claude, Perplexity, and ChatGPT's search results — while you keep seeing normal Google traffic.
Our free checker was built to surface all of these at once, without you needing to understand every AI provider's bot naming convention.
Frequently asked questions
Can AI actually see my website?
Sometimes yes, sometimes no. AI systems like ChatGPT, Claude, and Perplexity use crawlers (GPTBot, ClaudeBot, PerplexityBot, etc.) to fetch web pages. If your robots.txt blocks these crawlers, if you're behind a Cloudflare AI-blocking rule, or if your homepage requires JavaScript to render, AI systems can't read your content. Our free checker tests all of these in about 10 seconds.
How do I know if ChatGPT can read my site?
Three quick checks: (1) Is GPTBot allowed in your robots.txt? (2) Does your homepage show real content without JavaScript running? (3) Do you have JSON-LD structured data? If any of those fail, ChatGPT probably can't cite you accurately. Paste your URL into our free readiness checker to see all three at once.
What blocks AI from reading a website?
The five most common blockers: (1) robots.txt Disallow rules for specific AI crawlers, (2) Cloudflare's 'Block AI Scrapers' toggle turned on, (3) a JavaScript-only homepage that shows an empty shell to non-browser crawlers, (4) a paywall or authentication gate, (5) IP-range blocking at the CDN or WAF level.
What is llms.txt and do I need one?
llms.txt is a proposed standard — a plain Markdown file at your site root that tells AI assistants which pages are most important and how to describe them. It's like sitemap.xml but for AI. Not strictly required in 2026, but adoption is growing fast (Anthropic, Perplexity, and Cloudflare already support it). Adding one takes about 10 minutes with a free generator.
Does Google AI use my robots.txt?
Yes. Google has multiple AI-related crawlers: Googlebot (regular search), Google-Extended (Gemini training + AI Overviews), and GoogleOther. Blocking Googlebot removes you from Google Search entirely. Blocking Google-Extended keeps you in Search but opts you out of AI training and AI Overview citations.
How do I make my website ChatGPT-friendly?
Five steps: (1) Allow GPTBot, OAI-SearchBot, and ChatGPT-User in your robots.txt. (2) Make sure your homepage server-renders content (no JS-only shells). (3) Add JSON-LD Organization schema so ChatGPT knows your name, logo, and socials. (4) Add a meta description to every page. (5) Optionally add an llms.txt file. Our free readiness check tests every one of these.
What's the difference between AI crawlers and search crawlers?
Search crawlers (Googlebot, Bingbot) build indexes for keyword search. AI crawlers (GPTBot, ClaudeBot, PerplexityBot) build training data OR fetch pages in real time to answer user questions. Most AI providers run multiple bots: one for training, one for search-style indexing, and one for live user browsing. Blocking one doesn't block the others.
Should I allow or block AI bots?
For most businesses: allow the assistant bots (ChatGPT-User, Claude-User, PerplexityBot), block or opt-out of training-only bots (GPTBot, Google-Extended) if you're worried about that. The new Content-Signal directive lets you say 'ai-train=no, search=yes, ai-input=yes' in one line — the balanced default. Publishers with licensed content sometimes block everything.
How often should I check my AI readiness?
Once when you first check (fix what's broken), then quarterly. New AI crawlers launch every few months, robots.txt files can get overwritten during redesigns, and CDN policies change. For active tracking, our Pro tier automatically runs the check weekly and monitors whether AI actually mentions your brand for target prompts.
Is my website AI-ready by default?
Probably partially. Most WordPress, Wix, Squarespace, and Shopify sites are technically crawlable, but usually score 55-70/100 on a full AI readiness audit — missing structured data, no llms.txt, sometimes an accidental robots.txt block. A perfect 100 requires a few deliberate additions. Our free checker gives you the exact list.
Check your site now
Free, instant, no signup. Get a full readiness report in 10 seconds.