xcrawl.ai Website Audit
Speed Insights
95% performance
LCP
1.7s
Largest paint
TBT
143ms
Blocking time
CLS
0.006
Layout shift
46
out of 100
77
out of 100
116/150 rule points
72
out of 100
55/76 rule points
64
out of 100
54/84 rule points
90
out of 100
102/110 rule points
Category point breakdown
Search discovery is materially blocked.
Material high-impact failures span 3 categories.
Biggest score-losing checks
For your AI agent
Share this audit’s URL with ChatGPT, Claude, or your coding agent to turn findings into fixes. Your agent can open the saved evidence below when it needs more detail; these files never start a new scan.
Agent-readable files
Improve AI search and crawler readiness
Why this needs attention
One or more AI discovery, user-fetch, indexability, snippet, synthetic reachability, or sitemap signals need attention.
AI search systems need crawlable, reachable, indexable, and quotable public content. Training and data-use choices remain neutral publisher preferences.
Recommended correction
Evidence
FreeScan.app reached the public page with HTTP 200. Non-robots access or indexing restrictions affect OAI-SearchBot, Claude-SearchBot, Googlebot, and 5 more. OAI-SearchBot: Allowed by * allow: / on line 32. Synthetic User-Agent probe returned HTTP 200 with 99% sampled-content parity. The page has a noindex signal, so search and citation eligibility is blocked even if crawling is allowed. Relevant page controls: robots meta: index, follow; X-Robots-Tag: noindex, nofollow. Claude-SearchBot: Allowed by * allow: / on line 32. Synthetic User-Agent probe returned HTTP 200 with 99% sampled-content parity. The page has a noindex signal, so search and citation eligibility is blocked even if crawling is allowed. Relevant page controls: robots meta: index, follow; X-Robots-Tag: noindex, nofollow. Googlebot: Allowed by * allow: / on line 32. Synthetic User-Agent probe returned HTTP 200 with 99% sampled-content parity. The page has a noindex signal, so search and citation eligibility is blocked even if crawling is allowed. Relevant page controls: robots meta: index, follow; Googlebot meta: index, follow; X-Robots-Tag: noindex, nofollow. Bingbot: Allowed by * allow: / on line 32. Synthetic User-Agent probe returned HTTP 200 with 99% sampled-content parity. The page has a noindex signal, so search and citation eligibility is blocked even if crawling is allowed. Relevant page controls: robots meta: index, follow; X-Robots-Tag: noindex, nofollow. PerplexityBot: Allowed by * allow: / on line 32. Synthetic User-Agent probe returned HTTP 200 with 99% sampled-content parity. The page has a noindex signal, so search and citation eligibility is blocked even if crawling is allowed. Relevant page controls: robots meta: index, follow; X-Robots-Tag: noindex, nofollow. Applebot: Allowed by * allow: / on line 32. Synthetic User-Agent probe returned HTTP 200 with 99% sampled-content parity. The page has a noindex signal, so search and citation eligibility is blocked even if crawling is allowed. Relevant page controls: robots meta: index, follow; X-Robots-Tag: noindex, nofollow. MistralAI-Index: Allowed by * allow: / on line 32. Synthetic User-Agent probe returned HTTP 200 with 99% sampled-content parity. The page has a noindex signal, so search and citation eligibility is blocked even if crawling is allowed. Relevant page controls: robots meta: index, follow; X-Robots-Tag: noindex, nofollow. Amzn-SearchBot: Allowed by * allow: / on line 32. Synthetic User-Agent probe returned HTTP 200 with 99% sampled-content parity. The page has a noindex signal, so search and citation eligibility is blocked even if crawling is allowed. Relevant page controls: robots meta: index, follow; X-Robots-Tag: noindex, nofollow. GPTBot, ClaudeBot, Google-Extended, and 4 more are explicit training or data-use opt-outs and do not reduce the AEO score. AEO crawler readiness is 30%: discovery 0/45, index and snippet eligibility 5/25, synthetic reachability 20/20, sitemap freshness 5/10. robots.txt did not block the scored AI search or user-fetch agents for this page path.
Priority
Priority 2: fix before sharing the page publicly.
Add common public security headers
Why this needs attention
Several browser protection headers are missing or present only as weak, narrow policies.
These headers reduce avoidable browser-side risk and show a baseline of care before launch.
Recommended correction
Evidence
Detected 0 of 5 common public security controls. Present: none. Missing: strict-transport-security, content-security-policy, x-content-type-options, frame protection (x-frame-options or content-security-policy frame-ancestors), referrer-policy. Needs strengthening: none.
Priority
Priority 5: fix before sharing the page publicly.
Label public form fields
Why this needs attention
Some form controls do not have a detectable label or accessible name.
Unlabeled inputs make email capture, demo requests, and signups harder for assistive technology users.
Recommended correction
Evidence
Detected labels or accessible names for 0 of 1 eligible user-facing form control(s).
Priority
Priority 7: fix during launch polish.
Remove noindex signals
Why this needs attention
The page appears to tell crawlers not to index it.
A noindex directive can prevent the launch page from appearing in search results.
Recommended correction
Evidence
A noindex directive was detected.
Priority
Priority 2: fix before sharing the page publicly.
Fix rendered axe accessibility violations
Why this needs attention
The rendered page has accessibility rule violations detected by axe-core.
axe checks the actual browser-rendered page, so these issues can affect people using keyboards, screen readers, or other assistive technology.
Recommended correction
Evidence
axe found 3 violation rule(s); 3 non-contrast rule(s) are scored here, including 1 serious or critical rule(s). Color contrast is scored once by the dedicated contrast check. 0 result(s) require review. Top rules: link-name (78), landmark-unique (1), region (14).
Priority
Priority 5: fix before sharing the page publicly.
Affected elements
Links must have discernible text

Fix all of the following: Element is in tab order and does not have accessible text Fix any of the following: Element does not have text that is visible to screen readers aria-label attribute does not exist or is empty aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty Element has no title attribute
Links must have discernible text

Fix all of the following: Element is in tab order and does not have accessible text Fix any of the following: Element does not have text that is visible to screen readers aria-label attribute does not exist or is empty aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty Element has no title attribute
Links must have discernible text
Fix all of the following: Element is in tab order and does not have accessible text Fix any of the following: Element does not have text that is visible to screen readers aria-label attribute does not exist or is empty aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty Element has no title attribute
Landmarks should have a unique role or role/label/title (i.e. accessible name) combination

Fix any of the following: The landmark must have a unique aria-label, aria-labelledby, or title to make landmarks distinguishable
All page content should be contained by landmarks

Fix any of the following: Some page content is not contained by landmarks
All page content should be contained by landmarks
Fix any of the following: Some page content is not contained by landmarks
All page content should be contained by landmarks
Fix any of the following: Some page content is not contained by landmarks
SEO / AEO
Severity mix: 1 critical, 8 high, 3 medium, 1 low. Coverage: 0 review and 1 not applicable.
Security
Severity mix: 2 critical, 3 high, 0 medium, 0 low. Coverage: 0 review and 0 not applicable.
Accessibility
Severity mix: 0 critical, 3 high, 6 medium, 0 low. Coverage: 2 review and 0 not applicable.
Design
Severity mix: 0 critical, 3 high, 7 medium, 1 low. Coverage: 1 review and 0 not applicable.
SEO / AEO
Severity mix: 1 critical, 8 high, 3 medium, 1 low. Coverage: 0 review and 1 not applicable.
Accessibility
Severity mix: 0 critical, 3 high, 6 medium, 0 low. Coverage: 2 review and 0 not applicable.
Security
Severity mix: 2 critical, 3 high, 0 medium, 0 low. Coverage: 0 review and 0 not applicable.
Design
Severity mix: 0 critical, 3 high, 7 medium, 1 low. Coverage: 1 review and 0 not applicable.
Failed checks
These checks need attention.
Crawler access
Why this needs attention
One or more AI discovery, user-fetch, indexability, snippet, synthetic reachability, or sitemap signals need attention.
AI search systems need crawlable, reachable, indexable, and quotable public content. Training and data-use choices remain neutral publisher preferences.
Recommended correction
Evidence
FreeScan.app reached the public page with HTTP 200. Non-robots access or indexing restrictions affect OAI-SearchBot, Claude-SearchBot, Googlebot, and 5 more. OAI-SearchBot: Allowed by * allow: / on line 32. Synthetic User-Agent probe returned HTTP 200 with 99% sampled-content parity. The page has a noindex signal, so search and citation eligibility is blocked even if crawling is allowed. Relevant page controls: robots meta: index, follow; X-Robots-Tag: noindex, nofollow. Claude-SearchBot: Allowed by * allow: / on line 32. Synthetic User-Agent probe returned HTTP 200 with 99% sampled-content parity. The page has a noindex signal, so search and citation eligibility is blocked even if crawling is allowed. Relevant page controls: robots meta: index, follow; X-Robots-Tag: noindex, nofollow. Googlebot: Allowed by * allow: / on line 32. Synthetic User-Agent probe returned HTTP 200 with 99% sampled-content parity. The page has a noindex signal, so search and citation eligibility is blocked even if crawling is allowed. Relevant page controls: robots meta: index, follow; Googlebot meta: index, follow; X-Robots-Tag: noindex, nofollow. Bingbot: Allowed by * allow: / on line 32. Synthetic User-Agent probe returned HTTP 200 with 99% sampled-content parity. The page has a noindex signal, so search and citation eligibility is blocked even if crawling is allowed. Relevant page controls: robots meta: index, follow; X-Robots-Tag: noindex, nofollow. PerplexityBot: Allowed by * allow: / on line 32. Synthetic User-Agent probe returned HTTP 200 with 99% sampled-content parity. The page has a noindex signal, so search and citation eligibility is blocked even if crawling is allowed. Relevant page controls: robots meta: index, follow; X-Robots-Tag: noindex, nofollow. Applebot: Allowed by * allow: / on line 32. Synthetic User-Agent probe returned HTTP 200 with 99% sampled-content parity. The page has a noindex signal, so search and citation eligibility is blocked even if crawling is allowed. Relevant page controls: robots meta: index, follow; X-Robots-Tag: noindex, nofollow. MistralAI-Index: Allowed by * allow: / on line 32. Synthetic User-Agent probe returned HTTP 200 with 99% sampled-content parity. The page has a noindex signal, so search and citation eligibility is blocked even if crawling is allowed. Relevant page controls: robots meta: index, follow; X-Robots-Tag: noindex, nofollow. Amzn-SearchBot: Allowed by * allow: / on line 32. Synthetic User-Agent probe returned HTTP 200 with 99% sampled-content parity. The page has a noindex signal, so search and citation eligibility is blocked even if crawling is allowed. Relevant page controls: robots meta: index, follow; X-Robots-Tag: noindex, nofollow. GPTBot, ClaudeBot, Google-Extended, and 4 more are explicit training or data-use opt-outs and do not reduce the AEO score. AEO crawler readiness is 30%: discovery 0/45, index and snippet eligibility 5/25, synthetic reachability 20/20, sitemap freshness 5/10. robots.txt did not block the scored AI search or user-fetch agents for this page path. Failed: earned 4 of 14 points from partial coverage.
How FreeScan evaluates this check (opens in a new tab)Priority
Priority 2: fix before sharing the page publicly.
Answer-engine content quality
Why this needs attention
The page is technically discoverable, but its entity clarity, answer-ready copy, structured data, evidence, authorship, or freshness is not yet complete.
Search and answer systems need clear, extractable, verifiable facts in addition to crawl access and metadata.
Recommended correction
Evidence
Role-aware AEO content components: entity clarity 96/100, answer-ready text 75/100, semantic structured data 91/100. Detected role: home (high confidence). Failed: earned 35 of 40 points from partial coverage.
How FreeScan evaluates this check (opens in a new tab)Priority
Priority 6: fix during launch polish.
Canonical tag
Why this needs attention
The page does not declare which public URL is the preferred version.
A canonical tag reduces duplicate URL confusion before marketing links start spreading.
Recommended correction
Evidence
Canonical does not align with the scanned page: https://www.xcrawl.com/. Failed: earned 0 of 10 points.
How FreeScan evaluates this check (opens in a new tab)Priority
Priority 9: fix during launch polish.
Indexability signals
Why this needs attention
The page appears to tell crawlers not to index it.
A noindex directive can prevent the launch page from appearing in search results.
Recommended correction
Evidence
A noindex directive was detected. Failed: earned 0 of 9 points.
How FreeScan evaluates this check (opens in a new tab)Priority
Priority 2: fix before sharing the page publicly.
Common security headers
Why this needs attention
Several browser protection headers are missing or present only as weak, narrow policies.
These headers reduce avoidable browser-side risk and show a baseline of care before launch.
Recommended correction
Evidence
Detected 0 of 5 common public security controls. Present: none. Missing: strict-transport-security, content-security-policy, x-content-type-options, frame protection (x-frame-options or content-security-policy frame-ancestors), referrer-policy. Needs strengthening: none. Failed: earned 0 of 18 points.
How FreeScan evaluates this check (opens in a new tab)Priority
Priority 5: fix before sharing the page publicly.
Visible cookie flags
Why this needs attention
The safe public scan could not inspect private or browser-set cookies, so cookie flag coverage was not verified.
Session and preference cookies should avoid unnecessary exposure before early users sign in or submit forms.
Recommended correction
Evidence
Observed 1 cookie(s) without storing names or values. Secure: 0/1. HttpOnly: 0/1. SameSite: 0/1. SameSite=None without Secure: 0. Failed: earned 1 of 4 points from partial coverage.
How FreeScan evaluates this check (opens in a new tab)Priority
Priority 20: lower-risk cleanup after urgent launch blockers.
Form labels
Why this needs attention
Some form controls do not have a detectable label or accessible name.
Unlabeled inputs make email capture, demo requests, and signups harder for assistive technology users.
Recommended correction
Evidence
Detected labels or accessible names for 0 of 1 eligible user-facing form control(s). Failed: earned 0 of 14 points.
How FreeScan evaluates this check (opens in a new tab)Priority
Priority 7: fix during launch polish.
Basic contrast
Why this needs attention
The scanner found text/background color pairs that may be hard to read, or could not verify contrast.
Low contrast makes a launch page feel less polished and can exclude users with low vision.
Recommended correction
Evidence
15 of 237 rendered text color sample(s) across captured viewports missed WCAG contrast targets. Worst rendered ratio: 1.97. Mobile 390px viewport: 8 of 135 failed; worst ratio 1.97:1. Failed: earned 0 of 8 points.
How FreeScan evaluates this check (opens in a new tab)Priority
Priority 20: lower-risk cleanup after urgent launch blockers.
Affected elements
[.MD]
[.MD]
AI-Ready

# pip install xcrawl-py

# Scrape a website:
# pip install xcrawl-py

# Scrape a website:
"xc-YOUR_API_KEY"
XCrawl

Rendered axe accessibility violations
Why this needs attention
The rendered page has accessibility rule violations detected by axe-core.
axe checks the actual browser-rendered page, so these issues can affect people using keyboards, screen readers, or other assistive technology.
Recommended correction
Evidence
axe found 3 violation rule(s); 3 non-contrast rule(s) are scored here, including 1 serious or critical rule(s). Color contrast is scored once by the dedicated contrast check. 0 result(s) require review. Top rules: link-name (78), landmark-unique (1), region (14). Failed: earned 10 of 18 points from partial coverage.
How FreeScan evaluates this check (opens in a new tab)Priority
Priority 5: fix before sharing the page publicly.
Affected elements
Links must have discernible text

Fix all of the following: Element is in tab order and does not have accessible text Fix any of the following: Element does not have text that is visible to screen readers aria-label attribute does not exist or is empty aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty Element has no title attribute
Links must have discernible text

Fix all of the following: Element is in tab order and does not have accessible text Fix any of the following: Element does not have text that is visible to screen readers aria-label attribute does not exist or is empty aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty Element has no title attribute
Links must have discernible text
Fix all of the following: Element is in tab order and does not have accessible text Fix any of the following: Element does not have text that is visible to screen readers aria-label attribute does not exist or is empty aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty Element has no title attribute
Landmarks should have a unique role or role/label/title (i.e. accessible name) combination

Fix any of the following: The landmark must have a unique aria-label, aria-labelledby, or title to make landmarks distinguishable
All page content should be contained by landmarks

Fix any of the following: Some page content is not contained by landmarks
All page content should be contained by landmarks
Fix any of the following: Some page content is not contained by landmarks
All page content should be contained by landmarks
Fix any of the following: Some page content is not contained by landmarks
Visual hierarchy
Why this needs attention
Rendered typography, geometry, and contrast show that the main heading is not distinct or legible enough, or that the hierarchy needs manual review.
Hierarchy tells visitors what to read first and what action to take next.
Recommended correction
Evidence
The rendered H1 is 60px/700 weight versus a 14px/400 median body style (4.29× size). Captured 22 supporting heading(s) and 6 action(s). The main heading has a clear rendered emphasis over body copy. 15 of 237 rendered text samples missed contrast targets, so hierarchy is not considered fully legible. Failed: earned 6 of 9 points from partial coverage.
How FreeScan evaluates this check (opens in a new tab)Priority
Priority 12: fix during launch polish.
Page-level AEO readiness
Shared eligibility and content signals that affect every search and answer agent.
Index and citation eligibility
A general noindex directive prevents search indexing and citation eligibility.
Snippet and answer controls
No general nosnippet or max-snippet:0 control was detected.
Canonical alignment
The canonical does not resolve to the scanned URL: https://www.xcrawl.com/
Sitemap coverage and freshness
A sitemap index was found; this safe scan did not recursively fetch child sitemaps.
Structured data
Semantic schema score 91/100. Detected 5 JSON-LD block(s), 0 parse error(s), 27 graph node(s), and primary types FAQPage, Organization, Service, WebSite. Completeness: 60/100. Graph connectivity: 93/100. A primary schema type matches the detected page role. The schema entity is supported by visible page content. Limited search eligibility: FAQPage. Retired search feature: SearchAction; no coverage credit is awarded.
Public or paywalled content
No isAccessibleForFree structured-data value was detected.
Entity clarity
Entity alignment scored 96/100 for “AI-Powered Web Scraping API & Intelligent Scraper Tool”. Title/H1 token overlap is 86%; description match detected; identity schema detected.
Answer-ready visible text
Answerability scored 75/100 from 1745 main-content words, 9 concise answer block(s), 1 question heading(s), 13 definition(s), 0 list(s), and 0 table(s).
Authorship and accountability
Authorship is not a universal requirement for the detected home page role.
Published or updated date
Freshness metadata is contextual for the detected home page role.
Supporting-source links
Supporting sources are contextual for the detected home page role and do not affect readiness.
Rendered content availability
Rendered audit completed with 434 visible text element(s).
Synthetic search-agent reachability
8 of 8 completed search/discovery User-Agent probes returned reachable content.
Optional LLM-readable summary
No llms.txt file was found; this is optional and does not reduce core AEO readiness.
OpenAIChatGPT searchOAI-SearchBotFAIL · BLOCKED · Toggle crawler details
AI search
Robots: allowed by * allow: / · line 32
Synthetic probe: reachable · HTTP 200 · 99% sampled-content parity
Controls: robots meta: index, follow; X-Robots-Tag: noindex, nofollow
OpenAIOpenAI model trainingGPTBotOPTED OUT · Toggle crawler details
Model training
Robots: blocked by gptbot disallow: / · line 57
Synthetic probe: not applicable to this policy control
Controls: robots meta: index, follow; X-Robots-Tag: noindex, nofollow
OpenAIChatGPT user fetchesChatGPT-UserUNKNOWN · Toggle crawler details
User-requested fetch
Robots: not applicable to user-requested fetches
Synthetic probe: not tested · access is unverified
Controls: robots meta: index, follow; X-Robots-Tag: noindex, nofollow
AnthropicClaude model trainingClaudeBotOPTED OUT · Toggle crawler details
Model training
Robots: blocked by claudebot disallow: / · line 48
Synthetic probe: not applicable to this policy control
Controls: robots meta: index, follow; X-Robots-Tag: noindex, nofollow
AnthropicClaude searchClaude-SearchBotFAIL · BLOCKED · Toggle crawler details
AI search
Robots: allowed by * allow: / · line 32
Synthetic probe: reachable · HTTP 200 · 99% sampled-content parity
Controls: robots meta: index, follow; X-Robots-Tag: noindex, nofollow
AnthropicClaude user fetchesClaude-UserPASS · ALLOWED · Toggle crawler details
User-requested fetch
Robots: allowed by * allow: / · line 32
Synthetic probe: reachable · HTTP 200 · 99% sampled-content parity
Controls: robots meta: index, follow; X-Robots-Tag: noindex, nofollow
GoogleGoogle Search and AI featuresGooglebotFAIL · BLOCKED · Toggle crawler details
Search and discovery
Robots: allowed by * allow: / · line 32
Synthetic probe: reachable · HTTP 200 · 99% sampled-content parity
Controls: robots meta: index, follow; Googlebot meta: index, follow; X-Robots-Tag: noindex, nofollow
GoogleGemini data use and groundingGoogle-ExtendedOPTED OUT · Toggle crawler details
Data-use policy control
Robots: blocked by google-extended disallow: / · line 54
Synthetic probe: not applicable to this policy control
Controls: robots meta: index, follow; X-Robots-Tag: noindex, nofollow
MicrosoftBing and CopilotBingbotFAIL · BLOCKED · Toggle crawler details
Search and discovery
Robots: allowed by * allow: / · line 32
Synthetic probe: reachable · HTTP 200 · 99% sampled-content parity
Controls: robots meta: index, follow; X-Robots-Tag: noindex, nofollow
PerplexityPerplexity searchPerplexityBotFAIL · BLOCKED · Toggle crawler details
AI search
Robots: allowed by * allow: / · line 32
Synthetic probe: reachable · HTTP 200 · 99% sampled-content parity
Controls: robots meta: index, follow; X-Robots-Tag: noindex, nofollow
PerplexityPerplexity user fetchesPerplexity-UserUNKNOWN · Toggle crawler details
User-requested fetch
Robots: not applicable to user-requested fetches
Synthetic probe: not tested · access is unverified
Controls: robots meta: index, follow; X-Robots-Tag: noindex, nofollow
AppleSiri, Spotlight, and Safari searchApplebotFAIL · BLOCKED · Toggle crawler details
Search and discovery
Robots: allowed by * allow: / · line 32
Synthetic probe: reachable · HTTP 200 · 99% sampled-content parity
Controls: robots meta: index, follow; X-Robots-Tag: noindex, nofollow
AppleApple foundation-model trainingApplebot-ExtendedOPTED OUT · Toggle crawler details
Data-use policy control
Robots: blocked by applebot-extended disallow: / · line 39
Synthetic probe: not applicable to this policy control
Controls: robots meta: index, follow; X-Robots-Tag: noindex, nofollow
Mistral AIMistral searchMistralAI-IndexFAIL · BLOCKED · Toggle crawler details
AI search
Robots: allowed by * allow: / · line 32
Synthetic probe: reachable · HTTP 200 · 99% sampled-content parity
Controls: robots meta: index, follow; X-Robots-Tag: noindex, nofollow
Mistral AIMistral user fetchesMistralAI-UserPASS · ALLOWED · Toggle crawler details
User-requested fetch
Robots: allowed by * allow: / · line 32
Synthetic probe: reachable · HTTP 200 · 99% sampled-content parity
Controls: robots meta: index, follow; X-Robots-Tag: noindex, nofollow
MetaMeta AI model and product datameta-externalagentOPTED OUT · Toggle crawler details
Model training
Robots: blocked by meta-externalagent disallow: / · line 75
Synthetic probe: not applicable to this policy control
Controls: robots meta: index, follow; X-Robots-Tag: noindex, nofollow
MetaMeta AI user fetchesmeta-externalfetcherPASS · ALLOWED · Toggle crawler details
User-requested fetch
Robots: allowed by * allow: / · line 32
Synthetic probe: reachable · HTTP 200 · 99% sampled-content parity
Controls: robots meta: index, follow; X-Robots-Tag: noindex, nofollow
AmazonAmazon model trainingAmazonbotOPTED OUT · Toggle crawler details
Model training
Robots: blocked by amazonbot disallow: / · line 36
Synthetic probe: not applicable to this policy control
Controls: robots meta: index, follow; X-Robots-Tag: noindex, nofollow
AmazonAlexa and Rufus searchAmzn-SearchBotFAIL · BLOCKED · Toggle crawler details
AI search
Robots: allowed by * allow: / · line 32
Synthetic probe: reachable · HTTP 200 · 99% sampled-content parity
Controls: robots meta: index, follow; X-Robots-Tag: noindex, nofollow
AmazonAlexa user fetchesAmzn-UserPASS · ALLOWED · Toggle crawler details
User-requested fetch
Robots: allowed by * allow: / · line 32
Synthetic probe: reachable · HTTP 200 · 99% sampled-content parity
Controls: robots meta: index, follow; X-Robots-Tag: noindex, nofollow
Common CrawlOpen web datasetsCCBotOPTED OUT · Toggle crawler details
Model training
Robots: blocked by ccbot disallow: / · line 45
Synthetic probe: not applicable to this policy control
Controls: robots meta: index, follow; X-Robots-Tag: noindex, nofollow
Ready combines robots policy, a bounded synthetic User-Agent fetch, indexability, and snippet eligibility. Synthetic probes do not prove vendor-origin traffic. Preference states are training or data-use choices and do not lower the AEO score.
The image refreshes from FreeScan after a completed rescan. Clicking it always opens the newest public report saved for this exact scanned URL.
Scan evidence is included as untrusted data. The implementation brief instructs agents to verify findings before editing and to avoid destructive or speculative changes.
To refresh your Pro site score and fixes, open your Pro dashboard and run a site audit.