Static pre-check · Fri, 07 Aug 2026 06:49:20 GMT
firecrawl.dev
AI agents can effectively use Firecrawl's API, but usability features lag behind discovery and accessibility, limiting optimal interaction patterns.
Watch a real agent use your site →
A live Claude Code run against firecrawl.dev, streamed step by step.
Test this with real agents →
An audit: measured success rates from repeated real agent runs, with confidence intervals and downloadable evidence.
Discovery 20.6/22.2
Canonical URL declareddetailsWARN
no <link rel="canonical">
Fix: Add <link rel="canonical"> to the homepage so agents and crawlers converge on one URL for this page.
Homepage has a descriptive titledetailsWARN
title is 116 characters (truncated in most surfaces)
Fix: Trim the <title> to 70 characters or fewer so it is not cut off.
API reference linked from the homepagedetailsPASS
1 API link(s), e.g. https://docs.firecrawl.dev/api-reference/introduction
Documentation linked from the homepagedetailsPASS
5 docs link(s), e.g. https://docs.firecrawl.dev
Homepage has a meta descriptiondetailsPASS
165 characters
Open Graph tags presentdetailsPASS
og:title and og:description present
robots.txt allows agent crawlersdetailsPASS
all 6 agent crawlers may fetch /
robots.txt existsdetailsPASS
HTTP 200, 366 bytes
robots.txt points at the sitemapdetailsPASS
robots.txt has a Sitemap: line
sitemap.xml exists and parsesdetailsPASS
565 URL(s) listed
Brand findable in agent searchdetailsN/A
search-based checks are deferred — excluded from the score
Listed in an MCP registrydetailsN/A
registry lookup not implemented in v1 — excluded from the score rather than guessed
Accessibility 30.4/33.3
Structured data (JSON-LD) on the homepagedetailsWARN
no JSON-LD blocks on the homepage
Fix: Add a JSON-LD block (schema.org Organization or SoftwareApplication) so agents get your name, description and links as data rather than prose.
Serves markdown when asked for itdetailsWARN
Accept: text/markdown → text/html; charset=utf-8
Fix: Optional but increasingly expected: honour `Accept: text/markdown` by serving a markdown rendering of the page, so agents skip HTML parsing entirely.
Content-to-markup ratio is reasonabledetailsWARN
ratio 0.0162 — 19356 readable chars in 1194521 bytes of HTML
Fix: Most of what you serve is markup and inline script rather than content. Reduce inline payloads or server-render more text so the signal-to-noise ratio favours readers.
Docs are readable without JavaScriptdetailsPASS
11704 characters of text at /docs
Homepage responds to an agent requestdetailsPASS
HTTP 200 for / (agent UA)
Served over HTTPSdetailsPASS
https://www.firecrawl.dev/
llms-full.txt is publisheddetailsPASS
HTTP 200, 6922 bytes
llms.txt is publisheddetailsPASS
HTTP 200, 6922 bytes
Links in llms.txt resolvedetailsPASS
5 sampled link(s) all resolve
llms.txt is structured markdowndetailsPASS
has headings and markdown links
Same response for agents and browsersdetailsPASS
both User-Agents got HTTP 200
Homepage has readable text without JavaScriptdetailsPASS
19356 characters of text in the served HTML
No long redirect chain on the homepagedetailsPASS
1 redirect(s)
Usability 29.2/44.4
Errors are machine-readabledetailsWARN
404 returns HTML, not a machine-readable body
Fix: Content-negotiate your errors: return a JSON body with a stable `code` (and a `next_action` where recovery exists) to clients that ask for JSON.
OpenAPI document publisheddetailsWARN
/openapi.json not served (HTTP 404)
Fix: If you have an HTTP API, publish its OpenAPI document at a predictable path (/openapi.json). It is the difference between an agent reading your contract and an agent guessing it. If you have no HTTP API, ignore this.
.well-known/mcp.json advertises an MCP serverdetailsWARN
/.well-known/mcp.json not served (HTTP 404)
Fix: If you run an MCP server, advertise it at /.well-known/mcp.json so agents discover it without being told its URL.
agents.md is publisheddetailsPASS
HTTP 200, 17440 bytes
A documentation endpoint respondsdetailsPASS
/docs → HTTP 200
The /api root behaves predictablydetailsN/A
/api → HTTP 404 (no API at the conventional root)
MCP server completes the initialize handshakedetailsN/A
no MCP endpoint found to probe (none in the input URL and none advertised at /.well-known/mcp.json)
MCP tool parameters are typed and describeddetailsN/A
no listed MCP tools to inspect
Every MCP tool has a descriptiondetailsN/A
no listed MCP tools to inspect
MCP server lists toolsdetailsN/A
no MCP endpoint to probe
OpenAPI declares its auth schemedetailsN/A
no parseable OpenAPI document
OpenAPI operations are describeddetailsN/A
no parseable OpenAPI document
OpenAPI documents error responsesdetailsN/A
no parseable OpenAPI document
OpenAPI document is structurally validdetailsN/A
no parseable OpenAPI document
Payments not applicable — excluded from the score
Serves a 402 payment challengedetailsN/A
no agent-payment rail published — not applicable. This layer is excluded from the score rather than counted against the site.
Declares AP2 / ACP supportdetailsN/A
no agent-payment rail published — not applicable. This layer is excluded from the score rather than counted against the site.
Publishes x402 payment metadatadetailsN/A
no agent-payment rail published — not applicable. This layer is excluded from the score rather than counted against the site.
Badge
Shows the current scan score and grade for this domain, straight from the latest static pre-check.
[](/score/firecrawl.dev)<a href="/score/firecrawl.dev"><img src="/api/badge/firecrawl.dev" alt="agent readiness score" /></a>