Static pre-check · Fri, 07 Aug 2026 06:50:14 GMT
AI agents struggle discovering and accessing lens.dev content due to missing robots.txt, sitemap.xml, and llms.txt files that facilitate automated crawling.
Watch a real agent use your site →
A live Claude Code run against lens.dev, streamed step by step.
Measure this with real agents →
An audit: measured success rates from repeated real agent runs, with confidence intervals and downloadable evidence.
robots.txt blocks ClaudeBot, GPTBot, Google-Extended from /
Fix: Remove the Disallow rules for ClaudeBot, GPTBot, Google-Extended in /robots.txt, or narrow them to the paths you really want kept out. Blocking these crawlers is the single most effective way to make your product invisible to AI agents.
sitemap.xml not served (HTTP 404)
Fix: Publish /sitemap.xml listing your public pages, and reference it from robots.txt with a `Sitemap:` line.
Shows the current scan score and grade for this domain, straight from the latest static pre-check.
[](https://www.vorza.dev/score/lens.dev)<a href="https://www.vorza.dev/score/lens.dev"><img src="https://www.vorza.dev/api/badge/lens.dev" alt="agent readiness score" /></a>no link to an API reference in the homepage HTML
Fix: If you have an API, link its reference from the homepage. If you don't, this check is safe to ignore.
no <link rel="canonical">
Fix: Add <link rel="canonical"> to the homepage so agents and crawlers converge on one URL for this page.
description is only 44 characters
Fix: Expand the meta description to at least 50 characters.
title is only 4 characters: 'Lens'
Fix: Expand the <title> to 10–70 characters naming the product and what it does.
robots.txt does not reference a sitemap
Fix: Add `Sitemap: https://<domain>/sitemap.xml` to /robots.txt so crawlers find it without guessing.
4 docs link(s), e.g. /docs
og:title and og:description present
HTTP 200, 1836 bytes
search-based checks are deferred — excluded from the score
registry lookup not implemented in v1 — excluded from the score rather than guessed
/llms.txt not served (HTTP 404)
Fix: Publish /llms.txt: a plain-markdown index that tells an agent what this product is and where the authoritative docs live. It is the cheapest high-leverage file you can add.
no JSON-LD blocks on the homepage
Fix: Add a JSON-LD block (schema.org Organization or SoftwareApplication) so agents get your name, description and links as data rather than prose.
/llms-full.txt not served
Fix: Optional but cheap: publish /llms-full.txt with your docs inlined as one markdown file, so an agent can read everything in a single fetch.
Accept: text/markdown → text/html; charset=utf-8
Fix: Optional but increasingly expected: honour `Accept: text/markdown` by serving a markdown rendering of the page, so agents skip HTML parsing entirely.
ratio 0.0093 — 2062 readable chars in 222663 bytes of HTML
Fix: Most of what you serve is markup and inline script rather than content. Reduce inline payloads or server-render more text so the signal-to-noise ratio favours readers.
2515 characters of text at /docs
HTTP 200 for / (agent UA)
https://lens.xyz/
both User-Agents got HTTP 200
2062 characters of text in the served HTML
1 redirect(s)
no llms.txt
no llms.txt to parse
/agents.md not served (HTTP 404)
Fix: Publish /agents.md: what an agent may do here, which endpoints matter, auth requirements, and rate limits. It is the agent-facing counterpart to a README.
404 returns HTML, not a machine-readable body
Fix: Content-negotiate your errors: return a JSON body with a stable `code` (and a `next_action` where recovery exists) to clients that ask for JSON.
/openapi.json not served (HTTP 404)
Fix: If you have an HTTP API, publish its OpenAPI document at a predictable path (/openapi.json). It is the difference between an agent reading your contract and an agent guessing it. If you have no HTTP API, ignore this.
/.well-known/mcp.json not served (HTTP 404)
Fix: If you run an MCP server, advertise it at /.well-known/mcp.json so agents discover it without being told its URL.
/docs → HTTP 200
/api → HTTP 404 (no API at the conventional root)
no MCP endpoint found to probe (none in the input URL and none advertised at /.well-known/mcp.json)
no listed MCP tools to inspect
no listed MCP tools to inspect
no MCP endpoint to probe
no parseable OpenAPI document
no parseable OpenAPI document
no parseable OpenAPI document
no parseable OpenAPI document
no agent-payment rail published — not applicable. This layer is excluded from the score rather than counted against the site.
no agent-payment rail published — not applicable. This layer is excluded from the score rather than counted against the site.
no agent-payment rail published — not applicable. This layer is excluded from the score rather than counted against the site.