Strong technical baseline

Recommendation

Server-side render your homepage so AI crawlers see meaningful content without JavaScript. Ensure an H1 and 500+ chars of text in raw HTML.

Recommendation

Allowlist known AI agent User-Agents (ChatGPT-User, ClaudeBot, Google-Extended, DeepSeekBot) in your WAF or bot-detection rules.

Recommendation

Stop blocking user-triggered agents in robots.txt. Remove any 'Disallow: /' that applies to ChatGPT-User, Claude-User or Perplexity-User - including a blanket 'User-agent: * / Disallow: /', which covers them by default. These agents fetch a page only because a person just asked about you, so a block turns away your highest-intent traffic.

Recommendation

Replace meta-refresh and JavaScript-only redirects with real HTTP 301/302 redirects. Non-JS agents never execute `location.href` or wait for a meta refresh - they see only the stub page. Verify with `curl -sI <url>` - you should see a Location header, not a 200 with a near-empty body.

Recommendation

Serve your content pages without a login wall. Agents cannot complete auth flows while browsing - a 401/403 or a login-form page is invisible content. Keep public documentation public; if some content must stay gated, publish an ungated summary so agents can still represent it.

Recommendation

Publish an OpenAPI (Swagger) specification at /openapi.json or /api/openapi.yaml. This is how agents understand your API surface automatically.

Recommendation

On the responses that serve text/markdown via Accept negotiation, add Accept to the Vary header (Vary: Accept, Accept-Encoding). Without it, CDNs can serve the cached HTML variant to an agent asking for markdown (or vice versa), depending on which variant landed in cache first.

Recommendation

Verify that major agent User-Agents can reach the homepage. If robots.txt, your WAF, or bot rules block them, remove or narrow the blocking rule. Add an allow rule only when your security setup denies them by default.

Recommendation

Implement OAuth 2.0 for API authentication. Publish your authorization server metadata at /.well-known/oauth-authorization-server.

Recommendation

Support scoped API permissions (OAuth scopes or API key roles) so agents can request only the access they need.

Recommendation

Return structured JSON error responses with error codes, messages, and resolution hints. Agents can't parse HTML error pages.

Recommendation

If your MCP server advertises the resources capability in its initialize handshake, make sure resources/list returns at least one resource. If you don't intend to expose resources, omit the capability - the check returns na with no penalty for tool-only servers. Quality of the resources you do return is scored separately by mcp-resource-quality.

Recommendation

Return a real HTTP 404 (or 410) status for nonexistent paths - never a 200 with your app shell, which makes agents believe every path exists. For full credit, give the 404 response a short markdown body pointing agents at your sitemap, llms.txt, or docs index. Verify with `curl -s -o /dev/null -w "%{http_code}" https://yourdomain.com/some-path-that-does-not-exist` - it must print 404.

Recommendation

Make your developer resources (API docs, OpenAPI spec, auth docs, webhooks, MCP server) discoverable by name. Publish them at predictable URLs, list them in llms.txt, and include your product name in page titles and headings so search engines surface them for name-based queries.

Recommendation

Make sure a clean search for your brand name returns your own domain in the top results. If it does not, your brand may be too generic, conflict with a more established term, or not yet indexed. Strengthen brand-name search by claiming consistent NAP across listings, earning press mentions that link to the canonical domain, and avoiding redirect chains that mask the apex domain in search results.

Recommendation

Add explicit AI crawler directives in robots.txt. Allow the crawlers that feed answer engines ('User-agent: GPTBot' / 'Allow: /', same for ClaudeBot, PerplexityBot, OAI-SearchBot), and restrict training-only crawlers (CCBot, ByteSpider) with 'Disallow: /'. Content Signals ('Content-Signal: search=yes, ai-train=no') earns the same credit.

Recommendation

Add a valid XML sitemap at /sitemap.xml listing all indexable URLs. Include lastmod dates and keep it under 50MB.

Recommendation

Reduce markup overhead so readable text is at least 5% of your HTML. Strip unused inline scripts/styles, server-render content instead of shipping large JSON hydration blobs, and keep wrapper nesting shallow.

Recommendation

Add JSON-LD structured data to your homepage using the identity type that matches your site - SoftwareApplication for products, Organization or LocalBusiness for companies, Person for personal sites, Article for blogs - with name, description, url, and type-appropriate fields (offers, sameAs, author) so AI can parse your identity programmatically.

Recommendation

Publish API documentation at a discoverable URL (/docs, /api, /developers). Include authentication, endpoints, and example requests.

Recommendation

Tell agents when to reach for you: add a 'when to use this' section to your llms.txt (or a dedicated agent-instructions file) that names your best-fit use cases and how an agent should call you. Be specific about the jobs you are right for - generic marketing copy does not read as guidance.

Recommendation

Add all four signals to your homepage: <link rel="canonical">, <html lang="...">, <meta property="og:image">, and <meta property="og:type">. Agents use these for entity resolution and attribution.

Recommendation

Add Organization JSON-LD that includes both contactPoint (with email/phone and contactType) and address (PostalAddress). This lets AI verify your business legitimacy and answer contact queries.

Recommendation

Publish real /about, /contact, and /privacy pages with at least 500 characters of content each. These are the pages AI agents check to verify your business is legitimate before recommending you.

Recommendation

Keep each page's extracted text under ~100K characters (~25K tokens) so it fits an agent's context window without truncation. Split oversized reference pages into focused per-topic documents and link them from an index. Check a page with `curl -s <url> | wc -c` and remember agents read the extracted text, not the raw HTML.

Recommendation

Close every fenced code block (``` or ~~~) in your served markdown. CommonMark treats everything after an unclosed fence as code, so an agent parsing the document silently loses the rest of it. Count fence lines per file - the total must be even.

Recommendation

Create a developer portal at /developers with API keys, documentation, quickstart guides, and a sandbox environment.

Recommendation

Expose a public REST or GraphQL API. AI agents need programmatic access - not just a web UI - to integrate with your product.

Recommendation

Offer a free tier or trial, self-serve API key generation, and a sandbox environment. Agents can't fill out 'contact sales' forms.

Recommendation

Build an MCP (Model Context Protocol) server exposing your API as tools. Use Streamable HTTP transport for full score. This lets Claude, ChatGPT, and other AI agents call your product natively.

Recommendation

Return standard rate-limit headers on your API responses (the RFC RateLimit headers, plus Retry-After on a 429) so agents can self-throttle in real time, and document the conventions alongside your API.

Recommendation

Support an idempotency key on your write operations and declare it where agents can read it: an Idempotency-Key header parameter on your POST/PUT/PATCH operations in your OpenAPI spec for REST, or a client-supplied id argument on your GraphQL mutations. Agents retry on network failures, and without this a retry can double-charge or duplicate a record.

Recommendation

Document your error responses in your OpenAPI spec: give 4xx and 5xx responses a typed error schema (or use RFC 9457 application/problem+json). A consistent error object with a machine-readable code and a human-readable message lets agents handle failures without guessing.

Recommendation

Declare a versioning policy agents can rely on: version your API (in the URL path or a version header) and publish how you signal deprecation (a Sunset/Deprecation header or a documented timeline). Agents avoid integrating against a surface that can change without warning.

Recommendation

Use a consistent, documented pagination shape on your list endpoints (cursor-based preferred) and define the pagination fields in your OpenAPI response schemas, so agents can page through results without guessing the shape.

Recommendation

For long-running operations, return 202 Accepted and point agents at where to poll for the result (a status/location reference plus a job identifier in the body), documented in your OpenAPI spec, so work that does not finish in one request is still followable.

Recommendation

Publish an official CLI tool on npm, PyPI, or Homebrew. A CLI lets agents and developers script interactions with your product without building API integrations from scratch.

Recommendation

Publish official SDK packages across multiple language ecosystems (npm, PyPI, Go modules, RubyGems). Auto-generate them from your OpenAPI spec using tools like openapi-generator. For each package set the project URL or homepage to your product domain (package.json `repository`/`homepage`, PyPI `Home-Page` or `project_urls`, RubyGems `homepage_uri`) - this is how agents verify the package is your official SDK.

Recommendation

Define typed JSON response schemas for every endpoint in your OpenAPI spec. Agents rely on these to know what fields they will get back; missing or partial schemas force trial-and-error.

Recommendation

Make your API spec self-describing: a unique operationId and a description on every operation, typed parameters, and response schemas. For GraphQL, a fully typed schema with a documented cost or rate limit reads best.

Recommendation

Ensure API endpoints have unique operation IDs, typed schemas, and descriptions compatible with LLM function-calling formats.

Recommendation

Ensure every resource returned by resources/list reads cleanly via resources/read: declare a valid mimeType, return non-empty content, and make sure any URIs in the content resolve. Broken or empty resources break agent UX silently.

Recommendation

Establish a Wikipedia article and a Wikidata entity for your brand, with the domain set as the official website (Wikidata property P856) and a corresponding external link on Wikipedia. Wikipedia is the largest single source of citations in ChatGPT and a primary input to Knowledge Graphs across Google, Bing, and LLM training data. Earn third-party press coverage first to satisfy notability, then draft the article with cited references rather than self-promotion.

Recommendation

Register your MCP server on Smithery (smithery.ai) or mcp.so so agent platforms can discover your tools. Link the registry entry from your homepage or docs for bi-directional verification.

Recommendation

Publish a JavaScript/TypeScript SDK package on npm so developers can integrate your API programmatically. In package.json set `repository` to your source repo and `homepage` to your product domain - these links are how agents confirm the package is your official SDK rather than a third-party tool with a similar name.

Recommendation

Give your MCP server-card (at /.well-known/mcp/server-card.json) a display name, an icon or logo, and a description - all three together are what reads as a complete, branded listing agents can present.

Recommendation

Publish an Agent Skills index at /.well-known/agent-skills/index.json that lists your capabilities, with each skill carrying a name and a description so agents can find and parse what you offer.

Recommendation

Publish an Agent-to-Agent (A2A) agent card at /.well-known/agent-card.json describing your agent's capabilities, skills, and contact endpoint.

Recommendation

Create a /pricing.md file with your pricing tiers, features, and limits in plain markdown. This lets AI agents compare costs and recommend plans without scraping HTML pricing pages.

Recommendation

Serve your MCP server at /.well-known/mcp, publish a server-card.json at /.well-known/mcp/server-card.json, or reference it in llms.txt so agents can discover it automatically without manual URL input.

Recommendation

Add a ?mode=agent query parameter to your homepage that returns a structured, machine-readable view with API endpoints, authentication info, and key capabilities instead of marketing HTML.

Recommendation

Add HTTP Link: response headers (RFC 8288) advertising your sitemap, markdown alternates, API service descriptions, and API catalog. Example: Link: </sitemap.xml>; rel="sitemap", </index.md>; rel="alternate"; type="text/markdown".

Recommendation

Let agents fetch markdown by appending .md to page URLs. Required for any credit: serve a markdown homepage at /index.md. For full credit (2/2): also serve a .md twin for each content page (e.g. /docs/auth -> /docs/auth.md). Content-Type should be text/markdown and the body should start with a top-level heading (not HTML).

Recommendation

Add <lastmod> dates (W3C datetime, e.g. 2026-08-01) to your sitemap entries and update them when content actually changes. Aim for lastmod on at least half your entries with the newest within the last year. Verify with `curl https://yourdomain.com/sitemap.xml | grep lastmod`.

Recommendation

Create an llms.txt file at your domain root (/llms.txt) - the AI equivalent of robots.txt. Write at least 100 characters of real content: what your product is, what it does, and links to your key docs. Then verify it with `curl https://yourdomain.com/llms.txt` - you should see your text, not HTML. If your app returns its homepage for every URL (common with single-page apps), add a static file route so the raw text is served. A placeholder with just a heading earns no credit.

Recommendation

Format your llms.txt as a navigation index: start with a markdown heading, include markdown links to deeper resources, and keep it under 30,000 characters. If you have more to say, move long-form content into /llms-full.txt or per-section files (e.g. /docs/llms.txt, /api/llms.txt) and link to them from the main index.

Recommendation

Add schema.org speakable markup to your homepage indicating which sections are suitable for AI assistant text-to-speech readout. Use a Speakable property with CSS selectors targeting your key value proposition.

Recommendation

Expand your JSON-LD beyond Organization/WebSite. Add FAQPage for common questions, Service or Product for offerings, AggregateRating or Review for social proof, and BreadcrumbList for navigation context.

Recommendation

Make every link your llms.txt declares resolve to real content. Verify each one with `curl -L <url>` - you should see the linked document, not your homepage. If your app returns the homepage shell for unknown paths (common with single-page apps), a 200 status is not proof: check the body. Fix or remove any dead link; agents that follow the index treat a broken link as a dead end.

Recommendation

Advertise a markdown twin of each page with <link rel="alternate" type="text/markdown" href="..."> in the HTML head (or an equivalent Link response header), and make sure the advertised URL actually serves markdown - an advertisement pointing at HTML is worse than none. Verify the target with `curl -s <href>` and check the body starts with a heading, not <!doctype html>.

Recommendation

Publish an API catalog at /.well-known/api-catalog per RFC 9727. Serve it with Content-Type: application/linkset+json;profile="https://www.rfc-editor.org/info/rfc9727" and include a 'linkset' array with item entries pointing to your OpenAPI specs and service descriptions.

Recommendation

Pick one: (a) return Content-Type: text/markdown on GET <homepage> when the request sends Accept: text/markdown, or (b) publish a static /llms.md, /auth.md, or /agents.md file at your root with real markdown content. Option (b) is usually a single static file. This is the cold-discovery path for agents that land at your homepage from web search without reading llms.txt first.

Recommendation

Publish RFC 9728 protected-resource metadata at /.well-known/oauth-protected-resource. Include the resource field plus enough supporting metadata - your authorization servers, supported scopes, accepted bearer methods - that an agent can work out how to authenticate without first triggering a 401.

Recommendation

Publish /auth.md as a markdown prose walkthrough of how agents should obtain credentials. Serve it with Content-Type: text/markdown, lead with a top-level heading, and write at least ~200 chars of real content (not just a placeholder). See the WorkOS auth.md draft at https://workos.com/auth-md.

Recommendation

Structure /auth.md as the WorkOS spec prescribes: sections for Discover, Pick a method, Register, Claim, Use the credential, Errors, and Revocation, with spec anchor keywords (agent_auth, register_uri, identity_assertion, id-jag, WWW-Authenticate). Reference https://workos.com/auth-md.

Recommendation

Make your published auth-discovery chain traversable end to end: an agent starting at /auth.md (or your protected-resource metadata) should be able to follow the links to your authorization-server metadata and registration endpoint without hitting a dead link. Test the whole path, not just each file in isolation.

Recommendation

Publish RFC 9728 protected-resource metadata at /.well-known/oauth-protected-resource on your resource server (the host that actually serves the API, e.g. api.<apex>) with `resource` and `authorization_servers`. Publish RFC 8414 authorization-server metadata at /.well-known/oauth-authorization-server on the AS origin, and include the WorkOS auth.md `agent_auth` block with `register_uri`, `identity_types_supported` drawn from the spec enum (`anonymous`, `identity_assertion` - variants like `verified_email` or `urn:ietf:params:oauth:token-type:id-jag` belong inside `identity_assertion.assertion_types_supported`, not at the top level), and a sibling per-type block for each advertised type (`anonymous.credential_types_supported`; `identity_assertion.assertion_types_supported` + `credential_types_supported`) so agents can look up the request shape. Cross-link by listing the AS origin in PRM `authorization_servers`, and point `agent_auth.skill` back at your published /auth.md. Spec: https://workos.com/auth-md.

Recommendation

Return a 401 carrying a spec-shaped `WWW-Authenticate: Bearer resource_metadata="<your protected-resource metadata URL>"` header on your API's primary entry points, so an agent learns your auth requirements from one request instead of hunting for the well-known document. Point the metadata URL at /.well-known/oauth-protected-resource on the host that serves the API. Spec: https://workos.com/auth-md.

Recommendation

Make sure the URIs you advertise (in the AS metadata agent_auth block OR in your /auth.md prose) for register_uri, claim_uri, and revocation_uri actually resolve. An OPTIONS preflight should return any HTTP status (2xx/3xx/4xx that isn't 404). DNS-level failure or a 404 means the discovery block / prose is stale - either remove the URI or stand up the endpoint.

Recommendation

Publish a server card at /.well-known/mcp/server-card.json describing your MCP server. Required fields: name, description, version, serverUrl, tools[]. This lets agents preview your server before opening a transport connection.