# robots.txt # # ONE group, on purpose. Under RFC 9309 a crawler obeys only the group that names it: a bot with a # group of its own no longer reads the `User-agent: *` group at all, so a Disallow written there would # silently skip it. The AI assistants and answer engines are therefore named IN the `*` group — they # are welcome to the MCP pages, which are written to be quoted, and every rule below reaches all of # them (RobotsTxtTest). A bot that ever needs rules of its own gets a group of its own that repeats # every rule of this one; the test holds that too. User-agent: * User-agent: GPTBot User-agent: ChatGPT-User User-agent: OAI-SearchBot User-agent: ClaudeBot User-agent: Claude-User User-agent: anthropic-ai User-agent: PerplexityBot User-agent: Google-Extended User-agent: Bingbot Allow: /mcp Allow: /docs/mcp Allow: /docs/mcp.md Allow: /docs/mcp/skill.md Allow: /docs/mcp/chattee-skill.zip Allow: /docs/mcp/tools.json Allow: /llms.txt Allow: /llms-full.txt Allow: / Sitemap: https://chattee.ai/sitemap.xml