User-Agent: *
Allow: /
Disallow: /api/

User-Agent: GPTBot
Allow: /
Allow: /llms.txt
Allow: /llms-full.txt

User-Agent: ChatGPT-User
Allow: /
Allow: /llms.txt
Allow: /llms-full.txt

User-Agent: PerplexityBot
Allow: /
Allow: /llms.txt
Allow: /llms-full.txt

Sitemap: https://www.crabcut.ai/sitemap.xml