��# llms.txt for https://www.earwigacademic.com/ # Goal: Be discoverable in AI search results while controlling training access. # === OpenAI GPTBot (used for training ChatGPT) === User-Agent: GPTBot Disallow: / # === Google-Extended (used for Gemini AI search) === User-Agent: Google-Extended Allow: / # === Anthropic Claude (not currently public-facing, but safe to block) === User-Agent: anthropic-ai Disallow: / # === Perplexity.ai (AI search engine) === User-Agent: PerplexityBot Allow: / # === Common Crawl (data source for multiple models) === User-Agent: CCBot Disallow: / # === Microsoft Bing (used in ChatGPT browsing mode) === User-Agent: bingbot Allow: / # === Neeva (acquired by Snowflake, but used in some AI search) === User-Agent: NeevaAI Allow: /