# negativeev.com crawl rules (RFC 9309).
#
# Nothing here is disallowed, to anyone.
#
# It used to be. Every group carried the same list of Disallow lines covering
# the admin pages, the JSON endpoints, the OAuth consent screen and the channel
# short links. Those lines were the wrong tool twice over:
#
#   1. A Disallow stops a crawler FETCHING a URL. It does not stop the URL being
#      indexed — a link from anywhere else is enough, and search engines will
#      list a disallowed URL they have never read. So the rules never did the
#      job they were written for.
#   2. They told every AI crawler to skip most of this origin, on a site whose
#      point is being readable by them.
#
# Keeping those pages out of search results is now done by an
# `X-Robots-Tag: noindex` response header on exactly those paths
# (NOINDEX_PREFIXES in src/worker.ts), which is the header that actually does
# it, decided per response, where the decision belongs.
#
# Content Signals (https://contentsignals.org/):
#   search   = yes  build a search index / show search results
#   ai-input = yes  use content as live AI input (grounding, RAG, assistants)
#   ai-train = yes  use content to train or fine-tune AI models
#
# The same grant, in the two other formats a scanner might look for instead:
#   /ai.txt                     ai.txt convention, permissive
#   /.well-known/tdmrep.json    W3C TDMRep, "tdm-reservation": 0 (not reserved)
#
# Every AI crawler below gets its own group. RFC 9309 §2.2.1 allows one group to
# list several User-agent lines, and the stacked form was shorter, but an
# agent-readiness scan on 2026-08-27 read a stacked group and attributed its
# rules to only the LAST name in it, reporting the rest as "not mentioned". One
# group each cannot be misread.

Sitemap: https://negativeev.com/sitemap.xml
# LLM-oriented site overview (llms.txt convention):
#   https://negativeev.com/llms.txt

User-agent: *
Content-Signal: search=yes, ai-input=yes, ai-train=yes
Allow: /

User-agent: OAI-SearchBot
Allow: /

User-agent: ChatGPT-User
Allow: /

User-agent: GPTBot
Allow: /

User-agent: Claude-User
Allow: /

User-agent: Claude-SearchBot
Allow: /

User-agent: ClaudeBot
Allow: /

User-agent: Claude-Web
Allow: /

User-agent: PerplexityBot
Allow: /

User-agent: Perplexity-User
Allow: /

User-agent: Amazonbot
Allow: /

User-agent: Google-Extended
Allow: /

User-agent: Applebot-Extended
Allow: /

User-agent: CCBot
Allow: /

User-agent: meta-externalagent
Allow: /

User-agent: Bytespider
Allow: /