User-agent: * Allow: / Sitemap: https://ailearning.global/sitemap-index.xml # No Disallow lines here, deliberately. # # Client proposal areas (/icda/, /unicda/, /minerd/, /gruporamos/, /inicia/), the # per-recipient Challenge letters under /proposals/, the families consent template, # the Orientador's area and the unpublished unit drafts are all shared by link and # must not be indexed. They are kept out of search by # # X-Robots-Tag: noindex, nofollow, noarchive # # set per path in firebase.json, which reaches the PDFs and DOCX too — a document # cannot carry a meta tag, and until 2026-11-15 this site was on GitHub Pages, which # cannot send a header at all. That is why the paths were listed here instead. # # Listing them was worse than useless (cemi-unindexed-documents rule 3): a crawler # blocked by robots.txt never reads the noindex, so it can still list a bare URL it # finds linked elsewhere — and robots.txt is public, so the list itself advertised # exactly where the client documents live. # # Before adding a Disallow here, check whether an X-Robots-Tag in firebase.json is # the right answer instead. It almost always is.