cgit/robots.txt
June McEnroe cc6d9cc7fc ui-tree,ui-blame: bail from blame if blob is binary
This avoids piping binary blobs through the source-filter. Also prevent
robots from crawling it, since it's expensive.

Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
2022-12-19 15:14:05 +01:00

5 lines
68 B
Plaintext

User-agent: *
Disallow: /*/snapshot/*
Disallow: /*/blame/*
Allow: /