Commit Graph

2 Commits

Author SHA1 Message Date
June McEnroe
cc6d9cc7fc ui-tree,ui-blame: bail from blame if blob is binary
This avoids piping binary blobs through the source-filter. Also prevent
robots from crawling it, since it's expensive.

Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
2022-12-19 15:14:05 +01:00
Jason A. Donenfeld
23debef621 robots.txt: disallow access to snapshots
My dmesg is filled with the oom killer bringing down processes while the
Bingbot downloads every snapshot for every commit of the Linux kernel in
tar.xz format. Sure, I should be running with memory limits, and now I'm
using cgroups, but a more general solution is to prevent crawlers from
wasting resources like that in the first place.

Suggested-by: Natanael Copa <ncopa@alpinelinux.org>
Suggested-by: Julius Plenz <plenz@cis.fu-berlin.de>
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
2013-08-12 13:14:10 -06:00