1# robots.txt for http://www.gnu.org/ 2 3User-agent: * 4Crawl-delay: 2 5Disallow: /private/ 6Disallow: /savannah-checkouts/ 7Disallow: /server/trans-map.html 8 9# RT #1298215. 10User-agent: * 11Crawl-delay: 4 12Disallow: /server/trans-map.html 13 14# RT #1298215. 15User-agent: * 16Crawl-delay: 4 17Disallow: /server/select-language.html 18 19Sitemap: http://www.gnu.org/sitemap.xml 20 21