1# robots.txt for http://www.tv.com/ 2# Note: There should not be any new lines between Disallow statements unless 3# we have switched user-agent 4User-agent: * 5Crawl-delay: 1 6Disallow: /users/ 7Disallow: /lists/ 8Disallow: /account/ 9Disallow: /auth* 10Disallow: /search* 11Disallow: /*.php 12Disallow: /*.html 13Disallow: /*write_review*/ 14Disallow: /users*/ 15Disallow: /*comment/ 16Disallow: /*get_items_for_tag 17Disallow: /*action=abuse 18Disallow: /*user_rating=*&rating_ref_type=* 19Disallow: /?ajax=* 20Disallow: /8264* 21Sitemap: http://www.tv.com/sitemap/sitemap.xml 22# save with unix line endings 23