1# Tells Scanning Robots Where They Are And Are Not Welcome 2# 3# User-agent: can also specify by name; "*" is for everyone 4# Disallow: disallow if this matches first part of requested path 5# 6# For now disallow all we can modify this as needed to allow certain crawlers. 7# 8Sitemap: http://www.jpl.nasa.gov/sitemap/xml/sitemap.xml 9User-agent: * 10Disallow: /93 11Disallow: /97 12Disallow: /airs 13Disallow: /ambassador 14Disallow: /attachments 15Disallow: /basics 16Disallow: /cal 17Disallow: /calendar 18Disallow: /cassini 19Disallow: /cchen 20Disallow: /cf_test 21Disallow: /cfdocs 22Disallow: /CFIDE 23Disallow: /cgi-bin 24Disallow: /code 25Disallow: /coldfusion 26Disallow: /comet 27Disallow: /csmad 28Disallow: /dailyplanet 29Disallow: /ds1news 30Disallow: /earth_test 31Disallow: /ethics 32Disallow: /europaorbiter 33Disallow: /facts 34Disallow: /forum 35Disallow: /fweb 36Disallow: /fweb2 37Disallow: /fweb_logs 38Disallow: /galileo 39Disallow: /genesis 40Disallow: /genesismission 41Disallow: /gif 42Disallow: /grace 43Disallow: /iae 44Disallow: /ice_fire 45Disallow: /internal 46Disallow: /jason 47Disallow: /jpl-internal-only 48Disallow: /jupiterflyby 49Disallow: /kids 50Disallow: /lidar 51Disallow: /lost+found 52Disallow: /magellan 53Disallow: /newsclips 54Disallow: /neo 55Disallow: /openhouse 56Disallow: /outgoing 57Disallow: /pictures 58Disallow: /planpro 59Disallow: /projects 60Disallow: /pscischool 61Disallow: /pso 62Disallow: /pso_old 63Disallow: /radar 64Disallow: /radioastronomy 65Disallow: /rm.old.cassini 66Disallow: /sait 67Disallow: /saturn 68Disallow: /scijinks 69Disallow: /sespd 70Disallow: /sircxsar 71Disallow: /s19 72Disallow: /snc 73Disallow: /spaceplace 74Disallow: /spacepwr 75Disallow: /srtm 76Disallow: /stardust 77Disallow: /stylesheets 78Disallow: /tbd 79Disallow: /teachers 80Disallow: /temp 81Disallow: /tours 82Disallow: /vyger 83Disallow: /vyger.old 84Disallow: /webalizer 85Disallow: /about_archive