# robots.txt # Written by Kevin Cole 97.11.19 # Suggested by AltaVista (http://www.altavista.digital.com/) as a way to # limit spiders from indexing sites. AltaVista's pages were not clear # as to the proper location of this file... See "Adding a URL" on their # site for further info. (Scooter is AltaVista's spider.) User-agent: * # direct this to all spiders, not just Scooter Disallow: /cgi-bin # We don't want anyone looking here Disallow: /cgi-shl # Ditto Disallow: /images # Don't show them the Powered By stuff Disallow: /graphics # Why waste the database space? Disallow: /icons # Ditto Disallow: /lynx_help # Internal to this machine only Disallow: /manual # Apache internal to this machine only Disallow: /scripts # JavaScripts Disallow: /test # Internal to this machine only Disallow: /temp Disallow: /CSS # Cascading Style Sheets Disallow: /SysAdmin # System Administration Information Disallow: /Surveys # On-line data collection Disallow: /lists # Mailing lists are private Disallow: /misc # Not for production Disallow: /old # Like it says: Old Disallow: /piranha # I forget (RPM fancy thing) Disallow: /Priority/2006 # Not ready for prime time Disallow: /zopedocs # Installed by Zope Disallow: /ccg/x # I hope this discourages x*.html Disallow: /ccg/moved.html # This is what x*.html is linked to. Disallow: /rpmnew.html # This page is just the Apache stock page Disallow: /default.html # This page is just the Apache stock page Disallow: /default.ida # Reduce traffic from Code Red virus Disallow: /FNF.html # 404 File Not Found error page Disallow: /~root Disallow: /~kjcole/private Disallow: /~kjcole/test Disallow: /~kjcole/temp Disallow: /~kjcole/GRISJK