# This is the robots.txt file, it is a standard file found on the web these days. # Use multiple 'Disallow:' lines to specify which files (or directories) that # you do not want robots (or spiders) automatically linking to. User-agent: * Disallow: /cgi-bin/ Disallow: .htaccess Disallow: .htgroup Disallow: .htpasswd Disallow: /old/