# robots.txt --- Exclude cgi-bin User-agent: * Disallow: /cgi-bin Disallow: /images Disallow: /stats Disallow: /ads Disallow: /_private # Let Alexa Internet retain a copy. User-agent: ia_archiver Disallow: /cgi-davidreilly Disallow: /cgi-bin # Let I/AGENT take it all but cgi-davidreilly User-agent: I/AGENT Disallow: /cgi-bin