<!-----------start robots.txt--------->

User-agent: *
Disallow: /cgi-bin/
Disallow: /cgibin/

User-agent: msnbot
Crawl-delay: 3

User-agent: Teoma
Crawl-delay: 3

User-agent: Slurp
Crawl-delay: 3

<!------------end robots.txt------------>