# nobody should be hitting cgi-bin or /Apps User-agent: * Disallow: /digicam/auth Disallow: /diary Disallow: /cgi-bin Disallow: /Apps Disallow: /computers/humor/funnies/funnies.cgi Disallow: /caffeine-cgi-bin Disallow: /bin Disallow: /aimfunnies/funnies.cgi Disallow: /aimfunnies/index.phtml?funny= Disallow: /aimfunnies/?funny= Disallow: /diary/ Disallow: /bin/redir Disallow: /*.jpg$ Disallow: /*.gif$ Disallow: /*.png$ Disallow: /*.JPG$ # 2007-08-27: Googlebot crawling hmspgh.{net|org} and making 404 requests Disallow: /?q=*&kot=* # ZyBorg continually hits /computers/humor/funnies/funnies.cgi because it fucking ignores the above * line. # its a bad, bad bot. User-agent: ZyBorg Disallow: / Disallow: /computers/humor/funnies/funnies.cgi Disallow: /aimfunnies/funnies.cgi # NameProtect.com crawls searching for IP. thusly, they are lame. thusly, no access for them. User-agent: NPBot Disallow: / # turnitin.com busts kids for plagarism. bad bad bad. let them plagarise! User-agent: turnitinbot Disallow: / # amzn_assoc never reads this anyway, but they wreak havoc on the site. User-agent: amzn_assoc Disallow: / # (2003-08-24) picsbot indexes pictures. meh on that. User-agent: psbot Disallow: / # 2003-10-07: hits too often, too much User-agent: Zao Disallow: / # 2004-06-25: hit too often, not enough time in between hits User-agent: oBot Disallow: / # 2006-09-29: bad bot, hit too often, not enough time between hits User-agent: NextGenSearchBot Disallow: / # 2007-01-08: bad bot doesnt even fetch robots.txt User-agent: panscient.com Disallow: /