# /robots.txt for http://newsinc.net/ # See http://www.robotstxt.org/ # David M. Cole User-Agent: * Disallow: /newsinc/ Disallow: /sphider/ Disallow: /morgue/ Disallow: /isearch2/ Disallow: /2009/ Disallow: /2010/ Allow: / User-Agent: Googlebot Disallow: *.pdf Disallow: /newsinc/ Disallow: /sphider/ Disallow: /morgue/ Disallow: /isearch2/ Disallow: Reg_form.html Disallow: old_Reg_form.html Disallow: /2009/ Disallow: /2010/ Allow: /