# All robots will spider the domain User-agent: * User-agent: Mediapartners-Google* Disallow: User-agent: psbot Disallow: / User-agent: Yahoo-Blogs/v3.9 Crawl-delay: 60 User-agent: Slurp Crawl-delay: 60 User-agent: BecomeBot Crawl-Delay: 60 Dissallow: /cgi-bin/ Dissallow: /tmp/ Dissallow: /cache/ Dissallow: /class/ Dissallow: /images/ Dissallow: /include/ Dissallow: /install/ Dissallow: /kernel/ Dissallow: /language/ Dissallow: /templates_c/ Dissallow: /themes/ Dissallow: /uploads/ Dissallow: /leslie/ Dissallow: /articles/ezineready.php Dissallow: /articles/ezineready/ Disallow: /articles/comment.php Disallow: /articles/emailfriend.php Disallow: /articles/styles/ Disallow: /articles/cgi-bin/ Disallow: /articles/images/ Disallow: /articles/admin/ Disallow: /articles/admintemplates/ Disallow: /articles/emailtemplates/ Disallow: /articles/include/ Disallow: /articles/ioncube/ Disallow: /articles/photoimages/ Disallow: /articles/templates/ Dissallow: /articles/articles.php Dissallow: /articles/profiles/ # end of file