User-agent: * Disallow: /dev/ Disallow: /internal/ Disallow: /clients/ Disallow: /cgi-bin/ Disallow: /crm/ Disallow: /backups/ Disallow: /site.old/ Disallow: /temp/ Disallow: /tools/