# Allow crawling of all content User-agent: * Disallow: /recruiter/ Disallow: /change_language Crawl-delay: 1