User-agent: AhrefsBot Disallow: /blog/* Crawl-delay: 5 User-agent: * Disallow: /blog/* Crawl-delay: 5