User-agent: * Crawl-delay: 5 User-agent: * Disallow: /admin* Disallow: /settings* Disallow: /moderator* Disallow: /filelocker* Disallow: /dashboard* Disallow: /marketplace/vcr* Disallow: /marketplace/testing-queue* Disallow: /storage/* Disallow: /config/*