The robots.txt file format allows control over bots crawling on a site. It allows a site owner to direct web robots (crawlers) to mainly exclude sections of a website from crawling, and indirectly from search engine indexing. The file, placed at /robots.txt, follows a format as defined at robotstxt.org.

Redirected robots.txt

Google will follow a redirect from /robots.txt and will use a valid file in the redirect target location. Verified via Google’s testing tool in June 2022.

Note that this conflicts with Google’s webmaster reference, which states the file must be located on /robots.txt.

BING will also follow redirects, as confirmed with the testing tool for BING.