blog/source/robots-mdleom.com.txt

# This file only applies to mdleom.com via web server's internal rewrite
User-agent: *
Disallow: /404
Disallow: /404.html
Disallow: _headers
Disallow: _redirects
Disallow: _routes.json

Sitemap: https://mdleom.com/sitemap.xml
build: add special robots.txt for mdleom.com - will exclude mirror sites from web crawler 2021-02-26 10:53:29 +00:00			`# This file only applies to mdleom.com via web server's internal rewrite`
			`User-agent: *`
			`Disallow: /404`
			`Disallow: /404.html`
chore(robots): exclude non-pages 2025-01-28 09:24:07 +00:00			`Disallow: _headers`
			`Disallow: _redirects`
			`Disallow: _routes.json`
build: add special robots.txt for mdleom.com - will exclude mirror sites from web crawler 2021-02-26 10:53:29 +00:00
			`Sitemap: https://mdleom.com/sitemap.xml`