curben
/
blog
mirror of https://gitlab.com/curben/blog
1
0
Fork 0
Code Issues Releases Wiki Activity

build: add special robots.txt for mdleom.com 

- will exclude mirror sites from web crawler
This commit is contained in:
Ming Di Leom 2021-02-26 10:53:29 +00:00
parent 2379cd6672
commit 0640fbd8f7
No known key found for this signature in database
GPG Key ID: 32D3E28E96A695E8
1 changed files with 6 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
source/robots-mdleom.com.txt Normal file
View File

@ -0,0 +1,6 @@
# This file only applies to mdleom.com via web server's internal rewrite
User-agent: *
Disallow: /404
Disallow: /404.html
Sitemap: https://mdleom.com/sitemap.xml