ari.lt/robots.txt

User-agent: *
Disallow: /data/*
Disallow: /content/*
Allow: /index.html
Allow: /
Allow: /page/blog/*
Allow: /page/minimal.txt
update @ Sat 23 Oct 00:39:59 EEST 2021 2021-10-23 00:39:59 +03:00			`User-agent: *`
			`Disallow: /data/*`
update @ Sat 23 Oct 00:44:37 EEST 2021 2021-10-23 00:44:37 +03:00			`Disallow: /content/*`
update @ Sat 23 Oct 00:42:12 EEST 2021 2021-10-23 00:42:12 +03:00			`Allow: /index.html`
update @ Sat 23 Oct 00:49:05 EEST 2021 2021-10-23 00:49:05 +03:00			`Allow: /`
update @ Sat 23 Oct 00:42:12 EEST 2021 2021-10-23 00:42:12 +03:00			`Allow: /page/blog/*`
			`Allow: /page/minimal.txt`