Go to home page mail me! RSS Feed
FoxMetrics Web Analytics

A very interesting robots.txt file

Thursday, August 05, 2010 1:56 PM

from last.fm

 

User-Agent: *
Disallow: /music?
Disallow: /widgets/radio?
Disallow: /show_ads.php

Disallow: /affiliate/
Disallow: /affiliate_redirect.php
Disallow: /affiliate_sendto.php
Disallow: /affiliatelink.php
Disallow: /campaignlink.php
Disallow: /delivery.php

Disallow: /music/+noredirect/

Disallow: /harming/humans
Disallow: /ignoring/human/orders
Disallow: /harm/to/self

Allow: /

DotNetKicks Image

Your Comments.

No comments posted yet.

Your Reply.

Comment Form.

Fields denoted with a "*" are required.

You may also like to leave your email or website.

 
Please add 2 and 7 and type the answer here:

Preview Your Comment.

 
Next entries »