Blue Cat Buxton
08-17-2004, 06:47 AM
How can I ban the Alexa robot from going through my sites? It gobbles up bandwidth for no percieveable benefit?
Can I simply use the robots.txt ?
Are there any downsides that I am missing if I do this?
Would it be better to limit it just to say the front page. If so, how can I best do this?
Can I simply use the robots.txt ?
Are there any downsides that I am missing if I do this?
Would it be better to limit it just to say the front page. If so, how can I best do this?