How to Create the Robots txt file.

[Nexevo Technologi]

How to Create the Robots txt file.

[davidwarner033]

Robots.txt file is create by then click on the Tools menu option on the left-hand side of the screen after you select one of your verified sites. You’ll see a "Generate robots.txt" link among the tool options. That’s what you want. By default, the tool is designed to let you create a robots.txt file to allow all robots into your site. That’s kind of odd. By default, all robots will come into your site. If you want them, then there’s no need to have a robots.txt file at all. It’s like pinning a note to your chest reminding yourself to breathe.

[brknny]

Use a text editor to create the robots.txt file and add REP directives to block content from being visited by the bots.Create a list of Disallow directives listing the content you want blocked.