View Full Version : What Is The Limit of A Robot.txt File.?
ankitasharma123
02-21-2019, 03:13 AM
Hello friends,
What Is The Limit of A Robot.txt File.?
thaichacha
02-21-2019, 09:53 PM
there is a limit to robots.txt file in terms of filesize 500 KB as Vipin points out. A maximum file size may be enforced per crawler. Content which is after the maximum file size may be ignored. Google currently enforces a size limit of 500kb.
MVMinfotech18
02-22-2019, 12:22 AM
Google has a limit of only being able to process up to 500 KB present in the robots.txt file.
sandymary
02-22-2019, 02:36 AM
Google has setted a limit of process up to 500 KB in the robots.txt file.
RH-Calvin
02-26-2019, 10:52 PM
A maximum file size may be enforced per crawler. Content which is after the maximum file size may be ignored. Google currently enforces a size limit of 500kb.
Powered by vBulletin® Version 4.2.2 Copyright © 2024 vBulletin Solutions, Inc. All rights reserved.