What is the limit for a robot.txt file?

Discuss other engines here. AllTheWeb, AltaVista, Excite, Lycos, etc...
photoeditor
Posts: 117
Joined: Wed Mar 22, 2017 5:58 am
Contact:

What is the limit for a robot.txt file?

Postby photoeditor » Mon Apr 09, 2018 11:19 pm

Hello friends,
What is the limit for a robot.txt file?

Lovepop
Posts: 1
Joined: Wed Apr 18, 2018 3:10 am
Contact:

Re: What is the limit for a robot.txt file?

Postby Lovepop » Thu Apr 19, 2018 3:38 am

What is the limit for a robot.txt file? This is very interesting. Many people are interested in this. ไฮโล

nova62
Posts: 27
Joined: Tue Mar 27, 2018 9:48 pm
Contact:

Re: What is the limit for a robot.txt file?

Postby nova62 » Sun Apr 22, 2018 8:00 pm

It is recommended to use gloves when using this tool.














royalhill-online

breezer
Posts: 13
Joined: Sun Apr 22, 2018 3:25 am
Location: Mataram, Lombok
Contact:

Re: What is the limit for a robot.txt file?

Postby breezer » Mon Apr 23, 2018 3:06 am

500kb for .txt file is too big LOL

dombowkett
Posts: 86
Joined: Tue Feb 13, 2018 3:30 am
Location: CA
Contact:

Re: What is the limit for a robot.txt file?

Postby dombowkett » Sat Aug 25, 2018 9:44 am

Robots.txt file size can be max. 500kb for better crawling. http://raptorhomes.com


Return to “All Other Engines”

Who is online

Users browsing this forum: No registered users and 2 guests