What is Robot.txt file

Discuss other engines here. AllTheWeb, AltaVista, Excite, Lycos, etc...
photoeditor
Posts: 117
Joined: Wed Mar 22, 2017 5:58 am
Contact:

Re: What is Robot.txt file

Postby photoeditor » Mon Apr 09, 2018 11:13 pm

Robots.txt file helps to index any website in different search engines.

dombowkett
Posts: 86
Joined: Tue Feb 13, 2018 3:30 am
Location: CA
Contact:

Re: What is Robot.txt file

Postby dombowkett » Sat Aug 25, 2018 10:00 am

Robots.txt file is set of indexing instructions for search engine. http://raptorhomes.com

Josh44
Posts: 20
Joined: Thu Aug 30, 2018 11:26 pm
Contact:

Re: What is Robot.txt file

Postby Josh44 » Mon Sep 03, 2018 12:36 am

Robot.txt file helps search engine and other bots to crawl your website. You can instruct bots what to crawl and how to behave while indexing your website by including certain rules into the robots text file.








-----------------------------------------------------------------------------
https://www.netguru.co/services/nodejs
Hire The Best Node.js Developers

garmincomexpress
Posts: 5
Joined: Wed Oct 10, 2018 5:32 am
Location: Canada
Contact:

Re: What is Robot.txt file

Postby garmincomexpress » Mon Oct 15, 2018 10:11 pm

Robots.txt is a text (not html) file you put on your site to tell search robots which pages you would like them not to visit. Robots.txt is by no means mandatory for search engines but generally search engines obey what they are asked not to do.
Garmin.com/express


Return to “All Other Engines”

Who is online

Users browsing this forum: No registered users and 1 guest