robots.txt

[Ravishshah]

What is robots.txt?

[gautamseo]

It is a text file which tells Google which pages to crawl and which not to crawl.

[riya11]

Robot.txt is a file also know as google bot is used to tells Google which pages to crawl and which not to crawl.

[rais26]

A robots.txt file tells search engine crawlers which pages or files the crawler can or can't request from your site.

[Rasull]

Robots.txt is a text file which tells Google which pages to crawl and which not to crawl.

[Jvimala]

Robots.txt is a text file webmasters create to instruct robots (typically search engine robots) how to crawl & index pages on their website.

regards,
Ansularjs Training Institute in Chennai

[gagarihuhe]

Robots.txt is a text file webmasters create to instruct robots (typically search engine robots) how to crawl & index pages on their website dermatologo pavia.

regards,
Ansularjs Training Institute in Chennai

very interesting, thanck you

[vijaykl32]

robots.txt file mention the google'c crawler that which website to crawl or not

[Marquestech]

We tell Google crawler 'to crawl and not to crawl' since this is will help your valuable data to be hidden like admin data.
So we tell google using robots.txt file on the website.
Very simple and easy right.