robots.txt

For discussion of general Search Engine Optimization, not specific to any engine.
Ravishshah
Posts: 22
Joined: Wed May 08, 2019 4:07 am
Location: Connecticut USA
Contact:

robots.txt

Postby Ravishshah » Mon Jun 24, 2019 4:08 am

What is robots.txt?

gautamseo
Posts: 7
Joined: Tue May 21, 2019 3:01 am
Contact:

Re: robots.txt

Postby gautamseo » Mon Jun 24, 2019 4:17 am

It is a text file which tells Google which pages to crawl and which not to crawl.
BEST SEO Expert in India, SEO EXPERT IN KOLKATA, SEO EXPERT IN MUMBAI, SEO EXPERT IN CHANDIGARH, SEO FREELANCER IN HYDERABAD

riya11
Posts: 15
Joined: Thu Jun 20, 2019 11:26 pm
Contact:

Re: robots.txt

Postby riya11 » Tue Jun 25, 2019 12:24 am

Robot.txt is a file also know as google bot is used to tells Google which pages to crawl and which not to crawl.

rais26
Posts: 76
Joined: Wed Jul 03, 2019 3:10 am
Contact:

Re: robots.txt

Postby rais26 » Fri Jul 26, 2019 9:50 pm

A robots.txt file tells search engine crawlers which pages or files the crawler can or can't request from your site.

Rasull
Posts: 27
Joined: Mon Jul 29, 2019 12:44 am
Location: Dubai, UAE
Contact:

Re: robots.txt

Postby Rasull » Mon Jul 29, 2019 1:08 am

Robots.txt is a text file which tells Google which pages to crawl and which not to crawl.

Jvimala
Posts: 11
Joined: Mon Jul 29, 2019 5:03 am
Contact:

Re: robots.txt

Postby Jvimala » Mon Jul 29, 2019 5:33 am

Robots.txt is a text file webmasters create to instruct robots (typically search engine robots) how to crawl & index pages on their website.

regards,
Ansularjs Training Institute in Chennai

gagarihuhe
Posts: 1
Joined: Mon Jul 29, 2019 12:25 pm
Contact:

Re: robots.txt

Postby gagarihuhe » Mon Jul 29, 2019 12:29 pm

Jvimala wrote:Robots.txt is a text file webmasters create to instruct robots (typically search engine robots) how to crawl & index pages on their website dermatologo pavia.

regards,
Ansularjs Training Institute in Chennai


very interesting, thanck you :D

vijaykl32
Posts: 25
Joined: Thu Jul 25, 2019 1:50 am
Contact:

Re: robots.txt

Postby vijaykl32 » Mon Jul 29, 2019 10:09 pm

robots.txt file mention the google'c crawler that which website to crawl or not

Marquestech
Posts: 3
Joined: Tue Jul 30, 2019 4:01 am
Contact:

Re: robots.txt

Postby Marquestech » Tue Jul 30, 2019 4:50 am

We tell Google crawler 'to crawl and not to crawl' since this is will help your valuable data to be hidden like admin data.
So we tell google using robots.txt file on the website.
Very simple and easy right.


Return to “SEO Forum”

Who is online

Users browsing this forum: No registered users and 115 guests