What is Robot.txt file

Discuss other engines here. AllTheWeb, AltaVista, Excite, Lycos, etc...
stevedswtpl
Posts: 27
Joined: Wed Jun 25, 2014 5:04 am
Contact:

Re: What is Robot.txt file

Postby stevedswtpl » Thu Aug 07, 2014 5:40 am

Robot.txt is very useful for any site. Web pages you don't want to index in google are hidden by Robot.txt code. For example:

User-agent: *
Disallow: /Webpage name

rishitha
Posts: 14
Joined: Wed Sep 10, 2014 1:32 am
Location: bangalore
Contact:

Re: What is Robot.txt file

Postby rishitha » Wed Sep 10, 2014 2:09 am

The pages which you want to hide from google name with robots.txt
Apartments in Whitefield , [url=http://www.realtycompass.com/apartments-around-electronic-city-in-bangalore-south
]Apartments in Electronic City[/url]

othername0104
Posts: 83
Joined: Sat Aug 27, 2016 8:33 am
Contact:

Re: What is Robot.txt file

Postby othername0104 » Sat Oct 01, 2016 1:58 am

Robots.txt is a text (not html) file you put on your site to tell search robots which pages you would like them not to visit. Robots.txt is by no means mandatory for search engines but generally search engines obey what they are asked not to do.

benben1
Posts: 85
Joined: Wed Sep 28, 2016 10:56 am
Contact:

Re: What is Robot.txt file

Postby benben1 » Sat Oct 01, 2016 7:23 am

The robots.txt file tells Google and Bing (as well as other robots) what pages on a website (if any) they are not allowed to access. If you disallow Google and Bing from accessing a page on your website, they will not crawl the page, and they may or may not include the URL in their index.
:D
The Video Marketers Profit Mining System Vid Reaper Review

adamjones
Posts: 104
Joined: Mon Nov 28, 2016 9:36 pm
Contact:

Re: What is Robot.txt file

Postby adamjones » Mon Dec 12, 2016 3:48 am

The Robots Exclusion Protocol (REP) is a group of web standards that regulate web robot behavior and search engine indexing.

Frizost
Posts: 35
Joined: Sun Dec 18, 2016 6:20 am
Contact:

Re: What is Robot.txt file

Postby Frizost » Sun Dec 18, 2016 6:28 am

But reasonably all target their keywords to get top ranking in Google only.

kumarsatish
Posts: 282
Joined: Thu Feb 25, 2016 3:10 am
Contact:

Re: What is Robot.txt file

Postby kumarsatish » Thu Apr 06, 2017 6:27 am

The robots exclusion commonplace, conjointly referred to as the robots exclusion protocol or just robots.txt, may be a commonplace employed by internet sites to speak with internet crawlers and alternative web robots. the quality specifies a way to inform the net mechanism concerning that areas of the web site shouldn't be processed or scanned.

luyenyen
Posts: 20
Joined: Fri Mar 03, 2017 2:54 am
Contact:

Re: What is Robot.txt file

Postby luyenyen » Tue Apr 11, 2017 1:24 am

The Robots.txt file is used to guide the search engine automatically to which pages you want it to search and then index the page. Most websites also have directories and files that do not need the robots of the search engine to visit. So creating a robots.txt file can help you in SEO.

techlysolutions
Posts: 26
Joined: Mon Jul 10, 2017 1:04 am
Contact:

Re: What is Robot.txt file

Postby techlysolutions » Mon Jul 10, 2017 1:10 am

Hello to all of you,

Robots.txt is a text file webmasters create to instruct web robots (typically search engine robots) how to crawl pages on their website...

kushan007
Posts: 10
Joined: Sat Nov 12, 2016 9:54 pm
Contact:

Re: What is Robot.txt file

Postby kushan007 » Mon Aug 07, 2017 9:01 am

webmaster tool to let bots know. What to index or follow

jackar56
Posts: 123
Joined: Sun Dec 11, 2016 11:28 pm
Contact:

Re: What is Robot.txt file

Postby jackar56 » Sun Aug 27, 2017 11:24 pm

Robots.txt is a text file that lists webpages which contain instructions for search engines robots. The file lists webpages that are allowed and disallowed from search engine crawling.
Best Online QA Training with certification

jackar56
Posts: 123
Joined: Sun Dec 11, 2016 11:28 pm
Contact:

Re: What is Robot.txt file

Postby jackar56 » Sun Aug 27, 2017 11:26 pm

Robots.txt is a text file that lists webpages which contain instructions for search engines robots. The file lists webpages that are allowed and disallowed from search engine crawling.
Best Online QA Training with certification

Aaronerows
Posts: 2
Joined: Mon Sep 04, 2017 8:02 am
Location: Bulgaria
Contact:

What is Robot txt file

Postby Aaronerows » Sat Sep 09, 2017 3:26 pm

That depends on the part of canada you are from... and what type of water you are onwell or city

I have a PH out of the tap of 7.4-7.6.... hard water as well.
Reliable best trading system for reliable investments


Return to “All Other Engines”

Who is online

Users browsing this forum: No registered users and 3 guests