Mind blowing SEO improving and Google rank increasing Signature Rotator SEO Tool.


What is use of robot.txt in SEO ?

Christi Smith Offline referral

Posts: 34
Joined: Apr 2017
Reputation: 0

Junior Member
(01-30-2018 07:39 AM)Shivam Tripathi Wrote:  Hello Friends,

Actually i want to know litle bit information about robot.txt in SEO.so,please share with me.

Robots.txt file is used to tell Google, which pages of the website need to be crawled or which file need to blocked from crawling. It is also recommended to mention Sitemap file in it.
Memorials4u - Cremation Urn | Adult Urns
Shilpa111 Offline referral


Robot.txt file is controlling our website in the search engines. Example if we will disable Robot.txt file means our website is not showing under search engines. By using this we control the indexing of our website in the search result.
fayeseom Offline referral

Posts: 113
Joined: Feb 2016
Reputation: 1

The robots.txt is used to instruct robots (like Googlebot) what pages to index and which pages to ignore, on your website.
moazzem28 Offline referral

Posts: 38
Joined: May 2018
Reputation: 1

Junior Member
When you want to block search engine to crawl and index your webpage, you need the robots.txt file.

User(s) browsing this thread: 1 Guest(s)