Mind blowing SEO improving and Google rank increasing Signature Rotator SEO Tool.

Thread: 

robots[.]txt is blocking all spiders

Ads
banner
banner
banner
boopika Offline referral

Posts: 6
Joined: Jul 2014
Reputation: 0

#1
Junior Member
what does it meant?
robots[.]txt is blocking all spiders

how can i rectify this error
invisibe_dude Offline referral

Posts: 820
Joined: May 2013
Reputation: 24

#2
Posting Freak
This file is used by most major search engines. Spam-bots and other bogus ones will not be affected sorry.
conocer gente gratis, chatear y buscar amigos en chat gratis o ligar. Citas, contactos y encuentros.
marcus_avrelius Offline referral

Posts: 2,424
Joined: May 2013
Reputation: 102

#3
Support Team
Where do you get this message?

Don't use any auto generators do it manually!!!!

Make sure you don't have similar code in your file:

Code:
User-agent: Googlebot
Disallow: /
User-agent: googlebot-image
Disallow: /
User-agent: googlebot-mobile
Disallow: /
User-agent: MSNBot
Disallow: /
User-agent: Slurp
Disallow: /
User-agent: Teoma
Disallow: /
User-agent: twiceler
Disallow: /
User-agent: Gigabot
Disallow: /
User-agent: Scrubby
Disallow: /
User-agent: Robozilla
Disallow: /
User-agent: Nutch
Disallow: /
User-agent: ia_archiver
Disallow: /
User-agent: baiduspider
Disallow: /
User-agent: naverbot
Disallow: /
User-agent: yeti
Disallow: /
User-agent: yahoo-mmcrawler
Disallow: /
User-agent: psbot
Disallow: /
User-agent: asterias
Disallow: /
User-agent: yahoo-blogs/v3.9
Disallow: /
Looking for ways to make money fast? Read this article and start earning money todya fast, easy or for free.
KevinMax Offline referral

Posts: 575
Joined: Aug 2013
Reputation: 78

#4
Senior Member
Marcus is right! Check your robots file first, Understand why it is intended for... I am sure you file is blocking all the web spiders to crawl.
Check also this! https://support.google.com/webmasters/an...ic=6061961
SimranKaur Offline referral

Posts: 7
Joined: Jun 2014
Reputation: 0

#5
Junior Member
(07-17-2014 03:24 PM)boopika Wrote:  what does it meant?
robots[.]txt is blocking all spiders

how can i rectify this error

Have you used right codes in Robots txt file?

Do check your file, might be there is some mistake on your file.
boopika Offline referral

Posts: 6
Joined: Jul 2014
Reputation: 0

#6
Junior Member
I got this message from Bing webmaster, yes of course I'm not using any tools.
marcus_avrelius Offline referral

Posts: 2,424
Joined: May 2013
Reputation: 102

#7
Support Team
Use Fetch as Bingbot: http://www.bing.com/webmaster/help/fetch...t-fe18fa0d to check if you are really blocking Bing crawlers.

If Google says nothing then it's probably a problem with Bing cause Google will be the first to warn you about that!!
Looking for ways to make money fast? Read this article and start earning money todya fast, easy or for free.





User(s) browsing this thread: 1 Guest(s)