1. No Profile Picture
    Registered User
    SEO Chat Explorer (0 - 99 posts)

    Join Date
    Mar 2013
    Rep Power

    Exclamation Is this Robots.txt format Correct ? Plz Post a Correct Format

    User-agent: *
    User-agent: Googlebot
    User-agent: Googlebot-Image
    User-agent: MSNBot
    User-agent: Slurp
    User-agent: Gigabot
    User-agent: Scrubby
    User-agent: Robozilla

    Sitemap: wwwdotexampledotcom/sitemap.xml


  2. #2
  3. My views are my own ;)
    SEO Chat Good Citizen (1000 - 1499 posts)

    Join Date
    Jun 2012
    Clifton Park, New York
    Rep Power
    There's something missing: Action... are you trying to allow or disallow all the listed crawlers?
    Everything will be ok in the end

    If it's not ok, it's not the end
  4. #3
  5. SEO Consultant
    SEO Chat High Scholar (3500 - 3999 posts)

    Join Date
    Apr 2010
    Atlanta, GA
    Rep Power
    Rather than explain, it's easier to share a link: Robots.txt Tutorial
    www.joshz.net - SEO Web Design | On-Page SEO | SEO Audits & Reports | Link Building
    BASED IN ATLANTA, GA. Contact: Skype/Phone/Email (Discounted rates available for SEOChat members only - email josh@joshz.net)
    Please EMAIL me at the above address and do not send Private Messages. I rarely check them and your message will be lost.
  6. #4
  7. SEO Insultant
    SEO Chat Mastermind (5000+ posts)

    Join Date
    Oct 2005
    sharing a room with my ego
    Rep Power
    And no, it is not correct.

    For starters, following User-agent: *,
    why list all the other bots?

    In short: best for you to follow joshz' link to begin with. Good luck.

    Comments on this post

    • avinashmurthy agrees : Thanks for your reply dzine..

Similar Threads

  1. Will Disallow: /*?format= in robots blog all occurrences of ?format= in a URL?
    By geoffw8_ in forum Web Design, Coding and Programming
    Replies: 6
    Last Post: Jan 6th, 2013, 12:34 PM
  2. Correct site showing WRONG domain name! How to correct?
    By jeffmartini13 in forum Google Optimization
    Replies: 3
    Last Post: Oct 9th, 2009, 07:38 AM
  3. Robots.txt - is it correct ?
    By varun8211 in forum Search Engine Optimization
    Replies: 3
    Last Post: Jun 28th, 2009, 12:08 PM
  4. What is the correct format for robots meta?
    By malhyp in forum Web Design, Coding and Programming
    Replies: 7
    Last Post: Aug 17th, 2008, 11:42 AM
  5. Is my Robots.txt File Correct????
    By SquirrelJ in forum Search Engine Optimization
    Replies: 1
    Last Post: Feb 19th, 2008, 08:57 AM

IMN logo majestic logo threadwatch logo seochat tools logo