www.webdeveloper.com
Results 1 to 9 of 9

Thread: what is the role of robot.txt in SEO?

  1. #1

    what is the role of robot.txt in SEO?

    Hello Guys,
    Please share your review about robot.txt, what is this and how its important for seo. I am not more familiar to seo task so please help me to know about it. Thanks

  2. #2
    Join Date
    Jan 2013
    Posts
    52
    Robot text file is an essential part of site from crawlers point of view you may save all those files in it taht you dont want to read by crawlers.

  3. #3
    Join Date
    Mar 2013
    Posts
    139
    You can make use of robot.txt files if you don't want to crawl any page etc..

  4. #4
    Join Date
    Mar 2013
    Location
    USA
    Posts
    3
    It is important part for the site . From that you can restrict the crawler to carwl that particular page you doesn't want to crawl.

  5. #5
    Join Date
    Nov 2011
    Location
    chennai
    Posts
    73
    Robots.txt helps, not to index website personal pages in search engines. It acts like security for your personal page.

  6. #6
    Join Date
    Feb 2013
    Location
    Mississauga, Canada
    Posts
    152
    Robots.txt is a text (not html) file you put on your site to tell search robots which pages you would like them not to visit. Robots.txt is by no means mandatory for search engines but generally search engines obey what they are asked not to do. It is important to clarify that robots.txt is not a way from preventing search engines from crawling your site (i.e. it is not a firewall, or a kind of password protection) and the fact that you put a robots.txt file is something like putting a note “Please, do not enter” on an unlocked door – e.g. you cannot prevent thieves from coming in but the good guys will not open to door and enter. That is why we say that if you have really sen sitive data, it is too na´ve to rely on robots.txt to protect it from being indexed and displayed in search results.
    *** spam signature deleted by moderator - please do not put it back ***

  7. #7
    Join Date
    Apr 2013
    Posts
    9
    way to tell search engines which files and folders on your Web site to avoid is with the use of the Robots metatag.

  8. #8
    Join Date
    Sep 2012
    Posts
    720
    Robots.txt is a text file you put on your site to tell search robots which pages you would like them not to visit.

  9. #9
    Join Date
    Feb 2012
    Posts
    83
    It is used to restrict or allow bots to visit our site.

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
HTML5 Development Center



Recent Articles