508
04-04-2006, 02:24 AM
I have placed my website on different message boards. How can I use robots.txt or any other tool so the message boards cannot get indexed by a search engine?
|
Click to See Complete Forum and Search --> : Robots.txt or other tools with outside sites 508 04-04-2006, 02:24 AM I have placed my website on different message boards. How can I use robots.txt or any other tool so the message boards cannot get indexed by a search engine? pcthug 04-04-2006, 03:49 AM You cannot control whether or not an outside site is indexed by a search engine, the most you can do is add rel="nofollow" to any hyperlinks that you wish search engines to ignore TiGGi 04-04-2006, 08:46 AM If you want to deny access to a directory add this to your robots.txt User-agent: * Disallow: /directory/ You can add as many directories as you want. If you want to block crawlers from the whole site do this: User-agent: * Disallow: / webdeveloper.com
Copyright Internet.com Inc., All Rights Reserved. |