Benefits of Excluding robots
Excluding robots from certain sections of the site can benefit you :
1. You can prevent search engine crawlers from eating up unnecessary bandwidth of your site, For some hosting bandwidth is costly and it is important bandwidth is saved by excluding crawlers from certain sections.
2. Excluding robots from comments section if you run a blog is very good, It prevents robots from seeing or giving credit to any unnecessary links.
3. Excluding robots also helps in preventing sensitive information of your site not getting exposed in search results
I just know things above, so can you list more?
Users Browsing this Thread
There are currently 1 users browsing this thread. (0 members and 1 guests)