tims15
07-17-2005, 11:59 AM
What is the easiest way to have google and other search engines spider all parts of my website but only show specific pages in search results? I've read of a few ways, but also read that they aren't effective.
Click to See Complete Forum and Search --> : Spiders
felgall
07-17-2005, 05:02 PM
<meta name="robots" content="noindex,follow" /> on pages you don't want listed.
tims15
07-18-2005, 10:52 AM
Cheers
Spiderline
07-19-2005, 05:51 PM
Or the same cna be on a robots.txt file to limit robots you do not want to index your site.
Useing the Meta tag above will lower the chance of you comming at the top of the search results for anything listed in the page. "follow" will keep it running to all other pages.
A good trick is to use a sitemap.html with noindex and follow and have it attached to the splashpage with a hidden or small link. have this page linking everything on your site. It also make for easy indexing for other crawlers.
Useing the Meta tag above will lower the chance of you comming at the top of the search results for anything listed in the page. "follow" will keep it running to all other pages.
A good trick is to use a sitemap.html with noindex and follow and have it attached to the splashpage with a hidden or small link. have this page linking everything on your site. It also make for easy indexing for other crawlers.
TechEvangelist
07-22-2005, 08:59 AM
Hidden links are a violation of search engine guidelines and may get your site penalized. The site map is a good idea, but a hidden link may not be.
webdeveloper.com
Copyright Internet.com Inc., All Rights Reserved.
Copyright Internet.com Inc., All Rights Reserved.