Click to See Complete Forum and Search --> : Auto Meta Tag Generator v3.6?


dalesellers
05-05-2004, 10:10 AM
I got referred to this generator which is on this site. I like it compared to the other generators I've seen out there. It seems easy to use and has a lot of options. Here's the link:

http://webdeveloper.earthweb.com/repository/javascripts/2002/09/99061/meta.htm

My question is that I'm trying to figure out how to get Google and other SEs to not list certain pages on my site when someone is doing a search. For example, if I have an educational article, I don't want someone coming in on a link to the middle of the article - they need to come in on the first page and only the first page. I still want SEs to index my whole site and I don’t' want to do anything that effects my page ranking.

My guess is that this has to do with the Robots setting (like on the Meta tag Generator I mentioned). Anybody got any ideas on what setting I should put the Robots value on to get the results I’m looking for and if this even has anything to do with Robots settings? Thanks for any help.

ShrineDesigns
05-05-2004, 03:07 PM
"some" (and i do mean some) web crawlers, look for a file in your root directory called "robots.txt", robots.txt should have directives set for how web crawlers access your site

see:

http://www.robotstxt.org/wc/robots.html
or
http://www.searchtools.com/robots/robots-txt.html

dalesellers
05-06-2004, 07:21 AM
Thanks for the info. Have you ever heard of the robots files effecting page rank?