Spiders crawling nonlinked areas

[JuliePgh]

Can spiders crawl to areas of a website which aren't linked to the rest of a website? Do they crawl through links only or can they detect pages under the root directory?

The reason I'm asking is that I have some "private" areas not linked to the rest of my self, but under the same folder. I send links to people I want to see these areas.

[Kravvitz]

You should use a robots.txt file to tell robots where they are and are not allowed to go. (http://www.google.com/search?q=robots.txt+%7Etutorial)

[ray326]

Spiders cannot find unlinked content or index content that requires authentication.

[Kravvitz]

True, but people on other sites can link to files that don't have links posted on the public pages of a website -- thus a spider can find them that way.