How to block all search engines not to be visible?

[toplisek]

How to block all search engines not to be visible content with ROBOTS TXT

Simple question...

[LeeU]

Are you saying you don't want any of the search engines to spider your web site? Not sure what you mean by the "not to be visible" part.

[toplisek]

Hi,
I would like to block Subdomain and also folders as sometimes I need testing website which is in testing period but my friends can see.

Goal is not be seen in search results.

[LeeU]

You can do this:

User-agent: *
Disallow: /subDomainName/
Disallow: /testingArea/

That will block the search engines that read the robots.txt file. (While most of them do check the file, a few spiders don't so if it is sensitive material, you should password that directory.) Meanwhile, anybody will be able to directly access the directories listed.

[\\.\]

As with what LeeU said... if it is sensitive then secure it.

You do have lots of hosts that will allow you to password protect a folder but please note, some of the more "Money-Orientated" hosts will charge anything from £15 a year plus a one off fee.

Search hard and long and you will find budget hosts that let you have access to this feature for free.

Your alternate option is to hold these types of file outside the public_html / htdocs folder and use your server-side of choice to access it.

[toplisek]

You can do this:

User-agent: *
Disallow: /subDomainName/
Disallow: /testingArea/



Is correct:

Disallow: /http://www.subdomain.domain.com/
Disallow: /testing/

[LeeU]

No, use what I gave you above.

[toplisek]

No, use what I gave you above.

What is meaning (technically):testingArea and what subDomainName?

[LeeU]

That is what you gave in the example. Just replace it with the actual ones.