Robots.txt Is Not Hiding Postings

[design]

NONE

[philaweb]

I created this three weeks ago. However, I still see the postings from webdeveloper.com. Why isn't webdeveloper.com hiding the postings with my robots.txt?

I think you have misunderstood the purpose of the robots.txt file.

The robots.txt file is used to restrict access for robots visiting your website. You cannot restrict access via a robots.txt file to other websites.

[griff777]

My robots.txt on my server also
lists the following user-agents for disallowing:

AbachoBot, http://www.abcdatos.com/botlink/, abcdatos_botlink, AESOP_com_SpiderMan, ah-ha.com crawler, Scooter Mercator, FAST-WebCrawler, ArchitextSpider, UltraSeek, MantraAgent, Hotbot, Lycos, Amotz, WebCrawler, AltaVista, Infoseek, Pathfinder, msnbot, Yahoo-MMCrawler, Teoma, YahooBot, Slurp, Gigabot, Scrubby, Robozilla, BecomeBot, AskJeeves, Tarantula

I did not post them in the attachment for space reasons.

I created this three weeks ago. However, I still see the postings from webdeveloper.com. Why isn't webdeveloper.com hiding the postings with my robots.txt?

Can someone fix it?


Try this site http://www.searchengineworld.com

[LeeU]

You might try the article here (http://www.webreference.com/authoring/robots/) also.

[philaweb]

Or you might want to try robotstxt.org (http://slettes.dk/redirect_to/?url_name=http://www.robotstxt.org/wc/robots.html).

[design]

NONE

[philaweb]

I Already Saw All Of These Sites Before I Started To Post This Inquiry To Webdeveloper.com.
Great, then you would also know the answer.
Can Htaccess Block Webdeveloper.com.
No.