508
06-20-2006, 05:53 AM
WHEN I DO A SEARCH OF MY WEBSITE, I FIND IT POSTED ON OVER 1,000 ADDITIONAL SITES. THERE ARE LESS THAN 100 INTERNAL LINKS. HOW CAN I USE CGI, PERL, AND/OR OTHER PROGRAMMING LANGUAGES TO EITHER REMOVE THE SITES FROM SEARCHES, OR MAKE THOSE WEBSITES NON-INDEXABLE.
CURRENTLY, I HAVE A ROBOTS.TXT WHICH IS SET UP TO NOINDEX, NOARCHIVE. THE ONLY ALLOWED FILE IN ROBOTS.TXT IS THE HOMEPAGE.
I ALSO HAVE AN .HTACCESS FILE WHICH CONTAINS
RewriteEngine on
# Options +FollowSymlinks
RewriteCond %{HTTP_REFERER}
FOLLOWED BY THE SITES I WANT TO REJECT.
HOWEVER, IF I END WITH
RewriteRule .* - [F], ANY ATTEMPTS FROM ME TO VISIT MY PAGE IS AN ERROR 401 FOR THE INTERNET PAGES.
ANY ATTEMPT TO USE RewriteRule \.(gif|jpg|js|css)$ - [F]
WILL RESULT IN ME NOT BEING ABLE TO SEE MY OWN IMAGES.
I WANT TO BE ABLE TO SEE THE SITE, BUT I WANT OUTSIDE LINKS TO EITHER BE REMOVED OR NON-INDEXED.
INDEX OF MY OWN PAGES=PASS
INDEX OF THE 1,000 PAGES=FAIL
HOW CAN THIS BE DONE WITH CGI, PERL, HTACCESS, OR ANY OTHER PROGRAMMING LANGUAGE.
PLEASE SUBMIT A SAMPLE AND TELL ME WHERE TO REPLACE THE OUTSIDE URL AND WHERE TO SUBMIT MY OWN URL.
SINCERELY,
508
CURRENTLY, I HAVE A ROBOTS.TXT WHICH IS SET UP TO NOINDEX, NOARCHIVE. THE ONLY ALLOWED FILE IN ROBOTS.TXT IS THE HOMEPAGE.
I ALSO HAVE AN .HTACCESS FILE WHICH CONTAINS
RewriteEngine on
# Options +FollowSymlinks
RewriteCond %{HTTP_REFERER}
FOLLOWED BY THE SITES I WANT TO REJECT.
HOWEVER, IF I END WITH
RewriteRule .* - [F], ANY ATTEMPTS FROM ME TO VISIT MY PAGE IS AN ERROR 401 FOR THE INTERNET PAGES.
ANY ATTEMPT TO USE RewriteRule \.(gif|jpg|js|css)$ - [F]
WILL RESULT IN ME NOT BEING ABLE TO SEE MY OWN IMAGES.
I WANT TO BE ABLE TO SEE THE SITE, BUT I WANT OUTSIDE LINKS TO EITHER BE REMOVED OR NON-INDEXED.
INDEX OF MY OWN PAGES=PASS
INDEX OF THE 1,000 PAGES=FAIL
HOW CAN THIS BE DONE WITH CGI, PERL, HTACCESS, OR ANY OTHER PROGRAMMING LANGUAGE.
PLEASE SUBMIT A SAMPLE AND TELL ME WHERE TO REPLACE THE OUTSIDE URL AND WHERE TO SUBMIT MY OWN URL.
SINCERELY,
508