www.webdeveloper.com
Results 1 to 3 of 3

Thread: Sending search engines to a different page?

  1. #1
    Join Date
    Apr 2010
    Posts
    7

    Sending search engines to a different page?

    Firstly, before you reply telling me not to do this (and I fully understand why) please read my situation.

    The website I've created for myself loads the file "index.php" and doesn't go anywhere else. All content that the site uses is loaded through ajax - and to allow bookmarking I've created a system which writes the page name after the "#" sign in the url.
    Example: "http://localhost/#home"

    This tag, as I'm going to call it, is changed by the javascript when the new page is loaded through ajax, and because it's after a hash the page does not get reloaded (like I wanted).

    The problem is though, my site is impossible for search engines to crawl.

    The search engine will find "index.php", and that will have no content as JS hasn't loaded it yet.

    But my website works like this:
    http://localhost/#home - This loads - http://localhost/pages/home.php
    http://localhost/#blog - This loads - http://localhost/pages/blog.php

    Is there a htaccess rule that I can use which will allow me to point the search engines to the PHP scripts?
    (I can use the PHP user agent script to add the text hyperlinks where needed)

    Thanks for any help!

  2. #2
    Join Date
    Mar 2006
    Location
    Northern UK :((
    Posts
    667
    Thought about having a robot text file for engines to crawl? That way, they only visit the text file ...

    Googled a tutorial for you ....

    http://tools.seobook.com/robots-txt/

  3. #3
    Join Date
    Apr 2010
    Posts
    7
    ****! Sorry about forgetting to post my solution here - it actually stems from the idea you gave me.

    Essentially what I did was create a sitemap and robots.txt file and uploaded them to the website. These two files both point to the actual PHP pages.

    At the top of each of the PHP pages is a small script. This script detects whether the user is a standard user or a search engine. If it's a search engine then it adds a navigation element to the top of the page - to link all the pages together.

    So now when a search engine bot crawls my website it reads the PHP script as if it's the actual website. (well it is the actual website, it's just not viewed from there).

    So, for example, the contact part of my website is actually:
    "mywebsite/#contact"
    But in the google listing it's:
    "mywebsite/pages/contact.php"

    Now this same script which detects for search engines actually works in reverse too. If you're visiting the page without a certain "GET" variable set then the PHP script modifies the header and sends you to the nicely, and un-search engine friendly, site.
    So it kind of works:
    Google -> pages/contact.php -> user agent is real user -> get variable not set -> redirect to propper site

    propersite -> ajax load: pages/contact.php -> GET variable set -> give ajax the response it wants

    This approach also lets me easily display a mobile site under the exact same url.

    It's also obviously working to - the site's not been published long and I'm already on page 2 of google for searching just my name!

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
HTML5 Development Center



Recent Articles