dcsimg
www.webdeveloper.com
Results 1 to 9 of 9

Thread: Google webmaster and robots.txt indexing

Hybrid View

  1. #1
    Join Date
    Oct 2012
    Posts
    55

    Google webmaster and robots.txt indexing

    Hi

    I have a site of about 25 pages.
    9 of those pages are metric charts, full of numbers and tables.

    Of course I do not really want Google to include those in their index , so I added a meta tag as follows, to the nine files.

    HTML Code:
    <meta name="robots" content="noindex" />
    Sure I probably should also add "nofollow" and will do that today.

    My question is when I look at the Google webmaster site it says ''23 URLS Submitted - 9 Indexed''.

    Has Google indexed them simply as "noindex"?

    And why are my other files not indexed, I have not added any robot meta tags to the other files as the default is just meant to be follow,index.

    So I am not using any Robot.txt just a meta for the pages I don't want indexed and leaving the others blank for the default (follow index).

    Is this suitable?

    Dan

  2. #2
    Join Date
    Nov 2012
    Posts
    15
    Google will rarely index your whole site in one shot. It usually does that in time, so no worries, it'll come around again and eventually index your whole site.

    Do you have a site map as well?

  3. #3
    Join Date
    Oct 2012
    Posts
    55
    Quote Originally Posted by NorLine View Post
    Google will rarely index your whole site in one shot. It usually does that in time, so no worries, it'll come around again and eventually index your whole site.

    Do you have a site map as well?
    Thanks

    Yes I have a site map, according to the Google information, It has not even been crawled yet.

    Dan

  4. #4
    Join Date
    Nov 2012
    Posts
    15
    Quote Originally Posted by drunken_dev View Post
    Thanks

    Yes I have a site map, according to the Google information, It has not even been crawled yet.

    Dan
    You might also want to subscribe to Bing Yahoo webmaster services...it's a bit different, and they are also slow, but it provides another point of view for webstats

    Hope it helps!

  5. #5
    Join Date
    Oct 2012
    Posts
    55
    Quote Originally Posted by NorLine View Post
    You might also want to subscribe to Bing Yahoo webmaster services...it's a bit different, and they are also slow, but it provides another point of view for webstats

    Hope it helps!
    Yes Thanks I will.

    I have been reading a lot of Google documents and watching their webmaster videos.

    Overall they do hold a large share of the market however you would be naive to think they will be the "be all and end all of the web" in the future.

    They have a very market based philosophy, and while productive now, like any marketing business they also have fierce competitors.

    Dan

  6. #6
    Join Date
    Aug 2012
    Posts
    26
    I agree with NorLine, It will take time for Google management to set an index for your website. I will guide you to make a sitemap and simply submit it to Google and it will be easy for Google management to crawl your website from a single set page.

  7. #7
    Join Date
    Oct 2012
    Posts
    55
    Quote Originally Posted by abiabiah View Post
    I agree with NorLine, It will take time for Google management to set an index for your website. I will guide you to make a sitemap and simply submit it to Google and it will be easy for Google management to crawl your website from a single set page.
    Thanks, I have a site map.

    It is not a huge site so not so bad.

    Dan

  8. #8
    Join Date
    Nov 2012
    Posts
    15
    Totally agree...BIng/Yahoo have a long way to go before they catch up with Google...but it's good for the market that there are other players in the market as well.

    Take care!

  9. #9
    Join Date
    Nov 2012
    Location
    Orange County, California
    Posts
    26
    Create your sitemap properly, make it as sitemap.xml and ror.xml and .html also, and then submit your website in webmaster tools. Google have to crawl your website. You should try some bookmarks it can also help to crawl and indexing to your website.

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
HTML5 Development Center



Recent Articles