Getting the right Keywords

[golb0016]

I use google webmasters to track the keywords for my website but it is not turning out so hot. It seems to have pulled more keywords from the terms of use page than most other pages. I recently created a robot.txt file to disallow this page in attempt to hide it from crawlers (along with some other changes to SEO for the website). However, a week has passed and no change.

Was this the correct action? Do I just have to wait longer? How often do search engines crawl websites?

Thanks for any help.

[developerguru]

Well there is no specific time for search engines to crawl. It totally depends on them. They can crawl on the same day or they can take 3 months for the same thing.
If you are doing some thing related to seo then you need to have patience.
You can not take your next step until search crawl your site.

[golb0016]

Do you think it was the correct action to use a robot file to disallow pages from search engines? Will they still follow all internal links to all pages?

[developerguru]

well robot.txt is used for that purpose only but the thing is that if you have added urls incorrectly then it will cause problems
Means if you do not want a single page to be visited by crawler. then you just need to write that page url but if you write the url and * at the end then google will not crawl all the other pages.

so before giving any suggestion I want to know what you have written in robot. txt

[golb0016]

Here is a sample of my robot.txt file:

User-Agent: *
Disallow: /abuse.php
Allow: /about.php
Allow: /ac.php
Allow: /accprint.php
Allow: /acwprint.php
Disallow: /addappliances.php
Disallow: /addcomment.php
Disallow: /addquestion
Disallow: /adduser.php
Disallow: /admin.php
Disallow: /adminhome.php
Disallow: /adminstat.php
Allow: /appliances.php
Allow: /contact.php
Disallow: /contactadd.php
Allow: /contactus.php
Disallow: /contactusadd.php
Disallow: /detect.php
Allow: /dw.php
Allow: /dwprint.php
Disallow: /emailform.php
Disallow: /femail.php
Disallow: /forgot.php
Allow: /fp.php
Allow: /fpass.php
Allow: /fpprint.php
Allow: /fprepprint.php
Allow: /frig.php
Allow: /frigprint.php
Disallow: /fupdate.php
Allow: /furnace.php
Allow: /fvprint.php
Allow: /gd.php
Allow: /gdo.php
Allow: /gdoprint.php
Allow: /gdprint.php
Allow: /general.php
Allow: /generalprint.php
Allow: /help.php
Allow: /home.php
Allow: /hum.php
Allow: /humcprint.php
Allow: /humwprint.php
Allow: /ietip.php
Allow: /index.php

Disallow: /Links/

All the above examples are individual pages except the last one, which is a folder. I don't any pages crawled within the Links folder.

Thanks for the help!

[developerguru]

It seems ok.

Have you created xml sitemap for your site or not
If not then please do that.
And also create your "google webmaster tool account" for your site.

This account will provide you details related to the problems related to your site.
From that information you can get idea why you google is not indexing your site.

[diviny]

The first thing I want you to do is to go to the following website: Keyword Questions

This is a website where you can type in a broad search term to find out some of the “Long Tailed Keyword Questions” that your prospects are searching for.

[diviny]

your keyword should be relevant to the website..Google will crawl more often the website with high cache velocity....So try to do optimization on highly caching web site....

[stephen1983]

SEO really needs patience and an understanding on what you are doing. If you’re optimising a certain keyword on a certain page, make sure the keyword density is enough and that there is no duplicate content within your site.

[georgefuller]

SEO really needs patience and an understanding on what you are doing. If you’re optimising a certain keyword on a certain page, make sure the keyword density is enough and that there is no duplicate content within your site.

these are all true... seo is a very long process you wont get your goal in just one snap.

[airun0804]

I use google webmaster tools, Submit a new xml file, and soon google removed the ban pages.