Click to See Complete Forum and Search --> : Site not getting indexed, only home page


mdoigny
08-16-2005, 04:01 PM
Hello!

After reading my server logs for some days now, it seems that my site is not getting indexed like it should.

This is what i found in the logs:

The home page is accessed by google more than once a day, but only the main index "GET /"
girafebot (never heard of it) downloads the images in the home page, but nothing else
i provide a sitemap (xml) for google, that is downloaded more than once a day by googlebot
pages with adsense are downloaded by mediapartners-google whenever a visitor downloads the corresponding page (to provide targeted ads)

I provide a robots.txt file containingUser-agent: *
Disalow:
and a meta tag <META NAME="ROBOTS" CONTENT="INDEX,FOLLOW">

I try to provide anything that could make a spider happy; the right meta tags, plain href tags, enough content, ...

The pages are on a home server using non-standard ports. The ip itself is quite stable (the same for months).
Do i need to provide a base href containing the port number???

The home page (the only one that get's indexed) is located at
http://afr_upd.verfaillie.be:20800
What could be wrong with it (spider-wise?)

thanks

Bluetagpizza
08-16-2005, 04:25 PM
Google blocks all pages that contain "&ID=" in their address. I have no idea why Google does this but it is undoubtedly true as it comes directly from Google's site (http://www.google.com/webmasters/guidelines.html). You're link is not working at this time, so I cannot determine if this is the cause.

mdoigny
08-16-2005, 05:07 PM
To keep you informed:

The site got indexed by Google yesterday, some 95 hits in half an hour.

Google did use a very old site index, probably one month old (containing non-exitent pages), instead of using the links provided by the new pages. New pages were not indexed. I provieded an sitemap.xml, but they didn't use it.

It is as if the pages are just downloaded, but the links in them were not used.

Ruick
08-23-2005, 01:39 AM
you always have to give google, and i cant explain why your main pages didnt get indexed.