Click to See Complete Forum and Search --> : Inktomi bots are shy


iKwak
04-17-2004, 04:48 AM
I can't get inktomi bots to deep spider my sites.
It only comes and hits like 1 page or so.

AdamGundry
04-17-2004, 09:09 AM
Looking at your sites (the ones in your sig, at least), they tend to use a single page (index.php) with GET parameters to determine content. This can cause problems because some search engine bots ignore GET parameters, meaning they only see lots of references to index.php.

This article (http://www.zend.com/zend/spotlight/searchengine.php) explains a possible workaround.

Adam