Click to See Complete Forum and Search --> : Include VS querystring, webcrawlers
poolsat
07-28-2005, 11:01 PM
Hi all!
http://www.helpmesearchfor.com
Please check out my webpage and let me know what you think.
This page only uses index.php and that passes arguments on what the content should be, vs using many pages that poll the same include files. Essentially they would look the same. I was wondering if any of you have some info on how this will affect web crawlers. If google for example goes through my page, will it index each different link as index.php?foo=foo1, index.php?foo=foo2, will these show up as separate pages or does it just say there is only idex.php at this webpage?
Thanks a lot!
HostBreak
07-30-2005, 03:54 PM
There is spelling mistake on the main page and let me tell you as far as my knowledge is concerned I think Search Engines don't read pages generated on the fly
buntine
07-30-2005, 08:26 PM
There is spelling mistake on the main page and let me tell you as far as my knowledge is concerned I think Search Engines don't read pages generated on the fly
I would hope they do. The page is requested from the server and passed through whichever engines/interpreters that are appropriate. The crawler should request the page in the same way a Web Browser would.
Regards.
felgall
07-30-2005, 08:52 PM
There are only a very few links on the page for the web spiders to follow. Web spiders can't type in text boxes and so wont find anything from the search box..
buntine
07-30-2005, 09:28 PM
Web spiders can't type in text boxes and so wont find anything from the search box..
I always thought that some spiders can generate content and/or querystrings. Isn't that why some HTML forms generate images containing a string of text and then ask you to write it in a textbox?
Some of them are pretty smart.
Regards.
poolsat
07-31-2005, 09:52 AM
I was thinking the crawler wouldn't know the difference if it was a generated page since it generates plain html code that is read sent out.
I also have a follow up question, I ave been getting some strange traffic from the google bot, it comes to my page about once or twice a day and broses around a few pages and leaves, but when I search google for the title or description of my page it doesnt show up. Please, if you have seen this phenomena let us know how google behaves. Ask jeves bot comes about once every two days to check on the index page and leaves but the ask search actually returns my webpage.