www.webdeveloper.com
Results 1 to 15 of 17

Thread: How to start a Classified Ads Aggregator?

Hybrid View

  1. #1
    Join Date
    Apr 2009
    Posts
    4

    How to start a Classified Ads Aggregator?

    I would like to create a website that is essentially a classified ads aggregator. It would scrape and aggregate the ads from local websites posting classified ads in several categories such as buy/sell, help wanted, etc. It would then display them in similar categories. The user would be able to sort them by price, location, etc. The site would also accept the ads from users.

    The website will target the local market in one of the European countries. There are several dozens of local websites posting classified ads, either stand alone, or online versions of newspapers. I want my site to pull these ads and display in organized fashion for the user.

    There are sites using this model, such as oodle.com, or kijiji.com. However these are huge sites in scope. On the other hand, my project is targeting only 30-40 websites for now.

    Where can I find somebody who might be able to do this (I am not coder)?
    I see obstacles at the outset:

    1. overcoming the occasional changes in the targeted websites.
    2. the site will be non-English
    3. while basic ads categories are very similar, they are NOT spelled out exactly alike from website to website?
    4. I am on my own, no big $$ sponsors
    5. above 4 are just the ones that are coming to my mind (not a developer mind)

    Best regards,

    Andre

  2. #2
    Join Date
    Feb 2009
    Posts
    38
    Quote Originally Posted by aswidz View Post
    I would like to create a website that is essentially a classified ads aggregator. It would scrape and aggregate the ads from local websites posting classified ads in several categories such as buy/sell, help wanted, etc. It would then display them in similar categories. The user would be able to sort them by price, location, etc. The site would also accept the ads from users.

    The website will target the local market in one of the European countries. There are several dozens of local websites posting classified ads, either stand alone, or online versions of newspapers. I want my site to pull these ads and display in organized fashion for the user.

    There are sites using this model, such as oodle.com, or kijiji.com. However these are huge sites in scope. On the other hand, my project is targeting only 30-40 websites for now.

    Where can I find somebody who might be able to do this (I am not coder)?
    I see obstacles at the outset:

    1. overcoming the occasional changes in the targeted websites.
    2. the site will be non-English
    3. while basic ads categories are very similar, they are NOT spelled out exactly alike from website to website?
    4. I am on my own, no big $$ sponsors
    5. above 4 are just the ones that are coming to my mind (not a developer mind)

    Best regards,

    Andre
    To really cllassify you need the add Classifier, meaning the tool which will automatically categorize the website by its content. This task is not easy, but has a solution. Let me know if you are interested.
    WebsiteContentAnalysis.com - Content Analysis, Keywords, Related/Competitor Websites ...
    http://LogYourLink.com - Automated submission web link directory

  3. #3
    Join Date
    Apr 2009
    Posts
    4
    I agree that, like any search engine, I will need to some sort of way to categorize the website. But even before I consider this I need to answer several issues that I think I will face, such as:

    1. Is it possible at all to create such a search site that actively monitors individual websites for new ads, and displays them in selected categories? I identified 30-40 local websites that I would like my serch engine to crawl.

    2. How to address the changing nature of the source website? The webmasters occasionally change the sites, and what are the implication for the serching mechanism?

    3. I can imagine that the fact that I want to create a non-English website at some point sooner or later has to create issues - can only the native speaker address these?

    These are just two consideration I am looking for the answers to.

    Best regards,
    Andre

  4. #4
    Join Date
    Feb 2009
    Posts
    38
    Quote Originally Posted by aswidz View Post
    I agree that, like any search engine, I will need to some sort of way to categorize the website. But even before I consider this I need to answer several issues that I think I will face, such as:

    1. Is it possible at all to create such a search site that actively monitors individual websites for new ads, and displays them in selected categories? I identified 30-40 local websites that I would like my serch engine to crawl.

    2. How to address the changing nature of the source website? The webmasters occasionally change the sites, and what are the implication for the serching mechanism?

    3. I can imagine that the fact that I want to create a non-English website at some point sooner or later has to create issues - can only the native speaker address these?

    These are just two consideration I am looking for the answers to.

    Best regards,
    Andre
    1. Of course, but why do you need it?

    2. All sites are assumed to be changed. The site analyzer picks up the current state and if it is a background process then it stores and use it until the next crawling cycle. If its a real time operation then you get always updated results.

    3. I am not sure about, say oriental langauges, but if it's at least a European one there should not be a problem. For example although web content classifier classengine.com is trained now for English, it is not a problem to do the same for any European langauge as long as this langauge is supported by media, e.g. German Google search.
    WebsiteContentAnalysis.com - Content Analysis, Keywords, Related/Competitor Websites ...
    http://LogYourLink.com - Automated submission web link directory

  5. #5
    Join Date
    Apr 2009
    Posts
    4
    1. I came up with the idea of classifieds ads being pulled in one place because there is no such site addressing Polish language ads sites. There are mainly local 'stand alone' websites, or classifieds sections of online local newspapers.

    2. I am looking for somebody who might know how to do such site. I am looking for help on rentacoder.com so far. I know of getacoder.com, guru.com, and couple of other sites. I am looking for some feedback on these and similar sites, as well as, if this is reasonable way to look for somebody to create a website.

  6. #6
    Join Date
    Feb 2009
    Posts
    38
    Quote Originally Posted by aswidz View Post
    2. I am looking for somebody who might know how to do such site. I am looking for help on rentacoder.com so far. I know of getacoder.com, guru.com, and couple of other sites. I am looking for some feedback on these and similar sites, as well as, if this is reasonable way to look for somebody to create a website.
    I am afraid you are mistaken saying "to create a website". It's not just creating a website The problem is in implementation of what you want. Algorithms for what you want are not trivial and programming on the server side is required. So I do not think these "guru" - like services will help. They are ususlly good at web design but very far from algorithms.

    Is it your own project or something which is funded?
    WebsiteContentAnalysis.com - Content Analysis, Keywords, Related/Competitor Websites ...
    http://LogYourLink.com - Automated submission web link directory

  7. #7
    Join Date
    May 2004
    Location
    Manhattan NY
    Posts
    6,028
    Yes, you should be able to find what you're looking for there. Solicitations are prohibited here but you're already on the right track.

  8. #8
    Join Date
    Sep 2010
    Posts
    1
    Hi daydreamer
    I would like to talk with you about aggregation of the web. We are currently aggregating and archiving every car posting in the US (80 classifieds and 28k dealer websites soon to be 68k dealer websites) with our technology and three people. Here is a video on how we do it.
    http://youtu.be/Ps-qDB67PKA
    Here is my linkedin profile
    http://www.linkedin.com/pub/troy-campbell/31/a88/927
    Thanks
    Troy

  9. #9
    Join Date
    Aug 2013
    Location
    Phoenix, AZ
    Posts
    104
    There are plenty of Classified ads themes for wordpress out there. That would be the easiest more cost effective way to develop a classifieds ads website in my opinion. Then after you have explored the theme options I would move onto custom PHP coding.

  10. #10
    Join Date
    Oct 2013
    Posts
    45
    Choose the method for creating your aggregator feed. Search the Web and find the sites you want to use for aggregate fodder.Enter the syndication site's address into your aggregator module or program,

  11. #11
    Join Date
    Jan 2014
    Posts
    120
    Thanks for sharing. nice information.

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
HTML5 Development Center



Recent Articles