www.webdeveloper.com
Page 1 of 2 12 LastLast
Results 1 to 15 of 17

Thread: How to start a Classified Ads Aggregator?

  1. #1
    Join Date
    Apr 2009
    Posts
    4

    How to start a Classified Ads Aggregator?

    I would like to create a website that is essentially a classified ads aggregator. It would scrape and aggregate the ads from local websites posting classified ads in several categories such as buy/sell, help wanted, etc. It would then display them in similar categories. The user would be able to sort them by price, location, etc. The site would also accept the ads from users.

    The website will target the local market in one of the European countries. There are several dozens of local websites posting classified ads, either stand alone, or online versions of newspapers. I want my site to pull these ads and display in organized fashion for the user.

    There are sites using this model, such as oodle.com, or kijiji.com. However these are huge sites in scope. On the other hand, my project is targeting only 30-40 websites for now.

    Where can I find somebody who might be able to do this (I am not coder)?
    I see obstacles at the outset:

    1. overcoming the occasional changes in the targeted websites.
    2. the site will be non-English
    3. while basic ads categories are very similar, they are NOT spelled out exactly alike from website to website?
    4. I am on my own, no big $$ sponsors
    5. above 4 are just the ones that are coming to my mind (not a developer mind)

    Best regards,

    Andre

  2. #2
    Join Date
    Feb 2009
    Posts
    38
    Quote Originally Posted by aswidz View Post
    I would like to create a website that is essentially a classified ads aggregator. It would scrape and aggregate the ads from local websites posting classified ads in several categories such as buy/sell, help wanted, etc. It would then display them in similar categories. The user would be able to sort them by price, location, etc. The site would also accept the ads from users.

    The website will target the local market in one of the European countries. There are several dozens of local websites posting classified ads, either stand alone, or online versions of newspapers. I want my site to pull these ads and display in organized fashion for the user.

    There are sites using this model, such as oodle.com, or kijiji.com. However these are huge sites in scope. On the other hand, my project is targeting only 30-40 websites for now.

    Where can I find somebody who might be able to do this (I am not coder)?
    I see obstacles at the outset:

    1. overcoming the occasional changes in the targeted websites.
    2. the site will be non-English
    3. while basic ads categories are very similar, they are NOT spelled out exactly alike from website to website?
    4. I am on my own, no big $$ sponsors
    5. above 4 are just the ones that are coming to my mind (not a developer mind)

    Best regards,

    Andre
    To really cllassify you need the add Classifier, meaning the tool which will automatically categorize the website by its content. This task is not easy, but has a solution. Let me know if you are interested.
    WebsiteContentAnalysis.com - Content Analysis, Keywords, Related/Competitor Websites ...
    http://LogYourLink.com - Automated submission web link directory

  3. #3
    Join Date
    Apr 2009
    Posts
    4
    I agree that, like any search engine, I will need to some sort of way to categorize the website. But even before I consider this I need to answer several issues that I think I will face, such as:

    1. Is it possible at all to create such a search site that actively monitors individual websites for new ads, and displays them in selected categories? I identified 30-40 local websites that I would like my serch engine to crawl.

    2. How to address the changing nature of the source website? The webmasters occasionally change the sites, and what are the implication for the serching mechanism?

    3. I can imagine that the fact that I want to create a non-English website at some point sooner or later has to create issues - can only the native speaker address these?

    These are just two consideration I am looking for the answers to.

    Best regards,
    Andre

  4. #4
    Join Date
    Feb 2009
    Posts
    38
    Quote Originally Posted by aswidz View Post
    I agree that, like any search engine, I will need to some sort of way to categorize the website. But even before I consider this I need to answer several issues that I think I will face, such as:

    1. Is it possible at all to create such a search site that actively monitors individual websites for new ads, and displays them in selected categories? I identified 30-40 local websites that I would like my serch engine to crawl.

    2. How to address the changing nature of the source website? The webmasters occasionally change the sites, and what are the implication for the serching mechanism?

    3. I can imagine that the fact that I want to create a non-English website at some point sooner or later has to create issues - can only the native speaker address these?

    These are just two consideration I am looking for the answers to.

    Best regards,
    Andre
    1. Of course, but why do you need it?

    2. All sites are assumed to be changed. The site analyzer picks up the current state and if it is a background process then it stores and use it until the next crawling cycle. If its a real time operation then you get always updated results.

    3. I am not sure about, say oriental langauges, but if it's at least a European one there should not be a problem. For example although web content classifier classengine.com is trained now for English, it is not a problem to do the same for any European langauge as long as this langauge is supported by media, e.g. German Google search.
    WebsiteContentAnalysis.com - Content Analysis, Keywords, Related/Competitor Websites ...
    http://LogYourLink.com - Automated submission web link directory

  5. #5
    Join Date
    Apr 2009
    Posts
    4
    1. I came up with the idea of classifieds ads being pulled in one place because there is no such site addressing Polish language ads sites. There are mainly local 'stand alone' websites, or classifieds sections of online local newspapers.

    2. I am looking for somebody who might know how to do such site. I am looking for help on rentacoder.com so far. I know of getacoder.com, guru.com, and couple of other sites. I am looking for some feedback on these and similar sites, as well as, if this is reasonable way to look for somebody to create a website.

  6. #6
    Join Date
    May 2004
    Location
    Manhattan NY
    Posts
    6,028
    Yes, you should be able to find what you're looking for there. Solicitations are prohibited here but you're already on the right track.

  7. #7
    Join Date
    Feb 2009
    Posts
    38
    Quote Originally Posted by aswidz View Post
    2. I am looking for somebody who might know how to do such site. I am looking for help on rentacoder.com so far. I know of getacoder.com, guru.com, and couple of other sites. I am looking for some feedback on these and similar sites, as well as, if this is reasonable way to look for somebody to create a website.
    I am afraid you are mistaken saying "to create a website". It's not just creating a website The problem is in implementation of what you want. Algorithms for what you want are not trivial and programming on the server side is required. So I do not think these "guru" - like services will help. They are ususlly good at web design but very far from algorithms.

    Is it your own project or something which is funded?
    WebsiteContentAnalysis.com - Content Analysis, Keywords, Related/Competitor Websites ...
    http://LogYourLink.com - Automated submission web link directory

  8. #8
    Join Date
    Apr 2009
    Posts
    4
    Appreciate your insight. Which calls for the question how, or where do I start learning the programing. I basically want to at least start scratching the surface of programing. I agree with you on guru-like websites. I think long term I will be far better off if I at least try to explore this on my own. The prospect of working and learning on my own is more appealing to me.

    And, by the way, it is my own little project.

    Having said this, would you be able to suggest for a novice where to start learning programming? I know there is tons of info out there, but perhaps you could recommend a way to start. There has to be a beginning to this somewhere..

    Best regards,
    Andre

  9. #9
    Join Date
    Feb 2009
    Posts
    38
    Quote Originally Posted by aswidz View Post
    Having said this, would you be able to suggest for a novice where to start learning programming? I know there is tons of info out there, but perhaps you could recommend a way to start. There has to be a beginning to this somewhere..
    You need to choose first the programming language.
    Taking into account that in future you are going to program servers for web applications and prefer to use freeware and yet have an abilty to create programs on any level of complexity I'd suggest that you start with Perl.

    For the begining try to create a program which does something simple. For example "for any arbitrary set of words in the input stream print them in the alphabetical order".
    WebsiteContentAnalysis.com - Content Analysis, Keywords, Related/Competitor Websites ...
    http://LogYourLink.com - Automated submission web link directory

  10. #10
    Join Date
    Jan 2010
    Posts
    1

    Classifieds agregator

    hi, i have a classifieds site called www.telists.com, it's users who input ads manually, I was thinking of someway to get more ads from some well known classifieds site and inject them into my database, i could't find any tool or script that does that, what i think is that it involves complex programming to get this done, any idea??
    www.telists.com

  11. #11
    Join Date
    Jan 2012
    Posts
    1

    classified aggregator

    hi vit,

    i saw your discussion going on classified aggregator and would like to ask if you where able to develop it.

    i am also looking for a local classified aggregator for my country and hope we can make this happen.

    please let me know ur mail id so we can discuss further.

  12. #12
    Join Date
    Oct 2013
    Posts
    2

    I have a ready-made solution

    Hey Guys,

    I can see that post is really old. However, if it concerns anyone, I have a solution available which aggregates all classified ads data in an organized manner. Currently I have added around 50 websites across various languages and region. The system is very much customization, that means, if you need to add new website, it's quite easy.

    Reply to this thread if you need this.

  13. #13
    Join Date
    Sep 2010
    Posts
    1
    Hi daydreamer
    I would like to talk with you about aggregation of the web. We are currently aggregating and archiving every car posting in the US (80 classifieds and 28k dealer websites soon to be 68k dealer websites) with our technology and three people. Here is a video on how we do it.
    http://youtu.be/Ps-qDB67PKA
    Here is my linkedin profile
    http://www.linkedin.com/pub/troy-campbell/31/a88/927
    Thanks
    Troy

  14. #14
    Join Date
    Oct 2013
    Posts
    2

    Lightbulb Full-Automated Classified Website Aggregator scraping 12,000 URLs

    Hello People,

    Take a look at findmestuffs.com It is a classified website aggregator, aggregating listings from around 12,000 URLs ( All Indian classified websites ). The listings are organized in 18 categories as listed on the website. The website automatically updates for new listings on daily basis. The entire system is fully automated and does not require any maintenance and update. The current system is capable of scraping around 100,000 URLs on daily basis. The cost of running the entire system ( website + scraping system ) is 30$ per month.

    I am looking to sell the website. If you are interested, please get in touch with me at a[at]amitkumargupta.com I will be happy to provide post-sale support as well.

    Thanks,
    Amit

  15. #15
    Join Date
    Aug 2013
    Location
    Phoenix, AZ
    Posts
    76
    There are plenty of Classified ads themes for wordpress out there. That would be the easiest more cost effective way to develop a classifieds ads website in my opinion. Then after you have explored the theme options I would move onto custom PHP coding.

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
HTML5 Development Center



Recent Articles