What is the best language for this huge requirement?

FMTHIS

Technically speaking, what is the best way of doing the project below and why?

Summary:

Scrape a lot of data (text, pictures, videos) from multiple websites multiple times in a day (one example: all vehicle data from autotrader.co.uk)
Combine all data and check for any changes in the data being scraped (e.g new pictures)
Make data available on a website on a real time basis (database?)
Data is searchable with multiple filters (think autotrader.co.uk again)

Thank you

VITSUSA

Hi FMTHIS, I can suggest you the best way of doing the below project. You can contact us using my profile name on Google.

FMTHIS

VITSUSA Have sent a message on your website

NogDog

Everything else being equal, I might consider Python to leverage both its multi-threading capabilities (for the scraping) and its comprehensive database tools. But really pretty much any of the popular server-side languages could do it, with the database design/implementation perhaps being more important in terms of performance and scalability.

Truncheon

I also think the best way to approach this is with the use of Python.