Programming code to copy links from other websites (i.e. aggregate news sites)
I was working on a site that collected links from other websites and listed them on my page. For example, I wanted to links/threads from Craigslist and various forums and combine them all on my website. I was wondering if there was a way to automatically update my site with the new posts from the different websites on a daily basis. I could actually do this manually, but it would be a pain in the arse to do everyday. I'm grabbing the links from about 10 different websites. The links would be displayed in a very simple design, just like Craigslist or Google News.
Is this even possible?
I guess this isn't possible???
You'll need permission from those sites to copy them, so the easiest way would be to talk to them about whether they have some sort of data interface (XML, etc) where you could pull the info via your code.
On the other hand, they may be less than pleased if you have been using their copyright material without their permission. That's the trouble with filching content: what happens when you want to come clean???
Possible? Yes. (Search on "screen scraping")
Easy? Can be anything from moderately easy if you know something about server-side programming, to annoyingly difficult if the target web site's structure is malformed and inconsistent.
Ethical/Legal? Lots of possibilities there, from no big deal (maybe they'd be happy to sign an agreement with you to get the free advertising) to you better hope they don't notice and call their lawyers.
"Please give us a simple answer, so that we don't have to think, because if we think, we might find answers that don't fit the way we want the world to be."
~ Terry Pratchett in Nation
How to Ask Questions the Smart Way
(not affiliated with this site, but well worth reading)
If the sites have an RSS feed you can use a module like feed aggregator, that is if you're using a CMS like Drupal; then import the titles and links into your website.
Wordpress also has a theme for what you want called WP-Drudge
Users Browsing this Thread
There are currently 1 users browsing this thread. (0 members and 1 guests)