www.webdeveloper.com
Results 1 to 3 of 3

Thread: Retriving information from another website

  1. #1
    Join Date
    Aug 2009
    Posts
    79

    Exclamation Retriving information from another website

    What i want to do is create a database of lyrics for songs however i dont want to sit there and go and copy and paste for hours from another website and i know in php you can "crawl" a website and store it and i was wondering on how you do it??

  2. #2
    Join Date
    Nov 2010
    Location
    Croatia
    Posts
    31

  3. #3
    Join Date
    Jul 2010
    Location
    /ramdisk/
    Posts
    865
    Using only php you'd want to look into ob_fetch();

    There's a lot of ways to go about copying a database that doesn't belong to you. You could "scrape" the data from their webserver as a last resort. There are other ways to get more direct results from their dbms depending on how they put their application together.

    I haven't done it in a while, but urllib and urllib2 work for python. People also seem to like "beautiful soup" (a scraping API) for python.

    A few tips:
    Check for robots.txt
    Try asking first, it's too easy to not try.
    Don't be surprised if you get blocked from their webserver for "abuse".
    You need previous experience with html or you'll be lost when it comes time to parse the data.

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
HTML5 Development Center



Recent Articles