perl :: dbi - starting

[perl_girlie]

hi all

first - i am new to perl and i am new to this board.

well i need to collect some of the data out of a site - here is an example.
Note this is only an example.
Why do i need to collect the data, you may ask: i am an researcher and i want to do some socio-ethnographic research. Therefore i need the data.

Well - i am about to run a useragent against a site like the above mentioned. (note - this sites are only examples!! i am not interested in those site)

Well i have the data from the sites - i pulled them almost on my machine.
See the history of the idea http://forums.devshed.com/perl-programming-6/minor-change-in-lwp-need-ideas-how-to-accomplish-388061.html
(]http://www.bamaclubgp.org/forum/sitemap.php[/URL)

How to get ahead - how to get into a database: Any ideas were greatly appreciated. How to use Perl's Database Interface Module in order to bring the data into a tabular structure. DBI is the right way to go to insert the data into your database. The hard part is usually defining your database schema to begin with, but that's a different topic. I am sure that once I get over these initial issues, then it will be plain sailing.

So how to start with: well i have the data from the sites - i pulled them almost on my machine. how to get ahead - how to get into a database:

since the relational database is a bunch of rectangular tables. with each row of a table is a record about one person or thing; the record contains several pieces of information called fields.
Here is an example table:

LASTNAME FIRSTNAME ID POSTAL_CODE AGE SEX
Gauss Karl 119 19107 30 M
Smith Mark 3 T2V 3V4 53 M
Noether Emmy 118 19107 31 F
Smith Jeff 28 K2G 5J9 19 M
Hamilton William 247 10139 2 M


The names of the fields are LASTNAME, FIRSTNAME, ID, POSTAL_CODE, AGE, and SEX. Each line in the table is a record, or sometimes a row or tuple.
For example, the first row of the table represents a 30-year-old male whose name is Karl Gauss, who lives at postal code 19107, and whose ID number is 119.
Sometimes this is a very silly way to store information. When the information naturally has a tabular structure it's fine. When it doesn't, you have to squeeze it into a table, and some of the techniques for doing that are more successful than others.

How to start to use Perl::DBI ? How to run and to set up Perl::DBI
I think that i have to build up the table - structure...

to begin with: well i have the data in a bunch of plain text files on the local disk. well i need to collect some of the data out of a site - here is an example. [URL=]http://www.bamaclubgp.org/forum/sitemap.php

i look forward to any mail.

regards perl_girlie

[Sixtease]

Hello Perl Girlie and welcome!

My experience with databases is very limited, so my guidance may not be the best you can get. If what I write here doesn't help you, I suggest you ask on perlmonks.org - they are ready for questions like yours.

DBI is database-independent interface, which means that in order to use it, you must also have an actual data base system installed. You can opt for MySQL, PostgreSQL, SQLite or plain text files, to list some of the popular options. If I got it right and you don't need any security measures for people accessing your database and you'll not have millions of records, I'd suggest SQLite - easy to set up and fast. Google up about it and install it. If you want the fastest way, that doesn't involve any installation, you can also use CSV - plaintext files instead of a real database management system.

Once you have decided what DBMS you wish to use and have it installed, you'll install the needed Perl modules - DBI itself and a driver for your database. Let's assume you opted for SQLite.
perl -MCPAN -e 'install Bundle::DBI'
perl -MCPAN -e 'install DBD::SQLite'
Now test if the modules installed correctly. If the following command gives no output, all is right
perl -MDBI -MDBD::SQLite -e 1
Basic usage of the database is then like this (from Perl):
use DBI;
my $dbh = DBI->connect("dbi:SQLite:dbname=dbfile","",""); # varies with DB used
$dbh->do('SQL COMMAND');
Help on using DBI (http://search.cpan.org/~timb/DBI-1.604/DBI.pm) can be invoked with the perldoc DBI command. Generally, perldoc Package::Name shows documentation of a package. Google seems to give reasonable results for "Perl DBI tutorial", so I trust you'll have little difficulty getting this to work.