Click to See Complete Forum and Search --> : please Ineed help


rubaf
04-13-2007, 03:20 PM
Hi

I need your help to build application that will find the changes that take place on the main university timetable and use this information to update the affected groups by email : The access will not directly to the database so that will be a need to parse the html formate that is available using php : it would be hoped to write this into XML format for storage:
this is my project , but I donīt know from where and how I should start
Please help me, please .

Charles
04-13-2007, 06:16 PM
Well, you haven't given us a whole lot of information.

I'm guessing that this time table is in HTML and accessible on the internet or something. Myself, I'd write something in Perl using the LWP::Simple and HTML::TreeBuilder modules. It shouldn't take more than two dozen lines of code.

rubaf
04-14-2007, 04:39 AM
Hi Charles
Yes the time table is in HTML , My application will take any timetable , stored that timetable in xml storage and after 15minutes it will store the timetable again then it will compare the 2 previous storage and find the changes , store it after that send them the affected group:
If you could send your code to me: it may be helping me.
And thanks a lot

Charles
04-14-2007, 06:47 AM
This is a bit more involved than you will need, but without seeing the schedule there is no way to know just what you do need. I run this every night to check for updates to the web sites of some of my local dojos. For most sites I can get away with checking the response headers for changes to the modification date and the content length. Some sites don't send that info so I just count the characters myself. One guy has the current time stamped server side so I have to filter that out first. (Please people, don't stamp the current time on you web sites. I already own a clock.) The important thing to note is that the robot is a little bit different for each application.
#!c:\perl\bin\perl.exe

use strict;
use HTML::TreeBuilder;
use HTTP::Date;
use LWP::Simple;
require LWP::UserAgent;
use Win32;
use Win32::Process;

my $ua = LWP::UserAgent->new;

my %data = ();
my @data_changed = ();

sub ErrorReport{
print Win32::FormatMessage( Win32::GetLastError() );
}

$/ = '';

while (<DATA>) {
chomp;
my ($site, @data) = split "\n";
my $old_headers = $data[0];

if ($site eq '~http://www.asu.org/cgi-bin/display-cal.cgi') {

# to do

} elsif ($site =~ m|http://www.aikidosuimei.com/|) {

my $root = HTML::TreeBuilder->new_from_content (get $site);
$root->look_down ('id', 'dnn_dnnCURRENTDATE_lblDate')->delete;
my $count = split '', $root->as_text();
if ($count != $old_headers) {
$data {$site} = $count;
push @data_changed, $site;
print "updated\t$site\n";
} else {
$data {$site} = $old_headers;
print "OK\t$site\n";
};
$root->delete;

} elsif ($site eq 'http://www.aikido-shobukan.org/seminars/?seminarid') {

my $seminar_id = $old_headers || 1;
my $r = $ua->head ("$site=$seminar_id");
while ($r->is_success) {
push @data_changed, "$site=$seminar_id";
$seminar_id++;
$r = $ua->head ("$site=$seminar_id");
};
if ($seminar_id > $old_headers) {
$data {$site} = $seminar_id;
print "updated\t$site=$seminar_id\n";
} else {
$data {$site} = $old_headers;
print "OK\t$site=$seminar_id\n";
}

} else {

my $r = $ua->head ($site);
my $length = $r->header ('content-length');
my $time = str2time ($r->header('last-modified'));
unless ($length) {
my $root = HTML::TreeBuilder->new_from_content (get $site);
$length = split '', $root->as_text();
$time = undef;
$root->delete;
};
my $new_headers = join '+', $length, $time;
if ($old_headers ne $new_headers) {
$data {$site} = $new_headers;
push @data_changed, $site;
print "updated\t$site\n";
} else {
$data {$site} = $old_headers;
print "OK\t$site\n";
}
}

}

if (@data_changed) {

my $site_list = join ' ', @data_changed;
my $p;
Win32::Process::Create($p, 'C:\Program Files\Mozilla Firefox\firefox.exe', "firefox $site_list", 0, NORMAL_PRIORITY_CLASS, '.')|| die ErrorReport();

open (FILE, $0) or die;
$/ = "\n__DATA__";
my $this_script = <FILE>;
open (FILE, ">$0") or die;
print FILE $this_script;
print FILE "\n$_\n$data{$_}\n" foreach (sort keys %data);
close FILE;
print 'done';
}


__DATA__
http://www.aikido-nova.org/
9509+1174308529

http://www.aikido-nova.org/seminars.html
10523+1175214630

http://www.aikido-shobukan.org/
13676+

http://www.aikido-shobukan.org/?ref=53
13017+

http://www.aikido-shobukan.org/Seminars/
12751+

http://www.aikido-shobukan.org/seminars/?seminarid
65

http://www.aikido-westreading.org/
11838+1175452295

http://www.aikido-westreading.org/calendar.htm
6469+1175652338

http://www.aikidoeibukan.com/
1476+

http://www.aikidointhefan.org/
23+

http://www.aikidosuimei.com/Default.aspx?tabid=104
2436

http://www.aikidosuimei.com/Home/tabid/102/Default.aspx
3609

http://www.asu.org/cgi-bin/display-cal.cgi
5862+

http://www.capitalaikikai.org/events.html
1056+1152886712

http://www.georgetownaikikai.com/aikido/
4496+

http://www.goldmountainforge.com/aikido/seminars.htm
1350+

rubaf
04-14-2007, 01:37 PM
Hi Charlas

Thanks a lot for you .But I need code to generate xml file by taken the html page content and after 15 minutes generate a new xml file for the same page then find the differences between 2 file and store the changes in new file and send this file content to the affected people .
For example if DB teacher specify first exam. The code will find that there is a change then sends it to affected student's email.
But really I need more help in my project; I ‘am finished my
Project but I don't know from where and how I should start to design & programming .

Note :( the xml files saved in server, it means the access will not be directly to the database ) .

Charles
04-14-2007, 02:14 PM
Thanks a lot for you .But I need code to generate xml file by taken the html page content and after 15 minutes generate a new xml file for the same page then find the differences between 2 file and store the changes in new file and send this file content to the affected people . I got that from your first post. I'm also getting that this is way over your head. Perhaps you would do well to go find a programmer at your school. I bet if you bought him or her a few beers you could get lots of help.

rubaf
04-14-2007, 02:22 PM
Thanks a lot Charles :