Click to See Complete Forum and Search --> : plz check this program for StopWord Removal


kimskams80
04-18-2009, 09:16 AM
hi

I have a string $para_content (also contains new line characters) and I want to remove Stopwords from it that i have defined in an array of words @stop_words. I am using following program but it does not remove stop words from string ... can anybody please check it .. thanks

sub SWR() {
print "I am in SWR function\n";

my %stop_words = map {;$_=>1} @stop_words;

my $filtered_line = '';
for my $word (split /\b/, $para_content) {
if ($stop_words{ $word }) { # converts word (read from doc) to small letters before comparison
# do nothing
}
else {
$filtered_line .= $word;
}
}
print "****************************** After STOP WORD Removal ****************************\n";
print "$filtered_line\n";
} ## end of Sub routine

Sixtease
04-19-2009, 11:16 AM
The function works for me. Your problem lies somewhere else.
#!/usr/bin/perl

use strict;
use warnings;

my @stop_words = qw(
the a of
);
my $para_content;

sub SWR() {
print "I am in SWR function\n";

my %stop_words = map {;$_=>1} @stop_words;

my $filtered_line = '';
for my $word (split /\b/, $para_content) {
if ($stop_words{ $word }) { # converts word (read from doc) to small letters before comparison
# do nothing
}
else {
$filtered_line .= $word;
}
}
print "****************************** After STOP WORD Removal ****************************\n";
print "$filtered_line\n";
} ## end of Sub routine

$para_content = 'a rite of passage';
SWR();
prints ' rite passage' as expected.