I'm using this script in order to remove Stop-Words in Perl, I'm running on Windows and I could not find compatible version of:
Lingua::EN::StopWordList
Lingua::StopWords qw(getStopWords)
I have an array of Stop-Words but once I use the REGEX below, I loss critical white-space that causes words to clash. Note that the every word in the Stop-Word array has two spaces, one on the right and one on the left.
How can I remove Stop-Words efficiently without losing crucial white-spaces?
use strict;
use warnings;
use utf8;
use IO::File;
use String::Util 'trim';
my $inFile = "C:\\Users\\David\\Downloads\\InfoRet\\Explore the ways to get better grades.txt";
my $inFh = new IO::File $inFile, "r";
my $lineNum = 0;
my $line = undef;
my $loc = undef;
my $str = undef;
my @stopList = (" the ", " a ", " an ", " of ", " and ", " on ", " in ", " by ", " with ", " at ", " after ", " into ", " their ", " is ", " that ", " they ", " for ", " to ", " it ", " them ", " which ");
for(my $i = 1; $i <= 4; $i++) {
<$inFh>
}
while($line = <$inFh>) {
$lineNum++;
chomp $line;
$line =~ s/[\$#@~!&*()\[\];.,:?^`\\\/]+//g;
for my $planet (@stopList) {
$loc = index($line, $planet);
if($loc!=(-1)) {
#$line =~ s/$str//g;
$line =~ s/$planet//g;
}
}
print "$line\n";
}
and later,