Fix SnowballStemmer Error "AttributeError: 'English' object has no attribute 'Default"

Question

Hello there:) I am using the package SnowballStemmer and i am getting an Error. I am very glad for any kind of help :)

Code:

stem2 =[]

for word in stem:
    if word not in nlp.Default.stop_words: 
        stem2.append(word)

print(stem2)

Error here:

line 127, in <module>
    if word not in nlp.Default.stop_words:  
AttributeError: 'English' object has no attribute 'Default'

Answer 1

It would be easier to answer the question if you would show where the variable nlp is coming from.

But from what you are saying I assume that you refer to this package: https://pypi.org/project/snowballstemmer and as far as I can see, it does not define any stopwords.

If you are using the nltk package then you can do this:

import nltk

# needed once - nltk seems to cache it
nltk.download('stopwords')
# load cached stop words
stopwords = frozenset(nltk.corpus.stopwords.words('english'))

stem2 =[]
for word in stem:
    if word not in stopwords:
        stem2.append(word)

If you are using the spacy package you can do for example

from spacy.lang.en.stop_words import STOP_WORDS

for word in stem:
    if word not in STOP_WORDS:
        stem2.append(word)

Even faster should be a list comprehension:

stem2 = [word for word in stem if word not in STOP_WORDS]

The code above of course assumes that a variable stem is defined that would most probably be a list of strings. depending on your requirements, you might want to check the actual stopwords, they might be slightly differnt sets of words based on the library you choose, so the solution above do not generally return the same result.