I am working on text classifications and faced misspelling problem. I am trying to solve it with spacy_hunspell but can`t get what is the point of hunspell_spell property, cause it always returns None.
for token in nlp('I can haz cheezeburger.'):
print(token._.hunspell_spell)
if not token._.hunspell_spell:
print(token.text, token._.hunspell_suggest)
Code listed above returns:
None
I ['I', 'Ia', 'In', 'Ir', 'It', 'Io', 'IE', 'IA', 'AI', 'IN', 'RI', 'IT', 'IL', 'ID', 'DI']
None
can ['Can', 'van', 'cab', 'cam', 'ca', 'an', 'cane', 'cans', 'scan', 'cant', 'clan', 'cyan', 'Scan', 'car', 'ran']
None
haz ['ha', 'haze', 'hazy', 'has', 'hat', 'had', 'hag', 'ham', 'hap', 'hay', 'haw', 'ha z']
None
cheezeburger ['cheeseburger', 'vegeburger']
None
. []
At the beginning i thought that this property returns is token misspelled or not.
Does anybody know what is the meaning of property hunspell_spell?
How to check that word is misspelled with spacy_hunspell (does not comply with morphological rules)?
I think my implementation is not ideal.
for token in nlp('I can haz cheezeburger.'):
is_correct = any([token.text == str.lower(suggest) for suggest in token._.hunspell_suggest])
is_correct = "correct" if is_correct else "misspeled"
print(f"{token.text} is {is_correct}")
Code listed above returns:
I is misspeled
can is correct
haz is misspeled
cheezeburger is misspeled
. is misspeled
It's a free spell checker and morphological analyzer library and command-line tool