Pycharm doesn't recognize nltk (installed with Anaconda)

Question

I'm using PyCharm to write a program that uses the nltk package. My first line is:

 from nltk import word_tokenize, sent_tokenize

I have the nltk package imported in my 2.7 Python environment (the environment I'm working on) in PyCharm, as shown here:

However, PyCharm doesn't recognize the from nltk.. line. It's grayed out; it also shows this error:

This inspection detects names that should resolve but don't. Due to dynamic dispatch and duck typing, this is possible in a limited but useful number of cases. Top-level and class-level items are supported better than instance items.

Here is my code:

from nltk import word_tokenize, sent_tokenize

annot1 = [(500L, u'[they seldom desire anything unless it belongs to     others]')]
annot2 = (500L, u'[they seldom desire anything unless it belongs to others]')

def scope_match(annot1, annot2):
    tokens1 = annot2[1].encode('utf-8')
    print type(tokens1)
    for string in tokens1:
        tokens2 = nltk.word_tokenize(string)
        print 'these are the tokens: ', tokens2
        new2 = [a.strip('[]').encode('utf-8') for a in tokens2]
        print new2

scope_agr = scope_match(annot1, annot2)
print scope_agr

When I run the code, I get this error: `C:\Users\nepal\Anaconda3\envs\py27\python.exe /Users/nepal/PycharmProjects/ScopeCue/ScopeComparison/scope-compare-inter-annotation-agreement-TEST.py

Traceback (most recent call last):
  File "C:/Users/nepal/PycharmProjects/ScopeCue/ScopeComparison/scope-compare- inter-annotation-agreement-TEST.py", line 1, in <module>
from nltk import word_tokenize, sent_tokenize
ImportError: cannot import name word_tokenize

Process finished with exit code 1`

Can somebody guide me to solve this issue? Thanks so much in advance.

Answer 1

Your import error shows that the module nltk is found, but does not contain word_tokenize. 99% of the time this means that you have created a file nltk.py in the same directory as your script.

In fact you seem to be one of the exceptions-- sort of: The last error trace you posted in the comments shows that you have created an entire nltk package (a folder with __init__.py)! Get rid of it or rename it so that python can find the real nltk.

Answer 2

I resolved it by downloading the complete nltk package from using nltk.download() from the terminal.

So, I opened a new python session and then did:

import nltk

nltk.download()

A new window opened and asked me if I wanted to download, which I accepted. Now it's running fine.

I wonder if Anaconda does not install the complete nltk package?.... Before trying this solution, I re-installed using Anaconda twice (conda install -c anaconda nltk=3.2.1). But it seems that using that command does not get the whole nltk package...

Anyways, I hope it helps the next person.