AllenNLP fails on "to be" sentences. Something Broke along the Way

211 views Asked by At

Simple sentences involving the verb, "is" return no results for semantic role labeling, either via the demo page or by using AllenNLP in Python3.8 with the latest November Bert base model.

For example, "I am here." returns nothing.

In short:

  • Instances of simple "A is B" sentences don't return any results.
  • I believe there should be some sort of output, as other SRL engines do return results.
  • The same goes for "I am." The expected result is an ARG1 for "I" and a predicate of "am."

This used to work with an earlier version:

allennlp==1.0.0
allennlp-models==1.0.0

Related issues or possible duplicates

  • None

Environment

OS: macOS 10.15.7 (Catalina)

Python version: 3.8.6 (via home-brew)

allennlp==1.2.2
allennlp-models==1.2.2
attrs==20.3.0
blis==0.4.1
boto3==1.16.24
botocore==1.19.24
catalogue==1.0.0
certifi==2020.11.8
chardet==3.0.4
click==7.1.2
conllu==4.2.1
cymem==2.0.4
dataclasses==0.6
filelock==3.0.12
ftfy==5.8
future==0.18.2
h5py==3.1.0
idna==2.10
importlib-metadata==3.1.0
iniconfig==1.1.1
jmespath==0.10.0
joblib==0.17.0
jsonnet==0.17.0
jsonpickle==1.4.1
murmurhash==1.0.4
nltk==3.5
numpy==1.19.4
overrides==3.1.0
packaging==20.4
plac==1.1.3
pluggy==0.13.1
preshed==3.0.4
protobuf==3.14.0
py==1.9.0
py-rouge==1.1
pyparsing==2.4.7
pytest==6.1.2
python-dateutil==2.8.1
regex==2020.11.13
requests==2.25.0
s3transfer==0.3.3
sacremoses==0.0.43
scikit-learn==0.23.2
scipy==1.5.4
sentencepiece==0.1.91
six==1.15.0
spacy==2.3.2
srsly==1.0.4
tensorboardX==2.1
thinc==7.4.1
threadpoolctl==2.1.0
tokenizers==0.9.3
toml==0.10.2
torch==1.7.0
tqdm==4.53.0
transformers==3.5.1
typing-extensions==3.7.4.3
urllib3==1.26.2
wasabi==0.8.0
wcwidth==0.2.5
word2number==1.1
zipp==3.4.0

Steps to reproduce

Visit the demo website for SRL

Example:

Enter almost any variation of: "I am here." "We are people." "I am."

#  https://demo.allennlp.org/semantic-role-labeling/MjU3NDk3NA==

# or
from allennlp.predictors.predictor import Predictor

allen_predictor_srl = Predictor.from_path(
    "./models/bert-base-srl-2020.11.19.tar.gz"
)

output = allen_predictor_srl.predict(sentence="I am here.")
print(output)

# observe nothing

EDIT: I tried installing the exact library versions I used to have (i.e. AllenNLP 1.0), but the issue persists. I frankly have no idea what is going on.

EDIT 2:

Install script:

python3 -m venv env
source ./env/bin/activate


pip3 install --upgrade pip

pip3 install -U --no-cache-dir

pip3 install -U allennlp allennlp-models --no-cache-dir

python3 -m spacy download en_core_web_lg    --no-cache-dir
python3 -m spacy download en_core_web_sm    --no-cache-dir
python3 -m spacy download en_vectors_web_lg --no-cache-dir
python3 -m spacy download de_core_news_md   --no-cache-dir
1

There are 1 answers

2
Dirk Groeneveld On BEST ANSWER

To provide some closure, the issue was caused by an update in Spacy. We have a fix in https://github.com/allenai/allennlp-models/pull/178 (thank you https://github.com/wangrat), and it will be officially released in AllenNLP 1.3.

If you need this feature earlier than that, we recommend checking out the main branch of AllenNLP and installing it with pip install -e ..