Running CRFSuite examples

Question

I'm trying to use CRFSuite but I can't figure out how to use the example/ner.py and pos.py

Precisely, how do I make an input of the form:

# Ner.py
fields = 'y w pos chk'

or

# Pos.py
fields = 'w num cap sym p1 p2 p3 p4 s1 s2 s3 s4 y'

The "y w pos" I can get from a CoNNL model, for example, but the "chk" part and all those fields in pos.py I don't really get.

Also, is there a way to process a raw text (without all those tags) with CRFSuite given that I have a trained model?

Answer 1

@michele is right. This task requires another dataset. I believe the datasets are here: http://www.cnts.ua.ac.be/conll2003/ner/

Answer 2

It turned out it is simpler to slightly modify the pos.py file to do what it should be doing. Now the input format for pos.py is 'w y', while the features 'num cap sym p1 p2 p3 p4 s1 s2 s3 s4' are all generated by the script itself. This should solve the pos.py issues. Here is the gist:

https://gist.github.com/fnl/21116fa57527946c5dbe

As for the ner.py script, as answered by @Legend already, the relevant input data format can be found, for example, here:

http://www.cnts.ua.ac.be/conll2003/ner/

Answer 3

You cannot use ner.py or pos.py with the data provided by the author of the tutorial. You need a proper CoNLL-2000 data set. :)

Just as an example, you can find it here

I hope I have replied correctly to your question.