NLP Date Parsing

1.5k views Asked by At

I've been experimenting with a number of NLP text parsers, but have found that most fail at even some of the simplest tasks that occur in actual texts (aren't preprocessed to show how "great" the systems are. An example is the following:

From Sundays until Thursdays every week

I've yet to find a single parser that can parse this correctly. I've tried with quite a number including Stanford's sutime. Can anyone recommend software that can handle natural text dates?

1

There are 1 answers

0
Ian Mercer On

I did not find one either when I went looking so I wrote my own. It's part of my natural language engine for .NET.

Here's what the demo shows when you enter that phrase (qualified to next week rather than every week - it can handle that too but it's infinite):

Sample output

Some comments:

1) Handling all possible english language temporal expressions is a huge task. I've been working on this problem for years to come up with a clean way to represent temporal expressions plus the many rules needed to parse english expressions of time.

2) In addition to finding a way to represent typical calendar date times and ranges of such, you also need ways to represent infinite sequences like 'every monday', and half-infinite sequences like 'every weekday before ...'. And then you'll need an algebra on top of that for combining temporal expressions.

3) Temporal expressions are often ambiguous in the English language and interpretation may vary from culture to culture.

4) The result must often be interpreted in the context of the sentence and/or the conversation history. "Who called Monday?" is a different Monday from "Remind me on Monday" and is different again from "Show me statistics for Monday".