Walrus is not indexing the words properly

42 views Asked by At

In my Django project, I am using walrus to cache location names. eg: New Zealand, New York City, Newcastle e.t.c So, when I am searching for a key 'new', I am expecting it to return all the above locations but it is only giving me Newcastle. But I when I use 'n' or 'ne' as a key I am getting all of these. Any help is appreciated.

1

There are 1 answers

0
riyasyash On BEST ANSWER

Finally found out the issue, when you initialise an object of walrus there is an option to pass a stopwords_file . If you don't pass any the default file defined inside the library called stopwords.txt is taken. This stopwords.txt file had a lot of words listed like 'new'. So whenever a word from the stopwords file is found in the word to index, it will not index that particular word.

In my case 'new' was present in stopwords.txt. So when it indexed 'New York' it didn't map the word 'New york' to 'new', but it mapped it to 'york'. That is why I couldn't search with 'new'.

I solved it by initialising the walrus db search object with an empty stopwords_file.