I am using Universal sentence encoder to encode some documents into a 512 dimensional embeddings. These are then used to find similar items to a search query which is also encoded using USE. USE works pretty well on general english words in search query and documents but performs really bad when the search query contains rare keywords such as people's name etc. I am thinking of enabling a reranker over the search results that takes into account the number of rare words present in the search query and the document retrieved. This should boost the scores of documents which contain known words while reduce the score of documents that contain unknown words.
My question is How do I get the grammar of Universal sentence encoder to implement such re-ranker?