I am using “cosinesimil” as the knn.space_type in opendistro elasticsearch version 7.8.0. I indexed 3 documents with attribute type knn_vector(the only other attribute supplied other than the knn_vector was a status term with value 1). The 3 vectors for these docs were (2,2), (2,1) and (2,3).
Surprisingly, when I search for the vector (1,1) [with a post filter of status = 1] using the query:
{
"size": 1,
"query": {
"knn": {
"embedding": {
"vector": [1, 1],
"k": 1
}
}
},
"post_filter": {
"term": {"status": 1}
}
}
I am getting different recall vectors each time I execute the query. Sometimes , I get the doc with vector (2,1) with _score 0.5, while other times I get the doc (2,2) with _score 1.0.
Questions:
- Why am I getting different recall vectors when executing the same query?
- Why is the _score for doc with vector (2,1) from elasticsearch coming to be 0.5. Only cosinesimil should be influencing the score and cosinesimil between (2,1) and (1,1) is around 0.95?
- When using the same ES query with knn parameters k = 3 and size = 3, and execute it multiple times…sometimes I get the doc (2,1) with _score 0.5 and some other times I get it with score 0.95(the actual cosinesimil).