I have a vector database using the text-ada-002
embedding model. Each vector is just text representing a service or a product.
e.g. "Computer parts for upgrade" "Lights and desk lamps" "Couches, sofas" etc
I am converting a product description into a vector and then using that as the whole query.
However, it seems like irrelevant vectors are being returned. When I search "Computer desk", it's returning these results in my Pinecone vector database:
{
id: 'foobar-123',
score: 0.883229554,
values: [],
sparseValues: undefined,
metadata: {
text: 'Lights and desk lamps'
},
{
id: 'foobar-207',
score: 0.882196,
values: [],
sparseValues: undefined,
metadata: {
text: 'Computers, laptops, computer parts'
}
},
{
id: 'foobar-279',
score: 0.867525816,
values: [],
sparseValues: undefined,
metadata: {
text: 'Computer assembling or repair'
}
},
{
id: 'foobar-260',
score: 0.863857865,
values: [],
sparseValues: undefined,
metadata: {
text: 'non-adjustable tables, desk that does not raise to standing position'
}
},
{
id: 'foobar-278',
score: 0.859145403,
values: [],
sparseValues: undefined,
metadata: {
text: 'Computer parts for upgrade'
}
}
},
As you can see, most of these results are not really relevant despite having a high relevancy score. How can I improve the relevancy of the search results?
Out of the 5 results, the one I expect at the top would be
{
id: 'foobar-260',
score: 0.863857865,
values: [],
sparseValues: undefined,
metadata: {
text: 'non-adjustable tables, desk that does not raise to standing position'
}
},
and for "Lights and desk lamps" to not be in the list