Irrelevant data returned when querying simple string

46 views Asked by At

I have a vector database using the text-ada-002 embedding model. Each vector is just text representing a service or a product.

e.g. "Computer parts for upgrade" "Lights and desk lamps" "Couches, sofas" etc

I am converting a product description into a vector and then using that as the whole query.

However, it seems like irrelevant vectors are being returned. When I search "Computer desk", it's returning these results in my Pinecone vector database:

{
  id: 'foobar-123',
  score: 0.883229554,
  values: [],
  sparseValues: undefined,
  metadata: {
    text: 'Lights and desk lamps'
  },
{
  id: 'foobar-207',
  score: 0.882196,
  values: [],
  sparseValues: undefined,
  metadata: {
    text: 'Computers, laptops, computer parts'
  }
},
{
  id: 'foobar-279',
  score: 0.867525816,
  values: [],
  sparseValues: undefined,
  metadata: {
    text: 'Computer assembling or repair'
  }
},
{
  id: 'foobar-260',
  score: 0.863857865,
  values: [],
  sparseValues: undefined,
  metadata: {
    text: 'non-adjustable tables, desk that does not raise to standing position'
  }
},
{
  id: 'foobar-278',
  score: 0.859145403,
  values: [],
  sparseValues: undefined,
  metadata: {
    text: 'Computer parts for upgrade'
  }
}
},

As you can see, most of these results are not really relevant despite having a high relevancy score. How can I improve the relevancy of the search results?

Out of the 5 results, the one I expect at the top would be

{
  id: 'foobar-260',
  score: 0.863857865,
  values: [],
  sparseValues: undefined,
  metadata: {
    text: 'non-adjustable tables, desk that does not raise to standing position'
  }
},

and for "Lights and desk lamps" to not be in the list

0

There are 0 answers