Does the question/query vectorized in similaritySearch (Langchain) Does it use tokens

Question

const simpleRes = await vectorStore.similaritySearchWithScore(question);
console.log(simpleRes);

How can I know if it uses tokens in similarity search, does my question converted to vector and uses OpenAIEmbeddings before it search the closest vector in the database?

Answer 1

when you do a similarity search in the database there is no token usage.

Tokens are the basic units of text or code that an LLM AI uses to process and generate language.

You consume tokens when you send the most similar chunks to the LLM

Answer 2

As far as I can see on the code, langchain's similaritySearch method embeds your query so it can perform the search. You can confirm that by looking at model usage report at OpenAI dashboard, where you can find the calls to the embedding model.

It does not use tokens as defined in completion generations, though. You will find model usage as "n_context_tokens_total", which I find quite confusing.

HTH.

Juan Manuel.