Chatbox is returning one word answer instead of full sentence

Question

I am creating a Book recommendation system with Retrieval Augmented Generation(RAG) framework and vector database in LangChain. But I am getting just the answer not the whole sentence like a chatbot should.

This is the code:

embeddings = HuggingFaceEmbeddings(model_name='bert-base-uncased')
docsearch = FAISS.from_documents(texts, embeddings)
llm = HuggingFaceHub(repo_id = "google/flan-t5-base", model_kwargs={"temperature":0.6,"max_length": 500, "max_new_tokens": 200})
prompt_template = """

Compare the book given in question with others in the retriever based on genre and description.
Return a complete sentence with the full title of the book and describe the similarities between the books.

question: {question}
context: {context}
"""
prompt = PromptTemplate(template=prompt_template, input_variables=["context", "question"])
retriever=docsearch.as_retriever()
qa = RetrievalQA.from_chain_type(llm=llm, chain_type="stuff", retriever=retriever,chain_type_kwargs = {"prompt": prompt})
qa.run({"query": "Which book except 'To Kill A Mocking Bird' is similar to it?"} )

I am just getting the name of the book. I atleast need a full sentence and maybe some description. I cannot use bigger models as some dont work and others time out. OpenAI models also returned limit exceeded. Does it need a better prompt?

Answer 1

I ran your code. If I give

docsearch = FAISS.from_texts(
    ["harry potter's owl is in the castle. blahblah, The book is about 'To Kill A Mocking Swan'. There is another monkey"], embeddings)

The output will be To Kill A Mocking Swan, which seems expected. I think the reason of your "problem" is that the embedding data contains To Kill A Mocking Bird in VectorStore, which exactly matches by similarity_search