How to use Langchian SQL with Llama 2 GGUF

Question

I am trying to use Llama 2 GGUF 8 bit quantized model to run with Langchain SQL agent. In their docs, they use openAI's 3.5 turbo model and I saw someone use Photolens/llama-2-7b-langchain-chat model and I wanted to use the quantized version of it which is, YanaS/llama-2-7b-langchain-chat-GGUF. I always get errors. Here is my code below,

from langchain.utilities import SQLDatabase
from llama_cpp import Llama
from langchain_experimental.sql import SQLDatabaseChain

llm = Llama(model_path="./models/llama-2-7b-langchain-chat-Q8_0.gguf")

# Define your MySQL connection parameters
mysql_config = {
    "host": "localhost",
    "user": "root",
    "password": "myPassword",
    "database": "myDB",
}

connection_string = f"mysql+mysqlconnector://{mysql_config['user']}:{mysql_config['password']}@{mysql_config['host']}/{mysql_config['database']}"


# Connect to the MySQL database
db = SQLDatabase.from_uri(connection_string)

db_chain = SQLDatabaseChain.from_llm(llm, db, verbose=True)
db_chain.run("How many employees are there?")

I get the error because I am passing llm that is llama2 which expects BaseLanguageModel.

New to LLM. Would be a big help if somebody could help with this.

is there other way of doing this? or do I need to add/remove something? Any help is much appriciated! Thanks alot!!!