Trying to run the LLama-2-7B-chat-GGUF on local machine but getting NotImplementedError

92 views Asked by At

I am trying to run LLama-2-7B-chat-GGUF on local machine

import torch
import torch.nn as nn
!pip install ctransformers
from ctransformers import AutoModelForCausalLM, AutoTokenizer
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")


model = AutoModelForCausalLM.from_pretrained("TheBloke/Llama-2-7B-Chat-GGUF", hf=True)
tokenizer = AutoTokenizer.from_pretrained(model)

I've attached the error I get. I think it has something to do with tokenizer because later in the code when I try do anything with tokenizer, it says 'tokenizer' not defined

0

There are 0 answers