How to convert pretrained hugging face model to .pt and run it fully locally?

Question

I'm attempting to convert this model in .pt format. It's working fine for me so i dont want to fine-tune it. How can i export it to .pt and run interface?

I tried using this to convert to .pt:

from transformers import AutoConfig, AutoProcessor, AutoModelForCTC, AutoTokenizer, Wav2Vec2Processor
import librosa
import torch



# Define the model name
model_name = "UrukHan/wav2vec2-russian"

# Load the model and tokenizer
config = AutoConfig.from_pretrained(model_name)
model = AutoModelForCTC.from_pretrained(model_name, config=config)
processor = Wav2Vec2Processor.from_pretrained(model_name)
tokenizer = AutoTokenizer.from_pretrained(model_name)

# Save the model as a .pt file
torch.save(model.state_dict(), "model.pt")

# Save the tokenizer as well if needed
tokenizer.save_pretrained("model-tokenizer")

but unfortunately its not running the interface :

model = AutoModelForCTC.from_pretrained("model.pt")
processor = AutoProcessor.from_pretrained("model.pt")


# Perform inference with the model
FILE = 'here is wav.wav'
audio, _ = librosa.load(FILE, sr = 16000)
audio = list(audio)
def map_to_result(batch):
  with torch.no_grad():
    input_values = torch.tensor(batch, device="cpu").unsqueeze(0) #, device="cuda"
    logits = model(input_values).logits
  pred_ids = torch.argmax(logits, dim=-1)
  batch = processor.batch_decode(pred_ids)[0]
  return batch
map_to_result(audio)
print(map_to_result(audio))


model.eval()

And encountered an error: `model.pt is not a local folder and is not a valid model identifier listed on 'https://huggingface.co/models'

`

Answer 1

You don't have to worry about file endings in this case.

torch.save basically writes the input to a zip file (it also works for arbitrary python objects, not just weights). The .pt file ending is just convention.

Huggingface saves model weights to a directory, usually of the form model_name.hf/.

The error you are seeing is because AutoModelForCTC.from_pretrained("model.pt") results in huggingface code looking for a directory named model.pt, when no such directory exists.

When you download the model the first time, the weights are saved to cache, usually ~/.cache/huggingface/hub.

If you want to explicitly save the model weights to a different directory, you can run:

model = AutoModelForCTC.from_pretrained(model_name, config=config)
model.save_model("path_to_save")