Missing main.exe when installing llama2 model

186 views Asked by At

I am trying to install a language model called llama2 from here.

Followed this guide with some necessary adjustments (see reproducability below), but in step 7 it wants to use a Main.exe which is not there. I looked in several Llama2 repos but no one provides a Main.exe.

How do I find the Main.exe that is missing here to run the llama2 model?

Reproducability I have done the following:

  • in Git Bash: git clone https://github.com/sychhq/llama-cpp-setup.git && cd llama-cpp-setup && chmod +x setup.sh && ./setup.sh
  • It struggles with SSL licences so I add git config --global http.sslbackend schannel and rerun line above. It now works and asks for settings. I use default for both questions so press enter twice.
  • It takes 6 mins to download en ends up in C:/Users/…/llama-cpp-setup
  • I now clone the llama repo by git clone https://github.com/facebookresearch/llama.git
  • I now install wget and put it in C:/Windows/System32
  • Then I run /bin/bash ./download.sh where I put the URL that I requested from https://ai.meta.com/resources/models-and-libraries/llama-downloads/
  • I updated to Python 11 and run python -m venv llama2 from cd llama.cpp
  • Then to activate the model (source) llama2\Scripts\activate and python3 -m pip install -r requirements.txt
  • Then python3 convert.py --outfile models/7B/ggml-model-f16.bin --outtype f16 ../../llama-cpp-setup/llama/llama-2-7b --vocab-dir ../../llama-cpp-setup//llama/llama (let op de paths!)
  • quantize ./models/7B/ggml-model-f16.bin ./models/7B/ggml-model-q4_0.bin q4_0 comprimises the model. This happens from a quantize.exe program that is visible in the File Explorer.
  • Finally I run ./main -m ./models/7B/ggml-model-q4_0.bin -n 1024 --repeat_penalty 1.0 --color -i -r "User:" -f ./prompts/chat-with-bob.txt but it cannot find a main.exe . Which is obviously true, because there is no Main.exe in the directory (I checked in the Explorer).
0

There are 0 answers