I am trying to install a language model called llama2 from here.
Followed this guide with some necessary adjustments (see reproducability below), but in step 7 it wants to use a Main.exe which is not there. I looked in several Llama2 repos but no one provides a Main.exe.
How do I find the Main.exe that is missing here to run the llama2 model?
Reproducability I have done the following:
- in Git Bash:
git clone https://github.com/sychhq/llama-cpp-setup.git && cd llama-cpp-setup && chmod +x setup.sh && ./setup.sh
- It struggles with SSL licences so I add
git config --global http.sslbackend schannel
and rerun line above. It now works and asks for settings. I use default for both questions so press enter twice. - It takes 6 mins to download en ends up in C:/Users/…/llama-cpp-setup
- I now clone the llama repo by
git clone https://github.com/facebookresearch/llama.git
- I now install wget and put it in
C:/Windows/System32
- Then I run
/bin/bash ./download.sh
where I put the URL that I requested from https://ai.meta.com/resources/models-and-libraries/llama-downloads/ - I updated to Python 11 and run
python -m venv llama2
fromcd llama.cpp
- Then to activate the model (source)
llama2\Scripts\activate
andpython3 -m pip install -r requirements.txt
- Then
python3 convert.py --outfile models/7B/ggml-model-f16.bin --outtype f16 ../../llama-cpp-setup/llama/llama-2-7b --vocab-dir ../../llama-cpp-setup//llama/llama (let op de paths!)
quantize ./models/7B/ggml-model-f16.bin ./models/7B/ggml-model-q4_0.bin q4_0
comprimises the model. This happens from a quantize.exe program that is visible in the File Explorer.- Finally I run
./main -m ./models/7B/ggml-model-q4_0.bin -n 1024 --repeat_penalty 1.0 --color -i -r "User:" -f ./prompts/chat-with-bob.txt
but it cannot find a main.exe . Which is obviously true, because there is no Main.exe in the directory (I checked in the Explorer).