Safetensor model serving using pytorch torch serve container not starting

Question

I cannot start my saved safetensor model in my pytorch/torchserve container.

I can start my container using:

docker run --rm -p 127.0.0.1:8080:8080 --name test -v docker run --rm -it -p 127.0.0.1:8080:8080 -p 127.0.0.1:8081:8081 --name 805  -v C:\Users\.....t\output\TinyPixel\llama-42m:/home/model-server/model-store pytorch/torchserve:latest

I can enter the container, and the model directory is mounted.

model-server@ec5ebadcef2e:~$ ll

drwxr-xr-x 1 model-server model-server 4096 Nov 26 18:42 ./
drwxr-xr-x 1 root         root         4096 Aug 28 21:07 ../
-rw-r--r-- 1 model-server model-server  220 Feb 25  2020 .bash_logout
-rw-r--r-- 1 model-server model-server 3771 Feb 25  2020 .bashrc
-rw-r--r-- 1 model-server model-server  807 Feb 25  2020 .profile
-rw-r--r-- 1 root         root          246 Aug 28 21:01 config.properties
drwxr-xr-x 3 model-server model-server 4096 Nov 26 18:42 logs/
drwxrwxrwx 1 root         root          512 Nov 26 15:56 model-store/
drwxr-xr-x 1 model-server root         4096 Nov 26 18:42 tmp/

model-server@ec5ebadcef2e:~$ cd model-store/
model-server@ec5ebadcef2e:~/model-store$ ls

config.json generation_config.json model.safetensors special_tokens_map.json tokenizer.json tokenizer_config.json

No model seems to be available but torchserve is running

$ curl  http://localhost:8081/models

{
  "models": []
}

model-server@ec5ebadcef2e:~/model-store$ torchserve

TorchServe is already running, please use torchserve --stop to stop TorchServe.

curl  http://localhost:8080/ping

{ "status": "Healthy" }

 head -30 ts_log.log

2023-11-26T18:42:20,304 [WARN ] main org.pytorch.serve.util.ConfigManager - Your torchserve instance can access any URL to load models. When deploying to production, make sure to limit the set of allowed_urls in config.properties 2023-11-26T18:42:20,304 [WARN ] main org.pytorch.serve.util.ConfigManager - Your torchserve instance can access any URL to load models. When deploying to production, make sure to limit the set of allowed_urls in config.properties 2023-11-26T18:42:20,325 [INFO ] main org.pytorch.serve.servingsdk.impl.PluginsManager - Initializing plugins manager... 2023-11-26T18:42:20,325 [INFO ] main org.pytorch.serve.servingsdk.impl.PluginsManager - Initializing plugins manager... 2023-11-26T18:42:20,584 [INFO ] main org.pytorch.serve.metrics.configuration.MetricConfiguration - Successfully loaded metrics configuration from /home/venv/lib/python3.9/site-packages/ts/configs/metrics.yaml 2023-11-26T18:42:20,584 [INFO ] main org.pytorch.serve.metrics.configuration.MetricConfiguration - Successfully loaded metrics configuration from /home/venv/lib/python3.9/site-packages/ts/configs/metrics.yaml 2023-11-26T18:42:20,965 [INFO ] main org.pytorch.serve.ModelServer - Torchserve version: 0.8.2 TS Home: /home/venv/lib/python3.9/site-packages Current directory: /home/model-server Temp directory: /home/model-server/tmp Metrics config path: /home/venv/lib/python3.9/site-packages/ts/configs/metrics.yaml Number of GPUs: 0 Number of CPUs: 12 Max heap size: 2964 M Python executable: /home/venv/bin/python Config file: /home/model-server/config.properties Inference address: http://0.0.0.0:8080 Management address: http://0.0.0.0:8081 Metrics address: http://0.0.0.0:8082 Model Store: /home/model-server/model-store Initial Models: N/A Log dir: /home/model-server/logs Metrics dir: /home/model-server/logs Netty threads: 32 Netty client threads: 0 Default workers per model: 12 Blacklist Regex: N/A Maximum Response Size: 6553500 Maximum Request Size: 6553500

Any help is appreciated.

Thank you