Flask app continuously restarting after downloading huggingface models

Question

In my docker container I have a flask app (behind nginx and uwsgi) which instantiates a model from huggingface/transformers. For some reason, the app continuously restarts when trying to after downloading the models

App:

### app.py
server = Flask(__name__)
cors = CORS(server)
server.config["CORS_HEADERS"] = "Content-Type"
log.info("Instantiating model")
model = AutoModelForTokenClassification.from_pretrained('dbmdz/bert-large-cased-finetuned-conll03-english')
tokenizer = AutoTokenizer.from_pretrained('bert-base-cased')
pipe = pipeline('ner', model=model, tokenizer=tokenizer)

if __name__ == "__main__":
    server.run(host=appconf.host, port=appconf.port, debug=appconf.isdev, use_reloader=False)

Logs:

[2020-09-29 14:13:11,704] {./app.py:14} INFO - Instantiating model
[2020-09-29 14:13:11,708] {/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py:939} DEBUG - Starting new HTTPS connection (1): s3.amazonaws.com:443
[2020-09-29 14:13:12,170] {/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py:433} DEBUG - https://s3.amazonaws.com:443 "HEAD /models.huggingface.co/bert/dbmdz/bert-large-cased-finetuned-conll03-english/config.json HTTP/1.1" 200 0
[2020-09-29 14:13:12,176] {/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py:939} DEBUG - Starting new HTTPS connection (1): cdn.huggingface.co:443
[2020-09-29 14:13:12,317] {/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py:433} DEBUG - https://cdn.huggingface.co:443 "HEAD /dbmdz/bert-large-cased-finetuned-conll03-english/pytorch_model.bin HTTP/1.1" 200 0
[uWSGI] getting INI configuration from /etc/uwsgi/uwsgi.ini
*** Starting uWSGI 2.0.19.1 (64bit) on [Tue Sep 29 14:13:55 2020] ***
...
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 1727064 bytes (1686 KB) for 16 cores
*** Operational MODE: preforking ***
mounting app:server on /
Comment: FROM HERE IT REPEATS
[2020-09-29 14:13:57,254] {./app.py:14} INFO - Instantiating model <-- AGAIN!
[2020-09-29 14:13:57,257] {/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py:939} DEBUG - Starting new HTTPS connection (1): s3.amazonaws.com:443
[2020-09-29 14:13:57,686] {/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py:433} DEBUG - https://s3.amazonaws.com:443 "HEAD /models.huggingface.co/bert/dbmdz/bert-large-cased-finetuned-conll03-english/config.json HTTP/1.1" 200 0
[2020-09-29 14:13:57,693] {/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py:939} DEBUG - Starting new HTTPS connection (1): cdn.huggingface.co:443
[2020-09-29 14:13:57,790] {/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py:433} DEBUG - https://cdn.huggingface.co:443 "HEAD /dbmdz/bert-large-cased-finetuned-conll03-english/pytorch_model.bin HTTP/1.1" 200 0

I am sure to have http_proxy and https_proxy set within the Docker container.

Thanks for the help

Answer 1

The problem got resolved by increasing the CPU and memory capacity on the server where the container is running.