I have a docker swarm where I deploy 3 copies of my microservice. The job of the microservice is to let a client download files. I am currently testing with large files of up to 3GB in size and multiple such downloads in parallel. I am on 17.06.1-ce
My microservice has “docker.sock” mounted inside my service. It is the same socket that is on my mac-docker-vm
I have a bash script that whether I execute it inside the microservice or on the mac, should give me the same output(As the same socket is mounted inside the container). The output is 3 IP addresses. The script basically is nothing but just runs "docker inspect to get IP addresses". It does that fine. The bash script uses docker command and I think it uses the docker.sock internally to process those commands.
Problem description
- When I have made my microservice busy (I have more than one copy of the service running) that is streaming huge data streams, say up to 3 streams of 3GB files, the docker sock slows down I think. The reason I feel this is that when I send a download stream request, it hits the REST controller, the controller executes the bash script, and sits there waiting for script to finish. To verify my theory that script is the bottle neck and not “Scala’s” “Process” class, while this bottleneck is occurring, I executed the same bash script from my laptop. The script waited for over a minute to respond while the streaming was in progress. Remember, whether I execute the script from my laptop or from within my Scala code(which is inside the microservice, it is the same socket that is being used(as the same docker.sock is mounted)
- How do I debug this further to make sure that my theory is correct and get around it? I understand it is my code base that I wrote to support download of files, but could I be potentially leaving a resource open that makes the socket behave bad? I have not tested this on CentOS Docker. Not sure if behavior will remain the same there too as on Mac