How to run docker commands in azure devops build pipelines by not using standard tasks

Question

I need to run docker commands such as "docker build" and "docker push" in azure devops build pipelines. I know there are tasks available to do these things such as imagebuildingInfo@1 etc. But issue is I need to do this for a new ACR for each pipeline run, and imagebuildingInfo@1 task expects the service connection of type docker registry not azure rm. So I have to create a new service principal of every run of the pipeline which I can't do.

Is there a way to run docker commands in a script in pipeline? I am trying this

- task: AzureCLI@2
                #   displayName: Registry Login
                #   name: acrLogin
                #   env:
                #     ADME_SUBSCRIPTION: 'id'
                #     REGISTRY: 'acr name'
                #   inputs:
                #     azureSubscription: 'azure rm service principal'
                #     scriptType: 'bash'
                #     scriptLocation: 'inlineScript'
                #     inlineScript: |
                #      az acr login -n ${REGISTRY}

But while running this I am getting the error:

WARNING: You may want to use 'az acr login -n <acr_name> --expose-token' to get an access token, which does not require Docker to be installed.

ERROR: 2024-03-20 10:22:28.619871 An error occurred: DOCKER_COMMAND_ERROR
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?

Which means docker deamon is not running on the agent. We have to use a template which might be causing this not to run. Is there a way to run the docker Deamon explicitly in pipeline?

Answer 1

Self-hosted agentpool

If you are using a self-hosted agentpool where you install the agents yourself, and need to run the docker commands directly in other tasks, these are the commands on how I solved that for a self-hosted agentpool and the agent user on the VM:

# Install Docker (https://docs.docker.com/engine/install/ubuntu/#install-using-the-repository)
# Add Docker's official GPG key
sudo apt-get update
sudo apt-get install -y ca-certificates curl
sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc
# Add the repository to Apt sources
echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \
  $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update

# Install the Docker packages
sudo apt-get install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

# Set Docker permissions
sudo chmod 666 /var/run/docker.sock

# Add agent user to the Docker group
sudo usermod -aG docker <agent-user-here>

# Restart docker
sudo systemctl restart docker

Installing it in this manner should solve the error:

ERROR: 2024-03-20 10:22:28.619871 An error occurred: DOCKER_COMMAND_ERROR
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?

Once installed you should be i.e. be able to do the following:

az acr login -n ${REGISTRY}
docker build
docker push

I hope this helps, good luck!

Answer 2

Based on your description, I could reproduce the issue with a pipeline running on a Linux self-hosted agent machine, where docker and the pipeline agent service were newly installed.

This is because the user that the pipeline agent service is configured to run as, is not granted permission to connect to the Docker daemon. When installing agent service with sudo ./svc.sh install [username], if username parameter is not specified then the username is taken from the $SUDO_USER environment variable which is set by sudo command. This variable is always equal to the name of the user who invoked the sudo command.

Here are my steps to fix the error.

1. Add the user (Alvin in my case) to run pipeline agent service in docker admin group;

   sudo usermod -aG docker Alvin
   

2. Stop pipeline agent service in the agent physical location directory;

   sudo ./svc.sh stop
   

   
3. Configure agent service with environment update (docker permission assigned);

   ./env.sh
   

   
4. Start the pipeline agent service and Rerun failed jobs without changing anything in the pipeline;

   sudo ./svc.sh start