How to capture docker pull status

Question

I'm trying to post docker image downloading status to frontend in a real time manner. Currently it can just tell that docker pull is going on. I understand that there could be multiple layers of different sizes and the sizes of layers that haven't started downloading could be unknown. So I'm thinking about showing how many layers are there and the current downloading status of the layers that have already started downloading. Do you know how to capture the data that's shown on terminal console when you do docker pull <image>? My program is in Python so that I would like to get the data in python code and send it to frontend. I checked the docker documentation but didn't find any api that supports this. Looks like there's no public api that supports this functionality yet.

Do you have any other approaches that may realize my goal?

Any suggestions are welcomed and appreciated!

Thank you!

Answer 1

You need a low-level pull() call for that (docs). Here's an example:

>>> import docker
>>> import json
>>> client = docker.APIClient(base_url='unix://var/run/docker.sock')
>>> for line in client.pull('busybox', stream=True, decode=True):
...     print(json.dumps(line, indent=4))
{
    "status": "Pulling from library/busybox",
    "id": "latest"
}
{
    "status": "Pulling fs layer",
    "progressDetail": {},
    "id": "e5d9363303dd"
}
{
    "status": "Downloading",
    "progressDetail": {
        "current": 8635,
        "total": 764663
    },
    "progress": "[>                                                  ]  8.635kB/764.7kB",
    "id": "e5d9363303dd"
}
{
    "status": "Downloading",
    "progressDetail": {
        "current": 764663,
        "total": 764663
    },
    "progress": "[==================================================>]  764.7kB/764.7kB",
    "id": "e5d9363303dd"
}
{
    "status": "Verifying Checksum",
    "progressDetail": {},
    "id": "e5d9363303dd"
}
{
    "status": "Download complete",
    "progressDetail": {},
    "id": "e5d9363303dd"
}
{
    "status": "Extracting",
    "progressDetail": {
        "current": 32768,
        "total": 764663
    },
    "progress": "[==>                                                ]  32.77kB/764.7kB",
    "id": "e5d9363303dd"
}
{
    "status": "Extracting",
    "progressDetail": {
        "current": 764663,
        "total": 764663
    },
    "progress": "[==================================================>]  764.7kB/764.7kB",
    "id": "e5d9363303dd"
}
{
    "status": "Extracting",
    "progressDetail": {
        "current": 764663,
        "total": 764663
    },
    "progress": "[==================================================>]  764.7kB/764.7kB",
    "id": "e5d9363303dd"
}
{
    "status": "Pull complete",
    "progressDetail": {},
    "id": "e5d9363303dd"
}
{
    "status": "Digest: sha256:c5439d7db88ab5423999530349d327b04279ad3161d7596d2126dfb5b02bfd1f"
}
{
    "status": "Status: Downloaded newer image for busybox:latest"
}

Answer 2

docker-py provides low level APIs to pull image along with streaming status. You can use this snippet to show it like a progress bar similar to docker pull command. It first shows red progress-bars for showing download status, then green status bars for extract :-)

import docker
from rich.progress import Progress

tasks = {}

# Show task progress (red for download, green for extract)
def show_progress(line, progress):
    if line['status'] == 'Downloading':
        id = f'[red][Download {line["id"]}]'
    elif line['status'] == 'Extracting':
        id = f'[green][Extract  {line["id"]}]'
    else:
        # skip other statuses
        return

    if id not in tasks.keys():
        tasks[id] = progress.add_task(f"{id}", total=line['progressDetail']['total'])
    else:
        progress.update(tasks[id], completed=line['progressDetail']['current'])

def image_pull(image_name):
    print(f'Pulling image: {image_name}')
    with Progress() as progress:
        client = docker.from_env()
        resp = client.api.pull(image_name, stream=True, decode=True)
        for line in resp:
            show_progress(line, progress)

if __name__ == '__main__':
    # Pull a large image
    IMAGE_NAME = 'bitnami/pytorch'
    image_pull(IMAGE_NAME)

Progress bar for docker pull through Python API