Incorrect JSON format from docker compose ps

Question

I'm having a problem with JSON format produced by docker compose ps --format json command.

Here's my example docker-compose.yml file:

version: '3.5'

services:
  nginx:
    image: nginx
    links: 
      - php

  php:
    image: php:fpm

I start my containers by docker compose up -d. When I run docker compose ps it produces the correct output:

NAME                  IMAGE     COMMAND                                          SERVICE   CREATED         STATUS         PORTS
test_docker-nginx-1   nginx     "/docker-entrypoint.sh nginx -g 'daemon off;'"   nginx     4 minutes ago   Up 4 minutes   80/tcp
test_docker-php-1     php:fpm   "docker-php-entrypoint php-fpm"                  php       4 minutes ago   Up 4 minutes   9000/tcp

I wanted to customize the above output a little bit by using --format json and jq tool. However it appeared that docker compose ps --format json produces an incorrect output:

{"Command":"\"/docker-entrypoint.sh nginx -g 'daemon off;'\"","CreatedAt":"2023-09-20 14:32:55 +0200 CEST","ExitCode":0,"Health":"","ID":"956ee02dc6beb2b32fb15dac9cfb3be37bf09bcfd5fe05d9c23b31ba2fbc2108","Image":"nginx","Labels":"com.docker.compose.project.working_dir=/var/www/test_docker,com.docker.compose.version=2.21.0,com.docker.compose.config-hash=0483cc81f74ebb9392330ce8680a4d528986eecb650af475b34bdfe90bfa1dc3,com.docker.compose.container-number=1,com.docker.compose.depends_on=php:service_started:true,com.docker.compose.image=sha256:f5a6b296b8a29b4e3d89ffa99e4a86309874ae400e82b3d3993f84e1e3bb0eb9,com.docker.compose.project=test_docker,com.docker.compose.project.config_files=/var/www/test_docker/docker-compose.yml,maintainer=NGINX Docker Maintainers \[email protected]\u003e,com.docker.compose.oneoff=False,com.docker.compose.service=nginx","LocalVolumes":"0","Mounts":"","Name":"test_docker-nginx-1","Names":"test_docker-nginx-1","Networks":"test_docker_default","Ports":"80/tcp","Publishers":[{"URL":"","TargetPort":80,"PublishedPort":0,"Protocol":"tcp"}],"RunningFor":"6 minutes ago","Service":"nginx","Size":"0B","State":"running","Status":"Up 6 minutes"}
{"Command":"\"docker-php-entrypoint php-fpm\"","CreatedAt":"2023-09-20 14:32:53 +0200 CEST","ExitCode":0,"Health":"","ID":"9ace40ff342457bceeff0433fd882cf65e819ac85cef4216df8815b8630bb7df","Image":"php:fpm","Labels":"com.docker.compose.depends_on=,com.docker.compose.image=sha256:c6b042c2a60f05833e6b5041cb5fbd83ef70f42f9b510decb364dc7306dd8da2,com.docker.compose.project.config_files=/var/www/test_docker/docker-compose.yml,com.docker.compose.config-hash=816ffa2d0d86f46676ce2aba77f5d4ded1e538e99c78aee30db77ba4deb5835e,com.docker.compose.container-number=1,com.docker.compose.oneoff=False,com.docker.compose.project=test_docker,com.docker.compose.project.working_dir=/var/www/test_docker,com.docker.compose.service=php,com.docker.compose.version=2.21.0","LocalVolumes":"0","Mounts":"","Name":"test_docker-php-1","Names":"test_docker-php-1","Networks":"test_docker_default","Ports":"9000/tcp","Publishers":[{"URL":"","TargetPort":9000,"PublishedPort":0,"Protocol":"tcp"}],"RunningFor":"6 minutes ago","Service":"php","Size":"0B","State":"running","Status":"Up 6 minutes"}

It is formatted as {...} {...}, but I would rather expect something like [{...}, {...}].

I tested the same on other machines and it works perfectly OK there, so I assume it must be something with my installation / version / configuration.

Some details:

$ docker version
Client: Docker Engine - Community
 Version:           24.0.6
 API version:       1.43
 Go version:        go1.20.7
 Git commit:        ed223bc
 Built:             Mon Sep  4 12:31:44 2023
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          24.0.6
  API version:      1.43 (minimum version 1.12)
  Go version:       go1.20.7
  Git commit:       1a79695
  Built:            Mon Sep  4 12:31:44 2023
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.24
  GitCommit:        61f9fd88f79f081d64d6fa3bb1a0dc71ec870523
 runc:
  Version:          1.1.9
  GitCommit:        v1.1.9-0-gccaecfc
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

$ docker compose version
Docker Compose version v2.21.0

$ cat ~/.docker/config.json 
{
    "auths": {
        [redacted]
    }
}

---

Below is the output for the same docker-compose.yml file, ran on another server. It has a correct format, but in the same time has a few differences (e.g. no Labels or RunningFor properties, Created vs. CreatedAt, etc.)

$ docker compose ps --format json
[{"ID":"e51334d871cba0ea4cdcd5003b9293b9de7ae01cef0cf95f1df3949f8762066f","Name":"docker_test_nginx_1","Image":"nginx","Command":"/docker-entrypoint.sh nginx -g 'daemon off;'","Project":"docker_test","Service":"nginx","Created":1695218753,"State":"running","Status":"Up 5 hours","Health":"","ExitCode":0,"Publishers":[{"URL":"","TargetPort":80,"PublishedPort":0,"Protocol":"tcp"}]},{"ID":"438c39a82b6c102eb1b4b0923696b8330a7a2fbcacacb645e7dff69589ea61be","Name":"docker_test_php_1","Image":"php:fpm","Command":"docker-php-entrypoint php-fpm","Project":"docker_test","Service":"php","Created":1695218749,"State":"running","Status":"Up 5 hours","Health":"","ExitCode":0,"Publishers":[{"URL":"","TargetPort":9000,"PublishedPort":0,"Protocol":"tcp"}]}]

Is there anything I should check to make it to produce a properly formatted JSON? Any help will be appreciated!

Answer 1

Unfortunately, this is an expected behavior and breaking change introduced in docker compose 2.21. We are currently dealing with the same issue..

See https://github.com/docker/compose/issues/10958 for more info.

In the same vein from boppy's answer, here's one possible solution that solves both docker compose ps --format json output (no matter which docker compose's version basically):

BEFORE:

docker compose ps -a --format json | jq <...any additional transfo, if any...>

AFTER (the fix):

docker compose ps -a --format json | jq -sc '.[] | if type=="array" then .[] else . end' | jq -s | jq <...any additional transfo, if any...>

For sure, this adds more processing, but in reality, I haven't seen too much impact so far, this will depends on the number of records that you are dealing with.

Answer 2

jq offers a switch to merge multi-input JSON feeds into an array. It's the -s (slurp) option:

docker compose ps --format json | jq -s .