Why is consul health check returning an empty array?

Question

We have a docker-compose.yml file:

version: '3'
services:
  consul:
    container_name: consul
    command: agent -dev -config-dir=/consul/config
    dns: 8.8.8.8
    hostname: consul-docker
    image: docker-registry.mycompany.net/ap/consul-ent:0.9.2
    ports:
      - "8500:8500"
    volumes:
      - ./consul/config/conf.json:/consul/config/conf.json

# https://github.com/gliderlabs/registrator
  registrator:
    command: consul://consul:8500
    container_name: registrator
    depends_on:
      - consul
    image: gliderlabs/registrator:latest
    restart: always
    volumes:
      - /var/run/docker.sock:/tmp/docker.sock

  myservice1:
    container_name: myservice1
    image: docker-registry.mycompany.net/ap/myservice1/develop:latest
    ports:
      - "8080:8080"
      - "5000:5000"

# more services below

When I check the health status using consul v1 API by going to http://localhost:8500/v1/health/checks/myservice1 , it returns an empty array [].

According to docker ps and the logs of myservice1, myservice1 is Ready.

myservice1 shows green on http://localhost:8500/ui/#/dc1/services

myservice1 is a dropwizard based java microservice.

According to https://www.consul.io/api/health.html, sample response to

$ curl https://consul.rocks/v1/health/service/my-service

looks like:

[
  {
    "Node": {
      "ID": "40e4a748-2192-161a-0510-9bf59fe950b5",
      "Node": "foobar",
      "Address": "10.1.10.12",
      "Datacenter": "dc1",
      "TaggedAddresses": {
        "lan": "10.1.10.12",
        "wan": "10.1.10.12"
      },
      "Meta": {
        "instance_type": "t2.medium"
      }
    },
    "Service": {
      "ID": "redis",
      "Service": "redis",
      "Tags": ["primary"],
      "Address": "10.1.10.12",
      "Port": 8000
    },
    "Checks": [
      {
        "Node": "foobar",
        "CheckID": "service:redis",
        "Name": "Service 'redis' check",
        "Status": "passing",
        "Notes": "",
        "Output": "",
        "ServiceID": "redis",
        "ServiceName": "redis",
        "ServiceTags": ["primary"]
      },
      {
        "Node": "foobar",
        "CheckID": "serfHealth",
        "Name": "Serf Health Status",
        "Status": "passing",
        "Notes": "",
        "Output": "",
        "ServiceID": "",
        "ServiceName": "",
        "ServiceTags": []
      }
    ]
  }
]

According to registrator logs, it is seeing myservice1 fine and adding myservice1.

Any ideas on why I am seeing [] response would be greatly appreciated?

Answer 1

I was able to get basic health checks working after following the instructions at https://gliderlabs.com/registrator/latest/user/backends/. I was under the impression that health checks will work automatically after setting up registrator and consul. I needed to setup HTTP check by providing environment variables in docker-compose.yml like:

SERVICE_80_CHECK_HTTP=/health/endpoint/path
SERVICE_80_CHECK_INTERVAL=15s
SERVICE_80_CHECK_TIMEOUT=1s     # optional, Consul default used otherwise