How to access service running on host from inside Docker container in a cross platform way without using a bridge?

Question

I have read every answer to these two questions and more:

• From inside of a Docker container, how do I connect to the localhost of the machine?
• How to access host port from docker container

I have a CLI that runs some services natively on 127.0.0.1 on a variety of ports on the host system. It also spins up Postgres in a container and exposes it at 127.0.0.1:5432 on the host.

Once this is done, the CLI spins up a new container that connects to Postgres.

I want a single invocation for running this final container that works across all host platforms. In particular:

• MacOS (Docker Desktop)
• Windows (WSL) (Docker Desktop)
• Windows (native aka NT kernel) (Docker Desktop)
• Linux (Docker Desktop)

Based on the first question above, it seems like it should be possible to connect to host.docker.internal from inside the container on all platforms if I run the container like this. Assume the binary in the container knows where to connect to based on this ADDR env var:

docker run --add-host host.docker.internal:host-gateway -e ADDR=host.docker.internal:5432 my_image

The --add-host flag isn't necessary under the Docker Desktop environment but on Linux, this is necessary to let the container access the host. In reality, this only works on Linux with --network host. I can see from the logs that the name resolution works successfully (it tries to connect to 172.17.0.1 from within the container) but the connection fails without --network host, perhaps due to some permission issue (I run as root on Linux). Problem is, if I use --network host with Docker for Desktop, it fails to connect.

Assume I cannot solve this by creating a custom network, using bridge networking, and that I must bind all services both in and not in containers to 127.0.0.1.

Currently I'm solving this by having the CLI figure out if Docker Desktop is in use and changing the command accordingly:

• Without Docker for Desktop use --network host and connect to 127.0.0.1 from within the container.
• With Docker for Desktop do not use --network host and connect to host.docker.internal from within the container.

Issue is I'm finding the detection of whether the host is Docker for Desktop or not quite flaky: How to see if Docker daemon is Docker Desktop or not.

In all cases I'm using Docker version 24.0.7, build afdd53b.

Answer 1

You are fundamentally trying to connect from one container to another. This doesn't require anything special at the host level. You should not be using host.docker.internal or host networking to try to connect from one container to another.

The basic mechanism here is described in How to communicate between Docker containers via "hostname": if you create a Docker network and you make sure both containers are on the same network, then one container's name will be usable as a host name from the other container. This is fully portable across all host operating systems and Docker setups.

For example, this could look like

# Create a Docker network
docker network create a-network

# Start the database container
#   Attach it to a-network
#   Publish its port on the host's localhost interface
docker run \
  -d \
  --name a-database
  --net a-network \
  -p 127.0.0.1:5432:5432 \
  -v a-database-data:/var/lib/postgresql/data \
  postgres:15

# Start the application container
#   Attach it to a-network
#   Specify the database container's name as the database name
docker run \
  --net a-network \
  -e ADDR=a-database:5432
  my_image

The a-network, a-database, a-database-data names need to match each other within a particular invocation, but in the context of a CLI tool like you're discussing it could be reasonable to generate unique names for these per invocation.

This setup seems to meet your requirements: the database is accessible from outside Docker, but only on the same host; the application can connect to the database; the setup is independent of any particular Docker distribution.

Answer 2

I figured it out. I was running with this configuration:

• Postgres binds to 0.0.0.0 inside the container.
• I publish that to 127.0.0.1 on the host.

The solution was to publish to 0.0.0.0 on the host instead. In other words, when running the Postgres container, do this:

--publish 0.0.0.0:5432:5432

Instead of this:

--publish 127.0.0.1:5432:5432

Why this is necessary I'm not quite sure, but it works!