Unable to get Docker Swarm on Windows Server 2019 ingress network working between containers

Question

I have found some posts mentioning the support for routing mesh using an overlay network on Windows Server 2019 (in references bellow).

After lots of troubleshooting, I am unable to properly configure 2 simple containers on a user defined overlay network created using the following network and services:

docker network create -d overlay --attachable testnet
docker service create -d --name web --network testnet --publish 80:80 microsoft/iis
docker service create -d --network testnet --name pingweb mcr.microsoft.com/windows/nanoserver:1809 ping web

I am able to reach the iis website when browsing my docker host on port 80, but my other container pingweb is unable to ping my main web container when they are on the same overlay network.

PS C:\Users\me> docker network ls
NETWORK ID          NAME                DRIVER              SCOPE
ga8egf2nwsir        ingress             overlay             swarm
bf164fa77349        nat                 nat                 local
81fb626259e1        none                null                local
l9p7c8p2fy3g        testnet             overlay             swarm
PS C:\Users\me> docker service create -d --name web --network testnet --publish 80:80 microsoft/iis
mk3r1a7za4jk21321kmzlddxr
PS C:\Users\me> docker service create -d --network testnet --name pingweb mcr.microsoft.com/windows/nanoserver:1809 ping web
j3z0xso7shghctva3od9qct10
PS C:\Users\me> docker service logs pingweb
pingweb.1.wbtpizulcxvg@WS2019DockerNode1    |
pingweb.1.wbtpizulcxvg@WS2019DockerNode1    | Pinging web [10.0.29.180] with 32 bytes of data:
pingweb.1.wbtpizulcxvg@WS2019DockerNode1    | Request timed out.
pingweb.1.wbtpizulcxvg@WS2019DockerNode1    | Request timed out.
pingweb.1.wbtpizulcxvg@WS2019DockerNode1    | Request timed out.
pingweb.1.wbtpizulcxvg@WS2019DockerNode1    | Request timed out.
pingweb.1.wbtpizulcxvg@WS2019DockerNode1    |
pingweb.1.wbtpizulcxvg@WS2019DockerNode1    | Ping statistics for 10.0.29.180:
pingweb.1.wbtpizulcxvg@WS2019DockerNode1    |     Packets: Sent = 4, Received = 0, Lost = 4 (100% loss),
PS C:\Users\me>

I also have noticed that I am unable to ping external sites whenever my pingweb container is on the overlay network. I've tested pinging 8.8.8.8, but it doesn't work when running on the overlay network as I am getting the same Request timed out as when I am trying to ping my web container on the testnet network.

docker service create -d --network testnet --name pingweb mcr.microsoft.com/windows/nanoserver:1809 ping 8.8.8.8

Question(s):

1. Is this a known issue?
2. How can I get this to work?

References:

https://learn.microsoft.com/en-us/virtualization/community/team-blog/2017/20170926-docker-s-routing-mesh-available-with-windows-server-version-1709

https://www.docker.com/blog/docker-windows-server-1709/

Docker ingress mode service publishing on Windows

Parity with Linux service publishing options has been highly requested by Windows customers. Adding support for service publishing using ingress mode in Windows Server 1709 enables use of Docker’s routing mesh, allowing external endpoints to access a service via any node in the swarm regardless of which nodes are running tasks for the service.

These networking improvements also unlock VIP-based service discovery when using overlay networks so that Windows users are not limited to DNS Round Robin.

Check out the corresponding post on the Microsoft Virtualization blog for details on the improvements.

Answer 1

After lots of struggle on this, turns out the fix was provided as part of the Windows Server 2019 Update KB4580390

Github thread around the issue: https://github.com/moby/moby/issues/40998#issuecomment-719889423

Update fixing the issue: https://www.catalog.update.microsoft.com/Search.aspx?q=KB4580390