Xdebug in docker container fails to connect to PhpStorm on Qubes OS

Question

Xdebug within a Docker container does not connect to PhpStorm on my system.

I am trying to setup Xdebug with PhpStorm for a Docker environment on Linux (qubes-os / Fedora 30). Xdebug is enabled, and I can access error messages. In PhpStorm I always updated DBGp Proxy setting with the respective IPs I gave Xdebug as remote host. I tried many versions of Xdebug setups but all failed.

My current best guess is that something with the internal IP management is messed up. This could be due to qubes-os, but I'm not really convinced since it's a normal Fedora and I have never had issues like that before...

My Xdebug conf

zend_extension=xdebug.so

[Xdebug]
xdebug.idekey=PHPSTORM
xdebug.remote_enable=true
xdebug.remote_port=5902
xdebug.remote_host=host.docker.internal
xdebug.remote_log=/tmp/xdebug-remote.log

(I'm aware that host.docker.internal does not work for Linux. I'm using it anyway for easier debugging by setting an IP to this variable in the /etc/hosts file of the docker container)

My phpinfo()

xdebug support  enabled
Version 2.6.1
IDE Key PHPSTORM
xdebug.remote_enable    On  On
xdebug.remote_handler   dbgp    dbgp
xdebug.remote_host  host.docker.internal    host.docker.internal
xdebug.remote_log   /tmp/xdebug-remote.log  /tmp/xdebug-remote.log
xdebug.remote_mode  req req
xdebug.remote_port  5902    5902
xdebug.remote_timeout   200 200

My web-log tells me, that my requests are coming from 172.18.0.1 - - [31/Oct/2019:09:58:22 +0000] "GET / HTTP/1.1" 200 47698 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.70 Safari/537.36"

my ifconfig output host machine

br-8d5002ad7a3a: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 172.18.0.1  netmask 255.255.0.0  broadcast 172.18.255.255
        inet6 fe80::42:17ff:feaa:e865  prefixlen 64  scopeid 0x20<link>
        ether 02:42:17:aa:e8:65  txqueuelen 0  (Ethernet)
        RX packets 5  bytes 513 (513.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 11  bytes 866 (866.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

docker0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet 172.17.0.1  netmask 255.255.0.0  broadcast 172.17.255.255
        inet6 fe80::42:99ff:fe38:e669  prefixlen 64  scopeid 0x20<link>
        ether 02:42:99:38:e6:69  txqueuelen 0  (Ethernet)
        RX packets 4055  bytes 233615 (228.1 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 4367  bytes 55073512 (52.5 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.137.0.17  netmask 255.255.255.255  broadcast 10.255.255.255
        inet6 fe80::216:3eff:fe5e:6c00  prefixlen 64  scopeid 0x20<link>
        ether 00:16:3e:5e:6c:00  txqueuelen 1000  (Ethernet)
        RX packets 555370  bytes 785064402 (748.6 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 208464  bytes 13235820 (12.6 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 107  bytes 227427 (222.0 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 107  bytes 227427 (222.0 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

veth0271483: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet6 fe80::183d:fdff:fe2b:f8ce  prefixlen 64  scopeid 0x20<link>
        ether 1a:3d:fd:2b:f8:ce  txqueuelen 0  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 16  bytes 1379 (1.3 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

veth25193ce: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet6 fe80::603c:beff:fe87:6283  prefixlen 64  scopeid 0x20<link>
        ether 62:3c:be:87:62:83  txqueuelen 0  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 16  bytes 1379 (1.3 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

vetha36c6d7: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet6 fe80::789d:60ff:fe15:8eb4  prefixlen 64  scopeid 0x20<link>
        ether 7a:9d:60:15:8e:b4  txqueuelen 0  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 16  bytes 1379 (1.3 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

vethc039300: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet6 fe80::a0a9:4eff:fe3d:8338  prefixlen 64  scopeid 0x20<link>
        ether a2:a9:4e:3d:83:38  txqueuelen 0  (Ethernet)
        RX packets 5  bytes 513 (513.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 11  bytes 866 (866.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

vethe777af4: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet6 fe80::c07e:11ff:fe1a:9f6b  prefixlen 64  scopeid 0x20<link>
        ether c2:7e:11:1a:9f:6b  txqueuelen 0  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 16  bytes 1379 (1.3 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

my ifconfig output on the docker container

eth0      Link encap:Ethernet  HWaddr 02:42:AC:12:00:05  
          inet addr:172.18.0.5  Bcast:172.18.255.255  Mask:255.255.0.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:14 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:1260 (1.2 KiB)  TX bytes:0 (0.0 B)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

My netstat -ltn from my host machine

Proto Recv-Q Send-Q Local Address           Foreign Address         State      
tcp        0      0 127.0.0.1:631           0.0.0.0:*               LISTEN     
tcp        0      0 0.0.0.0:10137           0.0.0.0:*               LISTEN     
tcp        0      0 127.0.0.1:6942          0.0.0.0:*               LISTEN     
tcp        0      0 0.0.0.0:5902            0.0.0.0:*               LISTEN     
tcp        0      0 127.0.0.1:63342         0.0.0.0:*               LISTEN     
tcp        0      0 0.0.0.0:20080           0.0.0.0:*               LISTEN     
tcp6       0      0 ::1:631                 :::*                    LISTEN     
tcp6       0      0 :::9000                 :::*                    LISTEN     
tcp6       0      0 :::3306                 :::*                    LISTEN     
tcp6       0      0 :::80                   :::*                    LISTEN     
tcp6       0      0 :::81                   :::*                    LISTEN     
tcp6       0      0 :::8082                 :::*                    LISTEN    

My netstat -ltn from the docker container

Proto Recv-Q Send-Q Local Address           Foreign Address         State       
tcp        0      0 127.0.0.11:34183        0.0.0.0:*               LISTEN      
tcp        0      0 127.0.0.1:9000          0.0.0.0:*               LISTEN      
tcp        0      0 0.0.0.0:9001            0.0.0.0:*               LISTEN    

When using xdebug.remote_connect_back=1 I get

I: Checking remote connect back address.
I: Checking header 'HTTP_X_FORWARDED_FOR'.
I: Checking header 'REMOTE_ADDR'.
I: Remote address found, connecting to 172.18.0.1:5902.
E: Time-out connecting to client (Waited: 200 ms). :-(
Log closed at 2019-10-31 09:32:55

Also when I run netstat -nr | grep '^0\.0\.0\.0' | awk '{print $2}' within the docker container I get 172.18.0.1

Thus I would conclude that xdebug.remote_host = 172.18.0.1

But when I use the IP added by docker to the container's /etc/hosts (the IP changes to 172.17-18.0.1-4, right now it's 172.18.0.4) and look into the Xdebug logs I get

W: Creating socket for 'host.docker.internal:5902', poll success, but error: Operation in progress (29).
E: Could not connect to client. :-(

So since these IPs were somewhat inconclusive, I simply tried every IP that I encountered on my way as my xdebug.remot_host for the docker container. And I either one of the above failure logs from xdebug

Additionally the results of telnet and ping:

telnet 172.18.0.1 5902:
telnet: can't connect to remote host (172.18.0.1): Operation timed out
bash-4.4# telnet 172.18.0.4
telnet: can't connect to remote host (172.18.0.4): Connection refused

bash-4.4# ping 172.18.0.1:
5 packets transmitted, 0 packets received, 100% packet loss

bash-4.4# ping 172.18.0.4
PING 172.18.0.4 (172.18.0.4): 56 data bytes
5 packets transmitted, 5 packets received, 0% packet loss

Conclusion: I'm lost. I basically tried every possible IP address. Please help me understand what I need to do in order to debug my PHP code. Thanks!

Answer 1

If you are on a local machine and have a docker container there it's enough to set xdebug.remote_host=host.docker.internal and add host.docker.internal to /etc/hosts.

I do this in the entry sh script: netstat -nr | grep '^0\.0\.0\.0' | awk '{print $2" host.docker.internal"}' >> /etc/hosts

How to check that data is sent:

1. you can enable xdebug.remote_log=/var/log/xdebug_remote_log.log
2. I prefer to listen to ports on a host nc -l 5902 or nc -l 0.0.0.0 5902. Send text via telnet from docker telnet host.docker.internal 5902 and type something. You should see it in nc on the host

If your docker is on a remote host then you have to allow ssh GatewayPorts yes to listen 0.0.0.0:5904 or forward traffic to 127.0.0.0:5905. Look here

1. on the remote host run once: socat TCP-LISTEN:5904,fork TCP:127.0.0.1:5905
2. to get response on the local machine run: ssh -R 5905:localhost:5904

You can check the connection with nc and telnet.

Answer 2

TLDR: Specify in your docker container xdebug.remote_connect_back=0 and xdebug.remote_host=172.17.0.1 and it should work. Remember: Xdebug needs to connect from your webserver (inside Docker) to your IDE listening on port 9000.

You didn't show Docker's ifconfig, so I might have gotten the IP address above wrong. But, the important thing is that the IP address you specify in xdebug.remote_host is the one where your IDE listens. And that IP address needs to be reachable from Docker. You can test that by running "telnet IpAddress 9000" from within your Docker container, to see whether you got the right IP addess of your host.

But some other points:

On your docker container, your netstat shows:

tcp        0      0 127.0.0.1:9000          0.0.0.0:*               LISTEN      
tcp        0      0 0.0.0.0:9001            0.0.0.0:*               LISTEN 

Do you have a DBGp Proxy running there? You shouldn't need to do that. The proxy only makes things more complicated for your situation.

I see in your host's netstat:

tcp6       0      0 :::9000                 :::*                    LISTEN     

Are you using an IDE that only listens on IPv6? You might want to see if you can change that to run with IPv4.