I have figured out a way to start gdb inside a flatpak in a single command (instead of three steps), but it involves doing something strange to work around a quirk or possibly bug in gdb or in flatpak. Can someone explain to me why flatpak gets "Stopped" and why running sh with job control enabled inside the flatpak prevents it? I think there is some odd interaction here between how flatpak, sh and gdb handle signals, but I am not sure.
I have installed the "debug sdk" for Minetest from flathub:
flatpak install org.freedesktop.Sdk.Debug/x86_64/23.08
I want to run gdb inside the flatpak. This works as expected, but takes three steps (flatpak, gdb, run):
flatpak run --command=sh --devel net.minetest.Minetest
[ net.minetest.Minetest ~]$ gdb /bin/sleep
GNU gdb (GDB) 13.2
Copyright (C) 2023 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-unknown-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /bin/sleep...
Reading symbols from /usr/lib/debug//usr/bin/sleep.debug...
(gdb) run 30
Starting program: /usr/bin/sleep 30
This GDB supports auto-downloading debuginfo from the following URLs:
<https://debuginfod.fedoraproject.org/>
Enable debuginfod for this session? (y or [n])
Debuginfod has been disabled.
To make this setting permanent, add 'set debuginfod enabled off' to .gdbinit.
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/usr/lib/x86_64-linux-gnu/libthread_db.so.1".
[Inferior 1 (process 9) exited normally]
(gdb)
This on the other hand immediately says [1]+ Stopped without waiting 30 seconds
flatpak run --command=gdb --devel net.minetest.Minetest -ex run --args /bin/sleep 30
...
Reading symbols from /bin/sleep...
Reading symbols from /usr/lib/debug//usr/bin/sleep.debug...
Starting program: /usr/bin/sleep 30
This GDB supports auto-downloading debuginfo from the following URLs:
<https://debuginfod.fedoraproject.org/>
[1]+ Stopped flatpak run --command=gdb --devel net.minetest.Minetest -ex run --args /bin/sleep 30
jobs -l
[1]+ 59486 Stopped (tty output) flatpak run --command=gdb --devel net.minetest.Minetest -ex run --args /bin/sleep 30
I thought maybe gdb must be started via sh to work correctly, but nope - same result (note: I restarted the shell to get rid of stopped jobs):
flatpak run --command=sh --devel net.minetest.Minetest -c 'gdb -ex run --args /bin/sleep 30'
...
Reading symbols from /bin/sleep...
Reading symbols from /usr/lib/debug//usr/bin/sleep.debug...
Starting program: /usr/bin/sleep 30
This GDB supports auto-downloading debuginfo from the following URLs:
<https://debuginfod.fedoraproject.org/>
[1]+ Stopped flatpak run --command=sh --devel net.minetest.Minetest -c 'gdb -ex run --args /bin/sleep 30'
jobs -l
[1]+ 59664 Stopped (tty output) flatpak run --command=sh --devel net.minetest.Minetest -c 'gdb -ex run --args /bin/sleep 30'
pstree reveals something interesting: there is no sh in the process tree:
pstree $$
bash─┬─bwrap───bwrap───gdb─┬─sleep
│ └─4*[{gdb}]
└─pstree
OK. Easy fix sh -c '<command>; sleep 0'. Still broken, even though sh is in the process hierachy:
flatpak run --command=sh --devel net.minetest.Minetest -c 'gdb -ex run --args /bin/sleep 30; sleep 0'
...
Reading symbols from /bin/sleep...
Reading symbols from /usr/lib/debug//usr/bin/sleep.debug...
Starting program: /usr/bin/sleep 30
This GDB supports auto-downloading debuginfo from the following URLs:
<https://debuginfod.fedoraproject.org/>
[1]+ Stopped flatpak run --command=sh --devel net.minetest.Minetest -c 'gdb -ex run --args /bin/sleep 30; sleep 0'
pstree $$
bash─┬─bwrap───bwrap───sh───gdb─┬─sleep
│ └─4*[{gdb}]
└─pstree
jobs -l
[1]+ 59738 Stopped (tty output) flatpak run --command=sh --devel net.minetest.Minetest -c 'gdb -ex run --args /bin/sleep 30; sleep 0'
On a hunch I tried enabling job control and this fixed the problem:
flatpak run --command=sh --devel net.minetest.Minetest -c 'set -m; gdb -ex run --args /bin/sleep 30; sleep 0'
...
Reading symbols from /bin/sleep...
Reading symbols from /usr/lib/debug//usr/bin/sleep.debug...
Starting program: /usr/bin/sleep 30
This GDB supports auto-downloading debuginfo from the following URLs:
<https://debuginfod.fedoraproject.org/>
Enable debuginfod for this session? (y or [n])
Debuginfod has been disabled.
To make this setting permanent, add 'set debuginfod enabled off' to .gdbinit.
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/usr/lib/x86_64-linux-gnu/libthread_db.so.1".
[Inferior 1 (process 9) exited normally]
(gdb)
Note: job control does appear to be relevant somehow because the following command gets "Stopped" same as examples above:
flatpak run --command=sh --devel net.minetest.Minetest -c 'set -m; set +m; gdb -ex run --args /bin/sleep 30; sleep 0'
Note that gdb inside a flatpak reads input and responds to commands just fine until I tell it to run the program I am debugging:
flatpak run --command=gdb --devel net.minetest.Minetest --args /bin/sleep 30
...
Reading symbols from /bin/sleep...
Reading symbols from /usr/lib/debug//usr/bin/sleep.debug...
(gdb) info os
Type Description
cpus Listing of all cpus/cores on the system
files Listing of all file descriptors
modules Listing of all loaded kernel modules
msg Listing of all message queues
processes Listing of all processes
procgroups Listing of all process groups
semaphores Listing of all semaphores
shm Listing of all shared-memory regions
sockets Listing of all internet-domain sockets
threads Listing of all threads
(gdb) run
Starting program: /usr/bin/sleep 30
This GDB supports auto-downloading debuginfo from the following URLs:
<https://debuginfod.fedoraproject.org/>
[1]+ Stopped flatpak run --command=gdb --devel net.minetest.Minetest --args /bin/sleep 30
[1]+ 59879 Stopped (tty output) flatpak run --command=gdb --devel net.minetest.Minetest --args /bin/sleep 30
Why do I get "[1]+ Stopped"? What is happening? Why does set -m inside sh inside flatpak fix this?
cat /etc/fedora-release
Fedora release 38 (Thirty Eight)
flatpak --version
Flatpak 1.15.4
flatpak run --command=sh --devel net.minetest.Minetest --version | head --lines=1
GNU bash, version 5.2.21(1)-release (x86_64-unknown-linux-gnu)
flatpak run --command=gdb --devel net.minetest.Minetest --version | head --lines=1
GNU gdb (GDB) 13.2
flatpak info net.minetest.Minetest | grep Version
Version: 5.8.0
Edit: with the help of jobs -l (thanks Mark Plotnick) I got closer to understanding this. Apparently this is related to SIGTTOU, but I checked and tostop is always disabled:
flatpak run --command=/bin/stty --devel net.minetest.Minetest -a | tr ' ' '\n' | grep tostop
-tostop
flatpak run --command=sh --devel net.minetest.Minetest -c 'stty -a' | tr ' ' '\n' | grep tostop
-tostop
flatpak run --command=sh --devel net.minetest.Minetest -c 'set -m; stty -a' | tr ' ' '\n' | grep tostop
-tostop
Does this mean that something enables tostop when I tell gdb to start the program I am debugging? How does my workaround change this?