Unable to delete network bridges on Fedora 33 & Docker

Question

I am using Docker 20.10.7 on Fedora 33:

$ uname -r
5.12.8-200.fc33.x86_64
$ docker -v
Docker version 20.10.7, build f0df350

When trying to create a new docker network, I get the following error:

$ docker network create --subnet 172.18.128.0/17 si
Error response from daemon: cannot create network 87c3cae81d6e379b4fec55a671fc26e547debc218d16b9e762435b7aa6f9ca28 (br-87c3cae81d6e): conflicts with network 8247c1fe85afa51c174529086ad8bc58069e8aac336cc99239b2cce21311ecb9 (br-8247c1fe85af): networks have overlapping IPv4

Although no docker networks apart from the defaults seem to exist:

$ docker network ls
NETWORK ID     NAME      DRIVER    SCOPE
93e55f691850   bridge    bridge    local
57062f01a833   host      host      local
3f81078d2755   none      null      local

It seems that some network bridges are present. These may be remnants of docker networks which have since been deleted:

$ brctl show
bridge name bridge id       STP enabled interfaces
br-0ccb5edb60fa     8000.02425e65824c   no      
br-0ebff3222c7d     8000.0242ad165003   no      
br-13b92f8a402d     8000.0242fd54405d   no      
br-3fe8819f1dab     8000.0242df196682   no      
br-62855eeb66b1     8000.024279b970d4   no      
br-64f88c232b42     8000.024294a697d3   no      
br-6cb52985982b     8000.0242fa683006   no      
br-7b91629f9dc5     8000.02429075dacd   no      
br-8247c1fe85af     8000.0242da4e1ba9   no      
br-9e6e08fae543     8000.0242a0c94c05   no      
br-b04024820e6b     8000.024251673cfb   no      
br-bb03dc88def2     8000.0242d550a519   no      
br-c3bfd0e7bfac     8000.02420fc70d06   no      
br-f446efca8607     8000.02423c5f7dff   no      
br-fc4085566463     8000.0242c5a5a9ed   no      
docker0     8000.0242090fcff7   no

So my idea was to delete these bridges, thinking that creating the new docker network should work after that:

$ sudo ip link del br-0ccb5edb60fa # do this for all bridges
$ brctl show
bridge name bridge id       STP enabled interfaces
docker0     8000.0242090fcff7   no

But alas, the error remains:

$ docker network create --subnet 172.18.128.0/17 si
Error response from daemon: cannot create network 60b9a9c6f1b032cf54ac799e5b8f2a96b1d55a05492e5357ffb6b002f10a27de (br-60b9a9c6f1b0): conflicts with network fc4085566463d57a641dbb3f5bba4888dbdf3908868f30d4e0c46edf19c001e4 (br-fc4085566463): networks have overlapping IPv4

I thought restarting the docker service might solve the problem

$ sudo systemctl restart docker

But after restarting the docker service, the bridges reappear!

$ brctl show
bridge name bridge id       STP enabled interfaces
br-0ccb5edb60fa     8000.0242ba61c513   no      
br-0ebff3222c7d     8000.0242ae82887a   no      
br-13b92f8a402d     8000.02421fa70d86   no      
br-3fe8819f1dab     8000.0242a7c0d3cd   no      
br-62855eeb66b1     8000.02423b35c43d   no      
br-64f88c232b42     8000.0242e3235138   no      
br-6cb52985982b     8000.024264333192   no      
br-7b91629f9dc5     8000.02420c574876   no      
br-8247c1fe85af     8000.02423b3845ee   no      
br-9e6e08fae543     8000.024285a4da86   no      
br-b04024820e6b     8000.02425a16a19f   no      
br-bb03dc88def2     8000.02422c2da231   no      
br-c3bfd0e7bfac     8000.024250df5e93   no      
br-f446efca8607     8000.024234091377   no      
br-fc4085566463     8000.0242eda476ce   no      
docker0     8000.0242090fcff7   no

Other things I have tried, but which did not help:

$ docker network prune -f
$ docker system prune -f
# tried the following commands for all existing bridges
$ nmcli connection delete br-0ccb5edb60fa
$ sudo brctl delbr br-0ccb5edb60fa
$ sudo firewall-cmd --zone=docker --remove-interface=br-0ccb5edb60fa --permanent
Warning: NOT_ENABLED: br-0ccb5edb60fa
success

What may be preventing me from reliably deleting network bridges?

Answer 1

I figured out why docker kept recreating brigdes. They seem to be stored in /var/lib/docker/network/files/local-kv.db, a binary file (BoltDB), which explains why I didn't catch it with a grep.

Deleting the file and restarting docker solved my problem. The following commands (executed in zsh, other shells may be different) delete all bridges, the aforementioned file and restart the docker service. After that, I am able to create the docker network without errors.

while read; do
  if [ ! -z "$REPLY" ]; then
    sudo ip link del $REPLY
  fi
done <<<"$(brctl show | grep '^br-' | awk '{print $1}')"
sudo rm /var/lib/docker/network/files/local-kv.db
sudo systemctl restart docker