You are not logged in.

#1 2021-05-13 19:02:20

Geonai
Member
Registered: 2021-03-25
Posts: 4

Docker Networking not working

Hi,

I've been wrestling with a Docker issue for over a month now, seems that containers can't communicate between each other on a docker network.

I've verified that the docker files and docker-compose work by loading them onto a windows machine and then firing a request (Specifically C# api container to maria-db container) which gives back a result from the db. This error seems to be similar behaviour as to https://bbs.archlinux.org/viewtopic.php?id=200000 however the fixes here don't appear to work for me.

If on my arch setup I set network-mode=host in the docker-compose file then the containers can communicate via the localhost. I've also loaded a networking container (nicolaka/netshoot) and verified that the docker DNS is working as pinging the container name outputs the same ip address as from

docker inspect --format='{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' <container-name> 

however all packets are lost indicating no network traffic between the containers.

networkctl list outputs the following where IDX 52, 54 and 58 are from the containers generated by my docker-compose

IDX LINK            TYPE     OPERATIONAL SETUP
  1 lo              loopback carrier     unmanaged
  2 eno1            ether    routable    unmanaged
  5 docker0         bridge   no-carrier  unmanaged
 52 br-f0003a24549b bridge   routable    unmanaged
 54 vethe3d5310     ether    degraded    unmanaged
 58 vethafab2a7     ether    degraded    unmanaged

6 links listed.

I've also taken a look at https://wiki.archlinux.org/title/docker … d-networkd but it seems my port forwarding is all set to 1 (unless the mc_forwarding and bc_forwarding also needs to be 1)

sysctl -a | grep forward
sysctl: permission denied on key 'fs.protected_fifos'
sysctl: permission denied on key 'fs.protected_hardlinks'
sysctl: permission denied on key 'fs.protected_regular'
sysctl: permission denied on key 'fs.protected_symlinks'
sysctl: permission denied on key 'kernel.cad_pid'
sysctl: permission denied on key 'kernel.usermodehelper.bset'
sysctl: permission denied on key 'kernel.usermodehelper.inheritable'
sysctl: permission denied on key 'net.core.bpf_jit_harden'
sysctl: permission denied on key 'net.core.bpf_jit_kallsyms'
sysctl: permission denied on key 'net.core.bpf_jit_limit'
net.ipv4.conf.all.bc_forwarding = 0
net.ipv4.conf.all.forwarding = 1
net.ipv4.conf.all.mc_forwarding = 0
net.ipv4.conf.br-f0003a24549b.bc_forwarding = 0
net.ipv4.conf.br-f0003a24549b.forwarding = 1
net.ipv4.conf.br-f0003a24549b.mc_forwarding = 0
net.ipv4.conf.default.bc_forwarding = 0
net.ipv4.conf.default.forwarding = 1
net.ipv4.conf.default.mc_forwarding = 0
net.ipv4.conf.docker0.bc_forwarding = 0
net.ipv4.conf.docker0.forwarding = 1
net.ipv4.conf.docker0.mc_forwarding = 0
net.ipv4.conf.eno1.bc_forwarding = 0
net.ipv4.conf.eno1.forwarding = 1
net.ipv4.conf.eno1.mc_forwarding = 0
net.ipv4.conf.lo.bc_forwarding = 0
net.ipv4.conf.lo.forwarding = 1
net.ipv4.conf.lo.mc_forwarding = 0
net.ipv4.conf.vethafab2a7.bc_forwarding = 0
net.ipv4.conf.vethafab2a7.forwarding = 1
net.ipv4.conf.vethafab2a7.mc_forwarding = 0
net.ipv4.conf.vethe3d5310.bc_forwarding = 0
net.ipv4.conf.vethe3d5310.forwarding = 1
net.ipv4.conf.vethe3d5310.mc_forwarding = 0
net.ipv4.ip_forward = 1
net.ipv4.ip_forward_update_priority = 1
net.ipv4.ip_forward_use_pmtu = 0
sysctl: permission denied on key 'net.ipv4.tcp_fastopen_key'
sysctl: permission denied on key 'net.ipv6.conf.all.stable_secret'
net.ipv6.conf.all.forwarding = 0
net.ipv6.conf.all.mc_forwarding = 0
sysctl: permission denied on key 'net.ipv6.conf.br-f0003a24549b.stable_secret'
net.ipv6.conf.br-f0003a24549b.forwarding = 0
net.ipv6.conf.br-f0003a24549b.mc_forwarding = 0
sysctl: permission denied on key 'net.ipv6.conf.default.stable_secret'
net.ipv6.conf.default.forwarding = 0
net.ipv6.conf.default.mc_forwarding = 0
sysctl: permission denied on key 'net.ipv6.conf.docker0.stable_secret'
net.ipv6.conf.docker0.forwarding = 0
net.ipv6.conf.docker0.mc_forwarding = 0
sysctl: permission denied on key 'net.ipv6.conf.eno1.stable_secret'
net.ipv6.conf.eno1.forwarding = 0
net.ipv6.conf.eno1.mc_forwarding = 0
sysctl: permission denied on key 'net.ipv6.conf.lo.stable_secret'
net.ipv6.conf.lo.forwarding = 0
net.ipv6.conf.lo.mc_forwarding = 0
sysctl: permission denied on key 'net.ipv6.conf.vethafab2a7.stable_secret'
net.ipv6.conf.vethafab2a7.forwarding = 0
net.ipv6.conf.vethafab2a7.mc_forwarding = 0
sysctl: permission denied on key 'net.ipv6.conf.vethe3d5310.stable_secret'
net.ipv6.conf.vethe3d5310.forwarding = 0
net.ipv6.conf.vethe3d5310.mc_forwarding = 0
sysctl: permission denied on key 'vm.mmap_rnd_bits'
sysctl: permission denied on key 'vm.mmap_rnd_compat_bits'
sysctl: permission denied on key 'vm.stat_refresh'

ip addr gives

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 18:c0:4d:48:30:52 brd ff:ff:ff:ff:ff:ff
    altname enp3s0
    inet 192.168.1.103/24 brd 192.168.1.255 scope global dynamic noprefixroute eno1
       valid_lft 82805sec preferred_lft 72005sec
    inet6 2a01:4b00:a85f:df00:a233:833b:b2f6:4c55/64 scope global dynamic mngtmpaddr noprefixroute
       valid_lft 53040sec preferred_lft 38640sec
    inet6 fd00::ef7c:e14d:6607:14ff/64 scope global mngtmpaddr noprefixroute
       valid_lft forever preferred_lft forever
    inet6 fe80::ce1f:11f5:4742:38c9/64 scope link
       valid_lft forever preferred_lft forever
5: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
    link/ether 02:42:94:a4:c1:16 brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
       valid_lft forever preferred_lft forever
    inet6 fe80::42:94ff:fea4:c116/64 scope link
       valid_lft forever preferred_lft forever
52: br-f0003a24549b: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
    link/ether 02:42:7d:86:66:3e brd ff:ff:ff:ff:ff:ff
    inet 172.20.0.1/16 brd 172.20.255.255 scope global br-f0003a24549b
       valid_lft forever preferred_lft forever
    inet6 fe80::42:7dff:fe86:663e/64 scope link
       valid_lft forever preferred_lft forever
54: vethe3d5310@if53: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br-f0003a24549b state UP group default
    link/ether 32:a3:ea:9f:5c:d6 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet6 fe80::1028:a578:8d6f:3e30/64 scope link
       valid_lft forever preferred_lft forever
    inet6 fe80::30a3:eaff:fe9f:5cd6/64 scope link
       valid_lft forever preferred_lft forever
58: vethafab2a7@if57: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br-f0003a24549b state UP group default
    link/ether 32:6a:6c:b6:1a:35 brd ff:ff:ff:ff:ff:ff link-netnsid 1
    inet6 fe80::e0d7:43b9:c34a:ae38/64 scope link
       valid_lft forever preferred_lft forever
    inet6 fe80::306a:6cff:feb6:1a35/64 scope link
       valid_lft forever preferred_lft forever

Apologies if I've left out any useful information, my networking knowledge isn't the strongest!

Thanks for any help!

Offline

#2 2021-05-15 12:20:31

Geonai
Member
Registered: 2021-03-25
Posts: 4

Re: Docker Networking not working

So I seem to have resolved the issue, though not entirely sure what step was the fix. For reference in case it helps anyone the steps I performed were:

I followed the Official Docker installation for Binaries (https://docs.docker.com/engine/install/binaries/) and noticed I didn't have iptables installed so installed there.

I followed the trouble shooting from https://wiki.archlinux.org/title/docker … d-networkd primarily following:

nft flush chain inet filter forward

I also followed https://wiki.archlinux.org/title/docker … networking namely adding the below to /etc/docker/daemon.json

{
  "iptables": false
}

I then ran the following restarts:

systemctl restart systemd-networkd
systemctl restart iptables
systemctl restart docker

Thanks,

Offline

Board footer

Powered by FluxBB