[SOLVED] No network on Docker Container

KuriyamaMirai · 2017-12-15 11:34:47

I'm new to Docker and just intalled it on Arch, but when I run image, there are no network connection.

sudo docker run -it archlinux/base pacman -Syu
:: Synchronizing package databases...
error: failed retrieving file 'core.db' from mirror.leaseweb.net : Resolving timed out after 10545 milliseconds
error: failed to update core (download library error)

I'm using systemd-networkd and wpa_supplicant to connect to wifi.
I also tried suggested solution on wiki: https://wiki.archlinux.org/index.php/Do … containers but it didnot work

$ cat /etc/systemd/network/wlp3s0.network 
[Match]
Name=wlp3s0

[Network]
DHCP=yes
IPForward=kernel

$ sysctl -a | grep forward
net.ipv4.conf.all.forwarding = 1
net.ipv4.conf.all.mc_forwarding = 0
net.ipv4.conf.default.forwarding = 1
net.ipv4.conf.default.mc_forwarding = 0
net.ipv4.conf.docker0.forwarding = 1
net.ipv4.conf.docker0.mc_forwarding = 0
net.ipv4.conf.enp4s0f2.forwarding = 1
net.ipv4.conf.enp4s0f2.mc_forwarding = 0
net.ipv4.conf.lo.forwarding = 1
net.ipv4.conf.lo.mc_forwarding = 0
net.ipv4.conf.wlp3s0.forwarding = 1
net.ipv4.conf.wlp3s0.mc_forwarding = 0
net.ipv4.ip_forward = 1
net.ipv4.ip_forward_use_pmtu = 0
sysctl: permission denied on key 'net.ipv4.tcp_fastopen_key'
sysctl: permission denied on key 'net.ipv6.conf.all.stable_secret'
net.ipv6.conf.all.forwarding = 0
net.ipv6.conf.all.mc_forwarding = 0
sysctl: permission denied on key 'net.ipv6.conf.default.stable_secret'
net.ipv6.conf.default.forwarding = 0
net.ipv6.conf.default.mc_forwarding = 0
sysctl: permission denied on key 'net.ipv6.conf.docker0.stable_secret'
net.ipv6.conf.docker0.forwarding = 0
net.ipv6.conf.docker0.mc_forwarding = 0
sysctl: permission denied on key 'net.ipv6.conf.enp4s0f2.stable_secret'
net.ipv6.conf.enp4s0f2.forwarding = 0
net.ipv6.conf.enp4s0f2.mc_forwarding = 0
sysctl: permission denied on key 'net.ipv6.conf.lo.stable_secret'
net.ipv6.conf.lo.forwarding = 0
net.ipv6.conf.lo.mc_forwarding = 0
sysctl: permission denied on key 'net.ipv6.conf.wlp3s0.stable_secret'
net.ipv6.conf.wlp3s0.forwarding = 0
net.ipv6.conf.wlp3s0.mc_forwarding = 0

Tried re-installing, restarting docker.service, systemd-networkd.service, re-pulling image, ubuntu image but noting worked

I found this post: https://bbs.archlinux.org/viewtopic.php?id=200000 and run --net=host worked, but post owner's solution did not work

$ sudo docker run -it --net=host archlinux/base pacman -Syu --> This worked
:: Synchronizing package databases...
 core                         126.1 KiB   251K/s 00:01 [############################] 100%

$ cat /etc/sysctl.d/30-ipforward.conf
net.ipv4.ip_forward=1

Here is my iptables and other details:

$ pacman -Q systemd docker
systemd 235.38-4
docker 1:17.11.0-1

$ sudo iptables -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         

Chain FORWARD (policy DROP)
target     prot opt source               destination         
DOCKER-USER  all  --  anywhere             anywhere            
DOCKER-ISOLATION  all  --  anywhere             anywhere            
ACCEPT     all  --  anywhere             anywhere             ctstate RELATED,ESTABLISHED
DOCKER     all  --  anywhere             anywhere            
ACCEPT     all  --  anywhere             anywhere            
ACCEPT     all  --  anywhere             anywhere            

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination         

Chain DOCKER (1 references)
target     prot opt source               destination         

Chain DOCKER-ISOLATION (1 references)
target     prot opt source               destination         
RETURN     all  --  anywhere             anywhere            

Chain DOCKER-USER (1 references)
target     prot opt source               destination         
RETURN     all  --  anywhere             anywhere 

$ cat /etc/systemd/system/multi-user.target.wants/docker.service
...
[Service]
Type=notify
# the default is not to use systemd for cgroups because the delegate issues still
# exists and systemd currently does not support the cgroup feature set required
# for containers run by docker
ExecStart=/usr/bin/dockerd -H fd://
ExecReload=/bin/kill -s HUP $MAINPID

Last edited by KuriyamaMirai (2018-01-02 02:08:51)

firecat53 · 2017-12-15 15:21:05

I'm not sure if this is your issue, but check to see if your host machine IP address falls in the same range as the Docker bridge IP address. I have to drop in a systemd service file override to set '--bip=' on the Docker daemon to ensure the address spaces don't collide while I'm at work.

You can also drop into a shell in the container and check things like 'ip route' and 'ip addr' to see what's going on in there.

KuriyamaMirai · 2017-12-15 17:23:51

firecat53 wrote:

I'm not sure if this is your issue, but check to see if your host machine IP address falls in the same range as the Docker bridge IP address. I have to drop in a systemd service file override to set '--bip=' on the Docker daemon to ensure the address spaces don't collide while I'm at work.
You can also drop into a shell in the container and check things like 'ip route' and 'ip addr' to see what's going on in there.

This is my 'ip route' and 'ip addr' (I run docker with: docker run -v /myfolder:/folder -it --net=host archlinux/base /bin/bash)
wlp3s0 inet different with docker0 inet

$ ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: enp4s0f2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
3: wlp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
4: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default 

$ ip route
default via 192.x.x.1 dev wlp3s0 proto dhcp src 192.x.x.103 metric 1024 
172.x.x.x/16 dev docker0 proto kernel scope link src 172.x.x.x linkdown 
192.x.x.0/24 dev wlp3s0 proto kernel scope link src 192.x.x.103 
192.x.x.1 dev wlp3s0 proto dhcp scope link src 192.x.x.103 metric 1024

Without --net=host option, I can't connect to network so unable to run pacman -Syu base so 'ip: command not found'

Last edited by KuriyamaMirai (2018-01-02 02:14:34)

firecat53 · 2017-12-15 17:44:33

Try with just a stock Alpine or Ubuntu container and see if you get networking without net=host.

KuriyamaMirai · 2017-12-15 18:21:43

firecat53 wrote:

Try with just a stock Alpine or Ubuntu container and see if you get networking without net=host.

No, I tried with ubuntu and it is same as archlinux/base, it would not work if I did not add --net=host
I also tried re-installing docker, restarting docker.service, systemd-networkd.service but nothing worked

Last edited by KuriyamaMirai (2017-12-15 18:22:49)

firecat53 · 2017-12-15 18:28:26

Check your host ip address with 'ip addr', then use alpine or docker to check the network inside a container

docker run alpine ip a
docker run alpine ip r

Also, are you on a VPN or have anything else going on with networking? Bridges, hypervisors, etc?

Edit (again, sorry): also, take a look at 'journalctl -u docker.service' for any other clues/errors

Last edited by firecat53 (2017-12-15 18:32:53)

KuriyamaMirai · 2017-12-15 18:40:53

firecat53 wrote:

Check your host ip address with 'ip addr', then use alpine or docker to check the network inside a container
docker run alpine ip a
docker run alpine ip r
Also, are you on a VPN or have anything else going on with networking? Bridges, hypervisors, etc?
Edit (again, sorry): also, take a look at 'journalctl -u docker.service' for any other clues/errors

No, I'm not using any VPN, I just used google dns and systemd-networkd, wpa_supplicant to connect wifi
This is journalctl -u docker.service , look like there are any errors

16/12

Dec 16 01:19:58 Mirai dockerd[483]: time="2017-12-16T01:19:58+07:00" level=info msg="shim docker-containerd-shim started" address="/containerd-shim/moby/4d427682ab475f321dc691b66bd114460fea95fef17d5915bcb5a4ad0c40c501/shim.sock" debug=false module="containerd/tasks" pid=10115
Dec 16 01:20:13 Mirai dockerd[483]: time="2017-12-16T01:20:13.124443179+07:00" level=info msg="ignoring event" module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete"
Dec 16 01:20:13 Mirai dockerd[483]: time="2017-12-16T01:20:13+07:00" level=info msg="shim reaped" id=4d427682ab475f321dc691b66bd114460fea95fef17d5915bcb5a4ad0c40c501 module="containerd/tasks"
Dec 16 01:20:19 Mirai dockerd[483]: time="2017-12-16T01:20:19+07:00" level=info msg="shim docker-containerd-shim started" address="/containerd-shim/moby/f9eee76891ccc0c5cc6d66fbe2a3e553b1fa9c062448bce6bf4fc6d70f00aa4d/shim.sock" debug=false module="containerd/tasks" pid=10228
Dec 16 01:20:28 Mirai dockerd[483]: time="2017-12-16T01:20:28.185802772+07:00" level=info msg="ignoring event" module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete"
Dec 16 01:20:28 Mirai dockerd[483]: time="2017-12-16T01:20:28+07:00" level=info msg="shim reaped" id=f9eee76891ccc0c5cc6d66fbe2a3e553b1fa9c062448bce6bf4fc6d70f00aa4d module="containerd/tasks"

Last edited by KuriyamaMirai (2018-01-02 02:11:58)

progandy · 2017-12-15 18:55:31

cat /etc/systemd/network/wlp3s0.network

I'm not sure, but it seems that you might have to enable forwarding for docker0, too? The wiki says you need /etc/systemd/network/ipforward.network with exactly this content:

[Network]
IPForward=kernel

Edit: According to the page on Internet sharing, IPForward=kernel is not working anymore. Try and change it to "yes" or "ipv4".
https://wiki.archlinux.org/index.php/In … forwarding

Last edited by progandy (2017-12-15 19:03:14)

KuriyamaMirai · 2017-12-15 19:12:53

progandy wrote:

cat /etc/systemd/network/wlp3s0.network
I'm not sure, but it seems that you might have to enable forwarding for docker0, too? The wiki says you need /etc/systemd/network/ipforward.network with exactly this content:
[Network]
IPForward=kernel

If i recall conrrectly, there are not any problem with *.network name. I also tried adding ipforward.network but it did not work and it would make my internet connection unavaible on boot (lost network connection after reboot) and I must re-run dhcpcd wlp3s0.
Also tried IPForward=yes but not worked

progandy · 2017-12-15 19:29:24

KuriyamaMirai wrote:

If i recall conrrectly, there are not any problem with *.network name. I also tried adding ipforward.network but it did not work and it would make my internet connection unavaible on boot (lost network connection after reboot) and I must re-run dhcpcd wlp3s0.
Also tried IPForward=yes but not worked

The name is not important, but the additional file without a [Match] would have applied the option to all devices and not only to your wireless card. With the change [edit: in Systemd] that made IPForward global that was irrelevant, though. Sorry.

Last edited by progandy (2017-12-15 19:30:08)

KuriyamaMirai · 2017-12-15 19:43:56

progandy wrote:

KuriyamaMirai wrote:
If i recall conrrectly, there are not any problem with *.network name. I also tried adding ipforward.network but it did not work and it would make my internet connection unavaible on boot (lost network connection after reboot) and I must re-run dhcpcd wlp3s0.
Also tried IPForward=yes but not worked
The name is not important, but the additional file without a [Match] would have applied the option to all devices and not only to your wireless card. With the change [edit: in Systemd] that made IPForward global that was irrelevant, though. Sorry.

it's fine, I also tried it before, anyway thanks for your reply

firecat53 · 2017-12-15 19:59:44

What's the content of /etc/resolv.conf, both on the host and inside a running container? They should match.

A long shot, but perhaps attempt to setup NetworkManager on your host (laptop, I assume) and once you've established that, then try again. I can't imagine it should make a difference, but I'm running out of ideas!

I use NetworkManager on my laptop with Docker, and I use systemd-networkd on my server (albeit with an ethernet static IP bridged connection), also with Docker. Both work fine without any workarounds (other than the Docker bridge address range adjustment on my laptop).

Edit: so looking back more closely, your docker0 bridge shows as 'DOWN' both in 'ip route' and 'ip addr'. Maybe try:

sudo ip link set docker0 up

Last edited by firecat53 (2017-12-15 20:07:40)

KuriyamaMirai · 2017-12-15 20:12:03

firecat53 wrote:

What's the content of /etc/resolv.conf, both on the host and inside a running container? They should match.
A long shot, but perhaps attempt to setup NetworkManager on your host (laptop, I assume) and once you've established that, then try again. I can't imagine it should make a difference, but I'm running out of ideas!
I use NetworkManager on my laptop with Docker, and I use systemd-networkd on my server (albeit with an ethernet static IP bridged connection), also with Docker. Both work fine without any workarounds (other than the Docker bridge address range adjustment on my laptop).
Edit: so looking back more closely, your docker0 bridge shows as 'DOWN' both in 'ip route' and 'ip addr'. Maybe try:
sudo ip link set docker0 up

I use GG DNS, and make it unmotifiable with chattr +i to prevent networkd auto re-generate resolv.conf on boot

$ cat /etc/resolv.conf
# Google DNS server
nameserver 8.8.8.8
nameserver 8.8.4.4
# Generated by resolvconf
nameserver 203.113.188.1
nameserver 203.113.131.3

sudo ip link set docker0 up also not worked
I think I will try with NetworkManager tomorrow to see if it will work, I used netctl before and it not worked so I switched to systemd-networkd, thanks for your help.

Last edited by KuriyamaMirai (2017-12-15 20:20:39)

firecat53 · 2017-12-15 20:38:54

I'm pretty sure that docker0 being down is the actual cause of not having networking inside your containers. It has to be up for the default Docker networking or it just won't work. So regardless of what else you try, that has to be fixed.

You might try stopping Docker, then deleting the docker0 bridge 'brctl delbr docker0'. Restart Docker and it should be created again automatically...and hopefully in the 'UP' state.

Last edited by firecat53 (2017-12-15 20:40:40)

KuriyamaMirai · 2017-12-16 03:32:36

firecat53 wrote:

I'm pretty sure that docker0 being down is the actual cause of not having networking inside your containers. It has to be up for the default Docker networking or it just won't work. So regardless of what else you try, that has to be fixed.
You might try stopping Docker, then deleting the docker0 bridge 'brctl delbr docker0'. Restart Docker and it should be created again automatically...and hopefully in the 'UP' state.

It still is down no matter how I re-created it, I tried re-installing docker but it still not worked

# systemctl stop docker
# brctl delbr docker0
bridge docker0 is still up; can't delete it
# ip link set docker0 down
# brctl delbr docker0
# systemctl start docker
# ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000

2: enp4s0f2: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc fq_codel state DOWN mode DEFAULT group default qlen 1000

3: wlp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000

12: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN mode DEFAULT group default 


# ip link set docker0 up
# ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000

2: enp4s0f2: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc fq_codel state DOWN mode DEFAULT group default qlen 1000

3: wlp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000

12: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN mode DEFAULT group default

Last edited by KuriyamaMirai (2018-01-02 02:09:58)

positronik · 2017-12-16 08:52:59

How does the Docker bridge work?
Using LXC I discovered that if your host machine connects via wifi a simple bridge won't work and you need to create a subnet behind NAT for your containers.
Is this maybe the problem you are experiencing?

progandy · 2017-12-16 10:01:03

Is a docker container running when you check "ip link"? The bridge might show NO-CARRIER until at least one container is connected to it.

firecat53 · 2017-12-17 00:56:45

Progandy, you're right...my apologies. docker0 does indeed show as down without any running containers. I've always got a few running Sorry OP, I'm fresh out of ideas.

KuriyamaMirai · 2017-12-17 04:44:08

progandy wrote:

Is a docker container running when you check "ip link"? The bridge might show NO-CARRIER until at least one container is connected to it.

if there is a docker container running, docker0 is up

positronik wrote:

How does the Docker bridge work?
Using LXC I discovered that if your host machine connects via wifi a simple bridge won't work and you need to create a subnet behind NAT for your containers.
Is this maybe the problem you are experiencing?

Can you teach me how to do that? I tried all solution I found but none of them worked

Last edited by KuriyamaMirai (2017-12-17 04:45:58)

positronik · 2017-12-17 09:08:53

KuriyamaMirai wrote:

positronik wrote:
How does the Docker bridge work?
Using LXC I discovered that if your host machine connects via wifi a simple bridge won't work and you need to create a subnet behind NAT for your containers.
Is this maybe the problem you are experiencing?
Can you teach me how to do that? I tried all solution I found but none of them worked

Actually by looking more careful at the output of your commands it seems that docker takes care of it by itself, therefore I don't think that what I said is actually the problem here.
Sorry for not being of much help.

KuriyamaMirai · 2018-01-02 02:07:25

I just re-installed Arch and the problem solved.
I think the cause of problem may be a bit stupid but it is I set the same username and hosts (mirai and Mirai)
Sorry for trouble you and thanks for helping me

Arch Linux

#1 2017-12-15 11:34:47

[SOLVED] No network on Docker Container

#2 2017-12-15 15:21:05

Re: [SOLVED] No network on Docker Container

#3 2017-12-15 17:23:51

Re: [SOLVED] No network on Docker Container

#4 2017-12-15 17:44:33

Re: [SOLVED] No network on Docker Container

#5 2017-12-15 18:21:43

Re: [SOLVED] No network on Docker Container

#6 2017-12-15 18:28:26

Re: [SOLVED] No network on Docker Container

#7 2017-12-15 18:40:53

Re: [SOLVED] No network on Docker Container

#8 2017-12-15 18:55:31

Re: [SOLVED] No network on Docker Container

#9 2017-12-15 19:12:53

Re: [SOLVED] No network on Docker Container

#10 2017-12-15 19:29:24

Re: [SOLVED] No network on Docker Container

#11 2017-12-15 19:43:56

Re: [SOLVED] No network on Docker Container

#12 2017-12-15 19:59:44

Re: [SOLVED] No network on Docker Container

#13 2017-12-15 20:12:03

Re: [SOLVED] No network on Docker Container

#14 2017-12-15 20:38:54

Re: [SOLVED] No network on Docker Container

#15 2017-12-16 03:32:36

Re: [SOLVED] No network on Docker Container

#16 2017-12-16 08:52:59

Re: [SOLVED] No network on Docker Container

#17 2017-12-16 10:01:03

Re: [SOLVED] No network on Docker Container

#18 2017-12-17 00:56:45

Re: [SOLVED] No network on Docker Container

#19 2017-12-17 04:44:08

Re: [SOLVED] No network on Docker Container

#20 2017-12-17 09:08:53

Re: [SOLVED] No network on Docker Container

#21 2018-01-02 02:07:25

Re: [SOLVED] No network on Docker Container

Board footer