[SOLVED] openvpn and networkmanager invisibly fail on resume

Salkay · 2017-03-13 00:49:28

I use openvpn with NetworkManager. It's set up so that after connecting to either my wifi connection at home, or my ethernet at work, it automatically brings up the VPN. This used to work fine after resuming from suspend. Recently (perhaps in the last week?) the NetworkManager icon says that it's connected to the VPN, but I can tell from my public IP that I'm not. In addition, I can see the following output.

$ nmcli con
NAME                    UUID                                  TYPE             DEVICE  
Ethernet                12345678-1234-1234-1234-123456789abc  802-3-ethernet   enp10s0 
Mullvad OpenVPN         12345678-1234-1234-1234-123456789abc  vpn              enp10s0 
WifiNetwork             12345678-1234-1234-1234-123456789abc  802-11-wireless  --      
...

I can restart NetworkManager, which usually fixes the problem, i.e. I'm genuinely connect to the VPN. nmcli also gives different output, with the tun0 entry appearing.

$ nmcli con
NAME                    UUID                                  TYPE             DEVICE  
Ethernet                12345678-1234-1234-1234-123456789abc  802-3-ethernet   enp10s0 
Mullvad OpenVPN         12345678-1234-1234-1234-123456789abc  vpn              enp10s0 
tun0                    12345678-1234-1234-1234-123456789abc  tun              tun0    
WifiNetwork             12345678-1234-1234-1234-123456789abc  802-11-wireless  --      
...

However, when the VPN fails, there's no clear indication, so I don't know I have to restart NetworkManager. Does anyone know why NetworkManager is behaving like this, and can I prevent it? I keep my system up-to-date, so I don't think the networkmanager update from 2.5 weeks ago is to blame (and openvpn hasn't been updated for a while).

==EDIT==
In the meantime, I wrote a script to restart NetworkManager if this fails.

Create the following file at (e.g.) /etc/NetworkManager/dispatcher.d/97_vpn_fails.sh

#!/bin/sh
# Sometimes NetworkManager says it's connected to VPN, but actually isn't

IF=$1
STATUS=$2
# VPN goes up
if [ "$STATUS" = 'vpn-up' ]; then
  nmcli con | grep '^tun0' -q || (date >> /tmp/vpn-fail ; systemctl restart NetworkManager)
fi

It does some rudimentary logging. To remove that, just change the executed line to

nmcli con | grep '^tun0' -q || systemctl restart NetworkManager

Last edited by Salkay (2017-03-28 02:00:25)

vtyulb · 2017-03-14 13:07:37

Yep, I got same problem. Thank you for temporary fix.

R00KIE · 2017-03-14 14:08:31

I have stumbled upon something similar, but in my case it is right when connecting, more often than not the icon says connected but it actually isn't.

When checking the output of 'ip route' and 'ip address show' I can see that not all the routes that should be there are present and the tun device (in my case) does not have an IP address assigned. The logs have something like this when things are not working: "<error> [1489418374.9762] platform-linux: do-add-ip4-address[5: 10.10.1.2/24]: failure 19 (No such device)". However the tun device is there, I guess this must be some kind or race issue where networkmanager tried to set the IP before the tun device is fully ready.

I don't know what to blame, if networkmanager if kernel 4.10 as it seems there are people with other network connectivity problems with 4.10 (might be related might not be).

Edit:
I've done a few tests using openvpn directly and it never failed to set things up properly, so I'd blame this on networkmanager.

Last edited by R00KIE (2017-03-14 21:55:16)

Salkay · 2017-03-15 00:10:32

vtyulb wrote:

Yep, I got same problem. Thank you for temporary fix.

You are welcome. It seems to work fairly well, but often the icon seems to be absent in my tray, and I have to either restart NetworkManager again, or my DE.

R00KIE wrote:

I have stumbled upon something similar, but in my case it is right when connecting, more often than not the icon says connected but it actually isn't.

I'm not sure what you mean. That's what I see too, isn't it?

Salkay wrote:

the NetworkManager icon says that it's connected to the VPN, but I can tell from my public IP that I'm not.

R00KIE wrote:

so I'd blame this on networkmanager.

Presuming we are seeing the same thing, have you only encountered this bug recently? Just because NetworkManager hasn't been updated in ~3 weeks, so maybe it is kernel-related.

Also, I couldn't find an obvious place to post NetworkManager bugs. There's the Launchpad page, but I'm not sure if that is just for Ubuntu users, or if there is something upstream. Also, there's no recent bug similar to this, although Ubuntu is shipping a very old version.

solstice · 2017-03-15 06:30:09

I have the same problem.
If I manually launch the VPN from the network manager applet menu, the VPN is correctly setup. This only happens when configured to automatically connect.

I have open a bug at networkmanager repo: https://bugzilla.gnome.org/show_bug.cgi?id=780070

Salkay · 2017-03-15 06:32:47

solstice wrote:

I have open a bug at networkmanager repo: https://bugzilla.gnome.org/show_bug.cgi?id=780070

Thanks for doing that!

R00KIE · 2017-03-15 14:31:27

Salkay wrote:

R00KIE wrote:
I have stumbled upon something similar, but in my case it is right when connecting, more often than not the icon says connected but it actually isn't.
I'm not sure what you mean. That's what I see too, isn't it?

What I was trying to say is that I don't need to suspend and resume to see the problem, a normal boot, login and trying to connect manually is enough to see the problem.

Salkay wrote:

the NetworkManager icon says that it's connected to the VPN, but I can tell from my public IP that I'm not.
R00KIE wrote:
so I'd blame this on networkmanager.
Presuming we are seeing the same thing, have you only encountered this bug recently? Just because NetworkManager hasn't been updated in ~3 weeks, so maybe it is kernel-related.

I can't say. I don't use the VPN connection very often so it might be either a kernel problem or a networkmanager problem. I haven't been able to trigger the problem if connecting manually/directly with openvpn though so that would make me suspect networkmanager more than the kernel.

Salkay wrote:

Also, I couldn't find an obvious place to post NetworkManager bugs. There's the Launchpad page, but I'm not sure if that is just for Ubuntu users, or if there is something upstream. Also, there's no recent bug similar to this, although Ubuntu is shipping a very old version.

Networkmanager bugs go here https://bugzilla.gnome.org/ but when filling a bug report you have to put it in the correct project as that bug tracker is for everything gnome.

Salkay · 2017-03-15 22:31:47

R00KIE wrote:

What I was trying to say is that I don't need to suspend and resume to see the problem, a normal boot, login and trying to connect manually is enough to see the problem.

Ah okay. I rarely reboot, but that makes sense.

R00KIE wrote:

I can't say. I don't use the VPN connection very often so it might be either a kernel problem or a networkmanager problem. I haven't been able to trigger the problem if connecting manually/directly with openvpn though so that would make me suspect networkmanager more than the kernel.

Fair enough. Perhaps you might be able to post that information in the bug that solstice opened?

solstice · 2017-03-19 18:55:38

In the mean time I have adapted a script from the wiki to be used as dispatcher script to auto-enable the VPN.

It loops to disable-re-enable the VPN until it works.

while ! nmcli con show|grep -q tun0; do
        nmcli con down id "$VPN_NAME"
        nmcli con up id "$VPN_NAME"
done

so the script looks like this:

#!/bin/sh
VPN_NAME="vpn name of connection in network manager"
ESSID="$(iwgetid -r)"

interface=$1 status=$2
case $status in
  up|vpn-down)
    if iwgetid | grep -qs ":\"$ESSID\""; then
      nmcli con up id "$VPN_NAME"
      while ! nmcli con show|grep -q tun0; do
        nmcli con down id "$VPN_NAME"
        nmcli con up id "$VPN_NAME"
      done
    fi
    ;;
  down)
    if iwgetid | grep -qs ":\"$ESSID\""; then
      if nmcli con show --active | grep "$VPN_NAME"; then
        nmcli con down id "$VPN_NAME"
      fi
    fi
    ;;
esac

It seems to work here.
Enjoy !

Last edited by solstice (2017-03-20 19:17:09)

Salkay · 2017-03-19 22:59:23

Thanks solstice. Looks great! It's a lot more elegant than my script totally restarting NetworkManager. I previously found some slight oddities with my method, with the NetworkManager icon totally disappearing sometimes. Hopefully just toggling the VPN itself is less intrusive. I've also modified your script a little further, because:

1. I didn't want VPN to connect for just wifi.
2. I sometimes manually turn off the VPN, so I don't want it automatically going back up in this case. I just want it to fix NetworkManager's shortcomings (i.e. just have $status == vpn-up).
3. When connecting, I prevented the script from automatically connecting as a matter of course, to prevent potential race conditions with the "normal" NetworkManager.
4. I didn't need the $status == down part either, because I can trust NetworkManager with killing the VPN (I think), it's just going up that is the problem.
5. I added a sleep line in just in case the loop went rampant.

#!/bin/sh

VPN_NAME='vpn name of connection in network manager'

interface=$1 status=$2
case $status in
  vpn-up)
    while ! nmcli con show|grep -q tun0; do
      nmcli con down id "$VPN_NAME"
      sleep 0.5
      nmcli con up id "$VPN_NAME"
    done
    ;;
esac

Thanks again!

Salkay · 2017-03-22 09:44:21

Here we go with a new version of the script. I realised that if tun0 failed, the script would bring the vpn down then up again… which would trigger this script again anyway (vpn-up). So the script doesn't need a "while" loop, just an "if". I also added some more detailed logging lines.

I certainly see this problem very intermittently. On two separate occasions, it resumed fine. Then on the third, it took three attempts to get the vpn up again.

#!/bin/sh

VPN_NAME='vpn name of connection in network manager'

interface=$1 status=$2
case $status in
  vpn-up)
    echo "[$(date --rfc-3339=seconds)] vpn supposedly up" >> /tmp/vpn-fail
    if ! nmcli con show|grep -q tun0; then
      echo "[$(date --rfc-3339=seconds)] tun0 failed. reconnecting vpn" >> /tmp/vpn-fail
      nmcli con down id "$VPN_NAME"
      sleep 0.5
      nmcli con up id "$VPN_NAME"
    fi
    ;;
esac

solstice · 2017-03-22 20:25:39

yes the last version works better. I use it. thanks

kubrick · 2017-03-23 15:03:44

commit a79fa23c82a1b7ee3726ea348336fbe874130a7c of kernel 4.10.5 seems to fix openvpn not being able to set routes correctly on newer kernels

Salkay · 2017-03-23 21:18:00

kubrick wrote:

commit a79fa23c82a1b7ee3726ea348336fbe874130a7c of kernel 4.10.5 seems to fix openvpn not being able to set routes correctly on newer kernels

Thanks for reporting! Do you know if there's a patch that we can apply?

R00KIE · 2017-03-23 21:56:04

Unless I didn't check correctly, there is no such commit as a79fa23c82a1b7ee3726ea348336fbe874130a7c upstream. It should be commit b20e2d54789c6acbf6bd0efdbec2cf5fa4d90ef1 tun: fix premature POLLOUT notification on tun devices, and it should be included already in kernel 4.10.5 which is in [testing].

Salkay · 2017-03-24 00:13:57

R00KIE wrote:

Unless I didn't check correctly, there is no such commit as a79fa23c82a1b7ee3726ea348336fbe874130a7c upstream. It should be commit b20e2d54789c6acbf6bd0efdbec2cf5fa4d90ef1 tun: fix premature POLLOUT notification on tun devices, and it should be included already in kernel 4.10.5 which is in [testing].

I wasn't really sure where to look, so I just did a general web search and came up with this page. Anyway, thanks for the information; I've installed the [testing] version instead. I'll report back on how it's going in a little while.

Also, does anyone know where best to find information on bugs in unstable/[testing] packages? I searched the subforum here and the Arch bugtracker (and found nothing). Would they be the places that people post up feedback?

kubrick · 2017-03-24 09:42:00

R00KIE wrote:

Unless I didn't check correctly, there is no such commit as a79fa23c82a1b7ee3726ea348336fbe874130a7c upstream. It should be commit b20e2d54789c6acbf6bd0efdbec2cf5fa4d90ef1 tun: fix premature POLLOUT notification on tun devices, and it should be included already in kernel 4.10.5 which is in [testing].

You didn't check correctly
https://git.kernel.org/pub/scm/linux/ke … e874130a7c

solstice · 2017-03-24 12:36:08

kernel 4.10.5 is in stable now. It seems to work fine now at firt sight

Salkay · 2017-03-24 12:58:58

Yes, I agree. It has connected properly for me four times in a row.

Arch Linux

#1 2017-03-13 00:49:28

[SOLVED] openvpn and networkmanager invisibly fail on resume

#2 2017-03-14 13:07:37

Re: [SOLVED] openvpn and networkmanager invisibly fail on resume

#3 2017-03-14 14:08:31

Re: [SOLVED] openvpn and networkmanager invisibly fail on resume

#4 2017-03-15 00:10:32

Re: [SOLVED] openvpn and networkmanager invisibly fail on resume

#5 2017-03-15 06:30:09

Re: [SOLVED] openvpn and networkmanager invisibly fail on resume

#6 2017-03-15 06:32:47

Re: [SOLVED] openvpn and networkmanager invisibly fail on resume

#7 2017-03-15 14:31:27

Re: [SOLVED] openvpn and networkmanager invisibly fail on resume

#8 2017-03-15 22:31:47

Re: [SOLVED] openvpn and networkmanager invisibly fail on resume

#9 2017-03-19 18:55:38

Re: [SOLVED] openvpn and networkmanager invisibly fail on resume

#10 2017-03-19 22:59:23

Re: [SOLVED] openvpn and networkmanager invisibly fail on resume

#11 2017-03-22 09:44:21

Re: [SOLVED] openvpn and networkmanager invisibly fail on resume

#12 2017-03-22 20:25:39

Re: [SOLVED] openvpn and networkmanager invisibly fail on resume

#13 2017-03-23 15:03:44

Re: [SOLVED] openvpn and networkmanager invisibly fail on resume

#14 2017-03-23 21:18:00

Re: [SOLVED] openvpn and networkmanager invisibly fail on resume

#15 2017-03-23 21:56:04

Re: [SOLVED] openvpn and networkmanager invisibly fail on resume

#16 2017-03-24 00:13:57

Re: [SOLVED] openvpn and networkmanager invisibly fail on resume

#17 2017-03-24 09:42:00

Re: [SOLVED] openvpn and networkmanager invisibly fail on resume

#18 2017-03-24 12:36:08

Re: [SOLVED] openvpn and networkmanager invisibly fail on resume

#19 2017-03-24 12:58:58

Re: [SOLVED] openvpn and networkmanager invisibly fail on resume

Board footer