[SOLVED] Lost connectivity on resume with netctl-auto

eddy_hunter · 2017-11-02 13:18:56

I use my laptop exclusively on the university campus, access the internet over wifi. We have the same SSID across campus, but if I close the lid, go to a different building and open it again there, I lose connectivity.

When I try to access a website on my browser, it says it might be a DNS issue. I tried e.g. ping google.com and even telnet google.com 80, but nothing happens (hangs). Trying to dig google.com shows that the system can't resolve the IP. I poked around with ip a bit and found this:

ip n

10.2.128.1 dev wlp4s0 INCOMPLETE
fe80::da67:d9ff:fe0e:73c1 dev wlp4s0 lladdr d8:67:d9:0e:73:c1 router REACHABLE
fe80::6c5:a4ff:fee3:fb42 dev wlp4s0 router FAILED

This is what ip a says about my IP:

inet 10.2.129.42/20 brd 10.2.143.255 scope global wlp4s0
valid_lft forever preferred_lft forever

If I understand it correctly, I am in the 10.2.128.0/20 network (/20 subnet mask) but I can't reach the gateway (10.2.128.1), so I can't even resolve DNS queries. Since I went to a different building, I assume I should be in a different network. I did systemctl restart netctl-auto@wlp4s0, got a new IP and restored connectivity. This is how things look now:

ip n

10.2.32.1 dev wlp4s0 lladdr d8:67:d9:0e:73:c1 REACHABLE
fe80::da67:d9ff:fe0e:73c1 dev wlp4s0 lladdr d8:67:d9:0e:73:c1 router STALE

And my IP:

inet 10.2.36.12/19 brd 10.2.63.255 scope global wlp4s0
valid_lft forever preferred_lft forever

So I guess now I'm in the right network, I can reach the gateway, everything is fine. From my understanding of DHCP, I should receive an IP, subnet mask and gateway address from the router, but for some reason this isn't getting refreshed when the computer reconnects to the network upon waking up. This is what journalctl shows:

Nov 02 11:37:35 narch systemd[1]: Stopped target Suspend.
Nov 02 11:37:35 narch kernel: psmouse serio1: synaptics: queried max coordinates: x [..5678], y [..4754]
Nov 02 11:37:35 narch kernel: psmouse serio1: synaptics: queried min coordinates: x [1262..], y [1098..]
Nov 02 11:37:36 narch kernel: ish-hid {33AECD58-B679-4E54-9BD9-A04D34F0C226}: [hid-ish]: enum_devices_done OK, num_hid_devices=2
Nov 02 11:37:38 narch kernel: wlp4s0: authenticate with 2c:41:38:f4:57:a0
Nov 02 11:37:38 narch kernel: wlp4s0: send auth to 2c:41:38:f4:57:a0 (try 1/3)
Nov 02 11:37:38 narch kernel: wlp4s0: authenticated
Nov 02 11:37:38 narch kernel: wlp4s0: associate with 2c:41:38:f4:57:a0 (try 1/3)
Nov 02 11:37:38 narch kernel: wlp4s0: RX AssocResp from 2c:41:38:f4:57:a0 (capab=0x8011 status=0 aid=11)
Nov 02 11:37:38 narch kernel: wlp4s0: associated
Nov 02 11:37:38 narch wpa_actiond[441]: Interface 'wlp4s0' reestablished connection to network 'eth-5'

No DHCP negotiation at all. I looked at the wiki and found this (link):

Sometimes netctl-auto fails to reconnect when the system resumes from suspend, hibernate or hybrid-sleep. An easy solution is to restart the service for netctl-auto. This can be automated with an additional service like the following (netctl-auto-resumeAUR)

I understand how this would fix the problem, but is it really the proper solution? I get that netctl-auto wouldn't refresh the DHCP lease because the SSID hasn't changed, but at the same time, large campuses with a single SSID and many subnets aren't that uncommon, and colleagues who also use Arch (with networkmanager) don't suffer from the same problem.

In summary:
- am I interpreting the logs correctly?
- is the wiki solution what I need, or is my system otherwise misconfigured?

Last edited by eddy_hunter (2017-11-21 07:43:08)

lo1 · 2017-11-02 14:05:42

You got it perfectly: different areas of your campus belong to different subnets, hence you got to restart netctl. Your default gateway should provide a DNS server for you, but if the wireless card isn't provided of an ip address it can't have most basic level 3 connectivity. The dhcpcd should be called automatically by netctl. The wiki is ok, this may belong only to your case. Let's figure that out: what you got when, staying in the same subnet, you call a

systemctl restart netctl-auto@wlp4s0

?
And what's the output of

ip route

?

EDIT: also, do you mind to post the relevant profiles? (excluding any sensitive information, if that is the case...)

Last edited by lo1 (2017-11-02 14:09:00)

cfr · 2017-11-02 14:25:52

I have found that using dhclient for DHCP helps. But netctl still doesn't always handle this correctly by itself - it just doesn't fail to do it correctly as frequently. You don't need to here, but sometimes I also have to remove a lease manually in order to get it working again, as netctl doesn't handle cases in which the existing lease is 'corrupt'.

eddy_hunter · 2017-11-02 19:01:38

lo1 wrote:

You got it perfectly: different areas of your campus belong to different subnets, hence you got to restart netctl. Your default gateway should provide a DNS server for you, but if the wireless card isn't provided of an ip address it can't have most basic level 3 connectivity. The dhcpcd should be called automatically by netctl. The wiki is ok, this may belong only to your case. Let's figure that out: what you got when, staying in the same subnet, you call a
systemctl restart netctl-auto@wlp4s0
?
And what's the output of
ip route
?
EDIT: also, do you mind to post the relevant profiles? (excluding any sensitive information, if that is the case...)

Right now I'm connected and everything works. I restart netctl-auto, my IP stays the same. Here's the log:

Nov 02 18:47:15 narch sudo[10213]:     eddy : TTY=pts/4 ; PWD=/home/eddy/desktop/anothertest ; USER=root ; COMMAND=/usr/bin/systemctl restart netctl-auto@wlp4s0.service
Nov 02 18:47:15 narch sudo[10213]: pam_unix(sudo:session): session opened for user root by eddy(uid=0)
Nov 02 18:47:15 narch systemd[1]: Stopping Automatic wireless network connection using netctl profiles...
Nov 02 18:47:15 narch wpa_actiond[15736]: Interface 'wlp4s0' disconnected from network 'eth-5'
Nov 02 18:47:15 narch dhcpcd[10482]: sending signal TERM to pid 16314
Nov 02 18:47:15 narch dhcpcd[10482]: waiting for pid 16314 to exit
Nov 02 18:47:15 narch dhcpcd[16314]: received SIGTERM, stopping
Nov 02 18:47:15 narch dhcpcd[16314]: wlp4s0: removing interface
Nov 02 18:47:15 narch dhcpcd[16314]: dhcpcd exited
Nov 02 18:47:16 narch wpa_actiond[15736]: Terminating wpa_actiond session for interface 'wlp4s0'
Nov 02 18:47:16 narch kernel: wlp4s0: deauthenticating from 98:4b:e1:2b:a9:20 by local choice (Reason: 3=DEAUTH_LEAVING)
Nov 02 18:47:16 narch kernel: wlp4s0: failed to remove key (1, ff:ff:ff:ff:ff:ff) from hardware (-22)
Nov 02 18:47:17 narch systemd[1]: Stopped Automatic wireless network connection using netctl profiles.
Nov 02 18:47:17 narch systemd[1]: Starting Automatic wireless network connection using netctl profiles...
Nov 02 18:47:17 narch netctl-auto[10574]: Included profile 'eth'
Nov 02 18:47:17 narch kernel: IPv6: ADDRCONF(NETDEV_UP): wlp4s0: link is not ready
Nov 02 18:47:17 narch wpa_actiond[10647]: Starting wpa_actiond session for interface 'wlp4s0'
Nov 02 18:47:17 narch systemd[1]: Started Automatic wireless network connection using netctl profiles.
Nov 02 18:47:17 narch sudo[10213]: pam_unix(sudo:session): session closed for user root
Nov 02 18:47:20 narch kernel: wlp4s0: authenticate with 98:4b:e1:2b:a9:20
Nov 02 18:47:20 narch kernel: wlp4s0: send auth to 98:4b:e1:2b:a9:20 (try 1/3)
Nov 02 18:47:20 narch kernel: wlp4s0: authenticated
Nov 02 18:47:20 narch kernel: wlp4s0: associate with 98:4b:e1:2b:a9:20 (try 1/3)
Nov 02 18:47:20 narch kernel: wlp4s0: RX AssocResp from 98:4b:e1:2b:a9:20 (capab=0x8011 status=0 aid=3)
Nov 02 18:47:20 narch kernel: wlp4s0: associated
Nov 02 18:47:20 narch kernel: IPv6: ADDRCONF(NETDEV_CHANGE): wlp4s0: link becomes ready
Nov 02 18:47:21 narch wpa_actiond[10647]: Interface 'wlp4s0' connected to network 'eth-5'
Nov 02 18:47:21 narch dhcpcd[10862]: DUID 00:01:00:01:20:2f:7a:b2:e4:a7:a0:bd:e9:99
Nov 02 18:47:21 narch dhcpcd[10862]: wlp4s0: IAID a0:bd:e9:99
Nov 02 18:47:21 narch dhcpcd[10862]: wlp4s0: rebinding lease of 10.2.129.82
Nov 02 18:47:21 narch dhcpcd[10862]: wlp4s0: probing address 10.2.129.82/20
Nov 02 18:47:26 narch dhcpcd[10862]: wlp4s0: leased 10.2.129.82 for 3053 seconds
Nov 02 18:47:26 narch dhcpcd[10862]: wlp4s0: adding route to 10.2.128.0/20
Nov 02 18:47:26 narch dhcpcd[10862]: wlp4s0: adding default route via 10.2.128.1
Nov 02 18:47:26 narch dhcpcd[10862]: forked to background, child pid 11198

It looks like dhcpcd is asking for the same IP in the discover phase, which is fine. If it did that after I move to a different subnet, I guess the DHCP server would deny the lease and offer another IP, which would also be fine.

Here's my current IP route (after restarting):

default via 10.2.128.1 dev wlp4s0 src 10.2.129.82 metric 302 
10.2.128.0/20 dev wlp4s0 proto kernel scope link src 10.2.129.82 metric 302

I actually have the outputs from before, so I'll post them as well.

This is what it was when it was not working (wrong IP and gateway):

default via 10.2.128.1 dev wlp4s0 src 10.2.129.42 metric 302 
10.2.128.0/20 dev wlp4s0 proto kernel scope link src 10.2.129.42 metric 302

And after I restarted it then:

default via 10.2.32.1 dev wlp4s0 src 10.2.36.12 metric 302 
10.2.32.0/19 dev wlp4s0 proto kernel scope link src 10.2.36.12 metric 302

This is the network profile:

Description='Automatically generated profile by wifi-menu'
Interface=wlp4s0
Connection=wireless
Security=wpa-configsection
IP=dhcp
Priority=1
WPAConfigSection=(
        'ssid="eth-5"'
        'key_mgmt=WPA-EAP'
        'proto=RSN'
        'eap=PEAP'
        'identity="#"'
        'password="#"'
)

I have found that using dhclient for DHCP helps. But netctl still doesn't always handle this correctly by itself - it just doesn't fail to do it correctly as frequently. You don't need to here, but sometimes I also have to remove a lease manually in order to get it working again, as netctl doesn't handle cases in which the existing lease is 'corrupt'.

I tried dhclient as well as few days ago for different reasons (IPv6 isn't working, I'll post about that next haha) but still had the same problem...

lo1 · 2017-11-02 19:54:38

The configuration is flawless, but i'd try adding an interface-specific hook.
I'd try adding

ExecUpPost="dhcpcd wlp4so"
ExecDownPre="dhcpcd exit wlp4so"

Double check cause probably i got it wrong.

EDIT: of course, that will only work when you have that script service for restarting the nectl-auto profile.

Last edited by lo1 (2017-11-03 17:02:58)

cfr · 2017-11-03 00:05:08

Does it help if you enable the netctl-sleep.service unit?

eddy_hunter · 2017-11-13 09:22:21

Sorry about the late reply, didn't have a lot of time to play with this until now.

lo1 wrote:

The configuration is flawless, but i'd try adding an interface-specific hook.
I'd try adding
ExecUpPost="dhcpcd wlp4so"
ExecDownPre="dhcpcd exit wlp4so"
Double check cause probably i got it wrong.
EDIT: of course, that will only work when you have that script service for restarting the nectl-auto profile.

I guess that would work, but like you say it requires the netctl-auto restart script. I was hoping to avoid that script, so instead I tried the netctl-sleep solution. It doesn't seem do anything when using netctl-auto, so today I tried a different approach: I disabled netctl-auto and enabled the university wifi profile directly in netctl, and left netctl-sleep enabled. Just moved to a different building and the wifi works fine, here is the log:

Nov 13 10:05:06 narch systemd-sleep[16443]: System resumed.
Nov 13 10:05:06 narch systemd[1]: Started Suspend.
Nov 13 10:05:06 narch systemd[1]: sleep.target: Unit not needed anymore. Stopping.
Nov 13 10:05:06 narch systemd[1]: Stopped target Sleep.
Nov 13 10:05:06 narch systemd[1]: netctl-sleep.service: Unit not needed anymore. Stopping.
Nov 13 10:05:06 narch systemd[1]: Stopping netctl sleep hook...
Nov 13 10:05:06 narch systemd[1]: Reached target Suspend.
Nov 13 10:05:06 narch systemd-logind[331]: Operation 'sleep' finished.
Nov 13 10:05:06 narch systemd[1]: suspend.target: Unit not needed anymore. Stopping.
Nov 13 10:05:06 narch systemd[1]: Stopped target Suspend.
Nov 13 10:05:06 narch systemd[1]: Starting Automatically generated profile by wifi-menu...
Nov 13 10:05:06 narch network[16529]: Starting network profile 'eth'...
Nov 13 10:05:06 narch kernel: IPv6: ADDRCONF(NETDEV_UP): wlp4s0: link is not ready
Nov 13 10:05:06 narch kernel: psmouse serio1: synaptics: queried max coordinates: x [..5678], y [..4754]
Nov 13 10:05:06 narch kernel: psmouse serio1: synaptics: queried min coordinates: x [1262..], y [1098..]
Nov 13 10:05:06 narch kernel: ish-hid {33AECD58-B679-4E54-9BD9-A04D34F0C226}: [hid-ish]: enum_devices_done OK, num_hid_devices=2
Nov 13 10:05:09 narch kernel: wlp4s0: authenticate with 9c:50:ee:25:42:d0
Nov 13 10:05:09 narch kernel: wlp4s0: send auth to 9c:50:ee:25:42:d0 (try 1/3)
Nov 13 10:05:09 narch kernel: wlp4s0: authenticated
Nov 13 10:05:09 narch kernel: wlp4s0: associate with 9c:50:ee:25:42:d0 (try 1/3)
Nov 13 10:05:09 narch kernel: wlp4s0: RX AssocResp from 9c:50:ee:25:42:d0 (capab=0x1411 status=0 aid=14)
Nov 13 10:05:09 narch kernel: wlp4s0: associated
Nov 13 10:05:09 narch kernel: IPv6: ADDRCONF(NETDEV_CHANGE): wlp4s0: link becomes ready
Nov 13 10:05:09 narch kernel: wlp4s0: Limiting TX power to 20 (23 - 3) dBm as advertised by 9c:50:ee:25:42:d0
Nov 13 10:05:10 narch systemd[1]: Started Automatically generated profile by wifi-menu.
Nov 13 10:05:10 narch systemd[1]: Stopped netctl sleep hook.
Nov 13 10:05:10 narch dhcpcd[16867]: DUID 00:01:00:01:20:2f:7a:b2:e4:a7:a0:bd:e9:99
Nov 13 10:05:10 narch dhcpcd[16867]: wlp4s0: IAID a0:bd:e9:99
Nov 13 10:05:10 narch dhcpcd[16867]: wlp4s0: rebinding lease of 10.2.129.167
Nov 13 10:05:10 narch dhcpcd[16867]: wlp4s0: NAK: Requested address 10.2.129.167 is not valid for this link from 129.132.128.135
Nov 13 10:05:10 narch dhcpcd[16867]: wlp4s0: message: Requested address 10.2.129.167 is not valid for this link
Nov 13 10:05:10 narch dhcpcd[16867]: wlp4s0: soliciting a DHCP lease
Nov 13 10:05:10 narch dhcpcd[16867]: wlp4s0: offered 10.2.71.237 from 129.132.128.135
Nov 13 10:05:10 narch dhcpcd[16867]: wlp4s0: ignoring offer of 10.2.69.251 from 129.132.128.199
Nov 13 10:05:11 narch dhcpcd[16867]: wlp4s0: probing address 10.2.71.237/20
Nov 13 10:05:16 narch dhcpcd[16867]: wlp4s0: leased 10.2.71.237 for 900 seconds
Nov 13 10:05:16 narch dhcpcd[16867]: wlp4s0: adding route to 10.2.64.0/20
Nov 13 10:05:16 narch dhcpcd[16867]: wlp4s0: adding default route via 10.2.64.1
Nov 13 10:05:16 narch dhcpcd[16867]: forked to background, child pid 17368
Nov 13 10:05:16 narch network[16529]: Started network profile 'eth'

This is what I expected netctl-auto to do. The dhcp daemon requests the same IP, the server NAKs and offers an IP in the range of the building's subnet, which the daemon accepts and everything is fine. I can't tell from the log if netctl-sleep is required for this to happen, so this afternoon I'll try it again without it to see what happens.

Since I only use this network, I would be OK with enabling the profile in netctl and forgoing netctl-auto altogether, but it still bothers me that netctl-auto does not do this on its own. I'll update this when I have results for this afternoon's "experiment".

EDIT:
The renegotiation works even without netctl-sleep. I'm not sure what it actually does then. What does it mean in netctl to store the profile?

UPDATE:
After playing around with this some more, I think I found what's causing the problem. Looking at the logs and the netctl source, it seems that when using netctl, dhcpcd renegotiates the IP on its own (it does not need any kind of triggering). This does not happen with netctl-auto, which implies that dhcpcd is being executed differently. I then looked at the systemd status for the two units and found one difference: netctl-auto passes the

-K

flag to dhcpcd, whereas netctl does not. The man pages are not very detailed and I couldn't find much more about it online:

-K, --nolink
Don't receive link messages for carrier status. You should only have to use this with buggy device drivers or running dhcpcd through a network manager.

The way I understand it though, passing this flag prevents dhcpcd from spontaneously acting on carrier status changes, presumably because the network manager will handle the necessary steps such as IP renegotiation (for instance by restarting dhcpcd). Since netctl-auto does not do this, it should not actually be using this flag. I removed the flag from the argument list and now it appears to work.

For the sake of documenting the patch exactly, I opened this file:

/usr/lib/netctl/auto.action

And changed line 18 from this:

DhcpcdOptions+=" -K -L"

To this:

DhcpcdOptions+=" -L"

I am not sure if this is the proper fix, but since I did it yesterday it appears to work, so I'm marking it solved.

Last edited by eddy_hunter (2017-11-21 07:39:18)

Arch Linux

#1 2017-11-02 13:18:56

[SOLVED] Lost connectivity on resume with netctl-auto

#2 2017-11-02 14:05:42

Re: [SOLVED] Lost connectivity on resume with netctl-auto

#3 2017-11-02 14:25:52

Re: [SOLVED] Lost connectivity on resume with netctl-auto

#4 2017-11-02 19:01:38

Re: [SOLVED] Lost connectivity on resume with netctl-auto

#5 2017-11-02 19:54:38

Re: [SOLVED] Lost connectivity on resume with netctl-auto

#6 2017-11-03 00:05:08

Re: [SOLVED] Lost connectivity on resume with netctl-auto

#7 2017-11-13 09:22:21

Re: [SOLVED] Lost connectivity on resume with netctl-auto

Board footer