[Solved] iwd 1.22-1+ is segfaulting

loqs · 2022-02-07 17:19:10

If you install the iwd-debug that will provide the symbols matching iwd iwd 1.24-2 from community

pacman -U https://america.mirror.pkgbuild.com/community-debug/os/x86_64/iwd-debug-1.24-2-x86_64.pkg.tar.zst

Hopefully that will fill in

#1  0x0000559ac87367e9 in ?? ()

I_Use_Arch_BTW45 · 2022-02-07 22:23:27

loqs wrote:

If you install the iwd-debug that will provide the symbols matching iwd iwd 1.24-2 from community
pacman -U https://america.mirror.pkgbuild.com/community-debug/os/x86_64/iwd-debug-1.24-2-x86_64.pkg.tar.zst
Hopefully that will fill in
#1  0x0000559ac87367e9 in ?? ()

finally debug packages, honestly I was just waiting for the iwd debug package to arrive lol

I_Use_Arch_BTW45 · 2022-02-07 23:38:01

YES, something actually useful??
#0 0x00007ffff7f0f3ab in l_queue_pop_head () from /usr/lib/libell.so.0
#1 0x000055555558c7e9 in scan_request_triggered (msg=0x5555556384c0, userdata=0x555555639c00) at src/scan.c:256
#2 0x00007ffff7f18244 in ?? () from /usr/lib/libell.so.0
#3 0x00007ffff7f14741 in ?? () from /usr/lib/libell.so.0
#4 0x00007ffff7f1386d in l_main_iterate () from /usr/lib/libell.so.0
#5 0x00007ffff7f1393e in l_main_run () from /usr/lib/libell.so.0
#6 0x00007ffff7f13b84 in l_main_run_with_signal () from /usr/lib/libell.so.0
#7 0x0000555555567aba in main (argc=<optimized out>, argv=<optimized out>) at src/main.c:532

I_Use_Arch_BTW45 · 2022-02-07 23:49:52

maybe this commit fixed it, since this is my main machine, I don't want to try iwd-git
https://git.kernel.org/pub/scm/network/ … a6c0f5718c
EDIT: not it, was reading the backtrace wrong (bottom -> top instead of top->bottom)
so basically we have to look for all the commits that changed something related to scan.c?

Last edited by I_Use_Arch_BTW45 (2022-02-07 23:57:54)

loqs · 2022-02-08 00:27:54

I_Use_Arch_BTW45 wrote:

so basically we have to look for all the commits that changed something related to scan.c?

https://git.kernel.org/pub/scm/network/ … 86b8a9f022 may address the issue

As ell has not been rebuilt with debug symbols yet. Please try the following

# pacman -S devtools git # Install need packages to obtain PKGBUILD and build in a clean chroot
git clone https://github.com/archlinux/svntogit-packages.git --single-branch --branch "packages/ell" ell
cd ell/trunk

change line 14 of PKGBUILD from

options=('!lto')

to

options=('!lto' 'debug')

then

extra-x86_64-build # build package in a clean chroot
# pacman -U ell-0.48-1-x86_64.pkg.tar.zst pacman -U ell-debug-0.48-1-x86_64.pkg.tar.zst # install newly built packages

With ell-debug installed all the backtrace entries then hopefully have file and function names.
Edit:
The following builds iwd with the two new commits applied

git clone https://github.com/archlinux/svntogit-community.git --single-branch --branch "packages/iwd" iwd
cd iwd/trunk
curl -o PKGBUILD.diff http://ix.io/3OZm # download diff of changes to PKGBUILD
git apply -v PKGBUILD.diff
extra-x86_64-build
# pacman -U iwd-1.24-2.1-x86_64.pkg.tar.zst iwd-debug-1.24-2.1-x86_64.pkg.tar.zst

Last edited by loqs (2022-02-08 00:43:39)

jprestwo · 2022-02-08 00:43:40

I_Use_Arch_BTW45 wrote:

YES, something actually useful??
#0 0x00007ffff7f0f3ab in l_queue_pop_head () from /usr/lib/libell.so.0
#1 0x000055555558c7e9 in scan_request_triggered (msg=0x5555556384c0, userdata=0x555555639c00) at src/scan.c:256
#2 0x00007ffff7f18244 in ?? () from /usr/lib/libell.so.0
#3 0x00007ffff7f14741 in ?? () from /usr/lib/libell.so.0
#4 0x00007ffff7f1386d in l_main_iterate () from /usr/lib/libell.so.0
#5 0x00007ffff7f1393e in l_main_run () from /usr/lib/libell.so.0
#6 0x00007ffff7f13b84 in l_main_run_with_signal () from /usr/lib/libell.so.0
#7 0x0000555555567aba in main (argc=<optimized out>, argv=<optimized out>) at src/main.c:532

This was somewhat more helpful but a debug log from IWD would also help a great deal to see what exactly happened prior to the crash. We did just fix a scanning crash (today) but its not quite the same as what I'm seeing here. Its upstream so you could give it a try, but ultimately I'll need a debug log to figure this out.

Also we (the devs) will see these issues much faster posting to the IWD mailing list. Especially if its a crash we want to know about these things right away, and usually they get fixed very quickly.

I_Use_Arch_BTW45 · 2022-02-08 12:16:56

jprestwo wrote:

I_Use_Arch_BTW45 wrote:
YES, something actually useful??
#0 0x00007ffff7f0f3ab in l_queue_pop_head () from /usr/lib/libell.so.0
#1 0x000055555558c7e9 in scan_request_triggered (msg=0x5555556384c0, userdata=0x555555639c00) at src/scan.c:256
#2 0x00007ffff7f18244 in ?? () from /usr/lib/libell.so.0
#3 0x00007ffff7f14741 in ?? () from /usr/lib/libell.so.0
#4 0x00007ffff7f1386d in l_main_iterate () from /usr/lib/libell.so.0
#5 0x00007ffff7f1393e in l_main_run () from /usr/lib/libell.so.0
#6 0x00007ffff7f13b84 in l_main_run_with_signal () from /usr/lib/libell.so.0
#7 0x0000555555567aba in main (argc=<optimized out>, argv=<optimized out>) at src/main.c:532
This was somewhat more helpful but a debug log from IWD would also help a great deal to see what exactly happened prior to the crash. We did just fix a scanning crash (today) but its not quite the same as what I'm seeing here. Its upstream so you could give it a try, but ultimately I'll need a debug log to figure this out.
Also we (the devs) will see these issues much faster posting to the IWD mailing list. Especially if its a crash we want to know about these things right away, and usually they get fixed very quickly.

this happened right after booting up

jprestwo · 2022-02-08 16:32:42

This is actually a back trace, i.e. where IWD was when it crashed. What I need is a debug log, anything IWD had printed during its entire runtime.

https://iwd.wiki.kernel.org/debugging

Basically run IWD with the -d flag. If you're using systemd you'll probably need to change your unit file to pass the -d flag. You can see logs (with systemd) using journalctl:

journalctl -xe -u iwd

I_Use_Arch_BTW45 · 2022-02-09 00:30:36

Wireless daemon version 1.24
src/main.c:main() Using configuration directory /etc/iwd
src/storage.c:storage_create_dirs() Using state directory /var/lib/iwd
src/main.c:nl80211_appeared() Found nl80211 interface
src/module.c:iwd_modules_init()
station: Network configuration is disabled.
src/knownnetworks.c:known_network_frequencies_load() No known frequency file found.
src/wsc.c:wsc_init()
src/eap.c:__eap_method_enable()
src/eap-wsc.c:eap_wsc_init()
src/eap-sim.c:eap_sim_init()
src/eap-mschapv2.c:eap_mschapv2_init()
src/eap-ttls.c:eap_ttls_init()
src/eap-tls.c:eap_tls_init()
src/eap-md5.c:eap_md5_init()
src/eap-pwd.c:eap_pwd_init()
src/eap-gtc.c:eap_gtc_init()
src/eap-peap.c:eap_peap_init()
src/eap-aka.c:eap_aka_prime_init()
src/eap-aka.c:eap_aka_init()
rfkill id 0 can't be matched to a wiphy
src/manager.c:manager_wiphy_dump_callback() New wiphy phy0 added (0)
Wiphy: 0, Name: phy0
Permanent Address: 94:08:53:ba:7c:41
2.4Ghz Band:
Bitrates (non-HT):
1.0 Mbps
2.0 Mbps
5.5 Mbps
11.0 Mbps
6.0 Mbps
9.0 Mbps
12.0 Mbps
18.0 Mbps
24.0 Mbps
36.0 Mbps
48.0 Mbps
54.0 Mbps
HT Capabilities:
HT40
Short GI for 20Mhz
Short GI for 40Mhz
HT RX MCS indexes:
0-15
5Ghz Band:
Bitrates (non-HT):
6.0 Mbps
9.0 Mbps
12.0 Mbps
18.0 Mbps
24.0 Mbps
36.0 Mbps
48.0 Mbps
54.0 Mbps
HT Capabilities:
HT40
Short GI for 20Mhz
Short GI for 40Mhz
HT RX MCS indexes:
0-15
VHT Capabilities:
Short GI for 80Mhz
Max RX MCS: 0-9 for NSS: 2
Max TX MCS: 0-9 for NSS: 2
Ciphers: CCMP TKIP BIP
Supported iftypes: ad-hoc station ap p2p-client p2p-go p2p-device
src/manager.c:manager_create_interfaces() creating wlan0
src/manager.c:manager_create_interfaces() creating wlan0-p2p
src/wiphy.c:wiphy_update_reg_domain() New reg domain country code for phy0 is 99
src/manager.c:manager_config_notify() Notification of command New Interface(7)
src/netdev.c:netdev_link_notify() event 16 on ifindex 10
src/manager.c:manager_new_station_interface_cb()
src/netdev.c:netdev_create_from_genl() Created interface wlan0[10 1d]
src/manager.c:manager_config_notify() Notification of command New Interface(7)
src/manager.c:manager_new_p2p_interface_cb()
src/p2p.c:p2p_device_update_from_genl() Created P2P device 1e
src/netdev.c:netdev_link_notify() event 16 on ifindex 10
src/netdev.c:netdev_set_4addr() netdev: 10 use_4addr: 0
src/netdev.c:netdev_initial_up_cb() Interface 10 initialized
src/station.c:station_enter_state() Old State: disconnected, new state: autoconnect_quick
src/scan.c:scan_periodic_start() Starting periodic scan for wdev 1d
src/wiphy.c:wiphy_radio_work_insert() Inserting work item 1
src/wiphy.c:wiphy_radio_work_next() Starting work item 1
src/rrm.c:rrm_add_frame_watches()
src/netdev.c:netdev_link_notify() event 16 on ifindex 10
src/netdev.c:netdev_link_notify() event 16 on ifindex 10
src/manager.c:manager_config_notify() Notification of command Set Interface(6)
src/scan.c:scan_notify() Scan notification Trigger Scan(33)
src/scan.c:scan_request_triggered() Passive scan triggered for wdev 1d
src/scan.c:scan_periodic_triggered() Periodic scan triggered for wdev 1d
src/netdev.c:netdev_link_notify() event 16 on ifindex 10
src/netdev.c:netdev_link_notify() event 16 on ifindex 10
src/station.c:station_free()
src/scan.c:scan_periodic_stop() Stopping periodic scan for wdev 1d
src/scan.c:scan_cancel() Trying to cancel scan id 1 for wdev 1d
src/scan.c:scan_cancel() Scan has been triggered, wait for it to complete
src/station.c:station_roam_state_clear() 10
src/scan.c:scan_notify() Scan notification Scan Aborted(35)
src/wiphy.c:wiphy_radio_work_done() Work item 1 done
src/netdev.c:netdev_link_notify() event 16 on ifindex 10
src/netdev.c:netdev_link_notify() event 16 on ifindex 10
src/station.c:station_enter_state() Old State: disconnected, new state: autoconnect_quick
src/scan.c:scan_periodic_start() Starting periodic scan for wdev 1d
src/wiphy.c:wiphy_radio_work_insert() Inserting work item 2
src/wiphy.c:wiphy_radio_work_next() Starting work item 2
src/scan.c:scan_notify() Scan notification Trigger Scan(33)
src/scan.c:scan_request_triggered() Passive scan triggered for wdev 1d
[1] 61698 segmentation fault sudo /usr/lib/iwd/iwd -d

I_Use_Arch_BTW45 · 2022-02-09 00:33:32

seems to disconnect really frequently for some reason?
sudo wpa_cli status
gives
key_mgmt=WPA2-PSK
as the security mode

loqs · 2022-02-09 00:45:49

wpa_cli is for wpa_supplicant. Are both daemons active on the system at once?
Please use code tags for command, their outputs, listings e.t.c..
The output from 34 is without any patches applied to iwd 1.24?

I_Use_Arch_BTW45 · 2022-02-09 00:46:48

loqs wrote:

wpa_cli is for wpa_supplicant. Are both daemons active on the system at once?
Please use code tags for command, their outputs, listings e.t.c..
The output from 34 is without any patches applied to iwd 1.24?

yes, no patches
lemme check if wpa_supplicant is running
edit: yes, both are active, im guessing you are only supposed to have one.......
wifi with networkmanager dosen't work with only wpa_supplicant though

Last edited by I_Use_Arch_BTW45 (2022-02-09 01:14:32)

jprestwo · 2022-02-09 01:38:53

I_Use_Arch_BTW45 wrote:

loqs wrote:
wpa_cli is for wpa_supplicant. Are both daemons active on the system at once?
Please use code tags for command, their outputs, listings e.t.c..
The output from 34 is without any patches applied to iwd 1.24?
yes, no patches
lemme check if wpa_supplicant is running
edit: yes, both are active, im guessing you are only supposed to have one.......
wifi with networkmanager dosen't work with only wpa_supplicant though

Ah ok. Yes you're going to get strange behavior with both running at the same time. NetworkManager should work by setting the backend to IWD, topic for another discussion. In any case IWD should not be crashing, even with wpa_supplicant running so we need to fix this. Thank you for the debug log, I've got something to go off of now.

I_Use_Arch_BTW45 · 2022-02-09 03:15:10

jprestwo wrote:

I_Use_Arch_BTW45 wrote:
loqs wrote:
wpa_cli is for wpa_supplicant. Are both daemons active on the system at once?
Please use code tags for command, their outputs, listings e.t.c..
The output from 34 is without any patches applied to iwd 1.24?
yes, no patches
lemme check if wpa_supplicant is running
edit: yes, both are active, im guessing you are only supposed to have one.......
wifi with networkmanager dosen't work with only wpa_supplicant though
Ah ok. Yes you're going to get strange behavior with both running at the same time. NetworkManager should work by setting the backend to IWD, topic for another discussion. In any case IWD should not be crashing, even with wpa_supplicant running so we need to fix this. Thank you for the debug log, I've got something to go off of now.

yeah but I cannot use wifi without both running at the same time, How do I fix that?? From your response I guess I have to change the backend to IWD?

Last edited by I_Use_Arch_BTW45 (2022-02-09 03:15:46)

V1del · 2022-02-09 08:15:14

Either that or actually disable iwd, so wpa_supplicant/networkmanager is the only one managing the interface, both running can have adverse effects on both because they try to manipulate the same interface. So "wifi doesn't work with just networkmanager and wpa_supplicant" can be a direct consequence of the simultaneously running iwd, and vice-versa the reason for the iwd crash.

Last edited by V1del (2022-02-09 08:16:22)

I_Use_Arch_BTW45 · 2022-02-09 13:45:35

V1del wrote:

Either that or actually disable iwd, so wpa_supplicant/networkmanager is the only one managing the interface, both running can have adverse effects on both because they try to manipulate the same interface. So "wifi doesn't work with just networkmanager and wpa_supplicant" can be a direct consequence of the simultaneously running iwd, and vice-versa the reason for the iwd crash.

ok I stopped iwd, then restarted wpa_supplicant and networkmanager
surprise, surprise, wifi dosen't work (wifi card not detected by kde)

Last edited by I_Use_Arch_BTW45 (2022-02-09 13:47:23)

V1del · 2022-02-09 13:52:47

Without an actual reboot there are chances your HW/firmware state is completely broken due to the conflicting situation. I'm not aware of any hardware that is supported by iwd but not by wpa_supplicant, they both use the same kernel interfaces at the end of the day. Should this be the case against any odds, you can indeed just do the reverse and configure NetworkManager to use iwd as the backend.

Scimmia · 2022-02-09 13:55:29

Do you actually have wpa_supplicant enabled? If so, that's a problem as well. If you want to use NetworkManager, you enable that and nothing else.

progandy · 2022-02-09 15:04:45

ok I stopped iwd, then restarted wpa_supplicant and networkmanager

iwd deletes the original wifi interface and creates its own. That will be deleted as well when you stop it.
So either reboot with iwd disabled or manually restore the wifi device with "iw phy ..."
https://iwd.wiki.kernel.org/interface_l … ent_in_iwd

Last edited by progandy (2022-02-09 15:05:04)

jprestwo · 2022-02-09 17:15:20

I was unable to reproduce this but the logs are pointing to the same issue fixed here:

https://git.kernel.org/pub/scm/network/ … 86b8a9f022

IWD starts up and begins scanning. Something brings the interface down (could be wpa_supplicant, network manager, or ?). IWD cleans up but leaves a stale scan request in the queue (this is the now fixed bug). Device is brought up again, IWD starts scanning and pops this stale request and crashes.

Anyways this should be fixed, and hopefully nobody else is running into this as its quite rare and shouldn't happen unless the device is brought down at just the right moment. I suggest OP gets his system configuration sorted out, here is some more information about NM + IWD:

https://iwd.wiki.kernel.org/networkmanager

I_Use_Arch_BTW45 · 2022-02-09 17:35:49

ok works now, switched to wpa_supplicant and uninstalled iwd
how do I close this topic?

Last edited by I_Use_Arch_BTW45 (2022-02-09 17:40:26)

mcloaked · 2022-02-09 18:52:48

Go to your first post in the thread and edit the title.

adbs98 · 2022-02-10 03:11:30

sorry I didn't help more with this. Been out of pocket dealing with a family emeregency. I just got a chance to run thru the fix of disabling IWD, rebooting, and then uninstalling IWD and all is well now. Great support and awesome work. I owe a few a beer for this one!!!

tomby · 2022-02-10 09:40:42

I can confirm that switching to wpa_supplicant and removing iwd solves the issue.

progandy · 2022-02-10 12:54:51

tomby wrote:

I can confirm that switching to wpa_supplicant and removing iwd solves the issue.

It doesn't solve the issue, but avoids it. iwd cannot segfault when you do not start it.

Arch Linux

#26 2022-02-07 17:19:10

Re: [Solved] iwd 1.22-1+ is segfaulting

#27 2022-02-07 22:23:27

Re: [Solved] iwd 1.22-1+ is segfaulting

#28 2022-02-07 23:38:01

Re: [Solved] iwd 1.22-1+ is segfaulting

#29 2022-02-07 23:49:52

Re: [Solved] iwd 1.22-1+ is segfaulting

#30 2022-02-08 00:27:54

Re: [Solved] iwd 1.22-1+ is segfaulting

#31 2022-02-08 00:43:40

Re: [Solved] iwd 1.22-1+ is segfaulting

#32 2022-02-08 12:16:56

Re: [Solved] iwd 1.22-1+ is segfaulting

#33 2022-02-08 16:32:42

Re: [Solved] iwd 1.22-1+ is segfaulting

#34 2022-02-09 00:30:36

Re: [Solved] iwd 1.22-1+ is segfaulting

#35 2022-02-09 00:33:32

Re: [Solved] iwd 1.22-1+ is segfaulting

#36 2022-02-09 00:45:49

Re: [Solved] iwd 1.22-1+ is segfaulting

#37 2022-02-09 00:46:48

Re: [Solved] iwd 1.22-1+ is segfaulting

#38 2022-02-09 01:38:53

Re: [Solved] iwd 1.22-1+ is segfaulting

#39 2022-02-09 03:15:10

Re: [Solved] iwd 1.22-1+ is segfaulting

#40 2022-02-09 08:15:14

Re: [Solved] iwd 1.22-1+ is segfaulting

#41 2022-02-09 13:45:35

Re: [Solved] iwd 1.22-1+ is segfaulting

#42 2022-02-09 13:52:47

Re: [Solved] iwd 1.22-1+ is segfaulting

#43 2022-02-09 13:55:29

Re: [Solved] iwd 1.22-1+ is segfaulting

#44 2022-02-09 15:04:45

Re: [Solved] iwd 1.22-1+ is segfaulting

#45 2022-02-09 17:15:20

Re: [Solved] iwd 1.22-1+ is segfaulting

#46 2022-02-09 17:35:49

Re: [Solved] iwd 1.22-1+ is segfaulting

#47 2022-02-09 18:52:48

Re: [Solved] iwd 1.22-1+ is segfaulting

#48 2022-02-10 03:11:30

Re: [Solved] iwd 1.22-1+ is segfaulting

#49 2022-02-10 09:40:42

Re: [Solved] iwd 1.22-1+ is segfaulting

#50 2022-02-10 12:54:51

Re: [Solved] iwd 1.22-1+ is segfaulting

Board footer