[Solved] iwd 1.22-1+ is segfaulting

I_Use_Arch_BTW45 · 2022-01-27 22:49:42

I haven't seen anybody else with this issue yet, so I haven't posted it in bugs.
here is the dmesg output:
[ 4666.912745] wlan0: deauthenticating from 98:da:c4:83:95:b8 by local choice (Reason: 3=DEAUTH_LEAVING)
[ 4668.645676] traps: iwd[23938] general protection fault ip:7f22855083a3 sp:7ffe636ef1a0 error:0 in libell.so.0.0.2[7f2285506000+4a000]
[ 4669.537862] iwd[23977]: segfault at 556ecb4e7010 ip 0000556ecb4e7010 sp 00007ffe839c4268 error 15
[ 4669.537874] Code: Unable to access opcode bytes at RIP 0x556ecb4e6fe6.
[ 4670.304598] iwd[23991]: segfault at 55832f22f010 ip 000055832f22f010 sp 00007fff11f9f5e8 error 15
[ 4670.304607] Code: Unable to access opcode bytes at RIP 0x55832f22efe6.
[ 4671.094159] iwd[24013]: segfault at 560c5570b010 ip 0000560c5570b010 sp 00007ffeb5e5dd98 error 15
[ 4671.094168] Code: Unable to access opcode bytes at RIP 0x560c5570afe6.
[ 4672.037116] traps: iwd[24028] general protection fault ip:7f3c9b8b6e1d sp:7ffd3bcaa1d0 error:0 in libell.so.0.0.2[7f3c9b8ac000+4a000]
iwd 1.21-1 works great, and have not had a single issue with it, I have no idea what happened with this new release.
Solution: I was using IWD with wpa_supplicant and that seems to have caused the crash, I uninstalled iwd and everything now works

Last edited by I_Use_Arch_BTW45 (2022-02-09 20:50:37)

loqs · 2022-01-27 23:57:46

Could you rebuild both iwd and ell with debug symbols? That will not fix the issue but the backtrace would be more useful to upstream.

naruhax · 2022-01-28 13:54:09

I'm facing a similar issue, but might be completely different:
https://bbs.archlinux.org/viewtopic.php … 6#p2018826

mcloaked · 2022-01-28 21:03:33

Is the problem still happening with 1.23 which is now current?

po1o · 2022-01-29 20:53:11

FYI
It was not segfaulting for me on 1.23 (did not try 1.22) but I was getting the following error message when using iwctl as a non-root user.

Rejected send message, 2 matched rules; type="method_call", sender=":1.33" (uid=1000 pid=2821 comm="iwctl ") interface="org.freedesktop.DBus.ObjectManager" member="GetManagedObjects" error name="(unset)" requested_reply="0" destination="net.connman.iwd" (uid=0 pid=2812 comm="/usr/lib/iwd/iwd ")
Failed to retrieve IWD dbus objects, quitting...

Waiting for IWD to start...

Downgrading to 1.21 fixed the issue.

mcloaked · 2022-01-29 21:30:11

I wonder what is causing iwctl to fail in your system - for me all laptops run iwctl without any problem, for all recent versions of iwd, including 1.23.

loqs · 2022-01-29 21:34:47

mcloaked wrote:

Is the problem still happening with 1.23 which is now current?

Should be fixed by https://git.kernel.org/pub/scm/network/ … 4376d0a4b9 which was added in 1.23

mcloaked wrote:

I wonder what is causing iwctl to fail in your system - for me all laptops run iwctl without any problem, for all recent versions of iwd, including 1.23.

https://bbs.archlinux.org/viewtopic.php?id=273625 / https://git.kernel.org/pub/scm/network/ … f92334c293

I_Use_Arch_BTW45 · 2022-01-31 23:02:18

iwd 1.23 is also not working, am part of wheel and netdev
~ iwctl
Terminate
Waiting for IWD to start...
same segfault from before

Last edited by I_Use_Arch_BTW45 (2022-01-31 23:22:10)

loqs · 2022-01-31 23:36:45

@I_Use_Arch_BTW45 have you been able to produce a backtrace with debug symbols?

Astrono2 · 2022-02-02 15:45:26

I had the same problem after updating to iwd 1.23-1. In my case it caused a total system meltdown.

Trying to open the KDE NetworkManager panel froze plasmashell, trying to systemctl restart iwd hanged and then a lot of commands stopped working (I'm guessing because they tried to communicate with systemd and failed(?), including sudo.

I eventually found that systemd had failed to start iwd.service do to a crash, ran iwd and saw the segfault. I managed to downgrade to 1.21, but I'll build 1.23 and try to recreate the crash to get more info.

adbs98 · 2022-02-03 01:42:12

Same here. I had to keep hard rebooting at points to get the chance to downgrade the package

habizzle · 2022-02-03 05:45:45

I had similar problems with system freezes (e.g. sudo or networkctl did not work anymore) after having updated iwd.

This is what I found in my kernel logs:

Feb 03 05:39:10 hannes-schenkervia15pro audit[662]: ANOM_ABEND auid=4294967295 uid=0 gid=0 ses=4294967295 pid=662 comm="iwd" exe="/usr/lib/iwd/iwd" sig=11 res=1
Feb 03 05:39:10 hannes-schenkervia15pro kernel: show_signal_msg: 462 callbacks suppressed
Feb 03 05:39:10 hannes-schenkervia15pro kernel: iwd[662]: segfault at 55db7989da5e ip 000055de22d50105 sp 00007ffe52e794b0 error 4 in iwd[55de22d29000+85000]
Feb 03 05:39:10 hannes-schenkervia15pro kernel: Code: ff 15 1f f9 08 00 58 48 89 ef 5a 5d e9 c4 fd ff ff 0f 1f 40 00 f3 0f 1e fa 41 55 41 54 45 31 e4 55 53 48 83 ec 08 4c 8b 6f c0 <41> 8b 45 08 85 c0 74 13 48 83 c4 08 44 89 e0 5b 5d 41 5c 41 5d c3
Feb 03 05:39:10 hannes-schenkervia15pro kernel: audit: type=1701 audit(1643863150.238:215): auid=4294967295 uid=0 gid=0 ses=4294967295 pid=662 comm="iwd" exe="/usr/lib/iwd/iwd" sig=11 res=1
...
Feb 03 05:39:10 hannes-schenkervia15pro systemd-coredump[1797]: [?] Process 662 (iwd) of user 0 dumped core.
                                                                
                                                                Module linux-vdso.so.1 with build-id bd401e5bd0d642686e3a27e85c9cfb15e287f348
                                                                Module ld-linux-x86-64.so.2 with build-id 040cc3dd10461562f177df39e3be2f3704258c3c
                                                                Module libc.so.6 with build-id 4b406737057708c0e4c642345a703c47a61c73dc
                                                                Module libgcc_s.so.1 with build-id 7f8508bb914546ada778809b64b99d234337d835
                                                                Module libell.so.0 with build-id 964169963666de7b2e9dcda90fedc2d3cac8468e
                                                                Module iwd with build-id 3cd17ecc9328ed3bef7cea0537c3914232911172
                                                                Stack trace of thread 662:
                                                                #0  0x000055de22d50105 n/a (iwd + 0x39105)
                                                                #1  0x000055de22d50a28 n/a (iwd + 0x39a28)
                                                                #2  0x00007fb1287af09b n/a (libell.so.0 + 0x1a09b)
                                                                #3  0x00007fb1287ab741 n/a (libell.so.0 + 0x16741)
                                                                #4  0x00007fb1287aa86d l_main_iterate (libell.so.0 + 0x1586d)
                                                                #5  0x00007fb1287aa93e l_main_run (libell.so.0 + 0x1593e)
                                                                #6  0x00007fb1287aab84 l_main_run_with_signal (libell.so.0 + 0x15b84)
                                                                #7  0x000055de22d2aaba n/a (iwd + 0x13aba)
                                                                #8  0x00007fb1285d5b25 __libc_start_main (libc.so.6 + 0x27b25)
                                                                #9  0x000055de22d2b39e n/a (iwd + 0x1439e)
                                                                ELF object binary architecture: AMD x86-64

After I disabled WiFi in BIOS the system works again as expected and I can at least use ethernet ;-)

Failing IWD version is 1.23-1, Linux kernel is 5.16.5.arch1-1

Last edited by habizzle (2022-02-03 05:47:11)

adbs98 · 2022-02-04 02:49:48

I checked and I am a member of wheel and I do not have the netdev group so that won’t work as a fix. Only option is to stay on 1.21 . Hopefully the next version should be out in a few days and fixes this so I can start upgrading again

Last edited by adbs98 (2022-02-04 02:50:18)

MickeyMouse · 2022-02-04 08:01:29

habizzle wrote:

I had similar problems with system freezes (e.g. sudo or networkctl did not work anymore) after having updated iwd.
After I disabled WiFi in BIOS the system works again as expected and I can at least use ethernet ;-)
Failing IWD version is 1.23-1, Linux kernel is 5.16.5.arch1-1

Same here

adbs98 · 2022-02-05 02:59:10

1.24.1 is out for testing and should be available soon. Hopefully this will fix this issue

tomby · 2022-02-05 14:46:33

I tested 1.24-1 version from testing and is still failing

adbs98 · 2022-02-05 20:04:42

1.24.1 is out. Didn’t see it in the repository yet when I checked this am. I will check again in bit. Don’t have high confidence if Tomby says it didn’t fix

adbs98 · 2022-02-06 02:34:21

This is still occurring and not sure where to go next. So it looks like at least for me after a fresh reboot its good but then doing a suspend and resume it segfaults and stops. Here is the dump I have on this. I am willing to debug this if anyone has any ideas

[ab@ab-arch ~]$ sudo coredumpctl
TIME PID UID GID SIG COREFILE EXE SIZE
Sat 2022-02-05 21:15:32 EST 467 0 0 SIGSEGV present /usr/lib/iwd/iwd 45.1K
[ab@ab-arch ~]$ sudo coredumpctl gdb 467
PID: 467 (iwd)
UID: 0 (root)
GID: 0 (root)
Signal: 11 (SEGV)
Timestamp: Sat 2022-02-05 21:15:32 EST (10min ago)
Command Line: /usr/lib/iwd/iwd
Executable: /usr/lib/iwd/iwd
Control Group: /system.slice/iwd.service
Unit: iwd.service
Slice: system.slice
Boot ID: c94bdd9365504e5e9628a20dd14eaf63
Machine ID: d92798486f32469480af53dd5ee2b928
Hostname: ab-arch
Storage: /var/lib/systemd/coredump/core.iwd.0.c94bdd9365504e5e9628a20dd14eaf63.467.1644113732000000.zst (present)
Disk Size: 45.1K
Message: Process 467 (iwd) of user 0 dumped core.

Module linux-vdso.so.1 with build-id bd401e5bd0d642686e3a27e85c9cfb15e287f348
Module ld-linux-x86-64.so.2 with build-id 040cc3dd10461562f177df39e3be2f3704258c3c
Module libc.so.6 with build-id 4b406737057708c0e4c642345a703c47a61c73dc
Module libgcc_s.so.1 with build-id 7f8508bb914546ada778809b64b99d234337d835
Module libell.so.0 with build-id 964169963666de7b2e9dcda90fedc2d3cac8468e
Module iwd with build-id 34c1d21bea688ec5a2460841a54f592a09e129e0
Stack trace of thread 467:
#0 0x00007f460dde53a3 l_queue_pop_head (libell.so.0 + 0x113a3)
#1 0x000056341a3377e9 n/a (iwd + 0x387e9)
#2 0x00007f460ddee244 n/a (libell.so.0 + 0x1a244)
#3 0x00007f460ddea741 n/a (libell.so.0 + 0x16741)
#4 0x00007f460dde986d l_main_iterate (libell.so.0 + 0x1586d)
#5 0x00007f460dde993e l_main_run (libell.so.0 + 0x1593e)
#6 0x00007f460dde9b84 l_main_run_with_signal (libell.so.0 + 0x15b84)
#7 0x000056341a312aba n/a (iwd + 0x13aba)
#8 0x00007f460dc14b25 __libc_start_main (libc.so.6 + 0x27b25)
#9 0x000056341a31339e n/a (iwd + 0x1439e)
ELF object binary architecture: AMD x86-64

GNU gdb (GDB) 11.2
Copyright (C) 2022 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-pc-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/lib/iwd/iwd...
(No debugging symbols found in /usr/lib/iwd/iwd)
[New LWP 467]
Core was generated by `/usr/lib/iwd/iwd'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0 0x00007f460dde53a3 in l_queue_pop_head () from /usr/lib/libell.so.0

Last edited by adbs98 (2022-02-06 02:34:52)

adbs98 · 2022-02-06 03:03:04

I submitted a bug for this
FS#73641 - (iwd) of user 0 dumped core.

Please upvote so we can get more attention on this

I_Use_Arch_BTW45 · 2022-02-07 01:49:05

maybe this patch caused it?
https://git.kernel.org/pub/scm/network/ … 35aabeb0b5

loqs · 2022-02-07 02:00:30

I_Use_Arch_BTW45 wrote:

maybe this patch caused it?
https://git.kernel.org/pub/scm/network/ … 35aabeb0b5

Does the backtrace you have not posted match up with changes in the commit you linked?

adbs98 · 2022-02-07 04:33:57

So did a little work on this but didn’t make much progress. Part of the issue with getting a backtrack with symbols is when this occurs and it core dumps it brings my system to its knees and commands like sudo just sit. Hard reboot is the only option at that stage.

Seems to work ok for some time but will seem to fail at random on a suspend and a reboot doesn’t fix. The only way to straighten it out is to disable wireless and reboot and the reenable. This is my work laptop so I had to downgrade back to 1.21 so I can work this week. I am also on call at work the next 7 days and will need a stable system so I wont have time work to get this to pull a backtrace anytime soon. Hopefully someone can get a backtrace soon.

I_Use_Arch_BTW45 · 2022-02-07 13:31:44

adbs98 wrote:

So did a little work on this but didn’t make much progress. Part of the issue with getting a backtrack with symbols is when this occurs and it core dumps it brings my system to its knees and commands like sudo just sit. Hard reboot is the only option at that stage.
Seems to work ok for some time but will seem to fail at random on a suspend and a reboot doesn’t fix. The only way to straighten it out is to disable wireless and reboot and the reenable. This is my work laptop so I had to downgrade back to 1.21 so I can work this week. I am also on call at work the next 7 days and will need a stable system so I wont have time work to get this to pull a backtrace anytime soon. Hopefully someone can get a backtrace soon.

same for me, if I try switching iwd versions without rebooting, I legit cannot run or use anything

I_Use_Arch_BTW45 · 2022-02-07 13:33:07

loqs wrote:

I_Use_Arch_BTW45 wrote:
maybe this patch caused it?
https://git.kernel.org/pub/scm/network/ … 35aabeb0b5
Does the backtrace you have not posted match up with changes in the commit you linked?

this is just a guess rn, I'll try making a build with debug symbols
since its the only thing that has been changed with libell
and also @ads98 a stack trace without debug symbols is pretty useless btw

Last edited by I_Use_Arch_BTW45 (2022-02-07 13:45:59)

adbs98 · 2022-02-07 16:30:50

Here is the stack trace from yesterday that I forgot to post. Not sure if this would give you any more info

Program terminated with signal SIGSEGV, Segmentation fault.
#0 0x00007fbe1de443a3 in l_queue_pop_head () from /usr/lib/libell.so.0
(gdb) bt
#0 0x00007fbe1de443a3 in l_queue_pop_head () from /usr/lib/libell.so.0
#1 0x0000559ac87367e9 in ?? ()
#2 0x00007fbe1de4d244 in ?? () from /usr/lib/libell.so.0
#3 0x00007fbe1de49741 in ?? () from /usr/lib/libell.so.0
#4 0x00007fbe1de4886d in l_main_iterate () from /usr/lib/libell.so.0
#5 0x00007fbe1de4893e in l_main_run () from /usr/lib/libell.so.0
#6 0x00007fbe1de48b84 in l_main_run_with_signal () from /usr/lib/libell.so.0
#7 0x0000559ac8711aba in ?? ()
#8 0x00007fbe1dc73b25 in __libc_start_main () from /usr/lib/libc.so.6
#9 0x0000559ac871239e in ?? ()
(gdb) disas
Dump of assembler code for function l_queue_pop_head:
0x00007fbe1de44390 <+0>: endbr64
0x00007fbe1de44394 <+4>: push %r12
0x00007fbe1de44396 <+6>: push %rbx
0x00007fbe1de44397 <+7>: sub $0x8,%rsp
0x00007fbe1de4439b <+11>: test %rdi,%rdi
0x00007fbe1de4439e <+14>: je 0x7fbe1de443e8 <l_queue_pop_head+88>
0x00007fbe1de443a0 <+16>: mov %rdi,%rbx
=> 0x00007fbe1de443a3 <+19>: mov (%rdi),%rdi
0x00007fbe1de443a6 <+22>: test %rdi,%rdi
0x00007fbe1de443a9 <+25>: je 0x7fbe1de443e8 <l_queue_pop_head+88>
0x00007fbe1de443ab <+27>: mov 0x8(%rdi),%rax
0x00007fbe1de443af <+31>: test %rax,%rax
0x00007fbe1de443b2 <+34>: je 0x7fbe1de443d0 <l_queue_pop_head+64>
0x00007fbe1de443b4 <+36>: mov %rax,(%rbx)
0x00007fbe1de443b7 <+39>: mov (%rdi),%r12
0x00007fbe1de443ba <+42>: call *0x64b28(%rip) # 0x7fbe1dea8ee8
0x00007fbe1de443c0 <+48>: subl $0x1,0x10(%rbx)
0x00007fbe1de443c4 <+52>: add $0x8,%rsp
0x00007fbe1de443c8 <+56>: mov %r12,%rax
0x00007fbe1de443cb <+59>: pop %rbx
0x00007fbe1de443cc <+60>: pop %r12
0x00007fbe1de443ce <+62>: ret
0x00007fbe1de443cf <+63>: nop
0x00007fbe1de443d0 <+64>: movq $0x0,(%rbx)
0x00007fbe1de443d7 <+71>: movq $0x0,0x8(%rbx)
0x00007fbe1de443df <+79>: jmp 0x7fbe1de443b7 <l_queue_pop_head+39>
0x00007fbe1de443e1 <+81>: nopl 0x0(%rax)
0x00007fbe1de443e8 <+88>: xor %r12d,%r12d
0x00007fbe1de443eb <+91>: add $0x8,%rsp
0x00007fbe1de443ef <+95>: mov %r12,%rax
0x00007fbe1de443f2 <+98>: pop %rbx
0x00007fbe1de443f3 <+99>: pop %r12
0x00007fbe1de443f5 <+101>: ret
End of assembler dump.
(gdb) i r
rax 0x0 0
rbx 0x1100730103 73021980931
rcx 0x2 2
rdx 0x559ac8c44450 94123281630288
rsi 0x559ac8c4fd50 94123281677648
rdi 0x1100730103 73021980931
rbp 0x559ac8c503e0 0x559ac8c503e0
rsp 0x7fff61d60ed0 0x7fff61d60ed0
r8 0x559ac8c58330 94123281711920
r9 0x0 0
r10 0xffffffff 4294967295
r11 0x246 582
r12 0x559ac8c4fd50 94123281677648
r13 0x559ac8c5aeb0 94123281723056
r14 0x559ac8c5aeb0 94123281723056
r15 0x559ac87aa7c5 94123276806085
rip 0x7fbe1de443a3 0x7fbe1de443a3 <l_queue_pop_head+19>
eflags 0x10206 [ PF IF RF ]
cs 0x33 51
ss 0x2b 43
ds 0x0 0
es 0x0 0
fs 0x0 0
gs 0x0 0
(gdb)
rax 0x0 0
rbx 0x1100730103 73021980931
rcx 0x2 2
rdx 0x559ac8c44450 94123281630288
rsi 0x559ac8c4fd50 94123281677648
rdi 0x1100730103 73021980931
rbp 0x559ac8c503e0 0x559ac8c503e0
rsp 0x7fff61d60ed0 0x7fff61d60ed0
r8 0x559ac8c58330 94123281711920
r9 0x0 0
r10 0xffffffff 4294967295
r11 0x246 582
r12 0x559ac8c4fd50 94123281677648
r13 0x559ac8c5aeb0 94123281723056
r14 0x559ac8c5aeb0 94123281723056
r15 0x559ac87aa7c5 94123276806085
rip 0x7fbe1de443a3 0x7fbe1de443a3 <l_queue_pop_head+19>
eflags 0x10206 [ PF IF RF ]
cs 0x33 51
ss 0x2b 43
ds 0x0 0
es 0x0 0
fs 0x0 0
gs 0x0 0

Arch Linux

#1 2022-01-27 22:49:42

[Solved] iwd 1.22-1+ is segfaulting

#2 2022-01-27 23:57:46

Re: [Solved] iwd 1.22-1+ is segfaulting

#3 2022-01-28 13:54:09

Re: [Solved] iwd 1.22-1+ is segfaulting

#4 2022-01-28 21:03:33

Re: [Solved] iwd 1.22-1+ is segfaulting

#5 2022-01-29 20:53:11

Re: [Solved] iwd 1.22-1+ is segfaulting

#6 2022-01-29 21:30:11

Re: [Solved] iwd 1.22-1+ is segfaulting

#7 2022-01-29 21:34:47

Re: [Solved] iwd 1.22-1+ is segfaulting

#8 2022-01-31 23:02:18

Re: [Solved] iwd 1.22-1+ is segfaulting

#9 2022-01-31 23:36:45

Re: [Solved] iwd 1.22-1+ is segfaulting

#10 2022-02-02 15:45:26

Re: [Solved] iwd 1.22-1+ is segfaulting

#11 2022-02-03 01:42:12

Re: [Solved] iwd 1.22-1+ is segfaulting

#12 2022-02-03 05:45:45

Re: [Solved] iwd 1.22-1+ is segfaulting

#13 2022-02-04 02:49:48

Re: [Solved] iwd 1.22-1+ is segfaulting

#14 2022-02-04 08:01:29

Re: [Solved] iwd 1.22-1+ is segfaulting

#15 2022-02-05 02:59:10

Re: [Solved] iwd 1.22-1+ is segfaulting

#16 2022-02-05 14:46:33

Re: [Solved] iwd 1.22-1+ is segfaulting

#17 2022-02-05 20:04:42

Re: [Solved] iwd 1.22-1+ is segfaulting

#18 2022-02-06 02:34:21

Re: [Solved] iwd 1.22-1+ is segfaulting

#19 2022-02-06 03:03:04

Re: [Solved] iwd 1.22-1+ is segfaulting

#20 2022-02-07 01:49:05

Re: [Solved] iwd 1.22-1+ is segfaulting

#21 2022-02-07 02:00:30

Re: [Solved] iwd 1.22-1+ is segfaulting

#22 2022-02-07 04:33:57

Re: [Solved] iwd 1.22-1+ is segfaulting

#23 2022-02-07 13:31:44

Re: [Solved] iwd 1.22-1+ is segfaulting

#24 2022-02-07 13:33:07

Re: [Solved] iwd 1.22-1+ is segfaulting

#25 2022-02-07 16:30:50

Re: [Solved] iwd 1.22-1+ is segfaulting

Board footer