[SOLVED] Kernel stack is corrupted in skb_find_text

WFV · 2021-03-01 15:46:32

Update from 5.10.16 to 5.11.2, boot halts on kernel panic. Section below taken from photograph as it doesn't record in the journal after starting display manager:

[ OK ] Started LXDE Display Manager.
[   6.774078] Kernel panic - not syncing: stack-protector: Kernel stack is corrupted in: skb_find_text+0xd3/0xe0
[   6.775150] CPU: 1 PID: 560 Comm: lxdm-binary Tainted: G	OE	5.11.2-arch1-1 #1
[   6.776208] Hardware name: System manufacturer System Product Name/M5A88-M, BIOS 1702	05/01/2013
[   6.777293] Call Trace:
[   6.778373] dump_stack+0x6b/0x83
[   6.779456] panic+0x112/0x2e8
[   6.780540] ? kmp_find+0x58/0x160 [ts_kmp]
[   6.781609] ? skb_find_text+0xd3/0xe0
[   6.782556] __stack_chk_fail+0x10/0x10
[   6.783582] skb_find_text+0xd3/0xe0
[   6.784619] string_mt+0x24/0x31 [xt_string]
[   6.785651] ipt_do_table+0x28b/0x670 [ip_tables]
[   6.786709] nf_hook_slow+0x3f/0xb0
[   6.787764] __ip_local_out+oxf1/0x170
[   6.788824] ? ip_forward_options+0x180/0x180
[   6.789884] __ip_queue_xmit+0x163/0x410
[   6.790930] __tcp_transmit_skb+0xaaa/0xc80
[   6.791966] tcp_connect+0xb03/0xe60
[   6.792998] tcp_v4_connect+0x3f4/0x500
[   6.794036] __inet_stream_connect+0xc5/0x380
[   6.795072] ? security_file_alloc+0x48/0x90
[   6.796116] ? __alloc_file+0x89/0xe0
[   6.797177] inet_stream_connect+0x37/0x50
[   6.798233] __sys_connect+0xac/0xe0
[   6.799294] __x64_sys_connect+0x16/0x20
[   6.800349] do_syscall_64+0x33/0x40
[   6.801381] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[   6.802422] RIP: 0033:0x7f94e22a82f7
[   6.803462] Code: 64 89 01 48 83 c8 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0 75 10 b8 2a 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 51 c3 48 83 cc 18 89 54 24 ... #runs off camera screen...
[   6.804612] RSP: 002b:00007ffefe1f2d48 EFLAGS: 00000246 ORIG_RAX: 000000000000002a
[   6.805782] RAX: ffffffffffffffda RBX: 00007ffefe1f2d54 RCX: 00007f94e22a82f7
[   6.806969] RDX: 0000000000000010 RSI: 00005573b9dd4ce0 RDI: 0000000000000006
[   6.808162] RBP: 00005573b9dd4cb0 R08: 0000000000000004 R09: 00007f94e22a8210
[   6.809365] R10: 00007ffefe1f2d54 R11: 0000000000000246 R12: 0000000000000006
[   6.810545] R13: 00005573b9dd4ce0 R14: 0000000000000010 R15: 0000000000000006
[   6.811723] Kernel Offset: 0x5000000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff)
[   6.812613] ---[ end Kernel panci - not syncing: stack-protector: Kernel stack is corrupted in: skb_find_text+0xd3/0xe0 ]---
_

Host machine panics. Arch Virtualbox guest doesn't panic running 5.11.0 from testing nor updated to 5.11.2. Understood VBox guest isn't apples-for-apples comparison.
Rollback host to 5.10.16 no problem, and no problem with lts 5.10.19. Reviewed this bug: https://bugzilla.kernel.org/show_bug.cgi?id=211937
and this information about skb_find_text:
https://lwn.net/Articles/141166/
http://lkml.iu.edu/hypermail/linux/kern … 02130.html
https://github.com/torvalds/linux/blob/ … x/skbuff.h
Several mentions around iptables in the panic, is that where skb_find_text is failing?
The second information link (from 2015) use of "--to" in iptable rules indicates it could cause problem, which currently I have not a few such entries.
Thank you.

Last edited by WFV (2021-03-22 22:19:54)

loqs · 2021-03-02 14:19:11

WFV wrote:

Several mentions around iptables in the panic, is that where skb_find_text is failing?

Kernel netfilter code which is used by iptables.
You could try linux-mainline 5.12-rc1 from Unofficial_user_repositories#miffe. You could bisect between 5.10 and 5.11 to find the causal commit.

WFV · 2021-03-02 22:23:22

Thank you loqs. Maybe test 5.12-rc1 tomorrow. Bisect I've never done but am interested in learning however, have to wait to end of month as will be away from machine (work).
Temporary workaround, removed the instances of "--to" from iptables and no more kernel panic with 5.11.2 (have been reading up on iptables and a lot more reading/learning to do).

EDIT1: same panic with 5.12-rc1 kernel using the original iptables but otherwise no other noticeable issues with that release candidate.
There is also this bug report with research and bisecting.

EDIT2: panics with 5.11.3 testing.

EDIT3: resolved with the patch in the last bug report (somewhere between linux-5.11.3 and 5.11.8)

Last edited by WFV (2021-03-22 22:25:15)

Arch Linux

#1 2021-03-01 15:46:32

[SOLVED] Kernel stack is corrupted in skb_find_text

#2 2021-03-02 14:19:11

Re: [SOLVED] Kernel stack is corrupted in skb_find_text

#3 2021-03-02 22:23:22

Re: [SOLVED] Kernel stack is corrupted in skb_find_text

Board footer