You are not logged in.
After upgrading to Linux 6.8 (including the most recent 6.8.2.arch2-1), I noticed that the system hangs when swap is actively used.
My swap partition is encrypted with a random password as described in the wiki: https://wiki.archlinux.org/title/Dm-cry … sk_support
* I tried to use an unencrypted swap file on the same disk — the system does NOT hang (i.e. my disk is healthy and is probably not the cause of the hang).
* I tried to downgrade to linux 6.6.23-1-lts — the system does NOT hang.
Given these two facts, I suspect there is a bug in the 6.8 kernel, but I don't know how to properly describe and report it, so I decided to whine on this forum instead. Maybe people more knowledgeable than me can figure it out.
dmesg:
Mar 29 18:16:19 home kernel: INFO: task kswapd0:62 blocked for more than 122 seconds.
Mar 29 18:16:20 home kernel: Tainted: P OE 6.8.2-arch2-1 #1
Mar 29 18:16:38 home kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Mar 29 18:16:38 home kernel: task:kswapd0 state:D stack:0 pid:62 tgid:62 ppid:2 flags:0x00004000
Mar 29 18:16:38 home kernel: Call Trace:
Mar 29 18:16:38 home kernel: <TASK>
Mar 29 18:16:39 home kernel: __schedule+0x3e6/0x1520
Mar 29 18:16:39 home kernel: schedule+0x32/0xd0
Mar 29 18:16:39 home kernel: schedule_timeout+0x151/0x160
Mar 29 18:16:39 home kernel: __down_common+0x14e/0x270
Mar 29 18:16:39 home kernel: down+0x47/0x60
Mar 29 18:16:39 home kernel: __map_bio+0xb6/0x1e0 [dm_mod 3b1a0d62406cfe5a1c5b69746e4723d34150ba76]
Mar 29 18:16:40 home kernel: dm_submit_bio+0x1c3/0x580 [dm_mod 3b1a0d62406cfe5a1c5b69746e4723d34150ba76]
Mar 29 18:16:40 home kernel: __submit_bio+0xb0/0x170
Mar 29 18:16:41 home kernel: submit_bio_noacct_nocheck+0x159/0x370
Mar 29 18:16:41 home kernel: ? submit_bio_noacct+0x49/0x580
Mar 29 18:16:41 home kernel: zswap_writeback_entry+0xf2/0x180
Mar 29 18:16:42 home kernel: shrink_memcg_cb+0xe7/0x2f0
Mar 29 18:16:42 home kernel: ? xa_load+0x8c/0xe0
Mar 29 18:16:42 home kernel: ? __pfx_shrink_memcg_cb+0x10/0x10
Mar 29 18:16:43 home kernel: __list_lru_walk_one+0xb9/0x1d0
Mar 29 18:16:43 home kernel: ? __pfx_shrink_memcg_cb+0x10/0x10
Mar 29 18:16:43 home kernel: list_lru_walk_one+0x5d/0x90
Mar 29 18:16:44 home kernel: zswap_shrinker_scan+0xc4/0x130
Mar 29 18:16:44 home kernel: do_shrink_slab+0x13f/0x360
Mar 29 18:16:45 home kernel: shrink_slab+0x28e/0x3c0
Mar 29 18:16:45 home kernel: shrink_one+0x123/0x1b0
Mar 29 18:16:45 home kernel: shrink_node+0x97e/0xbc0
Mar 29 18:16:46 home kernel: balance_pgdat+0x523/0x960
Mar 29 18:16:46 home kernel: ? __schedule+0x3ee/0x1520
Mar 29 18:16:47 home kernel: ? __switch_to_asm+0x3e/0x70
Mar 29 18:16:47 home kernel: kswapd+0x20d/0x400
Mar 29 18:16:48 home kernel: ? __pfx_autoremove_wake_function+0x10/0x10
Mar 29 18:16:48 home kernel: ? __pfx_kswapd+0x10/0x10
Mar 29 18:16:48 home kernel: kthread+0xe5/0x120
Mar 29 18:16:49 home kernel: ? __pfx_kthread+0x10/0x10
Mar 29 18:16:49 home kernel: ret_from_fork+0x31/0x50
Mar 29 18:16:50 home kernel: ? __pfx_kthread+0x10/0x10
Mar 29 18:16:50 home kernel: ret_from_fork_asm+0x1b/0x30
Mar 29 18:16:51 home kernel: </TASK>
Mar 29 18:16:51 home kernel: INFO: task kworker/u8:5:20255 blocked for more than 122 seconds.
Mar 29 18:16:51 home kernel: Tainted: P OE 6.8.2-arch2-1 #1
Mar 29 18:16:52 home kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Mar 29 18:16:52 home kernel: task:kworker/u8:5 state:D stack:0 pid:20255 tgid:20255 ppid:2 flags:0x00004000
Mar 29 18:16:52 home kernel: Workqueue: kcryptd/254:0 kcryptd_crypt [dm_crypt]
Mar 29 18:16:53 home kernel: Call Trace:
Mar 29 18:16:53 home kernel: <TASK>
Mar 29 18:16:54 home kernel: __schedule+0x3e6/0x1520
Mar 29 18:16:54 home kernel: ? sysvec_apic_timer_interrupt+0xe/0x90
Mar 29 18:16:54 home kernel: schedule+0x32/0xd0
Mar 29 18:16:55 home kernel: schedule_preempt_disabled+0x15/0x30
Mar 29 18:16:55 home kernel: __mutex_lock.constprop.0+0x39a/0x6a0
Mar 29 18:16:55 home kernel: crypt_alloc_buffer+0x228/0x240 [dm_crypt 6f52c98e1f968012bd75812a29a17dce7ea3edc1]
Mar 29 18:16:56 home kernel: kcryptd_crypt_write_convert+0x91/0x1e0 [dm_crypt 6f52c98e1f968012bd75812a29a17dce7ea3edc1]
Mar 29 18:16:56 home kernel: process_one_work+0x183/0x370
Mar 29 18:16:56 home kernel: worker_thread+0x3ab/0x4f0
Mar 29 18:16:57 home kernel: ? __pfx_worker_thread+0x10/0x10
Mar 29 18:16:57 home kernel: kthread+0xe5/0x120
Mar 29 18:16:57 home kernel: ? __pfx_kthread+0x10/0x10
Mar 29 18:16:58 home kernel: ret_from_fork+0x31/0x50
Mar 29 18:16:58 home kernel: ? __pfx_kthread+0x10/0x10
Mar 29 18:16:58 home kernel: ret_from_fork_asm+0x1b/0x30
Mar 29 18:16:58 home kernel: </TASK>
Mar 29 18:16:58 home kernel: INFO: task kworker/u8:3:23688 blocked for more than 122 seconds.
Mar 29 18:16:59 home kernel: Tainted: P OE 6.8.2-arch2-1 #1
Mar 29 18:16:59 home kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Mar 29 18:16:59 home kernel: task:kworker/u8:3 state:D stack:0 pid:23688 tgid:23688 ppid:2 flags:0x00004000
Mar 29 18:17:00 home kernel: Workqueue: kcryptd/254:0 kcryptd_crypt [dm_crypt]
Mar 29 18:17:00 home kernel: Call Trace:
Mar 29 18:17:00 home kernel: <TASK>
Mar 29 18:17:08 home kernel: __schedule+0x3e6/0x1520
Mar 29 18:17:08 home kernel: schedule+0x32/0xd0
Mar 29 18:17:08 home kernel: schedule_preempt_disabled+0x15/0x30
Mar 29 18:17:08 home kernel: __mutex_lock.constprop.0+0x39a/0x6a0
Mar 29 18:17:08 home kernel: crypt_alloc_buffer+0x228/0x240 [dm_crypt 6f52c98e1f968012bd75812a29a17dce7ea3edc1]
Mar 29 18:17:08 home kernel: kcryptd_crypt_write_convert+0x91/0x1e0 [dm_crypt 6f52c98e1f968012bd75812a29a17dce7ea3edc1]
Mar 29 18:17:08 home kernel: process_one_work+0x183/0x370
Mar 29 18:17:08 home kernel: worker_thread+0x3ab/0x4f0
Mar 29 18:17:08 home kernel: ? __pfx_worker_thread+0x10/0x10
Mar 29 18:17:08 home kernel: kthread+0xe5/0x120
Mar 29 18:17:08 home kernel: ? __pfx_kthread+0x10/0x10
Mar 29 18:17:08 home kernel: ret_from_fork+0x31/0x50
Mar 29 18:17:08 home kernel: ? __pfx_kthread+0x10/0x10
Mar 29 18:17:08 home kernel: ret_from_fork_asm+0x1b/0x30
Mar 29 18:17:08 home kernel: </TASK>
Mar 29 18:17:08 home kernel: INFO: task kworker/u8:0:25663 blocked for more than 122 seconds.
Mar 29 18:17:08 home kernel: Tainted: P OE 6.8.2-arch2-1 #1
Mar 29 18:17:08 home kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Mar 29 18:17:08 home kernel: task:kworker/u8:0 state:D stack:0 pid:25663 tgid:25663 ppid:2 flags:0x00004000
Mar 29 18:17:08 home kernel: Workqueue: kcryptd/254:0 kcryptd_crypt [dm_crypt]
Mar 29 18:17:08 home kernel: Call Trace:
Mar 29 18:17:08 home kernel: <TASK>
Mar 29 18:17:08 home kernel: __schedule+0x3e6/0x1520
Mar 29 18:17:08 home kernel: schedule+0x32/0xd0
Mar 29 18:17:08 home kernel: schedule_timeout+0x151/0x160
Mar 29 18:17:08 home kernel: __down_common+0x14e/0x270
Mar 29 18:17:08 home kernel: down+0x47/0x60
Mar 29 18:17:08 home kernel: __map_bio+0xb6/0x1e0 [dm_mod 3b1a0d62406cfe5a1c5b69746e4723d34150ba76]
Mar 29 18:17:09 home kernel: dm_submit_bio+0x1c3/0x580 [dm_mod 3b1a0d62406cfe5a1c5b69746e4723d34150ba76]
Mar 29 18:17:09 home kernel: __submit_bio+0xb0/0x170
Mar 29 18:17:09 home kernel: submit_bio_noacct_nocheck+0x159/0x370
Mar 29 18:17:09 home kernel: ? submit_bio_noacct+0x49/0x580
Mar 29 18:17:09 home kernel: zswap_writeback_entry+0xf2/0x180
Mar 29 18:17:09 home kernel: shrink_memcg_cb+0xe7/0x2f0
Mar 29 18:17:10 home kernel: ? xa_load+0x8c/0xe0
Mar 29 18:17:10 home kernel: ? __pfx_shrink_memcg_cb+0x10/0x10
Mar 29 18:17:10 home kernel: __list_lru_walk_one+0xb9/0x1d0
Mar 29 18:17:10 home kernel: ? __pfx_shrink_memcg_cb+0x10/0x10
Mar 29 18:17:10 home kernel: list_lru_walk_one+0x5d/0x90
Mar 29 18:17:11 home kernel: zswap_shrinker_scan+0xc4/0x130
Mar 29 18:17:11 home kernel: do_shrink_slab+0x13f/0x360
Mar 29 18:17:11 home kernel: shrink_slab+0x28e/0x3c0
Mar 29 18:17:11 home kernel: shrink_one+0x123/0x1b0
Mar 29 18:17:11 home kernel: shrink_node+0x97e/0xbc0
Mar 29 18:17:12 home kernel: do_try_to_free_pages+0xe7/0x5b0
Mar 29 18:17:12 home kernel: try_to_free_pages+0xe1/0x200
Mar 29 18:17:12 home kernel: __alloc_pages_slowpath.constprop.0+0x343/0xde0
Mar 29 18:17:13 home kernel: ? cbc_encrypt+0x97/0xd0 [aesni_intel 085fbf7cf52ecbd3eb6480274b7e7c625cd6073a]
Mar 29 18:17:13 home kernel: __alloc_pages+0x32d/0x350
Mar 29 18:17:13 home kernel: alloc_pages_mpol+0x95/0x1f0
Mar 29 18:17:14 home kernel: ? bio_associate_blkg_from_css+0xf8/0x340
Mar 29 18:17:14 home kernel: crypt_page_alloc+0x2e/0x60 [dm_crypt 6f52c98e1f968012bd75812a29a17dce7ea3edc1]
Mar 29 18:17:14 home kernel: mempool_alloc+0x86/0x1b0
Mar 29 18:17:15 home kernel: crypt_alloc_buffer+0x168/0x240 [dm_crypt 6f52c98e1f968012bd75812a29a17dce7ea3edc1]
Mar 29 18:17:15 home kernel: kcryptd_crypt_write_convert+0x91/0x1e0 [dm_crypt 6f52c98e1f968012bd75812a29a17dce7ea3edc1]
Mar 29 18:17:15 home kernel: process_one_work+0x183/0x370
Mar 29 18:17:16 home kernel: worker_thread+0x3ab/0x4f0
Mar 29 18:17:16 home kernel: ? __pfx_worker_thread+0x10/0x10
Mar 29 18:17:16 home kernel: kthread+0xe5/0x120
Mar 29 18:17:16 home kernel: ? __pfx_kthread+0x10/0x10
Mar 29 18:17:16 home kernel: ret_from_fork+0x31/0x50
Mar 29 18:17:17 home kernel: ? __pfx_kthread+0x10/0x10
Mar 29 18:17:17 home kernel: ret_from_fork_asm+0x1b/0x30
Mar 29 18:17:17 home kernel: </TASK>
Mar 29 18:17:17 home kernel: INFO: task kworker/u8:1:27614 blocked for more than 122 seconds.
Mar 29 18:17:18 home kernel: Tainted: P OE 6.8.2-arch2-1 #1
Mar 29 18:17:18 home kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Mar 29 18:17:18 home kernel: task:kworker/u8:1 state:D stack:0 pid:27614 tgid:27614 ppid:2 flags:0x00004000
Mar 29 18:17:18 home kernel: Workqueue: kcryptd/254:0 kcryptd_crypt [dm_crypt]
Mar 29 18:17:19 home kernel: Call Trace:
Mar 29 18:17:19 home kernel: <TASK>
Mar 29 18:17:19 home kernel: __schedule+0x3e6/0x1520
Mar 29 18:17:19 home kernel: schedule+0x32/0xd0
Mar 29 18:17:20 home kernel: schedule_preempt_disabled+0x15/0x30
Mar 29 18:17:20 home kernel: __mutex_lock.constprop.0+0x39a/0x6a0
Mar 29 18:17:20 home kernel: crypt_alloc_buffer+0x228/0x240 [dm_crypt 6f52c98e1f968012bd75812a29a17dce7ea3edc1]
Mar 29 18:17:20 home kernel: kcryptd_crypt_write_convert+0x91/0x1e0 [dm_crypt 6f52c98e1f968012bd75812a29a17dce7ea3edc1]
Mar 29 18:17:20 home kernel: process_one_work+0x183/0x370
Mar 29 18:17:21 home kernel: worker_thread+0x3ab/0x4f0
Mar 29 18:17:21 home kernel: ? __pfx_worker_thread+0x10/0x10
Mar 29 18:17:21 home kernel: kthread+0xe5/0x120
Mar 29 18:17:21 home kernel: ? __pfx_kthread+0x10/0x10
Mar 29 18:17:22 home kernel: ret_from_fork+0x31/0x50
Mar 29 18:17:22 home kernel: ? __pfx_kthread+0x10/0x10
Mar 29 18:17:22 home kernel: ret_from_fork_asm+0x1b/0x30
Mar 29 18:17:22 home kernel: </TASK>
Last edited by andreymal (2024-04-14 20:17:18)
Offline
It starts w/ zswap writebacks, so you might want the check whether that's the issue, https://wiki.archlinux.org/title/Zswap#Toggling_zswap
Did you check whether an unencrypted swap /partition/ still works?
Offline
With zswap disabled, it doesn't hang.
With an unencrypted swap partition, it doesn't hang (I just reused the same partition I use for encrypted swap).
So it seems that only the combination of zswap + encrypted swap causes it to hang
Offline
You're not alone …
https://bbs.archlinux.org/viewtopic.php?id=294481
Offline
Probably fixed in 6.8.3-arch1-1 (at least I can't manually trigger the hang). If it doesn't hang again, I'll mark this thread as solved
Offline
6.8.5-arch1-1 doesn't hang, nice
Offline