Arch Linux

mr.MikyMaus

Member

From: disabled

Registered: 2006-03-31

Posts: 285

"rcu_preempt detected stalls on CPUs/tasks" likely caused by btrfs

Hi,

I have a bit advanced storage configuration relying heavily on btrfs and its cool features:

SSD drive:
sda1: EFI partition
sda2: LUKS partition with BTRFS on it. BTRFS contains root, /var and /home subvolumes, no quotas, 6.5 GB free space, no special configuration
sda4: LUKS partition with SWAP

Hard drives:
sdb2 + sdc3: independent LUKS partitions holding a single BTRFS filesystem configured as data+metadata RAID1, 400 GB free space, 2x1.2TB
sdb3 + sdc4: independent LUKS partitions holding a single BTRFS filesystem configured as data SINGLE (combined space) and metadata RAID1, 1.5 TB free space, 600 GB + 1.4 TB.

Other partitions are not used by the running kernel in any way.

I'm having quite a lot of these in journal. Repeats exactly evey three minutes:

Sep 25 20:31:04 gremlin kernel: INFO: rcu_preempt detected stalls on CPUs/tasks: {} (detected by 2, t=2040057 jiffies, g=74163, c=74162, q=215856)
Sep 25 20:31:04 gremlin kernel: All QSes seen, last rcu_preempt kthread activity 2 (4314532618-4314532616), jiffies_till_next_fqs=3, root ->qsmask 0x0
Sep 25 20:31:04 gremlin kernel: btrfs-transacti R  running task        0  1916      2 0x00000000
Sep 25 20:31:04 gremlin kernel:  ffff88020cd22850 000000000707549c ffff88021fa83d88 ffffffff810a2bc1
Sep 25 20:31:04 gremlin kernel:  ffff88021fa96440 ffffffff81849b80 ffff88021fa83e18 ffffffff810c38dd
Sep 25 20:31:04 gremlin kernel:  0000000000000000 0000000000000046 ffff88021fa83e18 ffffffff810d07d8
Sep 25 20:31:04 gremlin kernel: Call Trace:
Sep 25 20:31:04 gremlin kernel:  <IRQ>  [<ffffffff810a2bc1>] sched_show_task+0xc1/0x140
Sep 25 20:31:04 gremlin kernel:  [<ffffffff810c38dd>] rcu_check_callbacks+0x95d/0x970
Sep 25 20:31:04 gremlin kernel:  [<ffffffff810d07d8>] ? update_wall_time+0x418/0x6b0
Sep 25 20:31:04 gremlin kernel:  [<ffffffff810c88e9>] update_process_times+0x39/0x60
Sep 25 20:31:04 gremlin kernel:  [<ffffffff810d94c1>] tick_sched_handle.isra.6+0x21/0x60
Sep 25 20:31:04 gremlin kernel:  [<ffffffff810d9549>] tick_sched_timer+0x49/0x90
Sep 25 20:31:04 gremlin kernel:  [<ffffffff810c8f27>] __run_hrtimer+0x77/0x250
Sep 25 20:31:04 gremlin kernel:  [<ffffffff810d9500>] ? tick_sched_handle.isra.6+0x60/0x60
Sep 25 20:31:04 gremlin kernel:  [<ffffffff810c9b13>] hrtimer_interrupt+0x113/0x290
Sep 25 20:31:04 gremlin kernel:  [<ffffffff8105094c>] local_apic_timer_interrupt+0x3c/0x70
Sep 25 20:31:04 gremlin kernel:  [<ffffffff81574831>] smp_apic_timer_interrupt+0x41/0x60
Sep 25 20:31:04 gremlin kernel:  [<ffffffff81572a5e>] apic_timer_interrupt+0x6e/0x80
Sep 25 20:31:04 gremlin kernel:  <EOI>  [<ffffffff81572799>] ? retint_kernel+0x1b/0x1d
Sep 25 20:31:04 gremlin kernel:  [<ffffffff812b7ee8>] ? __radix_tree_lookup+0x68/0xb0
Sep 25 20:31:04 gremlin kernel:  [<ffffffff812b7f52>] radix_tree_lookup_slot+0x22/0x50
Sep 25 20:31:04 gremlin kernel:  [<ffffffff81147dfc>] find_get_entry+0x2c/0xd0
Sep 25 20:31:04 gremlin kernel:  [<ffffffff811485ec>] pagecache_get_page+0x2c/0x1f0
Sep 25 20:31:04 gremlin kernel:  [<ffffffffa009d366>] alloc_extent_buffer+0xf6/0x3c0 [btrfs]
Sep 25 20:31:04 gremlin kernel:  [<ffffffffa0071c55>] btrfs_find_create_tree_block+0x15/0x20 [btrfs]
Sep 25 20:31:04 gremlin kernel:  [<ffffffffa00656bc>] btrfs_alloc_tree_block+0x22c/0x500 [btrfs]
Sep 25 20:31:04 gremlin kernel:  [<ffffffffa004e29f>] __btrfs_cow_block+0x13f/0x520 [btrfs]
Sep 25 20:31:04 gremlin kernel:  [<ffffffffa004e826>] btrfs_cow_block+0x116/0x1e0 [btrfs]
Sep 25 20:31:04 gremlin kernel:  [<ffffffffa0052749>] btrfs_search_slot+0x1e9/0xa90 [btrfs]
Sep 25 20:31:04 gremlin kernel:  [<ffffffff810cee97>] ? ktime_get+0x37/0xb0
Sep 25 20:31:04 gremlin kernel:  [<ffffffffa006e82e>] btrfs_lookup_inode+0x3e/0xc0 [btrfs]
Sep 25 20:31:04 gremlin kernel:  [<ffffffffa00d2c13>] ? btrfs_delete_delayed_items+0x333/0x370 [btrfs]
Sep 25 20:31:04 gremlin kernel:  [<ffffffffa00d2e59>] __btrfs_update_delayed_inode+0x69/0x210 [btrfs]
Sep 25 20:31:04 gremlin kernel:  [<ffffffffa004d50a>] ? btrfs_alloc_path+0x1a/0x20 [btrfs]
Sep 25 20:31:04 gremlin kernel:  [<ffffffffa00d3357>] __btrfs_run_delayed_items+0x157/0x1d0 [btrfs]
Sep 25 20:31:04 gremlin kernel:  [<ffffffffa00d3763>] btrfs_run_delayed_items+0x13/0x20 [btrfs]
Sep 25 20:31:04 gremlin kernel:  [<ffffffffa00792c3>] btrfs_commit_transaction+0x2f3/0xc80 [btrfs]
Sep 25 20:31:04 gremlin kernel:  [<ffffffffa0079ce5>] ? start_transaction+0x95/0x5a0 [btrfs]
Sep 25 20:31:04 gremlin kernel:  [<ffffffffa0074b15>] transaction_kthread+0x1d5/0x240 [btrfs]
Sep 25 20:31:04 gremlin kernel:  [<ffffffffa0074940>] ? btrfs_cleanup_transaction+0x5a0/0x5a0 [btrfs]
Sep 25 20:31:04 gremlin kernel:  [<ffffffff81096418>] kthread+0xd8/0xf0
Sep 25 20:31:04 gremlin kernel:  [<ffffffff81096340>] ? kthread_worker_fn+0x170/0x170
Sep 25 20:31:04 gremlin kernel:  [<ffffffff81571fe2>] ret_from_fork+0x42/0x70
Sep 25 20:31:04 gremlin kernel:  [<ffffffff81096340>] ? kthread_worker_fn+0x170/0x170

Checked SMART, did fscks on all btrfs volumes (for what it's worth), no signs of trouble. Should I be worried about my data? What is happening?

thanks for any info,

-miky.

---

What happened to Arch's KISS? systemd sure is stupid but I must have missed the simple part ...

... and who is general Failure and why is he reading my harddisk?