You are not logged in.
Hi All,
Summary: the kernel can't use an existing /dev/bcache0 as backing device for /dev/bcache1, and throws a lot of errors in syslog.
In detail:
I would like to use a small slow ssd as a write cache for a hard drive, and a large, fast SSD as read cache. Therefore I'm trying to stack two bcaches on top of each other like this: (Note, I have done all tests on one HDD with a lot of partitions, so all devices start with /dev/sde)
+---------------------------------+
| btrfs file system |
+---------------------------------+
| /dev/Bcache1 |
+---------------------------------+
| Read cache on fast ssd |
| /dev/sde3 (5GB) |
+---------------------------------+
| /dev/Bcache0 |
+---------------------------------+
| Write Cache on slow SSD |
| /dev/sde2 (2GB) |
+---------------------------------+
| Data on big spinning hard drive |
| /dev/sde1 (40GB) |
+---------------------------------+I create the stack from bottom to top:
1) Format the big spinning hard drive as backing device
# make-bcache -B /dev/sde1
Name /dev/sde1
Label
Type data
UUID: a2b424ce-c7dd-4d16-8600-fa4b47306865
Set UUID: f9698940-513b-4649-9d0e-58216587bd6f
version: 1
block_size_in_sectors: 1
data_offset_in_sectors: 16
# dmesg
[52742.372299] bcache: register_bdev() registered backing device sde1
2) Format the slow SSD as cache device
# make-bcache -C /dev/sde2
Name /dev/sde2
Label
Type cache
UUID: 58092ee5-2412-4f25-af21-ed8d30fb3f1c
Set UUID: a5bfbb35-dd42-4593-af63-9af53a5cc3a0
version: 0
nbuckets: 3814
block_size_in_sectors: 1
bucket_size_in_sectors: 1024
nr_in_set: 1
nr_this_dev: 0
first_bucket: 1
# dmesg
[52965.412606] bcache: run_cache_set() invalidating existing data
[52965.438891] bcache: register_cache() registered cache device sde2
3) Attach the slow ssd cache (/dev/sde2) to the big spinning hard drive (/dev/sde1)
# bcache-super-show /dev/sde2 | grep cset
cset.uuid a5bfbb35-dd42-4593-af63-9af53a5cc3a0
echo a5bfbb35-dd42-4593-af63-9af53a5cc3a0 > /sys/block/bcache0/bcache/attach
# dmesg
Mar 2 17:45:17 cedric kernel: bcache: bch_cached_dev_run() cached dev sde1 is running already
Mar 2 17:45:17 cedric kernel: bcache: bch_cached_dev_attach() Caching sde1 as bcache0 on set a5bfbb35-dd42-4593-af63-9af53a5cc3a0
4) Format /dev/bcache0 as backing device
# make-bcache -B /dev/bcache0
Name /dev/bcache0
Label
Type data
UUID: 21cc37f3-9aa7-4eba-b652-8cd6b1f812ae
Set UUID: 2d45c3c2-1efc-43bb-80e0-33859649aba2
version: 1
block_size_in_sectors: 1
data_offset_in_sectors: 16
# tail -F /var/log/everything.log
Mar 2 17:48:47 cedric kernel: sysfs: cannot create duplicate filename '/devices/virtual/block/bcache0/bcache'
Mar 2 17:48:47 cedric kernel: CPU: 1 PID: 37431 Comm: kworker/1:4 Tainted: G OE 5.16.11-arch1-1 #1 ded1ca8dd1dd660648f829b67fad213afe36c9c9
Mar 2 17:48:47 cedric kernel: Hardware name: Gigabyte Technology Co., Ltd. B550 AORUS PRO AC/B550 AORUS PRO AC, BIOS F12 01/18/2021
Mar 2 17:48:47 cedric kernel: Workqueue: events register_bdev_worker [bcache]
Mar 2 17:48:47 cedric kernel: Call Trace:
Mar 2 17:48:47 cedric kernel: <TASK>
Mar 2 17:48:47 cedric kernel: dump_stack_lvl+0x48/0x5e
Mar 2 17:48:47 cedric kernel: sysfs_warn_dup.cold+0x17/0x24
Mar 2 17:48:47 cedric kernel: sysfs_create_dir_ns+0xc6/0xe0
Mar 2 17:48:47 cedric kernel: kobject_add_internal+0xbd/0x2c0
Mar 2 17:48:47 cedric kernel: kobject_add+0x98/0xd0
Mar 2 17:48:47 cedric kernel: ? bcache_device_init+0x242/0x2a0 [bcache eb8586620a25cfc9a2e260d15875c9beb6c2953d]
Mar 2 17:48:47 cedric kernel: register_bdev_worker+0x30d/0x3b0 [bcache eb8586620a25cfc9a2e260d15875c9beb6c2953d]
Mar 2 17:48:47 cedric kernel: process_one_work+0x1e8/0x3c0
Mar 2 17:48:47 cedric kernel: worker_thread+0x50/0x3b0
Mar 2 17:48:47 cedric kernel: ? rescuer_thread+0x3a0/0x3a0
Mar 2 17:48:47 cedric kernel: kthread+0x15c/0x180
Mar 2 17:48:47 cedric kernel: ? set_kthread_struct+0x40/0x40
Mar 2 17:48:47 cedric kernel: ret_from_fork+0x22/0x30
Mar 2 17:48:47 cedric kernel: </TASK>
Mar 2 17:48:47 cedric kernel: kobject_add_internal failed for bcache with -EEXIST, don't try to register things with the same name in the same directory.
Mar 2 17:48:47 cedric kernel: bcache: register_bdev() error bcache0: error creating kobject
Mar 2 17:48:47 cedric kernel: bcache: register_bdev_worker() error /dev/bcache0: fail to register backing device
Mar 2 17:48:47 cedric kernel: bcache: bcache_device_free() bcache1 stopped
# ls -l /dev/bca*
brw-rw---- 1 root disk 254, 0 Mar 2 17:48 /dev/bcache0
Expected result:
I expected step 4 to create the new bcache device /dev/bcache1. Then I would have been able to to these steps:
5) Format the fast SSD as cache device:
# make-bcache -C /dev/sde3
6) Attach the fast ssd cache (/dev/sde3) to the big spinning hard drive/slow SSD pair (/dev/bcache1) created in step 4
# bcache-super-show /dev/sde3 | grep cset
cset.uuid 957377e0-ae6f-45fb-9ad8-9ff7ca83a861
# echo 957377e0-ae6f-45fb-9ad8-9ff7ca83a861 > /sys/block/bcache1/bcache/attach
My Kernel:
# uname -a
Linux cedric 5.16.11-arch1-1 #1 SMP PREEMPT Thu, 24 Feb 2022 02:18:20 +0000 x86_64 GNU/Linux
I'm new at kernel debugging, how can I start debugging this?
Where can I best report this problem?
Is there a workaround for this problem?
Kind regards,
Cedric
Offline
Was 5.16.11-arch1-1 the first kernel with the issue i.e. 5.16.10-arch1-1 does not have the issue?
Does 5.16.12-arch1-1 have the issue? Does 5.17-rc6 have the issue?
Offline
Thanks loqs, this is a long standing problem in the kernel. I've also noticed this in kernel 5.9.14
https://wiki.archlinux.org/title/Talk:B … _of_bcache
# ls -l /dev/bcache5
brw-rw---- 1 root disk 254, 640 Dec 27 10:14 /dev/bcache5
# make-bcache -B /dev/bcache5
Device /dev/bcache5 already has a non-bcache superblock,remove it using wipefs and wipefs -a
# wipefs -a /dev/bcache5
# make-bcache -B /dev/bcache5
Name /dev/bcache5
Label
Type data
UUID: 2b081bcb-5699-4748-85f2-2cc3ca4d2b18
Set UUID: c75253fd-4bed-4774-98e2-897ea766a917
version: 1
block_size_in_sectors: 1
data_offset_in_sectors: 16
# uname -a
Linux bcache-test 5.9.14-arch1-1 #1 SMP PREEMPT Sat, 12 Dec 2020 14:37:12 +0000 x86_64 GNU/Linux
# dmesg
[ 1464.820001] sysfs: cannot create duplicate filename '/devices/virtual/block/bcache5/bcache'
[ 1464.820005] CPU: 2 PID: 536 Comm: bcache-register Not tainted 5.9.14-arch1-1 #1
[ 1464.820006] Hardware name: Hewlett-Packard HP Compaq 8200 Elite CMT PC/1494, BIOS J01 v02.28 03/24/2015
[ 1464.820007] Call Trace:
[ 1464.820016] dump_stack+0x6b/0x83
[ 1464.820020] sysfs_warn_dup.cold+0x17/0x24
[ 1464.820026] sysfs_create_dir_ns+0xc6/0xe0
[ 1464.820030] kobject_add_internal+0xab/0x2f0
[ 1464.820033] kobject_add+0x98/0xd0
[ 1464.820037] ? blk_queue_write_cache+0x2f/0x60
[ 1464.820049] register_bdev+0x337/0x360 [bcache]
[ 1464.820059] register_bcache+0x43c/0x910 [bcache]
[ 1464.820062] ? kernfs_fop_write+0xce/0x1b0
[ 1464.820069] ? register_cache+0x1290/0x1290 [bcache]
[ 1464.820071] kernfs_fop_write+0xce/0x1b0
[ 1464.820074] vfs_write+0xc7/0x210
[ 1464.820076] ksys_write+0x67/0xe0
[ 1464.820079] do_syscall_64+0x33/0x40
[ 1464.820081] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[ 1464.820084] RIP: 0033:0x7feaba7faf67
[ 1464.820087] Code: 0d 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b7 0f 1f 00 f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0 75 10 b8 01 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 51 c3 48 83 ec 28 48 89 54 24 18 48 89 74 24
[ 1464.820088] RSP: 002b:00007ffd19cd32f8 EFLAGS: 00000246 ORIG_RAX: 0000000000000001
[ 1464.820090] RAX: ffffffffffffffda RBX: 000000000000000d RCX: 00007feaba7faf67
[ 1464.820091] RDX: 000000000000000d RSI: 000056216b5532a0 RDI: 0000000000000003
[ 1464.820092] RBP: 000056216b5532a0 R08: 00007feaba891040 R09: 00007feaba8910c0
[ 1464.820093] R10: 00007feaba890fc0 R11: 0000000000000246 R12: 000000000000000d
[ 1464.820094] R13: 00007ffd19cd3380 R14: 000000000000000d R15: 00007feaba8cd720
[ 1464.820097] kobject_add_internal failed for bcache with -EEXIST, don't try to register things with the same name in the same directory.
[ 1464.820098] bcache: register_bdev() error bcache5: error creating kobject
[ 1464.820108] bcache: bcache_device_free() bcache6 stopped
[ 1464.820128] bcache: register_bcache() error : failed to register deviceOffline
In order to file a bug report at https://bugzilla.kernel.org/ I need to compile and run the latest upstream kernel.
As a reminder to myself, here's the wiki article to do this: https://wiki.archlinux.org/title/Kernel … ompilation
Offline
You can obtain linux-mainline prebuilt from miffe or archlinuxcn.
Offline