Arch Linux

silverwolf

Member

Registered: 2023-03-12

Posts: 13

Consistent input/output error with BTRFS (csum failed)

Hey everyone!

Recently I've encountered a really weird issue with my Samsung 870 EVO 4TB SATA SSD drive (cryptsetup LUKS encrypted partition), 3 files on it got corrupted out of nowhere and BTRFS reported that their checksums are not matching.
I did a little bit of research, since I've never dealt with an issue like this, so that's what I've done so far
1) tested my RAM through the night for about 9 hours via memtest86+, 0 errors found
2) ran extended SMART self-test on the SSD, no errors
3) ran full scan via Samsung Magician on Windows for 10 hours, no error. no bad blocks or anytting like that

After doing all of this I decided to recreate the drive's partitions with ext4 instead of BTRFS, also encrypted with LUKS like it used to be with btrfs, I moved all my files from backup drive back to the newly create ext4 partition.
After moving back files to my 4TB Samsung SATA drive just out of pure curiosity I tried moving the same files to my main M2 NVMe Samsun 990 Pro 2TB drive, also encrypted with LUKS and BTRFS, which is used as root for my system.
Everything worked fine.
Then I tried copying those copied files from M2 NVMe Samsung 990 Pro 2TB (LUKS+BTRFS) system drive AGAIN to the same Samsung 870 EVO 4TB SATA drive (LUKS+ext4), and this time BTRFS reported an error. One of the files checksum changed.

[ 1702.156104] ata3.00: Enabling discard_zeroes_data
[ 1702.175145]  sdb: sdb1
[ 1702.176273] ata3.00: Enabling discard_zeroes_data
[ 1702.193738]  sdb: sdb1
[ 2227.026530] EXT4-fs (dm-1): mounted filesystem 0ddde686-35d5-407f-a28f-ce7da9d820c5 r/w with ordered data mode. Quota mode: none.
[ 4031.944211] EXT4-fs (dm-1): unmounting filesystem 0ddde686-35d5-407f-a28f-ce7da9d820c5.
[ 4206.927575] EXT4-fs (dm-1): mounted filesystem 0ddde686-35d5-407f-a28f-ce7da9d820c5 r/w with ordered data mode. Quota mode: none.
[ 4519.546197] EXT4-fs (dm-1): unmounting filesystem 0ddde686-35d5-407f-a28f-ce7da9d820c5.
[ 4599.568649] EXT4-fs (dm-1): mounted filesystem 0ddde686-35d5-407f-a28f-ce7da9d820c5 r/w with ordered data mode. Quota mode: none.
[ 9120.564074] BTRFS warning (device dm-0): csum failed root 269 ino 63654 off 483569664 csum 0x0bbcd88c expected csum 0xb82f8869 mirror 1
[ 9120.564083] BTRFS error (device dm-0): bdev /dev/mapper/cryptroot errs: wr 0, rd 0, flush 0, corrupt 1, gen 0
[ 9120.584295] BTRFS warning (device dm-0): csum failed root 269 ino 63654 off 483569664 csum 0x0bbcd88c expected csum 0xb82f8869 mirror 1
[ 9120.584299] BTRFS error (device dm-0): bdev /dev/mapper/cryptroot errs: wr 0, rd 0, flush 0, corrupt 2, gen 0
[ 9120.584331] BTRFS warning (device dm-0): csum failed root 269 ino 63654 off 483569664 csum 0x0bbcd88c expected csum 0xb82f8869 mirror 1
[ 9120.584332] BTRFS error (device dm-0): bdev /dev/mapper/cryptroot errs: wr 0, rd 0, flush 0, corrupt 3, gen 0
[ 9183.636487] BTRFS info (device dm-0): scrub: started on devid 1
[ 9226.713489] BTRFS error (device dm-0): scrub: unable to fixup (regular) error at logical 265493282816 on dev /dev/mapper/cryptroot physical 260166516736
[ 9226.713547] BTRFS warning (device dm-0): scrub: checksum error at logical 265493282816 on dev /dev/mapper/cryptroot, physical 260166516736 root 269 inode 63654 offset 483524608 length 4096 links 1 (path: [REDACTED])
[ 9226.713551] BTRFS error (device dm-0): bdev /dev/mapper/cryptroot errs: wr 0, rd 0, flush 0, corrupt 4, gen 0
[ 9237.009394] BTRFS info (device dm-0): scrub: finished on devid 1 with status: 0

I'm completely lost here, I've never encountered errors like these. What could be the cause of it?
I recently switched my motherboard, CPU and RAM.

I'm aware that my CPU is affected by RDSEED Failure on AMD “Zen 5” Processors - https://www.amd.com/en/resources/produc … -7055.html
Could this RDSEED failure be related? But as far as I can tell it should've been fixed with recent AMD microcode update.

Since I have no idea what could be the cause and hardware is seemingly acting normal, what I use:
- Wayland
- KDE
- Dolphin
- Linux stable kernel
- LUKS via cryptsetup

Hardware:
- be quiet! DARK POWER 12 750W
- NVIDIA RTX 4070
- AMD Ryzen 9600X
- Gigabyte X870 AORUS ELITE WIFI7 ICE
- Kingston FURY 64GB (32x2 sticks, compatible model according to motherboard documentation)
- Samsung 990 Pro 2TB (M.2)
- Samsung 870 EVO 1TB (SATA)
- Samsung 870 EVO 4TB (SATA)
- WD Blue 2TB (SATA hard drive)

topcat01

Member

Registered: 2019-09-17

Posts: 258

Re: Consistent input/output error with BTRFS (csum failed)

That does not look like a bit flip problem. Are you able to get on to #btrfs on libera IRC?

silverwolf

Member

Registered: 2023-03-12

Posts: 13

Re: Consistent input/output error with BTRFS (csum failed)

That does not look like a bit flip problem. Are you able to get on to #btrfs on libera IRC?

Hey! I'm not very familiar with IRC, how can I join this chat?

I have also updated my BIOS, which also introduced a whole new headache unrelated to my data corruption issue, but well, at least after hours of moving files around it seems like no corruptions so far.
Although I still wish to get to the bottom of this and figure out what exactly caused all of this.

Also interestingly enough, Gigabyte completely removed the BIOS version I had before from their website?? That's weird, to say the least. But now I'm on F8 BIOS version and I was on F8d (just saying for logging purposes, maybe it'll come useful to a degree)

Last edited by silverwolf (2025-12-01 20:54:28)

topcat01

Member

Registered: 2019-09-17

Posts: 258

Re: Consistent input/output error with BTRFS (csum failed)

Typically this kind of corruption is the result of the hardware not respecting write barriers, thus corrupting data. It could be broken write cache in the drive (quite common) or some power management issue. Hard to tell without more testing. Hopefully it's solved now.

silverwolf

Member

Registered: 2023-03-12

Posts: 13

Re: Consistent input/output error with BTRFS (csum failed)

Typically this kind of corruption is the result of the hardware not respecting write barriers, thus corrupting data. It could be broken write cache in the drive (quite common) or some power management issue. Hard to tell without more testing. Hopefully it's solved now.

I see, I've tried testing pretty much all that I could, so honestly still no idea what caused it. My biggest suspicion was SATA cable, but I suppose I would've been getting corrupted files a lot more often and by this point would've had a lot more data corrupted than just 3 files on the 4TB SATA drive. As of right now I don't really have a spare SATA cable, but it should arrive in about 3 days.

One more idea I have is that maybe it could be related to my undervolting settings of the CPU? I've had it set for all cores at -20 in curve optimizaer, with PBO limits being set to PPT 85000mW, TDC 90000mA, EDC 150000mA. Also I had a custom thermal limit of 80 Celsius.
Now I've changed the settings and used AMD's new PBO feature "curve shaper", so now I don't apply any kind of negative multiple for low CPU clock frequencies, set -15 to the medium range of frequencies and set -20 for higher frequencies, changed PPT limit to 88000mW, left TDC and EDC untouched. Still with 80 Celsius thermal limit.

One more thing that I noticed today while working on Windows - apparently my Samsung 990 Pro drives both have been running on old firmware and different versions. I updated both to the latest versions. However, SATA 870 EVO drive firmware was already latest.

Regarding power - I recently also replaced my case fans, since one of my older be quiet! fans broke, so now I have their new fans with some shiny god damn RGB lights, I wasn't really looking for RGB, it's just what was available. I didn't really care about the lights , but still connected them to be quiet!'s ARGB hub which is powered by cable from PSU that is meant for SATA drives power (I don't really know what it's called, but hopefully I made it easy to understand). Could that ARGB hub somehow affecting power delivery to the drives? The cable has 3 connectors, two of which go into that one 4TB EVO drive and my other WD HDD drive.

So far:
- updated BIOS from F8d to F8 (F8d was removed from GIGABYTE website for some unknown reason)
- updated Samsung 990 Pro drives firmware, which are both used for main OS drives (Arch Linux on one, Windows on another)
- ran Memtest86+ twice with no errors
- "shred" the drive that corrupted 3 files initially (870 EVO 4TB SATA drive) and reformatted it back to LUKS+BTRFS
- switched from "curve optimizer" set to -20 on all cores to "curve shaper" which allows me to avoid applying undervolting at lower CPU speeds, which is exactly what I did

After all that I keep moving about 650 GB of data back and forth between my 870 EVO 4TB SATA and my OS drive which is Samsung 990 Pro, ran btrfs scrub on both multiple times, no errors so far.

Maybe in an hour or two I plan on running OCCT stability tests on CPU+Memory for at least 4 hours (I don't know if I can just leave the PC running while sleeping, I don't really want to cause damage because I went sleeping).

I understand that this is a lot of information, however due to the nature of my original issue being so god damn weird I don't really know which of my actions could potentially have an effect on data traveling in my system, maybe someone would know and some of this information could end up being useful.

P.S. Also got really annoyed at how slow that 870 EVO drive is while running btrfs scrub on it, so ordered a new 4TB Crucial M2 drive instead... Maybe it'll also help with data integrity? Hopefully? Maybe? I don't really know, but I do know that all M.2 drives are running on the same bus at least, correct me if I'm wrong here if anyone knows something about it

topcat01

Member

Registered: 2019-09-17

Posts: 258

Re: Consistent input/output error with BTRFS (csum failed)

I don't have a clear idea what the issue was, but the irc.libera.chat #btrfs channel has some very helpful experts. The mailing lists can also help. However, you will need to sign up to investigate further. This is not an Arch specific problem it seems.

silverwolf

Member

Registered: 2023-03-12

Posts: 13

Re: Consistent input/output error with BTRFS (csum failed)

I don't have a clear idea what the issue was, but the irc.libera.chat #btrfs channel has some very helpful experts. The mailing lists can also help. However, you will need to sign up to investigate further. This is not an Arch specific problem it seems.

I see, I just tried the website you mentioned and seems like it's basically like discord, but I didn't really read anything about this place and only tried entering as a guest. I'll try asking people in chat tomorrow or Friday (kinda depends, because work)