ata timeout leading to I/O errors

seth · 2024-06-01 18:33:29

seth wrote:

This looks increasingly like the disk is dying.

Can you get the smart data from it when attached to the other system?

0xlogn · 2024-06-01 18:49:33

Smart! (pun intended)
I'm running an extended test but it's going to take a few hours. Here's the current results: https://0x0.st/XNF4.txt

0xlogn · 2024-06-01 18:50:55

The error "revalidation failed (errno=-5)" has begun to appear a few times, and smartctl has frozen when running another command.

Update: "INQUIRY failed" if I re-run `smartctl -a`. dmesg: http://0x0.st/XNFt.txt

Last edited by 0xlogn (2024-06-01 18:51:59)

seth · 2024-06-01 18:59:57

  5 Reallocated_Sector_Ct   0x0033   095   095   036    Pre-fail  Always       -       2288
183 Runtime_Bad_Block       0x0032   099   099   000    Old_age   Always       -       1
196 Reallocated_Event_Count 0x0032   095   095   000    Old_age   Always       -       2288

You've backups, right?

0xlogn · 2024-06-01 19:03:04

Yes, I take it this disk is dead?

Oddly enough smartctl -H showed it as PASSING still.

seth · 2024-06-01 19:08:44

https://bbs.archlinux.org/viewtopic.php … 1#p2174971

The re-allocation exceeds the limits by some thousands - either the disk is by tihs time mostly rust or the firmware got fried.
Either way, you cannot expect any kind of reliable behavior from the disk anymore. And apparently it starts to choke after some access.

Don't cry - it will move to a serverfarm upstate and only store the funnies memes and cutest kitten pictures for the rest of time.
And porn, of course.

0xlogn · 2024-06-01 19:11:55

I've had some issues understanding how to read SMART results: on reallocated sector ct isn't VALUE 95 and not 2288?

I don't know man, if this thing is that fried maybe it can't even store the memes :sob:

Either way, thank you for helping me figure out if it was the disk specifically. Appreciate it lots!

seth · 2024-06-01 19:20:13

The raw value column is the relevant one.

Don't worry, the upstate serverfarm is a special place where data gets handled with infinite throughput and no latency and not bit has ever flipped.
It's the happiest place wher any drive can ever be.

0xlogn · 2024-06-01 19:46:37

HDD uninstalled and I'm still seeing the PCI physical layer errors. What could it even be at this point?? Should I be concerned?

seth · 2024-06-02 06:58:13

https://bbs.archlinux.org/viewtopic.php … 7#p2174797 ist still a bus error, the operative term is "Correctable" but it means that the signal on the bus is polluted.
Does it help to remove the nvme? Though since you rely on an OOT driver, you might have issues testing this w/ some live distro from a USB key…

0xlogn · 2024-06-03 15:33:27

I don't care for the wireless NIC, I don't even plan on installing the driver. This acts as a server, so it's hardwired.

seth · 2024-06-03 15:35:14

And can you/have you tried to simply disable or remove it (in case this has further impact on the nvme)?

0xlogn · 2024-06-03 15:46:59

When we still had the HDD installed I had removed both the nvme SSD and the NIC, and it was still there. Could it be the bus at fault?

seth · 2024-06-03 16:36:41

The HDD seems the HDD's fault, so that's not a good indicator.
The errors in https://bbs.archlinux.org/viewtopic.php … 7#p2174797 are from the NIC and imply there's too much noise on the bus.
Whether this is because the bus lines are degrading or some device is yelling around there would to be seen.
Since the messages there are all from the wifi NIC I'd disable it and see whether you still get bus errors (In this case likely from the nvme, maybe anything else)

As long as the errors can be corrected, this will cause some slow-down, but isn't critical.
If it's too annoying you can **suppress** (this doesn't fix anything!) them w/ "pci=noaer", https://wiki.archlinux.org/title/Kernel_parameters

Arch Linux

#26 2024-06-01 18:33:29

Re: ata timeout leading to I/O errors

#27 2024-06-01 18:49:33

Re: ata timeout leading to I/O errors

#28 2024-06-01 18:50:55

Re: ata timeout leading to I/O errors

#29 2024-06-01 18:59:57

Re: ata timeout leading to I/O errors

#30 2024-06-01 19:03:04

Re: ata timeout leading to I/O errors

#31 2024-06-01 19:08:44

Re: ata timeout leading to I/O errors

#32 2024-06-01 19:11:55

Re: ata timeout leading to I/O errors

#33 2024-06-01 19:20:13

Re: ata timeout leading to I/O errors

#34 2024-06-01 19:46:37

Re: ata timeout leading to I/O errors

#35 2024-06-02 06:58:13

Re: ata timeout leading to I/O errors

#36 2024-06-03 15:33:27

Re: ata timeout leading to I/O errors

#37 2024-06-03 15:35:14

Re: ata timeout leading to I/O errors

#38 2024-06-03 15:46:59

Re: ata timeout leading to I/O errors

#39 2024-06-03 16:36:41

Re: ata timeout leading to I/O errors

Board footer