You are not logged in.

#1 2017-12-15 23:35:00

Batou
Member
Registered: 2017-01-03
Posts: 259

Linux 4.14.5 crash: whole filesystem was down; "kswapd0 tainted"?

Hi all,

Earlier today I suffered a crash: my applications started failing and freezing and my filesystem was completely locked up. I also couldn't start any new applications. I couldn't even start a new shell in my terminal and when I issued "ls" in one of the open shells, it locked up. Not even REISUB worked and I had to resort to a hard reset.

Luckily, just as one of my applications crashed, I looked at the dmesg and noticed the info and hit PrtScr. I managed to capture only the last page of the crash and stuff above was not saved:

https://i.imgur.com/KKr1pVB.jpg

Can anyone tell me what happened based on this screen or is it impossible to say? In all these years of using Linux I've never had this happen to me...
Also, I use ext4 and can anyone tell me if there's any need to run fsck on the FS after a reboot in situations like this or does the FS automatically recover? And is the hard reset the only way to get out of this situation or is there a better way?

Thanks!


Please vote for all the AUR packages you're using. You can mass-vote for all of them by doing: "pacman -Qqm | xargs aurvote -v" (make sure to run "aurvote --configure"  first)

Offline

#2 2017-12-16 00:49:45

R00KIE
Forum Fellow
From: Between a computer and a chair
Registered: 2008-09-14
Posts: 4,734

Re: Linux 4.14.5 crash: whole filesystem was down; "kswapd0 tainted"?

So your machine crashed hard, how did you manage to capture the printscreen? Was arch running on a virtual machine?

That said the crash does look to be related to the filesystem but probably it was caused by something else, one guess can be it was not possible to access the hard disk anymore, another guess can be bad ram that in time can lead to filesystem corruption.

I would start by booting the arch install iso and run memtest to make sure the ram is fine, then maybe try to mount the filesystem read-only and backup everything important, after that definitely force a filesystem check (fsck -f).


R00KIE
Tm90aGluZyB0byBzZWUgaGVyZSwgbW92ZSBhbG9uZy4K

Offline

#3 2017-12-16 03:24:55

Batou
Member
Registered: 2017-01-03
Posts: 259

Re: Linux 4.14.5 crash: whole filesystem was down; "kswapd0 tainted"?

R00KIE wrote:

So your machine crashed hard, how did you manage to capture the printscreen? Was arch running on a virtual machine?

That said the crash does look to be related to the filesystem but probably it was caused by something else, one guess can be it was not possible to access the hard disk anymore, another guess can be bad ram that in time can lead to filesystem corruption.

I would start by booting the arch install iso and run memtest to make sure the ram is fine, then maybe try to mount the filesystem read-only and backup everything important, after that definitely force a filesystem check (fsck -f).

Not sure how this screenshot got saved. Subsequent ones (when I scrolled up) didn't get saved at all. I just did the memtest86 (I have it in my Grub menu) and it passed. RAM is not the issue. I'll try the fsck the disk next but I don't think it's an issue (it's an SSD that's been performing flawlessly for the past year).

I've had few weird issues with 4.14.x so far and I'm suspecting it's at fault. I'll go and try the LTS version for few days and see if there's any change.

Last edited by Batou (2017-12-16 03:54:53)


Please vote for all the AUR packages you're using. You can mass-vote for all of them by doing: "pacman -Qqm | xargs aurvote -v" (make sure to run "aurvote --configure"  first)

Offline

#4 2017-12-16 09:15:28

seth
Member
Registered: 2012-09-03
Posts: 49,981

Re: Linux 4.14.5 crash: whole filesystem was down; "kswapd0 tainted"?

Looks like the swapfile grew until the containing partition (likely /root) was out of space (or at least writing there failed for other reasons)
If your $HOME is on a different partition and the shot went there and the screenshot taking process was running (WM or so) that's no black magic.

Offline

#5 2017-12-16 13:55:49

R00KIE
Forum Fellow
From: Between a computer and a chair
Registered: 2008-09-14
Posts: 4,734

Re: Linux 4.14.5 crash: whole filesystem was down; "kswapd0 tainted"?

@seth
I think we can rule out the swap file growing, as far as I'm aware on linux swap files don't grow dynamically, you have to preallocate the whole file and do a mkswap on it before you can use it.

@Batou
You don't do a fsck to check if the ssd is ok, you do it because you want to check the data written to the ssd (the filesystem structures) makes sense and does not have problems.


R00KIE
Tm90aGluZyB0byBzZWUgaGVyZSwgbW92ZSBhbG9uZy4K

Offline

#6 2017-12-16 15:27:11

seth
Member
Registered: 2012-09-03
Posts: 49,981

Re: Linux 4.14.5 crash: whole filesystem was down; "kswapd0 tainted"?

Just ftr:
https://aur.archlinux.org/packages/swapspace/
If it wasn't for the "ext4" bit (which I missed and actually also wasn't aware of the feature lack) file compression could have cause that.

Running a SMART test is probably still a good idea, but of course it could just be a bug in the jbd2 module.

Offline

Board footer

Powered by FluxBB