[SOLVED] Suddenly FS in Read-only mode -> SSD not detected

atedesch1 · 2023-05-05 19:29:08

I am getting some really weird behavior:
Suddenly my system stops working, I get that the file system is in read-only mode. Can't even shutdown. I force shutdown.
Next, my SSD is not detected. I set secure-mode to ON again reboot and after some trial and error my SSD is recognized (I dont know if I the "redetecting" is because I set secure-mode to ON or not).
I set secure-mode back to OFF.
After trying to boot to my system with grub i get:

452: out of range pointer: 0x185e2020

Helpful output of fdisk -l for next part:

Disk /dev/nvme0n1: 476.94 GiB, 512110190592 bytes, 1000215216 sectors
Disk model: SAMSUNG MZVLB512HAJQ-00000
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 42206031-D397-4E71-8E4F-3F0A129F0533
Device Start End Sectors Size Type
/dev/nvme0n1p1 2048 1048575 1046528 511M EFI System
/dev/nvme0n1p2 1050624 185298943 184248320 87.9G Linux filesystem
/dev/nvme0n1p3 185300992 1000214527 814913536 388.6G Linux filesystem

Disk /dev/zram0: 4 GiB, 4294967296 bytes, 1048576 sectors
Units: sectors of 1 * 4096 = 4096 bytes
Sector size (logical/physical): 4096 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes

I fix this by fixing grub:
Boot from a live usb and then execute

mount /dev/nvme0n1p2 /mnt
mount /dev/nvme0n1p1 /mnt/boot
arch-chroot /mnt
grub-install --efi-directory=/boot (without this it cant find the dir)
grub-mkconfig -o /boot/grub/grub.cfg
...

I reboot and seems like its working again. However, randomly after some time system suddenly stops working again and the cycle repeats.
I suppose this started happening after I had a crash when updating the system and had some broken packages, I think i fixed them by reading the forums and reinstalling the packages by overwriting but I am not sure.

Last edited by atedesch1 (2023-05-10 20:34:12)

seth · 2023-05-05 20:33:26

Check "smartctl -a /dev/nvme0n1", post the output of "lsblk -f" and see https://wiki.archlinux.org/title/Solid_ … leshooting
(nvme_core.default_ps_max_latency_us=0 & iommu=soft)

I think i fixed them

sudo LC_ALL=C pacman -Qkk | grep -v ', 0 altered files'

atedesch1 · 2023-05-05 21:02:19

seth wrote:

Check "smartctl -a /dev/nvme0n1", post the output of "lsblk -f" and see https://wiki.archlinux.org/title/Solid_ … leshooting
(nvme_core.default_ps_max_latency_us=0 & iommu=soft)
I think i fixed them
sudo LC_ALL=C pacman -Qkk | grep -v ', 0 altered files'

Output of smartctl:

smartctl 7.3 2022-02-28 r5338 [x86_64-linux-6.3.1-arch1-1] (local build)
Copyright (C) 2002-22, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Number:                       SAMSUNG MZVLB512HAJQ-00000
Serial Number:                      S3W8NB0M100258
Firmware Version:                   EXA7301Q
PCI Vendor/Subsystem ID:            0x144d
IEEE OUI Identifier:                0x002538
Total NVM Capacity:                 512,110,190,592 [512 GB]
Unallocated NVM Capacity:           0
Controller ID:                      4
NVMe Version:                       1.2
Number of Namespaces:               1
Namespace 1 Size/Capacity:          512,110,190,592 [512 GB]
Namespace 1 Utilization:            397,807,443,968 [397 GB]
Namespace 1 Formatted LBA Size:     512
Namespace 1 IEEE EUI-64:            002538 8191b02ee3
Local Time is:                      Fri May  5 17:46:14 2023 -03
Firmware Updates (0x16):            3 Slots, no Reset required
Optional Admin Commands (0x0017):   Security Format Frmw_DL Self_Test
Optional NVM Commands (0x001f):     Comp Wr_Unc DS_Mngmt Wr_Zero Sav/Sel_Feat
Log Page Attributes (0x03):         S/H_per_NS Cmd_Eff_Lg
Maximum Data Transfer Size:         512 Pages
Warning  Comp. Temp. Threshold:     81 Celsius
Critical Comp. Temp. Threshold:     82 Celsius

Supported Power States
St Op     Max   Active     Idle   RL RT WL WT  Ent_Lat  Ex_Lat
 0 +     7.02W       -        -    0  0  0  0        0       0
 1 +     6.30W       -        -    1  1  1  1        0       0
 2 +     3.50W       -        -    2  2  2  2        0       0
 3 -   0.0760W       -        -    3  3  3  3      210    1200
 4 -   0.0050W       -        -    4  4  4  4     2000    8000

Supported LBA Sizes (NSID 0x1)
Id Fmt  Data  Metadt  Rel_Perf
 0 +     512       0         0

=== START OF SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

SMART/Health Information (NVMe Log 0x02)
Critical Warning:                   0x00
Temperature:                        46 Celsius
Available Spare:                    98%
Available Spare Threshold:          10%
Percentage Used:                    5%
Data Units Read:                    51,451,278 [26.3 TB]
Data Units Written:                 60,388,252 [30.9 TB]
Host Read Commands:                 741,864,124
Host Write Commands:                1,131,811,277
Controller Busy Time:               3,899
Power Cycles:                       3,863
Power On Hours:                     7,720
Unsafe Shutdowns:                   284
Media and Data Integrity Errors:    0
Error Information Log Entries:      4,249
Warning  Comp. Temperature Time:    121
Critical Comp. Temperature Time:    5
Temperature Sensor 1:               46 Celsius
Temperature Sensor 2:               55 Celsius

Error Information (NVMe Log 0x01, 16 of 64 entries)
Num   ErrCount  SQId   CmdId  Status  PELoc          LBA  NSID    VS
  0       4249     0  0x001c  0x4004      -            0     0     -

Output of lsblk:

NAME        FSTYPE FSVER LABEL UUID                                 FSAVAIL FSUSE% MOUNTPOINTS
sda
sdb
zram0                                                                              [SWAP]
nvme0n1
├─nvme0n1p1 vfat   FAT32       E567-224F                             424.9M    17% /boot
├─nvme0n1p2 ext4   1.0         9cb9e5a1-98cd-4e44-862d-a7a9fd86b045   29.5G    61% /
└─nvme0n1p3 ext4   1.0         06a0c83a-73d0-497f-9a11-da0e6ce5b96a   63.9G    78% /home

Output of pacman -Qkk:

backup file: bluez: /etc/bluetooth/main.conf (Modification time mismatch)
backup file: bluez: /etc/bluetooth/main.conf (Size mismatch)
backup file: bluez: /etc/bluetooth/main.conf (MD5 checksum mismatch)
backup file: bluez: /etc/bluetooth/main.conf (SHA256 checksum mismatch)
bluez: 63 total files, 1 altered file
bluez-utils: 66 total files, 1 altered file
backup file: filesystem: /etc/fstab (Modification time mismatch)
backup file: filesystem: /etc/fstab (Size mismatch)
backup file: filesystem: /etc/fstab (MD5 checksum mismatch)
backup file: filesystem: /etc/fstab (SHA256 checksum mismatch)
backup file: filesystem: /etc/group (Modification time mismatch)
backup file: filesystem: /etc/group (Size mismatch)
backup file: filesystem: /etc/group (MD5 checksum mismatch)
backup file: filesystem: /etc/group (SHA256 checksum mismatch)
backup file: filesystem: /etc/gshadow (Modification time mismatch)
backup file: filesystem: /etc/gshadow (Size mismatch)
backup file: filesystem: /etc/gshadow (MD5 checksum mismatch)
backup file: filesystem: /etc/gshadow (SHA256 checksum mismatch)
backup file: filesystem: /etc/passwd (Modification time mismatch)
backup file: filesystem: /etc/passwd (Size mismatch)
backup file: filesystem: /etc/passwd (MD5 checksum mismatch)
backup file: filesystem: /etc/passwd (SHA256 checksum mismatch)
backup file: filesystem: /etc/profile (Modification time mismatch)
backup file: filesystem: /etc/profile (Size mismatch)
backup file: filesystem: /etc/profile (MD5 checksum mismatch)
backup file: filesystem: /etc/profile (SHA256 checksum mismatch)
backup file: filesystem: /etc/resolv.conf (Modification time mismatch)
backup file: filesystem: /etc/resolv.conf (Size mismatch)
backup file: filesystem: /etc/resolv.conf (MD5 checksum mismatch)
backup file: filesystem: /etc/resolv.conf (SHA256 checksum mismatch)
backup file: filesystem: /etc/shadow (Modification time mismatch)
backup file: filesystem: /etc/shadow (Size mismatch)
backup file: filesystem: /etc/shadow (MD5 checksum mismatch)
backup file: filesystem: /etc/shadow (SHA256 checksum mismatch)
backup file: filesystem: /etc/shells (Modification time mismatch)
backup file: filesystem: /etc/shells (Size mismatch)
backup file: filesystem: /etc/shells (MD5 checksum mismatch)
backup file: filesystem: /etc/shells (SHA256 checksum mismatch)
filesystem: 120 total files, 1 altered file
backup file: glibc: /etc/locale.gen (Modification time mismatch)
backup file: glibc: /etc/locale.gen (Size mismatch)
backup file: glibc: /etc/locale.gen (MD5 checksum mismatch)
backup file: glibc: /etc/locale.gen (SHA256 checksum mismatch)
backup file: grub: /etc/default/grub (Modification time mismatch)
backup file: grub: /etc/default/grub (Size mismatch)
backup file: grub: /etc/default/grub (MD5 checksum mismatch)
backup file: grub: /etc/default/grub (SHA256 checksum mismatch)
intel-ucode: 7 total files, 1 altered file
java-runtime-common: 21 total files, 2 altered files
libutempter: 20 total files, 1 altered file
light: 17 total files, 1 altered file
backup file: linux-wifi-hotspot: /etc/create_ap.conf (Modification time mismatch)
backup file: linux-wifi-hotspot: /etc/create_ap.conf (Size mismatch)
backup file: linux-wifi-hotspot: /etc/create_ap.conf (MD5 checksum mismatch)
backup file: linux-wifi-hotspot: /etc/create_ap.conf (SHA256 checksum mismatch)
linux-wifi-hotspot: 48 total files, 1 altered file
backup file: mkinitcpio: /etc/mkinitcpio.conf (Modification time mismatch)
backup file: mkinitcpio: /etc/mkinitcpio.conf (Size mismatch)
backup file: mkinitcpio: /etc/mkinitcpio.conf (MD5 checksum mismatch)
backup file: mkinitcpio: /etc/mkinitcpio.conf (SHA256 checksum mismatch)
backup file: networkmanager: /etc/NetworkManager/NetworkManager.conf (Modification time mismatch)
backup file: networkmanager: /etc/NetworkManager/NetworkManager.conf (Size mismatch)
backup file: networkmanager: /etc/NetworkManager/NetworkManager.conf (MD5 checksum mismatch)
backup file: networkmanager: /etc/NetworkManager/NetworkManager.conf (SHA256 checksum mismatch)
backup file: pacman-mirrorlist: /etc/pacman.d/mirrorlist (Modification time mismatch)
backup file: pacman-mirrorlist: /etc/pacman.d/mirrorlist (Size mismatch)
backup file: pacman-mirrorlist: /etc/pacman.d/mirrorlist (MD5 checksum mismatch)
backup file: pacman-mirrorlist: /etc/pacman.d/mirrorlist (SHA256 checksum mismatch)
backup file: sudo: /etc/sudoers (Modification time mismatch)
backup file: sudo: /etc/sudoers (Size mismatch)
backup file: sudo: /etc/sudoers (MD5 checksum mismatch)
backup file: sudo: /etc/sudoers (SHA256 checksum mismatch)
backup file: systemd: /etc/systemd/timesyncd.conf (Modification time mismatch)
backup file: systemd: /etc/systemd/timesyncd.conf (Size mismatch)
backup file: systemd: /etc/systemd/timesyncd.conf (MD5 checksum mismatch)
backup file: systemd: /etc/systemd/timesyncd.conf (SHA256 checksum mismatch)
systemd: 1330 total files, 1 altered file
backup file: texlive-core: /etc/texmf/web2c/fmtutil.cnf (Modification time mismatch)
backup file: texlive-core: /etc/texmf/web2c/fmtutil.cnf (Size mismatch)
backup file: texlive-core: /etc/texmf/web2c/fmtutil.cnf (MD5 checksum mismatch)
backup file: texlive-core: /etc/texmf/web2c/fmtutil.cnf (SHA256 checksum mismatch)
vlc: 1060 total files, 1 altered file

Currently looking at the troubleshooting you gave me, thanks!

Last edited by atedesch1 (2023-05-05 21:03:06)

seth · 2023-05-05 21:15:54

Packages look fine, ext4 would run fsck, smartctl looks ok-ish (some spare is gone, but you've also written > 30TB)
Next to the nvme specific things, do you mount any drives "discard"?

atedesch1 · 2023-05-05 21:28:09

seth wrote:

Packages look fine, ext4 would run fsck, smartctl looks ok-ish (some spare is gone, but you've also written > 30TB)
Next to the nvme specific things, do you mount any drives "discard"?

I have ran fsck, it didn't show anything out of the ordinary, only asked if I wanted to shorten the partition (fragmented?).
I do believe I don't mount them "discard". The following command yields nothing:

cat /proc/mounts | grep discard

I have added the kernel parameters you said. The only problem is I don't know if it fixed the problem as it really occurs randomly when I'm coding.
The problem with the out of range pointer also occurred when I shutdown my laptop, disconnected the power chord and tried booting it back up (maybe this sheds some light as to what might be happening?).

atedesch1 · 2023-05-05 21:49:05

atedesch1 wrote:

seth wrote:
Packages look fine, ext4 would run fsck, smartctl looks ok-ish (some spare is gone, but you've also written > 30TB)
Next to the nvme specific things, do you mount any drives "discard"?
I have ran fsck, it didn't show anything out of the ordinary, only asked if I wanted to shorten the partition (fragmented?).
I do believe I don't mount them "discard". The following command yields nothing:
cat /proc/mounts | grep discard
I have added the kernel parameters you said. The only problem is I don't know if it fixed the problem as it really occurs randomly when I'm coding.
The problem with the out of range pointer also occurred when I shutdown my laptop, disconnected the power chord and tried booting it back up (maybe this sheds some light as to what might be happening?).

I just tried disconnecting the power chord and everything from it, shutting it down and turning it back up. The secure-mode is back on, I switch it off, the ssd is there but out of range pointer occurs again. I boot into the live usb.
I then run fsck:

fsck -f /dev/nvme0n1p2
Superblock last mount time (Fri May 5 21:16:56 2023, now = Wed May 3 17:22:33 2023) is in the future.
Fix<y>? yes
Superblock last write time (Fri May 5 21:16:56 2023, now = Wed May 3 17:22:33 2023) is in the future.
Fix<y>? yes
...

Everything else is ok. Same happens to /dev/nvme0n1p3.
I try rebooting but out of range pointer still occurs.
I boot into the live usb and do the same thing.
I run fsck and the output now reads:

fsck -f /dev/nvme0n1p2
Superblock last mount time is in the future (by less than a day, probably due to the hardware clock being incorrectly set).
Superblock last mount time is in the future (by less than a day, probably due to the hardware clock being incorrectly set).
...

Everything else is ok. Same happens to /dev/nvme0n1p3.
I fix grub like I did before and system boots.
How do I fix this hardware clock?

Last edited by atedesch1 (2023-05-05 21:57:21)

atedesch1 · 2023-05-06 03:44:21

I now strongly believe it is an issue with the CMOS battery on my Razer Blade 15 2019 RTX 2070 as removing the chord and rebooting it makes the system clock go to the past and cause the out of range pointer. I will try replacing the battery and will update if this issue stops.

Last edited by atedesch1 (2023-05-06 03:44:55)

atedesch1 · 2023-05-10 19:23:52

I replaced the CMOS battery and now the issues have stopped. Thanks!

seth · 2023-05-10 20:21:28

Please always remember to mark resolved threads by editing your initial posts subject - so others will know that there's no task left, but maybe a solution to find.
Thanks.

Arch Linux

#1 2023-05-05 19:29:08

[SOLVED] Suddenly FS in Read-only mode -> SSD not detected

#2 2023-05-05 20:33:26

Re: [SOLVED] Suddenly FS in Read-only mode -> SSD not detected

#3 2023-05-05 21:02:19

Re: [SOLVED] Suddenly FS in Read-only mode -> SSD not detected

#4 2023-05-05 21:15:54

Re: [SOLVED] Suddenly FS in Read-only mode -> SSD not detected

#5 2023-05-05 21:28:09

Re: [SOLVED] Suddenly FS in Read-only mode -> SSD not detected

#6 2023-05-05 21:49:05

Re: [SOLVED] Suddenly FS in Read-only mode -> SSD not detected

#7 2023-05-06 03:44:21

Re: [SOLVED] Suddenly FS in Read-only mode -> SSD not detected

#8 2023-05-10 19:23:52

Re: [SOLVED] Suddenly FS in Read-only mode -> SSD not detected

#9 2023-05-10 20:21:28

Re: [SOLVED] Suddenly FS in Read-only mode -> SSD not detected

Board footer