Udev fails, failing to boot, NTFS unrecognized

ZurkinWood497 · 2019-11-06 18:13:35

I apologize if this is in the wrong thread or something. I'm new here... ._.

To start, my system consists of four drives: The Arch drive (ext4), two NTFS drives, and another drive Arch has never been able to read; but that drive, I'm not concerned about.

The short: I cannot boot the system with either of the NTFS drives connected. Once disconnected, Arch boots.

The long:

I had set the two NTFS drives by their UUID in /etc/fstab to mount in respective directories under /mnt. They always mounted, no problems. This morning, I boot to find the system continually failing to boot, on default and on fallback, stating that 'NTFS' is unrecognized. This is shortly after udev starts, it doesn't even reach the cleanup phase.

After finding nothing obviously wrong in the emergency shell, I disconnected the two drives and tried to boot Arch... It reached the point of mounting the drives listed in /etc/fstab, which obviously aren't there. I booted into a live Arch session and commented the lines out. Now the system boots. But this means my daily procedure consists of physically disconnecting the two drives, starting Arch, physically reconnecting the two drives, and manually mounting the two drives into their respective /mnt directories. After that, the drives work fine; I'm playing music off of the media NTFS drive right now.

Re-installation won't be an option for a long time, as there's no internet out here, excluding a tethered connection through my phone.

It's also worth nothing that /dev/console is missing... I thought that was supposed to be there...? I reinstalled the kernel just in case that was an issue, and, no difference.

I hate asking others like this, but I'm left without options here, and I didn't see anything like this happening before in these here forums, so... Any help will be ever-so-greatly appreciated. (:

WorMzy · 2019-11-06 18:18:43

From what you've described, this sounds like a classic case of booting into a different kernel to the one you have installed. While booted into Arch, please post the output from the following commands:

uname -a
file /usr/lib/modules/*/vmlinuz
lsblk -f

Please also post the content of your /etc/fstab

ZurkinWood497 · 2019-11-06 18:28:56

>uname -a
Linux Jormas 5.3.8-arch1-1 #1 SMP PREEMPT @1572357769 x86_64 GNU/Linux

>file /usr/lib/modules/*/vmlinuz
/usr/lib/modules/5.3.8-arch1-1/vmlinuz: Linux kernel x86 boot executable bzImage, version 5.3.8-arch1-1 (linux@archlinux) #1 SMP PREEMPT @1572357769, RO-rootFS, swap_dev 0x5, Normal VGA

>lsblk -f
NAME   FSTYPE LABEL           UUID                                 FSAVAIL FSUSE% MOUNTPOINT
sda                                                                               
|-sda1 swap                   43c86715-7606-49cb-a401-0fe9693968d8                [SWAP]
`-sda2 ext4                   542dd05a-e2b0-40aa-b1a4-5a78d2fcf3ac  737.2G    14% /
sdb                                                                               
`-sdb1 ufs                    5dc278d8ad8740c8                                    
sdd                                                                               
|-sdd1 ntfs   System Reserved 39B3E529B0AACC7A                                    
`-sdd2 ntfs                   F81499DF903B4407                      228.1G    76% /mnt/windows
sde                                                                               
`-sde1 ntfs   :/              64FA118EFA115E1C                      300.6G    68% /mnt/media
sr0

Contents of /etc/fstab:

# Static information about the filesystems.
# See fstab(5) for details.

# <file system> <dir> <type> <options> <dump> <pass>
# UUID=542dd05a-e2b0-40aa-b1a4-5a78d2fcf3ac
/dev/sda2               /               ext4            rw,relatime     0 1

# UUID=43c86715-7606-49cb-a401-0fe9693968d8
/dev/sda1               none            swap            defaults        0 0

#Media Drive
#UUID=64FA118EFA115E1C  /mnt/media      ntfs-3g         user            0 2

# Windows Drive
#UUID=F81499DF903B4407  /mnt/windows    ntfs-3g         user            0 3

(The bottom two are what I added. The ones above it are the default.)

I'll keep this session of Arch going for now.

WorMzy · 2019-11-06 18:42:59

Hm, well my theory was wrong -- your booted kernel matches the one that is installed (and you don't have a separate boot partition, so you shouldn't get that sort of disconnect anyway).

I note that you have the two partitions mounted now, but the two entries are commented out in the fstab, so you must have mounted them manually? Please post the commands you used, as well as the output of `mount`.

ZurkinWood497 · 2019-11-06 19:40:14

The kernel that I grabbed is the one from the Arch repos - When I said I reinstalled, I ran ">pacman -S linux". I forgot to specify that, sorry.

Yes, the two I commented out are the two NTFS drives I have to physically disconnect for Arch to start. Once it starts, I re-connect the drives SATA cables and run this:

# I use su because I'm paranoid, using sudo makes no difference.
>su 
>mount /dev/sdX /mnt/windows
>mount /dev/sdX /mnt/media

There's no readout on the terminal when I run the "mount" commands, it just returns the prompt. The reason I label them as "/dev/sdX", is because the letters are always changing on each startup, thus why I went to using UUID in the /etc/fstab file.

After the drives are mounted, the readout of ">mount" is as follows:

>mount
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
sys on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
dev on /dev type devtmpfs (rw,nosuid,relatime,size=12281764k,nr_inodes=3070441,mode=755)
run on /run type tmpfs (rw,nosuid,nodev,relatime,mode=755)
/dev/sda2 on / type ext4 (rw,relatime)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup2 on /sys/fs/cgroup/unified type cgroup2 (rw,nosuid,nodev,noexec,relatime,nsdelegate)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
bpf on /sys/fs/bpf type bpf (rw,nosuid,nodev,noexec,relatime,mode=700)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_cls,net_prio)
cgroup on /sys/fs/cgroup/rdma type cgroup (rw,nosuid,nodev,noexec,relatime,rdma)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpu,cpuacct)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2675)
debugfs on /sys/kernel/debug type debugfs (rw,nosuid,nodev,noexec,relatime)
mqueue on /dev/mqueue type mqueue (rw,nosuid,nodev,noexec,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime,pagesize=2M)
configfs on /sys/kernel/config type configfs (rw,nosuid,nodev,noexec,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,nosuid,nodev,noexec,relatime)
tmpfs on /tmp type tmpfs (rw,nosuid,nodev)
tmpfs on /run/user/1000 type tmpfs (rw,nosuid,nodev,relatime,size=2464436k,mode=700,uid=1000,gid=1000)
fusectl on /sys/fs/fuse/connections type fusectl (rw,nosuid,nodev,noexec,relatime)
/dev/sdd2 on /mnt/windows type fuseblk (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other,blksize=4096)
/dev/sde1 on /mnt/media type fuseblk (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other,blksize=4096)

It might be worth noting that I've had it fail during Udev before on rare occasion, but I could always restart the system and it'd be good to go. I poked around on other sites, and came across the error code it gave me those times:

-sh: can't access tty: job control turned off

No idea what that's about.

Also, thanks a bunch for bearing through with me on this, I very much appreciate it! (:

loqs · 2019-11-06 19:47:22

Is there anything logged to the journal from a boot with the issue?
From the emergency shell can you manually mount the file systems?

ZurkinWood497 · 2019-11-06 20:05:36

loqs wrote:

Is there anything logged to the journal from a boot with the issue?

I opened up ">journalctl", and I saw about an hours gap between logs, where it plainly says "reboot". Every recorded boot in there doesn't show my other two drives, so I'm believing that it's not even logging the error boots.

logs wrote:

From the emergency shell can you manually mount the file systems?

I'll have to see, but I'm writing all this on the machine in question. I'll reboot with the drives connected and see how things go.

Would a purging and re-installation of the entire init system correct this? Right now that's looking like the easier way to go, if that's at all possible.

loqs · 2019-11-06 20:25:45

Impossible to say what would fix it without knowing the cause.
The symptoms suggest a udev issue in early boot.
If udev is not creating the /dev/disk/by-uuid/* symlinks that would cause the issue.

What is the contents of /etc/mkinitcpio.conf?
Did you perform a pacman update between the last successful boot and the onset of the issue?

seth · 2019-11-06 21:03:33

Try

UUID=64FA118EFA115E1C  /mnt/media      ntfs-3g         user,noauto,x-systemd.automount    0 2

The drive will not be mounted before you attempt to access /mnt/media.

Since those seem USB drives, they might just show up really slowly or run into some usb powersaving, thus have systemd timeout.

ZurkinWood497 · 2019-11-06 23:00:50

First off, thank you seth for what you mentioned regarding /etc/fstab! I didn't know how you could keep it from auto-mounting before. That's so good to know. (:

loqs wrote:

Did you perform a pacman update between the last successful boot and the onset of the issue?

What you said got me thinking... And led to me getting this thing back up and running. I can boot with my drives auto-mounting, and all is fine again. Thank you for stating that, loqs! But... Udev still has a problem.

I did update the entire system yesterday, and didn't shut it down until late that day. So I no doubt got the issue started then.

First, these are all internal, 3.5" drives. None of them are external. When I say connect/disconnect, I'm taking the SATA cables on/off of the drives.

Here's the spill of what happened since my last post:

I decided to try both a normal boot and a fallback boot (I use SysLinux so both options are readily available,) and write down the error logs by hand. While writing them down, I noticed this line when trying the normal:

 : : running hook [udev]
 : : Triggering events...
Waiting 10 seconds for device /dev/sda2
 : : performing fsck on '/dev/sda2'
 : : mounting '/dev/sda2' on real root
mount: /new_root: unknown filesystem type 'ntfs'.
You are now being dropped into an emergency shell.
sh: can't access tty: job control turned off

This read the same every time I had /mnt/windows physically connected. My /mnt/windows drive has two ntfs partitions... So I thought: "Is it trying to boot that drive?" I disconnected it and left the other NTFS drive connected (/mnt/media), and got this result:

 : : running hook [udev]
 : : Triggering events...
Waiting 10 seconds for device /dev/sda2
 : : performing fsck on '/dev/sda2'
 : : mounting '/dev/sda2' on real root
[    2.619103] ntfs: (device sda2): load_system_files(): Volume is dirty. Mounting read-only. Run chkdisk and mount in Windows.
 : : running cleanup hook [udev]
ERROR: Root device mounted successful, but /sbin/init does not exist.
Bailing out, tou are on your own. Good luck.
sh: can't access tty: job control turned off

/mnt/media only has one partition. This read out the same every time I had it connected, with /mnt/windows disconnected.

I had both of those drives before the one Arch is installed on, so I hopped into the BIOS settings, and noticed this:

- /mnt/windows = SATA1
- /mnt/media = SATA2
- Arch = SATA3

I swapped Arch over to SATA1, hoping that it actually wouldn't fix it, and... It did. I've re-enabled the /etc/fstab automount lines, restarted the entire system... six times now? It started flawlessly every time. Now I just gotta check that my other OSes haven't messed up, heh.

I honestly consider this a 'quick fix' for the problem. Something happened that makes it think /dev/sda is on SATA1, and I don't know what. But long as I keep my Arch drive on that physical port on the motherboard, everything's fine. It hasn't done it for the past 7 months I've had Arch running on here, so... I ain't got a clue as to why this happens now.

Scimmia · 2019-11-06 23:04:59

You'll be wanting this: https://wiki.archlinux.org/index.php/Pe … ice_naming

Don't use /dev/sdxX for anything.

ZurkinWood497 · 2019-11-06 23:14:55

Scimmia wrote:

Don't use /dev/sdxX for anything.

I'm realizing more and more that using that for anything (aside from manual mounting in a terminal) causes so many issues it's insane. I've got to head out for a couple hours, but I'll swap everything over to UUIDs. I'll revert my Arch drive back to SATA3, and if using UUID corrects all of this, I'll consider this closed for good. Thank you Scimmia! (:

I'll report back with my findings when I find them.

seth · 2019-11-07 08:07:23

[ 2.619103] ntfs: (device sda2): load_system_files(): Volume is dirty. Mounting read-only. Run chkdisk and mount in Windows.

Disable https://wiki.archlinux.org/index.php/Du … t_Start-Up

Arch Linux

#1 2019-11-06 18:13:35

Udev fails, failing to boot, NTFS unrecognized

#2 2019-11-06 18:18:43

Re: Udev fails, failing to boot, NTFS unrecognized

#3 2019-11-06 18:28:56

Re: Udev fails, failing to boot, NTFS unrecognized

#4 2019-11-06 18:42:59

Re: Udev fails, failing to boot, NTFS unrecognized

#5 2019-11-06 19:40:14

Re: Udev fails, failing to boot, NTFS unrecognized

#6 2019-11-06 19:47:22

Re: Udev fails, failing to boot, NTFS unrecognized

#7 2019-11-06 20:05:36

Re: Udev fails, failing to boot, NTFS unrecognized

#8 2019-11-06 20:25:45

Re: Udev fails, failing to boot, NTFS unrecognized

#9 2019-11-06 21:03:33

Re: Udev fails, failing to boot, NTFS unrecognized

#10 2019-11-06 23:00:50

Re: Udev fails, failing to boot, NTFS unrecognized

#11 2019-11-06 23:04:59

Re: Udev fails, failing to boot, NTFS unrecognized

#12 2019-11-06 23:14:55

Re: Udev fails, failing to boot, NTFS unrecognized

#13 2019-11-07 08:07:23

Re: Udev fails, failing to boot, NTFS unrecognized

Board footer