You are not logged in.

#1 2019-01-09 08:02:17

Kotrfa
Member
Registered: 2012-10-25
Posts: 213

[solved] Boot stuck on systemd 240 - A start job is running for device

Hi fellow Archers.

Yesterday, I updated my Arch system to the latest as I do every approx 3 days and stumbled upon a weird issue. When I wanted to turn off the PC after the upgrade, it hanged for 2min1s on a job with something like (didn't take the picture, unfortunately, but there wasn't info about what the job is, just that it's root user related)

A stop job is running for Session ... for user ...

and then it shutdown fine and it doesn't show up on consequent shutdowns/reboots. Looking at the journal, I found:


Jan 07 23:35:24 think460s systemd[1]: Unmounting /home/dan...
Jan 07 23:35:24 think460s systemd[1256]: Stopping Sound Service...
Jan 07 23:35:24 think460s systemd[1256]: Stopping Virtual filesystem service - disk device monitor...
Jan 07 23:35:24 think460s systemd[1256]: Stopping D-Bus User Message Bus...
Jan 07 23:35:24 think460s systemd[1256]: Stopping Accessibility services bus...
Jan 07 23:35:24 think460s audit[1]: SERVICE_STOP pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=systemd-fsck@dev-mapper-home\x2ddan\x2dencr comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Jan 07 23:35:24 think460s umount[13213]: umount: /home/dan: target is busy.
Jan 07 23:35:24 think460s systemd[1256]: Stopping Virtual filesystem service...
Jan 07 23:35:24 think460s systemd[1256]: Stopped target Default.
Jan 07 23:35:24 think460s systemd[1]: home-dan.mount: Mount process exited, code=exited status=32
Jan 07 23:35:24 think460s systemd[1]: Failed unmounting /home/dan.
Jan 07 23:35:24 think460s systemd[1]: Stopped File System Check on /dev/mapper/home-dan-encr.
Jan 07 23:35:24 think460s systemd[1]: Unmounted /run/user/1000/gvfs.

When I booted my laptop (Thinkpad T460s) the next morning though, I got a following A start job is running for dev-mapper-homex2ddanx2dencr for 1m30s and then it timed out. This is my home partition encrypted via dmcrypt (which has been working fine for several months just fine). I can't say for sure these two are related, but it's very likely. It seems like there was something wrong during the last shutdown and now it has some issues with fscheck (which seems to go fine, just does not notify systemd correctly - the unit it says success, see below).

relevant part of the systemctl start boot log:

Jan 09 08:46:29 think460s systemd[1]: dev-mapper-home\x2ddan\x2dencr.device: Job dev-mapper-home\x2ddan\x2dencr.device/start timed out.
Jan 09 08:46:29 think460s systemd[1]: Timed out waiting for device /dev/mapper/home-dan-encr.
Jan 09 08:46:29 think460s systemd[1]: Dependency failed for /home/dan.
Jan 09 08:46:29 think460s systemd[1]: home-dan.mount: Job home-dan.mount/start failed with result 'dependency'.
Jan 09 08:46:29 think460s systemd[1]: Dependency failed for File System Check on /dev/mapper/home-dan-encr.
Jan 09 08:46:29 think460s systemd[1]: systemd-fsck@dev-mapper-home\x2ddan\x2dencr.service: Job systemd-fsck@dev-mapper-home\x2ddan\x2dencr.service/start failed with result 'dependency'.
Jan 09 08:46:29 think460s systemd[1]: dev-mapper-home\x2ddan\x2dencr.device: Job dev-mapper-home\x2ddan\x2dencr.device/start failed with result 'timeout'.
Jan 09 08:46:29 think460s systemd[1]: Reached target Local File Systems.

... few lines below...
Jan 09 08:46:40 think460s kernel: BTRFS: device fsid 83518ae3-55f8-48b2-abc6-39ffe70d0414 devid 1 transid 390001 /dev/dm-0
Jan 09 08:46:40 think460s systemd[1]: Found device /dev/mapper/home-dan-encr.
Jan 09 08:46:40 think460s systemd[1]: Starting File System Check on /dev/mapper/home-dan-encr...
Jan 09 08:46:40 think460s kernel: kauditd_printk_skb: 8 callbacks suppressed
Jan 09 08:46:40 think460s kernel: audit: type=1006 audit(1547020000.613:36): pid=943 uid=0 old-auid=4294967295 auid=1000 tty=tty1 old-ses=4294967295 ses=1 res=1
Jan 09 08:46:40 think460s login[943]: pam_unix(login:session): session opened for user dan by LOGIN(uid=0)
Jan 09 08:46:40 think460s systemd[1]: Created slice User Slice of UID 1000.
Jan 09 08:46:40 think460s systemd[1]: Starting User Runtime Directory /run/user/1000...
Jan 09 08:46:40 think460s systemd[1]: Started File System Check on /dev/mapper/home-dan-encr.
Jan 09 08:46:40 think460s audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=systemd-fsck@dev-mapper-home\x2ddan\x2dencr comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'

$ systemctl status systemd-fsck@dev-mapper-home\\x2ddan\\x2dencr.service

● systemd-fsck@dev-mapper-home\x2ddan\x2dencr.service - File System Check on /dev/mapper/home-dan-encr
   Loaded: loaded (/usr/lib/systemd/system/systemd-fsck@.service; static; vendor preset: disabled)
   Active: active (exited) since Wed 2019-01-09 08:46:40 CET; 9min ago
     Docs: man:systemd-fsck@.service(8)
  Process: 1239 ExecStart=/usr/lib/systemd/systemd-fsck /dev/mapper/home-dan-encr (code=exited, status=0/SUCCESS)
 Main PID: 1239 (code=exited, status=0/SUCCESS)

Jan 09 08:46:40 think460s systemd[1]: Starting File System Check on /dev/mapper/home-dan-encr...
Jan 09 08:46:40 think460s systemd[1]: Started File System Check on /dev/mapper/home-dan-encr.

The system boots in the end, but this is a nono thing I want to have. Additionally, the laptop keyboard just "randomly" stopped working 2/5 times when started to reboot under my regular user (dan), but I didn't reliably reproduced that (and works fine under root) and would rather solve the data issue first. Any ideas? Thanks a lot!

Last edited by Kotrfa (2019-01-20 21:46:37)

Offline

#2 2019-01-12 16:14:01

Kotrfa
Member
Registered: 2012-10-25
Posts: 213

Re: [solved] Boot stuck on systemd 240 - A start job is running for device

Anyone please? Here are those two suspected unit files generated by systemd-fstab. This problem occurred shortly after installing systemd 240.

$ systemctl cat home-dan.mount
# /run/systemd/generator/home-dan.mount
# Automatically generated by systemd-fstab-generator

[Unit]
SourcePath=/etc/fstab
Documentation=man:fstab(5) man:systemd-fstab-generator(8)
Requires=systemd-fsck@dev-mapper-home\x2ddan\x2dencr.service
After=systemd-fsck@dev-mapper-home\x2ddan\x2dencr.service

[Mount]
Where=/home/dan
What=/dev/mapper/home-dan-encr
Type=btrfs
Options=rw,noatime,nodiratime,ssd,space_cache,nofail,subvol=/,compress=lzo,subvolid=5


$ systemctl cat systemd-fsck@dev-mapper-home\x2ddan\x2dencr.service
# /usr/lib/systemd/system/systemd-fsck@.service
[Unit]
Description=File System Check on %f
Documentation=man:systemd-fsck@.service(8)
DefaultDependencies=no
BindsTo=%i.device
Conflicts=shutdown.target
After=%i.device systemd-fsck-root.service local-fs-pre.target
Before=systemd-quotacheck.service shutdown.target

[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/usr/lib/systemd/systemd-fsck %f
TimeoutSec=0

$ cat /etc/fstab
...
# encrypted storage
/dev/mapper/home-dan-encr /home/dan btrfs	rw,noatime,nodiratime,ssd,space_cache,nofail,subvol=/,compress=lzo,subvolid=5	0 2
$ systemd-analyze critical-chain
...
                          └─local-fs.target @30.467s
                            └─home-dan.mount @41.792s +41ms
                              └─systemd-fsck@dev-mapper-home\x2ddan\x2dencr.service @41.779s +10ms
                                └─dev-mapper-home\x2ddan\x2dencr.device @41.778s

I was able to reduce the timeout to 10 sec instead of 90 secs by adding `,x-systemd.device-timeout=10` to the fstab options.

Last edited by Kotrfa (2019-01-12 16:47:24)

Offline

#3 2019-01-12 16:55:02

dmartins
Member
Registered: 2006-09-23
Posts: 360

Re: [solved] Boot stuck on systemd 240 - A start job is running for device

Could you post more complete boot logs to a pastebin (bpaste.net/gist.github.com/etc.)? I suspect if any problem is shown, it is a bit before what you shared already.

Offline

#4 2019-01-12 22:47:40

Kotrfa
Member
Registered: 2012-10-25
Posts: 213

Re: [solved] Boot stuck on systemd 240 - A start job is running for device

Offline

#5 2019-01-13 20:54:48

dmartins
Member
Registered: 2006-09-23
Posts: 360

Re: [solved] Boot stuck on systemd 240 - A start job is running for device

I can't find anything obvious there, but a new version of systemd has been moved to the core repo recently. Could you try updating your system?

Offline

#6 2019-01-19 08:05:01

Kotrfa
Member
Registered: 2012-10-25
Posts: 213

Re: [solved] Boot stuck on systemd 240 - A start job is running for device

My system is up to date, systemd 240.34-3. Still the same :-(

Offline

#7 2019-01-19 10:25:09

seth
Member
Registered: 2012-09-03
Posts: 49,985

Re: [solved] Boot stuck on systemd 240 - A start job is running for device

The journal you posted does not suggest /home/dan is ever successfully mounted.
Does it?
If not, can you mount it from a live system (install iso)?

If it does, the (continuing? the journal doesn't cover any reboot) issue seems to be the failure to unmount your /home/dan partition during a shutdown.
I'd suggest to logout, switch to a linux console (ctrl+alt+f1/2/3/…), login as root, check whether /home/dan is still mounted, attempt to umount it and if that fails "lsof" to figure what is still in use (and hopefully why)

A second approach based on your suspicion was to downgrade to systemd 239, https://wiki.archlinux.org/index.php/Downgrade

Offline

#8 2019-01-19 10:50:18

Kotrfa
Member
Registered: 2012-10-25
Posts: 213

Re: [solved] Boot stuck on systemd 240 - A start job is running for device

It is. It is mounted automatically after a sign in as `dan` user. It works as expected (after a login). The same way as manually decrypting it and mounting under root.

I did what you suggested: logged out from `dan`, logged in as user, `systemctl stop user-1000.slice` to end all `dan`'s user still-on processes, lsof returned nothing on `/home/dan` (which is still mounted - again, expected, there is no automatic unmount on logging out) and I could happily `umount /home/dan`, no problem.

Yeah, living with downgraded systemd 239 is not really a nice situation given that almost everything depends on it.

Offline

#9 2019-01-19 14:56:04

seth
Member
Registered: 2012-09-03
Posts: 49,985

Re: [solved] Boot stuck on systemd 240 - A start job is running for device

The idea was more to test whether the systemd version actually makes a difference to isolate the culprit - not to solve the problem by sticking to an old version.

"systemctl stop user-1000.slice" somewhat defeats the idea to check what's lingering around and might impede the umount, so I'd suggest to try w/o to pin some possibly problematic processes.

Offline

#10 2019-01-20 19:34:52

Kotrfa
Member
Registered: 2012-10-25
Posts: 213

Re: [solved] Boot stuck on systemd 240 - A start job is running for device

Got it, thanks!

So I did `lsof | grep /home/dan > a.log` and got the following:

lsof: WARNING: can't stat() fuse.gvfsd-fuse file system /run/user/1000/gvfs
      Output information may be incomplete.
dbus-daem 1383                             dan  cwd       DIR               0,55     2254        256 /home/dan
devmon    1389                             dan  cwd       DIR               0,55     2254        256 /home/dan
devmon    1389                             dan    1w      REG               0,55     4621        347 /home/dan/.xorg.log
devmon    1389                             dan    2w      REG               0,55     4621        347 /home/dan/.xorg.log
combine_b 1403                             dan  cwd       DIR               0,55     2254        256 /home/dan
combine_b 1403                             dan    1w      REG               0,55     4621        347 /home/dan/.xorg.log
combine_b 1403                             dan    2w      REG               0,55     4621        347 /home/dan/.xorg.log
combine_b 1403                             dan  255r      REG               0,55      292      42713 /home/dan/.local/bin/combine_batteries.sh
gvfsd     1462                             dan  cwd       DIR               0,55     2254        256 /home/dan
gvfsd     1462 1464 gmain                  dan  cwd       DIR               0,55     2254        256 /home/dan
gvfsd     1462 1465 gdbus                  dan  cwd       DIR               0,55     2254        256 /home/dan
at-spi-bu 1472                             dan  cwd       DIR               0,55     2254        256 /home/dan
at-spi-bu 1472                             dan  mem       REG               0,53             4632398 /home/dan/.config/dconf/user (path dev=0,55)
at-spi-bu 1472 1498 gmain                  dan  cwd       DIR               0,55     2254        256 /home/dan
at-spi-bu 1472 1498 gmain                  dan  mem       REG               0,53             4632398 /home/dan/.config/dconf/user (path dev=0,55)
at-spi-bu 1472 1499 dconf\x20              dan  cwd       DIR               0,55     2254        256 /home/dan
at-spi-bu 1472 1499 dconf\x20              dan  mem       REG               0,53             4632398 /home/dan/.config/dconf/user (path dev=0,55)
at-spi-bu 1472 1502 gdbus                  dan  cwd       DIR               0,55     2254        256 /home/dan
at-spi-bu 1472 1502 gdbus                  dan  mem       REG               0,53             4632398 /home/dan/.config/dconf/user (path dev=0,55)
gvfsd-fus 1474                             dan  cwd       DIR               0,55     2254        256 /home/dan
gvfsd-fus 1474 1494 gvfsd-fus              dan  cwd       DIR               0,55     2254        256 /home/dan
gvfsd-fus 1474 1495 gvfsd-fus              dan  cwd       DIR               0,55     2254        256 /home/dan
gvfsd-fus 1474 1496 gmain                  dan  cwd       DIR               0,55     2254        256 /home/dan
gvfsd-fus 1474 1497 gdbus                  dan  cwd       DIR               0,55     2254        256 /home/dan
gvfsd-fus 1474 1501 gvfs-fuse              dan  cwd       DIR               0,55     2254        256 /home/dan
dbus-daem 1505                             dan  cwd       DIR               0,55     2254        256 /home/dan
redshift  1599                             dan  cwd       DIR               0,55     2254        256 /home/dan
redshift  1599 1601 geoclue2               dan  cwd       DIR               0,55     2254        256 /home/dan
redshift  1599 1602 gmain                  dan  cwd       DIR               0,55     2254        256 /home/dan
redshift  1599 1604 gdbus                  dan  cwd       DIR               0,55     2254        256 /home/dan
xfconfd   1899                             dan  cwd       DIR               0,55     2254        256 /home/dan
udevil    2134                             dan  cwd       DIR               0,55     2254        256 /home/dan
udevil    2134                             dan    2w      REG               0,55     4621        347 /home/dan/.xorg.log
sleep     2980                             dan  cwd       DIR               0,55     2254        256 /home/dan
sleep     2980                             dan    1w      REG               0,55     4621        347 /home/dan/.xorg.log
sleep     2980                             dan    2w      REG               0,55     4621        347 /home/dan/.xorg.log

Regarding the downgrade - I did the downgrade but got a lot of errors, do you think it's safe to reboot?

sudo pacman -U /var/cache/pacman/pkg/systemd-239.370-1-x86_64.pkg.tar.xz
loading packages...
warning: downgrading package systemd (240.34-3 => 239.370-1)
resolving dependencies...
looking for conflicting packages...

Package (1)  Old Version  New Version  Net Change

systemd      240.34-3     239.370-1     -0.79 MiB

Total Installed Size:  19.37 MiB
Net Upgrade Size:      -0.79 MiB

:: Proceed with installation? [Y/n] y
(1/1) checking keys in keyring                                                                                                                  [########################################################################################] 100%
(1/1) checking package integrity                                                                                                                [########################################################################################] 100%
(1/1) loading package files                                                                                                                     [########################################################################################] 100%
(1/1) checking for file conflicts                                                                                                               [########################################################################################] 100%
(1/1) checking available disk space                                                                                                             [########################################################################################] 100%
:: Processing package changes...
(1/1) downgrading systemd                                                                                                                       [########################################################################################] 100%
systemd-sysusers: error while loading shared libraries: libidn2.so.0: cannot open shared object file: No such file or directory
journalctl: error while loading shared libraries: libidn2.so.0: cannot open shared object file: No such file or directory
systemctl: error while loading shared libraries: libidn2.so.0: cannot open shared object file: No such file or directory
:: Running post-transaction hooks...
(1/9) Updating linux initcpios...
==> Building image from preset: /etc/mkinitcpio.d/linux.preset: 'default'
  -> -k /boot/vmlinuz-linux -c /etc/mkinitcpio.conf -g /boot/initramfs-linux.img
==> Starting build: 4.20.1-arch1-1-ARCH
  -> Running build hook: [base]
  -> Running build hook: [udev]
==> ERROR: binary dependency `libidn2.so.0' not found for `/usr/lib/systemd/systemd-udevd'
==> ERROR: binary dependency `libidn2.so.0' not found for `/usr/bin/udevadm'
==> ERROR: binary dependency `libidn2.so.0' not found for `/usr/bin/systemd-tmpfiles'
  -> Running build hook: [autodetect]
  -> Running build hook: [modconf]
  -> Running build hook: [block]
  -> Running build hook: [filesystems]
  -> Running build hook: [keyboard]
  -> Running build hook: [fsck]
==> Generating module dependencies
==> Creating gzip-compressed initcpio image: /boot/initramfs-linux.img
==> WARNING: errors were encountered during the build. The image may not be complete.
==> Building image from preset: /etc/mkinitcpio.d/linux.preset: 'fallback'
  -> -k /boot/vmlinuz-linux -c /etc/mkinitcpio.conf -g /boot/initramfs-linux-fallback.img -S autodetect
==> Starting build: 4.20.1-arch1-1-ARCH
  -> Running build hook: [base]
  -> Running build hook: [udev]
==> ERROR: binary dependency `libidn2.so.0' not found for `/usr/lib/systemd/systemd-udevd'
==> ERROR: binary dependency `libidn2.so.0' not found for `/usr/bin/udevadm'
==> ERROR: binary dependency `libidn2.so.0' not found for `/usr/bin/systemd-tmpfiles'
  -> Running build hook: [modconf]
  -> Running build hook: [block]
==> WARNING: Possibly missing firmware for module: aic94xx
==> WARNING: Possibly missing firmware for module: wd719x
  -> Running build hook: [filesystems]
  -> Running build hook: [keyboard]
  -> Running build hook: [fsck]
==> Generating module dependencies
==> Creating gzip-compressed initcpio image: /boot/initramfs-linux-fallback.img
==> WARNING: errors were encountered during the build. The image may not be complete.
error: command failed to execute correctly
(2/9) Updating journal message catalog...
/usr/bin/journalctl: error while loading shared libraries: libidn2.so.0: cannot open shared object file: No such file or directory
error: command failed to execute correctly
(3/9) Reloading system manager configuration...
/usr/bin/systemctl: error while loading shared libraries: libidn2.so.0: cannot open shared object file: No such file or directory
error: command failed to execute correctly
(4/9) Updating udev hardware database...
/usr/bin/systemd-hwdb: error while loading shared libraries: libidn2.so.0: cannot open shared object file: No such file or directory
error: command failed to execute correctly
(5/9) Applying kernel sysctl settings...
/usr/lib/systemd/systemd-sysctl: error while loading shared libraries: libidn2.so.0: cannot open shared object file: No such file or directory
error: command failed to execute correctly
(6/9) Creating system user accounts...
/usr/bin/systemd-sysusers: error while loading shared libraries: libidn2.so.0: cannot open shared object file: No such file or directory
error: command failed to execute correctly
(7/9) Creating temporary files...
/usr/bin/systemd-tmpfiles: error while loading shared libraries: libidn2.so.0: cannot open shared object file: No such file or directory
error: command failed to execute correctly
(8/9) Reloading device manager configuration...
/usr/bin/udevadm: error while loading shared libraries: libidn2.so.0: cannot open shared object file: No such file or directory
error: command failed to execute correctly
(9/9) Arming ConditionNeedsUpdate...

Offline

#11 2019-01-20 20:38:26

seth
Member
Registered: 2012-09-03
Posts: 49,985

Re: [solved] Boot stuck on systemd 240 - A start job is running for device

NO! You'll also have to downgrade at least libidn2

You're using startx/xinit? Please post your xinitrc.

Offline

#12 2019-01-20 20:41:26

Kotrfa
Member
Registered: 2012-10-25
Posts: 213

Re: [solved] Boot stuck on systemd 240 - A start job is running for device

OK, will try to get to a state when I see no warnings...

.xinitrc contains:

source /etc/X11/xinit/xinitrc.d/50-systemd-user.sh # notifications
exec i3

where:

$ cat /etc/X11/xinit/xinitrc.d/50-systemd-user.sh     
#!/bin/sh

systemctl --user import-environment DISPLAY XAUTHORITY

if command -v dbus-update-activation-environment >/dev/null 2>&1; then
        dbus-update-activation-environment DISPLAY XAUTHORITY
fi

but it's definitely nothing new. Have had that there for ages.

I really appreciate your help. Really.

Offline

#13 2019-01-20 20:49:54

Kotrfa
Member
Registered: 2012-10-25
Posts: 213

Re: [solved] Boot stuck on systemd 240 - A start job is running for device

By the way I tried to downgrade the `libidn2`, which broke pacman entirely complaining about ` error while loading shared libraries: libidn2.so` and also almost everything broke down such as xfce terminal didn't start. I had to untar the new version in different tty, manually copy it to /usr/lib and it made the pacman work again. Usually, when I hit issues such this I stop - do you think it's worth trying to find a suitable version of all dependent packages, given that it would be "just" for narrowing the scope?

Offline

#14 2019-01-20 20:53:14

seth
Member
Registered: 2012-09-03
Posts: 49,985

Re: [solved] Boot stuck on systemd 240 - A start job is running for device

That's beyond the point ;-)

I guess you're running some custom script on i3 (or they're kicked off by i3bar etc.) incl. combine_batteries.sh and they trigger at least devmon/udevil and redshift and are run out of the sessions process group.
IOW those PIDs are not killed w/ the session and I assume one of them prevents the umount of your $HOME.

So check which are still running and bisect for the culprit(s), ie. kill half of them and see if you get a clean shutdown. If yes, you narrowed down the contenders, if not, there's at least an issue w/ the remaining half.

Ultimately, draw the culprits (or all of them) into the i3 process group (since it's most likely to support something in the graphical session anyway?)

You can postpone systemd downgrades, rather maintain a usable system.

Offline

#15 2019-01-20 21:10:34

Kotrfa
Member
Registered: 2012-10-25
Posts: 213

Re: [solved] Boot stuck on systemd 240 - A start job is running for device

Cool! Good thinking.

I tried not to login into the graphical session (i3), but just stayed in the console (sorry, didn't realize it could influence the result). So I just tried to login in tty to the user `dan`, then logged out and did the same `lsof` as root and the output was clean (no processes on /home/dan)!

Nevertheless, when I rebooted the machine, the result was the same - on the next boot, I got the timeout. So it can't be i3 (although you found some mess :-D )...

Offline

#16 2019-01-20 21:20:57

seth
Member
Registered: 2012-09-03
Posts: 49,985

Re: [solved] Boot stuck on systemd 240 - A start job is running for device

After you explicitly umount'ed before rebooting, did you still get the same boot delay?
Try to explicitly fsck the partition before a supposingly clean reboot.

Offline

#17 2019-01-20 21:30:07

Kotrfa
Member
Registered: 2012-10-25
Posts: 213

Re: [solved] Boot stuck on systemd 240 - A start job is running for device

Yes, I get the delay even after I do a clean unmount (I had to manually kill dbus-daemon - that one is causing problem of unmounting it cleanly, after that there is no "target is busy" - solved by https://github.com/systemd/systemd/issues/2975).

I did the filesystem `btrfs check` on the partition which is causing the issue - no errors nor warnings. So really don't know :-/ .

Oh and I checked the shutdown log, the relevant part seems fine:

# mounting when signing in as a user
Jan 20 22:27:08 think460s kernel: device-mapper: uevent: version 1.0.3
Jan 20 22:27:08 think460s kernel: device-mapper: ioctl: 4.39.0-ioctl (2018-04-03) initialised: dm-devel@redhat.com
Jan 20 22:27:09 think460s kernel: NET: Registered protocol family 38
Jan 20 22:27:09 think460s kernel: BTRFS: device fsid 83518ae3-55f8-48b2-abc6-39ffe70d0414 devid 1 transid 405630 /dev/dm-0
Jan 20 22:27:09 think460s systemd[1]: Found device /dev/mapper/home-dan-encr.
Jan 20 22:27:09 think460s systemd[1]: Starting File System Check on /dev/mapper/home-dan-encr...
Jan 20 22:27:09 think460s kernel: audit: type=1006 audit(1548019629.369:30): pid=933 uid=0 old-auid=4294967295 auid=1000 tty=tty1 old-ses=4294967295 ses=1 res=1
Jan 20 22:27:09 think460s login[933]: pam_unix(login:session): session opened for user dan by LOGIN(uid=0)
Jan 20 22:27:09 think460s systemd[1]: Started File System Check on /dev/mapper/home-dan-encr.
Jan 20 22:27:09 think460s audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=systemd-fsck@dev-mapper-home\x2ddan\x2dencr comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Jan 20 22:27:09 think460s kernel: audit: type=1130 audit(1548019629.379:31): pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=systemd-fsck@dev-mapper-home\x2ddan\x2dencr comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? >
Jan 20 22:27:09 think460s systemd-logind[706]: New session 1 of user dan.
Jan 20 22:27:09 think460s systemd[1]: home-dan.mount: Directory /home/dan to mount over is not empty, mounting anyway.
Jan 20 22:27:09 think460s systemd[1]: Mounting /home/dan...
Jan 20 22:27:09 think460s systemd[1]: Created slice User Slice of UID 1000.
Jan 20 22:27:09 think460s systemd[1]: Starting User Runtime Directory /run/user/1000...


# unmounting
Jan 20 22:33:15 think460s systemd[1]: Stopped target Local File Systems.
Jan 20 22:33:15 think460s systemd[1]: Unmounting /home/dan...
Jan 20 22:33:15 think460s systemd[1]: Unmounting Temporary Directory (/tmp)...
Jan 20 22:33:15 think460s systemd[1]: Unmounting /boot...
Jan 20 22:33:15 think460s systemd[1]: tmp.mount: Succeeded.
Jan 20 22:33:15 think460s systemd[1]: Unmounted Temporary Directory (/tmp).
Jan 20 22:33:15 think460s systemd[1]: boot.mount: Succeeded.
Jan 20 22:33:15 think460s systemd[1]: Unmounted /boot.
Jan 20 22:33:15 think460s systemd[1]: systemd-fsck@dev-sda2.service: Succeeded.
Jan 20 22:33:15 think460s systemd[1]: Stopped File System Check on /dev/sda2.
Jan 20 22:33:15 think460s audit[1]: SERVICE_STOP pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=systemd-fsck@dev-sda2 comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Jan 20 22:33:15 think460s systemd[1]: Stopped target Swap.
Jan 20 22:33:15 think460s systemd[1]: Deactivating swap /dev/disk/by-partuuid/2f6ae529-28b5-4a25-a6a7-579d8ded0168...
Jan 20 22:33:15 think460s systemd[1]: dev-disk-by\x2dpartlabel-swap.swap: Succeeded.
Jan 20 22:33:15 think460s systemd[1]: Deactivated swap /dev/disk/by-partlabel/swap.
Jan 20 22:33:15 think460s systemd[1]: dev-disk-by\x2did-wwn\x2d0x5001b448b41b9c53\x2dpart7.swap: Succeeded.
Jan 20 22:33:15 think460s systemd[1]: Deactivated swap /dev/disk/by-id/wwn-0x5001b448b41b9c53-part7.
Jan 20 22:33:15 think460s systemd[1]: dev-disk-by\x2dlabel-swap.swap: Succeeded.
Jan 20 22:33:15 think460s systemd[1]: Deactivated swap /dev/disk/by-label/swap.
Jan 20 22:33:15 think460s systemd[1]: dev-disk-by\x2did-ata\x2dSanDisk_SD8TN8U512G1001_170343800337\x2dpart7.swap: Succeeded.
Jan 20 22:33:15 think460s systemd[1]: Deactivated swap /dev/disk/by-id/ata-SanDisk_SD8TN8U512G1001_170343800337-part7.
Jan 20 22:33:15 think460s systemd[1]: dev-disk-by\x2dpath-pci\x2d0000:00:17.0\x2data\x2d2\x2dpart7.swap: Succeeded.
Jan 20 22:33:15 think460s systemd[1]: Deactivated swap /dev/disk/by-path/pci-0000:00:17.0-ata-2-part7.
Jan 20 22:33:15 think460s systemd[1]: dev-disk-by\x2dpartuuid-2f6ae529\x2d28b5\x2d4a25\x2da6a7\x2d579d8ded0168.swap: Succeeded.
Jan 20 22:33:15 think460s systemd[1]: Deactivated swap /dev/disk/by-partuuid/2f6ae529-28b5-4a25-a6a7-579d8ded0168.
Jan 20 22:33:15 think460s systemd[1]: dev-sda7.swap: Succeeded.
Jan 20 22:33:15 think460s systemd[1]: Deactivated swap /dev/sda7.
Jan 20 22:33:15 think460s systemd[1]: dev-disk-by\x2duuid-dc41d95d\x2def9f\x2d4d5c\x2dbb9b\x2de8ff02ca5aff.swap: Succeeded.
Jan 20 22:33:15 think460s systemd[1]: Deactivated swap /dev/disk/by-uuid/dc41d95d-ef9f-4d5c-bb9b-e8ff02ca5aff.
Jan 20 22:33:15 think460s systemd[1]: home-dan.mount: Succeeded.
Jan 20 22:33:15 think460s systemd[1]: Unmounted /home/dan.
Jan 20 22:33:15 think460s systemd[1]: Reached target Unmount All Filesystems.

But trying hard, isn't it weird that the system is trying to filesystem check an encrypted system? Then I would properly expect it would fail?

Jan 20 22:33:39 think460s systemd[1]: dev-mapper-home\x2ddan\x2dencr.device: Job dev-mapper-home\x2ddan\x2dencr.device/start timed out.
Jan 20 22:33:39 think460s systemd[1]: Timed out waiting for device /dev/mapper/home-dan-encr.
Jan 20 22:33:39 think460s systemd[1]: Dependency failed for File System Check on /dev/mapper/home-dan-encr.
Jan 20 22:33:39 think460s systemd[1]: Dependency failed for /home/dan.
Jan 20 22:33:39 think460s systemd[1]: home-dan.mount: Job home-dan.mount/start failed with result 'dependency'.
Jan 20 22:33:39 think460s systemd[1]: systemd-fsck@dev-mapper-home\x2ddan\x2dencr.service: Job systemd-fsck@dev-mapper-home\x2ddan\x2dencr.service/start failed with result 'dependency'.
Jan 20 22:33:39 think460s systemd[1]: dev-mapper-home\x2ddan\x2dencr.device: Job dev-mapper-home\x2ddan\x2dencr.device/start failed with result 'timeout'.
Jan 20 22:33:39 think460s systemd[1]: Reached target Local File Systems.

not sure why it would do it now and not before...

Last edited by Kotrfa (2019-01-20 21:41:56)

Offline

#18 2019-01-20 21:46:21

Kotrfa
Member
Registered: 2012-10-25
Posts: 213

Re: [solved] Boot stuck on systemd 240 - A start job is running for device

YES! That last thing was that! I didn't had `noauto` in my fstab entry for the encrypted partition. I have no idea why it started to do this just recently, but it may be some change in behavior of systemd 240. Anyway, solved.

I really appreciate your help seth and dmartins!

Offline

Board footer

Powered by FluxBB