You are not logged in.

#1 2012-12-05 12:28:42

heman
Banned
Registered: 2012-12-02
Posts: 14

Systemd error when using separate /var partition

I recently moved /var to a new partition and have noticed this error in journalctl log

Dec 05 07:48:00 archlinux systemd[1]: Started Console System Shutdown Logging.
Dec 05 07:48:00 archlinux systemd[1]: var.mount mount process exited, code=exited status=32
Dec 05 07:48:00 archlinux systemd[1]: Failed unmounting /var.

I added an entry in fstab for /var

/dev/sda4 /var  ext4 defaults,noatime,discard 0 1

Drive is ssd and I simply added the options I have for / is this correct?

Offline

#2 2012-12-05 13:21:33

10PinkPanther
Member
From: Croatia
Registered: 2012-11-10
Posts: 374

Re: Systemd error when using separate /var partition

Did you only add this to fstab or did you actualy made /var partition with cfdisk or fdisk?

Offline

#3 2012-12-05 13:22:18

krest
Member
From: Athens, Greece
Registered: 2009-01-22
Posts: 10

Re: Systemd error when using separate /var partition

Probably not relevant, but change

/dev/sda4 /var  ext4 defaults,noatime,discard 0 1

to

/dev/sda4 /var  ext4 defaults,noatime,discard 0 2

Offline

#4 2012-12-05 15:36:06

heman
Banned
Registered: 2012-12-02
Posts: 14

Re: Systemd error when using separate /var partition

@10PinkPanther Yes I created partition and it is mounted ok.

Offline

#5 2012-12-05 18:07:14

Jristz
Member
From: America/Santiago
Registered: 2011-06-11
Posts: 1,022

Re: Systemd error when using separate /var partition

why not use UUID, probably the reboots meed the order of the drives (orchange the name or anithynk like it)


Well, I suppose that this is somekind of signature, no?

Offline

#6 2012-12-07 17:49:06

Foucault
Member
From: Athens, Greece
Registered: 2010-04-06
Posts: 214

Re: Systemd error when using separate /var partition

I have the exact same problem. I can not even umount /var manually even though lsof returns nothing. The relevant fstab line

# /dev/sdb2 LABEL=var
UUID=a24cb81b-a0b1-4c4d-a625-55d1c9c2a88a       /var            reiserfs        rw,relatime     0 2

Does anyone have any insight on this issue? I've also tried adding the shutdown hook to mkinitcpio as per this post but no luck.

Last edited by Foucault (2012-12-07 18:02:20)

Offline

#7 2012-12-10 23:00:09

nixpunk
Member
Registered: 2009-11-23
Posts: 271

Re: Systemd error when using separate /var partition

Same issue here with SATA drive and separate /var partition.  Shutdown hook didn't help either.  I remember having this issue in the past before using systemd, but I can't remember how I fixed it.

Last edited by nixpunk (2012-12-10 23:00:52)

Offline

#8 2012-12-11 04:13:49

hunterthomson
Member
Registered: 2008-06-22
Posts: 794
Website

Re: Systemd error when using separate /var partition

Yep, this is a problem with journald

Systemd can not unmount /var becuase journald is still writing log files to /var/log/journal/

The easy way to go is to delete the folder /var/log/journal
Then journald will write it's logs to /run

The correct way to fix this is to edit the UNIT files so it dose not try to umount /var until after journald has been stopped. However, I do not know if this is possible.

Last edited by hunterthomson (2012-12-11 04:14:49)


OpenBSD-current Thinkpad X230, i7-3520M, 16GB CL9 Kingston, Samsung 830 256GB
Contributor: linux-grsec

Offline

#9 2012-12-11 04:21:10

falconindy
Developer
From: New York, USA
Registered: 2009-10-22
Posts: 4,111
Website

Re: Systemd error when using separate /var partition

No, the correct "fix" for this is to use mkinitcpio's shutdown hook -- consider it a requirement for having /var on a separate partition (and this isn't particularly anything new). Deleting /var/log/journal is futile, as it's packaged with systemd.

Offline

#10 2012-12-11 04:27:19

hunterthomson
Member
Registered: 2008-06-22
Posts: 794
Website

Re: Systemd error when using separate /var partition

Well, it is not futile, when you delete it journald dose not try to recreate it. It just writes the logs to /run by default.

Awe okay, so like I should change my HOOKS= array from this

HOOKS="base udev autodetect block keymap encrypt lvm2 filesystems usbinput fsck"

To this?

HOOKS="base udev autodetect block keymap encrypt lvm2 filesystems usbinput fsck var shutdown"

Or what should the order be?

Last edited by hunterthomson (2012-12-11 04:29:10)


OpenBSD-current Thinkpad X230, i7-3520M, 16GB CL9 Kingston, Samsung 830 256GB
Contributor: linux-grsec

Offline

#11 2012-12-11 04:31:29

WonderWoofy
Member
From: Los Gatos, CA
Registered: 2012-05-19
Posts: 8,414

Re: Systemd error when using separate /var partition

I don't think there is a var hook.  Only a usr hook.

If you want to stop persistent journaling, I think the better solution would be to use /etc/systemd/journald.conf.

Offline

#12 2012-12-11 04:47:21

hunterthomson
Member
Registered: 2008-06-22
Posts: 794
Website

Re: Systemd error when using separate /var partition

Well falconindy is an arch dev. After reading his post again... I think he said I just need to add the "shutdown" hook, not anything ellse.

I'll give it a shot and report back.

But ya, WonderWoofy. I think I could like add "Storage=/run/journal" to /etc/systemd/journald.conf would work too.


OpenBSD-current Thinkpad X230, i7-3520M, 16GB CL9 Kingston, Samsung 830 256GB
Contributor: linux-grsec

Offline

#13 2012-12-11 04:51:28

falconindy
Developer
From: New York, USA
Registered: 2009-10-22
Posts: 4,111
Website

Re: Systemd error when using separate /var partition

journald.conf(5) would tell you the correct value is "volatile" if you want the journal stored in /run/log, rather than /var/log

Offline

#14 2012-12-11 04:56:37

HungGarTiger
Member
From: nz/auckland/
Registered: 2012-06-27
Posts: 187

Re: Systemd error when using separate /var partition

same problem here, I followed the advice in the other thread posted but the problem persists. Here is my mkinitcpio..

HOOKS="base udev autodetect modconf block filesystems usbinput fsck shutdown"

Also checked journald.conf, but couldn't really find any relevent options. Mostly just detailing speed/where to write to journal

Last edited by HungGarTiger (2012-12-11 04:57:45)


"No sympathy for the devil. If you buy the ticket, take the ride."
- Hunter S. Thompson

Offline

#15 2012-12-11 05:16:00

hunterthomson
Member
Registered: 2008-06-22
Posts: 794
Website

Re: Systemd error when using separate /var partition

falconindy wrote:

journald.conf(5) would tell you the correct value is "volatile" if you want the journal stored in /run/log, rather than /var/log

Awe yes, after a few reboots. I decided reading the man page would be a good way to go tongue
man journald.conf

Adding shutdown to the end of the HOOKS array and rebuilding the initramfs did not work. I still got the failed to umount /var

But, adding Storage=volatile to /etc/systemd/journald.conf did the trick and I deleted /var/log/journal Now my logs are in /run/log/journal which is fine with me.

Last edited by hunterthomson (2012-12-11 05:17:12)


OpenBSD-current Thinkpad X230, i7-3520M, 16GB CL9 Kingston, Samsung 830 256GB
Contributor: linux-grsec

Offline

#16 2012-12-11 05:17:17

falconindy
Developer
From: New York, USA
Registered: 2009-10-22
Posts: 4,111
Website

Re: Systemd error when using separate /var partition

Right, systemd will still try to unmount it. It's still going to fail. The shutdown hook will succeed, but it isn't going to tell you about it.

Offline

#17 2012-12-11 05:31:07

hunterthomson
Member
Registered: 2008-06-22
Posts: 794
Website

Re: Systemd error when using separate /var partition

Okay, that makes complete sense. Journald is stoped at that time so how could it write that /var was latter umounted correctly to a log on a filesytem that is not even mounted anymore.

Last edited by hunterthomson (2012-12-11 05:46:35)


OpenBSD-current Thinkpad X230, i7-3520M, 16GB CL9 Kingston, Samsung 830 256GB
Contributor: linux-grsec

Offline

#18 2012-12-11 05:55:55

HungGarTiger
Member
From: nz/auckland/
Registered: 2012-06-27
Posts: 187

Re: Systemd error when using separate /var partition

hunterthomson wrote:

Okay, that makes complete sense. Journald is stoped at that time so how could it write that /var was latter umounted correctly to a log on a filesytem that is not even mounted anymore.

Hmmmm, I never thought of it so logically. So leaving things as they are is fine then? And the error message is no problem?

The error message is actually showing a successful unmount by journald not being able to write to the log file....


"No sympathy for the devil. If you buy the ticket, take the ride."
- Hunter S. Thompson

Offline

#19 2012-12-11 06:17:08

hunterthomson
Member
Registered: 2008-06-22
Posts: 794
Website

Re: Systemd error when using separate /var partition

HungGarTiger wrote:

The error message is actually showing a successful unmount by journald not being able to write to the log file....

What? I don't understand.

The log says "umounting /var" then "Failed to umount /var"

What I think falconindy is saying is that, yes systemd will still try and fail to umount /var. However, if you add "shutdown" to the end of your HOOKS="" array in /etc/minitcpio.conf and rebuild the initramfs. That shutdown hook will umount /var after all the systemd and journald stuff is no longer running.

Personaly, I'm just going have journald write the journal to /run/log/journal. I run AIDE before and after every upgrade, so I will know if anything funky happens to my system. A log would be usefull, but as long as I know what files have been changed I can recover them from my backups.

Last edited by hunterthomson (2012-12-11 06:22:36)


OpenBSD-current Thinkpad X230, i7-3520M, 16GB CL9 Kingston, Samsung 830 256GB
Contributor: linux-grsec

Offline

#20 2012-12-11 06:42:28

HungGarTiger
Member
From: nz/auckland/
Registered: 2012-06-27
Posts: 187

Re: Systemd error when using separate /var partition

hunterthomson wrote:
HungGarTiger wrote:

The error message is actually showing a successful unmount by journald not being able to write to the log file....

What? I don't understand.

The log says "umounting /var" then "Failed to umount /var"

What I think falconindy is saying is that, yes systemd will still try and fail to umount /var. However, if you add "shutdown" to the end of your HOOKS="" array in /etc/minitcpio.conf and rebuild the initramfs. That shutdown hook will umount /var after all the systemd and journald stuff is no longer running.

Ok I get it now, thanks for the insight dude.


"No sympathy for the devil. If you buy the ticket, take the ride."
- Hunter S. Thompson

Offline

#21 2012-12-11 14:01:45

nixpunk
Member
Registered: 2009-11-23
Posts: 271

Re: Systemd error when using separate /var partition

falconindy wrote:

Right, systemd will still try to unmount it. It's still going to fail. The shutdown hook will succeed, but it isn't going to tell you about it.

Ahh.  That makes sense.

Offline

#22 2013-03-18 23:30:25

cfr
Member
From: Cymru
Registered: 2011-11-27
Posts: 7,130

Re: Systemd error when using separate /var partition

Does this only work with newer kernels? With the LTS kernel, I get

device-mapper: remove ioctl on lvm failed: Device or resource busy

repeated many times (more than a screen) on shutdown and I find "Failed unmounting /var." in the journal. A bit later, I then get "kernel: watchdog watchdog0: watchdog did not stop!" in the journal and the next message notes that journald has stopped.

I don't see the same chaos usually. (I'm not about to risk rebooting with the current kernel as I have problems booting right now but I certainly didn't see this prior to today's update with stable current kernels.)


CLI Paste | How To Ask Questions

Arch Linux | x86_64 | GPT | EFI boot | refind | stub loader | systemd | LVM2 on LUKS
Lenovo x270 | Intel(R) Core(TM) i5-7200U CPU @ 2.50GHz | Intel Wireless 8265/8275 | US keyboard w/ Euro | 512G NVMe INTEL SSDPEKKF512G7L

Offline

#23 2015-11-09 19:32:51

mkkot
Member
From: Poland
Registered: 2009-12-20
Posts: 287

Re: Systemd error when using separate /var partition

Hello all,

I have /var on separate partition and the same problem with Failed unmounting /var message when powering off my system. Within few months I had 2 times problems with filesystem on /var (which was xfs) so I finally got angry and re-formatted it to ext4 (as I can't really check if xfs filesystem was cleanly unmounted or not because of it's utilities which won't tell me that or I can't force them to do that).

After that I made a simple test: I reset on reset button my pc, booted from pendrive and checked filesystem via fsck. On each partition there was a message "Replaying journal" and then filesystem clean. There obviously should be no Replaying journal message beforehand when filesystem gets unmounted in proper manner.

So in next step I started my Archlinux and rebooted it. I saw a message that /var was not unmounted properly, so I booted up again live archlinux from pendrive and checked all partitions with fsck.ext4. The result is that all of them were clean, including /var. So there is nothing to be worried about, as in my case the filesystem is really being unmounted in proper manner.

Offline

Board footer

Powered by FluxBB