You are not logged in.

#1 2013-01-01 17:00:44

toothandnail
Member
From: Oxfordshire, UK
Registered: 2012-05-05
Posts: 88

Shutdown problem unmounting NFS shares

I have two remote machines that provide NFS mounts. One is a small server (Slackware based) the other a NAS running an OpenWrt implementation.

I can mount either or both shares and use them without any problems. My laptop will shut down without problems with either of the shares mounted. However, if both are mounted, the system will get as far as 'unmounting file systems' and hang.

Behaviour is quite consistent. So long as I remember to unmount one of the shares before shutdown, I have no problems. If I forget and both are still mounted, the system fails to shutdown and I have to use the power switch to get the machine to shutdown.

Does anyone have any ideas as to what I should be checking to find the cause of the problem?

Paul.

Offline

#2 2013-01-01 17:35:37

der_joachim
Member
From: the Netherlands
Registered: 2008-12-17
Posts: 143
Website

Re: Shutdown problem unmounting NFS shares

When shutting down, are the NFS servers still up and running? I had a similar problem. At home I have a NAS that is only up during evenings and weekend days. When shutting down outside these hours, my laptop would hang on unmounting the NFS shares. I found two solutions:

  • Mount the shares with the SOFT option. Beware though: the man page warns of possible data corruption. Never had any though.

  • Configure autofs and have it mount and unmount on the fly. Whenever a share goes up or down, the share is automagically mounted or unmounted. The autofs wiki page is very helpful.

This will hopefully help you along as well. smile


Geek, runner, motorcyclist and professional know-it-all

Offline

#3 2013-01-01 19:05:30

toothandnail
Member
From: Oxfordshire, UK
Registered: 2012-05-05
Posts: 88

Re: Shutdown problem unmounting NFS shares

der_joachim wrote:

When shutting down, are the NFS servers still up and running? I had a similar problem. At home I have a NAS that is only up during evenings and weekend days. When shutting down outside these hours, my laptop would hang on unmounting the NFS shares. I found two solutions:

Both servers are up and running. But, thinking about it, I have thought of something that might affect things. My homebrew server shuts down the storage drives after 10 minutes of inactivity. The Raidsonic NAS also shuts its drives down, though I can't remember what the timeout is. I'm now wondering if that is what is causing the hang - maybe the system is waiting for the drives to spin up. Though if that is the case, it seems that the drives on the homebrew box aren't comig back up. Not sure if I can do anything to change that, if it is the cause of the problem.

  • Mount the shares with the SOFT option. Beware though: the man page warns of possible data corruption. Never had any though.

  • Configure autofs and have it mount and unmount on the fly. Whenever a share goes up or down, the share is automagically mounted or unmounted. The autofs wiki page is very helpful.

I've never liked the soft mount option - I read the man page a long time ago... Autofs might be an option

This will hopefully help you along as well. smile

Thanks. Given me a few things to think of, anyway. I tried adding an couple of unmount commands to /etc/rc.local.shutdown, which worked once, but didn't the second time I tried it (which might well have been after the drives had powered down).

Paul.

Offline

#4 2013-01-02 00:22:25

skunktrader
Member
From: Brisbane, Australia
Registered: 2010-02-14
Posts: 1,538

Re: Shutdown problem unmounting NFS shares

In my experience it doesn't actually "hang" if the nfs server is down.  There is a long (2 minute?) timeout, after which the shutdown completes sucessfully

Offline

#5 2013-01-21 07:39:45

toothandnail
Member
From: Oxfordshire, UK
Registered: 2012-05-05
Posts: 88

Re: Shutdown problem unmounting NFS shares

skunktrader wrote:

In my experience it doesn't actually "hang" if the nfs server is down.  There is a long (2 minute?) timeout, after which the shutdown completes sucessfully

Sorry for the slow response - bad couple of weeks....

In this instance, it really is a hang - I've waited for more than 30 minutes on a couple of occasions.

I've also discovered a few other things. If I shut the laptop down very soon after mounting the NFS shares, it usually doesn't hang. It will also hang on either share if the share has been mounted for a while (or in use, not sure if that is the deciding factor).

I've tried changing the export parameters, which doesn't seem to make any difference. At the moment, I'm running these:

/mnt/sda2/stor 192.168.1.0/255.255.255.0(rw,sync,no_wdelay,nohide,no_root_squash)

(that's for the Raidsonic NAS) and:

/home/files/data-1/stor	192.168.1.0/255.255.255.0(no_root_squash,rw)

for the Slackware server.

I've also discovered that the problem is Arch specific. I've got two other distros installed on my laptop - SalineOS (based on Debian stable) and Salix (Slackware based). The problem doesn't happen with either of those machines. I'm wondering if it is something I've not got correctly set in systemd, though at the moment I'm at a loss to work out what.

Still hoping someone has an idea how to fix this - it is really annoying smile

Paul.

Offline

#6 2013-03-13 13:15:30

skanky
Member
From: WAIS
Registered: 2009-10-23
Posts: 1,847

Re: Shutdown problem unmounting NFS shares

I don't think this is a necro-bump - it's less than two months.

@toothandnail - did you resolve this?

Anyway, I've been getting the same problem, even with "soft" mounts. I've installed AutoFS and it generally solves the problem. There is one situation where I still get the hang, and that's if I've accessed the drive and shut down the "server" machine, then the laptop, before the AutoFS timeout. This doesn't happen very often, but it's not unheard of at the end of the day if I qiuickly access something before shutting down. To reduce the occurrance even further I've greatly reduced the timeout in AutoFS. It's usable but a bit messy.

I did look at running a script on shutdown to check for the NFS shares and forcibly umount them if there, but I can't get my head round how systemd shuts down, in what calls what - esp. the systemd-poweroff.service uses force:

ExecStart=/usr/bin/systemctl --force poweroff

Something tells me that if I could get a handle on that, I could possibly configure it to handle the missing NFS mounts.

I've got a mate who works in a Unix shop and they can similar issues with NFS mounts, but that "soft" mounting should fix it, though they tend not to. I realise NFS on Linux is different to NFS on Unix, but anyone know why "soft" mounts don't work here?


"...one cannot be angry when one looks at a penguin."  - John Ruskin
"Life in general is a bit shit, and so too is the internet. And that's all there is." - scepticisle

Offline

#7 2013-03-14 06:15:48

Padfoot
Member
Registered: 2010-09-03
Posts: 381

Re: Shutdown problem unmounting NFS shares

I used to experience a similar issue but with cifs (samba) shares on my network.

I found that during shutdown, the network was brought down before all the network shares were unmounted, causing the system hang during shutdown.

This sounds similar in that when you have one mounted, it works ok, but with both, the hang occurs. This makes me thing the network comes down before the second network share is unmounted.

What network manager are you using? I am using netcfg, and it does have options to run scripts post up and pre down, allowing you to mount/unmount your shares automatically based on the network state. NetworkManager has a similar config.

Perhaps this could be your issue?

Cheers.

Offline

#8 2013-03-14 09:59:57

skanky
Member
From: WAIS
Registered: 2009-10-23
Posts: 1,847

Re: Shutdown problem unmounting NFS shares

Thanks for the response.
That may (have) be(en) an issue, as it rings a bell. I don't recall there being a hang though, if I shut the laptop down first recently (even before using autofs). It may be that the soft mounts "fixed" that issue.
I'll do a test at some point, hopefully today.

I don't use a network manager at the moment, just a self-written systemd unit for wireless and dhcpd for ethernet. This will change though only to introduce ifpulgd at some point.


"...one cannot be angry when one looks at a penguin."  - John Ruskin
"Life in general is a bit shit, and so too is the internet. And that's all there is." - scepticisle

Offline

#9 2013-03-15 09:24:33

skanky
Member
From: WAIS
Registered: 2009-10-23
Posts: 1,847

Re: Shutdown problem unmounting NFS shares

An update, I tried shutting down while the NFS shares were mounted yesterday and there was no problem. So it is just the umount of stale/dead shares that's the issue.


"...one cannot be angry when one looks at a penguin."  - John Ruskin
"Life in general is a bit shit, and so too is the internet. And that's all there is." - scepticisle

Offline

#10 2013-08-22 20:18:03

Markus.N2
Member
From: Germany
Registered: 2013-08-22
Posts: 99

Re: Shutdown problem unmounting NFS shares

Is there any progress on this topic ?

I am new to Arch Linux and I stumbled on this issue, too. My installation is based on the 2013-08 snapshot CD. I have seven NFS mounts. When I umount one of them, the system shuts down normally. But when all of them are mounted, shutdown takes hours. In fact, it does not hang completely. When I issue the shutdown command and go to bed, it is shut down next morning. On the console, it enumerates my NFS mounts and complains about stale NFS handles for each of them. And there is more than one loop.

BTW... it also fails in automounting the NFS shares at boot (options=soft,bg) and mount -a takes about 20-25 seconds.
Umount all NFS shares and mount -a again is fast.

Offline

#11 2013-08-23 01:40:34

toothandnail
Member
From: Oxfordshire, UK
Registered: 2012-05-05
Posts: 88

Re: Shutdown problem unmounting NFS shares

skanky wrote:

I don't think this is a necro-bump - it's less than two months.

smile

@toothandnail - did you resolve this?

No, still the same problem. I thought I had it at least partially fixed a little while ago by forcing NFS v3 mounts, but the nfs-utils package has been updated a couple of times since then and the problem is back, just as it was before...

Anyway, I've been getting the same problem, even with "soft" mounts. I've installed AutoFS and it generally solves the problem. There is one situation where I still get the hang, and that's if I've accessed the drive and shut down the "server" machine, then the laptop, before the AutoFS timeout. This doesn't happen very often, but it's not unheard of at the end of the day if I qiuickly access something before shutting down. To reduce the occurrance even further I've greatly reduced the timeout in AutoFS. It's usable but a bit messy.

I've thought about installing AutoFS, but haven't got round to doing anything about it - lots of other things to occupy my limited free time. Sounds as though that may be at least a short term solution. The problem is annoying - happened this afternoon before I came to work, so I had to switch the machine off - didn't have time to wait and hope it might eventually time out and shutdown. I hate doing that.

I did look at running a script on shutdown to check for the NFS shares and forcibly umount them if there, but I can't get my head round how systemd shuts down, in what calls what - esp. the systemd-poweroff.service uses force:

ExecStart=/usr/bin/systemctl --force poweroff

Something tells me that if I could get a handle on that, I could possibly configure it to handle the missing NFS mounts.

I tried adding umount commands to /etc/rc.local.shutdown, but it didn't work very well - if the NFS mounts weren't up, I would see the umount commands go by, but if they were, the system would just lock up. One of the things I wondered about was NetworkManager - I installed it when I installed Arch, but it seems that when I exit X, it drops the network. I'm currently running Wicd (installed in an attempt to get around another problem) which leaves the network running when X terminates. Unfortunately, that hasn't fixed the problem either - still hangs if the mount(s) are up when I attempt to shut the machine down.

smile I've been very suspicious of systemd as well - Arch is the only distro I've currently got installed that uses it, and its the only distro that has this problem. I've been reading through as much systemd documentation as I can find, but so far, I've not found anything that helps much.

I've got a mate who works in a Unix shop and they can similar issues with NFS mounts, but that "soft" mounting should fix it, though they tend not to. I realise NFS on Linux is different to NFS on Unix, but anyone know why "soft" mounts don't work here?

I tried 'soft' mounts as well, but also didn't help my problem. I'd love a proper solution to it...

Paul.

Offline

#12 2013-08-23 01:53:37

toothandnail
Member
From: Oxfordshire, UK
Registered: 2012-05-05
Posts: 88

Re: Shutdown problem unmounting NFS shares

Markus.N2 wrote:

Is there any progress on this topic ?

I'm certainly still having the same problem. Fine if I remember to umount the NFS mounts before I shut down, but a pretty much certain hang if I forget (which happens often enough to be annoying).

I am new to Arch Linux and I stumbled on this issue, too. My installation is based on the 2013-08 snapshot CD. I have seven NFS mounts. When I umount one of them, the system shuts down normally. But when all of them are mounted, shutdown takes hours. In fact, it does not hang completely. When I issue the shutdown command and go to bed, it is shut down next morning. On the console, it enumerates my NFS mounts and complains about stale NFS handles for each of them. And there is more than one loop.

I've never waited that long (usually when it happens, I'm on my way out and need the laptop with me, so can't wait too long).

BTW... it also fails in automounting the NFS shares at boot (options=soft,bg) and mount -a takes about 20-25 seconds.
Umount all NFS shares and mount -a again is fast.

I've not tried automounting NFS shares so I can't comment on that. Looking at some of the other responses, I will probably try a predisconnect script for Wicd and see if that makes any difference. If it does, I may also try a postconnect script to automount the NFS mounts when the ESID is the one for my home network. Hopefully in the next few days, so I'll post back here if it helps.

Paul.

Offline

#13 2013-08-23 08:19:18

Markus.N2
Member
From: Germany
Registered: 2013-08-22
Posts: 99

Re: Shutdown problem unmounting NFS shares

Some additional info:
I am neither using networkmanager nor wicd, because it is not a laptop. I'm using only dhcpcd.

Offline

#14 2013-08-26 04:32:46

toothandnail
Member
From: Oxfordshire, UK
Registered: 2012-05-05
Posts: 88

Re: Shutdown problem unmounting NFS shares

I just tired adding a wicd pre-disconnect script. Not real complicated, checks whether the mounts are up and (should) umount them if they are. Seemed to work the first time, but when I tried again, left me with a hard lock up when I tried to shut down.

Really don't understand the problem. If I manually umount the shares before shudown, no problem. The script seems to be running, and it is running before wicd drops the network connection, but I'm still getting hard lock ups.

sad What is really annoying is the fact that none of the other distros I've got on my laptop have this problem - only Arch.... Naturally, Arch is also the distro that I use 90+% of the time.

Paul.
.

Offline

#15 2013-10-10 21:10:44

bagheera
Member
Registered: 2009-06-13
Posts: 84

Re: Shutdown problem unmounting NFS shares

Same issue here. Any luck resolving it?


arch linux user

Offline

#16 2013-10-11 14:45:17

kzoli429
Member
Registered: 2013-07-20
Posts: 3

Re: Shutdown problem unmounting NFS shares

I also had the same trouble with 'Idontwanttodisconnect' nfs mounts at shutdown/reboot.
I have created a service to kill nfs services before systemd shuts down the system (see Arch Wiki on sshfs).
Just try the following:

# nano /etc/systemd/system/killnfs.service

and insert the lines:

[Unit]
After=network.target

[Service]
RemainAfterExit=yes
ExecStart=-/bin/true
ExecStop=-/usr/bin/pkill rpc-mountd
ExecStop=-/usr/bin/pkill rpc-statd

[Install]
WantedBy=multi-user.target

Then enable the service by:

# systemctl enable killnfs.service

and you are done (hopefully).

Offline

#17 2013-10-16 20:44:05

Padfoot
Member
Registered: 2010-09-03
Posts: 381

Re: Shutdown problem unmounting NFS shares

I have had similar issues with cifs shares, system would hang if the nas has spun down the drives when I shut down. Over time, different solutions worked, but with further updates they stopped working.

I ran custom scripts with netctl and other points in my boot/shutdown to mount/unmount the shares and all at some point resulted in a hang on shutdown.

The solution I found consistently works (regardless of the shares being switched on or not and regardless of the disks being spun down to an idle state) is using systemd's automount.

Add the mounts to your /etc/fstab as follows (of course, modify the options and mounts to your system):

//LENOVO/Backups /mnt/lenovo/backups/ cifs noauto,nounix,sec=ntlm,file_mode=0777,dir_mode=0777,credentials=/etc/conf.d/nas.cred,x-systemd.automount 0 0

The key options are the noauto and x-systemd.automount systemd will mount the shares only if they exist and are accessed. Upon shutdown, systemd seems to have no issue unmounting regardless of what happened to the share during your session (share may have been turned off, disks spun down due to inactivity etc).

Hope this helps.

Cheers.

Offline

#18 2015-09-30 15:18:18

cuh7b5
Member
Registered: 2011-09-19
Posts: 3

Re: Shutdown problem unmounting NFS shares

@Padfoot Thanks for the tip, x-systemd.automount works great for network mounts!


Thinkpad Edge 11

Offline

#19 2015-09-30 15:53:23

WorMzy
Forum Moderator
From: Scotland
Registered: 2010-06-16
Posts: 11,787
Website

Re: Shutdown problem unmounting NFS shares

Please don't necrobump, cuh7b5.

Closing.


Sakura:-
Mobo: MSI MAG X570S TORPEDO MAX // Processor: AMD Ryzen 9 5950X @4.9GHz // GFX: AMD Radeon RX 5700 XT // RAM: 32GB (4x 8GB) Corsair DDR4 (@ 3000MHz) // Storage: 1x 3TB HDD, 6x 1TB SSD, 2x 120GB SSD, 1x 275GB M2 SSD

Making lemonade from lemons since 2015.

Offline

Board footer

Powered by FluxBB