sshfs disconnects without unmounting, keeps making everything freeze

whoops · 2010-04-07 12:33:36

Hi!

Every time sshfs stops working (guess it times out after a while?), every application that tries to access the folder freezes. After I "killall sshfs", they unfreeze, but the folder stays inaccessible until I manually umount them as root (which sometimes doesn't work).

while it's still "mounted", ls says:

ls: cannot access sshdmount: Transport endpoint is not connected

"lsof | grep ssh" just says…

lsof: WARNING: can't stat() fuse.sshfs file system /home/user/folder
      Output information may be incomplete.
lsof: WARNING: can't stat() fuse.sshfs file system /home/user/sshdmount
      Output information may be incomplete.

… and doesn't finish.

Is that supposed to be like that?
- If yes, how do you handle it? Just do a script that kills sshfs and unmounts?
- If no: Then… why? How do I get rid of this behaviour?

Thanks!

Last edited by whoops (2010-09-15 13:39:48)

whoops · 2010-09-20 11:49:54

Hi!

Still the same problem. Sometimes the whole system freezes after resume from pm-hibernate. Am I supposed to somehow unmount all mounted "network folders" in the hibernation scripts or something?

Straaange, I for some reason feel I'm missing the point here. Where is it?

Last edited by whoops (2010-09-20 11:50:16)

R00KIE · 2010-09-20 12:46:03

So you hibernate, killing the ssh connection by timeout on the remote machine side and then expect things to work after turning your pc on again?

As you say you need to unmount all sshfs mounts and remount them when you resume from hibernation.

whoops · 2010-09-20 13:54:45

R00KIE wrote:

So you hibernate, killing the ssh connection by timeout on the remote machine side and then expect things to work after turning your pc on again?

No, but I do expect the machine not to freeze. Or lock. It just goes black screen, tty not working, no intervention possible.

From my limited perspective, it would be nice if sshfs or something just unmounted the folders when it lost connection (no matter if due to hibernating or something else) instead of going into some sort of endless loop or whatever is blocking the processes that try to access those folders... but being able to kill sshfs manually should be the minimum, shouldn't it? I don't even mount as root...

So, is all of this behaviour normal / intended / is this an ask-upstream-case?

Last edited by whoops (2010-09-20 13:56:33)

R00KIE · 2010-09-21 11:12:18

I guess this may be a case of ask upstream but I suspect that it is just how it works (or you need to specify more mount options).

Maybe using -o reconnect will help, I have experienced what you mention but usually I don't use extra options (I don't use sshfs that much).

whoops · 2011-03-24 08:14:09

Thanks!

Took me quite a while ("loosing terminals" or even exservers somehow made me afraid of experimenting much - I hate having to reboot), but:

-o reconnect -C -o workaround=all

Seems to work better. Didn't try if some of those options can be left out.

Still I think this looks a lot like a baaad bug (the last time I had it running and lost wifi for 0.5 seconds ,3 Desktops and many terminals full of stuff were frozen and I accidentally hit <tab>completion while I was "near" the folder and "lost" my last tty that was still working -.-"). And I have no Idea, where this should be addressed. I don't think, sshfs itsself would be the right place... should it really even be able to do something like that? Should "the kernel" or whatever there's in between "let it"?

Well - I lack whatever basics / knowledge are needed to "complain" about this in the right place, so I'm out at this point... hope someone who knows more is annoyed enough to pursue further or something.

Cdh · 2011-03-24 09:28:58

IIRC the processes trying to access disconnected sshfs shares go into uninterruptible sleep. That is maybe correct behaviour, but it really isn't good for usability. For example when I have a sshfs mount at ~/mnt which gets disconnected I can't use any tabcompletion to umount it and starting filemanagers that try to read the whole ~ is extremely slow.

I think the behaviour in sshfs should be changed that it returns something when the share is not available at the moment. Maybe throw something like "transport endpoint not connected". So that you simply would try again if the share is in fact connected but only too slow.

whoops · 2011-03-24 10:54:14

I just think, stuff shouldn't freeze - everything beyond that point is beyond me. Theoretically I can mount something in my home directory as user with sshfs and then disconnect the server to "cripple" root (a little). Yes, easy way around that for root if root knows what "he" is doing, but still - doesn't sound very linux-ish to me. And the problems I had when resuming from hibernate sometimes were beyond recovery (unless you count "reboot" as "recovery") even by "root intervention".

If someone would ask me, where I'd put anti-freeze, I'd say: "Kernel". Then again - guess that's one of those things that sound moronic for someone who knows how stuff works. I'd put everything in the kernel. Always .

rwd · 2011-03-27 19:50:49

To keep ssh sessions alive even with the client suspended you could increase 'ClientAliveInterval' in /etc/ssh/sshd_config on the remote side, for example to 600 minutes . Because now sessions stay open on the server you need to prevent ending up with dozens of sessions, so put this in ~/.ssh/config on the client to reuse sessions instead of creating new ones:

# prevent connection from hanging   
    ServerAliveInterval 15
# set connection sharing
    ControlMaster auto
    ControlPath ~/.ssh/master-%r@%h:%p

Eventually sessions will die and so this is a way to make sure sshfs unmounts and mounts when suspending:
( For mounts done as non-root user you would need to change the line with starting with fusermount to something like: su -c 'fusermount -u ..' otherusername)

/etc/pm/sleep.d/00_sshfs_automount.sh:

                #!/bin/bash
                # unmount all sshfs mountpoints before suspending/hibernating
                # to prevent hanging sshfs after resume/thaw
                case $1 in
                hibernate | suspend)
                    mountpoint_count=$(egrep -c '(/[^ ]+) fuse.sshfs' /etc/mtab)
                    if [ $mountpoint_count -gt 0 ]; then
                        mountpoints=($(awk '/(\/[^ ]+) fuse.sshfs/ { print $2 }' /etc/mtab))
                        for element in  $(seq 0 $((${#mountpoints[@]} - 1)))
                        do
                            fusermount -u "${mountpoints[$element]}"
                        done
                    fi
                    ;;
                thaw | resume )
                    mount -a
                    ;;
                #   exit 0
                #   ;;
                esac
            exit 0

Last edited by rwd (2011-03-27 23:05:32)

squiidux · 2013-06-03 13:16:29

fusermount -u /path/to/sshfs/mount/point

novilog · 2017-05-19 20:11:03

I hate to necrobump for my first post, but there is useful information that could yet help future visitors to this thread, since it seems to be a pretty hot issue. As mentioned in a reply by kubanczyk on this serverfault thread

kubanczyk wrote:

Use
-o reconnect,ServerAliveInterval=15,ServerAliveCountMax=3
These ServerAlive options cause the I/O errors to pop out after one minute of network outage. Without these options, processes which experience I/O hang seem to sleep indefinitely, even after the sshfs gets reconnected.

For clarity:

You can just drop that option in-line with your sshfs command.
The ServerAliveInterval option changes how long your system waits for the server to respond.
The ServerAliveCountMax option changes how many times your system will tell the server that it's waiting.

In this example, when your computer is disconnected from the sshfs session improperly, it will wait for a response for 15 seconds, then notify the server that it's waiting and wait for another 15 seconds. This continues for one minute, then it times out as the user might expect, throwing an error and exiting the process.

Last edited by novilog (2017-05-19 20:14:11)

Arch Linux

#1 2010-04-07 12:33:36

sshfs disconnects without unmounting, keeps making everything freeze

#2 2010-09-20 11:49:54

Re: sshfs disconnects without unmounting, keeps making everything freeze

#3 2010-09-20 12:46:03

Re: sshfs disconnects without unmounting, keeps making everything freeze

#4 2010-09-20 13:54:45

Re: sshfs disconnects without unmounting, keeps making everything freeze

#5 2010-09-21 11:12:18

Re: sshfs disconnects without unmounting, keeps making everything freeze

#6 2011-03-24 08:14:09

Re: sshfs disconnects without unmounting, keeps making everything freeze

#7 2011-03-24 09:28:58

Re: sshfs disconnects without unmounting, keeps making everything freeze

#8 2011-03-24 10:54:14

Re: sshfs disconnects without unmounting, keeps making everything freeze

#9 2011-03-27 19:50:49

Re: sshfs disconnects without unmounting, keeps making everything freeze

#10 2013-06-03 13:16:29

Re: sshfs disconnects without unmounting, keeps making everything freeze

#11 2017-05-19 20:11:03

Re: sshfs disconnects without unmounting, keeps making everything freeze

Board footer