[SOLVED] Files disappeared from home directory after boot

greyseal96 · 2014-04-16 23:59:53

Hello,

I'm trying to troubleshoot a really strange problem that I've never seen before. I'm hoping that the symptoms might sound familiar or similar to someone and that somebody might be able to point me in the right direction so that I can figure out the cause. On Sunday afternoon, I did a full update (-Syu). I've got my boot, root and home partitions all in LVM so that before I do updates I can snapshot the partitions. This helps me so that if something goes wrong after the update I can quickly revert to the prior state. Since the update included a kernel update, I rebooted and continued using my computer. There didn't seem to be any problems so I thought nothing of it. This install of Arch is fairly new so I'm still installing things as I need them. This particular afternoon, I set up remote access to my other Linux server so I set up my SSH pub/private key pairs and got SSH working. I then shut down my computer for the day without any problems.

On Monday when I started up my computer, I went in to connect up to my Linux server again but the pub/private SSH key pairs were missing and the known_hosts file was empty (even though it previously had an entry for the Linux server that I had connected to the day before). I thought that it was pretty strange, but I regenerated the SSH key pairs and this time I set up a VNC connection using SSH and saved it in Remmina. I also had to replace the public key file on the destination server that I'm working with. I was only working on it for a little bit but once again, I saved everything and shut down without any problems. Yesterday when I started up my computer, I noticed that the connection that I'd put into Remmina for the Linux server was missing but the SSH key pairs were there. However, after re-adding the connection in Remmina and connecting, I had to re-accept the Linux server into the known_hosts file and the server wouldn't accept my key. I discovered that the original SSH key pairs from Sunday were back again! The weird thing in all of this is what has happened with my Firefox tabs. I frequently have a lot of tabs open in Firefox and when I open up Firefox, I always open the previous session. At no point did I lose any of the tabs that I had opened on Sunday, Monday or Tuesday. So whatever caused the weirdness with the key pairs disappearing and reappearing didn't touch my saved Firefox tabs for some reason.

At this point, I'm pretty baffled as to what is causing this problem. I don't know if the snapshots could be part of the problem, but it's the only thing that I can think of that might cause something to disappear and then reappear. But it mostly makes me really uneasy because I'm wondering what might be next to disappear. I'd like to try to troubleshoot this, but I'm not sure where to start. I've been searching online and in these forums to try to find something similar but I haven't found anything yet. I'm hoping that somebody has some ideas or that somebody might have seen this before. Thanks in advance for any help.

Last edited by greyseal96 (2014-04-23 01:26:32)

ewaller · 2014-04-17 02:17:48

Until I got to the bit about Firefox tabs, I was wondering if your home partition might not have been mounted. I still want to check.
look at the output of mount and ensure that your home partition is mounted on /home. Then, as root, unmount /home and then look in the directory at /home (it was your mount point. Now, it should be empty) Ensure that it is empty. If not, are they your wayward files?

Also, check that there are not multiple things mounted on /home. I don't know if you can actually do that in Arch, but I have seen it happen on my DD-WRT based router.

greyseal96 · 2014-04-17 03:55:30

Thanks for responding, ewaller. I tried what you suggested and everything looked normal; the /home directory was mounted to the correct partition and there was nothing there when I unmounted home. But that got me thinking... what if, somehow, in some weird way, the snapshot had gotten mounted as my home directory? So I mounted the snapshot at /mnt and took a look for the .ssh directory because that should not have been in the snapshot. However, I found the .ssh directory with the pub/private keys inside. Just to be sure, I checked on my server to see if the old public key matched the public key from the snapshot and it did.

So now, I've got two questions, both equally weird. First, how the heck did the snapshot of my home partition get mounted instead of my "real" home partition. My /etc/fstab uses UUIDs to find the partitions, and I was under the impression that the UUID for the snapshot partition would be different than the UUID for the "real" home partition so that this sort of thing wouldn't happen. Second, if the snapshot partition was mounted, how come the Firefox tabs were still there when I opened it up when my "real" home partition was correctly mounted? I would think that that would have been stored in my home directory as well.

I guess a related question would be, if the snapshot somehow got mounted, how likely is it that it could happen again or that other snapshots could get mounted like the root or boot snapshot? That would suck to have to check this every time... Has anyone ever run across something like this?

ewaller · 2014-04-17 04:20:05

I am not an expert in UUID's; not by a long shot.

It has been a while, but I think it is possible to copy a partition in a manner so as to preserve its UUID. I think tools like dd have that affect.
Check the output of sudo blkid and see if you have managed to duplicate a uuid.

If so, this article looks helpful.

greyseal96 · 2014-04-17 14:51:32

Thanks again for your help and response, ewaller. I ran blkid and I also ran lsblk -f to see what the output was. The interesting, and slightly disconcerting, thing that I found was that the snapshots DO have the same UUID as the parent partitions. I guess that this makes sense if they are exact copies, but that seems to me to be something that could potentially cause problems, especially if one's fstab mounts partitions by UUID. I did a little bit of searching online and came across a forum post here: http://forums.opensuse.org/showthread.p … d-old-home (not sure if it's OK to post links to other forums, but the information is very relevant...) in which one poster expresses frustration that (in his or her estimation) about 15% of the time, the snapshots are mounted instead of the parent partitions because the UUIDs are the same. The response is basically that this is to be expected since the snapshots are exact clones and the UUIDs are the same. The suggestion is made to change the UUIDs of the snapshots to avoid this problem. So I guess that this may have happened to me. I still don't understand how the Firefox session stuck around, but I guess I may never know that. The SSH key files in the snapshot are enough to convince me that the snapshot must have been mounted somehow.

To avoid this problem in the future, I decided that I needed to change my fstab. Since I don't really want to have to change my snapshot UUIDs every time, I just changed my fstab to mount by partition name, rather than UUID. I figure that LVM will already take care of finding the correct physical partition so that there will then be no problem finding the logical partition by name. Since I know that the names are different between parent and snapshot partitions, this should avoid the problem. I tested this out and it seems to work. If anybody foresees a problem with this, let me know.

Thanks again for your help, ewaller. You got me going in the right direction.

Arch Linux

#1 2014-04-16 23:59:53

[SOLVED] Files disappeared from home directory after boot

#2 2014-04-17 02:17:48

Re: [SOLVED] Files disappeared from home directory after boot

#3 2014-04-17 03:55:30

Re: [SOLVED] Files disappeared from home directory after boot

#4 2014-04-17 04:20:05

Re: [SOLVED] Files disappeared from home directory after boot

#5 2014-04-17 14:51:32

Re: [SOLVED] Files disappeared from home directory after boot

Board footer