You are not logged in.

#1 2013-06-16 19:58:11

ANOKNUSA
Member
Registered: 2010-10-22
Posts: 2,141

Files suddenly empty?

Howdy-ha, folks.  I want to apologize in advance if there's some mundane reason for this, but the problem seens so bizarre that I don't really know how to begin troubleshooting it.  Twice now over the last few weeks, I've opened my music collection to discover the same specific directory populated by empty files.  That is, the files all exist as "*.mp3," but contain zero bytes of information, e.g.:

$ ls -l aud/music/classic.hardcore/DI/1985\ -\ Ancient\ Artifacts
total 0
-rw------- 1 anoknusa users 0 Nov  4  2012 01 - O.C. Life.mp3
-rw------- 1 anoknusa users 0 Nov  4  2012 02 - Purgatory II.mp3
-rw------- 1 anoknusa users 0 Nov  4  2012 03 - Stand Up.mp3
-rw------- 1 anoknusa users 0 Nov  4  2012 04 - Eringzo.mp3
-rw------- 1 anoknusa users 0 Nov  4  2012 05 - (I Hate) Surfin' in H.B..mp3
-rw------- 1 anoknusa users 0 Nov  4  2012 06 - Falling Out.mp3
-rw------- 1 anoknusa users 0 Nov  4  2012 07 - Hang Ten in East Berlin.mp3
-rw------- 1 anoknusa users 0 Nov  4  2012 08 - Wounds From Within.mp3
-rw------- 1 anoknusa users 0 Nov  4  2012 09 - Spiritual Law.mp3
-rw------- 1 anoknusa users 0 Nov  4  2012 10 - Imminent War.mp3
-rw------- 1 anoknusa users 0 Nov  4  2012 11 - Pervert Nurse.mp3
-rw------- 1 anoknusa users 0 Nov  4  2012 12 - Guns.mp3
-rw------- 1 anoknusa users 0 Nov  4  2012 13 - Reagan's Der Fuhrer.mp3
-rw------- 1 anoknusa users 0 Nov  4  2012 14 - Little Land.mp3
-rw------- 1 anoknusa users 0 Nov  4  2012 15 - Kids of the Black Hole.mp3
-rw------- 1 anoknusa users 0 Nov  4  2012 Cover.jpg

That's one folder containing one album; the affected directory contains about fifty sub-folders. There hasn't been any crash, there's no error message I'm aware of--in fact, the only reason I became aware of this happening was that beets was complaining about "unreadable files."  I ran a S.M.A.R.T. test on the disk, and everything checks out; the filesystem in this case is brand new, and isn't corrupted.  I'm not sure what I'd look for in the journals or anything.  Can someone point me in the right direction, so I can at least try and determine where to start?  The files are all backed up, so testing different circumstances that might be the cause won't involve any real risk.

Offline

#2 2013-06-16 20:03:30

karol
Archivist
Registered: 2009-05-06
Posts: 25,440

Re: Files suddenly empty?

What filesystem is it?

Offline

#3 2013-06-16 20:04:35

WonderWoofy
Member
From: Los Gatos, CA
Registered: 2012-05-19
Posts: 8,414

Re: Files suddenly empty?

You say the filesystem is brand new, so I assume this means that when this filesystem was created the data was transferred from the mentioned backups.  During that time, have you actually played/accessed these files with success, or are they files that have been copied but dormant, and you were just suddenly thworn this error?  In other words, ahve you actually ever verified that these particular copies ever contained any data?

Offline

#4 2013-06-16 20:17:58

ANOKNUSA
Member
Registered: 2010-10-22
Posts: 2,141

Re: Files suddenly empty?

@karol: BTRFS.  While I know it's not exactly "stable," if BTRFS is the cause of this, it's the only problem I've ever had with it.

@WonderWoofy:  To clarify, this first happend a couple months back, after which I restored from a viable back-up and checked and re-checked several of the sub-dirs.  I was listening to a couple of the albums in question a little over a week ago.  The back-ups are on DVDs made over  a year ago, so they're (virtually) non-volatile.  The filesystem is new because my current installation is new, on account of a super-awesome manufacturing defect I ran into a week or so past.

Offline

#5 2013-06-16 21:14:59

karol
Archivist
Registered: 2009-05-06
Posts: 25,440

Re: Files suddenly empty?

ANOKNUSA wrote:

I was listening to a couple of the albums in question a little over a week ago.

Was this after you've created this filesystem i.e. on this installation?

Offline

#6 2013-06-16 21:30:11

ANOKNUSA
Member
Registered: 2010-10-22
Posts: 2,141

Re: Files suddenly empty?

Yes and no.  I'm not sure how helpful this info will be, but I didn't specifically create the filesystem in question for a new installation; the new installation was forced on me due to hardware failure.  I was in the process of encrypting my laptop's disks, and the hardware failure occured during that transition.  However, the first time this happend was quite some time ago, before I even considered undertaking the encryption process, so the two issues probably wouldn't be related.

Offline

#7 2013-06-16 22:45:48

karol
Archivist
Registered: 2009-05-06
Posts: 25,440

Re: Files suddenly empty?

I found only a comment https://www.linux.com/learn/docs/667605 … lesystems/

K. Darien Freeheart wrote:

There was an early bug where files would accidentlaly get replaced with empty files.

but I have no idea why would this bug reappear and whether he was talking about power loss during renaming the files or what.

btrfs has some tools that should throw errors / warnings if something's fishy is going on, right?


The permissions look kind of weird too, but I'm not a guru on this matter.

Offline

#8 2013-06-16 23:41:30

WonderWoofy
Member
From: Los Gatos, CA
Registered: 2012-05-19
Posts: 8,414

Re: Files suddenly empty?

If you use btrfs, I take it you do not use raid1 at all.  Have you done a scrub to see if btrfs recognizes these files as failing the checksum?

Offline

#9 2013-06-17 04:25:33

ANOKNUSA
Member
Registered: 2010-10-22
Posts: 2,141

Re: Files suddenly empty?

Thanks karol; I'll dig around for that bug report.  The permissions seemed different to me at first too---they don't match the rest of my user directory---but they do match all other files throughout my music library, which hasn't had any problems.

@WonderWoofy: No, I haven't; I actually just started thinking about a multi-disk setup shortly before this started (using BTRFS in earnest was the next step after encrypting the drives, as I have multiple BTRFS partitions  across two disks I wanted to combine)*, so "scrub" is still a totally new concept to me.  I'll look into that as well as soon as I get a chance.

* There are three partitions across two disks, two being encrypted with LUKS/dm-crypt, so I wasn't sure if I could use "btrfs device add" on them; that's a subject for a different thread, though.

Offline

#10 2013-06-17 04:51:31

WonderWoofy
Member
From: Los Gatos, CA
Registered: 2012-05-19
Posts: 8,414

Re: Files suddenly empty?

You can have it "scrub" the disk which bascailly just checks the data on disk against its metadata checksums.  To do this, all you need to do is "btrfs scrub start <filesystem>"  Then you can see how it is doing with btrfs scrub status.  I think it will print you a report when it finishes.

Offline

#11 2013-06-17 14:01:35

ANOKNUSA
Member
Registered: 2010-10-22
Posts: 2,141

Re: Files suddenly empty?

Thanks, WonderWoofy; running the scrub now.

Offline

#12 2013-06-17 15:10:55

WonderWoofy
Member
From: Los Gatos, CA
Registered: 2012-05-19
Posts: 8,414

Re: Files suddenly empty?

A bit OT, in regard to the empty files, but does relate to the filesystem in use:

This is also a really good reason to have your metatdata in a raid1.  Ideally you would keep your whole filesystem in a raid1, which will enable self-healing functionality when  you do these scrubs.  That is when it does the checksums, if there are inconsistencies anywhere it can determine if either is actually the right data and filx the offending corruption.  But if you keep your data in "single" mode, it can at least tell you if there are inconsistencies, but it can't really do much about it.  So having two copies of the metadata is typically a good thing in that it can determine if its metadata is valid as well and heal that if necessary.

You should know that raid1 (and the other raid implementations in btrfs) are not quite like what you think they are.  For example, if you hvae two disks in a raid1, it will obviously be mirroring across both drives.  But if you hvae more than two drives in a raid1, it will still only keep two copies if every extent, whereas with mdadm you would have an equivalent number of copies to the number of disks in the array.  I guess this is done because it helps it handle multiple different sized disks without space loss (or it minimizes space loss).

I just recently decided that if I was going to use btrfs, I wanted to actually learn about the things it can do.  I was already taking advantage of the subvolumes (and snapshots to a lesser extent), but there are so many things that this filesystem can do, it is rather incredible, and I started subscribing to the btrfs mailing list and more and more features are added all the time.

Offline

#13 2013-06-17 15:38:12

ANOKNUSA
Member
Registered: 2010-10-22
Posts: 2,141

Re: Files suddenly empty?

Yeah, I should probably take the time to look through the mailing list myself; until recently, I didn't really think about treating all three partitions as a single filesystem, and much of the information I've found on why it would be a good idea to do so seems to come from professionals with use cases far more advanced than my own (and seems to assume I already know about mdadm and can figure out for myself why BTRFS's mirroring is superior).  Near as I can tell, single-disk use of BTRFS is discouraged because of the present lack of redundancy and the limitations of btrfsck.

I've been using BTRFS for some time, but until recently just used it for simple reasons (/boot as a subvolume to roll back kernel updates, SSD optimization, compression).  Most of the more advanced stuff is still beyond me.

Offline

#14 2013-06-17 16:01:52

karol
Archivist
Registered: 2009-05-06
Posts: 25,440

Re: Files suddenly empty?

WonderWoofy wrote:

I just recently decided that if I was going to use btrfs, I wanted to actually learn about the things it can do.  I was already taking advantage of the subvolumes (and snapshots to a lesser extent), but there are so many things that this filesystem can do, it is rather incredible, and I started subscribing to the btrfs mailing list and more and more features are added all the time.

Are you saying that removing data from files is a btrfs feature? A radical space-optimization algorithm decides which files are non-essential and kinda gets rid of them. You should have backups anyway, so you can get these files back if needed ;P


@ANOKNUSA
You can add something like

find /home/username/media -type f -empty >> ~/.logs/empty.log

(with some excludes for false positives) to your daily cron to try and catch these empty files ASAP.

Offline

Board footer

Powered by FluxBB