Trying to extract .tar, exits before everything is extracted

raggerv8 · 2011-01-13 12:45:17

Hi.

A couple of days ago i backed up my home directory cause i reinstalled arch to get a separate /home partition. I made a tarball of my home dir and put it on a external hard drive (NTFS).
The tarball is 316GB big, and when I'm trying to extract it, it exits when it have extracted 309GB and i get the error message

tar: Exiting with failure status due to previous errors

My theory is that it has to do with permission issues. I have mounted the hard drive with pmount as a normal user.

Have you guys (and gals) here any idea what to do? I am available at Archlinux IRC @ Freenode under the nick name Rebellen.

linux-ka · 2011-01-13 12:48:46

Have your tried to untar the ball via live cd? anouther suggestion, could you reach a maximum filesize? maybe one file of 316Gb is too large to extract.

Btw, why have to tared so many data? a copy of the folder would have been much easier....and faster....

raggerv8 · 2011-01-13 12:51:50

linux-ka wrote:

Have your tried to untar the ball via live cd? anouther suggestion, could you reach a maximum filesize? maybe one file of 316Gb is too large to extract.
Btw, why have to tared so many data? a copy of the folder would have been much easier....and faster....

I haven't tried to untar via LiveCD, why would that make a difference? I don't know if it's too big.

I asked in #archlinux and they/someone told me to make a tarball of my home. Therefor i made a tarball. Never gonna do it again.

Last edited by raggerv8 (2011-01-13 12:52:01)

xiwenc · 2011-01-13 13:10:48

LiveCD won't make a difference. NTFS maximum filesize is 16TB[1] and tar's theoretical max is 2^63-1 bytes [2]. Much greater than NTFS.

You could try:
1. untar as root, with: tar xvfp /my/tar ; post the last ~5 lines when it fails
2. If there are corrupted files in the archive, exclude them from unpacking
3. copy the tar archive over to an ext2/3/4 mount point then verify the duplication was successful by say compare checksums; this to rule out the fact the complete archive could be read successfully. If this fails, probably a bug in ntfs-3g.

ref:
1. http://en.wikipedia.org/wiki/NTFS
2. http://answers.google.com/answers/threadview?id=25116

Last edited by xiwenc (2011-01-13 13:21:07)

raggerv8 · 2011-01-13 13:21:25

xiwenc wrote:

LiveCD won't make a difference.
You could try:
1. untar as root, with: tar xvfp /my/tar ; post the last ~5 lines when it fails
2. If there are corrupted files in the archive, exclude them from unpacking
3. copy the tar archive over to an ext2/3/4 mount point then verify the duplication was successful by say compare checksums; this to rule out the fact the complete archive could be read successfully. If this fails, probably a bug in ntfs-3g.

1. I'm doing it right now, thx!
2. How do i exclude them?

xiwenc · 2011-01-13 13:26:32

raggerv8 wrote:

2. How do i exclude them?

man tar

raggerv8 · 2011-01-13 13:30:03

xiwenc wrote:

raggerv8 wrote:
2. How do i exclude them?
man tar

I know there are some --exclude* commands but i don't really know which one I'm suppose to use and how the path should look like.

Now when I'm extracting the path is like this "home/me/folder/file"
Should i then run this?

tar xvfp /media/usb/me.tar --exclude=home/me/folder

raggerv8 · 2011-01-13 13:30:44

Sorry, double post.

Last edited by raggerv8 (2011-01-13 13:38:00)

xiwenc · 2011-01-13 13:35:18

raggerv8 wrote:

I know there are some --exclude* commands but i don't really know which one I'm suppose to use and how the path should look like.
Now when I'm extracting the path is like this "home/me/folder/file"
Should i then run this?
tar xvfp /media/usb/me.tar --exclude=home/me/folder

yes, or do -X /tmp/exlcudethese
with /tmp/excludethese as contents:

home/me/folder
home/me/afile
etc...

i8degrees · 2011-01-13 13:46:55

Hi, there!

raggerv8 wrote:

I asked in #archlinux and they/someone told me to make a tarball of my home. Therefor i made a tarball. Never gonna do it again.

First of all, the advice given in regards to tarring your home partition for backup was not bad advice per se, as this is both what I personally do and would also recommend to others to do and is not especially prone to error above or below that of any other standard backup operation. Whereas there are plenty of valid & great reasons as to why one would choose such a format, this I feel is beyond the "scope" of the topic. I simply wish to clear up any unfairly biased fear that might have been created from this sure to be stressful situation on your hands. Honestly, I do not know *anything* in regards to you system state leading up to nor during the archival backup, thus who knows -- I sure will not.

raggerv8 wrote:

A couple of days ago i backed up my home directory cause i reinstalled arch to get a separate /home partition.

OK. this is normal thus far from what little I know.

raggerv8 wrote:

I made a tarball of my home dir and put it on a external hard drive (NTFS).

What was the precise syntax of the archival operation of which created this tar ball for you? Even better is if we were to know how precisely it were moved over and also any reason to believe that power outage could have complicated these matters for it is within the realm of possibilities, not knowing anything more.

raggerv8 wrote:

The tarball is 316GB big, and when I'm trying to extract it,

By chance, do you know if this file size verifies back to the original contents of /home accurately, within a few megabytes or so, give or take? This might be a very important part of the equation to understand, for your sake above all else, but also beneficial for potential diagnostics from us.

raggerv8 wrote:

it exits when it have extracted 309GB and i get the error message
tar: Exiting with failure status due to previous errors

Ah, yes, these sorts of vague, cryptic messages are less than ideal, aye? :-] Unfortunately, this debug does relatively little from a diagnostics standpoint to help us in part due to the common occurring / usage of said debug message. We know there's a problem that has a rough variable number of solutions that is beyond that of what either one of us want to know.

raggerv8 wrote:

My theory is that it has to do with permission issues. I have mounted the hard drive with pmount as a normal user.
Have you guys (and gals) here any idea what to do? I am available at Archlinux IRC @ Freenode under the nick name Rebellen.

AFAIK, assuming typical filesystem -- ext2,ext3,ext4,reiserfs and other well known & supported systems -- in use that are setup with respect paid to user permissions, which ought to be already set right upon initial installation.... I have no reason to be led to believe that it is permissions related. I don't know. I do have short list of questions in addition to above to ask that may help me or another one better help you assess and determine the situation as follows below:

WARNING: you would be wise to replicate this tarball before trying anything else on it, *regardless* of where the suggestions are coming from and why.

1. I simply assume that you are extracting this tarball to the destination filesystem on the new home partition of yours? (I'm sorry if this is obvious but it must be said; wouldn't be my first!)

2. Inspect the integrity of the tarball by passing the --test-label parameter from the 'tar' software onto your tarball package? I am really hoping this tests more than just simply an archive label, honestly, but of course maybe the definition of archive label escapes me, too? =/ Either way, 'tis worth a shot.

3. http://www.urbanophile.com/arenn/coding/gzrt/gzrt.html | this piece of software *could* prove promising in recovery if the methods in which said recovery software uses / looks for is indeed what manifested onto your tarball and created this problem of yours.

4. Lastly, and this is all the more of a shot in the dark: the piece of software called 'star' has been made reference to once or twice as per my own personal research in potentially aiding one in the recovery process, if this is what the situation boils down to.

raggerv8 · 2011-01-13 15:04:59

i8degrees wrote:

Hi, there!
raggerv8 wrote:
I asked in #archlinux and they/someone told me to make a tarball of my home. Therefor i made a tarball. Never gonna do it again.
First of all, the advice given in regards to tarring your home partition for backup was not bad advice per se, as this is both what I personally do and would also recommend to others to do and is not especially prone to error above or below that of any other standard backup operation. Whereas there are plenty of valid & great reasons as to why one would choose such a format, this I feel is beyond the "scope" of the topic. I simply wish to clear up any unfairly biased fear that might have been created from this sure to be stressful situation on your hands. Honestly, I do not know *anything* in regards to you system state leading up to nor during the archival backup, thus who knows -- I sure will not.

raggerv8 wrote:
A couple of days ago i backed up my home directory cause i reinstalled arch to get a separate /home partition.
OK. this is normal thus far from what little I know.
raggerv8 wrote:
I made a tarball of my home dir and put it on a external hard drive (NTFS).
What was the precise syntax of the archival operation of which created this tar ball for you? Even better is if we were to know how precisely it were moved over and also any reason to believe that power outage could have complicated these matters for it is within the realm of possibilities, not knowing anything more.
raggerv8 wrote:
The tarball is 316GB big, and when I'm trying to extract it,
By chance, do you know if this file size verifies back to the original contents of /home accurately, within a few megabytes or so, give or take? This might be a very important part of the equation to understand, for your sake above all else, but also beneficial for potential diagnostics from us.
raggerv8 wrote:
it exits when it have extracted 309GB and i get the error message
tar: Exiting with failure status due to previous errors
Ah, yes, these sorts of vague, cryptic messages are less than ideal, aye? :-] Unfortunately, this debug does relatively little from a diagnostics standpoint to help us in part due to the common occurring / usage of said debug message. We know there's a problem that has a rough variable number of solutions that is beyond that of what either one of us want to know.
raggerv8 wrote:
My theory is that it has to do with permission issues. I have mounted the hard drive with pmount as a normal user.
Have you guys (and gals) here any idea what to do? I am available at Archlinux IRC @ Freenode under the nick name Rebellen.
AFAIK, assuming typical filesystem -- ext2,ext3,ext4,reiserfs and other well known & supported systems -- in use that are setup with respect paid to user permissions, which ought to be already set right upon initial installation.... I have no reason to be led to believe that it is permissions related. I don't know. I do have short list of questions in addition to above to ask that may help me or another one better help you assess and determine the situation as follows below:
WARNING: you would be wise to replicate this tarball before trying anything else on it, *regardless* of where the suggestions are coming from and why.
1. I simply assume that you are extracting this tarball to the destination filesystem on the new home partition of yours? (I'm sorry if this is obvious but it must be said; wouldn't be my first!)
2. Inspect the integrity of the tarball by passing the --test-label parameter from the 'tar' software onto your tarball package? I am really hoping this tests more than just simply an archive label, honestly, but of course maybe the definition of archive label escapes me, too? =/ Either way, 'tis worth a shot.
3. http://www.urbanophile.com/arenn/coding/gzrt/gzrt.html | this piece of software *could* prove promising in recovery if the methods in which said recovery software uses / looks for is indeed what manifested onto your tarball and created this problem of yours.
4. Lastly, and this is all the more of a shot in the dark: the piece of software called 'star' has been made reference to once or twice as per my own personal research in potentially aiding one in the recovery process, if this is what the situation boils down to.

I don't really know what the exact syntax were when i made the tarball. I dont know the options but it was something like: tar [options] /home/me /media/usb/me.tar

I ran a du -hs /home/me and the same to the tarball when i made the tarball and both were 316GB. If it weren't, why is the tarball alot bigger than the extracted content?

1. No. Im just extracting it to the same dir and drive.
2. I will try that later.

raggerv8 · 2011-01-13 22:01:04

xiwenc wrote:

LiveCD won't make a difference. NTFS maximum filesize is 16TB[1] and tar's theoretical max is 2^63-1 bytes [2]. Much greater than NTFS.
You could try:
1. untar as root, with: tar xvfp /my/tar ; post the last ~5 lines when it fails
2. If there are corrupted files in the archive, exclude them from unpacking
3. copy the tar archive over to an ext2/3/4 mount point then verify the duplication was successful by say compare checksums; this to rule out the fact the complete archive could be read successfully. If this fails, probably a bug in ntfs-3g.
ref:
1. http://en.wikipedia.org/wiki/NTFS
2. http://answers.google.com/answers/threadview?id=25116

home/ragger/.rigsofrods/
home/ragger/.rigsofrods/ogre.cfg
home/ragger/.rigsofrods/packs/
home/ragger/.rigsofrods/RoR.log
home/ragger/.rigsofrods/RoR.cfg
tar: Exiting with failure status due to previous errors
[root@fetta usb]#

It always exits on the same file. But i dont know which file is after, i guess i can exclude the .rigsofrods/ directory if it is a file in that directory.

Last edited by raggerv8 (2011-01-13 22:01:21)

raggerv8 · 2011-01-14 11:59:30

I excluded the folder it exited at but it still exits after the folder/files before those i excluded. So i ran tar -tvf and listed all the files and I'm gonna see which files or folder is after, so i can exclude them.

edit: or else i guess i have to settle with 309GB of 316GB

edit2: Okey guys i listed the file and home/ragger/.rigsofrods/RoR.cfg seems to be the latest file, can it be right that the tarball is 316GB and the extracted content is 309GB?

Last edited by raggerv8 (2011-01-14 12:07:20)

thisoldman · 2011-01-14 12:23:53

tar adds overhead for things such as indexing:

 $ touch bvbvbv
$ tar -cvf bv.tar bvbvbv
bvbvbv

 $ tar -tvf bv.tar 
-rw-r--r-- casey/users       0 2011-01-14 07:13 bvbvbv

$ ls -l bv*
-rw-r--r-- 1 casey users     0 Jan 14 07:13 bvbvbv
-rw-r--r-- 1 casey users 10240 Jan 14 07:13 bv.tar

How else would it be able to perform operations on single files within the archive? Remember, 'tar' was designed for tape archives.

Last edited by thisoldman (2011-01-14 12:30:51)

Arch Linux

#1 2011-01-13 12:45:17

Trying to extract .tar, exits before everything is extracted

#2 2011-01-13 12:48:46

Re: Trying to extract .tar, exits before everything is extracted

#3 2011-01-13 12:51:50

Re: Trying to extract .tar, exits before everything is extracted

#4 2011-01-13 13:10:48

Re: Trying to extract .tar, exits before everything is extracted

#5 2011-01-13 13:21:25

Re: Trying to extract .tar, exits before everything is extracted

#6 2011-01-13 13:26:32

Re: Trying to extract .tar, exits before everything is extracted

#7 2011-01-13 13:30:03

Re: Trying to extract .tar, exits before everything is extracted

#8 2011-01-13 13:30:44

Re: Trying to extract .tar, exits before everything is extracted

#9 2011-01-13 13:35:18

Re: Trying to extract .tar, exits before everything is extracted

#10 2011-01-13 13:46:55

Re: Trying to extract .tar, exits before everything is extracted

#11 2011-01-13 15:04:59

Re: Trying to extract .tar, exits before everything is extracted

#12 2011-01-13 22:01:04

Re: Trying to extract .tar, exits before everything is extracted

#13 2011-01-14 11:59:30

Re: Trying to extract .tar, exits before everything is extracted

#14 2011-01-14 12:23:53

Re: Trying to extract .tar, exits before everything is extracted

Board footer