You are not logged in.

#1 2015-12-25 20:09:27

agent0
Member
From: Russia
Registered: 2014-05-04
Posts: 74
Website

[solved] LiveUSB becomes unbootable after resizing filesystem

Hello.
I have a laptop with bios and there is windows installed on it. Under windows I have created a bootable recovery usb flash drive.
After that I have checked that that bootable drive is working normal. Of cource boot order is set to load from usb first.
Windows did not provide an option to create an iso file to be able to create bootable usb when you want. And because I do not know utilities which can do correct iso from live usb I decided to make a DD clone of the entire live usb.

Later I decided to cut free space from bootable partition because there was only 3,7 Gb used. I did this with kde partition manager and also tried with gparted. After resising fat32 partition, live usb becomes unbootable... Not unbootable at all, but I can watch black screen with blinking cursor.

If I clone originaly backed up image to flash drive, it becomes bootable again. But I do not want to store 8 Gb image with remaining half just containing empty space.
How can I determine reason why it becomes unbootable?

Here is log of applying operation of resizing fat 32 partition:

checkig before resizing:
Command: fsck.msdos -a -w -v /dev/sdb1
fsck.fat 3.0.28 (2015-05-16)
Checking we can access the last sector of the filesystem
Boot sector contents:
System ID "MSDOS5.0"
Media byte 0xf8 (hard disk)
       512 bytes per logical sector
      4096 bytes per cluster
      3018 reserved sectors
First FAT starts at byte 1545216 (sector 3018)
         2 FATs, 32 bit entries
   7616000 bytes per FAT (= 14875 sectors)
Root directory start at cluster 2 (arbitrary size)
Data area starts at byte 16777216 (sector 32768)
   1903872 data clusters (7798259712 bytes)
63 sectors/track, 255 heads
      2048 hidden sectors
  15263744 sectors total
Reclaiming unconnected clusters.
Checking free cluster summary.
/dev/sdb1: 59 files, 969072/1903872 clusters

resizing:
Resizing a fat32 file system using internal backend functions.
Successfully resized file system using internal backend functions.

some task for changing geometry (maybe this is cause?):
Changing geometry of partition «/dev/sdb1»: First sector: 2048, length: 14383104 : Done

checking fs after resizing:
Command: fsck.msdos -a -w -v /dev/sdb1
fsck.fat 3.0.28 (2015-05-16)
Checking we can access the last sector of the filesystem
Boot sector contents:
System ID "MSWIN4.1"
Media byte 0xf8 (hard disk)
       512 bytes per logical sector
      4096 bytes per cluster
        46 reserved sectors
First FAT starts at byte 23552 (sector 46)
         2 FATs, 32 bit entries
   7184896 bytes per FAT (= 14033 sectors)
Root directory start at cluster 969656 (arbitrary size)
Data area starts at byte 14393344 (sector 28112)
   1794374 data clusters (7349755904 bytes)
63 sectors/track, 255 heads
      2048 hidden sectors
  14383104 sectors total
Reclaiming unconnected clusters.
Checking free cluster summary.
/dev/sdb1: 59 files, 969072/1794374 clusters

I have checked the following:

  • boot flag was still on partition, so this is not cause

  • Mbr is not a reason, because 446 bytes are untouched and only bytes about partition placement are changed.

  • Files of bootable drive are untouched at their tree position, so they are not the reason.

  • I thought that bytes from 512 to 2047 could contain something, but they are untouched and contains zeroes, so it is not a reason.

  • I do not know utility that could change OEM ID
    Originally it was "MSDOS5.0", but gparted and kde partition manager are changing it to "MSWIN4.1".
    So I counted offset of this messages (there are two of them): 0010:0003 and 0010:0C03 in my case. Then I downloaded corresponding bytes with dd, edited them with hex editor and then uploaded them back with dd.
    I tryed to change "MSWIN4.1" on unbootable-after-resise drive to "MSDOS5.0" and it was still unbootable.
    Also I tryed to change original "MSDOS5.0" on bootable-before-resizing drive to "MSWIN4.1" and surprise it was still booting.
    So, System ID is not a reason.

  • I also tryed to modify uuid of fat32 partition (using this manual) and checked bootableness. It was able to boot, so uuid does not matter.

Maybe somebody who knows FAT32 internals could help me?

I suspect that it could contain some needed information in that reserved blocks (originally there were 3018 of them, after resize there are just 46 of them). Maybe graphical frontend of gparted just ignored that maybe needed information with the actually data? Because originally Data area starts at byte 16777216 (sector 32768), but after resize Data area starts at byte 14393344 (sector 28112), i.e. at sectors of previously reserved blocks location.
Or maybe it expects some files to be physically located at some addresses? (originally Root directory start at cluster 2, after resize Root directory start at cluster 969656 )
And what does that task mean about "changing geometry"? Why? Didn't it just resized a partition with a previous task?

Last edited by agent0 (2015-12-29 03:42:39)

Offline

#2 2015-12-26 03:36:48

tom.ty89
Member
Registered: 2012-11-15
Posts: 897

Re: [solved] LiveUSB becomes unbootable after resizing filesystem

Offline

#3 2015-12-26 11:24:16

Lone_Wolf
Member
From: Netherlands, Europe
Registered: 2005-10-04
Posts: 11,868

Re: [solved] LiveUSB becomes unbootable after resizing filesystem

Also check https://wiki.archlinux.org/index.php/US … tion_media

especially pay attention to section 1.2


Disliking systemd intensely, but not satisfied with alternatives so focusing on taming systemd.


(A works at time B)  && (time C > time B ) ≠  (A works at time C)

Offline

#4 2015-12-26 22:29:19

Regis
Member
Registered: 2013-12-26
Posts: 5

Re: [solved] LiveUSB becomes unbootable after resizing filesystem

agent0 wrote:

Or maybe it expects some files to be physically located at some addresses? (originally Root directory start at cluster 2, after resize Root directory start at cluster 969656 )

I'm no MBR expert, so take all what I'm saying in this post with caution, but I'd say that the problem is indeed related to a wrong address.

My understanding of FAT32 boot process is the following:

1/ The MBR contains instructions to jump to the first sector of the boot partition. The first sector of your boot partition has probably not moved, so I think this step runs normally after resize.

2/ The first two sectors of your FAT32 partition contains a jump to the boot loader code, which is somewhere in the reserved sectors of the partition (the exact address varies). The FAT32 boot sector of your partition has undergone major change (switching from "MSDOS5.0" version to "MSWIN4.1" version, more about that at http://thestarman.pcministry.com/asm/mbr/MSWIN41.htm and http://thestarman.pcministry.com/asm/mbr/DOS50FDB.htm ), and if gparted did not update the jump address contained in the first two sectors then the boot process cannot work properly.

There might be other problems in addition to this one. The boot loader code itself could contain fixed addresses...

Again, I'm not a boot expert and all this is just a guess of what might be happening.

Have you tried to run a Windows boot-record restoration tool on this disk?

Offline

#5 2015-12-26 23:07:21

agent0
Member
From: Russia
Registered: 2014-05-04
Posts: 74
Website

Re: [solved] LiveUSB becomes unbootable after resizing filesystem

Thanks for reply.
No, I was not applying  Windows boot-record restoration tool on this disk. You are talking about ms-sys, right?
I think I need to learn deaper of boot process and learn fat32 structure as in link above. I think it would be better to make another physicall copy of not-working-resized usb flash and ingeneer it's contents for significant differences against original one.
Also, I could upload them (or just interested bytes from them) to some cloud store for everybody could help out with this...

Offline

#6 2015-12-27 05:21:32

tom.ty89
Member
Registered: 2012-11-15
Posts: 897

Re: [solved] LiveUSB becomes unbootable after resizing filesystem

Well: https://gist.github.com/tomty89/49e16c0 … diff=split

Truth is not only the boot sectors are completely destroyed. Apparently I can't even mount it on Windows anymore. It seems to work fine in Linux though. (NOT limited to recovery drive, but general FAT32)

FWIW, interestingly, one can shrink an NTFS in Windows' disk management utility, but not a FAT. Also there is no resize program in dosfstools but ntfsresize in ntfs-3g.

The FAT resizing is done by libparted. However, the feature is probably considered deprecated, since parted no longer has commands that manipulate filesystems. For example, "resize" is removed since 3.0. Now there is only "resizepart".

Last edited by tom.ty89 (2015-12-27 06:24:04)

Offline

#7 2015-12-28 04:24:29

tom.ty89
Member
Registered: 2012-11-15
Posts: 897

Re: [solved] LiveUSB becomes unbootable after resizing filesystem

I filed a bug report here: https://bugzilla.gnome.org/show_bug.cgi?id=759916

Btw any FAT32 created in Windows (10) actually contains the boot code for bootmgr anyway: https://gist.github.com/tomty89/f1830cc … diff=split
The differences of the filesystems are trivial and irrelevant to booting. Therefore you can copy the content from the recovery drive out, recreate a smaller partition, and the copy the content back to it.
Just make sure you flag the partition as "active" with diskpart or fdisk.

I also tried dumping the jump instruction and boot code back to the gparted-resized FAT32, and it boots as well:
https://gist.github.com/tomty89/72ff0df … diff=split

A very nice article about the whole thing: https://en.wikipedia.org/wiki/Design_of … ile_system

Last edited by tom.ty89 (2015-12-28 04:36:39)

Offline

#8 2015-12-28 22:36:15

agent0
Member
From: Russia
Registered: 2014-05-04
Posts: 74
Website

Re: [solved] LiveUSB becomes unbootable after resizing filesystem

tom.ty89, thank you for your exploration and for filing a bug report.
You say that there is no resize program in dosfstools. Who's fault is it: upstream or maintainer?
You say that  "resize" was removed from libpartd and now there is only "resizepart". Name of component are pretty the same. This means that resizing of fat in gparted should be done via resizepart instead of libparted?

tom.ty89 wrote:

you can copy the content from the recovery drive out, recreate a smaller partition, and the copy the content back to it.
Just make sure you flag the partition as "active" with diskpart or fdisk.

Thank you for workaround suggestion. But is it possible only with windows? What if windows becomes broken, and I want to create LiveUSB from linux? Can I create a partition with gparted with correct boot code and place files on that partition? Flag partition as active means applying "boot" flag for partition in gparted or code 80 in mbr?

If no, than I would like to dump the jump instruction and boot code (what sectors?) and then place it back after gparted corrupted FAT32 boot sector while resizing.
So to able to create my recovery usb I will keep files themselves and also a "jump instruction with boot code" in binary file. And when I need to create LiveUSB, I will create partition, fix its "jump instruction with boot code" (how?) and then place files on it. Or is it even possible to create iso image with correct "jump instruction with boot code"?

Offline

#9 2015-12-28 23:48:52

tom.ty89
Member
Registered: 2012-11-15
Posts: 897

Re: [solved] LiveUSB becomes unbootable after resizing filesystem

Well I didn't mean to say that anyone has fault. As I've said, not even Windows support shrinking FAT with its own utility while it support that for NTFS. My point was just that libparted is what being used here.
Filesystem resizing is not removed from "libparted", but only "parted", that means the "official" command-line program can no longer do this job, but "third-party" program like "Gparted" which use "libparted" as backend can still do that.
Partition and filesystem resizing are two seperate thing. "resizepart" is only for the former, and "resize", which only exists in older "parted", is for the latter (or maybe both, I don't know). Gparted simply do both at a time, so that it looks like one thing.

I am not sure what you mean by "create LiveUSB from linux". What LiveUSB? Windows Recovery? Arch ISO? Different medium requires different thing. For example, grub does not even rely on the active flag. And the boot code embedded on the FAT32 is for loading bootmgr only and is generated when you format the drive in Windows. mkfs.fat does not generate such code. I think active flag in gparted is "boot" flag, no idea what code 80 is.

Last edited by tom.ty89 (2015-12-28 23:51:36)

Offline

#10 2015-12-29 00:24:32

agent0
Member
From: Russia
Registered: 2014-05-04
Posts: 74
Website

Re: [solved] LiveUSB becomes unbootable after resizing filesystem

tom.ty89 wrote:

I am not sure what you mean by "create LiveUSB from linux". What LiveUSB? Windows Recovery? Arch ISO? Different medium requires different thing.

I understand that different medium could require different thing. It would be nice if I could create multiboot LiveUSB with Arch and Windows recovery. But I meant I want to be able to create at least just Windows Recovery LiveUSB. From Linux.

tom.ty89 wrote:

I think active flag in gparted is "boot" flag, no idea what code 80 is.

I was talking about first field of partition description in mbr. From wikipedia: Status or physical drive (bit 7 set is for active or bootable, old MBRs only accept 80h, 00h means inactive, and 01h–7Fh stand for invalid).

tom.ty89 wrote:

And the boot code embedded on the FAT32 is for loading bootmgr only and is generated when you format the drive in Windows. mkfs.fat does not generate such code.

Thank you, that info I needed. So, in situation when I have broken windows (i.e. cannot format drive using windows), the only way I can repair drive to be bootable is restore that code for bootmgr. But how can I dump that embedded code and recreate it on drive?

Offline

#11 2015-12-29 00:59:20

tom.ty89
Member
Registered: 2012-11-15
Posts: 897

Re: [solved] LiveUSB becomes unbootable after resizing filesystem

It's not possible (unless you run a Windows VM). It's not just the boot code won't be generated by any tool in Linux, all the files of the Recovery drive comes from Windows itself.

In that case I guess the boot flag / active flag / bit 7 is all the same thing behind the scene.

It's pretty complicated to be honest. Since the jump instruction / boot code are bytes within a sector. You can see the git diff I pasted for details. Anyway since you don't have Windows, you don't have the actual source of the boot code for dd anyway. Maybe I can attach it to you here.

Offline

#12 2015-12-29 01:22:03

agent0
Member
From: Russia
Registered: 2014-05-04
Posts: 74
Website

Re: [solved] LiveUSB becomes unbootable after resizing filesystem

For now I have fresh installed and working windows 10. And also I have created recovery usb from windows and made dd copy of that drive. And it becomes bootable again when dd back to drive. I just want to store not that 8GB dd copy, but files that are on recovery media and the boot sector for bootmgr.
I am interesting this problem because I want to use my flash drive as usual. And only when I need to recover windows in the future, I want to be able to create liveusb on any flash drive.

Some posts before you say:
I also tried dumping the jump instruction and boot code back to the gparted-resized FAT32, and it boots as well:
https://gist.github.com/tomty89/72ff0df … diff=split

I am interesting how you did that? I mean what exactly sectors should I copy from my dd copy and can I just place them back to original location or FAT on drive would become corrupted?

Offline

#13 2015-12-29 01:56:23

tom.ty89
Member
Registered: 2012-11-15
Posts: 897

Re: [solved] LiveUSB becomes unbootable after resizing filesystem

This is how I fixed it: https://gist.github.com/tomty89/c15bea1ed6496b6b4d57

However, the story will probably be different if you want to format from scratch with mkfs.fat (gparted), and then apply windows boot code, since the step taken only fix what gparted destroyed during the resize, instead dumping everything needed.

Actually if you have the recovery files in hand and want to "start from scratch" in Linux, grub (the ntldr command) may be a better option. Also none of these is needed if your PC support UEFI booting, since the files are all it takes to boot.

EDIT: just tested that it boots with a drive prepared by fdisk + mkfs.fat + grub-install

Last edited by tom.ty89 (2015-12-29 02:17:30)

Offline

#14 2015-12-29 02:20:39

agent0
Member
From: Russia
Registered: 2014-05-04
Posts: 74
Website

Re: [solved] LiveUSB becomes unbootable after resizing filesystem

Wow, thank you! It worked. I see that you are placing bytes in their positions. Maybe my question will be very stupid, but will the filesystem be workable if I place all that 7k block to drive?
I have just dumped the differences in hexdump -C -n 7k /dev/sdb1 before and after resizing fat32 partition. Here are they: click
I also created bin dumps of first 7K with sudo dd if=/dev/sdb1 of=fat32_bootable_original.bin bs=1 count=7K
and sudo dd if=/dev/sdb1 of=fat32_unbootable_after_resize.bin bs=1 count=7K
Then make resize. Can I now dd that fat32_bootable_original.bin to /dev/sdb1?
EDIT: of cource. Disk error Press any key to restart.

That laptop has a bios interface only, but thanks for info.
Yes, really grub (the ntldr command) may be a better option. And even it would be possible to make a la multiboot liveusb. Thank you for info, I will learn about it.

Last edited by agent0 (2015-12-29 02:34:20)

Offline

#15 2015-12-29 02:30:27

tom.ty89
Member
Registered: 2012-11-15
Posts: 897

Re: [solved] LiveUSB becomes unbootable after resizing filesystem

I didn't try that, so I don't know. But there are figures of the filesystem (see the wiki page I linked for details) recorded within those gaps, so things can break because the old figures does not apply in the resized filesystem. It's like (partially) reverting the resize.

fdisk /dev/sdX
mkfs.fat /dev/sdXY
mount /dev/sdXY /mnt
grub-install --boot-directory /mnt /dev/sdX

then you will get a working grub shell; and after you copy the recovery files to it, all you need to do is typing in `ntldr /bootmgr` and `boot` in grub shell

P.S. Only the first 6KiB (6144 bytes) is touched by gparted. I only dumped the 7th for reference/comparision.

Last edited by tom.ty89 (2015-12-29 02:36:06)

Offline

#16 2015-12-29 03:39:44

agent0
Member
From: Russia
Registered: 2014-05-04
Posts: 74
Website

Re: [solved] LiveUSB becomes unbootable after resizing filesystem

Yes, that worked also! And it is more convinient solution.
A small correction: grub-install --target=i386-pc --boot-directory=/mnt /dev/sdX
Also, it is possible to create config file for grub for not typing commands in it's shell manually.

And also for somebody who searches I want to repeat your solution about recovering bootableness after resizing in gparted:
if somebody want to use native microsoft's fat32 boot sector, then he/she should copy needed sectors of original partition (before resizing) [while that bug will not be fixed]. If you have done dd copy of whole drive, then you should use offset to point to your partition in that image. Else if you have your bootable drive in hands, than you can just mount that partition (here we will use X as drive letter and Y as partition of this drive)

dd if=/dev/sdXY of=win.code bs=1 skip=80 count=432
dd if=/dev/sdXY of=jump.code bs=1 count=3

Now you can resize your partition with gparted. After that do the following (again XY is your partition):

dd if=win.code of=/dev/sdXY bs=1 seek=80
dd if=win.code of=/dev/sdXY bs=1 seek=3152
dd if=jump.code of=/dev/sdXY
dd if=jump.code of=/dev/sdXY seek=6

Now your drive is able to boot again!

tom.ty89, thank you very much! I will mark this topic as solved. A lot of respect to you.

Last edited by agent0 (2015-12-29 03:41:44)

Offline

#17 2015-12-29 03:58:23

tom.ty89
Member
Registered: 2012-11-15
Posts: 897

Re: [solved] LiveUSB becomes unbootable after resizing filesystem

--target is optional unless you're doing some "cross-platform" installation, like installing grub x86_64-efi when booted in legacy mode or vice versa.

And sure you can write a grub.cfg, I just wanted to point out that the command is simple enough that you don't necessarily need one.

One doesn't need the ORIGINAL filesystem to get the boot code, any FAT32 formatted in Windows (at least those made in Windows 10) will contain the boot code, so he can fix it even if he is aware of the issue AFTER he broke it.

Using `losetup -f -P` (-P is for partition probing on the image) to set up loopback device for the disk image(s) is probably better than calculating the offset yourself. Just remember to detach it after the work is done.

Last edited by tom.ty89 (2015-12-29 04:00:37)

Offline

#18 2015-12-31 20:20:48

agent0
Member
From: Russia
Registered: 2014-05-04
Posts: 74
Website

Re: [solved] LiveUSB becomes unbootable after resizing filesystem

Yes, thanks again for correct notices.
The only involved sector as I understand is 0 sector (first 512 bytes of partition). Backup is stored in 6 sector (i.e. from 512*6=3072 byte), and is not involved in boot process as I think. And also microsoft uses 12 sector (begins at byte 0x1800). In our case it was not touched by gparted, but somebody may want to make it if making liveusb from scratch in linux.

Also, I wanted to correct you a bit. In string dd if=jump.code of=/dev/sdXY seek=6 you mean to restore 6 sector (not 6 byte), so you should use dd if=jump.code of=/dev/sdXY bs=512 seek=6 or dd if=jump.code of=/dev/sdXY bs=1 seek=3072 instead.
Also, I did not find out info about DBR of fat32 from wikipedia article you linked above (it only describes bios parameter block (0x00-0x59), but os boot code offset is 0x5A).
By the way, you could dump just that bytes from 0x5A to 0x1FF, so you could use the following:

dd if=/dev/sdXY of=jump.code bs=1 count=3          # taking jump instruction code
dd if=/dev/sdXY of=win.code bs=1 skip=90 count=422 # taking os boot code
dd if=/dev/sdXY of=ms12.code bs=1 skip=6144 count=512 # taking 0xC sector (microsoft)
# restoring sector 0:
dd if=jump.code of=/dev/sdXY                # byte 0x0000
dd if=win.code of=/dev/sdXY bs=1 seek=90    # byte 0x005A
# restoring sector 6:
dd if=jump.code of=/dev/sdXY bs=1 seek=3072 # byte 0x0C00
dd if=win.code of=/dev/sdXY bs=1 seek=3162  # byte 0x0C5A
# restoring sector 12:
dd if=ms12.code of=/dev/sdXY bs=1 seek=6144 # byte 0x1800

I think it would be better to store that sectors' contents in binary format in some utility to be able to easy recreate them (maybe in ms-sys utility?).

Offline

#19 2016-01-01 03:57:08

tom.ty89
Member
Registered: 2012-11-15
Posts: 897

Re: [solved] LiveUSB becomes unbootable after resizing filesystem

It doesn't mean to restore 6 sector:

seek=N skip N obs-sized blocks at start of output

So dd skips LBA 0-5 (6 sectors) and outputs at LBA 6. And I don't need to specify bs/count here because it simply writes the whole dump (jump.code, which is three-byte).

I don't quite know about the things followed. I didn't bother to dig as deep as I can tell where exactly does the necessary boot code start and end or so. If you really want to capture every boot bits Windows generate, I guess you need to dump the code in the MBR as well.

Offline

#20 2016-01-03 20:13:35

agent0
Member
From: Russia
Registered: 2014-05-04
Posts: 74
Website

Re: [solved] LiveUSB becomes unbootable after resizing filesystem

Sorry about correcting you, I thought that bs is 1 by default. If bs or obs is not specified then obs is 512.
So all three variants have the same result:
dd if=jump.code of=/dev/sdXY seek=6
dd if=jump.code of=/dev/sdXY bs=512 seek=6
dd if=jump.code of=/dev/sdXY bs=1 seek=3072

Yes, I could also dump mbr, but I can restore it with ms-sys. Funny thing is that I could even restore that corrupted boot code either.
So to workaround gparted bug you can just do the following (for example, your device letter is 'b' and partition number is '1'):

ms-sys --mbr7 /dev/sdb
ms-sys --fat32pe /dev/sdb1

After that it is bootable again.
EDIT: creating ms mbr is only needed when creating liveusb from scratch. You can skip that step in case of restoring after resizing in gparted, due to gparted do not touches boot code in mbr.

Last edited by agent0 (2016-01-04 09:02:27)

Offline

#21 2016-01-04 03:48:32

tom.ty89
Member
Registered: 2012-11-15
Posts: 897

Re: [solved] LiveUSB becomes unbootable after resizing filesystem

Oh okay. Didn't know such thing exist.

Offline

Board footer

Powered by FluxBB