Booting problem with an external USB Hard Disk

casiso · 2012-09-01 05:47:55

I have a 750GB 2.5 inch hard disk in a NextStar3 USB3/ESata enclosure running Archlinux. The boot loader used is Grub2.

This disk is able to boot without problems when connected to an ESata port. With USB ports, it can boot boot on some PCs but not in others. The strange thing is that it mainly fails to boot on recent Dell & HP PCs and laptops. In each case, I have been able to boot the PCs from a 8GB USB sysrescue pendrive, but those PCs failed to detect the external disk (It is not shown at the bios boot list!). From the PCs bios setup, they are supposed to boot from USB-floppy and USB-Hard disk.

One thing I tried today was to boot grub-legacy from the USB sysrescue pendrive and edited a grub.conf entry to try to load the kernel from the external hard disk, but grub-legacy will not detect the external drive too.

So the questions are if that is a bios problem on those PCs or it could be something related to the external disk enclosure, the external disk size, the used partition scheme (MBR) or to the Grub2 boot loader? Is it worth it to try to replace the external disk partition scheme (MBR to GPT), replace Grub2 with Syslinux or use a boot manager like Plop (http://www.plop.at/en/bootmanager/index.html)?

Thanks in advance.

Regards,

WonderWoofy · 2012-09-01 06:04:00

Did you add the usb hook to mkinitcpio.conf... it sounds kinda like the necessary modules are not loaded by the intramfs. Hence eSATA works but not new USB... maybe 3.0?

DSpider · 2012-09-01 06:10:53

Set the BIOS to boot from "Removable Devices" first, "Optical Media" second and "Internal Storage" third - or something to that extent (it depends on the BIOS).

If the hardware changes (and chances are that it might, if you plan on whoring it from computer to computer), you need to re-generate the initramfs, tailored to that specific hardware. Or, you could use the Fallback image to load all SATA, IDE, ext2/3/4, etc modules instead of the smaller (3.5 MB), more compact "initramfs-linux.img". Don't forget the "usb" hook in mkinitcpio.conf when you do this.

Adds USB modules to the image. Use this if your root device is on a USB mass storage device or if your USB mass storage device needs to be accessed otherwise (checked, mounted, etc.) at boot time.

https://wiki.archlinux.org/index.php/Mkinitcpio#HOOKS

WonderWoofy · 2012-09-01 06:16:58

^ yeah I guess that is a little better of an explanation...

casiso · 2012-09-01 16:34:29

Thanks for yours answer,

1.- The mkinitcpio.conf usb hook is added. In fact, the disk boot from USB on a variety of systems.
2.- I disagree with the statement "you need to re-generate the initramfs, tailored to that specific hardware". In fact, to load the kernel you just need to have the udev, usb, scsi and sata hooks (to boot from USB or ESATA) with the corresponding filesystem modules (ext2 & ext4 in my case). The rest of the hardware is initialized by udev after the kernel is loaded when it has access to all compiled modules at /lib/modules/... I have checked the kernel configuration file ("/proc/config.gz") for the current kernel (3.5.3-1-ARCH in my case) and the list of modules included on the kernel cover almost all systems I will/can put my hands on.
3.- On the faulty systems, I have enabled any available boot options with no success. The fact that the sysrecue pendrive that boots on those system is parted as a hard disk, with 2 partitions (sysrecue 2GB booteable and a NTFS 6G partition) discards partition table problems (bios with USB floppy/CDROM only boot).
4.- CPU and memory speed seems not to be a factor here too.

I have done some googling around and I think the problem could be around the following:

1.- Limits on the bios to detect big USB disks. I don't know how to verify this as I have just one 750GB@7200rpm external disk (well, in fact I have two but they are identical). Some boot managers seems to address that (plop by the way). Have anybody any experience with that?

2.- Warm up time of the USB hard disk. Maybe the disk is taking so long to warm up, so the bios fail to detect the USB disk drive presence at all. The fact that the disk does not shows at the Bios Boot device list (when pressing Fn keys at boot time) and that grub-legacy fails to detect the USB drive when it is booted from a USB pen drive suggest that, assuming grub-legacy get the available drive list from the bios (not sure about that). Is there any way to determine the disk warm up time? (a benchmark program maybe?).

If the problem is warm up time related, I have some options left. Assuming grub2/syslinux/plop implements their own disk detection (not the bios one), it is possible to run the boot loader from a pen drive and direct it to boot the kernel from the USB hard disk, not an ideal solution by the way. Another option could be to use a SSD disk, assuming that they have the same or better warm up times than a pendrive, but I would like to validate that before spending 400$ on a 500G SSD. Maybe the problem is in the NextStar3 USB3/ESata enclosure that takes too much time to initialize the disk drive (I will by another USB/SATA enclosure and test it with the other disk I have). Any ideas on this would be appreciated.

Thanks again,

Regards,

lilsirecho · 2012-09-01 22:27:31

Perhaps AFAICT grub2 will not permit USB boots. This has been in posts for a year or more.

WonderWoofy · 2012-09-01 22:46:00

casiso wrote:

2.- I disagree with the statement "you need to re-generate the initramfs, tailored to that specific hardware". In fact, to load the kernel you just need to have the udev, usb, scsi and sata hooks (to boot from USB or ESATA) with the corresponding filesystem modules (ext2 & ext4 in my case). The rest of the hardware is initialized by udev after the kernel is loaded when it has access to all compiled modules at /lib/modules/... I have checked the kernel configuration file ("/proc/config.gz") for the current kernel (3.5.3-1-ARCH in my case) and the list of modules included on the kernel cover almost all systems I will/can put my hands on.

This is not entirely true. While yes it will boot on most new systems these days, the purpose of the initramfs is (as you seem to understand) provide any modules that are not compiled into the kernel itself taht are required to boot. If you, for instance, only include the sata hook, taking that drive to an older pata or scsi driven computer will not result in a boot. This is why it was recommended above that you may use the fallback intramfs. The fallback simply omits the autodetect hook, which does its best to detect which modules you are actually using, and exclude those that don't seem necessary for boot. So if you do not use autodetect, and say you have a pata and sata disk in your system, the initramfs will contain both. But if your system is actually booting off the sata only, it will rid itself of the pata modules.

Of course the config.gz is going to contain the necessary modules for a usable system on nearly any machine. That is the only way a distribution can manage to support so much hardware with a stock kernel. But if your computer cannot get to the point of loading udev to detect the rest of the necessary modules, you're sh*t out of luck.

BTW, it very well could be your disk is not spinning up fast enough, as I have actually expeienced this once. But I think that with a 750GB drive you are nowhere near the limits of the bios. I am fairly certain that the legacy bios limits are 2TB, so you're not even half way there... even if you did a RAID JBOD, you would still only be 3/4 of the way there, so I think that is not it.

casiso · 2012-09-02 04:55:54

lilsirecho wrote:

Perhaps AFAICT grub2 will not permit USB boots. This has been in posts for a year or more.

Regarding grub2 not to be able to boot from USB, I am sure it can. My system does and I have been doing that even with grub-legacy for a few years.

WonderWoofy wrote:

This is not entirely true. While yes it will boot on most new systems these days, the purpose of the initramfs is (as you seem to understand) provide any modules that are not compiled into the kernel itself taht are required to boot. If you, for instance, only include the sata hook, taking that drive to an older pata or scsi driven computer will not result in a boot. This is why it was recommended above that you may use the fallback intramfs. The fallback simply omits the autodetect hook, which does its best to detect which modules you are actually using, and exclude those that don't seem necessary for boot. So if you do not use autodetect, and say you have a pata and sata disk in your system, the initramfs will contain both. But if your system is actually booting off the sata only, it will rid itself of the pata modules.

Please take into account that when booting from a USB disk, it does not matter what hardware (pata, scsi, sata), file systems etc. are on the target machine. You just need the modules required to boot from the USB in the kernel and initramfs and include udev or set a kernel option to force load the modules at boot time (In Gentoo, it used to be doload=). Once the kernel is loaded and the root partition (in the USB disk) is mounted, udev will take care of setting up whatever hardware is on the machine (while it can detect it and there is a corresponding module available at /lib/modules/...).

By the way, you can't boot a sata drive in an old pata or scsi computer no matter what initramfs you use :-).

WonderWoofy wrote:

But I think that with a 750GB drive you are nowhere near the limits of the bios. I am fairly certain that the legacy bios limits are 2TB, so you're not even half way there... even if you did a RAID JBOD, you would still only be 3/4 of the way there, so I think that is not it.

I believe the 2TB limit is a file system limit. I understand that legacy bios have problems to boot partitions of that size and that is one of the main reasons to use GPT partition tables instead of MBR. Additionally, there is indication that some bios may have a 128GByte limit regarding USB devices (Please see http://www.plop.at/en/bootmanager/features.html ).

Thank you for yours answers.

WonderWoofy · 2012-09-02 07:00:03

That is a very good point that you are using a usb stick... for some reason I was picturing in my head, you taking a raw hdd from computer to computer. You sir would be right on that.

Also, yes I know that the 2TB limit is a filesystem limit... but I was certainly not aware of this 128GB USB limit. What a bunch of crap! These days, 128GB is nothing these days. But regarding your original mention that grub/syslinux/etc use their own detection mechanisms and you could therefore boot from them, I think you would be right. I have no evidince to back this up, and I know this is not the same thing, but I used to use a MacBook that would not boot from a USB anything in bios compatibility mode. My solution was to put an entry into grub that I could mess with on the fly. Not quite the same restriction, but it seems like a restiction all the same, and thus exemplifies its ability to work around and do its own thing.

the.ridikulus.rat · 2012-09-02 07:56:38

You need to load grub(2)'s usb module for it to detect usb drives.

insmod usbms

casiso · 2012-09-17 15:11:11

I have made some additional testing and it seems the grub's usbms module has some issues. I tested it on a Desktop and a Laptop. On the desktop, loading usbms, grub failed to detect the USB disk. On the Laptop, loading usbms freeze the machine. Then I tried with Supergrub (http://www.supergrubdisk.org/) but it had problems in both machines, mainly related with video. It was not able to detect the USB disk.

I have been able to test the USB disk in a HP Pavillion DV9700 and found the BIOS has a disk warm up delay. Setting this delay, the USB disk has been able to boot. So it seems to confirm that the problem is timing related, but I have not seen this delay in any other bios.

Anyway, I will try to make the test with an SSD in the next few weeks as to discard any problem related to the external USB enclosure.

Arch Linux

#1 2012-09-01 05:47:55

Booting problem with an external USB Hard Disk

#2 2012-09-01 06:04:00

Re: Booting problem with an external USB Hard Disk

#3 2012-09-01 06:10:53

Re: Booting problem with an external USB Hard Disk

#4 2012-09-01 06:16:58

Re: Booting problem with an external USB Hard Disk

#5 2012-09-01 16:34:29

Re: Booting problem with an external USB Hard Disk

#6 2012-09-01 22:27:31

Re: Booting problem with an external USB Hard Disk

#7 2012-09-01 22:46:00

Re: Booting problem with an external USB Hard Disk

#8 2012-09-02 04:55:54

Re: Booting problem with an external USB Hard Disk

#9 2012-09-02 07:00:03

Re: Booting problem with an external USB Hard Disk

#10 2012-09-02 07:56:38

Re: Booting problem with an external USB Hard Disk

#11 2012-09-17 15:11:11

Re: Booting problem with an external USB Hard Disk

Board footer