Syslinux Installer / Update Script - Testers Needed

pyther · 2011-01-16 18:07:35

For those who don't follow the ML that closely....

Hello Community,

Over the last few weeks I have been working on Syslinux support for the installer. With the help Thomas and Dieter I am nearing the completion of this project. As part of this project, I have written a script that will help install and update Syslinux (similar to that of grub-install).

Some key features of the script: syslinux-install_update.sh
* Install Syslinux to the FS + Partition Boot Loader (extlinux --install /boot/syslinux)
* Install Syslinux MBR
* Detect and optionally set the boot flag on the boot partition
* Update Syslinux – copy files and execute (extilnux --update /boot/syslinux)
* Support for GPT disks
* Support for RAID configurations

The goal is to include this script in the official Syslinux package. Therefore we need your help to test it.

syslinux-install_update.sh -i -a -m ..... install Syslinux, set the boot flag (if needed), and install the MBR

We need tests for the following setups:
/ + /boot on the *same* partition
/ + /boot on the *same* partition - RAID
/boot + root on *separate* partition
/boot + root on *separate* partition - RAID
All of the above using but using the GPT partition layout

NOTE: This is an alpha/beta stage script. The script modifies the first 440 bytes of the disk (using dd) and the partition table (using either sfdisk or sgdisk). Although the script should be safe to run, I am not responsible for any data loss that may occur.

Let us know the following:
* Did the script work for you?
* What was your partition setup? (see above)
* What version did you use?
* If the script did not work, please provide as much information as possible

Get the script here: https://gist.github.com/772138
Syslinux Sample Config File: http://projects.archlinux.org/svntogit/ … slinux.cfg

The Syslinux package in testing includes the above configuration file.

Cheers,
pyther

falconindy · 2011-01-16 19:49:02

May I propose some Bash advice?

* The shebang needs to be line 1. You're using bash4 features, so this is particularly true. You don't want /bin/sh accidentally getting hold of this. Even though /bin/sh is a symlink, bash behaves differently when invoked as /bin/sh.
* The 12 calls to findmnt, grep and awk can be condensed into 2 findmnt calls:

read rootdev rootfs < <(findmnt -run -o SOURCE,FSTYPE "$CHROOT")
read bootdev bootfs < <(findmnt -run -o SOURCE,FSTYPE "$CHROOT/boot")

* lines 270, 272, 343, and 345, you're directly calling /sbin/extlinux instead of using the predefined variable from line 24.
* mdraid_is_raid should return 0 or 1 instead of echoing, since you're not interested in the return string. This lets you simply it to:

mdraid_is_raid()
{
    # returns 0 on true, 1 on false
    if [[ -z $1 || "$(mdadm --query "$1" | cut -d':' -f2)" == " is not an md array" ]]; then
        return 1
    fi  
    
    return 0
}

You can then simplify the calls to the function and just look for pass/fail instead of calling [[. For example, line 160 becomes:

if mdraid_is_raid "$bootpart"; then

lines 268 and 341 would have to be changed as well. I should also add that matching against an exact string like this probably isn't a fantastic solution, but I unfortunately don't have a better idea at the moment. Parsing /proc/mdstat may be a better solution as it's less prone to change. I'll raise a similar caveat for line 148 -- that warning will likely go away one day.

I'd love to try this out, but I'm stuck on a laptop which is too old to run a VM without eating itself alive.

pyther · 2011-01-16 20:10:29

Thanks for the suggestions/tips. I added them to the script. If you want to you could try the script on your laptop. :-)

I'll have to explore the raid utilities to see if I can improve the mdraid_is_raid function. With the fdisk check for GPT disks, I haven't found another tool that will give detect the partition table. There is parted, but I don't want to depend on that, mainly since it isn't in core.

falconindy · 2011-01-16 20:35:33

I think I've found something more reliable. As per a reference from kernel.org, a block device with a major ID of 9 is always a 'metadata' (raid) device. major ID 8 are your /dev/sd?? devices. So, we can tidy up the mdadm function a fair bit:

mdraid_is_raid() {
  [[ $1 && -f /proc/mdadm ]] || return 1
  local devmajor=$(stat -c %t "$1")
  (( devmajor == 9 ))
}

This is a bit implicit -- the return value is the result of the comparison, which still adheres to the 0=true, 1=false schema.

Dieter@be · 2011-01-16 20:55:55

falconindy wrote:

I think I've found something more reliable. As per a reference from kernel.org, a block device with a major ID of 9 is always a 'metadata' (raid) device. major ID 8 are your /dev/sd?? devices. So, we can tidy up the mdadm function a fair bit:
mdraid_is_raid() {
  [[ $1 && -f /proc/mdadm ]] || return 1
  local devmajor=$(stat -c %t "$1")
  (( devmajor == 9 ))
}
This is a bit implicit -- the return value is the result of the comparison, which still adheres to the 0=true, 1=false schema.

you can write some pretty bash, you know that?

Mr Green · 2011-01-16 21:18:21

What is the reason/advantages for using syslinux over Grub?

pyther · 2011-01-16 21:52:38

As Thomas puts it:

Thomas wrote:

Syslinux is way more flexible and extensible than grub or lilo, actively developped (by someone who knows what he is doing and is always willing to fix bugs like yours) and feature-rich.

http://mailman.archlinux.org/pipermail/ … 17368.html

In regards to bootloaders in core

Pierre wrote:

ATM. we have grub1 in core/base and install that by default. The problem is that this project is virtually dead for a long time now and also not available on x86_64. Technically it has to be in the multilib repo.
Grub2 is currently in extra. Upstream development is still in flux. Imho its quite heavy and complex. An alternative successor would be extlinux from the syslinux package. It's very simple, easy to configure, actively maintained and reliable. Sure, it only supports booting from ext* and btrfs afaik but to be honest, if you use any other FS you should have a separate /boot even when using grub.

http://mailman.archlinux.org/pipermail/ … 18445.html

Why I like it:
* It is simple and easy to understand (see https://wiki.archlinux.org/index.php/Sy … t_Process)
* The devs are willing to help you out (they have helped me understand the Syslinux boot process and write this script)
* It is modular in nature and the Hardware Detection Tool (HDT) com32 module is cool
* Configuration is simple
* KISS (IMHO)
* GPT Support

Disadvantages:
* Can't boot from LVM volumes
* only ext2/3/4, btrfs, vfat file systems supported

Last edited by pyther (2011-01-16 21:56:41)

Chriffer · 2011-01-16 23:11:33

I just tested the script, and it worked fine. I had to make /boot/syslinux/ myself (but this sounds like it's just not in the current package). My computer has 3 hard drives currently, with /boot as a 500mb vfat partition. Everything worked as expected when I ran the script with -i to install the files. I put together a syslinux.cfg and told it to write to the mbr. Everything worked as expected including the vesa mode.

falconindy · 2011-01-16 23:51:23

GPT specification says that LBA1 (bytes 513 through 1024), start with a specific signature -- the words 'EFI PART' will literally be emblazoned into the first 8 bytes. Soo....

device_is_gpt() {
    local device=${1%%[[:digit:]]*}
    local partsig=$(dd if=$device skip=64 bs=8 count=1 2>/dev/null)
    [[ $partsig = "EFI PART" ]]
}

Last edited by falconindy (2011-01-21 04:19:20)

pyther · 2011-01-17 02:23:30

@Chriffer thanks! The package in testing has an example config /boot/syslinux/syslinux.cfg, therefore when that file gets installed /boot/syslinux gets created.

falconindy, that function is amazing! Thanks!

I updated the code using the new functions.

skottish · 2011-01-21 03:16:56

Hi pyther,

I'm not sure if this has been covered yet, but the version in testing didn't check to see if I already had a syslinux.cfg file. I moved mine out of the way, installed the new package, and then put my copy back. I feel that creating the .pacnew file is the best solution, assuming that there isn't user intervention required.

Here is my insanely simple setup and probably not real helpful (comments removed):

syslinux.conf

PROMPT 1
TIMEOUT 50
DEFAULT arch

LABEL arch
        LINUX /vmlinuz26
        APPEND root=/dev/sda3 ro
        INITRD /kernel26.img

LABEL archfallback
        LINUX /vmlinuz26
        APPEND root=/dev/sda3 ro
        INITRD /kernel26-fallback.img

fstab

devpts                 /dev/pts      devpts    defaults            0      0
shm                    /dev/shm      tmpfs     nodev,nosuid        0      0

UUID=46df4be7-a9e6-4a47-9bea-35ec18a693ae swap swap defaults 0 0
UUID=8b595dcd-0712-4603-801e-6a9c7c8c02fe /home ext4 defaults 0 1
UUID=98b2483a-f20e-4f7e-9e40-956b15cd844d /boot ext2 defaults 0 1
UUID=f5ef9643-d408-4139-938b-795ec29f1c77 / ext4 defaults 0 1
UUID=da0e15dc-bd5e-430d-a621-174be68cd135 /backup ext4 defaults 0 1

Mr Green wrote:

What is the reason/advantages for using syslinux over Grub?

It's really slick in that it can be as KISS or complex as one wants it to be, provided that it supports the user's choice of file system. I've had troubles with both GRUB and GRUB2 in the past, and I've experienced none so far with syslinux.

pyther · 2011-01-21 03:34:15

To the best of my knowledge there is no way to create a pacnew file since pacman does not know about syslinux.cfg. However, I may be wrong since my knowledge on packaging is mediocre at best.

joeDeuce · 2011-01-21 12:49:20

The script worked flawlessly for me, on a fresh syslinux install.

/boot (ext2) and / (ext3) on separate partitions
using syslinux 4.03-2

pyther · 2011-01-24 00:53:13

Could really use some testers, especially with RAID and GPT setups!

Misfit138 · 2011-01-24 01:05:05

Since development is active, did you get any indication from the devs if they were planning on adding support for other filesystems?

kazuo · 2011-01-24 01:59:16

pyther wrote:

Could really use some testers, especially with RAID and GPT setups!

Sorry for the delayed test (helping gf with an exam/presentation if I said "sweet, I can't reach the .tex of your report, because pyther broke my boot!" she would break my, err.. you know ). Its worked for me with a minor problem.

But, nothing new. /boot in /dev/sda1 (ext2) and / in /dev/sda2 (ext4) (and a /home dm-crypt in /dev/sda3).

The problem is, for some reason I had a boot flag in the /boot partition (I used the -a switch) AND my swap had a boot flag too (I think, that I put windows on it to flash my bios a long time ago...)

And the system don't booted, said that 'multiple bootable partitions bla bla bla, insert disk to continue....' or something like this. Unflaged the dumb swap boot flag and all is ok now.

What about checking for this? i.e. see if the user have multiple boot flags and warn him, so he can take some actions (and don't get the error I got). I see that the script had some check for active partition, but (in my limited bash foo) its looks its only check for >= 1, and not for == 1.

Thanks pyther for the work, syslinux looks real nice.

falconindy · 2011-01-24 02:06:14

mdraid_all_slaves ()
{
    shopt -s nullglob
    local slave=
    local slaves=
    for slave in /sys/class/block/${1##*/}/slaves/*; do
        source "$slave/uevent"
        slaves="$slaves/dev/$DEVNAME "
        unset DEVNAME
    done
    echo $slaves
}

This doesn't do what you want it to. Just echo inside the foreach rather than building a string and then echoing. You're also abusing the environment by enabling nullglob here. It'll be enabled for the remainder of the script. If this is really what you want, just declare it up at the top of the script so its more visible to someone else reading the code.

mdraid_all_slaves() {
    for slave in /sys/class/block/${1##*/}/slaves/*; do
        source "$slave/uevent"
        echo "$slave/dev/$DEVNAME"
        unset DEVNAME
    done
    unset slave
}

Last edited by falconindy (2011-01-24 02:10:26)

pyther · 2011-01-25 00:22:17

@Falconindy thanks for pointing out my nullglob mistake, someone mistakenly told me it only applied to the local function

Misfit138 wrote:

Since development is active, did you get any indication from the devs if they were planning on adding support for other filesystems?

hpa, a syslinux dev, told me that someone is working on NTFS support and that many have requested xfs support.

@kazuo thanks for testing the script. Indeed we should check that only one partition has the boot flag set.

With GPT disks I can glob through /sys/block/sdx/sdx* and run sgdisk /dev/sda --attributes=1:show:2, however with MBR disks I'll have to sort through the output of fdisk -l /dev/sdx

falconindy · 2011-01-25 01:25:23

pyther wrote:

I'll have to sort through the output of fdisk -l /dev/sdx

You might as well go and scrape your nails down a chalkboard. Stop that. To the MBR standard we go.

part_is_active() {
  local device=$1
  local partnum=${device//[[:alpha:]]/}
  local offset=$(( 446 + (partnum - 1) * 16 ))
  local active parttype

  [[ -e /dev/$device ]] || return 2 # ENOENT

  # we won't install to a logical partition
  (( partnum > 4 )) && return 3

  # get the partition type -- we won't install to an extended partition
  read _ parttype < <(hexdump -s $(( offset + 4 )) -n 1 /dev/${device%$partnum})
  [[ $parttype == 0005 ]] && return 3

  read _ active < <(hexdump -s $offset -n 1 /dev/${device%$partnum})
  [[ $active == 0080 ]]
}

This expects a parameter such as 'sdc1'.

Last edited by falconindy (2011-01-25 01:58:39)

el mariachi · 2011-01-28 00:29:05

the script worked but now tuxonice won't (it did with grub) any idea why this would happen?
I have a separate /boot btw

edit: it fixed magically..

Last edited by el mariachi (2011-01-29 01:17:24)

pyther · 2011-02-06 21:46:48

Over the last few weeks I have been working on improving this script. Huge thanks to falconindy! He has written code which uses hexdump to determine if a disk is using the GPT partition table and code which detects if a partition is marked as active. Prior to this code I was greping and awking the output of fdisk!

Notable Changes
* Use hexdump to determine if boot flag is set
* Use hexdump to determine partition type (GPT)
* Copy/Symlink pci.ids to /boot/syslinux
* Lots of rewritten code

Much of the detection code has changed. So if you tested before, please test again!

Syslinux-install_update: https://gist.github.com/772138
Sample Usage: syslinux-install_update -iam (install, set /boot part(s) as active, install mbr)

The syslinux package in testing has a sample config. If you don't want to use the package from testing you can get the config from here: http://projects.archlinux.org/svntogit/ … slinux.cfg

Tell us as much information as possible about your setup (see first post) and whether or not the script worked.

P.S. Thanks to e36freak who has greatly helped improve my bash syntax and the syntax in this script!

Last edited by pyther (2011-02-06 21:48:32)

valium97582 · 2011-02-10 02:56:29

Worked nicely for me using the tried and true partition system:

/
/boot
/home
swap

BTW, I'm using MBR.

Ledti · 2011-02-13 01:42:59

Worked fine for me using d1c1c2. I've also used two other version, one from a few days ago and one of the initial versions.

/dev/sda:
 /sda1: windows reserved ...
 /sda2: windows ...
 /sda3: boot *
 /sda4: -- (extended)
  /sda5: root
  /sda6: swap
  /sda7: home

I tested -s, -s again after removing the SYSLINUX_AUTOUPDATE file (it worked as intended), -u, and then finally -i (as a test). I was surprised to see that -i didn't overwrite my custom syslinux.cfg file which was nice (I have syslinux-4.03-2 installed).

I've also used the -c and -m options before, but that was with one of the previous versions.

Last edited by Ledti (2011-02-13 01:45:24)

pyther · 2011-02-13 02:15:31

I updated the script again this weekend, so I could really use some testers.
*Different exit codes depending what part fails
*dev_is_part - checks to make sure the block device is partitioned (prevents mbr/bootflag being installed/set on /dev/sda)
*clear_gpt_attr2 - clears attribute 2 (BIOS Legacy Bootable) on all partitions of a disk
*Added support for cciss and ida controllers

@Ledti Thanks for the detailed information!

kazuo · 2011-02-13 03:05:13

I'm getting a strange problem with syslinux (I'm using the pack from testing and pyther install script).

90% of time, the boot sequence get frozen at the syslinux copyright screen (the one before the menu screen). I can leave it all day and it don't continue.

But if I smash (as 'hit it as fast I can') the enter key before the copyright screen comes up, its boot ok (of course the menu screen pass in a flash with all the enter key smashing).

This can be related to syslinux packaging and/or pyther script? Or you think that this is an upstream problem?

EDIT: my syslinux.conf http://codepad.org/0yhg8gAr

Last edited by kazuo (2011-02-13 03:07:36)

Arch Linux

#1 2011-01-16 18:07:35

Syslinux Installer / Update Script - Testers Needed

#2 2011-01-16 19:49:02

Re: Syslinux Installer / Update Script - Testers Needed

#3 2011-01-16 20:10:29

Re: Syslinux Installer / Update Script - Testers Needed

#4 2011-01-16 20:35:33

Re: Syslinux Installer / Update Script - Testers Needed

#5 2011-01-16 20:55:55

Re: Syslinux Installer / Update Script - Testers Needed

#6 2011-01-16 21:18:21

Re: Syslinux Installer / Update Script - Testers Needed

#7 2011-01-16 21:52:38

Re: Syslinux Installer / Update Script - Testers Needed

#8 2011-01-16 23:11:33

Re: Syslinux Installer / Update Script - Testers Needed

#9 2011-01-16 23:51:23

Re: Syslinux Installer / Update Script - Testers Needed

#10 2011-01-17 02:23:30

Re: Syslinux Installer / Update Script - Testers Needed

#11 2011-01-21 03:16:56

Re: Syslinux Installer / Update Script - Testers Needed

#12 2011-01-21 03:34:15

Re: Syslinux Installer / Update Script - Testers Needed

#13 2011-01-21 12:49:20

Re: Syslinux Installer / Update Script - Testers Needed

#14 2011-01-24 00:53:13

Re: Syslinux Installer / Update Script - Testers Needed

#15 2011-01-24 01:05:05

Re: Syslinux Installer / Update Script - Testers Needed

#16 2011-01-24 01:59:16

Re: Syslinux Installer / Update Script - Testers Needed

#17 2011-01-24 02:06:14

Re: Syslinux Installer / Update Script - Testers Needed

#18 2011-01-25 00:22:17

Re: Syslinux Installer / Update Script - Testers Needed

#19 2011-01-25 01:25:23

Re: Syslinux Installer / Update Script - Testers Needed

#20 2011-01-28 00:29:05

Re: Syslinux Installer / Update Script - Testers Needed

#21 2011-02-06 21:46:48

Re: Syslinux Installer / Update Script - Testers Needed

#22 2011-02-10 02:56:29

Re: Syslinux Installer / Update Script - Testers Needed

#23 2011-02-13 01:42:59

Re: Syslinux Installer / Update Script - Testers Needed

#24 2011-02-13 02:15:31

Re: Syslinux Installer / Update Script - Testers Needed

#25 2011-02-13 03:05:13

Re: Syslinux Installer / Update Script - Testers Needed

Board footer