Block Device Detection Code - Testers Needed

pyther · 2011-02-20 02:20:35

Hello!

Tonight I have been working an rewriting the finddisks() and findblockdevices() functions in AIF.

finddisks - locate partitionable blockdevices (sda, sdb, /dev/cciss/c1d0
fubdev - find partitions, lvm volumes, RAID + md_crypt devices

In relationship to AIF
finddisks = Partitioner and Bootloader
fubdev = mkfs

Example:

Disk Devices
/dev/sda /dev/sdb /dev/sdc 
Usable Devices
/dev/sda1 /dev/sda2 /dev/md0

Since /dev/sdb1 and /dev/sdc1 are part of /dev/md0 they are not listed under "Usable Devices"

What I need you to do
*Run script as root (pvscan only works as root)
*Paste output of script
*Paste output of findmnt -s <== If you don't mind
*Let me know if the are any devices that shouldn't be included in "Usable Devices"
*Any other issues, errors, warning, etc

Thanks

#!/bin/bash

# find partitionable blockdevices
# $1 extra things to echo for each device (optional) (backslash escapes will get interpreted)
finddisks() {
    shopt -s nullglob

    # Block Devices
    if [[ -d /sys/block ]]; then
        for dev in /sys/block/*; do
            if [[ -f $dev/device/type ]]; then
                local type
                read type < /sys/block/${dev##*/}/device/type
                # Type 5 is a ROM Device
                if [[ $type != 5 ]]; then
                    source "$dev/uevent"
                    echo -ne "/dev/$DEVNAME $1"
                    unset DEVNAME
                fi
            fi
        done
    fi

    # cciss controllers
    if [[ -d /dev/cciss ]]; then
        for dev in /dev/cciss/*; do
            if [[ $dev != *[[:digit:]]p[[:digit:]]* ]]; then
                echo "$dev $1"
            fi
        done
    fi
    # Smart 2 Controller
    if [[ -d /dev/ida ]]; then
        for dev in /dev/ida/*; do
            if [[ $dev != *[[:digit:]]p[[:digit:]]* ]]; then
                echo "$dev $1"
            fi
        done
    fi
    shopt -u nullglob
}

# find block devices, both partionable or not (i.e. partitions themselves)
# $1 extra things to echo for each partition (optional) (backslash escapes will get interpreted)
# find usable block devices
fubdevs() {
    shopt -s nullglob

    local parts

    # Cycle through all non-partitional block devices (sda, sdb, etc...)
    # Look for partitions and include them only if they are not part of a
    # RAID or LVM. Also, exclude the extended partition
    for devpath in $(finddisks); do
        hidebldev=
        unset parts

        # Glob allows for following matches:
        # /dev/sda -> /dev/sda1
        # /dev/cciss/c0d1 -> /dev/cciss/c0d1p1
        for dev in ${devpath}*[[:digit:]]*; do
            local disk="${dev%%[[:digit:]]*}"
            local partnum="${dev##*[[:alpha:]]}"

            # Don't display extened partition (not logical parts)
            sfdisk -c "$disk" "$partnum" 2>/dev/null | grep -q '5' && continue;

            # Don't list parts that are part of RAID or LVM volumes
            if grep -q "${dev##*/}" /proc/mdstat 2>/dev/null || { pvscan -s 2>/dev/null | grep -q "$dev"; }; then
                hidebldev="True"
            else
                parts+=("$dev")
            fi
        done

        # If we have no partitions then print the root of the block device
        # Otherwise print the partitions
        if [[ -z $hidebldev ]] && (( ${#parts[@]} < 1 )); then
            echo -ne "$devpath $1"
        elif (( ${#parts[@]} )); then
            for part in ${parts[@]}; do
                echo -ne "$part $1"
            done
        fi
    done

    # mapped devices
    for devpath in /dev/mapper/*; do
        # Exclude /control directory and other non-block files
        if [[ -b $devpath ]]; then
            echo -ne "$devpath $1"
        fi
    done

    # raid md devices
    for devpath in /dev/md[0-9]*; do
        if grep -qw $(echo $devpath /proc/mdstat | sed -e 's|/dev/||g')
        then
            echo -ne "$devpath $1"
        fi
    done

    shopt -u nullglob
}

# Make sure only root can run our script
if (( $(id -u) != 0 )); then
echo "This script must be run as root." 1>&2
exit 1
fi

echo "Disk Devices"
finddisks

echo -e "\nUsable Devices"
fubdevs

echo -e "\n"

EDIT: Thanks chee for the great new function name fubdev!
EDIT 2: Update the script to exclude root blockdevice for partition disks (sda, sdb, etc...)
EDIT 3: Fixed issue where root blockdevice would not appear (sda, sdb, etc...)

Last edited by pyther (2011-02-20 05:07:18)

c00kiemon5ter · 2011-02-20 02:33:12

two links for you

the actual run
http://sprunge.us/NgJQ

some comments
http://sprunge.us/NACM

lvm partitions are on /dev/sda

knopwob · 2011-02-20 03:00:07

Disk Devices
/dev/mmcblk1 /dev/sda /dev/sdb 
Block Devices
/dev/mmcblk1 /dev/mmcblk1p1 /dev/sda /dev/sda1 /dev/sda2 /dev/sda3 /dev/sda4 /dev/sdb /dev/md127

I'm a bit confused about the /dev/md127. It could be a fragment of my previous installation in which i had set up a raid.

knopwob@thinky ~ % findmnt -s
TARGET           SOURCE                                    FSTYPE   OPTIONS
/dev/pts         devpts                                    devpts   defaults
/dev/shm         shm                                       tmpfs    nodev,nosuid
/                UUID=ac083663-697f-4ffe-8bb8-62bde6203b41 ext2     defaults,noatime
/boot            UUID=dc9ee236-ec73-4da3-9b5f-b5c5b7bfc440 ext2     defaults,noatime
swap             UUID=e2cb9fc7-a4c4-4a06-99ab-eab960a098a8 swap     defaults,noatime
/squashed/usr/ro /squashed/usr/usr.sfs                     squashfs loop,ro
/usr             usr                                       aufs     udba=reval,br:/squashed/usr/rw:/squashed/usr/ro

I removed some samba shares mounted with cifs from the listing of findmnt

c00kiemon5ter · 2011-02-20 03:32:36

testing with the updated script

http://sprunge.us/jDJG
http://sprunge.us/bMBa (listed is clearer )

all seems good here

falconindy · 2011-02-20 04:14:20

if grep -qw $(echo $devpath /proc/mdstat | sed -e 's|/dev/||g')

That can't possibly be what you meant to put in there... maybe this is what you meant?

if grep -qw "${devpath//\/dev\//}" /proc/mdstat

You probably want to look at lsblk in util-linux 2.19. Would probably make all this a whole lot simpler...

pyther · 2011-02-20 05:02:06

falconindy wrote:

if grep -qw $(echo $devpath /proc/mdstat | sed -e 's|/dev/||g')
That can't possibly be what you meant to put in there... maybe this is what you meant?
if grep -qw "${devpath//\/dev\//}" /proc/mdstat
You probably want to look at lsblk in util-linux 2.19. Would probably make all this a whole lot simpler...

That is a lot better, but would

${devpath##*/}

make more sense or does it not really matter?

Dieter@be · 2011-02-20 15:10:29

* fubdev is a really weird name.
why not name it find_usable_blockdev or something
*

# ./script.sh
Disk Devices
/dev/sda /dev/sdb 
Usable Devices
/dev/sda1 /dev/sda2 /dev/sdb /dev/mapper/cryptpool-home /dev/mapper/cryptpoolhost /dev/mapper/cryptpool-root /dev/mapper/cryptpool-swap 

# findmnt -s
TARGET                  SOURCE                                    FSTYPE OPTIONS
/dev/pts                devpts                                    devpts 
/dev/shm                shm                                       tmpfs  nodev,nosuid
/media/cd               /dev/cdrom                                auto   ro,unhide,user,noauto
/media/dvd              /dev/dvd                                  auto   ro,unhide,user,noauto
/media/fl               /dev/fd0                                  auto   user,noauto
/home                   /dev/mapper/cryptpool-home                ext3   
/                       /dev/mapper/cryptpool-root                ext3   
swap                    /dev/mapper/cryptpool-swap                swap   
/boot                   UUID=823383c8-94c9-42ee-9325-41a8b3d17a2d ext3   
/home/dieter/server-nfs server:/                                  nfs4   
/home/dieter/n900       /dev/disk/by-label/N900_MYDOCS            vfat   noauto,user

/dev/sdb is my empty card reader (not attached on a physical sata interface, but by a small flatcable)
it should only show up when not empty, I'll see if I can find a card somewhere to put into it.
I'm not sure what the cryptpoolhost is (it's not a VG), it points to /dev/dm-0
I think there must be a way to get the data we want. I'm a bit interigued by `fdisk -l`, which just checks /proc/partitions.

JokerBoy · 2011-02-21 04:27:11

Disk Devices
/dev/sda /dev/sdb 
Usable Devices
/dev/sda1 /dev/sda2 /dev/sda3 /dev/sda4 /dev/sda5 /dev/sdb1 /dev/sdb2

findmnt -s
TARGET            SOURCE    FSTYPE   OPTIONS
/dev/pts          devpts    devpts   defaults
/dev/shm          shm       tmpfs    nodev,nosuid
/boot             /dev/sda1 ext2     defaults,noatime
swap              /dev/sda2 swap     defaults
/                 /dev/sda3 ext4     defaults,noatime
/var              /dev/sda4 reiserfs defaults,noatime
/home             /dev/sda5 ext4     defaults,noatime
/media/Stuff      /dev/sdb1 ext4     defaults,noatime
/media/Multimedia /dev/sdb2 ext4     defaults,noatime
/tmp              tmpfs     tmpfs    defaults,nodev,nosuid,noatime,mode=1777
/var/lock         tmpfs     tmpfs    defaults,nodev,nosuid,noexec,noatime,mode=1777,size=10m
/var/run          tmpfs     tmpfs    defaults,nodev,nosuid,noexec,noatime,mode=0755,size=10m

anonymous_user · 2011-02-21 05:45:49

Disk Devices
/dev/sda /dev/sdb 
Usable Devices
/dev/sda1 /dev/sda5 /dev/sda6 /dev/sda7 /dev/sda8 /dev/sdb1 /dev/sdb2

TARGET   SOURCE    FSTYPE OPTIONS
/dev/pts devpts    devpts defaults
/dev/shm shm       tmpfs  nodev,nosuid
/boot    /dev/sda5 ext2   defaults
swap     /dev/sda6 swap   defaults
/        /dev/sda7 ext4   defaults
/home    /dev/sda8 ext4   defaults

Last edited by anonymous_user (2011-02-21 05:47:19)

dmz · 2011-02-21 05:57:08

> sh s.sh 
Disk Devices
/dev/sda /dev/sdb /dev/sdc /dev/sdd /dev/sde /dev/sdf /dev/sdg /dev/sdh /dev/sdi /dev/sdj /dev/sdk /dev/sdl /dev/sdm /dev/sdn /dev/sdo 
Usable Devices
/dev/sda1 /dev/sda2 /dev/sda3 /dev/sda4 /dev/sdb1 /dev/sdc1 /dev/sdd1 /dev/sde1 /dev/sdf1 /dev/sdg1 /dev/sdh1 /dev/sdi1 /dev/sdj1 /dev/sdk1 /dev/sdl1 /dev/sdm1 /dev/sdn /dev/sdo1

TARGET        SOURCE                                    FSTYPE OPTIONS
/dev/pts      none                                      devpts defaults
/dev/shm      none                                      tmpfs  defaults
/tmp          none                                      tmpfs  defaults
/             UUID=7b0fe8ef-1604-4420-b330-827cdd8f725e ext3   defaults
/boot         UUID=801613e9-86fd-42e7-80d0-42beda0519ef ext2   defaults
/mnt/Books    UUID=cd54ce5d-7f03-47e4-86e8-9764301e3c5f ext4   user,noatime,data=writeback
/mnt/Docs     UUID=d4a3faa4-742e-4332-8396-39a1f4b96f4d ext4   user,noatime,data=writeback
/mnt/Games_1  UUID=0ea2b7c1-1b87-466c-ac7c-790a22e5c466 ext4   user,noatime,data=writeback
/mnt/Games_2  UUID=2ee72fe5-5972-44a0-bae0-53efeeb69cae ext4   user,noatime
/mnt/Movies_1 UUID=31fce00e-016a-466e-9be3-2a7a394d9f05 ext4   user,noatime
/mnt/Movies_2 UUID=830ae517-f33e-4061-97b0-a9cc73cc246a ext4   user,noatime
/mnt/Movies_3 UUID=3be8e09a-cb58-4c16-8dfe-ccf691ae2844 btrfs  user,noacl
/mnt/Movies_4 UUID=244fe090-9377-489c-bb29-332b966a92e3 ext4   user,noatime
/mnt/TV_1     UUID=375dc6c2-e56a-4072-95c0-33ea19ed0b0a ext4   user,noatime
/mnt/TV_2     UUID=4c6804ce-10da-4622-8c99-eaaebdc66d05 ext4   user,noatime
/mnt/TV_3     UUID=a073de24-b627-491a-83c6-1ae10b0509b2 ext4   user,noatime
/mnt/Mvids    UUID=f7adb40f-02be-463f-8219-37e94a01c2a4 ext4   user,noatime
/mnt/Mvids_2  UUID=9dec18f9-cdf4-4d41-8915-ce5d7ced695d btrfs  user,noacl,compress
/mnt/Leftover UUID=44b06165-37c5-4f87-aa81-7d635ca77a79 xfs    user,defaults
/mnt/Music_1  sshfs#192.168.0.100:/mnt/Music_1          fuse   users,noauto,transform_symlinks,follow_symlinks,reconnect
/mnt/Music_2  sshfs#192.168.0.100:/mnt/Music_2          fuse   users,noauto,transform_symlinks,follow_symlinks,reconnect
/mnt/Music_3  sshfs#192.168.0.100:/mnt/Music_3          fuse   users,noauto,transform_symlinks,follow_symlinks,reconnect
/mnt/Music_4  sshfs#192.168.0.100:/mnt/Music_4          fuse   users,noauto,transform_symlinks,follow_symlinks,reconnect
/mnt/Music_5  sshfs#192.168.0.100:/mnt/Music_5          fuse   users,noauto,transform_symlinks,follow_symlinks,reconnect
/mnt/Music_6  sshfs#192.168.0.100:/mnt/Music_6          fuse   users,noauto,transform_symlinks,follow_symlinks,reconnect
/mnt/Music_7  sshfs#192.168.0.100:/mnt/Music_7          fuse   users,noauto,transform_symlinks,follow_symlinks,reconnect

kgas · 2011-02-21 14:12:39

here is the out put

Disk Devices
/dev/sda 
Usable Devices
/dev/sda1 /dev/sda2 /dev/sda3 /dev/sda5 /dev/sda6 /dev/sda7 /dev/sda8 

findmnt -s
TARGET     SOURCE      FSTYPE   OPTIONS
/dev/pts   devpts      devpts   defaults
/dev/shm   shm         tmpfs    nodev,nosuid
/boot      LABEL=boot  ext2     defaults
/home      LABEL=home  ext4     defaults
/mnt/Media LABEL=media ext4     defaults
/          LABEL=root  ext4     defaults
swap       LABEL=swap  swap     defaults
/tmp       LABEL=tmp   reiserfs defaults
/var       LABEL=var   reiserfs defaults

gaelic · 2011-02-26 12:04:27

Script Output:

Disk Devices
/dev/sda
Usable Devices
/dev/sda1 /dev/sda2 /dev/sda3 /dev/sda4

findmnt:

TARGET SOURCE FSTYPE OPTIONS
/dev/pts devpts devpts defaults
/dev/shm shm tmpfs nodev,nosuid
/boot /dev/sda1 ext2 defaults
swap /dev/sda2 swap defaults
/ /dev/sda3 btrfs defaults
/home /dev/sda4 btrfs subvol=home
/data /dev/sda4 btrfs subvol=data

--
Note: I've only one disk attached with 4 partitions, so it seems reasonable.

Arch Linux

#1 2011-02-20 02:20:35

Block Device Detection Code - Testers Needed

#2 2011-02-20 02:33:12

Re: Block Device Detection Code - Testers Needed

#3 2011-02-20 03:00:07

Re: Block Device Detection Code - Testers Needed

#4 2011-02-20 03:32:36

Re: Block Device Detection Code - Testers Needed

#5 2011-02-20 04:14:20

Re: Block Device Detection Code - Testers Needed

#6 2011-02-20 05:02:06

Re: Block Device Detection Code - Testers Needed

#7 2011-02-20 15:10:29

Re: Block Device Detection Code - Testers Needed

#8 2011-02-21 04:27:11

Re: Block Device Detection Code - Testers Needed

#9 2011-02-21 05:45:49

Re: Block Device Detection Code - Testers Needed

#10 2011-02-21 05:57:08

Re: Block Device Detection Code - Testers Needed

#11 2011-02-21 14:12:39

Re: Block Device Detection Code - Testers Needed

#12 2011-02-26 12:04:27

Re: Block Device Detection Code - Testers Needed

Board footer