You are not logged in.

#3701 2022-08-14 02:17:01

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 27,564
Website

Re: Post your handy self made command line utilities

If you haven't seen it before, this is a great sed tutorial.  And while it's not the only website that suggests an inverse correlation between the quality of a website's content and it's aesthetics, it's certainly a prime example.

Though I've learned sed just suites me well - it works like I think.

But in this context, this is even an abuse of sed.  The whole approach makes some pretty big assumptions about the consistency of format of the nat-geo html.  Ideally you should use an html or at least xml parser to extract tags/properties from the content.  Or you could likely use w3m / links to do much of this for you (and these would also take care of text encoding themselves too).

Last edited by Trilby (2022-08-14 02:29:21)


"UNIX is simple and coherent..." - Dennis Ritchie, "GNU's Not UNIX" -  Richard Stallman

Online

#3702 2022-08-21 14:40:09

schard
Member
From: Hannover
Registered: 2016-05-06
Posts: 1,557
Website

Re: Post your handy self made command line utilities

Got tired of manually performing the same steps on my AUR packages again and again:

0 ✓ rne@thinkpad ~/Projekte/aur/python-rcon $ cat /usr/local/bin/versionbump 
#! /bin/bash

test -n "$1" || exit 1
test -f PKGBUILD || exit 2

if [ -n "$2" ]; then
        RELEASE="$2"
else
        RELEASE="1"
fi

sed -i -e "s/pkgver=.*/pkgver=$1/" PKGBUILD
sed -i -e "s/pkgrel=.*/pkgrel=$RELEASE/" PKGBUILD
makepkg --printsrcinfo > .SRCINFO
git add PKGBUILD .SRCINFO
git commit -m "Update to $1-$RELEASE"
git push

Солідарність з Україною

Offline

#3703 2022-08-21 15:44:49

salonkommunist
Member
From: Germany
Registered: 2020-02-18
Posts: 34

Re: Post your handy self made command line utilities

This script takes two IPv4 addresses as arguments and returns their smallest common subnet, along with some additional information. The output is supposed to resemble that of ipcalc. Criticism and suggestions are most welcome.

#!/bin/bash

USAGE () {
    echo "Invalid argument(s)."
    echo "Note: This script takes exactly two IPv4 addresses as arguments."
    exit 1
}

[[ $# -ne 2 ]] && USAGE

VALIDATE_ARGUMENTS () {
    local VALID_PATTERN='^([0-9]+)\.([0-9]+)\.([0-9]+)\.([0-9]+)$'

    [[ $SPECIMEN =~ $VALID_PATTERN ]] || USAGE

    for OCTET in {1..4} ; do
        local OCTET_DEC
        OCTET_DEC=$(echo "$SPECIMEN" | cut -d '.' -f "$OCTET")
        [[ $OCTET_DEC -gt 255 || $OCTET_DEC -ne 0 && $OCTET_DEC =~ ^0 ]] && USAGE
    done
}

SPECIMEN="$1"
VALIDATE_ARGUMENTS "$SPECIMEN"
SPECIMEN="$2"
VALIDATE_ARGUMENTS "$SPECIMEN"

ADDRESS_A=$(echo -e "$1\n$2" | sort -n | head -n 1)
ADDRESS_B=$(echo -e "$1\n$2" | sort -n | tail -n 1)

OCTET_1A_DEC=$(echo "$ADDRESS_A" | cut -d '.' -f 1)
OCTET_1B_DEC=$(echo "$ADDRESS_A" | cut -d '.' -f 2)
OCTET_1C_DEC=$(echo "$ADDRESS_A" | cut -d '.' -f 3)
OCTET_1D_DEC=$(echo "$ADDRESS_A" | cut -d '.' -f 4)

OCTET_2A_DEC=$(echo "$ADDRESS_B" | cut -d '.' -f 1)
OCTET_2B_DEC=$(echo "$ADDRESS_B" | cut -d '.' -f 2)
OCTET_2C_DEC=$(echo "$ADDRESS_B" | cut -d '.' -f 3)
OCTET_2D_DEC=$(echo "$ADDRESS_B" | cut -d '.' -f 4)

D2B=({0..1}{0..1}{0..1}{0..1}{0..1}{0..1}{0..1}{0..1})

OCTET_1A_BIN=${D2B[$OCTET_1A_DEC]}
OCTET_1B_BIN=${D2B[$OCTET_1B_DEC]}
OCTET_1C_BIN=${D2B[$OCTET_1C_DEC]}
OCTET_1D_BIN=${D2B[$OCTET_1D_DEC]}

OCTET_2A_BIN=${D2B[$OCTET_2A_DEC]}
OCTET_2B_BIN=${D2B[$OCTET_2B_DEC]}
OCTET_2C_BIN=${D2B[$OCTET_2C_DEC]}
OCTET_2D_BIN=${D2B[$OCTET_2D_DEC]}

ADDRESS_A_BIN_DOTS="${OCTET_1A_BIN}.${OCTET_1B_BIN}.${OCTET_1C_BIN}.${OCTET_1D_BIN}"
ADDRESS_B_BIN_DOTS="${OCTET_2A_BIN}.${OCTET_2B_BIN}.${OCTET_2C_BIN}.${OCTET_2D_BIN}"
ADDRESS_A_BIN=$(echo "$ADDRESS_A_BIN_DOTS" | tr -d .)
ADDRESS_B_BIN=$(echo "$ADDRESS_B_BIN_DOTS" | tr -d .)

for BIT in {1..32} ; do
    CIDR=$((32-BIT))
    [ "${ADDRESS_A_BIN::-$BIT}" == "${ADDRESS_B_BIN::-$BIT}" ] && break
done

case $CIDR in
    0)  NETMASK=("0.0.0.0" "255.255.255.255" "4,294,967,294") ;;
    1)  NETMASK=("128.0.0.0" "127.255.255.255" "2,147,483,646") ;;
    2)  NETMASK=("192.0.0.0" "63.255.255.255" "1,073,741,822") ;;
    3)  NETMASK=("224.0.0.0" "31.255.255.255" "536,870,910") ;;
    4)  NETMASK=("240.0.0.0" "15.255.255.255" "268,435,454") ;;
    5)  NETMASK=("248.0.0.0" "7.255.255.255" "134,217,726") ;;
    6)  NETMASK=("252.0.0.0" "3.255.255.255" "67,108,862") ;;
    7)  NETMASK=("254.0.0.0" "1.255.255.255" "33,554,430") ;;
    8)  NETMASK=("255.0.0.0" "0.255.255.255" "16,777,214") ;;
    9)  NETMASK=("255.128.0.0" "0.127.255.255" "8,388,606") ;;
    10) NETMASK=("255.192.0.0" "0.63.255.255" "4,194,302") ;;
    11) NETMASK=("255.224.0.0" "0.31.255.255" "2,097,150") ;;
    12) NETMASK=("255.240.0.0" "0.15.255.255" "1,048,574") ;;
    13) NETMASK=("255.248.0.0" "0.7.255.255" "524,286") ;;
    14) NETMASK=("255.252.0.0" "0.3.255.255" "262,142") ;;
    15) NETMASK=("255.254.0.0" "0.1.255.255" "131,070") ;;
    16) NETMASK=("255.255.0.0" "0.0.255.255" "65,534") ;;
    17) NETMASK=("255.255.128.0" "0.0.127.255" "32,766") ;;
    18) NETMASK=("255.255.192.0" "0.0.63.255" "16,382") ;;
    19) NETMASK=("255.255.224.0" "0.0.31.255" "8,190") ;;
    20) NETMASK=("255.255.240.0" "0.0.15.255" "4,094") ;;
    21) NETMASK=("255.255.248.0" "0.0.7.255" "2,046") ;;
    22) NETMASK=("255.255.252.0" "0.0.3.255" "1,022") ;;
    23) NETMASK=("255.255.254.0" "0.0.1.255" "510") ;;
    24) NETMASK=("255.255.255.0" "0.0.0.255" "254") ;;
    25) NETMASK=("255.255.255.128" "0.0.0.127" "126") ;;
    26) NETMASK=("255.255.255.192" "0.0.0.63" "62") ;;
    27) NETMASK=("255.255.255.224" "0.0.0.31" "30") ;;
    28) NETMASK=("255.255.255.240" "0.0.0.15" "14") ;;
    29) NETMASK=("255.255.255.248" "0.0.0.7" "6") ;;
    30) NETMASK=("255.255.255.252" "0.0.0.3" "2") ;;
    31) CIDR=30
        NETMASK=("255.255.255.252" "0.0.0.3" "2") ;;
esac

NETMASK_OCTET_1=$(echo "${NETMASK[0]}" | cut -d '.' -f 1)
NETMASK_OCTET_2=$(echo "${NETMASK[0]}" | cut -d '.' -f 2)
NETMASK_OCTET_3=$(echo "${NETMASK[0]}" | cut -d '.' -f 3)
NETMASK_OCTET_4=$(echo "${NETMASK[0]}" | cut -d '.' -f 4)

NETMASK_OCTET_1_BIN=${D2B[$NETMASK_OCTET_1]}
NETMASK_OCTET_2_BIN=${D2B[$NETMASK_OCTET_2]}
NETMASK_OCTET_3_BIN=${D2B[$NETMASK_OCTET_3]}
NETMASK_OCTET_4_BIN=${D2B[$NETMASK_OCTET_4]}

NETMASK_BIN_DOTS="$NETMASK_OCTET_1_BIN.$NETMASK_OCTET_2_BIN.$NETMASK_OCTET_3_BIN.$NETMASK_OCTET_4_BIN"

WILDCARD_OCTET_1=$(echo "${NETMASK[1]}" | cut -d '.' -f 1)
WILDCARD_OCTET_2=$(echo "${NETMASK[1]}" | cut -d '.' -f 2)
WILDCARD_OCTET_3=$(echo "${NETMASK[1]}" | cut -d '.' -f 3)
WILDCARD_OCTET_4=$(echo "${NETMASK[1]}" | cut -d '.' -f 4)

WILDCARD_OCTET_1_BIN=${D2B[$WILDCARD_OCTET_1]}
WILDCARD_OCTET_2_BIN=${D2B[$WILDCARD_OCTET_2]}
WILDCARD_OCTET_3_BIN=${D2B[$WILDCARD_OCTET_3]}
WILDCARD_OCTET_4_BIN=${D2B[$WILDCARD_OCTET_4]}

WILDCARD_BIN_DOTS="$WILDCARD_OCTET_1_BIN.$WILDCARD_OCTET_2_BIN.$WILDCARD_OCTET_3_BIN.$WILDCARD_OCTET_4_BIN"

NETWORK_ADDRESS=$(printf "%d.%d.%d.%d" "$((OCTET_1A_DEC & NETMASK_OCTET_1))" "$((OCTET_1B_DEC & NETMASK_OCTET_2))" "$((OCTET_1C_DEC & NETMASK_OCTET_3))" "$((OCTET_1D_DEC & NETMASK_OCTET_4))")

BROADCAST_ADDRESS=$(printf "%d.%d.%d.%d" "$((OCTET_1A_DEC & NETMASK_OCTET_1 | 255-NETMASK_OCTET_1))" "$((OCTET_1B_DEC & NETMASK_OCTET_2 | 255-NETMASK_OCTET_2))" "$((OCTET_1C_DEC & NETMASK_OCTET_3 | 255-NETMASK_OCTET_3))" "$((OCTET_1D_DEC & NETMASK_OCTET_4 | 255-NETMASK_OCTET_4))")

NETWORK_OCTET_1=$(echo "$NETWORK_ADDRESS" | cut -d '.' -f 1)
NETWORK_OCTET_2=$(echo "$NETWORK_ADDRESS" | cut -d '.' -f 2)
NETWORK_OCTET_3=$(echo "$NETWORK_ADDRESS" | cut -d '.' -f 3)
NETWORK_OCTET_4=$(echo "$NETWORK_ADDRESS" | cut -d '.' -f 4)

NETWORK_OCTET_1_BIN=${D2B[$NETWORK_OCTET_1]}
NETWORK_OCTET_2_BIN=${D2B[$NETWORK_OCTET_2]}
NETWORK_OCTET_3_BIN=${D2B[$NETWORK_OCTET_3]}
NETWORK_OCTET_4_BIN=${D2B[$NETWORK_OCTET_4]}

NETWORK_BIN_DOTS="$NETWORK_OCTET_1_BIN.$NETWORK_OCTET_2_BIN.$NETWORK_OCTET_3_BIN.$NETWORK_OCTET_4_BIN"

BROADCAST_OCTET_1=$(echo "$BROADCAST_ADDRESS" | cut -d '.' -f 1)
BROADCAST_OCTET_2=$(echo "$BROADCAST_ADDRESS" | cut -d '.' -f 2)
BROADCAST_OCTET_3=$(echo "$BROADCAST_ADDRESS" | cut -d '.' -f 3)
BROADCAST_OCTET_4=$(echo "$BROADCAST_ADDRESS" | cut -d '.' -f 4)

BROADCAST_OCTET_1_BIN=${D2B[$BROADCAST_OCTET_1]}
BROADCAST_OCTET_2_BIN=${D2B[$BROADCAST_OCTET_2]}
BROADCAST_OCTET_3_BIN=${D2B[$BROADCAST_OCTET_3]}
BROADCAST_OCTET_4_BIN=${D2B[$BROADCAST_OCTET_4]}

BROADCAST_BIN_DOTS="$BROADCAST_OCTET_1_BIN.$BROADCAST_OCTET_2_BIN.$BROADCAST_OCTET_3_BIN.$BROADCAST_OCTET_4_BIN"

ADDRESS_A_CLASS="Public internet"
[[ $OCTET_1A_DEC -eq 0 ]] && ADDRESS_A_CLASS="IANA special use (RFC 791)"
[[ $OCTET_1A_DEC -eq 10 ]] && ADDRESS_A_CLASS="Private internet, Class A (RFC 1918)"
[[ $OCTET_1A_DEC -eq 100 && $OCTET_1B_DEC -ge 64 && $OCTET_1B_DEC -le 127 ]] && ADDRESS_A_CLASS="IANA special use (RFC 6598)"
[[ $OCTET_1A_DEC -eq 127 ]] && ADDRESS_A_CLASS="Loopback address (RFC 1122)"
[[ $OCTET_1A_DEC -eq 172 && $OCTET_1B_DEC -ge 16 && $OCTET_1B_DEC -le 31 ]] && ADDRESS_A_CLASS="Private internet, Class B (RFC 1918)"
[[ $OCTET_1A_DEC -eq 169 && $OCTET_1B_DEC -eq 254 ]] && ADDRESS_A_CLASS="Link local address (RFC 3927)"
[[ $OCTET_1A_DEC -eq 192 && $OCTET_1B_DEC -eq 0 && $OCTET_1C_DEC -eq 0 ]] && ADDRESS_A_CLASS="IANA special use (RFC 6890)"
[[ $OCTET_1A_DEC -eq 192 && $OCTET_1B_DEC -eq 168 ]] && ADDRESS_A_CLASS="Private internet, Class C (RFC 1918)"
[[ $OCTET_1A_DEC -eq 192 && $OCTET_1B_DEC -eq 0 && $OCTET_1C_DEC -eq 0 && $OCTET_1D_DEC -le 7 ]] && ADDRESS_A_CLASS="IANA special use (RFC 7335)"
[[ $OCTET_1A_DEC -eq 192 && $OCTET_1B_DEC -eq 0 && $OCTET_1C_DEC -eq 2 ]] && ADDRESS_A_CLASS="IANA special use (RFC 5737)"
[[ $OCTET_1A_DEC -eq 192 && $OCTET_1B_DEC -eq 31 && $OCTET_1C_DEC -eq 196 ]] && ADDRESS_A_CLASS="IANA special use (RFC 7535)"
[[ $OCTET_1A_DEC -eq 192 && $OCTET_1B_DEC -eq 52 && $OCTET_1C_DEC -eq 193 ]] && ADDRESS_A_CLASS="IANA special use (RFC 7450)"
[[ $OCTET_1A_DEC -eq 192 && $OCTET_1B_DEC -eq 88 && $OCTET_1C_DEC -eq 99 ]] && ADDRESS_A_CLASS="6to4 relay anycast (RFC 7450) [Deprecated]"
[[ $OCTET_1A_DEC -eq 192 && $OCTET_1B_DEC -eq 175 && $OCTET_1C_DEC -eq 48 ]] && ADDRESS_A_CLASS="IANA special use (RFC 7534)"
[[ $OCTET_1A_DEC -eq 198 && $OCTET_1B_DEC -ge 18 && $OCTET_1B_DEC -le 19 ]] && ADDRESS_A_CLASS="IANA special use (RFC 2544)"
[[ $OCTET_1A_DEC -eq 198 && $OCTET_1B_DEC -eq 51 && $OCTET_1C_DEC -eq 100 ]] && ADDRESS_A_CLASS="IANA special use (RFC 5737)"
[[ $OCTET_1A_DEC -eq 203 && $OCTET_1B_DEC -eq 0 && $OCTET_1C_DEC -eq 113 ]] && ADDRESS_A_CLASS="IANA special use (RFC 5737)"
[[ $OCTET_1A_DEC -ge 240 && $OCTET_1A_DEC -le 255 ]] && ADDRESS_A_CLASS="IANA reserved (RFC 1112)"
[[ $ADDRESS_A == "192.0.0.8" ]] && ADDRESS_A_CLASS="IANA special use (RFC 7600)"
[[ $ADDRESS_A == "192.0.0.9" ]] && ADDRESS_A_CLASS="IANA special use (RFC 7723)"
[[ $ADDRESS_A == "192.0.0.10" ]] && ADDRESS_A_CLASS="IANA special use (RFC 8155)"
[[ $ADDRESS_A == "192.0.0.170" ]] && ADDRESS_A_CLASS="IANA special use (RFC 8880)"
[[ $ADDRESS_A == "192.0.0.171" ]] && ADDRESS_A_CLASS="IANA special use (RFC 7050)"
[[ $ADDRESS_A == "255.255.255.255" ]] && ADDRESS_A_CLASS="Limited broadcast (RFC 919 / RFC 8190)"

ADDRESS_B_CLASS="Public internet"
[[ $OCTET_2A_DEC -eq 0 ]] && ADDRESS_B_CLASS="IANA special use (RFC 791)"
[[ $OCTET_2A_DEC -eq 10 ]] && ADDRESS_B_CLASS="Private internet, Class A (RFC 1918)"
[[ $OCTET_2A_DEC -eq 100 && $OCTET_2B_DEC -ge 64 && $OCTET_2B_DEC -le 127 ]] && ADDRESS_B_CLASS="IANA special use (RFC 6598)"
[[ $OCTET_2A_DEC -eq 127 ]] && ADDRESS_B_CLASS="Loopback address (RFC 1122)"
[[ $OCTET_2A_DEC -eq 172 && $OCTET_2B_DEC -ge 16 && $OCTET_2B_DEC -le 31 ]] && ADDRESS_B_CLASS="Private internet, Class B (RFC 1918)"
[[ $OCTET_2A_DEC -eq 169 && $OCTET_2B_DEC -eq 254 ]] && ADDRESS_B_CLASS="Link local address (RFC 3927)"
[[ $OCTET_2A_DEC -eq 192 && $OCTET_2B_DEC -eq 0 && $OCTET_2C_DEC -eq 0 ]] && ADDRESS_B_CLASS="IANA special use (RFC 6890)"
[[ $OCTET_2A_DEC -eq 192 && $OCTET_2B_DEC -eq 168 ]] && ADDRESS_B_CLASS="Private internet, Class C (RFC 1918)"
[[ $OCTET_2A_DEC -eq 192 && $OCTET_2B_DEC -eq 0 && $OCTET_2C_DEC -eq 0 && $OCTET_2D_DEC -le 7 ]] && ADDRESS_B_CLASS="IANA special use (RFC 7335)"
[[ $OCTET_2A_DEC -eq 192 && $OCTET_2B_DEC -eq 0 && $OCTET_2C_DEC -eq 2 ]] && ADDRESS_B_CLASS="IANA special use (RFC 5737)"
[[ $OCTET_2A_DEC -eq 192 && $OCTET_2B_DEC -eq 31 && $OCTET_2C_DEC -eq 196 ]] && ADDRESS_B_CLASS="IANA special use (RFC 7535)"
[[ $OCTET_2A_DEC -eq 192 && $OCTET_2B_DEC -eq 52 && $OCTET_2C_DEC -eq 193 ]] && ADDRESS_B_CLASS="IANA special use (RFC 7450)"
[[ $OCTET_2A_DEC -eq 192 && $OCTET_2B_DEC -eq 88 && $OCTET_2C_DEC -eq 99 ]] && ADDRESS_B_CLASS="6to4 relay anycast (RFC 7450) [Deprecated]"
[[ $OCTET_2A_DEC -eq 192 && $OCTET_2B_DEC -eq 175 && $OCTET_2C_DEC -eq 48 ]] && ADDRESS_B_CLASS="IANA special use (RFC 7534)"
[[ $OCTET_2A_DEC -eq 198 && $OCTET_2B_DEC -ge 18 && $OCTET_2B_DEC -le 19 ]] && ADDRESS_B_CLASS="IANA special use (RFC 2544)"
[[ $OCTET_2A_DEC -eq 198 && $OCTET_2B_DEC -eq 51 && $OCTET_2C_DEC -eq 100 ]] && ADDRESS_B_CLASS="IANA special use (RFC 5737)"
[[ $OCTET_2A_DEC -eq 203 && $OCTET_2B_DEC -eq 0 && $OCTET_2C_DEC -eq 113 ]] && ADDRESS_B_CLASS="IANA special use (RFC 5737)"
[[ $OCTET_2A_DEC -ge 240 && $OCTET_2A_DEC -le 255 ]] && ADDRESS_B_CLASS="IANA reserved (RFC 1112)"
[[ $ADDRESS_B == "192.0.0.8" ]] && ADDRESS_B_CLASS="IANA special use (RFC 7600)"
[[ $ADDRESS_B == "192.0.0.9" ]] && ADDRESS_B_CLASS="IANA special use (RFC 7723)"
[[ $ADDRESS_B == "192.0.0.10" ]] && ADDRESS_B_CLASS="IANA special use (RFC 8155)"
[[ $ADDRESS_B == "192.0.0.170" ]] && ADDRESS_B_CLASS="IANA special use (RFC 8880)"
[[ $ADDRESS_B == "192.0.0.171" ]] && ADDRESS_B_CLASS="IANA special use (RFC 7050)"
[[ $ADDRESS_B == "255.255.255.255" ]] && ADDRESS_B_CLASS="Limited broadcast (RFC 919 / RFC 8190)"

if [ "$1" == "$2" ] ; then
    echo -e '\n'
    printf "%12s   %-20s %38s\n" "Address:" "$ADDRESS_A" "$ADDRESS_A_BIN_DOTS"
    printf "%12s   %s\n" "" "--> $ADDRESS_A_CLASS"
    echo -e '\n'
    printf "%12s   %-20s %38s\n" "Netmask:" "255.255.255.255 = 32" "11111111.11111111.11111111.11111111"
    printf "%12s   %-20s %38s\n" "Wildcard:" "0.0.0.0" "00000000.00000000.00000000.00000000"
    printf "%12s   %-20s %38s\n" "Network:" "$1" "$NETWORK_BIN_DOTS"
    printf "%12s   %-20s\n" "Max hosts:" "1"
    exit
fi

echo -e '\n'
printf "%12s   %-20s %38s\n" "Address A:" "$ADDRESS_A" "$ADDRESS_A_BIN_DOTS"
printf "%12s   %s\n" "" "--> $ADDRESS_A_CLASS"
echo -e '\n'
printf "%12s   %-20s %38s\n" "Address B:" "$ADDRESS_B" "$ADDRESS_B_BIN_DOTS"
printf "%12s   %s\n" "" "--> $ADDRESS_B_CLASS"
echo -e '\n'
printf "%12s   %-20s %38s\n" "Netmask:" "${NETMASK[0]} = $CIDR" "$NETMASK_BIN_DOTS"
printf "%12s   %-20s %38s\n" "Wildcard:" "${NETMASK[1]}" "$WILDCARD_BIN_DOTS"
printf "%12s   %-20s %38s\n" "Network:" "$NETWORK_ADDRESS" "$NETWORK_BIN_DOTS"
printf "%12s   %-20s %38s\n" "Broadcast:" "$BROADCAST_ADDRESS" "$BROADCAST_BIN_DOTS"
printf "%12s   %-20s\n" "Max hosts:" "${NETMASK[2]}"

Edit: Renamed one variable more sensibly and heeded some shellcheck advice.

Last edited by salonkommunist (2022-08-22 16:55:04)

Offline

#3704 2022-08-29 23:27:16

karabaja4
Member
From: Croatia
Registered: 2008-09-14
Posts: 965

Re: Post your handy self made command line utilities

Need to play a sound in your bash script? Why not embed the sound as text in the script itself? smile

Offline

#3705 2022-09-18 09:41:30

Docbroke
Member
From: India
Registered: 2015-06-13
Posts: 1,376

Re: Post your handy self made command line utilities

Pacman wrapper to search & install / remove packages, using fzf.

To install

#!/bin/sh

if [[ -n "$1" ]]; then
    doas pacman -S "$@" && exit
else
echo -e '\e[1;37m[PACMAN] \e[1;32mInstall new packages (TAB to select, ENTER to install, PREVIEW-WINDOW: ?- toggle, shift+up/down- movement)\e[0m';\
# apk list | sed 's/-[0-9].*//' |\
pacman -Ssq |\
 fzf -e --multi --preview='pacman -Si {1}' --reverse --info=inline --height='80%' \
 --color='hl:148,hl+:154,pointer:032,marker:010,bg+:237,gutter:008' \
 --prompt='> ' --pointer='▶' --marker='✓' \
 --bind '?:toggle-preview' \
 --bind 'shift-up:preview-up' \
 --bind 'shift-down:preview-down' \
 --bind 'ctrl-a:select-all' |\
 xargs -ro doas pacman -S
fi

To remove

#!/bin/sh

if [[ -n "$1" ]]; then
    doas pacman -Rns $@ && exit
else
echo -e '\e[1;37m[PACMAN] \e[1;35mRemove packages (TAB to select, ENTER to install, PREVIEW_WINDOW: ?- toggle, shift+up/down- movement)\e[0m';\
pacman -Qq |\
 fzf -e --multi --preview='pacman -Qi {1}' --reverse --info=inline --height='80%' \
 --color='hl:148,hl+:154,pointer:032,marker:010,bg+:237,gutter:008' \
 --prompt='> ' --pointer='▶' --marker='✓' \
 --bind '?:toggle-preview' \
 --bind 'shift-up:preview-up' \
 --bind 'shift-down:preview-down' \
 --bind 'ctrl-a:select-all' |\
 xargs -ro doas pacman -Rns
fi

Note: original idea is already  in wiki, I have added some colors and keybindings.

Last edited by Docbroke (2022-09-18 19:15:17)


Arch is home!
cwm rofi weaver clifm vis lizzy pass terminator
https://github.com/Docbroke

Offline

#3706 2022-09-19 10:34:29

xerus
Member
Registered: 2021-05-11
Posts: 20

Re: Post your handy self made command line utilities

@docbroke made a similar thing in https://git.jfischer.org:444/xeruf/dotf … shell/arch called `yas`/`yar` using `yay`.

As for AUR maintanence helpers (@schard): https://git.jfischer.org:444/xeruf/dotf … ts/git-aur

My dotfiles have thousands of lines full of gems, I should really repackage parts of them...

Offline

#3707 2022-09-27 05:44:08

Alad
Wiki Admin/IRC Op
From: Bagelstan
Registered: 2014-05-04
Posts: 2,360
Website

Re: Post your handy self made command line utilities

As for AUR maintanence helpers (@schard):

Which runs:

find ... -print -exec sudo rm -rI {} +;;

Just run git clean -xf.

Last edited by Alad (2022-09-27 05:44:37)


Mods are just community members who have the occasionally necessary option to move threads around and edit posts. -- Trilby

Offline

#3708 2022-10-26 14:24:48

idoit
Member
Registered: 2013-06-15
Posts: 13

Re: Post your handy self made command line utilities

xerus wrote:

As for AUR maintanence helpers (@schard): https://git.jfischer.org:444/xeruf/dotf … ts/git-aur

This website is expired!

xerus wrote:

My dotfiles have thousands of lines full of gems, I should really repackage parts of them...

Please do!

Offline

#3709 2022-10-26 20:29:57

jaywk
Member
Registered: 2020-12-14
Posts: 5

Re: Post your handy self made command line utilities

Change directory in Krusader using fzf

Script:

#!/bin/bash

# prerequisites:  xdotool, fzf, fd

cd /
DIR=$(fd -t d -a | fzf --preview="tree -C -L 1 {}" --bind="space:toggle-preview")
[[ -z $DIR ]] && exit

WIN=$(xdotool search --limit 1 --onlyvisible --class Krusader)
xdotool windowactivate $WIN && xdotool key  --window $WIN ctrl+l
xdotool windowactivate $WIN && xdotool type --window $WIN $DIR
xdotool windowactivate $WIN && xdotool key  --window $WIN KP_Enter

exit

Define a UserAction in Krusader to call the script.

Last edited by jaywk (2022-10-27 15:52:05)

Offline

#3710 2022-11-02 21:53:08

schard
Member
From: Hannover
Registered: 2016-05-06
Posts: 1,557
Website

Re: Post your handy self made command line utilities

Inspired by this discussion:

#! /usr/bin/env python3
"""Encode arbitrary strings with ANSI background coloring."""

from argparse import ArgumentParser, Namespace
from sys import stdin, stdout


def get_args(description: str = __doc__) -> Namespace:
    """Parse command line arguments."""

    parser = ArgumentParser(description=description)
    parser.add_argument('-d', '--decode', action='store_true')
    return parser.parse_args()


def color_code(data: bytes, *, end: str = '\x1b[0m') -> str:
    """Color code the bytes."""

    return ''.join(
        f'\x1b[{int(octet) + 40}m ' for octet
        in reversed(oct(int.from_bytes(data, 'little'))[2:])
    ) + end


def decode_colors(text: str) -> bytes:
    """Decode a color code."""

    return (i := int(''.join(
        str(int(code) - 40) for code
        in reversed(text.replace('\x1b[', '').replace('m', '').split()[:-1])
    ), 8)).to_bytes(i.bit_length() // 8 + bool(i.bit_length() % 8), 'little')


def main() -> None:
    """Run the script."""

    args = get_args()

    if args.decode:
        for line in stdin:
            stdout.buffer.write(decode_colors(line))
            stdout.flush()
    else:
        print(color_code(stdin.buffer.read()))


if __name__ == '__main__':
    main()

Update
Now also available in Rust: https://github.com/conqp/color-code

Last edited by schard (2022-11-03 20:27:16)


Солідарність з Україною

Offline

#3711 2022-11-03 02:17:07

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 27,564
Website

Re: Post your handy self made command line utilities

schard wrote:

Inspired by this discussion...

Nice.  How about a hexdump / sed version that's as ugly as it is tiny:

#!/bin/sh

hexdump -bve '/1 "|%0.3o|\n"' | sed -n '
/|[0-9]\+|/ { s/|\(.\)\(.\)\(.\).*/^[[4\1m ^[[4\2m ^[[4\3m /;H; }
$ { g;s/\n//g;s/$/^[[0m/;p; }
'

If anyone knows how to get hexdump to not print offsets this could be simplified a fair bit.

EDIT: escape codes don't copy paste cleanly ... standby.  EDIT 2: I added their representation back in, but this cant just be copied and used as is.  The "^[" sequences need to be replaced by actual escapes (e.g., in vim type "ctrl-v esc" to insert these).  As an interesting irony, I could feed this script to itself and post a screenshot of the resulting image which would less ambiguously represent the actual content of the script with escape codes than does the text above!  New forum rule: never post an image of text ... unless it's an image of octal-color-coded blocks.

And also a C version:

#include <stdio.h>

int main(void) {
	int c;
	while ((c=getchar()) != EOF)
		printf("\033[4%hhum \033[4%hhum \033[4%hhum ", c & 7, (c>>3) & 7, (c>>6) & 7);
	printf("\033[0m\n");
	return 0;
}

Last edited by Trilby (2022-11-03 14:59:33)


"UNIX is simple and coherent..." - Dennis Ritchie, "GNU's Not UNIX" -  Richard Stallman

Online

#3712 2022-11-04 16:12:13

CarbonChauvinist
Member
Registered: 2012-06-16
Posts: 408
Website

Re: Post your handy self made command line utilities

Finally got around to trying to improve on my fumbling about in bash to download Nat Geo picture of the day.

Took Trilby's advice to use an actual parser, and did it in python which I'm currently learning:

#!/usr/bin/env python3

from bs4 import BeautifulSoup
from datetime import date
from pathlib import Path
from requests import get


def get_image_url(soup: BeautifulSoup) -> str:
    """Find image link tag and pull out url contained in "content" attribute"""

    image_tag = soup.find("meta", property="og:image")
    return image_tag["content"]


def get_image_name(url: str) -> str:
    """Split image url into list of strings by path seperator and return last element"""

    return url.split("/")[-1]


def get_description(soup: BeautifulSoup) -> str:
    """Find description tag and pull out description text in "content" attribute"""

    description_tag = soup.find("meta", attrs={"name": "description"})
    return description_tag["content"]


def newest_file(path: Path, pattern: str = "*.jpg") -> Path:
    """Return newest jpg in passed path, if no file match return passed path"""

    files = path.glob(pattern)
    return max(files, key=lambda x: x.stat().st_ctime, default=path)


BASE_PATH = Path("/home/ghost/Pictures/NGWallpapers/")
INDEX_FILE = "image-details.txt"
URL = "https://www.nationalgeographic.com/photo-of-the-day"

soup = BeautifulSoup(get(URL).text, "html.parser")
today = date.today()

image_url = get_image_url(soup)
remote_ngpotd = get_image_name(image_url)
newest_local_ngpotd = newest_file(BASE_PATH)
description_text = get_description(soup)

if newest_local_ngpotd is BASE_PATH or newest_local_ngpotd.name != remote_ngpotd:
    response = get(image_url)

    open(BASE_PATH.joinpath(remote_ngpotd), "wb").write(response.content)
    print(f"{remote_ngpotd} downloaded succesfully")

    f = open(BASE_PATH.joinpath(INDEX_FILE), "a")
    f.write(f"{today}: {description_text}\n")
    f.close()
else:
    print(f"{remote_ngpotd} already exists!")

Any feedback is certainly welcomed.

Last edited by CarbonChauvinist (2022-11-10 05:33:49)


"the wind-blown way, wanna win? don't play"

Offline

#3713 2022-11-09 05:36:38

bad3r
Member
Registered: 2021-08-17
Posts: 1

Re: Post your handy self made command line utilities

CarbonChauvinist wrote:

Finally got around to trying to improve on my fumbling about in bash to download Nat Geo picture of the day.

Took Trilby's advice to use an actual parser, and did it in python which I'm currently learning:

...
Any feedback is certainly welcomed.

Cool script thanks for sharing! I noticed that the script will fail if `BASE_PATH` did not contain any `.jpg` files. The issue is in the function `newest_file()`

def newest_file(path: Path, pattern: str = "*.jpg") -> Path:
    """Return newest jpg in passed path"""

    files = path.glob(pattern)
    return max(files, key=lambda x: x.stat().st_ctime)

The error:

❯ ./sss-Nat-Geo-POD    
Traceback (most recent call last):
  File "sss-Nat-Geo-POD", line 45, in <module>
    newest_local_ngpotd = newest_file(BASE_PATH)
  File "sss-Nat-Geo-POD", line 33, in newest_file
    return max(files, key=lambda x: x.stat().st_ctime)
ValueError: max() arg is an empty sequence

Offline

#3714 2022-11-09 13:50:19

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 27,564
Website

Re: Post your handy self made command line utilities

The fix would be to give a default argument to max():

   return max(files, key=lambda x: x.stat().st_ctime, default=None)

Note that "None" may not be a good default, perhaps a new-constructed empty Path would be better - but I don't have much experience with pathlib, e.g.:

   return max(files, key=lambda x: x.stat().st_ctime, default=Path())

Last edited by Trilby (2022-11-09 13:51:10)


"UNIX is simple and coherent..." - Dennis Ritchie, "GNU's Not UNIX" -  Richard Stallman

Online

#3715 2022-11-09 21:00:19

CarbonChauvinist
Member
Registered: 2012-06-16
Posts: 408
Website

Re: Post your handy self made command line utilities

Wow, thanks for feedback! Didn't really contemplate someone else actually using the script tbh. So I didn't make it that robust with exception handling etc. On my system the path would never not have a jpg, and on any new system that would only be true the first time running the script.

Trilby's suggestion I think is best -- use a default value in max (Python 3.4+ only, but this is Arch after all, so that's not a problem). I decided to just use the same path that was passed to the function and not newly constructed path though.

def newest_file(path: Path, pattern: str = "*.jpg") -> Path:
    """Return newest jpg in passed path"""

    files = path.glob(pattern)
    return max(files, key=lambda x: x.stat().st_ctime, default=path)

and then added as another test in the if clause:

if newest_local_ngpotd is BASE_PATH or newest_local_ngpotd.name != remote_ngpotd:

I've edited my earlier post to reflect these changes.


"the wind-blown way, wanna win? don't play"

Offline

#3716 2022-11-15 11:09:31

Alad
Wiki Admin/IRC Op
From: Bagelstan
Registered: 2014-05-04
Posts: 2,360
Website

Re: Post your handy self made command line utilities

AUR helpers are often mentioned, but how they resolve dependencies is usually left in the dark. Here's a basic example which, despite the bulk functionality being below 40 lines, handles split packages and nested dependencies. It also caches dependencies to reduce load on the AUR.

#!/usr/bin/python3
import json
import requests
import re
import logging
import sys

from urllib.parse import quote
from copy import copy


def aurjson(pkgs):
    endpoint = 'https://aur.archlinux.org/rpc/v5/info'
    payload  = {'arg[]': [quote(x) for x in pkgs] }
    response = requests.post(endpoint, data=payload)
    logging.debug(pkgs)

    if len(pkgs) == 0:
        return []
    if response.status_code == 200:
        response_dict = json.loads(response.text)
        
        if response_dict['type'] == 'error':
            logging.error(response_dict['error'])
            raise RuntimeError('response error')
        return response_dict['results']

    response.raise_for_status()


def depends(pkgs, types, max_req=30):
    depends = copy(pkgs)
    cache   = {}  # cache to avoid duplicate entries
    reqby   = {}  # reverse depends (parent nodes in dependency graph)

    for a in range(0, max_req):
        level = aurjson(depends)
        if not len(level):
            break  # no results
        depends.clear()

        for node in level:  # iterate over array of dicts
            cache[node['Name']] = copy(node)

            for dtype in types:
                if dtype not in node:
                    continue  # no dependency of this type

                for spec in node[dtype]:
                    # split versioned dependency
                    nver = re.split(r'<=|>=|<|=|>', spec)
                    dep  = nver[0]
                    ver  = nver[1] if len(nver) == 2 else None

                    # populate reverse depends
                    if dep in reqby:
                        reqby[dep].append(node['Name'])
                    else:
                        reqby[dep] = [node['Name']]

                    # check for cache hits
                    if dep in cache:
                        continue
                    depends.append(dep)

                    # mark as incomplete (dep retrieved in next step or repo package)
                    cache[dep] = None
        
    return cache, reqby


if __name__ == "__main__":
    logging.basicConfig(stream=sys.stderr, level=logging.ERROR)
    types = ['Depends', 'MakeDepends']  # modify to suit

    results, reqby = depends(sys.argv, types)

    # From here onwards, you can do anything with the results.
    # The below gives dependency <-> package pairs suitable for tsort(1)
    for dep in reqby:
        for pkg in reqby[dep]:
            #print(dep, pkg)  # to print everything
            if results[dep] is not None:
                #print(dep, pkg)  # to only print AUR targets
                print(results[dep]['PackageBase'], results[pkg]['PackageBase']) # as above, but AUR pkgbase instead of pkgname

The "requires by" dictionary is optional, but allows to e.g. filter out AUR dependencies that are already provided by some other repository package.

Side-note: AUR helpers tend to use much more complicated logic than the above, because they are trying to make things work with pacman -U. With pacman -S (local repositories), most dependency tasks can be left directly to pacman, and the above suffices.

Last edited by Alad (2022-11-15 11:16:57)


Mods are just community members who have the occasionally necessary option to move threads around and edit posts. -- Trilby

Offline

#3717 2022-11-18 12:17:32

schard
Member
From: Hannover
Registered: 2016-05-06
Posts: 1,557
Website

Re: Post your handy self made command line utilities

I needed to analyze some systemd journal entries.
So I wrote a script to search a journal JSON dump for keywords and count their occurrence.
Additionally the script can log the boots.
This is in order to use the resulting JSON file with canvas.js or the likes to plot the stuff on a web page:

#! /usr/bin/env python3
"""Parse journal timestamps for event plotting."""

from argparse import ArgumentParser, Namespace
from collections import defaultdict
from datetime import datetime
from functools import partial
from json import dumps, loads
from pathlib import Path
from typing import Iterable, Iterator


def iter_journal(filename: Path) -> Iterator[dict[str, str]]:
    """Yield journal entries as JSON objects."""

    with filename.open('r', encoding='utf-8') as file:
        for line in file:
            yield loads(line)


def count(
    keywords: Iterable[str],
    entries: Iterable[dict[str, str]],
    *,
    boots: bool = False,
    normalize: bool = False
) -> dict[str, dict]:
    """Count the entries for the respective keywords
    at the respective timestamps.

    If boots is True, also accumulate the boot timestamps.
    """

    events = defaultdict(partial(defaultdict, int))

    for entry in entries:
        if isinstance(message := entry.get('MESSAGE'), str):
            for keyword in keywords:
                if keyword in message:
                    events[keyword][
                        get_timestamp(entry, normalize=normalize).isoformat()
                    ] += 1

        if boots and (bootid := entry.get('_BOOT_ID')) not in events['boots']:
            events['boots'][bootid] = get_timestamp(
                entry, normalize=normalize
            ).isoformat()

    return events


def get_timestamp(
    entry: dict[str, str],
    *,
    normalize: bool = False
) -> datetime:
    """Parse a datetime timestamp from a journal entry."""

    timestamp = datetime.fromtimestamp(
        int(entry['__REALTIME_TIMESTAMP']) / 1000000
    )

    if normalize:
        return timestamp.replace(microsecond=0)

    return timestamp


def get_args(description: str = __doc__) -> Namespace:
    """Parse the command line arguments."""

    parser = ArgumentParser(description=description)
    parser.add_argument(
        'file', type=Path, help='systemd journal file in JSON format'
    )
    parser.add_argument(
        'keyword', nargs='*', help='keywords to filter for (REGEX)'
    )
    parser.add_argument(
        '-b', '--boots', action='store_true', help='add boots to plot'
    )
    parser.add_argument(
        '-i', '--indent', type=int, help='indentation for JSON output'
    )
    parser.add_argument(
        '-n', '--normalize', action='store_true',
        help='normalize timestamps to seconds'
    )
    return parser.parse_args()


def main() -> None:
    """Run the script."""

    args = get_args()

    result = count(
        args.keyword,
        iter_journal(args.file),
        boots=args.boots,
        normalize=args.normalize
    )
    print(dumps(result, indent=args.indent))


if __name__ == '__main__':
    main()

Last edited by schard (2022-11-18 12:21:47)


Солідарність з Україною

Offline

Board footer

Powered by FluxBB