Post your handy self made command line utilities

schard · 2020-10-08 08:12:00

Since we're using Arch Linux as the main driver for our digital signage systems in our company since ~2015 and are using it with different generations of configurations, especially bootloaders (GRUB, systemd-boot and Syslinux), I wrote a script to automagically detect the boot configuration and update the corresponding bootloaders.

#! /bin/bash

if test ${EUID} -ne 0; then
        echo "This script must be run as root." >&2
        exit 1
fi

DEV="/dev/sda"
LOADER="/boot/loader/loader.conf"

if test -d "/sys/firmware/efi"; then
        echo "System uses EFI boot."
        if test -f "${LOADER}"; then
                echo "EFI loader is systemd-boot."
                sed -ie 's/^default digital-signage$/default digital-signage\.conf/' "${LOADER}"
                bootctl update
        else
                echo "Unknown EFI loader." >&2
                exit 2
        fi
else
        echo "System uses BIOS/MBR boot."
        if test -b "${DEV}"; then
                BOOTLOADER="Syslinux"
                dd if="${DEV}" bs=512 count=1 2>/dev/null | grep -q GRUB && BOOTLOADER="GRUB"
                echo "Bootloader is ${BOOTLOADER}."

                if [ "${BOOTLOADER}" == "GRUB" ]; then
                        grub-install --target=i386-pc "${DEV}"
                        grub-mkconfig -o /boot/grub/grub.cfg
                else
                        syslinux-install_update -i -a -m
                fi
        else
                echo "Device ${DEV} does not exist." >&2
                exit 3
        fi
fi

Limitations
This script was written by me for our digital signage systems, of which I know, that
1) they either use systemd-boot in EFI mode with /boot as ESP or GRUB (legacy) resp. Syslinux (new) for BIOS/MBR boot
and
2) that the internal disk is guaranteed to be /dev/sda if present.

Last edited by schard (2020-10-08 08:22:01)

Awebb · 2020-10-08 20:37:20

Here could be my awesome bash script that makes my default changes to makepkg.conf on a new machine by looping over an associative bash array and sed -i'ing the whole thing, but then I accidentally read the man page and realized that $HOME/.makepkg.conf is a thing, as well as an XDG_CONFIG_HOME.

eschwartz · 2020-10-09 04:35:45

Awebb wrote:

but then I accidentally read the man page

For shame, you scoundrel!

Awebb · 2020-10-09 15:30:17

eschwartz wrote:

Awebb wrote:
but then I accidentally read the man page
For shame, you scoundrel!

I only read it, because I hoped there would be some sort of /etc/makepkg.conf.d where I could drop overrides.

karabaja4 · 2020-10-24 12:17:51

Switch between headset and speakers using ALSA and not messing around with config files or PulseAudio (may require some modifying depending on your system):

#!/bin/bash
set -uo pipefail

_usage() {
    echo "usage: ${0} [ <card_name> | list ]"
    exit 1
}

(( ${#} == 0 )) && _usage

_unmute_max_all() {
    for channel in $(amixer | grep -B1 "Capabilities:.*pvolume" | grep -oP "(?<=Simple mixer control ').+(?=')")
    do
        echo "Unmuting ${channel} to 100%"
        amixer set "${channel}" unmute &> /dev/null
        amixer set "${channel}" 100% &> /dev/null
    done
}

_write_asoundrc() {
    echo -e "defaults.ctl.card ${1}\ndefaults.pcm.card ${1}" > "${HOME}/.asoundrc"
}

_switch_card() {
    local -r search="$(grep -iwH "^${1}$" /proc/asound/card*/id)"
    if [[ -z ${search} ]]
    then
        echo "Card ${1} not found, exiting."
        exit 1
    fi
    local -r index="$(echo "${search%:*}" | grep -oP '(?<=/proc/asound/card)[0-9]+(?=/id)')"
    _write_asoundrc "${index}"
    echo "Switched to card ${search##*:} (${index})"
}

_list() {
    cat /proc/asound/card*/id
    exit 1
}

case "${1}" in
list)
    _list
    ;;
*)
    _switch_card "${1}"
    _unmute_max_all
    ;;
esac

EDIT:
v2: updated this a bit to not have duplicate code...
v3: check if card exists
v4: refactor

Last edited by karabaja4 (2021-05-13 01:55:00)

Trilby · 2020-10-24 12:59:52

cat /proc/asound/cards | grep PCH | head -n1 | cut -c2

1. useless use of cat
2. piping grep to head
3. piping head to cut
4. etc ...

I think you mean:

awk '/PCH/{print $1; exit;}' /proc/asound/cards

karabaja4 · 2020-10-24 13:12:49

Trilby wrote:

cat /proc/asound/cards | grep PCH | head -n1 | cut -c2
1. useless use of cat
2. piping grep to head
3. piping head to cut
4. etc ...
I think you mean:
awk '/PCH/{print $1; exit;}' /proc/asound/cards

Thanks, fixed.

ondoho · 2020-10-24 14:45:22

youtube-search

A python script that searches youtube for a given search term and extracts video URLs and metadata. Does not require youtube-dl. Torsocks wrapper included.
Get it here or here.

Last edited by ondoho (2020-10-25 18:49:18)

dcaspar · 2020-11-09 15:09:07

i recently made a couple of scripts to batch convert all audio files in a directory to AIFF or 320 MP3.
these are my first ever scripts so they are very simple and probably a bit ugly, but here they are:

#!/bin/bash
#Convert all audio files in a folder to aiff

for FILE in *
do
        notify-send "? Converting $FILE to aiff..."
        ffmpeg -v 16 -i "$FILE" "${FILE%.*}.aiff"
        rm -f "$FILE"
done
        notify-send "? Converting finished"

#!/bin/bash
#Convert all audio files in a folder to mp3

for FILE in *
do
        notify-send "? Converting $FILE to mp3(320)..."
        ffmpeg -v 16 -i "$FILE" -ab 320k "${FILE%.*}.mp3"
        rm -f "$FILE"
done
        notify-send "? Converting finished"

so far they seem to work well, but maybe a little inelegant (it would be nice to only run on detected audio formats for example)
if anyone has any other improvements i'd love to hear

Last edited by dcaspar (2020-11-09 15:15:42)

Awebb · 2020-11-09 15:16:57

dcaspar wrote:

i recently made a couple of scripts to batch convert all audio files in a directory to AIFF or 320 MP3.
these are my first ever scripts so they are very simple and probably a bit ugly, but here they are:
#!/bin/bash
#Convert all audio files in a folder to aiff

for FILE in *
do
        notify-send "? Converting $FILE to aiff..."
        ffmpeg -v 16 -i "$FILE" "${FILE%.*}.aiff"
        rm -f "$FILE"
done
        notify-send "? Converting finished"
#!/bin/bash
#Convert all audio files in a folder to mp3

for FILE in *
do
        notify-send "? Converting $FILE to mp3(320)..."
        ffmpeg -v 16 -i "$FILE" -ab 320k "${FILE%.*}.mp3"
        rm -f "$FILE"
done
        notify-send "? Converting finished"
so far they seem to work well, but maybe a little inelegant (it would be nice to only run on detected audio formats for example)
if anyone has any improvements i'd love to hear

That'll gonna bite off something juicy if you accidentally run it in the wrong folder. You might want to rm the file only if the ffmpeg line doesn't run into an error. Or imagine your disk is full at some point and the output files turn out to be slightly larger than the input files. I'd add some sanity check and then: https://tldp.org/LDP/abs/html/loopcontrol.html. Maybe find out how noisily ffmpeg fails (&& or || can be of service) or check for the mp3 to actually exist.

Awebb · 2020-11-09 15:28:58

Trilby wrote:

awk

There is something about awk that makes it absolutely unfun to learn. It's so convenient to think in little steps and pipe them into each other. I always end up with something that does the job and runs reasonably well and every time I feel like a series of pipes is a performance problem, I find myself in front of something with a tripple digit MHz CPU clock and the impression, that it shouldn't be a shell script in the first place. Maybe it's time to write a tool, that compresses/compiles common shell pipe constructs into awk. Now THAT would be a reason to get familiar with it.

a821 · 2020-11-09 15:39:23

dcaspar wrote:

i recently made a couple of scripts to batch convert all audio files in a directory to AIFF or 320 MP3.

you can use `parallel(1)` for the task, eg (adapted from the manpage)

parallel ffmpeg -v 16 -i {} -ab 320k {.}.mp3 ::: *.wav

Trilby · 2020-11-09 16:03:49

RE: awk vs pipelines:

Even without awk, that'd still just be grep + cut, not cat+grep+head+cut. In fact in the case of that script, just grep would do, there's no need even for cut if the resulting variable 'id' is passed to the next function unquoted. That way the number at the start of the line would be $1 in the called function as intended (effectively using the built in shell argument parsing to do the 'cut').

Last edited by Trilby (2020-11-09 16:04:32)

ugjka · 2020-11-09 19:17:14

Hacked on on some abandoned mDNS Golang library, so that I can advertise my Android phones (via Termux) on my LAN.

https://github.com/ugjka/mdns

It was fun

Last edited by ugjka (2020-11-09 19:17:43)

Awebb · 2020-11-10 04:17:25

ugjka wrote:

Hacked on on some abandoned mDNS Golang library, so that I can advertise my Android phones (via Termux) on my LAN.
https://github.com/ugjka/mdns
It was fun

I think telling my computer to "go get" something is my favourite part of the go language.

aralkis · 2020-11-21 00:36:05

#!/bin/python

from googleapiclient.discovery import build
from time import gmtime,strftime
import sys
import googleapiclient as gapi
import string
import re

def convertTimeArray(timeList):
    ans=0
    hourPattern = re.compile(r'(\d+)H')
    minPattern = re.compile(r'(\d+)M')
    secPattern = re.compile(r'(\d+)S')
    for time in timeList:
        hourStr = hourPattern.search(time)
        hour = int(hourStr[1]) if hourStr != None else 0
        minStr = minPattern.search(time)
        minute = int(minStr[1]) if minStr != None else 0
        secStr = secPattern.search(time)
        second = int(secStr[1]) if secStr != None else 0
        ans+=(3600 * hour + 60 * minute + second)
    return ans
def main():
    api_key = "<your google api key goes here as a str>"
    maxResult = 50
# That is the max of videos to be queried at once. 
# There is a way to go trough pages of videos, but I didn't implemented it yet.                               
    youtube = build('youtube', 'v3', developerKey=api_key)
    playlistID = str(sys.argv[1]) 
    pl_request = youtube.playlistItems().list(part="contentDetails", maxResults=maxResult, playlistId=playlistID)
    
    pl_response = pl_request.execute()
    
    vidIds = []
    for item in pl_response['items']:
        vidIds.append(item ['contentDetails']['videoId'])
    
    vid_request = youtube.videos().list(part="contentDetails", id=','.join(vidIds))
    vid_response = vid_request.execute()
    durations = []
    for item in vid_response['items']:
        durations.append(item['contentDetails']['duration'])
    time = strftime('%H:%M:%S', gmtime(convertTimeArray(durations)))
    print (time)
    return 0 

if __name__ == '__main__':
    main()

A python script to calculate the total time of the videos in a youtube playlist. It takes the playlist id (that can be taken from the url) as a console input. Handy for your scheduling of your online studies.

ondoho · 2020-11-21 07:59:00

^ It requires a Google API key.

Last edited by ondoho (2020-11-23 06:51:05)

Awebb · 2020-11-21 11:37:02

ondoho wrote:

^ requires googleapiclient, so presumably it requires a Google API key?

The code you read, knowing what you ask you will.

icar · 2020-11-21 13:18:44

In my laptop, I use this quite a lot. Small but handy. Pretty sure it can be done better, but works for me.

cpuctl() {
    if [[ "$1" == "performance" || "$1" == "ondemand" || "$1" == "powersave" || "$1" == "conservative" || "$1" == "schedutil" ]]; then
        echo "$1" | sudo tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor
    elif [[ "$1" == "status" ]]; then
        cat /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor
    else
        echo "Invalid argument. Valid ones:\n 'performance', 'ondemand', 'powersave', 'conservative', 'schedutil' or 'status'"
    fi
}

Also, the one I spam the most:

full-upgrade() {
    sudo echo -e ${PALETTE_UNDERLINE} Updating with repos and flatpak${PALETTE_RESET}
    echo -e  ${PALETTE_GREEN}:: ${PALETTE_RESET}Running pacmatic and yay... ${PALETTE_GREEN}:: ${PALETTE_RESET}
    sudo pacmatic -Syu; yay -Syu --aur
    echo -e ${PALETTE_GREEN}:: ${PALETTE_RESET}Checking Flatpak... ${PALETTE_GREEN}:: ${PALETTE_RESET}
    flatpak update
}

The PALETTE are just some colours defined.

Trilby · 2020-11-21 13:52:20

icar, as a first improvement, cpuctl could really use a case statement rather than those many if conditions:

case $1 in
   performance|ondemand|powersave...) echo $1 | sudo tee ... ;; #note fill in "..."
   status) cat /sys/... ;;
   *) echo "Invalid argument ..." ;;
esac

But then you might want to remove the hardcoded available governors as you can ask the system for these as they are provided in the the "scaling_available_governors" file in the same path.

teckk · 2020-11-21 16:09:41

You can get all the info that you want from youtube without an api key, import youtube-dl, and spit out the info you want.

Example:

#!/usr/bin/python

from youtube_dl import YoutubeDL
from datetime import datetime

def getUtube(url):
    ulist = []
    yt = YoutubeDL()
    #Extract info from page
    info = yt.extract_info(url, download=False)
    
    try:
        #Append items to list with labels
        ulist.extend(['', 'Uploader:', info['uploader'], ''])
        ulist.extend(['Upload Date:', info['upload_date'], ''])
        ulist.extend(['Title:', info['title'], ''])
        ulist.extend(['Description:', info['description'], ''])
        ulist.extend(['Youtube URL:', info['webpage_url'], ''])
    
        for i in info['formats']:
            ulist.extend(['Format ID:', i['format_id'], ''])
            ulist.extend(['Type:', i['format'], i['ext'], ''])
            ulist.extend(['Video URL or Manifest:', i['url'], ''])
            ulist.extend(['File Size:', str(i['filesize']), '----'])
            
    #On error
    except:
        print('\n', 'Something on youtube\'s page has changed.')
        print('\n', 'Look at utinfo.log and fix this script')
        #Dump of yt.extract_info to inspect
        with open('utinfo.log', 'a') as f:
            f.write(str(info))
            
    #Join list into separate line           
    utList  = '\n'.join(ulist)
    #Date/time
    now = str(datetime.now())
    
    #Print and append utList to logfile
    print(utList, '_'*70)
    with open('uts.log', 'a') as f:
        f.write(('{}\n'*3).format(now, utList, '_'*70))
    
#Main loop.
while True:
    #url = input('Enter/Paste Utube vid url: ')
    url = 'https://m.youtube.com/watch?v=_uQrJ0TkZlc'
    getUtube(url)

icar · 2020-11-21 16:22:11

Trilby wrote:

icar, as a first improvement, cpuctl could really use a case statement rather than those many if conditions:
case $1 in
   performance|ondemand|powersave...) echo $1 | sudo tee ... ;; #note fill in "..."
   status) cat /sys/... ;;
   *) echo "Invalid argument ..." ;;
esac
But then you might want to remove the hardcoded available governors as you can ask the system for these as they are provided in the the "scaling_available_governors" file in the same path.

Thanks for the suggestion. It seems easier to look at your implementation, I'll probably change it to a case statement.

About the "scaling_available_governors". It's weird. I had already tried it and it seems it doesn't show all governors available. I checked the ones that exist here and after trying them, I hardcoded them.

Trilby · 2020-11-21 16:38:04

In that case, and assuming you are using bash, you could put them in a variable for easier editing if/when needed:

#!/bin/bash

shopt -s extglob

opts='performance|ondemand|powersave|conservative|schedutil'

case $1 in
   @($opts)) echo "$1" | sudo tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor ;;
   status) cat /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor ;;
   *) echo -e "Invalid argument. Valid ones:\n$opts" | tr '|' ' ' ;;
esac

qinohe · 2020-11-21 18:27:58

@Trilby, unless I'm missing something;) since your using on array and case , why not use it 'full' right away?
BTW, shellcheck also spits an(useless?) error ')'

#!/bin/bash

shopt -s extglob

opts=(performance ondemand powersave conservative schedutil)

select opt in "${opts[@]}" exit; do
    case $opt in
      exit) exit;;
         *) printf "%s" "$opt" | sudo tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor; break;;
    esac
done

Trilby · 2020-11-21 19:29:20

I wasn't using an array. And yours is interesting, but it does something completely different by offering a menu rather than using the command line argument.

Arch Linux

#3401 2020-10-08 08:12:00

Re: Post your handy self made command line utilities

#3402 2020-10-08 20:37:20

Re: Post your handy self made command line utilities

#3403 2020-10-09 04:35:45

Re: Post your handy self made command line utilities

#3404 2020-10-09 15:30:17

Re: Post your handy self made command line utilities

#3405 2020-10-24 12:17:51

Re: Post your handy self made command line utilities

#3406 2020-10-24 12:59:52

Re: Post your handy self made command line utilities

#3407 2020-10-24 13:12:49

Re: Post your handy self made command line utilities

#3408 2020-10-24 14:45:22

Re: Post your handy self made command line utilities

#3409 2020-11-09 15:09:07

Re: Post your handy self made command line utilities

#3410 2020-11-09 15:16:57

Re: Post your handy self made command line utilities

#3411 2020-11-09 15:28:58

Re: Post your handy self made command line utilities

#3412 2020-11-09 15:39:23

Re: Post your handy self made command line utilities

#3413 2020-11-09 16:03:49

Re: Post your handy self made command line utilities

#3414 2020-11-09 19:17:14

Re: Post your handy self made command line utilities

#3415 2020-11-10 04:17:25

Re: Post your handy self made command line utilities

#3416 2020-11-21 00:36:05

Re: Post your handy self made command line utilities

#3417 2020-11-21 07:59:00

Re: Post your handy self made command line utilities

#3418 2020-11-21 11:37:02

Re: Post your handy self made command line utilities

#3419 2020-11-21 13:18:44

Re: Post your handy self made command line utilities

#3420 2020-11-21 13:52:20

Re: Post your handy self made command line utilities

#3421 2020-11-21 16:09:41

Re: Post your handy self made command line utilities

#3422 2020-11-21 16:22:11

Re: Post your handy self made command line utilities

#3423 2020-11-21 16:38:04

Re: Post your handy self made command line utilities

#3424 2020-11-21 18:27:58

Re: Post your handy self made command line utilities

#3425 2020-11-21 19:29:20

Re: Post your handy self made command line utilities

Board footer