Post your handy self made command line utilities

jasonwryan · 2016-02-09 01:10:45

Well, I don't know what you are trying to achieve with it; but if you just want the list of mounted devices from /dev/sd[b-z]:

awk -F/ '/sd[b-z]/ {print $3}' /proc/mounts

Trilby · 2016-02-09 02:22:43

awk can also number the lines. And awk can check for specific line numbers or count matches. So grep + awk + nl + grep + awk = awk:

lsblk -r | awk '/^sd[b-z][1-9]/ { if (++count == '$ans1') print $1; }'

ewaller · 2016-02-09 03:09:44

duyinthee wrote:

Its shame, I just know it does but not how to.
Please suggest me how that line should be.

Here at Arch, we are happy to point you in the right direction. Pattern matching is lesson 1 in awk.

Might I suggest you start here in the awk documentation?

duyinthee · 2016-02-09 05:10:23

thanks for pointing awk does pattern match.
this is what I need.

Trilby wrote:

awk can also number the lines. And awk can check for specific line numbers or count matches. So grep + awk + nl + grep + awk = awk:
lsblk -r | awk '/^sd[b-z][1-9]/ { if (++count == '$ans1') print $1; }'

and thanks for the link to awk doc.

Ambrevar · 2016-02-09 08:13:33

I'd also suggest having a look at the man page of the One True Implementation of AWK (nawk on Arch). Its clarity and conciseness are examplary and make it a great reference for POSIX AWK.

Alad · 2016-02-09 17:08:10

I've always found urgent workspaces in i3 easy to miss, so I wanted some way to get a notification when this happens. Right now, all I've come up with is a copy-pasted perl script and an ugly bash wrapper. Any suggestions are welcome.

#!/bin/bash
get_urgent() {
    i3-msg -t get_workspaces | \
	jshon -a -e urgent -u -p -e name -u -p -e focused | \
	xargs -L3 | awk '$1 ~ /true/ && $3 ~ /false/ {print $2}'
}

i3subscribe window | while read -r event; do
    if [[ $event == window:urgent ]]; then
	wname=($(get_urgent))

	if [[ ${wname[@]} ]]; then
	    notify-send -u critical -t 3500 "Urgent window on workspace: ${wname[@]}"
	fi
    fi
done

Last edited by Alad (2016-03-02 17:37:18)

x33a · 2016-02-10 05:02:43

@Alad, have you tried different colour combinations? For me the default red on black is sufficiently attention grabbing.

bleach · 2016-02-10 05:22:06

thank you Alad I like this and can learn from this.

Alad · 2016-02-10 11:07:31

x33a wrote:

@Alad, have you tried different colour combinations? For me the default red on black is sufficiently attention grabbing.

I'm using the tango theme (j4 themes), as it looks similar to the default but with slightly better contrast for the title bars. Maybe I'm simply forgetting the task bar is there at all ... I'll have to experiment more, cheers.

bleach wrote:

thank you Alad I like this and can learn from this.

No problem, though besides the right way™ of using a dedicated Perl/Python script, you can probably leave out xargs somehow ...

Last edited by Alad (2016-02-10 11:07:46)

shmibs · 2016-02-10 16:48:05

wrote a zsh function to list the counts and percentages of different file containers in my mpd library. it's kind of verbose, but functions.

should probably be generalised into a pretty output ranking script and a wrapper, so it could be used for types of files etc in general (for an image library, or things like that)

# list stats about mpd library file types, to motivate me
# to do better!
mpd-filetypes() {
	local pattern

	# if args exist, read them as file extensions in a pattern
	if 
	if [[ ! -z "$@" ]]; then
		pattern=$(echo "$@" | sed -e 's/\s/|/g' -e 's/\(.*\)/(\1)/')
	else
	# else hackily yoink recognised types from mpd --version
		pattern=$(mpd --version \
			| sed -n '1h; 1!H; ${g; s/.*Decoders plugins:\(.*\)Output.*/\1/g; p}' \
			| sed -e 's/.*\]//' | tr -d '\n' \
			| sed -e 's/^ //' -e 's/\s/|/g' -e 's/\(.*\)/(\1)/')
	fi

	# find file counts
	local i=1
	local total=0
	local totalstr='total'
	local counts=()
	local types=()
	find ~/music -type f -regextype posix-extended \
		-regex ".*\.$pattern" | grep -oE "\.$pattern$" \
		| sort | uniq -c | sed -e 's/^\s*\([0-9]* \)\./\1/' | \
	while read -A line; do
		counts[$i]=${line[1]}
		types[$i]=${line[2]}
		total=$(( $total + ${line[1]} ))
		i=$(( $i + 1 ))
	done

	# calculate percentages
	local twidth=0
	local cwidth=0
	local percentages=()
	for i in {1..${#types}}; do
		percentages[$i]="%$(calc -p \
			"a=${counts[$i]} * 100 / $total; round(a, 5-digits(a), 16)" \
			| tr -d '~')"

	done

	types=( 'type' "${types[@]}" 'total' )
	counts=( 'count' "${counts[@]}" $total )
	percentages=( 'percent' "${percentages[@]}" '       ')

	# calculate padding widths
	for i in {1..${#types}}; do
		if [[ ${#types[$i]} -gt $twidth ]]; then
			twidth=${#types[$i]}
		fi

		if [[ ${#counts[$i]} -gt $cwidth ]]; then
			cwidth=${#counts[$i]}
		fi
	done


	# print results
	local linelen=$(( $twidth + $cwidth + 13 ))
	local j
	echo -n "┌"
	for (( j=0; j < $linelen; j++ )); do
		echo -n "─"
	done
	echo "┐"

	for i in {1..${#types}}; do
		if [[ $i -eq 2 || $i -eq ${#types} ]]; then
			echo -n "├"
			for (( j=0; j < $linelen; j++ )); do
				echo -n "─"
			done
			echo "┤"
		fi

		if [[ $i -eq 1 ]]; then
			printf "│ %${twidth}s  " "${types[$i]}"
		else
			printf "│ %${twidth}s: " "${types[$i]}"
		fi

		printf "%${cwidth}s" "${counts[$i]}"

		if [[ $i -eq 1 || $i -eq ${#types} ]]; then
			printf "  %-7s │\n" "${percentages[$i]}"
		else
			printf ", %-7s │\n" "${percentages[$i]}"
		fi
	done

	echo -n "└"
	for (( j=0; j < $linelen; j++ )); do
		echo -n "─"
	done
	echo "┘"
}

example output:

┌[shmibs@lain ~]
└: mpd-filetypes 
┌───────────────────────┐
│  type  count  percent │
├───────────────────────┤
│  flac:     4, %0.0275 │
│   m4a:    31, %0.2128 │
│   mp3:  7157, %49.135 │
│   ogg:  7316, %50.227 │
│   wma:    58, %0.3982 │
├───────────────────────┤
│ total: 14566          │
└───────────────────────┘

┌[shmibs@lain ~]
└: mpd-filetypes ogg mp3 
┌───────────────────────┐
│  type  count  percent │
├───────────────────────┤
│   mp3:  7157, %49.451 │
│   ogg:  7316, %50.549 │
├───────────────────────┤
│ total: 14473          │
└───────────────────────┘

Trilby · 2016-02-10 17:01:59

A more generic, shorter, but less "pretty" version of the above:

find $YOUR_DIR -type f -exec file -b --mime-type '{}' \; | awk '// { count[$1]++; total++; } END {printf "type\tcount\tpercent\n"; for (ft in count) printf "%s\t%s\t%s\n", ft, count[ft], 100 * count[ft] / total
; }'

Or to apply some similar formatting still with just one find and one awk:

#!/bin/bash

find $1 -type f -exec file -b --mime-type '{}' \; | awk '// { count[$1]++; total++; }
END {
	printf "┌───────────────────────────────────────┐\n"
	printf "│ %-15s\t%-7s\t%-7s │\n", "type", "count", "percent"
	printf "├───────────────────────────────────────┤\n"
	for (ft in count)
	printf "│ %-15s\t%-7s\t%-7s │\n", ft ":", count[ft], 100 * count[ft] / total
	printf "├───────────────────────────────────────┤\n"
	printf "│ %-15s\t%-7s\t%-7s │\n", "total:", total, ""
	printf "└───────────────────────────────────────┘\n"
}'

easysid · 2016-02-14 16:52:52

A script I use to generate portable passwords. Based on ideas found on this thread. Requires xsel, or a similar clipboard tool.

#!/bin/bash

#
# Usage passd.sh <seed> <name>
# seed is an easy to remember phrase, and name is the site name or code
#

if [[ $# -ne 2 ]]; then
    echo 'Requires two arguments'
    exit 101
fi
len=20
xsel -c  # clear the clipboard
# generate password, and copy to clipboard
# just replace every occurance of space in $1 with $2
echo -n "$1" | sed "s/ /$2/g" | sha256sum | head -c $len | base64 | xsel -b
sleep 9 
echo "Hello World" | xsel -b    # flush

I use Lastpass with 2FA to manage the passwords, but I managed to get myself locked out of an account because I could not log into lastpass, and I did not know the randomly generated password. With this, the chances of that happening are slim, as I can always reproduce the password on a linux box (or even a web page).

Last edited by easysid (2016-03-05 15:20:46)

quequotion · 2016-02-27 17:21:59

Dump raw pulseaudio source to encoded stdout:
/usr/local/sbin/pulsecat

#!/bin/bash

# Source to record from (pactl list sources)
monitor="alsa_output.pci-0000_00_1b.0.iec958-ac3-surround-51.monitor"

#Output FLAC
#encoder="flac"
#encopts="--best -e --force-raw-format --endian=little --sign=signed --channels=6 --bps=16 --sample-rate=48000 - -c"
#chanmap="--channel-map=front-left,front-right,front-center,lfe,rear-left,rear-right"

#Output OGG
encoder="oggenc"
encopts="--raw --raw-bits=16 --raw-chan=6 --raw-rate=48000 --raw-endianness=0 -"
chanmap="--channel-map=front-left,front-center,front-right,rear-left,rear-right,lfe"

parec --fix-{format,rate} --channels=6 ${chanmap} -d "${monitor}" | "${encoder}" ${encopts}

OGG allows for another script to update the metadata (see ezstream manual):
/usr/local/sbin/get-metadata

#!/bin/bash

case "${1}" in
  artist)
    #Try MPRIS (playerctl) first, then ask to enter the metadata by hand (zenity)
    metadata="$(playerctl metadata artist 2>/dev/null)"
    [[ -z "${metadata}" ]] || [[ "${metadata}" == " " ]] && \
      metadata="$(zenity --timeout 15 --entry --text="Artist?" || printf "Unknown Artist")"
  ;;
  title)
    #Try MPRIS (playerctl) first, then ask to enter the metadata by hand (zenity)
    metadata="$(playerctl metadata title 2>/dev/null)"
    [[ -z "${metadata}" ]] || [[ "${metadata}" == " " ]] && \
      metadata="$(zenity --timeout 15 --entry --text="Title?" || printf "Unknown Title")"
  ;;
  ""|*)
    #Title of your stream
    metadata="Stream Title"
  ;;
esac

#Output requested metadata
exec printf "${metadata}"

Example ezstream (icecast) config (streaming from stdin):

  <ezstream>
     <url>http://127.0.0.1:8000/queradio.ogg</url>
     <sourceuser>source</sourceuser>
     <sourcepassword>supersecretsourcepw</sourcepassword>
     <format>VORBIS</format>
     <filename>stdin</filename>
     <metadata_progname>/usr/local/sbin/get-metadata</metadata_progname>
     <metadata_format>@s@: @a@ - @t@</metadata_format>
     <metadata_refreshinterval>60</metadata_refreshinterval>
  </ezstream>

Broadcast:

pulsecat | ezstream -c yourstationconfig.xml

In theory this works for any pulseaudio source, with appropriate encopts and chanmap.

This example broadcasts from the monitor of my main soundcard: a Realtek ALC892 configured for 5.1 surround with the ALSA a52 plugin. This allows me to broadcast from any and all applications that produce audio, eg an ordinary music player can broadcast and a tts engine can dj, any number of applications can broadcast their audio simultaneously, etc.

Last edited by quequotion (2018-06-01 10:23:59)

Kallestofeles · 2016-03-02 15:00:15

Hi all!
Just wanted to share a little helper I made for compiling kernels. As I use multiple kernels for testing, it's just a quick reference to ArchWiki's commands.
Put it in your ~/.bashrc if you're interested.

alias kernel_guide='echo Check that the kernel package is clean: ; echo -e "\e[92m$ make mrproper\e[0m" ; echo Copy the config from currently running kernel: ; echo -e "\e[92m$ zcat /proc/config.gz > .config\e[0m" ; echo Make the initial configuration for compilation: ; echo -e "\e[92m$ make menuconfig\e[0m" ; echo Compile the kernel: ; echo -e "\e[92m$ make\e[0m" ; echo Copy the compiled modules to /lib/modules/ dir: ; echo -e "\e[91m# make modules_install\e[0m" ; echo Copy the kernel to /boot directory: ; echo -e "\e[91m# cp -v arch/x86/boot/bzImage /boot/vmlinuz-YourKernelName\e[0m" ; echo Make initial RAM disk: ; echo -e "\e[91m# mkinitcpio -k FullKernelName -c /etc/mkinitcpio.conf -g /boot/initramfs-YourKernelName.img\e[0m" ; echo Finally, just update the GRUB: ; echo -e "\e[91m# update-grub\e[0m"'

As it is a simple alias, just enter kernel_guide to terminal or change the alias as you wish. The output should be coloured and look something like this:

[kalle@EELAKALLE ~]$ kernel_guide 
Check that the kernel package is clean:
$ make mrproper
Copy the config from currently running kernel:
$ zcat /proc/config.gz > .config
Make the initial configuration for compilation:
$ make menuconfig
Compile the kernel:
$ make
Copy the compiled modules to /lib/modules/ dir:
# make modules_install
Copy the kernel to /boot directory:
# cp -v arch/x86/boot/bzImage /boot/vmlinuz-YourKernelName
Make initial RAM disk:
# mkinitcpio -k FullKernelName -c /etc/mkinitcpio.conf -g /boot/initramfs-YourKernelName.img
Finally, just update the GRUB:
# update-grub

Have a great day!

Alad · 2016-03-02 16:04:16

http://unix.stackexchange.com/a/65819
http://www.in-ulm.de/~mascheck/various/echo+printf/

cat <<EOF
foo
bar
baz
EOF

Or you could, you know, create a text file...

Last edited by Alad (2016-03-02 16:04:30)

boban_dj · 2016-03-06 06:49:10

Just to keep some configs portable to another computer, I use this small script. It's just basically cp -a

#!/bin/bash

# The backup directory, BK is created automatically
# It tars (compresses) the BK folder, names it hostname-date.tar.gz
# You can edit the filepaths below, or make a file backup_files

# Usage ./copy_configs

touch ./backup_files
cat <<EOT >> backup_files
/etc/httpd/conf
/etc/makepkg.conf
/etc/pacman.d/mirrorlist
/etc/php/php.ini
/etc/mysql/my.cnf
/etc/ssh/sshd_config
/home/$USER/.config/xfce4
/home/$USER/.config/Thunar
/home/$USER/Scripts    
/home/$USER/.bash_profile
/home/$USER/.bashrc
/home/$USER/.dir_colors
/home/$USER/.dir_colors_256
/home/$USER/.i3
/home/$USER/.i3status.conf
/home/$USER/.nanorc
/home/$USER/Packages
/home/$USER/Packages.aur
/home/$USER/Scripts
/home/$USER/.xinitrc
/home/$USER/.Xresources
/home/$USER/.Xresources.d
/home/$USER/.config/geany/plugins
/home/$USER/.cache/sessions
/usr/share/mc/skins
/home/$USER/.config/mc
EOT

# List pacman and AUR installs for easy install
# sudo xargs -a Packages pacman -S --noconfirm --needed
pacman -Qqe | grep -vx "$(pacman -Qqm)" > /home/$USER/Packages
pacman -Qqm > /home/$USER/Packages.aur

for i in $(cat ./backup_files)
do
echo "Backing up" $i"..."
cp -a $i ./BK
done

echo "----------------"
echo "Done backing up."

# Make tar from backup files in BK
backup_files=./BK

# Destination of Backup.
dest="."

# Create archive filename.
day=$(date +%Y-%m-%d)
hostname=$(hostname -s)
archive_file="$hostname-$day.tar.gz"

# Print start status message.
echo "Backing up $backup_files to $dest/$archive_file"
date
echo

# Backup The Files using tar.
tar -zcvf $dest/BK/$archive_file $backup_files

# Print end status message.
echo
echo "Backup finished"
date

# cleanup
rm -r ./backup_files

# Long listing of files in $dest to check file sizes.
tree -L 1 ./BK

Last edited by boban_dj (2016-03-06 06:50:28)

Earnestly · 2016-03-06 13:01:14

Use rsync (or duplicity)

Last edited by Earnestly (2016-03-06 13:01:24)

boban_dj · 2016-03-08 06:39:30

Export google-chrome/chromium passwords to plain csv, browser must be closed.

Google Chrome

    sqlite3 -header -csv -separator "," ~/.config/google-chrome/Default/Login\ Data "SELECT * FROM logins" > ~/Passwords.csv

Chromium

    sqlite3 -header -csv -separator "," ~/.config/chromium/Default/Login\ Data "SELECT * FROM logins" > ~/Passwords.csv

Docbroke · 2016-03-08 10:03:23

I did some modification in script posted by Alad (post 2706)

#!/bin/bash

i3-msg -t get_workspaces | jshon -a -e urgent -u | while read -r event; do
    [[ $event == true ]] && {
	wname=( $(i3-msg -t get_workspaces | \
		       jshon -a -e urgent -u -p -e name -u -p -e focused | \
		       xargs -L3 | awk '$1 ~ /true/ && $3 ~ /false/ {print $2}') )
	
	(( ${#wname[@]} )) && {
	    notify-send -u critical -t 3500 "Urgent window on workspace: ${wname[@]}"
	}
    }
done

as opposed to script by Alad, this doesn't depend on i3subscribe perl-script and it's dependencies, however this script is oneshot so to keep getting notification I am running this with conky right now, by adding below to my existing conky configuration.

${exec ~/bin/urgent}

and as per my conky config this script keeps running every second, so that notification window will stay till urgent window is focused.

Last edited by Docbroke (2016-03-08 11:01:07)

Alad · 2016-03-08 16:58:53

Well, running jshon + i3msg every second shouldn't matter much. As an impatient Archer however, waiting a whole second for the first urgent notification is not my cup of tea.

Last edited by Alad (2016-03-08 17:00:04)

Docbroke · 2016-03-09 02:49:57

I think that can be solved by creating a timer service. However i am not sure which method is better on resources.

Earnestly · 2016-03-09 16:24:22

Nevermind, the issue here is how conky works in intervals.

Last edited by Earnestly (2016-03-09 16:25:10)

parchd · 2016-03-10 09:27:30

Got fed up of searching my history for the stream I'm after, so I made this script which I save as ~/.bin/radio - threw it together this morning so will probably tweak it in the future.
It would be nice if I could use mpd instead of mpv, but I have mpv installed for videos anyway so nevermind.

#!/bin/zsh

ctgry=$1

typeset -A streams
streams=(
        electroswing "http://groove.wavestreamer.com:7115/live"
        riprock "http://s9.viastreaming.net:7160/listen.pls"
        bbc1 "http://www.listenlive.eu/bbcradio1.m3u"
        bbc2 "http://www.listenlive.eu/bbcradio2.m3u"
        bbc4 "http://www.listenlive.eu/bbcradio4.m3u"
        bbc5 "http://www.listenlive.eu/bbc5live.m3u"
        bbc5e "http://www.listenlive.eu/bbc5liveextra.m3u"
        bbc6 "http://www.listenlive.eu/bbc6music.m3u"
        bbcwm "http://open.live.bbc.co.uk/mediaselector/5/select/version/2.0/mediaset/http-icy-mp3-a/vpid/bbc_wm/format/pls.pls"
)

stream=$streams[$ctgry]

if [[ $stream ]]; then
        mpv $stream
else
        echo "No such category"
fi

awesome8x · 2016-03-17 23:34:08

Here's a script I wrote to uninstall unneeded packages via pacman. It gets the output of pacman -Qdt, parses it, and removes the resulting packages. You have to run it a few times to get rid of all of them (because dependency hell), but I figured it's safer than a loop. Really, this is just helpful if you accidentally used pacman -R instead of -Rs or is -Rs tries to remove optional dependencies of other programs.

#!/bin/bash
STR=""
PAC="$(pacman -Qdt)"
while read -r line; do
	STR+=$(echo "$line" | cut -d' ' -f1)
	STR+=" "
done <<< "$PAC"
pacman -R $STR

Trilby · 2016-03-17 23:38:17

awesome8x, you're really reinventing the wheel there. First, you can do string manipulation on an array so the following would do the same as your entire loop:

STR=(${PAC[@]/ */})

But much more imporantly, you can just tell pacman to not output the version info:

PAC="$(pacman -Qdtq)"

So the whole thing becomes:

pacman -R $(pacman -Qdtq)

And there should be no need to run it multiple times. You say yourself the clutter is due to running pacman -R rather than -Rs, so why not do it right the first time:

pacman -Rsn $(pacman -Qdtq)

EDIT: my array example doesn't work right as is, but it doesn't matter as that is not the right way to do this anyways.

Arch Linux

#2701 2016-02-09 01:10:45

Re: Post your handy self made command line utilities

#2702 2016-02-09 02:22:43

Re: Post your handy self made command line utilities

#2703 2016-02-09 03:09:44

Re: Post your handy self made command line utilities

#2704 2016-02-09 05:10:23

Re: Post your handy self made command line utilities

#2705 2016-02-09 08:13:33

Re: Post your handy self made command line utilities

#2706 2016-02-09 17:08:10

Re: Post your handy self made command line utilities

#2707 2016-02-10 05:02:43

Re: Post your handy self made command line utilities

#2708 2016-02-10 05:22:06

Re: Post your handy self made command line utilities

#2709 2016-02-10 11:07:31

Re: Post your handy self made command line utilities

#2710 2016-02-10 16:48:05

Re: Post your handy self made command line utilities

#2711 2016-02-10 17:01:59

Re: Post your handy self made command line utilities

#2712 2016-02-14 16:52:52

Re: Post your handy self made command line utilities

#2713 2016-02-27 17:21:59

Re: Post your handy self made command line utilities

#2714 2016-03-02 15:00:15

Re: Post your handy self made command line utilities

#2715 2016-03-02 16:04:16

Re: Post your handy self made command line utilities

#2716 2016-03-06 06:49:10

Re: Post your handy self made command line utilities

#2717 2016-03-06 13:01:14

Re: Post your handy self made command line utilities

#2718 2016-03-08 06:39:30

Re: Post your handy self made command line utilities

#2719 2016-03-08 10:03:23

Re: Post your handy self made command line utilities

#2720 2016-03-08 16:58:53

Re: Post your handy self made command line utilities

#2721 2016-03-09 02:49:57

Re: Post your handy self made command line utilities

#2722 2016-03-09 16:24:22

Re: Post your handy self made command line utilities

#2723 2016-03-10 09:27:30

Re: Post your handy self made command line utilities

#2724 2016-03-17 23:34:08

Re: Post your handy self made command line utilities

#2725 2016-03-17 23:38:17

Re: Post your handy self made command line utilities

Board footer