Post your handy self made command line utilities

basica · 2016-09-12 12:36:36

To get sound via HDMI on my SOC, I need to apply a patch. Since I want to use the ABS, I automated the kernel downloading and patching. I could even add the makepkg -s at the end too I suppose. Until support is merged into the mainline kernel, I figure this'll come in handy whenever Arch updates the kernel.

#!/bin/bash
# linux kernel stuff
version="4.7.2"
major_release="linux-4.7.tar.xz"
linux_folder="linux-$version"
linux_file="$linux_folder.tar.xz"
linuxURL="https://cdn.kernel.org/pub/linux/kernel/v4.x/"

# the cherry trail patched kernel stuff
cherry_folder="sound-byt-cht-hdmi-v4.7"
cherry_file="byt-cht-hdmi-v4.7.tar.gz"
cherryURL="https://github.com/plbossart/sound/archive/"

# folders and misc settings
path="/home/username/src"
pkg_folder="core/linux"
downloader="wget" # or "curl -o" if you prefer..
config="config.x86_64"

# checks if a command exists and exits the script if it doesn't
cmdExists() {
    command -v "$1" >/dev/null 2>&1 || { printf >&2 "$1 required, but not installed..aborting..\n"; exit 1; }
}

# checks if the file is already downloaded and if not, downloads it if the downloader exists
download(){
    if [ ! -f "$2" ]; then
        cmdExists "$downloader"
        ${downloader} "$1$2"
    fi
}

# makes sure we can actually download the core/linux files
cmdExists "abs"

printf "downloading and extracting files..\n"
if [ ! -d "$path" ]; then
    printf "$path doesn't exist, creating it..\n'"
    mkdir -p "$path"
    if [ ! -d "$path" ]; then
        printf "issue creating $path..aborting..\n"
        exit 1
    fi
fi

cd "$path"

download "$linuxURL" "$linux_file"
tar -axf "$linux_file"

if [ ! -d "$cherry_folder" ]; then
    download "$cherryURL" "$cherry_file"
    tar -axf "$cherry_file"
fi

if [ ! -d "$linux_folder" ] || [ ! -d "$cherry_folder" ]; then
    printf "issue downloading/extracting files..aborting..\n"
    exit 1
fi

printf "producing patch..\n"
sed -i 's/inline//g' "$cherry_folder"/sound/hdmi_audio/intel_mid_hdmi_audio.h
diff -ENwbur {"$linux_folder","$cherry_folder"}/drivers/gpu/drm/i915 > cherry.patch
diff -ENwbur {"$linux_folder","$cherry_folder"}/sound >> cherry.patch

printf "downloading pkgfiles..\n"
ABSROOT=$path abs "$pkg_folder"
if [ -f "$major_release" ]; then
    cp "$major_release" "$pkg_folder"
fi
cp cherry.patch "$pkg_folder"
cd "$pkg_folder"

printf "patching files and updating checksums..\n"
echo "CONFIG_SUPPORT_HDMI=y" >> "$config"
sed -i "/'linux.preset'/a 'cherry.patch'" PKGBUILD
sed -i '/patch -p1 -i \"\${srcdir}\/patch-\${pkgver}\"/a patch -p1 -i \"\${srcdir}/cherry.patch\"' PKGBUILD
updpkgsums

printf "should be ready to run makepkg -s now..\n"

A few things I was trying out just to see if they'd work (namely the downloader part), but if there's better ways to do things I'd appreciate the feedback.

eschwartz · 2016-09-13 18:34:40

Why do you need to use a patch if all it takes is a line in sed? I didn't notice at first you are using a fork of an entire susbsystem, why don't you just copy it over instead of generating a patch to do the same thing?

Anyway, I'd probably just use asp to create a checkout of the svntogit ABS sources, and use git rebase to auto-add my custom changes. It is only slightly different from keeping a persistent clone of an AUR package, with custom changes committed on top.

git rebase is awesome.

Last edited by eschwartz (2016-09-13 18:41:46)

Fixxer · 2016-09-13 19:00:33

I hope it wasn't yet ... ;]

[root@arch ~]# cat /usr/local/bin/nohash
#!/bin/sh
grep -v "#" $1 | sed '/^\s*$/d'

r0b0t · 2016-09-14 04:18:12

#!/bin/bash
PATH=/bin:/usr/bin:/sbin:/usr/sbin
NTOPLOG=/var/log/ntopng/ntopmemcheck.log
MSGK=/var/log/messages
MPROC="$(ps auxf | sort -nr -k 4 | head -1 | awk '{print $2'})"
MONIT="$(ps auxf | sort -nr -k 4 | head -1 | awk '{print $11'} | xargs basename)"
echo "=========================================================================================" >> $NTOPLOG
date >> $NTOPLOG
ps auxf | sort -nr -k 4 | head -2 >> $NTOPLOG
echo "${MPROC}" | xargs lsof -p >> $NTOPLOG
monit status $MONIT >> $NTOPLOG
if grep -q oom-killer "$MSGK"; then
   cat /proc/"${MPROC}"/smaps >> $NTOPLOG
 fi
echo "=========================================================================================" >> $NTOPLOG
exit 0

Simple script to monitor ntop activity on a system that was causing RAM problems if I remember correctly (it seems by the script), if oom-killer is on than get some memory info about the process.

basica · 2016-09-14 06:21:04

Eschwartz wrote:

Why do you need to use a patch if all it takes is a line in sed? I didn't notice at first you are using a fork of an entire susbsystem, why don't you just copy it over instead of generating a patch to do the same thing?

Good question. I was initially patching a bunch of files from a subsystem, but when I realized it touched all the files, I changed it to the whole folder. Didn't click that copying it across would be easier once I made the change to the subsystem. I love complicating things

EschwartzAnyway wrote:

, I'd probably just use asp to create a checkout of the svntogit ABS sources, and use git rebase to auto-add my custom changes. It is only slightly different from keeping a persistent clone of an AUR package, with custom changes committed on top.
git rebase is awesome.

This sounds like a good idea, thanks. I'll look into it.

papajoke · 2016-09-14 13:47:41

some usage stats on my arch (only boot not login)

#!/usr/bin/env bash
# usage : boot-stats.sh [ yyyy-mm ]
param="$1" # format 2016-11
[ -z "$param" ] && param=$(date +"%Y-%m")

total=0
difference() {
    local diff=$(($2-$1))
    ((total+=diff/60))
}

declare -a dates
declare -a days

journalctl --list-boots | grep -Eo " [a-z]{3}\. ${param}.*" | sed -e 's/CEST—/CEST ;/' -e 's/CET—/CET ;/' -e 's/[a-z].//g'>/tmp/boot-stats
while IFS=';' read a b; do  
   #echo "-- $a --> $b"  
   j="${a:10:2}"
   ((j=${j#0}+0))
   days[$j]=$j
   difference "$(date -d "$a" +%s)" "$(date -d "$b" +%s)"
done < /tmp/boot-stats

nbdays="${#days[@]}"
((total/=60))
echo "$param"
echo "$((total)) hours"
((nbdays>0)) && echo "average $((total/${nbdays})) hours /day (${nbdays} days)"
echo "used ${#days[@]} days, this mouth: ${days[*]}"
rm /tmp/boot-stats &>/dev/null

Last edited by papajoke (2016-09-14 14:30:10)

ackalker · 2016-09-15 15:15:04

I use this little scriptlet to compare boot logs before and after a kernel upgrade.

Usage: bootdiff boot#1 boot#2

where boot#1 and boot#2 are boot numbers (-1 for previous boot, 0 for current boot). See `-b` option in `man journalctl`

The script uses `diff -u` by default for diff output, but you can specify a different diffing tool (such as `meld` (*)) and options by setting an environment variable `DIFF="..."`.
A snippet of sample output for today's kernel upgrade:

$ DIFF="diff -U0" bootdiff -1 0 | grep "kernel: "
-localhost kernel: Linux version 4.7.2-1-ARCH (builduser@tobias) (gcc version 6.1.1 20160802 (GCC) ) #1 SMP PREEMPT Sat Aug 20 23:02:56 CEST 2016
+localhost kernel: Linux version 4.7.3-2-ARCH (builduser@tobias) (gcc version 6.2.1 20160830 (GCC) ) #1 SMP PREEMPT Thu Sep 8 09:44:02 CEST 2016
-localhost kernel: RAMDISK: [mem 0x7f90c000-0x7fffffff]
+localhost kernel: RAMDISK: [mem 0x7f90b000-0x7fffffff]
-localhost kernel: Memory: 8102364K/8319672K available (6028K kernel code, 977K rwdata, 1836K rodata, 1244K init, 1164K bss, 217308K reserved, 0K cma-reserved)
+localhost kernel: Memory: 8102360K/8319672K available (6000K kernel code, 978K rwdata, 1888K rodata, 1244K init, 1164K bss, 217312K reserved, 0K cma-reserved)
-localhost kernel: tsc: Detected 3600.281 MHz processor
-localhost kernel: Calibrating delay loop (skipped), value calculated using timer frequency.. 7203.58 BogoMIPS (lpj=12000936)
+localhost kernel: spurious 8259A interrupt: IRQ7.
+localhost kernel: tsc: Detected 3600.122 MHz processor
+localhost kernel: Calibrating delay loop (skipped), value calculated using timer frequency.. 7203.25 BogoMIPS (lpj=12000406)
-localhost kernel: ftrace: allocating 23806 entries in 93 pages
+localhost kernel: ftrace: allocating 23847 entries in 94 pages
-localhost kernel: smpboot: Total of 8 processors activated (57635.43 BogoMIPS)
+localhost kernel: smpboot: Total of 8 processors activated (57633.81 BogoMIPS)
-localhost kernel: RTC time: 14:27:27, date: 09/15/16
+localhost kernel: RTC time: 14:31:06, date: 09/15/16
-localhost kernel: ACPI: SSDT 0xFFFF880213D51800 0005AA (v02 PmRef  ApIst    00003000 INTL 20051117)
+localhost kernel: ACPI: SSDT 0xFFFF880213DB2000 0005AA (v02 PmRef  ApIst    00003000 INTL 20051117)
-localhost kernel: ACPI: SSDT 0xFFFF880213925400 000119 (v02 PmRef  ApCst    00003000 INTL 20051117)
+localhost kernel: ACPI: SSDT 0xFFFF880213939200 000119 (v02 PmRef  ApCst    00003000 INTL 20051117)
-localhost kernel: Freeing initrd memory: 7120K (ffff88007f90c000 - ffff880080000000)
+localhost kernel: Freeing initrd memory: 7124K (ffff88007f90b000 - ffff880080000000)
-localhost kernel: Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
+localhost kernel: Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[...]

Yup, we booted the new kernel, the initrd has gotten a bit bigger, there's a wee bit less memory available, and the serial driver now has IRQ sharing enabled. Interesting.

bootdiff:

#!/bin/bash

# Strip timestamp from output of `journalctl`
# Preferred over `journalctl -o cat` because that also strips
# event source such as "kernel: "
journalctl_no_ts() {
    journalctl "$@" | cut -d' ' -f 4-
}

[[ $# -ne 2 ]] && echo "Usage: $(basename "$0") boot#1 boot#2" && exit 1

${DIFF:-diff -u} <(journalctl_no_ts -b $1) <(journalctl_no_ts -b $2)

(*) Visual diffing tools like `meld` are useful because log messages can shift around a little due to asynchronous initialization.

Last edited by ackalker (2016-09-15 17:51:25)

Sachiko · 2016-09-16 06:02:08

This script I kinda have to share for the Debian/Ubuntu users who recently moved to Arch and use the AUR via pacaur.
No, it isn't mine completely, just the modifications are. The base belongs wholly to arcetera. I found the aptpac wrapper on the AUR and added a sane help input so it doesn't spam you with the help when you forget to put the proper parameters in and I switched from using pacman to using pacaur to add AUR support.

#!/bin/bash
#
#               __                             
#              /\ \__                          
#   __    _____\ \ ,_\ _____     __      ___   
# /'__`\ /\ '__`\ \ \//\ '__`\ /'__`\   /'___\ 
#/\ \L\.\\ \ \L\ \ \ \\ \ \L\ /\ \L\.\_/\ \__/ 
#\ \__/.\_\ \ ,__/\ \__\ \ ,__\ \__/.\_\ \____\
# \/__/\/_/\ \ \/  \/__/\ \ \/ \/__/\/_/\/____/
#           \ \_\        \ \_\                 
#            \/_/         \/_/    
# a pacaur wrapper with syntax based on debian's apt
# (c) arcetera 2015 - wtfpl
# Modified by Sachiko to add AUR support and sane help handling

SYNTAX=$1
INPUT=$2

if [ "$SYNTAX" == "install" ]
then
        pacaur -S $INPUT
elif [ "$SYNTAX" == "search" ]
then
        pacaur -Ss $INPUT
elif [ "$SYNTAX" == "remove" ]
then
        pacaur -Rs $INPUT
elif [ "$SYNTAX" == "upgrade" ]
then
        pacaur -Syu
elif [ "$SYNTAX" == "update" ]
then
        pacaur -Sy
        echo "run aptpac upgrade *immediately*. pacaur does not support partial upgrades. running merely 'upgrade' would suffice. failure to do this could result in a broken installation."
elif [ "$SYNTAX" == "download" ]
then
        pacaur -Sw $INPUT
elif [ "$SYNTAX" == "autoremove" ]
then
        pacaur -Qdtq | pacaur -Rs -
elif [ "$SYNTAX" == "show" ]
then
        pacaur -Qi $INPUT
elif [ "$SYNTAX" == "clean" ]
then
        pacaur -Sc
elif [ "$SYNTAX" == "autoclean" ]
then
        pacaur -Sc
elif [ "$SYNTAX" == "policy" ]
then
        less /etc/pacman.d/mirrorlist
elif [ "$SYNTAX" == "list" ]
then
        pacaur -Q
elif [ "$SYNTAX" == "listmore" ]
then
        pacaur -Qi
elif [ "$SYNTAX" == "listless" ]
then
        pacaur -Q | wc -l
elif [ "$SYNTAX" == "build" ]
then
        makepkg -sri
elif [ "$SYNTAX" == "help " ]
then
        echo "aptpac: a pacaur wrapper with apt syntax"
        echo "help - print this help"
        echo "install - installs a package"
        echo "search - searches for a package in the repos"
        echo "remove - removes a package"
        echo "upgrade - upgrades the system fully, refreshing repos and upgrading packages"
        echo "update - only refreshes the repos (bad practice, do not run this without running 'upgrade' immediately after"
        echo "download - only download a package into pacman's cache without installing it"
        echo "autoremove - remove dependencies that are no longer needed (usually should not be needed as 'remove' should remove dependencies along with the package)"
        echo "show - shows information about the package"
        echo "clean/autoclean - clears pacman's cache"
        echo "policy - prints mirrorlist"
        echo "list - lists all installed packages"
        echo "listmore - lists all installed packages with all info"
        echo "listless - lists how many packages are installed"
        echo "build - builds package from PKGBUILD"
else
        echo "Insufficent parameters. Use 'aptpac help' for more info"
fi

And because I felt the if statement spam was a bit much...

I took the same concept, used cases and added proper cache flushing.

#!/bin/bash
#                                /$$   /$$                     /$$          
#                               |__/  | $$                    | $$          
#   /$$$$$$   /$$$$$$   /$$$$$$$ /$$ /$$$$$$   /$$   /$$  /$$$$$$$  /$$$$$$ 
#  /$$__  $$ |____  $$ /$$_____/| $$|_  $$_/  | $$  | $$ /$$__  $$ /$$__  $$
# | $$  \ $$  /$$$$$$$| $$      | $$  | $$    | $$  | $$| $$  | $$| $$$$$$$$
# | $$  | $$ /$$__  $$| $$      | $$  | $$ /$$| $$  | $$| $$  | $$| $$_____/
# | $$$$$$$/|  $$$$$$$|  $$$$$$$| $$  |  $$$$/|  $$$$$$/|  $$$$$$$|  $$$$$$$
# | $$____/  \_______/ \_______/|__/   \___/   \______/  \_______/ \_______/
# | $$                                                                      
# | $$                                                                      
# |__/                                                                      
#
#
# A pacman and pacaur wrapper with apt inspired syntax


SYNTAX=$1
INPUT=$2

case "$SYNTAX" in
        install)
                pacaur -S $INPUT
                ;;
        search)
                pacaur -Ss $INPUT
                ;;
        remove)
                pacaur -Rs $INPUT
                ;;
        upgrade)
                pacaur -Su
                ;;
        update)
                pacaur -Sy
                ;;
        download)
                pacaur -Sw $INPUT
                ;;
        autoremove)
                pacaur-Qdtq | pacaur -Rs -
                ;;
        show)
                pacaur -Qi $INPUT
                ;;
        clean)
                printf "%s\n" $"cleaning pacman cache..."
                sleep 1s
                pacaur -Sc
                printf "%s\n" $"cleaning pacaur cache..."
                sleep 1s
                rm -rf $HOME/.cache/pacaur/*
                ;;
        policy)
                less /etc/pacman.d/mirrorlist
                ;;
        list)
                pacaur -Q
                ;;
        listm)
                pacaur -Qi
                ;;
        listl)
                pacaur -Q | wc -l
                ;;
        build)
                makepkg -sri
                ;;
        help)
                cat <<-EOF
	            pacitude - a pacaur wrapper with apt inspired syntax
                          help - print this help
                          install <package> - installs a package
                          search <package> - searches for a package in the repos
                          remove <package> - removes a package
                          upgrade - upgrades the system
                          update - only refreshes the repos (bad practice, do not run this without running 'upgrade' immediately after
                          download <package> - only download a package into pacman's cache without installing it
                          autoremove - remove dependencies that are no longer needed (usually should not be needed as 'remove' should remove dependencies along with the package)
                          show <package> - shows information about the package
                          clean - clears pacman and pacaur's caches
                          policy - prints mirrorlist
                          list - lists all installed packages
                          listm - lists all installed packages with all info
                          listl - lists how many packages are installed
                          build - builds package from PKGBUILD
EOF
                ;;
        *)
                printf "%s\n" $"Insufficient paramaters. Use 'pacitude help' for more info"
                ;;
esac

Of course, I had to come up with a new name for the myriad of changes that I added.

EDIT: For those who want pacitude to work like aptpac, I have made a PKGBUILD for anyone to use (Which it is now in the AUR).

EDIT2: Prior to releasing to the AUR, I did change to a cat EOF construct like Alad suggested.

Last edited by Sachiko (2016-09-16 20:23:11)

Alad · 2016-09-16 12:07:12

pacaur -Sy

sleep 1s

?

echo ...
echo ...
....
echo

cat is a thing:

https://wiki.archlinux.org/index.php/Core_utilities#cat

As to the general idea... blurring the lines between AUR and repos is a bad idea, especially for new users who aren't familiar with pacman yet.

Last edited by Alad (2016-09-16 12:13:30)

Sachiko · 2016-09-16 19:28:06

Alad wrote:

pacaur -Sy

This is to keep it in tandem with how a former Debian/Ubuntu user expects the package manager to run. You update the repos and then you upgrade the packages(which is explicitly stated in the help section.)

sleep 1s

?

This is to make it seem like the script is contemplating something for a decision. More aesthetic than practical.

echo ...
echo ...
....
echo
cat is a thing:
https://wiki.archlinux.org/index.php/Core_utilities#cat

I did look into the printf portion and changed it accordingly. HOWEVER, I elected to put each line in it's own printf so people can enable and disable each section based on their personal preferences(same with the cases)

As to the general idea... blurring the lines between AUR and repos is a bad idea, especially for new users who aren't familiar with pacman yet.

While it isn't a good idea for new users, I really geared the creation of my flavor of aptpac(Which, as you saw above, I elected to call pacitude) to the somewhat experienced Arch users who miss the blatant way of telling the package manager how to do things instead of memorizing flags and what they do while keeping the base functionality of the package manager and adding in, imho, a crucial part of the Arch experience(using the AUR)

Alad · 2016-09-16 19:48:51

New users who wish to retain the syntax of their old distribution, and experienced users who don't mind blurring the lines between AUR and repos, seem like two worlds to me. Assuming former, there's certain habits you won't break by bolting on a note in the help text; like doing an "apt-get update" alone when I get a 404 on installing.

Regarding printf, it's no harder to delete unwanted lines from a cat/EOF construct than it is to delete a printf line, though former saves a lot of boiler plate.

Trilby · 2016-09-16 20:15:16

Sachiko wrote:

This is to make it seem like the script is contemplating something...

You've previously worked for Microsoft?

Sachiko · 2016-09-17 02:01:21

Trilby wrote:

Sachiko wrote:
This is to make it seem like the script is contemplating something...
You've previously worked for Microsoft?

If I did, I'd pull a Snowden and release info they don't want people to know, like how their current OS is basically a technological version of the NSA. Oopsies.

On a serious note, I've yet to run into a user that solely relies on repos or solely relies on the AUR. It's always a "blurred" world of both. In fact, the two friends of mine who convinced me to go Arch insist that using the AUR is a must for the complete Arch experience, hence my pacitude script's creation.

ewaller · 2016-09-17 05:16:23

Sachiko wrote:

If I did, I'd pull a Snowden and release info they don't want people to know, like how their current OS is basically a technological version of the NSA. Oopsies.

I know you're being humorous, but...
https://wiki.archlinux.org/index.php/Co … ial_topics
https://wiki.archlinux.org/index.php/Co … d_projects

Alad · 2016-09-17 11:54:52

On a serious note, I've yet to run into a user that solely relies on repos or solely relies on the AUR. It's always a "blurred" world of both. In fact, the two friends of mine who convinced me to go Arch insist that using the AUR is a must for the complete Arch experience, hence my pacitude script's creation.

The post is specifically about yaourt, but the general point applies:

http://jasonwryan.com/blog/2013/04/09/helpers/

P.S. Nice avatar

Last edited by Alad (2016-09-17 11:57:20)

Sachiko · 2016-09-17 16:19:54

@ewaller Sorry about that. I'll refrain from doing so in the future.

@Alad I do realize that by creating pacitude I was "scratching [my] own itch", however, I have been using pacman for a while and do know the flags. I also know how some can struggle to learn these. Also, pacitude isn't intended to be a cure-all by any means. "It is intended to help [them], not preclude [them] from being able to accomplish the most simple and critical task of system maintenance..."

P.S. Thanks. It's actually a much bigger image which is a part of my massive wallpaper collection that I cycle through using the first script I posted here(which you also helped me with[using find over ls]).

AaronBP · 2016-09-21 08:32:35

Speaking of pacman stuff, here's makepkg.py, a makepkg wrapper I just wrote to reduce some typing on my end. If someone has a similar workflow to me, they might find it useful.

#!/usr/bin/python3
import os
import os.path
import sys
import subprocess
import glob
ESC="\033"
RESET=0
BOLD=1
BLACK=30
RED=31
DONT_CHECKOUT = ["firefox-developer", "dosbox-x-git"]
def tput(n):
    return "{}[{}m".format(ESC, n)
if __name__ == "__main__":
    if not os.path.exists("PKGBUILD"):
        print("{bold}{red}===> ERROR:{reset}{bold} PKGBUILD does not exist.{reset}"
              .format(
                  bold=tput(BOLD),
                  red=tput(RED),
                  reset=tput(RESET)),
              file=sys.stderr)
        sys.exit(1)
    base = os.path.basename(os.getcwd())
    command_list = ["makepkg"] + sys.argv[1:]
    if base not in DONT_CHECKOUT:
        subprocess.run(["git", "checkout", "--", "PKGBUILD"])
    try:
        subprocess.run(["git", "pull"])
        tars = glob.glob("*.tar*")
        subprocess.run(["gvfs-trash"] + tars, stderr=subprocess.DEVNULL)
        try:
            subprocess.run(command_list, check=True)
            pkgs = glob.glob("*.pkg.tar.xz")
            subprocess.run(["sudo", "pacman", "-U"] + pkgs)
        except subprocess.CalledProcessError:
            pass
    except KeyboardInterrupt:
        pass
    subprocess.run(["rm", "-rf", "src"])
    subprocess.run(["rm", "-rf", "pkg"])

Awebb · 2016-09-21 09:43:53

AaronBP wrote:

Speaking of pacman stuff, here's makepkg.py, a makepkg wrapper I just wrote to reduce some typing on my end. If someone has a similar workflow to me, they might find it useful.

#!/usr/bin/python3
import os
import os.path
import sys
import subprocess
import glob
ESC="\033"
RESET=0
BOLD=1
BLACK=30
RED=31
DONT_CHECKOUT = ["firefox-developer", "dosbox-x-git"]
def tput(n):
    return "{}[{}m".format(ESC, n)
if __name__ == "__main__":
    if not os.path.exists("PKGBUILD"):
        print("{bold}{red}===> ERROR:{reset}{bold} PKGBUILD does not exist.{reset}"
              .format(
                  bold=tput(BOLD),
                  red=tput(RED),
                  reset=tput(RESET)),
              file=sys.stderr)
        sys.exit(1)
    base = os.path.basename(os.getcwd())
    command_list = ["makepkg"] + sys.argv[1:]
    if base not in DONT_CHECKOUT:
        subprocess.run(["git", "checkout", "--", "PKGBUILD"])
    try:
        subprocess.run(["git", "pull"])
        tars = glob.glob("*.tar*")
        subprocess.run(["gvfs-trash"] + tars, stderr=subprocess.DEVNULL)
        try:
            subprocess.run(command_list, check=True)
            pkgs = glob.glob("*.pkg.tar.xz")
            subprocess.run(["sudo", "pacman", "-U"] + pkgs)
        except subprocess.CalledProcessError:
            pass
    except KeyboardInterrupt:
        pass
    subprocess.run(["rm", "-rf", "src"])
    subprocess.run(["rm", "-rf", "pkg"])

Mind if I use this as an example next time somebody asks me, why I picked $LANGAUGE for $TASK? By the way, I hope you don't believe in any kind of afterlife with some sort of punishment mechanism, because "except : pass" has a dedicated area in any hell-type post-mortem places I know of.

Alad · 2016-09-21 10:10:28

Let's start from the end...

subprocess.run(["rm", "-rf", "src"])
    subprocess.run(["rm", "-rf", "pkg"])

can be replaced with:

makepkg -Cc

subprocess.run(["sudo", "pacman", "-U"] + pkgs)

Everyone does it apparently but seeing pacman -U is still sad. A local repository has many benefits, e.g. for pkgfile, split packages, distribution across machines:

https://wiki.archlinux.org/index.php/Pa … repository
https://wiki.archlinux.org/index.php/Pa … #Utilities (repose / repoctl)

tars = glob.glob("*.tar*")
        subprocess.run(["gvfs-trash"] + tars, stderr=subprocess.DEVNULL)

If you want to delete everything but the PKGBUILD you should just run git clean:

git clean -xi

or, non-interactively:

git clean -xf

if base not in DONT_CHECKOUT:
        subprocess.run(["git", "checkout", "--", "PKGBUILD"])
    try:
        subprocess.run(["git", "pull"])

Easier is to just use git submodules. You can also pull in parallel that way, and repos you don't want to pull you simply don't add as a submodule:

https://git-scm.com/docs/git-submodule

subprocess.run(["git", "checkout", "--", "PKGBUILD"])

Not sure what's the point in checking out the PKGBUILD specifically... you might as well edit a patch or .install file which would also prevent a successful pull. git stash is an alternative.

RED=31
...

This being Python, isn't there some library which already does all of that and more?

To summarize, I'd do something like this:

# Change dir to superproject
cd /path/to/aur

# Assumes submodules are configured, see `man git submodule`
git submodule update --jobs 5

# Repos are left as an exercise. Hint: use a repo wrapper or `find -L -mmin`
# Note: just builds by LC_COLLATE order
git submodule foreach makepkg -Ccirs

# Clean up
git submodule foreach git clean -xf

P.S. I don't know enough Python to judge the code itself -- but I think Awebb covered that part.

Last edited by Alad (2016-09-21 10:22:19)

AaronBP · 2016-09-22 03:03:57

Mind if I use this as an example next time somebody asks me, why I picked $LANGAUGE for $TASK? By the way, I hope you don't believe in any kind of afterlife with some sort of punishment mechanism, because "except : pass" has a dedicated area in any hell-type post-mortem places I know of.

It's cool. I'm an atheist. I'm aware that using exceptions for control flow is considered bad practice. I didn't particularly care, though.

Everyone does it apparently but seeing pacman -U is still sad. A local repository has many benefits, e.g. for pkgfile, split packages, distribution across machines:

Sounds like a handy feature, but not one I have a use for personally.

If you want to delete everything but the PKGBUILD you should just run git clean:

No. I trash the old packages instead of deleting them so I can recover them easily if there's a regression. Though that precaution has turned out to be pointless so far. I've got GNOME set up to automatically empty the trash every week.

Not sure what's the point in checking out the PKGBUILD specifically... you might as well edit a patch or .install file which would also prevent a successful pull. git stash is an alternative.

IIRC (and this is a port of a fish function I wrote some time ago, so I could be misremembering) it's because makepkg was updating the PKGBUILD with new version info in cvs packages, which caused git pull to fail.

This being Python, isn't there some library which already does all of that and more?

I looked but didn't see anything, except for curses which is obviously excessive.

dmerej · 2016-09-22 18:13:37

You may consider using termcolor or blessings

snakeroot · 2016-09-24 23:18:01

Alad wrote:

subprocess.run(["sudo", "pacman", "-U"] + pkgs)
Everyone does it apparently but seeing pacman -U is still sad. A local repository has many benefits, e.g. for pkgfile, split packages, distribution across machines:
https://wiki.archlinux.org/index.php/Pa … repository
https://wiki.archlinux.org/index.php/Pa … #Utilities (repose / repoctl)

This is brilliant! How have I lived without this? Since I found the documentation is a little opaque, here's what I did and plan to do.

BACKGROUND

Single user system, with all AUR packages held in their respective build directories and installed via "pacman -U".

STEPS

1. Create new "/home/<user>/packages" directory
2. Move all current AUR packages into "/home/<user>/packages" directory
3. Edit "/etc/makepkg.conf" to add the line

PKGDEST=/home/<user>/packages

4. Add the following new lines to "/home/<user>/.bashrc":

alias aurupdate="repo-add -n -R /home/chris/packages/AUR.db.tar.xz /home/chris/packages/*.pkg.tar.xz"
alias aurinstall="sudo pacman -Syu $(pacman -Sl AUR | grep -v installed | cut -d ' ' -f 2)"

5. source .bashrc and run aurupdate
6. Going forward, after building new AUR packages run aurinstall

Does this make sense? Do you have any suggestions on better ways to do this?

Alad · 2016-09-30 23:39:52

Coincidentally, I have tips on how to use a local repository (with repose) in aurutils(7). I should probably merge that to the wiki at one point ...

Re your Steps section, I'm not sure what the aurinstall alias tries to achieve: pacman -Syu will update all installed packages, regardless of the targets you specify. I.e., a -Syu would suffice (you can also only sync/update your local repository, but that's a more advanced topic). Looks like you got the basics though.

Last edited by Alad (2016-09-30 23:42:27)

escondida · 2016-10-01 02:12:35

~/local/bin/template: print, copy or edit a template file

#!/usr/bin/env rc

switch ($#*) {
case 1
	if (~ $1 -h) {
		echo Usage: template name [output_file]
		echo template -e name
	}
	if not {
		cat $XDG_TEMPLATES_DIR/$1
	}
case 2
	if (~ $1 -e) {
		plumb -s template -d edit $XDG_TEMPLATES_DIR/$2
	}
	if not {
		cp $XDG_TEMPLATES_DIR/$1 $2
	}
case *
	echo Usage: template name [output_file] >[1=2]
	return 1
}

~/local/bin/bookmark: plumb (general purpose ipc and thing-runner) url of bookmark, or print title/description and url. Bookmark file has the format:

Topic header

	Title/description
		<url>

#!/usr/bin/env rc

if (~ $1 -p) {
	shift
	sed -n /$1/'{p;n;p;}' $home/notes/bookmarks
}
if not {
	plumb `{sed -n /$1/'{n;p;}' $home/notes/bookmarks}
}

~/local/bin/gg: git grep -n * (implemented as a tiny script rather than a shell function/alias for use in editor)

~/local/bin/mon: Monitor a file or directory for changes, run command when changes occur, and write all output to a dedicated acme editor window (clearing it before each new iteration). Could probably be adapted to your editor of choice pretty easily; super useful for (say) writing LaTeX or debugging code.

I got the idea from Russ Cox's video on the basics of acme. I couldn't find the source of his mon program online (granted, I didn't look too hard), so I wrote my own.

#!/usr/bin/env rc
# mon: monitor file or directory for changes, run command when changes occur

PLAN9=/usr/lib/plan9
. $PLAN9/lib/acme.rc

if (~ $#* [01]) {
	echo 'Usage: mon watched_path command [args]' >[1=2]
	exit 1
}

fn acme_stat {
	9p stat acme/$1 >/dev/null >[2=1]
}

file=`{realpath $1}
name=`{dirname $file}^/+$1.monitor

cmd=$2
shift 2

newwindow
winname $name
while (acme_stat $winid) {
	if (inotifywait -qq -t 60 -e modify $file) {
		echo -n , | winwrite addr
		$cmd $* >[2=1] | winwrite data
		echo clean | winwrite ctl
	}
}

# acme_stat's exit status should always be 1;
# this doesn't mean that mon has failed.
exit 0

~/local/bin/plumbsel: plumb the contents of X selection or clipboard. Useful for launching arbitrary things on any selectable text, anywhere, particularly if bound to a key with sxhkd or xbindkeys. For instance, any url can immediately go to a Web browser; the string <stdio.h> anywhere becomes a link to /usr/include/stdio.h, dict:foo looks up "foo" using dictd, etc. User-definable arbitrary text hyperlinks (-:

#!/usr/bin/env rc

switch ($1) {
case -c
	sel=`{xclip -selection clipboard -o}
case *
	sel=`{xclip -selection primary -o}
}

plumb $sel

#!/usr/bin/env sh
# This is completely senseless.

eject
sleep ${1:-5}
eject -t

Last edited by jasonwryan (2016-10-01 02:37:22)

snakeroot · 2016-10-01 16:40:20

Alad wrote:

I'm not sure what the aurinstall alias tries to achieve: pacman -Syu will update all installed packages, regardless of the targets you specify. I.e., a -Syu would suffice.

Thanks Alad. aurinstall does the inverse of pacman -Syu: it goes through the AUR repository and installs all uninstalled packages. So I can build multiple packages and then run "aurinstall" at the end and they'll all be installed, and everything else updated to boot.

Arch Linux

#2801 2016-09-12 12:36:36

Re: Post your handy self made command line utilities

#2802 2016-09-13 18:34:40

Re: Post your handy self made command line utilities

#2803 2016-09-13 19:00:33

Re: Post your handy self made command line utilities

#2804 2016-09-14 04:18:12

Re: Post your handy self made command line utilities

#2805 2016-09-14 06:21:04

Re: Post your handy self made command line utilities

#2806 2016-09-14 13:47:41

Re: Post your handy self made command line utilities

#2807 2016-09-15 15:15:04

Re: Post your handy self made command line utilities

#2808 2016-09-16 06:02:08

Re: Post your handy self made command line utilities

#2809 2016-09-16 12:07:12

Re: Post your handy self made command line utilities

#2810 2016-09-16 19:28:06

Re: Post your handy self made command line utilities

#2811 2016-09-16 19:48:51

Re: Post your handy self made command line utilities

#2812 2016-09-16 20:15:16

Re: Post your handy self made command line utilities

#2813 2016-09-17 02:01:21

Re: Post your handy self made command line utilities

#2814 2016-09-17 05:16:23

Re: Post your handy self made command line utilities

#2815 2016-09-17 11:54:52

Re: Post your handy self made command line utilities

#2816 2016-09-17 16:19:54

Re: Post your handy self made command line utilities

#2817 2016-09-21 08:32:35

Re: Post your handy self made command line utilities

#2818 2016-09-21 09:43:53

Re: Post your handy self made command line utilities

#2819 2016-09-21 10:10:28

Re: Post your handy self made command line utilities

#2820 2016-09-22 03:03:57

Re: Post your handy self made command line utilities

#2821 2016-09-22 18:13:37

Re: Post your handy self made command line utilities

#2822 2016-09-24 23:18:01

Re: Post your handy self made command line utilities

#2823 2016-09-30 23:39:52

Re: Post your handy self made command line utilities

#2824 2016-10-01 02:12:35

Re: Post your handy self made command line utilities

#2825 2016-10-01 16:40:20

Re: Post your handy self made command line utilities

Board footer