Post your handy self made command line utilities

vik_k · 2014-04-27 21:54:50

sorry for OT again, ^^ but yeah, probably HUGE resolution it is

BradPJ · 2014-04-29 20:27:27

Earnestly wrote:

BradPJ wrote:
I figured that since the rest was all '#' it wouldn't have any effect.
See: binfmt_script.c:25

Thanks, I'm trying to understand it but I honestly have no idea what is going on there. I've read up a bit on what a shebang does and understand it all a bit better now and see that what I done originally was completely stupid. Thanks for the reply though.

Last edited by BradPJ (2014-04-29 20:28:23)

Neven · 2014-05-03 21:27:56

A short script for building and installing a package from a repo directory in the ABS tree without modifying the PKGBUILD:
#! /usr/bin/env sh
absroot=/var/abs
buildroot=/tmp
repo=$1
package=$2
cp -r $absroot/$repo/$package $buildroot
cd $buildroot/$package
makepkg -ci
cd ..
rm -rf makepkg $package

Usage example:
build_package core zlib

The example command downloads zlib source, builds it according to the ABS PKGBUILD, installs zlib with pacman and cleans a bit after itself.

jasonwryan · 2014-05-03 21:40:43

Please use code tags when pasting to the boards: https://wiki.archlinux.org/index.php/Fo … s_and_Code

Also, why?

jakobcreutzfeldt · 2014-05-03 23:57:28

jasonwryan wrote:

Please use code tags when pasting to the boards: https://wiki.archlinux.org/index.php/Fo … s_and_Code
Also, why?

Haven't you ever gone to the store, bought a TV, brought it home, unboxed it, disassembled it, and then hand-assembled it? I heard the picture's clearer if you do it that way.

lolilolicon · 2014-05-04 05:01:54

jakobcreutzfeldt wrote:

Haven't you ever gone to the store, bought a TV, brought it home, unboxed it, disassembled it, and then hand-assembled it? I heard the picture's clearer if you do it that way.

We all knew it. AND It's proven! It's called the IKEA effect. I LOOKED IT UP MYSELF!

qinohe · 2014-05-09 12:51:17

I'm a frequent user of archiso, the project to create your own ISO.
Since I maintain more than one, I created a script to do it for me.
Hope you like it.

#!/bin/bash
# Script to automate archiso creation.
# Note: directory name=ISO name
# Note: all is based on user directory, not chroot!
# change the next things to your needs: USER, iso1-5, path1 & 2
USER='user'

# Change the name for your archiso project.
iso1='archiso'
iso2='archiso-bank'
iso3='archiso-home'
iso4='archiso-server'
iso5='archiso-testing'

# Date is  used in Archiso ISO naming, 
# only change if you changed this.
DATE=$(date +%Y.%m.%d)

# No need to change these vars.
isoo=$(isoo 2>/dev/null) || isoo=/tmp/isoo
trap "rm -f /tmp/isoo*" 0 1 2 5 15
DIALOG=${DIALOG=dialog}

$DIALOG --backtitle "Archiso 'maker'." \
            --title "Which ISO do want to make" --clear \
        --radiolist "" 11 30 5 \
"$iso1" "" off \
"$iso2" "" off \
"$iso3" "" off \
"$iso4" "" off \
"$iso5" "" off 2>/tmp/isoo;

retval=$?

ISONAME=$(cat "$isoo")

# Change these paths to your needs.
path1=/home/"$USER"/"$ISONAME"/releng     # path to archiso/releng dir
path2=/home/"$USER"/isoos                 # path to iso storage

case $retval in
  0)
		    cd "$path1"
		      rm -r work/ out/
		        sh ./build.sh -v
		          echo "$ISONAME" ISO is created.
	                     mv out/"$ISONAME"-"$DATE"-x86_64.iso "$path2"
		         echo "  ISO moved to ISO dir."
		       rm -r work/ out/
		    echo " Removed 'out/' & 'work/'"
		;;
  1)
			echo "Canceled."
		;;
  255)
			echo "Escaped."
		;;
esac

trap "rm -f /tmp/isoo*" 0 1 2 5 15

exit 0

jasonwryan · 2014-05-16 07:43:05

Check which AUR packages you have voted for:

while read -r pkg 0; do printf "%s " "${pkg}:"; aurvote -c "$pkg"; done < <(pacman -Qm)

Earnestly · 2014-05-16 16:27:46

jasonwryan wrote:

Check which AUR packages you have voted for

If I may:

# You can use '_' to discard a value during a read (based on IFS)
while read -r pkg _; do …

# …but I noticed that it might not even be necessary as you can pass `-q` to `-Q`
while read -r pkg; do printf '%s: ' "$pkg"; aurvote -c "$pkg"; done < <(pacman -Qqm)

# Try to put all static text, such as the ':', in the format section of printf; isolating the variables you pass to the formatters.

ralvez · 2014-05-16 17:48:36

There are some apps. I only want to run one instance of, like my mail client for example.
To do that I created this small script:

APPDIR=$HOME/Apps/thunderbird;
APPSTATUS=`pgrep -c thunderbird`;

if [ $APPSTATUS == 1 ]; then
    kdialog --title "Email Client Notification" --passivepopup "Thunderbird is running ..." 5
else
    $APPDIR/thunderbird&
fi

I hope this will be helpful to someone.

R.

jasonwryan · 2014-05-16 19:20:55

Earnestly wrote:

jasonwryan wrote:
Check which AUR packages you have voted for
If I may:
# You can use '_' to discard a value during a read (based on IFS)
while read -r pkg _; do …

You may; zsh doesn't like the `_`, hence the 0. I should have specified that.

# …but I noticed that it might not even be necessary as you can pass `-q` to `-Q`
while read -r pkg; do printf '%s: ' "$pkg"; aurvote -c "$pkg"; done < <(pacman -Qqm)

# Try to put all static text, such as the ':', in the format section of printf; isolating the variables you pass to the formatters.

Excellent: thank you.

Earnestly · 2014-05-16 19:24:20

That's right, _ is a bashism

esko997 · 2014-05-18 03:49:30

Not sure if this is useful for anyone, just something I quickly wrote so that I could search google from the commandline. Very simple, but maybe someone can use this.

#!/bin/bash

# This script takes user input and then uses it to search google.

echo "Enter your search terms: "
read search

searchRefine=$(echo $search | sed -e 's/ /+/g')

firefox https://www.google.com/#q=$searchRefine

WonderWoofy · 2014-05-18 04:44:17

esko997 wrote:

Not sure if this is useful for anyone, just something I quickly wrote so that I could search google from the commandline. Very simple, but maybe someone can use this.
#!/bin/bash

# This script takes user input and then uses it to search google.

echo "Enter your search terms: "
read search

searchRefine=$(echo $search | sed -e 's/ /+/g')

firefox https://www.google.com/#q=$searchRefine

@esko997, have you ever heard of surfraw? You should check it out.

quequotion · 2014-05-18 10:37:15

This thread is 95 pages long!

Perhaps it's time to start on an organized Arch User Script Repository?

Some stuff I've scattered about the forums:
If you use the BFS patchset, you can get improved control of individual program's performance with isosched & idleprio.
The throttle script handles a large number of generic powersaving settings with a manual switch.

Sometimes I use the above without a desktop, but overheating and such worry me, so I put together this simple one-liner to keep an eye on things:
sensorsweep (checks lm-sensors every 10 seconds and refreshes the display)

while [ true ]; do sensors; sleep 10; clear; done

One of the things that I use most of the above for is converting optical media to digital files, for which I have these handy scripts:
unfreecode (x264 and AC3, which have unfree patents):

#! /bin/bash
HandBrakeCLI --main-feature -e x264 -x no-psnr:no-ssim:trellis=2:subq=7:partitions=all:me=esa:frameref=12:threads="$(getconf _NPROCESSORS_ONLN)" -q 22 -a 1,2 -E copy:ac3 -6 6ch -s 1,2 -i "${1}" -o "${2}"

# Injectables:
#
# -P # use opencl hardware
# --main-feature # Auto-select main title, delays start; can fail: select title manually: -t #
# --decomb # Auto-deinterlace. Slow, may cause *visual* lag.
# :threads=$(getconf _NPROCESSORS_ONLN) # after -x, use all cores (increase for faster / decrease  for lighter)
# --gain 4 -D 1.5  # Boost audio and balance volume, good for home video
# -q 22 #Quantitizer 22

#Encode a blu-ray transport stream
#HandBrakeCLI --main-feature -e x264 -x level=4.1:weightp=0:nal-hrd=vbr:vbv-maxrate=15000:vbv-bufsize=15000:bluray-compat:aud:keyint=24:bframes=3:b-pyramid=strict:open-gop:slices=4:fake-interlaced:colorprim=bt709:colormatrix=bt709:no-psnr:no-ssim:trellis=2:subq=7:partitions=all:me=esa:frameref=12:threads="$(getconf _NPROCESSORS_ONLN)" -q 22 -a 1-30 -E copy -6 6ch -i "${1}" -o "${1}.m2ts"

freecode (unfreecode's hippy cousin, Theora and Vorbis):

HandBrakeCLI -P -main-feature -e theora -2 -q 20 -a 1,2 -E vorbis -6 6ch -s 1,2 -i $1 -o $2

# --main-feature # Auto-select main title, delays start; can fail: select title manually: -t #
# --decomb # Auto-deinterlace. Slow, may cause visual lag.
# --gain 4 -D 1.5  # Boost audio and balance volume, good for home video

Theora's had several updates since this was put together; there may be more options available.

Both scripts take two arguments, input output, where input is any file or mounted disc (image)'s directory and output is the desired filename, including dot-three extension (for container format). They also pull in the first two audio and subtitle tracks when available; which is because I need bilingual media (English and Japanese for most bilingual media released in Japan; not necessarily in that order) to watch with friends. These also do two-pass encoding, which can take a long time but makes much better quality encodes.

When I need an encoding done full speed, and I'm not planning to use the pc, I do that like:

throttle full; isoched unfreecode /var/run/media/$USER/DVD\ DISC/ ~/Videos/Very/Intricate/Directory/Organization/Title\ of\ DVD.mkv

However, if I'm using the pc, but need to get it started anyway, I do that like:

idleprio unfreecode /var/run/media/$USER/DVD\ DISC/ ~/Videos/Very/Intricate/Directory/Organization/Title\ of\ DVD.mkv

Last edited by quequotion (2017-11-12 17:47:21)

esko997 · 2014-05-19 00:33:08

WonderWoofy wrote:

esko997 wrote:
Not sure if this is useful for anyone, just something I quickly wrote so that I could search google from the commandline. Very simple, but maybe someone can use this.
#!/bin/bash

# This script takes user input and then uses it to search google.

echo "Enter your search terms: "
read search

searchRefine=$(echo $search | sed -e 's/ /+/g')

firefox https://www.google.com/#q=$searchRefine
@esko997, have you ever heard of surfraw? You should check it out.

Woah, looks like exactly what I was looking for when I wrote that script. Thanks mate.

Trilby · 2014-05-19 00:39:08

Perhaps this belongs in the "oh nice" thread - but DWB doesn't require any of this. I can type `dwb "some search terms"` and the default search engine will be fired up in dwb with the search terms.

Example: `dwb "\!archwiki beginner's guide"` brings up the beginners' guide instantly.

Last edited by Trilby (2014-05-19 00:40:14)

jakobcreutzfeldt · 2014-05-19 09:04:24

Trilby wrote:

Perhaps this belongs in the "oh nice" thread - but DWB doesn't require any of this. I can type `dwb "some search terms"` and the default search engine will be fired up in dwb with the search terms.
Example: `dwb "\!archwiki beginner's guide"` brings up the beginners' guide instantly.

Conkeror too

Luiji · 2014-05-23 15:21:05

Considering I incidently posted a question that already had a topic on the same page (:O) I figured the best way to make it up was post a script I wrote here. I don't use IceWM much but when I did I immediately hated the idea of having to hand-write my menus so I wrote a script to generate it from the .desktop files. I call it "update-icewm-menu". It doesn't handle the "only show in this desktop" and "don't show in that desktop" directives, but you could add those in pretty easily -- just add a check in the "case key.downcase" and either use split(";").include? or a regex match, whatever floats your boat.

If you want to remove the AudioVideo category or something of the sort just remove it from the list at the top. E.g. I've considered doing this because it mostly just duplicates entries from Audio and Video.

#!/usr/bin/env ruby
# Generate the ~/.icewm/menu file from .desktop files.
#
# Written by Luiji Maryo in 2013, updated 2014
#
# Released under the CC0 license linked below, which is just a public domain
# release that handles countries that don't recognize the public domain or the
# ability of authors to release content directly into it.
#
# CC0 License: https://creativecommons.org/publicdomain/zero/1.0/

require 'fileutils'

Encoding.default_internal = 'UTF-8'
Encoding.default_external = 'UTF-8'

CATEGORIES = %w{
  AudioVideo Audio Video Development Education Game Graphics Network Office
  Settings System Utility
}

menus = {}

data_dirs = ENV['XDG_DATA_DIRS']
if data_dirs.nil? or data_dirs.empty?
  data_dirs = '/usr/local/share:/usr/share'
end

data_dirs.split(':').each do |dir|
  Dir[dir + '/applications/*.desktop'].each do |filename|
    puts filename

    File.open(filename) do |file|
      name = ''
      icon = ''
      exec = ''
      categories = ''

      file.each_line do |line|
	next if not line.include? '='

	key, value = line.chomp.split('=')
	value = (value or '')

	case key.downcase
	  when 'name' then name = value if name.empty?
	  when 'icon' then icon = value if icon.empty?
	  when 'exec' then exec = value if exec.empty?
	  when 'categories' then categories = value if categories.empty?
	end
      end

      exec.gsub! /%./, ''

      app = "  prog \"#{name}\" \"#{icon}\" #{exec}\n"
      in_category = false

      categories.split(';').each do |category|
	next unless CATEGORIES.include? category
	menus[category] = {} unless menus.include? category
	menus[category][name] = app
	in_category = true
      end

      unless in_category
	menus['Other'] = {} unless menus.include? 'Other'
	menus['Other'][name] = app
      end
    end
  end
end

unless Dir.exist? ENV['HOME'] + '/.icewm'
  FileUtils.mkdir_p ENV['HOME'] + '/.icewm'
end

File.open(ENV['HOME'] + '/.icewm/menu', 'w') do |output|
  menus.sort.each do |name, menu|
    output.write "menu \"#{name}\" \"\" {\n"

    menu.sort.each do |name, app|
      output.write app
    end

    output.write "}\n\n"
  end
end

# vim:set ts=8 sts=2 sw=2 noet:

I also put it in a gist: https://gist.github.com/Luiji/cdc5aa5df7dc9398e67e

Oh, and I like that thought on an Arch User Script Repository. Perhaps we could use the GitHub gist system and just do something like "gist install Luiji/cdc5aa5df7dc9398e67e" and take advantage of the gist revision capability for "gist update".

EDIT: Before anybody says it, I did just notice that there's a tool in the repositories that already does this (menumaker). Well, at least mine's quite a bit simpler.

Last edited by Luiji (2014-05-23 18:40:22)

publicus · 2014-06-03 17:02:29

Not mine code, but found this little utility xmllint and it solved a problem for me. This is a really good explanation:
http://www.fedorenchik.org/tooltips/bas … _xml.xhtml

publicus · 2014-06-04 18:32:04

Not sure how useful others will find this. I had issues with deleting certain files while keeping others (say config files that you've tried and perfected, but want to get rid of the cruft.) In the past I would have specify every bloody file by hand and delete it. This was messy, time consuming and error prone (oh, you misspelled that file? Well, congrats, you just blew away the one that you needed!)

There had to be a better way...

After doing some thinking, I needed a utility to "store" files temporarily and then regurgitate them when needed, hence "pouch"*.

This consists of these files:
pouch.sh: when you need to tuck away some data.

#!/bin/sh

# - Part 1 -
# in this part, make sure that we have all of the directories that we need in
#   the home directory.

# check if the home directory exists.
if [ ! -d ~/.pouch ]
then
  # our home pouch directory does not exist, bummer, time to exit.
  exit 1
fi

# check if the data directory exists.  This is where we would keep all of the
#   files that we push down.
if [ ! -d ~/.pouch/data ]
then
  exit 1
fi

# - Part 2 -
# this is where we will do the necessary copying backing of files.

# in the current directory, first see if there are any files that you can copy.
if [ 0 -ne $# ]
then
  # at this stage, proceed to first check if the file exists in the current
  #   directory and if it does, copy it to our data directory.
  for i in $@
  do
    if [ -f $i ] || [ -d $i ]
    then
      # the reason for using copy and then removing the file rather than mv is
      #   that it reduces the chance of file corruption when copying from/to
      #   an unreliable file system.
      cp $i ~/.pouch/data
      rm -rf $i
    else
      echo "ERROR: $i does not exist as either a file or a directory."
      echo ""
    fi
  done
fi

# we've done everything correctly if we're at this point.  Simply exit.
exit 0

unpouch.sh: when you need to return the saved data.

#!/bin/sh

# copy all information that is in our data directory to the current directory,
#   afterwards delete it (again, first copy, then delete, same as the pouch
#   utility.)  Do this for the hidden and not hidden files.
HIDDEN_FILES=`ls -a1 ~/.pouch/data/`

for i in $HIDDEN_FILES
do
  if [ $i != '.' ] && [ $i != '..' ]
  then
    cp -r ~/.pouch/data/$i .
    rm -rf ~/.pouch/data/$i
  fi  
done

pouchempty.sh: when you figure that what you've stored is no longer needed.

#!/bin/sh

# delete the contents of the entire ~/.pouch/data directory.  We would use
#   this when we've changed our minds about the stored data and would just
#   like to get rid of it.
rm -rf ~/.pouch/data
mkdir -p ~/.pouch/data

Now, in your home directory, do this: mkdir ~/.pouch/data and put the above scripts inside of ~/.pouch.

In your .bashrc (or whichever shell you're using, .zshrc for Z-shell, etc.) do this:

alias pouch='~/.pouch/pouch.sh'
alias unpouch='~/.pouch/unpouch.sh'
alias pouchempty='~/.pouch/pouchempty.sh'
alias pouche='pouchempty'

Then do source ~/.bashrc and you're good!

Now, there are known issues with this:
1 - If you store the same file foo.txt, the one stored later will clobber the one stored previously. At the moment, since this is just a prototype, this is done on purpose (if this proves to be useful, I'll figure out a way to handle this somehow.)
2 - This is not terribly fast being written in shell.
3 - Pretty I/O intensive since I'm copying. Let me know what you think about using 'mv' or making that optional.
4 - Not multi-processor safe. If one process does pouchempty and the other does pouch and this was not somehow handled more elegantly, bad things could happen...

I'd like to know what you think about this. Do you think that it has value? How would you improve it?

*It's what a hamster uses to store a several rugby balls worth of food neard its head... cute when it's a hamster, creepy when it's a human.

Last edited by publicus (2014-06-04 18:34:26)

Saint0fCloud · 2014-06-04 19:02:34

publicus wrote:

HIDDEN_FILES=`ls -a1 ~/.pouch/data/`

No need to spawn a subshell just to run `ls`. Instead use file globbing.

3 - Pretty I/O intensive since I'm copying. Let me know what you think about using 'mv' or making that optional.

Aggregating any I/O operations into a single command (ie only do one `cp` and one `rm`) will significantly speed up your script.

4 - Not multi-processor safe. If one process does pouchempty and the other does pouch and this was not somehow handled more elegantly, bad things could happen...

If you're really concerned about race conditions use `flock`.

There're a couple other things that you could change -- for example: no need to check $#, use [[ foo || bar ]] rather than [ foo ] || [ bar ], combine everything into one script -- but most of these are superfluous so I'll leave that to your discretion.

publicus · 2014-06-04 20:02:28

Saint0fCloud wrote:

publicus wrote:
HIDDEN_FILES=`ls -a1 ~/.pouch/data/`
No need to spawn a subshell just to run `ls`. Instead use file globbing.
3 - Pretty I/O intensive since I'm copying. Let me know what you think about using 'mv' or making that optional.
Aggregating any I/O operations into a single command (ie only do one `cp` and one `rm`) will significantly speed up your script.
4 - Not multi-processor safe. If one process does pouchempty and the other does pouch and this was not somehow handled more elegantly, bad things could happen...
If you're really concerned about race conditions use `flock`.

Thanks, I'll look into this a little later.

Saint0fCloud wrote:

There're a couple other things that you could change -- for example: no need to check $#, use [[ foo || bar ]] rather than [ foo ] || [ bar ], combine everything into one script -- but most of these are superfluous so I'll leave that to your discretion.

I'll be honest, I'm a n00b when it comes to shell scripting, so I'm sure that there are things that I could easily change to improve this utility.

As for multiple scripts, that was by intent. I was thinking of different commands that can be run and the files associated with them and that design transferred very well into different files that would do different tasks. Having just one command can be done (not difficult), but I'm curious how others will respond.

mid-kid · 2014-06-12 21:16:28

for package in $(pacman -Sql midrepo); do if [ ! "$(package-query -f %v -A $package)" = "$(package-query -f %v -S $package)" ]; then echo "$package"; fi; done

Check for updates on your aur binary repository with package-query. While slow, it beats the manual stuff.

EDIT: Faster version:

packages=($(pacman -Sql midrepo)); repo=($(package-query -f '%v' -S $packages)); aur=($(package-query -f '%v' -A $packages)); for x in {0..$[${#packages[@]}-1]}; do if [ ! "${repo[$x]}" = "${aur[$x]}" ]; then echo ${packages[$x]}; fi; done

Last edited by mid-kid (2014-06-12 21:30:44)

karol · 2014-06-12 21:30:25

What is an aur binary repository? If midrepo is a regular repo, why not just run 'pacman -Syu'?

Arch Linux

#2351 2014-04-27 21:54:50

Re: Post your handy self made command line utilities

#2352 2014-04-29 20:27:27

Re: Post your handy self made command line utilities

#2353 2014-05-03 21:27:56

Re: Post your handy self made command line utilities

#2354 2014-05-03 21:40:43

Re: Post your handy self made command line utilities

#2355 2014-05-03 23:57:28

Re: Post your handy self made command line utilities

#2356 2014-05-04 05:01:54

Re: Post your handy self made command line utilities

#2357 2014-05-09 12:51:17

Re: Post your handy self made command line utilities

#2358 2014-05-16 07:43:05

Re: Post your handy self made command line utilities

#2359 2014-05-16 16:27:46

Re: Post your handy self made command line utilities

#2360 2014-05-16 17:48:36

Re: Post your handy self made command line utilities

#2361 2014-05-16 19:20:55

Re: Post your handy self made command line utilities

#2362 2014-05-16 19:24:20

Re: Post your handy self made command line utilities

#2363 2014-05-18 03:49:30

Re: Post your handy self made command line utilities

#2364 2014-05-18 04:44:17

Re: Post your handy self made command line utilities

#2365 2014-05-18 10:37:15

Re: Post your handy self made command line utilities

#2366 2014-05-19 00:33:08

Re: Post your handy self made command line utilities

#2367 2014-05-19 00:39:08

Re: Post your handy self made command line utilities

#2368 2014-05-19 09:04:24

Re: Post your handy self made command line utilities

#2369 2014-05-23 15:21:05

Re: Post your handy self made command line utilities

#2370 2014-06-03 17:02:29

Re: Post your handy self made command line utilities

#2371 2014-06-04 18:32:04

Re: Post your handy self made command line utilities

#2372 2014-06-04 19:02:34

Re: Post your handy self made command line utilities

#2373 2014-06-04 20:02:28

Re: Post your handy self made command line utilities

#2374 2014-06-12 21:16:28

Re: Post your handy self made command line utilities

#2375 2014-06-12 21:30:25

Re: Post your handy self made command line utilities

Board footer