Post your handy self made command line utilities

Trilby · 2022-03-08 00:01:28

I must be missing something, why curl a snapshot then untar it? Is that just to prevent requiring 'git'?

qinohe · 2022-03-08 00:07:13

No, it's just an old script I never updated to use git;-) which still works fine to get 'any' package from AUR
What I meant is he can convert his script to use it with any package instead of a script for every package...

edit: and here's that part rewritten for git;-)

buildir="$HOME"/build
prog=$1

get() {
  [ -d "$buildir/$prog" ] && mv "$buildir/$prog" "$buildir/${prog}.bak"
   cd "$buildir" || exit
  git clone https://aur.archlinux.org/"${prog}"
}
get

build() {
  cd "${buildir}"/"${prog}" || exit
  makepkg --noconfirm -sic
}
build

edit: add or exit) & build

Last edited by qinohe (2022-03-08 01:43:48)

lenhuppe · 2022-03-08 00:57:29

qinohe wrote:

Or get 'any' package you want and get on building it the way you like, my get part:
url='https://aur.archlinux.org/cgit/aur.git/snapshot/'      
buildir="$HOME"/build
prog=$2

_get() {
  [ -d "$buildir/$prog" ] && mv "$buildir/$prog" "$buildir/${prog}.bak"

  curl -sL "${url}${prog}.tar.gz" -o "$buildir/${prog}.tar.gz"

  tar zxvf "$buildir/${prog}.tar.gz" -C "$buildir" #&& rm "$buildir/${prog}.tar.gz"

}
_get "$@"
I mean your two scripts are for one package each, which you could make into 'one rules them all';-) mostly that is!

Nice ... I like that idea. I'll have a go at it when I'm not crazy busy.

qinohe · 2022-03-08 01:07:57

Yeah, then be sure to use the git one from #3677 although the one you quoted works fine too.
Also be sure it exits when not finding the 'buildir', I'll add it to the script...

edit: I added a build part too, do with it what you like!

edit: Ow, and I also forgot to say, you don't need to cd to '/tmp'.. what for?
That's something you set in makepkg.conf, mostly;-)

Last edited by qinohe (2022-03-08 02:04:27)

Tarqi · 2022-03-08 23:26:11

seth wrote:

wget/linode speedtest for reference

Some superfluous polish..

#!/bin/bash

locations=("Newark, USA"            "newark"
           "Singapore"              "singapore"
           "London, UK"             "london"
           "Frankfurt, Deutschland" "frankfurt"
           "Dallas, USA"            "dallas"
           "Toronta, Canada"        "toronto1"
           "Sydney, Australia"      "syd1"
           "Atlanta, USA"           "atlanta"
           "Tokyo, Japan"           "tokyo2"
           "Bombay, India"          "mumbai1"
           "Fremont, USA"           "fremont")

sizes=("100MB" "1GB")

for ((i=0; i<${#locations[@]}; i+=2)); do
  locs[$i]="${locations[$i]}"
done

echo
PS3=$'\n'"Enter location: "
select t in "${locs[@]}"; do
  location="${locations[(($REPLY*2-1))]}"
  [[ $REPLY -ge 1 && $REPLY -le ${#locs[@]} ]] && break
done
unset REPLY

echo
PS3=$'\n'"Select size: "
select size in "${sizes[@]}"; do
  [[ $REPLY -ge 1 && $REPLY -le ${#sizes[@]} ]] && break
done

echo
[[ -n "${location}" && -n "${size}" ]] && wget -nv --show-progress -O /dev/null "http://speedtest.${location}.linode.com/${size}-${location}.bin"

Last edited by Tarqi (2022-03-09 00:22:44)

Tarqi · 2022-03-08 23:31:55

zynex wrote:

A simple script to check your internal and external IP addresses..

Bash function, useful for own or foreign IPs. The AWK is a bit weird regarding the ordering of the variables, if someone can shed some light on this...

ipi ()
{
    curl -s "https://ipapi.co/$*/yaml" | awk '/ip:/{IP=$2} /asn:/{ASN=$2} /city:/{$1="";CITY=$0;} /country_name:/{$1="";COUNTRY=$0} /org:/{$1="";print IP ", " ASN "," $0 "," CITY "," COUNTRY}'
}

Bash function for local IPs:

ips ()
{
    ip -c -o a | awk '{print $2,$4}' | column -t
}

Last edited by Tarqi (2022-03-09 00:30:03)

Trilby · 2022-03-08 23:41:43

Why parse yaml with awk? Just get the json and use jq? I gather you've not seen posts 3661-3663 in this very thread ... as an identical goal was discussed with examples.

Tarqi · 2022-03-08 23:53:26

I saw the posts, the goal was not to use jq, as it is not installed as default on some systems I administrate, but AWK is. And YAML is (in this case) more friendly for parsing with AWK.

Last edited by Tarqi (2022-03-09 00:26:35)

Trilby · 2022-03-09 00:55:49

In that case, you should probably use awk's -F option as below. I also threw in a sed version if you don't care about the order of the fields (and just used a google ip for demonstration purposes):

#!/bin/sh

curl -s "https://ipapi.co/8.8.8.8/yaml" | awk -F': ' '
	/ip:/ { ip=$2 }
	/asn:/{ asn=$2 }
	/city:/{ city=$2;}
	/country_name:/ { country=$2}
	/org:/{ printf "%s, %s, %s, %s, %s\n", ip, asn, $2, city, country; }'

curl -s "https://ipapi.co/8.8.8.8/yaml" | sed -n '
	/^\(ip\|asn\|city\|country_name\|org\):/{s/[^:]*: //;H;}
	${x;s/\n//;s/\n/, /g;p;}'

And once you used the colon as the field separator, it becomes even easier:

curl -s "https://ipapi.co/8.8.8.8/yaml" | awk -F': ' '{ v[$1]=$2; }
	END { printf "%s, %s, %s, %s, %s\n", v["ip"], v["asn"], v["org"], v["city"], v["country_name"]; }'

Last edited by Trilby (2022-03-09 00:58:58)

Tarqi · 2022-03-09 13:20:14

I missed the field separator somehow... and the solution with the "dynamic dictionary" is great, thanks for that.

xyproto · 2022-04-07 13:09:16

Entering a directory via a symlink, and not quite sure where you really are on the file system?

The really alias that does cd "`pwd -P` has been useful to me.

For easy copy&paste to ie. .zshrc:

alias really="cd \"\`pwd -P\`\""

(yes, the backticks can probably be replaced with $())

Trilby · 2022-04-07 13:35:38

The backticks and backslashes can all be removed:

alias really='cd "$(pwd -P)"'

However, that readlly doesn't do anything at all. The only result I could imagine it having is changing how the current working directory is displayed in a prompt string if you use it there. But if what you really want in the prompt string is the actual path with symlinks resolved, you don't need an alias that you'll run manually, just write your prompt command to show what you actually want in the first place.

Last edited by Trilby (2022-04-07 13:39:17)

schard · 2022-04-07 14:24:52

cd -P .

clayrisser · 2022-06-27 16:47:39

My favorite script I've written is a simple and elegant templating engine.

tmpl.sh

#!/bin/sh

EOF=EOF
exec cat <<EOF | sh
cat <<EOF
$(cat $1 | \
    sed 's|\\|\\\\|g' | \
    sed 's|`|\\`|g' | \
    sed 's|\$|\\\$|g' | \
    sed "s|${OPEN:-<%}|\`eval echo |g" | \
    sed "s|${CLOSE:-%>}| 2>/dev/null \`|g")
$EOF
EOF

You can read more about it at the link below.

https://stackoverflow.com/a/72775596/5179133

Last edited by clayrisser (2022-06-27 16:56:22)

tsallinia86 · 2022-07-22 08:51:23

Hi all,

First a disclaimer: I am kind of newbie, so apologies to all the programmers and power users out there.

As a user I hate find and prefer locate to research for files from my terminal. However, to get the best possible results you need to perform updatedb often. Starting it as a systemd service was not an option as it adds to the boot up time. Plus, some days I just don't need to search for files, so updatedb is not needed. Hence, I created my own script that updatesdb on demand and while doing so you just type in the name of the file you want to be searched. My command is performed by typing search The search is not case sensitive.

Simply copy this code to a file, chmod 777, and paste to /usr/local/bin

 
#! /bin/bash
sudo -s updatedb && read varname &&  locate -e -i $varname

Last edited by tsallinia86 (2022-07-22 09:06:08)

2ManyDogs · 2022-07-22 11:54:25

Topics merged.

kokoko3k · 2022-07-22 12:39:32

tsallinia86 wrote:

Hi all,
First a disclaimer: I am kind of newbie, so apologies to all the programmers and power users out there.
As a user I hate find and prefer locate to research for files from my terminal. However, to get the best possible results you need to perform updatedb often. Starting it as a systemd service was not an option as it adds to the boot up time. Plus, some days I just don't need to search for files, so updatedb is not needed. Hence, I created my own script that updatesdb on demand and while doing so you just type in the name of the file you want to be searched. My command is performed by typing search The search is not case sensitive.
Simply copy this code to a file, chmod 777, and paste to /usr/local/bin
 
#! /bin/bash
sudo -s updatedb && read varname &&  locate -e -i $varname
 

You're defeating the purpose of locate and turned it into a crippled (and less efficient than the) find that you hate or am i missing something here?

Trilby · 2022-07-22 12:50:26

tsallinia86 wrote:

Starting it as a systemd service was not an option as it adds to the boot up time.

Then you created your service incorrectly. This should be triggered by a systemd timer which could have a delay after boot and could run periodically from then on. In fact the mlocate package comes with a timer that should do just this.

dogknowsnx · 2022-07-22 13:44:38

tsallinia86 wrote:

I am kind of newbie, so apologies to all the programmers and power users out there

Don't apologize - just keep learning...

https://wiki.archlinux.org/

tsallinia86 wrote:

As a user I hate find

Your aversion is likely rooted in ignorance...

man find

tsallinia86 wrote:

Simply copy this code to a file, chmod 777, and paste to /usr/local/bin

You don't want your scripts to be write- and executable by the whole world...

https://wiki.archlinux.org/title/File_p … attributes

tsallinia86 · 2022-07-23 08:01:37

Thank you for your feedback. I don't think my laptop performs updatedb periodically. I have checked this morning after I created a random file and locate -e -i still can't find it.

In what way is the script crippled? I t uses all the arguments I normally use when I invoke locate.

Alad · 2022-07-23 09:55:30

Find supports many more things than just searching by file names (by permissions, owner, different path depths, and so on) so locate can hardly replace it.

Maybe you might like "fd", it advertises itself as "user-friendly": https://github.com/sharkdp/fd

CarbonChauvinist · 2022-08-13 17:18:31

So, in the depressingly decreasing free time I can carve out, I sometimes plod away at various small scripts that interest me in the hopes of better learning bash scripting, purely for enjoyment. With that as a backdrop...

I have a script I'm using to scrape the National Geographic picture of the day. It works fairly well for my needs considering my relative inexperience with Bash. I guess the one thing I was hoping for some suggestions on is how to better handle character set encoding issues I guess?

My script basically scrapes the html of the page for the image description and the actual file/image link buried in the code. The image is downloaded and the image description is appended to a txt file, an index of sorts. I then confirm that both happened successfully and display the image and print the description to the screen. But when a character has accents or the like I get what looks like a the html utf instead of the character it represents? Example below which in this case is just supposed to be an apostrophe:

~ $ ng
NationalGeographic_1215640.jpg downloaded successfully
"A person prays at Tibet&#x27;s most sacred temple, the Jokhang, in this image from our photography archives. written to index successfully
2022-08-13: "A person prays at Tibet&#x27;s most sacred temple, the Jokhang, in this image from our photography archives.

What can I do to address this? Are there any other general recommendations to improve the overall script?

Here's the script btw:

#!/usr/bin/env bash

# save root photo of the day website
readonly POTD_URL="https://www.nationalgeographic.com/photo-of-the-day"

# store destination directory and indexfile in vars
readonly DEST_DIR="$HOME/Pictures/NGWallpapers"
readonly INDEX_FILE="$DEST_DIR/image-details.txt"

# function to display newest dated file from destination directory
# use rg when outputting decription line in order to highlight date in red
ngpic() {
  local newest
  read -r newest <<<"$(ls -t "$DEST_DIR"/*.jpg)"
  imv-wayland -f "$newest" && tail -1 "$INDEX_FILE" | rg '^[0-9]{4}-[0-9]{1,}-[0-9]{1,}'
}

# array to hold html tags for parsing imgurl and desc
declare -a data

# search for 'meta data-react-helmet' lines and add to data array for parsing
while read -r -d '>' LINE; do
  if [[ $LINE =~ "<meta data-react-helmet" ]]; then
    data+=("$LINE")
  fi
done <<<"$(curl -s "$POTD_URL")"

# extract image description
image_desc=$(grep "name=\"description\" content=\"" <(printf "%s\n" "${data[@]}"))
image_desc=${image_desc#*content=}
image_desc=${image_desc%\"/*}

# extract line containing image url
image_url_line=$(grep "\"og:image\" content=\"" <(printf "%s\n" "${data[@]}"))

# extract image url
image_url=${image_url_line#*content=\"}
image_url=${image_url%.jpg*}.jpg

# extract image filename only
image_file=${image_url##*/}

# download current wallpaper to tmpfile
TMPFILE=$(mktemp) || exit 3
curl -s "$image_url" >"$TMPFILE"
trap 'rm -f "$TMPFILE"' EXIT

# sanity checks
if [[ ! -d $DEST_DIR ]] || [[ ! -f $INDEX_FILE ]]; then
  echo "check your dir or file, something's off" && exit 1
elif [[ -e "$DEST_DIR/$image_file" ]]; then
  # TODO - find other way to compare files that doesn't rely on cmp
  if cmp -s "$TMPFILE" "$DEST_DIR/$image_file"; then
    echo "$image_file already exists"
    ngpic
    exit 2
  fi
fi

# if passes all sanity checks then proceed to copying to Pictures folder
# and appending to imagefile index
cp "$TMPFILE" "$DEST_DIR/$image_file"
[[ -e "$DEST_DIR/$image_file" ]] && echo "$image_file downloaded successfully"

# append desc to indexfile
if echo "$(printf '%(%Y-%m-%d)T'): $image_desc" >>"$INDEX_FILE"; then
  echo "$image_desc written to index successfully"
fi

Last edited by CarbonChauvinist (2022-08-13 18:21:33)

Trilby · 2022-08-13 19:10:54

First, wouldn't the following do the same without all the excess tests and redirection steps:

#!/bin/sh

url=https://www.nationalgeographic.com/photo-of-the-day
destdir=$HOME/Pictures/NGWallpapers
index=$destdir/image-details.txt

eval "$(curl -s $url | sed -n '
	/meta data-react-helmet/H
	$ {
		g; s/.*name="description" content="\([^"]*\).*/image_desc="\1"/p
		g; s/.*property="og:image" content="\([^"]*\).*/image_url="\1"/p
	}
')"

grep -q "$image_desc" $index && exit

date -f "%Y-%m-%d: $image_desc" >> $index
curl -s -O --outdir $destdir --create-dirs "$image_url"
imv-wayland -f "$(ls -t "$destdir/*.jpg")"

seth · 2022-08-13 19:15:20

… | recode html..utf-8

https://archlinux.org/packages/extra/x86_64/recode/
Afaik iconv can't do this.

CarbonChauvinist · 2022-08-14 01:57:23

Thank you both for your time and input!

@Seth, recode works perfectly as a drop in and in initial testing handles the conversion just as I'd hoped, thanks a lot!

@Trilby, your tag really should be changed from 'Inspector Parrot' to 'Parsimonious Parrot' - I will need some time to digest the sed-fu you've suggested and may just change over to that logic instead once I feel I've fully wrapped my head around it (I still use some throw-away python you'd suggested in a previous thread to rank mirrors by speed as replacement for rankmirrors/reflector btw).

TBH I don't use sed that much, and though it may not be immediately apparent, my script is somewhat contrived on purpose in the sense that I was trying to practice prioritizing using bash builtins where possible (i.e string manipulations, printf for dates/times, etc.) and wanted to limit additional binaries (even though I used a fair number including grep, cmp, etc.)

Last edited by CarbonChauvinist (2022-08-14 02:01:38)

Arch Linux

#3676 2022-03-08 00:01:28

Re: Post your handy self made command line utilities

#3677 2022-03-08 00:07:13

Re: Post your handy self made command line utilities

#3678 2022-03-08 00:57:29

Re: Post your handy self made command line utilities

#3679 2022-03-08 01:07:57

Re: Post your handy self made command line utilities

#3680 2022-03-08 23:26:11

Re: Post your handy self made command line utilities

#3681 2022-03-08 23:31:55

Re: Post your handy self made command line utilities

#3682 2022-03-08 23:41:43

Re: Post your handy self made command line utilities

#3683 2022-03-08 23:53:26

Re: Post your handy self made command line utilities

#3684 2022-03-09 00:55:49

Re: Post your handy self made command line utilities

#3685 2022-03-09 13:20:14

Re: Post your handy self made command line utilities

#3686 2022-04-07 13:09:16

Re: Post your handy self made command line utilities

#3687 2022-04-07 13:35:38

Re: Post your handy self made command line utilities

#3688 2022-04-07 14:24:52

Re: Post your handy self made command line utilities

#3689 2022-06-27 16:47:39

Re: Post your handy self made command line utilities

#3690 2022-07-22 08:51:23

Re: Post your handy self made command line utilities

#3691 2022-07-22 11:54:25

Re: Post your handy self made command line utilities

#3692 2022-07-22 12:39:32

Re: Post your handy self made command line utilities

#3693 2022-07-22 12:50:26

Re: Post your handy self made command line utilities

#3694 2022-07-22 13:44:38

Re: Post your handy self made command line utilities

#3695 2022-07-23 08:01:37

Re: Post your handy self made command line utilities

#3696 2022-07-23 09:55:30

Re: Post your handy self made command line utilities

#3697 2022-08-13 17:18:31

Re: Post your handy self made command line utilities

#3698 2022-08-13 19:10:54

Re: Post your handy self made command line utilities

#3699 2022-08-13 19:15:20

Re: Post your handy self made command line utilities

#3700 2022-08-14 01:57:23

Re: Post your handy self made command line utilities

Board footer