You are not logged in.

#3676 2022-03-08 00:01:28

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 29,442
Website

Re: Post your handy self made command line utilities

I must be missing something, why curl a snapshot then untar it?  Is that just to prevent requiring 'git'?


"UNIX is simple and coherent..." - Dennis Ritchie, "GNU's Not UNIX" -  Richard Stallman

Offline

#3677 2022-03-08 00:07:13

qinohe
Member
From: Netherlands
Registered: 2012-06-20
Posts: 1,494

Re: Post your handy self made command line utilities

No, it's just an old script I never updated to use git;-) which still works fine to get 'any' package from AUR
What I meant is he can convert his script to use it with any package instead of a script for every package...

edit: and here's that part rewritten for git;-)

buildir="$HOME"/build
prog=$1

get() {
  [ -d "$buildir/$prog" ] && mv "$buildir/$prog" "$buildir/${prog}.bak"
   cd "$buildir" || exit
  git clone https://aur.archlinux.org/"${prog}"
}
get

build() {
  cd "${buildir}"/"${prog}" || exit
  makepkg --noconfirm -sic
}
build

edit: add or exit) & build

Last edited by qinohe (2022-03-08 01:43:48)

Offline

#3678 2022-03-08 00:57:29

lenhuppe
Member
From: New Hampshire USA
Registered: 2018-12-10
Posts: 271
Website

Re: Post your handy self made command line utilities

qinohe wrote:

Or get 'any' package you want and get on building it the way you like, my get part:

url='https://aur.archlinux.org/cgit/aur.git/snapshot/'      
buildir="$HOME"/build
prog=$2

_get() {
  [ -d "$buildir/$prog" ] && mv "$buildir/$prog" "$buildir/${prog}.bak"

  curl -sL "${url}${prog}.tar.gz" -o "$buildir/${prog}.tar.gz"

  tar zxvf "$buildir/${prog}.tar.gz" -C "$buildir" #&& rm "$buildir/${prog}.tar.gz"

}
_get "$@"

I mean your two scripts are for one package each, which you could make into 'one rules them all';-) mostly that is!

Nice ... I like that idea. I'll have a go at it when I'm not crazy busy.


Why do we drive on the parkway and then park in the driveway?

Offline

#3679 2022-03-08 01:07:57

qinohe
Member
From: Netherlands
Registered: 2012-06-20
Posts: 1,494

Re: Post your handy self made command line utilities

Yeah, then be sure to use the git one from #3677 although the one you quoted works fine too.
Also be sure it exits when not finding the 'buildir', I'll add it to the script...

edit: I added a build part too, do with it what you like!

edit: Ow, and I also forgot to say, you don't need to cd to '/tmp'.. what for?
That's something you set in makepkg.conf, mostly;-)

Last edited by qinohe (2022-03-08 02:04:27)

Offline

#3680 2022-03-08 23:26:11

Tarqi
Member
From: Ixtlan
Registered: 2012-11-27
Posts: 179
Website

Re: Post your handy self made command line utilities

seth wrote:

wget/linode speedtest for reference

Some superfluous polish..

#!/bin/bash

locations=("Newark, USA"            "newark"
           "Singapore"              "singapore"
           "London, UK"             "london"
           "Frankfurt, Deutschland" "frankfurt"
           "Dallas, USA"            "dallas"
           "Toronta, Canada"        "toronto1"
           "Sydney, Australia"      "syd1"
           "Atlanta, USA"           "atlanta"
           "Tokyo, Japan"           "tokyo2"
           "Bombay, India"          "mumbai1"
           "Fremont, USA"           "fremont")

sizes=("100MB" "1GB")

for ((i=0; i<${#locations[@]}; i+=2)); do
  locs[$i]="${locations[$i]}"
done

echo
PS3=$'\n'"Enter location: "
select t in "${locs[@]}"; do
  location="${locations[(($REPLY*2-1))]}"
  [[ $REPLY -ge 1 && $REPLY -le ${#locs[@]} ]] && break
done
unset REPLY

echo
PS3=$'\n'"Select size: "
select size in "${sizes[@]}"; do
  [[ $REPLY -ge 1 && $REPLY -le ${#sizes[@]} ]] && break
done

echo
[[ -n "${location}" && -n "${size}" ]] && wget -nv --show-progress -O /dev/null "http://speedtest.${location}.linode.com/${size}-${location}.bin"

Last edited by Tarqi (2022-03-09 00:22:44)


Knowing others is wisdom, knowing yourself is enlightenment. ~Lao Tse

Offline

#3681 2022-03-08 23:31:55

Tarqi
Member
From: Ixtlan
Registered: 2012-11-27
Posts: 179
Website

Re: Post your handy self made command line utilities

zynex wrote:

A simple script to check your internal and external IP addresses..

Bash function, useful for own or foreign IPs. The AWK is a bit weird regarding the ordering of the variables, if someone can shed some light on this...

ipi ()
{
    curl -s "https://ipapi.co/$*/yaml" | awk '/ip:/{IP=$2} /asn:/{ASN=$2} /city:/{$1="";CITY=$0;} /country_name:/{$1="";COUNTRY=$0} /org:/{$1="";print IP ", " ASN "," $0 "," CITY "," COUNTRY}'
}

Bash function for local IPs:

ips ()
{
    ip -c -o a | awk '{print $2,$4}' | column -t
}

Last edited by Tarqi (2022-03-09 00:30:03)


Knowing others is wisdom, knowing yourself is enlightenment. ~Lao Tse

Offline

#3682 2022-03-08 23:41:43

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 29,442
Website

Re: Post your handy self made command line utilities

Why parse yaml with awk?  Just get the json and use jq?  I gather you've not seen posts 3661-3663 in this very thread ... as an identical goal was discussed with examples.


"UNIX is simple and coherent..." - Dennis Ritchie, "GNU's Not UNIX" -  Richard Stallman

Offline

#3683 2022-03-08 23:53:26

Tarqi
Member
From: Ixtlan
Registered: 2012-11-27
Posts: 179
Website

Re: Post your handy self made command line utilities

I saw the posts, the goal was not to use jq, as it is not installed as default on some systems I administrate, but AWK is. And YAML is (in this case) more friendly for parsing with AWK.

Last edited by Tarqi (2022-03-09 00:26:35)


Knowing others is wisdom, knowing yourself is enlightenment. ~Lao Tse

Offline

#3684 2022-03-09 00:55:49

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 29,442
Website

Re: Post your handy self made command line utilities

In that case, you should probably use awk's -F option as below.  I also threw in a sed version if you don't care about the order of the fields (and just used a google ip for demonstration purposes):

#!/bin/sh

curl -s "https://ipapi.co/8.8.8.8/yaml" | awk -F': ' '
	/ip:/ { ip=$2 }
	/asn:/{ asn=$2 }
	/city:/{ city=$2;}
	/country_name:/ { country=$2}
	/org:/{ printf "%s, %s, %s, %s, %s\n", ip, asn, $2, city, country; }'

curl -s "https://ipapi.co/8.8.8.8/yaml" | sed -n '
	/^\(ip\|asn\|city\|country_name\|org\):/{s/[^:]*: //;H;}
	${x;s/\n//;s/\n/, /g;p;}'

And once you used the colon as the field separator, it becomes even easier:

curl -s "https://ipapi.co/8.8.8.8/yaml" | awk -F': ' '{ v[$1]=$2; }
	END { printf "%s, %s, %s, %s, %s\n", v["ip"], v["asn"], v["org"], v["city"], v["country_name"]; }'

Last edited by Trilby (2022-03-09 00:58:58)


"UNIX is simple and coherent..." - Dennis Ritchie, "GNU's Not UNIX" -  Richard Stallman

Offline

#3685 2022-03-09 13:20:14

Tarqi
Member
From: Ixtlan
Registered: 2012-11-27
Posts: 179
Website

Re: Post your handy self made command line utilities

I missed the field separator somehow... and the solution with the "dynamic dictionary" is great, thanks for that.


Knowing others is wisdom, knowing yourself is enlightenment. ~Lao Tse

Offline

#3686 2022-04-07 13:09:16

xyproto
Package Maintainer (PM)
From: Oslo
Registered: 2011-01-11
Posts: 43
Website

Re: Post your handy self made command line utilities

Entering a directory via a symlink, and not quite sure where you really are on the file system?

The really alias that does cd "`pwd -P` has been useful to me.

For easy copy&paste to ie. .zshrc:

alias really="cd \"\`pwd -P\`\""

(yes, the backticks can probably be replaced with $())

Offline

#3687 2022-04-07 13:35:38

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 29,442
Website

Re: Post your handy self made command line utilities

The backticks and backslashes can all be removed:

alias really='cd "$(pwd -P)"'

However, that readlly doesn't do anything at all.  The only result I could imagine it having is changing how the current working directory is displayed in a prompt string if you use it there.  But if what you really want in the prompt string is the actual path with symlinks resolved, you don't need an alias that you'll run manually, just write your prompt command to show what you actually want in the first place.

Last edited by Trilby (2022-04-07 13:39:17)


"UNIX is simple and coherent..." - Dennis Ritchie, "GNU's Not UNIX" -  Richard Stallman

Offline

#3688 2022-04-07 14:24:52

schard
Member
From: Hannover
Registered: 2016-05-06
Posts: 1,932
Website

Re: Post your handy self made command line utilities

cd -P .

Offline

#3689 2022-06-27 16:47:39

clayrisser
Member
Registered: 2022-06-27
Posts: 1

Re: Post your handy self made command line utilities

My favorite script I've written is a simple and elegant templating engine.

tmpl.sh

#!/bin/sh

EOF=EOF
exec cat <<EOF | sh
cat <<EOF
$(cat $1 | \
    sed 's|\\|\\\\|g' | \
    sed 's|`|\\`|g' | \
    sed 's|\$|\\\$|g' | \
    sed "s|${OPEN:-<%}|\`eval echo |g" | \
    sed "s|${CLOSE:-%>}| 2>/dev/null \`|g")
$EOF
EOF

You can read more about it at the link below.

https://stackoverflow.com/a/72775596/5179133

Last edited by clayrisser (2022-06-27 16:56:22)

Offline

#3690 2022-07-22 08:51:23

tsallinia86
Member
Registered: 2021-04-17
Posts: 6

Re: Post your handy self made command line utilities

Hi all,

First a disclaimer: I am kind of newbie, so apologies to all the programmers and power users out there.

As a user I hate find and prefer locate to research for files from my terminal. However, to get the best possible results you need to perform updatedb often. Starting it as a systemd service was not an option as it adds to the boot up time. Plus, some days I just don't need to search for files, so updatedb is not needed. Hence, I created my own script that updatesdb on demand and while doing so you just type in the name of the file you want to be searched. My command is performed by typing search The search is not case sensitive.

Simply copy this code to a file, chmod 777, and paste to /usr/local/bin

 
#! /bin/bash
sudo -s updatedb && read varname &&  locate -e -i $varname
 

Last edited by tsallinia86 (2022-07-22 09:06:08)

Offline

#3691 2022-07-22 11:54:25

2ManyDogs
Forum Moderator
Registered: 2012-01-15
Posts: 4,645

Re: Post your handy self made command line utilities

Topics merged.


How to post. A sincere effort to use modest and proper language and grammar is a sign of respect toward the community.

Offline

#3692 2022-07-22 12:39:32

kokoko3k
Member
Registered: 2008-11-14
Posts: 2,390

Re: Post your handy self made command line utilities

tsallinia86 wrote:

Hi all,

First a disclaimer: I am kind of newbie, so apologies to all the programmers and power users out there.

As a user I hate find and prefer locate to research for files from my terminal. However, to get the best possible results you need to perform updatedb often. Starting it as a systemd service was not an option as it adds to the boot up time. Plus, some days I just don't need to search for files, so updatedb is not needed. Hence, I created my own script that updatesdb on demand and while doing so you just type in the name of the file you want to be searched. My command is performed by typing search The search is not case sensitive.

Simply copy this code to a file, chmod 777, and paste to /usr/local/bin

 
#! /bin/bash
sudo -s updatedb && read varname &&  locate -e -i $varname
 

You're defeating the purpose of locate and turned it into a crippled (and less efficient than the) find that you hate or am i missing something here?


Help me to improve ssh-rdp !
Retroarch User? Try my koko-aio shader !

Offline

#3693 2022-07-22 12:50:26

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 29,442
Website

Re: Post your handy self made command line utilities

tsallinia86 wrote:

Starting it as a systemd service was not an option as it adds to the boot up time.

Then you created your service incorrectly.  This should be triggered by a systemd timer which could have a delay after boot and could run periodically from then on.  In fact the mlocate package comes with a timer that should do just this.


"UNIX is simple and coherent..." - Dennis Ritchie, "GNU's Not UNIX" -  Richard Stallman

Offline

#3694 2022-07-22 13:44:38

dogknowsnx
Member
Registered: 2021-04-12
Posts: 648

Re: Post your handy self made command line utilities

tsallinia86 wrote:

I am kind of newbie, so apologies to all the programmers and power users out there

Don't apologize - just keep learning...

https://wiki.archlinux.org/

tsallinia86 wrote:

As a user I hate find

Your aversion is likely rooted in ignorance...

man find
tsallinia86 wrote:

Simply copy this code to a file, chmod 777, and paste to /usr/local/bin

You don't want your scripts to be write- and executable by the whole world...

https://wiki.archlinux.org/title/File_p … attributes


Notifications for Arch Linux package updates
RI - Rest your Eyes and Self

"We are eternal, all this pain is an illusion" - Maynard James Keenan

Offline

#3695 2022-07-23 08:01:37

tsallinia86
Member
Registered: 2021-04-17
Posts: 6

Re: Post your handy self made command line utilities

Thank you for your feedback. I don't think my laptop performs updatedb periodically. I have checked this morning after I created a random file and locate -e -i still can't find it.

In what way is the script crippled? I t uses all the arguments I normally use when I invoke locate.

Offline

#3696 2022-07-23 09:55:30

Alad
Wiki Admin/IRC Op
From: Bagelstan
Registered: 2014-05-04
Posts: 2,407
Website

Re: Post your handy self made command line utilities

Find supports many more things than just searching by file names (by permissions, owner, different path depths, and so on) so locate can hardly replace it.

Maybe you might like "fd", it advertises itself as "user-friendly": https://github.com/sharkdp/fd


Mods are just community members who have the occasionally necessary option to move threads around and edit posts. -- Trilby

Offline

#3697 2022-08-13 17:18:31

CarbonChauvinist
Member
Registered: 2012-06-16
Posts: 412
Website

Re: Post your handy self made command line utilities

So, in the depressingly decreasing free time I can carve out, I sometimes plod away at various small scripts that interest me in the hopes of better learning bash scripting, purely for enjoyment. With that as a backdrop...

I have a script I'm using to scrape the National Geographic picture of the day. It works fairly well for my needs considering my relative inexperience with Bash. I guess the one thing I was hoping for some suggestions on is how to better handle character set encoding issues I guess?

My script basically scrapes the html of the page for the image description and the actual file/image link buried in the code. The image is downloaded and the image description is appended to a txt file, an index of sorts. I then confirm that both happened successfully and display the image and print the description to the screen. But when a character has accents or the like I get what looks like a the html utf instead of the character it represents? Example below which in this case is just supposed to be an apostrophe:

~ $ ng
NationalGeographic_1215640.jpg downloaded successfully
"A person prays at Tibet&#x27;s most sacred temple, the Jokhang, in this image from our photography archives. written to index successfully
2022-08-13: "A person prays at Tibet&#x27;s most sacred temple, the Jokhang, in this image from our photography archives.

What can I do to address this? Are there any other general recommendations to improve the overall script?

Here's the script btw:

#!/usr/bin/env bash

# save root photo of the day website
readonly POTD_URL="https://www.nationalgeographic.com/photo-of-the-day"

# store destination directory and indexfile in vars
readonly DEST_DIR="$HOME/Pictures/NGWallpapers"
readonly INDEX_FILE="$DEST_DIR/image-details.txt"

# function to display newest dated file from destination directory
# use rg when outputting decription line in order to highlight date in red
ngpic() {
  local newest
  read -r newest <<<"$(ls -t "$DEST_DIR"/*.jpg)"
  imv-wayland -f "$newest" && tail -1 "$INDEX_FILE" | rg '^[0-9]{4}-[0-9]{1,}-[0-9]{1,}'
}

# array to hold html tags for parsing imgurl and desc
declare -a data

# search for 'meta data-react-helmet' lines and add to data array for parsing
while read -r -d '>' LINE; do
  if [[ $LINE =~ "<meta data-react-helmet" ]]; then
    data+=("$LINE")
  fi
done <<<"$(curl -s "$POTD_URL")"

# extract image description
image_desc=$(grep "name=\"description\" content=\"" <(printf "%s\n" "${data[@]}"))
image_desc=${image_desc#*content=}
image_desc=${image_desc%\"/*}

# extract line containing image url
image_url_line=$(grep "\"og:image\" content=\"" <(printf "%s\n" "${data[@]}"))

# extract image url
image_url=${image_url_line#*content=\"}
image_url=${image_url%.jpg*}.jpg

# extract image filename only
image_file=${image_url##*/}

# download current wallpaper to tmpfile
TMPFILE=$(mktemp) || exit 3
curl -s "$image_url" >"$TMPFILE"
trap 'rm -f "$TMPFILE"' EXIT

# sanity checks
if [[ ! -d $DEST_DIR ]] || [[ ! -f $INDEX_FILE ]]; then
  echo "check your dir or file, something's off" && exit 1
elif [[ -e "$DEST_DIR/$image_file" ]]; then
  # TODO - find other way to compare files that doesn't rely on cmp
  if cmp -s "$TMPFILE" "$DEST_DIR/$image_file"; then
    echo "$image_file already exists"
    ngpic
    exit 2
  fi
fi

# if passes all sanity checks then proceed to copying to Pictures folder
# and appending to imagefile index
cp "$TMPFILE" "$DEST_DIR/$image_file"
[[ -e "$DEST_DIR/$image_file" ]] && echo "$image_file downloaded successfully"

# append desc to indexfile
if echo "$(printf '%(%Y-%m-%d)T'): $image_desc" >>"$INDEX_FILE"; then
  echo "$image_desc written to index successfully"
fi

Last edited by CarbonChauvinist (2022-08-13 18:21:33)


"the wind-blown way, wanna win? don't play"

Offline

#3698 2022-08-13 19:10:54

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 29,442
Website

Re: Post your handy self made command line utilities

First, wouldn't the following do the same without all the excess tests and redirection steps:

#!/bin/sh

url=https://www.nationalgeographic.com/photo-of-the-day
destdir=$HOME/Pictures/NGWallpapers
index=$destdir/image-details.txt

eval "$(curl -s $url | sed -n '
	/meta data-react-helmet/H
	$ {
		g; s/.*name="description" content="\([^"]*\).*/image_desc="\1"/p
		g; s/.*property="og:image" content="\([^"]*\).*/image_url="\1"/p
	}
')"

grep -q "$image_desc" $index && exit

date -f "%Y-%m-%d: $image_desc" >> $index
curl -s -O --outdir $destdir --create-dirs "$image_url"
imv-wayland -f "$(ls -t "$destdir/*.jpg")"

"UNIX is simple and coherent..." - Dennis Ritchie, "GNU's Not UNIX" -  Richard Stallman

Offline

#3699 2022-08-13 19:15:20

seth
Member
Registered: 2012-09-03
Posts: 49,966

Re: Post your handy self made command line utilities

… | recode html..utf-8

https://archlinux.org/packages/extra/x86_64/recode/
Afaik iconv can't do this.

Offline

#3700 2022-08-14 01:57:23

CarbonChauvinist
Member
Registered: 2012-06-16
Posts: 412
Website

Re: Post your handy self made command line utilities

Thank you both for your time and input!

@Seth, recode works perfectly as a drop in and in initial testing handles the conversion just as I'd hoped, thanks a lot!

@Trilby, your tag really should be changed from 'Inspector Parrot' to 'Parsimonious Parrot' - I will need some time to digest the sed-fu you've suggested and may just change over to that logic instead once I feel I've fully wrapped my head around it (I still use some throw-away python you'd suggested in a previous thread to rank mirrors by speed as replacement for rankmirrors/reflector btw).

TBH I don't use sed that much, and though it may not be immediately apparent, my script is somewhat contrived on purpose in the sense that I was trying to practice prioritizing using bash builtins where possible (i.e string manipulations, printf for dates/times, etc.) and wanted to limit additional binaries (even though I used a fair number including grep, cmp, etc.)

Last edited by CarbonChauvinist (2022-08-14 02:01:38)


"the wind-blown way, wanna win? don't play"

Offline

Board footer

Powered by FluxBB