Post your handy self made command line utilities

Rasi · 2015-08-25 23:35:46

Alad wrote:

...you know, getopts is a thing.
http://mywiki.wooledge.org/BashFAQ/035#getopts

Yeah, should probably use it, not a fan of short options though

Last edited by Rasi (2015-08-25 23:42:43)

losko · 2015-08-26 09:03:51

Hello everyone, bfblock is a script I made to block IPs trying to brute-force an SSH server.
If, for some reason, you need to run SSH on port 22 and your server is subject to brute-force attacks, this script can be used as a minimalistic and ultra-light alternative to sshguard and fail2ban.

Brute-force attacks are very annoying, 99.9% of them are against root account (hopefully disabled for everyone). The major drawback is that the size of /var/log/btmp may become very huge and fill your partition.

Prerequisites: 'ufw' installed, enabled and running.

My approach to the problem is very simple, the attacker IP will be blocked 'for a while'...
The provided systemd service/timer runs the script every hour.
The script collects all the IPs failing to login in the last hour.
IPs with more than 100 failed attempts go 'in the blacklist'.
The blacklist is very short, just 10 items.
When a new IP is found the item #10 goes out of the list with a first-in/last-out logic.
Ufw rules are updated accordingly to the blacklist.
Then the /var/log/btmp is cleared.

I'm using it on real servers, in most cases it takes 3/4 days before an IP goes from 1st to 10th position (and then out), but this may not be the same for everyone.

/etc/systemd/system/bfblock.timer

[Unit]
Description=brute-force block service

[Timer]
OnCalendar=hourly
Persistent=false

[Install]
WantedBy=multi-user.target

/etc/systemd/system/bfblock.service

[Unit]
Description=brute-force block service

[Service]
Type=simple
ExecStart=/usr/bin/bfblock

/usr/bin/bfblock

#!/bin/bash

THRS_CNT=100
LIST_CNT=10

F_BTMP=/var/log/btmp
F_BBAK=/var/log/bbak.lst
F_BLCK=/var/log/bblk.lst

if [ ! -f $F_BLCK ]; then
  > $F_BLCK
fi

lastb | grep -o '[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}' > $F_BBAK

while [ -s $F_BBAK ]
do
  IP_GET=$(tail -1 $F_BBAK)
  IP_CNT=$(cat $F_BBAK | grep -c $IP_GET)

  if [ $IP_CNT -gt $THRS_CNT ]; then
    echo $IP_GET >> $F_BLCK
    ufw insert 1 deny from $IP_GET

    L_CNT=$(cat $F_BLCK | wc -l)

    if [ $L_CNT -gt $LIST_CNT ]; then
      IP_OUT=$(head -1 $F_BLCK)
      sed -i '1d' $F_BLCK
      ufw delete deny from $IP_OUT
    fi
  fi

  sed -i "/$IP_GET/d" $F_BBAK
done

> $F_BTMP

Make 'bfblock' executable:

# chmod +x /usr/bin/bfblock

Enable and/or start systemd service as usual:

# systemctl enable bfblock.timer
# systemctl start bfblock.timer

Now you have to wait until it's triggered at least once (or more).

To check the current blacklisted IPs (if any):

$ cat /var/log/bblk.lst

The list is upside-down, items at the bottom are the newest, at the top the oldest.

To check the denied IPs on the firewall (should match the blacklist, but in reverse order):

# ufw status

Further improvements:
- Reduce time between checks (1h -> 30 min)
- Lower the threshold (100 -> 50)
- Increase the blacklisted items (10 -> 20)
- Block whole subnet instead of single IP (11.22.33.44 -> 11.22.33.0/24)

I hope it helps.
Bye

Earnestly · 2015-08-26 10:45:05

# With nft
tcp dport 22 limit rate 3/minute log prefix "SSH Connection: " group 2 accept

# With iptables
-A INPUT -p tcp --dport 22 -m recent --name ssh --set
-A INPUT -p tcp --dport 22 -m recent --name ssh --rcheck --seconds 60 --hitcount 3 -j LOG --log-prefix "SSH connection: "
-A INPUT -p tcp --dport 22 -m recent --name ssh --rcheck --seconds 60 --hitcount 3 -j REJECT

betseg · 2015-08-26 11:02:25

fortune | cowsay | lolcat

losko · 2015-08-26 11:20:13

@Earnestly

another one:

# ufw limit SSH

Rate limiters can co-exist with my script:
3/min * 60 min = 180/hour
Will be blocked anyway...

Last edited by losko (2015-08-26 11:37:08)

x33a · 2015-08-26 11:33:28

Rasi wrote:

Alad wrote:
...you know, getopts is a thing.
http://mywiki.wooledge.org/BashFAQ/035#getopts
Yeah, should probably use it, not a fan of short options though

That is exactly the reason preventing me from using getopts. I like a combo of short and long options.

Earnestly · 2015-08-26 12:06:52

losko wrote:

@Earnestly
another one:
# ufw limit SSH
Rate limiters can co-exist with my script:
3/min * 60 min = 180/hour
Will be blocked anyway...

Except ufw is pretty ugly if you look at what it does with all the unnecessary chains it produces. Much nicer to get closer to the truth and use the core tools themselves if possible.

losko · 2015-08-26 12:42:37

Earnestly wrote:

use the core tools themselves if possible.

a good advice is always welcome

Alad · 2015-08-27 00:04:11

x33a wrote:

Rasi wrote:
Alad wrote:
...you know, getopts is a thing.
http://mywiki.wooledge.org/BashFAQ/035#getopts
Yeah, should probably use it, not a fan of short options though
That is exactly the reason preventing me from using getopts. I like a combo of short and long options.

Doesn't mean you should fallback to 20 lines of [[ $1 == --lol || $1 == -l ]].

http://mywiki.wooledge.org/ComplexOptionParsing

Trilby · 2015-08-27 00:16:54

Also note that getopts != getopt. If one isn't your cup of tea, check out the other.

esa · 2015-08-30 16:24:20

Easy way to get the ASCII value of a char/word:

charnum "'"
\047

Even:

charnum a b
\141
\142

charnum "a b"
\141\040\142

or

charnum "Hello World" "How are you?"
\110\145\154\154\157\040\127\157\162\154\144
\110\157\167\040\141\162\145\040\171\157\165\077

Scriptfile: charnum

#!/bin/bash
get_char_num() { # CHAR
# Parses numbers 0 - 999 and checks their result
# for matches against passed CHAR
	PRE=000
	MAX=999
	C=0
	while [ $C -lt $MAX ]
	do	num="${PRE:${#C}}$C"
		[ "$(printf \\$num)" = "$1" ] && \
			printf '%s' "\\$num" && \
			break
		((C++))
	done
}
parse_args() { # ARGS
# Parses each entry of ARGS as word
# and moves char by char to its length
	for word in "${@}"
	do	char=0
		while [ $char -lt ${#word} ]
		do 	get_char_num "${word:$char:1}"
			((char++))
		done
		printf '\n'
	done
}
case $# in
0)	echo "Usage: ${0##*/} \"'\"|x|a b|\"a b\"|hello world"
	exit 99
	;;
1)	case ${#1} in
	1)	get_char_num "$1"
		exit $?
		;;
	*)	parse_args "${@}"
		exit $?
		;;
	esac
	;;
*)	parse_args "${@}"
	exit $?
	;;
esac

Hope this helps & Have fun

Vain · 2015-08-30 17:17:52

Don't want to spoil the fun, but do you know about tools like "od"?

$ echo 'a b' | od -b
0000000 141 040 142 012
0000004

"od" can also create other output formats -- hex, decimal, ... Just have a look at the manpage.

Trilby · 2015-08-30 21:02:21

Or `man ascii`.

studentik · 2015-09-11 23:08:09

# pacman.gawk - Converts output of 'pacman -Qi' into SQLite  pacman table
# 
# Can be used in like this:
# shell> gawk -f ./pacman.gawk ./tmp > ./sql; sqlite3 -init ./sql
# sqlite> SELECT package, descr FROM pacman WHERE install_reason LIKE 'Explicit%' AND groups NOT LIKE '%xorg%' AND groups NOT LIKE '%base%' ORDER BY package;

BEGIN { 
 if (ARGC!=2) {
  print "Must have tmp file argument"
  exit
 }
 tmp=ARGV[ARGC-1] 
 command = "/usr/bin/pacman -Qi > " tmp
 if (system(command)!=0) {
  print "Error executing command: " command
  exit
 }
}

BEGINFILE {
 RS="\n\n"
 FS="\n"
 OFS=","
 print "DROP TABLE IF EXISTS pacman;"
 print "CREATE TABLE pacman (package, version, descr, arch, url, licenses, groups, provides, depends, optional_depends, required_by, optional_for, conflicts_with, replaces, installed_size, packager, build_date, install_date, install_reason, install_script, validated_by);"
}

# Optional Deps may be multiline
{ gsub("\n +", " ") }

{
 for (i=1;i<=NF;i++) {
  gsub("\"","",$i)
  $i = "\"" substr($i, index($i,":")+2) "\""
 }
 print "INSERT INTO pacman SELECT " $0 ";"
}

Last edited by studentik (2015-09-11 23:14:44)

bslackr · 2015-09-23 01:07:43

I needed something to add lines to the beginning of a file, so I added this to my .zshrc.

prepend() { 
    local pto="$1"
    local ptotemp=$(mktemp -t tmp.XXXXXX)
    shift
    echo "$@" | cat - "$pto" > "$ptotemp" &&
        mv "$ptotemp" "$pto" ||
        echo "Unable to comply."                                                
}

Trilby · 2015-09-23 01:25:33

sed -i '1i stuff to prepend' file

no need for a function

Or just append to the file, then move the last line to the first with ed:

echo -e '$m0\nwq\n' | ed filename

Earnestly · 2015-09-23 11:23:44

No one should ever use echo anymore. Ed can also be used similarly:

# Add H to the beginning if you want useful error reporting.
printf '%s\n' 1i "text to insert" . w | ed -s file

Note: ed is an editor, it will edit in place. Even with the option -i to sed, it will create a temporary file.

Last edited by Earnestly (2015-09-23 11:27:03)

basica · 2015-09-26 08:16:59

I hope this is allowed since this is for OSX..

I occasionally like to try out distros on my spare pc, this simplifies the process for when I want to put an ISO onto a USB when I'm using my mac.

#!/bin/bash

# iso2usb is a simple script for OSX to ease the conversion of an iso to an img file
# and then dd that img file to a USB of your choosing. You simply give it the iso
# as a parameter and then the disk number and it will take care of the rest.

# based on the commands here: http://www.ubuntu.com/download/desktop/create-a-usb-stick-on-mac-osx
# and the color commands here:  http://stackoverflow.com/questions/5947742/how-to-change-the-output-color-of-echo-in-linux

# exits out of the script upon error
set -e

# colors

# feel free to replace the color choices
# I made below with any of the following

BLACK='\033[0;30m'     
DGRAY='\033[1;30m'
RED='\033[0;31m'     
LRED='\033[1;31m'
GREEN='\033[0;32m'     
LGREEN='\033[1;32m'
ORANGE='\033[0;33m'     
YELLOW='\033[1;33m'
BLUE='\033[0;34m'     
LBLUE='\033[1;34m'
PURPLE='\033[0;35m'     
LPURPLE='\033[1;35m'
CYAN='\033[0;36m'     
LCYAN='\033[1;36m'
LGRAY='\033[0;37m'     
WHITE='\033[1;37m'
NC='\033[0m'

# beginning of actual script

if [ ! -z $1 ]; then
	printf "\nNOTE: If you do not see ${GREEN}## \
COMPLETED SUCCESSFULLY ##${NC} prior to the script \
exitingthen something has gone wrong and you will \
need to investigate further.\n"
	printf "\n${RED}DO NOT ASSUME THAT THE USB IS USABLE!${NC}\n\n"
	printf "${GREEN}## FILE CONVERSION ##${NC}\n\n"
	img=$(echo $1 | sed -e 's/iso/img/g')
	printf "original: ${CYAN}$1${NC}\n" 
	printf "new file: ${PURPLE}$img${NC}\n"
	read -p "proceed (y/n): " ans
	if [ "$ans" == "y" ]; then	
		hdiutil convert -format UDRW -o $img $1
		mv $img.dmg $img
	else
		printf "exiting script..\n"
		exit 0
	fi
	printf "\n${GREEN}## EJECT USB ##${NC}\n\n"
	diskutil list
	echo ""
	read -p "which disk is being ejected - enter number only: " usrinp
	printf "eject ${GREEN}/dev/disk$usrinp${NC} and img (y/n): " 
	read ans
	if [ "$ans" == "y" ]; then
		diskutil unmountDisk "/dev/disk$usrinp"
		printf "\n${GREEN}## IMAGING USB ##${NC}\n\n"
		printf "If requested, enter password to begin imaging..\n"
		sudo dd if="$img" of="/dev/rdisk$usrinp" bs=1m
		printf "\n${GREEN}## COMPLETED SUCCESSFULLY ##${NC}\n\n"
	else
		printf "exiting script..\n"
		exit 0
	fi	
else
	printf "${RED}ERROR:${NC} No file name given\n"
fi

Made the code cleaner by replacing the echo "" lines with newline characters per the advice below.

Last edited by basica (2015-09-27 08:18:11)

Alad · 2015-09-26 20:32:31

What's not allowed is this:

	echo ""
	printf "${RED}DO NOT ASSUME THAT THE USB IS USABLE!${NC}\n"
<repeated 20 times>

You seem to know escape sequences, so just put them at the front as well: printf '\nfoo\n' or repeat them: printf 'bar\n\n'

Last edited by Alad (2015-09-26 20:36:29)

Awebb · 2015-09-26 21:02:06

Earnestly wrote:

No one should ever use echo anymore.

Is there anything wrong with echo, besides not being very powerful?

Earnestly · 2015-09-26 22:45:18

Awebb wrote:

Earnestly wrote:
No one should ever use echo anymore.
Is there anything wrong with echo, besides not being very powerful?

Tidied up the output a little because sigh at everything web related.

<greybot> echo outputs a string. echo has many portability problems, and should never be used with option flags.
Consider printf instead: printf 'name: %s\n' "$name". 
http://wiki.bash-hackers.org/commands/builtin/echo
http://cfajohnson.com/shell/cus-faq.html#Q0b
http://www.in-ulm.de/~mascheck/various/echo+printf

Last edited by Earnestly (2015-09-26 22:46:28)

jasonwryan · 2015-09-27 00:31:04

See also Stéphane's answer here: http://unix.stackexchange.com/questions … -than-echo

basica · 2015-09-27 07:41:58

Alad wrote:

What's not allowed is this:
	echo ""
	printf "${RED}DO NOT ASSUME THAT THE USB IS USABLE!${NC}\n"
<repeated 20 times>
You seem to know escape sequences, so just put them at the front as well: printf '\nfoo\n' or repeat them: printf 'bar\n\n'

Of course, how silly of me No idea how that didn't occur to me at the time. Thanks for pointing it out.

basica · 2015-09-28 01:06:45

quequotion wrote:

Poor user's dynamic IP domain management with ssmtp:
E-mail your public IP address to user@domain.ext
#!/bin/bash

wget -qO- http://ipecho.net/plain | mail -s "$HOSTNAME woke up!" user@domain.ext
TODO: encrypt the address, ipv6?
NOTE: Remember, in a home or office network, the public IP address belongs to the router.
Systemd service to do this at boot:
[Unit]
Description=ip-drop
After=systemd-networkd-wait-online.service
Wants=systemd-networkd-wait-online.service

[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/usr/bin/bash "/usr/local/sbin/ip-drop"
StandardOutput=null
StandardError=journal

[Install]
WantedBy=multi-user.target
TODO: trigger on any DHCP cycle (how to detect from behind a router?)
NOTE: the *-wait-online services are likely to increase boot time.
I configured my router to forward SSH so I can access my home PC from wherever + installed mpd from a cellphone, on a bus .

Clever idea, and somewhat similar to what I did except was use the command below as it was a lot faster in retrieving the IP address. I also had a bit of a lazier route and just had a cron job to check if the ip had changed since last set and it checked every hour or so off the top of my head. Since then I purchased a new router and it can actually email you whenever your IP address changes which was far more convenient for me seeing that my IP address doesn't change all that often.

dig +short myip.opendns.com @resolver1.opendns.com

PrometheusBit · 2015-10-04 06:55:47

#!/usr/bin/env python

# importing modules
from __future__ import division
import math

# asking for user input
inputing = input("> ")
print inputing

# creating a loop
while abs(inputing):
	inputing = input("> ")
	print inputing

Really simple calculator that I've made quite a while ago. I know it can be improved and made shorter, but I'm not that good on Python, although I started to 'learn' it again.

Arch Linux

#2626 2015-08-25 23:35:46

Re: Post your handy self made command line utilities

#2627 2015-08-26 09:03:51

Re: Post your handy self made command line utilities

#2628 2015-08-26 10:45:05

Re: Post your handy self made command line utilities

#2629 2015-08-26 11:02:25

Re: Post your handy self made command line utilities

#2630 2015-08-26 11:20:13

Re: Post your handy self made command line utilities

#2631 2015-08-26 11:33:28

Re: Post your handy self made command line utilities

#2632 2015-08-26 12:06:52

Re: Post your handy self made command line utilities

#2633 2015-08-26 12:42:37

Re: Post your handy self made command line utilities

#2634 2015-08-27 00:04:11

Re: Post your handy self made command line utilities

#2635 2015-08-27 00:16:54

Re: Post your handy self made command line utilities

#2636 2015-08-30 16:24:20

Re: Post your handy self made command line utilities

#2637 2015-08-30 17:17:52

Re: Post your handy self made command line utilities

#2638 2015-08-30 21:02:21

Re: Post your handy self made command line utilities

#2639 2015-09-11 23:08:09

Re: Post your handy self made command line utilities

#2640 2015-09-23 01:07:43

Re: Post your handy self made command line utilities

#2641 2015-09-23 01:25:33

Re: Post your handy self made command line utilities

#2642 2015-09-23 11:23:44

Re: Post your handy self made command line utilities

#2643 2015-09-26 08:16:59

Re: Post your handy self made command line utilities

#2644 2015-09-26 20:32:31

Re: Post your handy self made command line utilities

#2645 2015-09-26 21:02:06

Re: Post your handy self made command line utilities

#2646 2015-09-26 22:45:18

Re: Post your handy self made command line utilities

#2647 2015-09-27 00:31:04

Re: Post your handy self made command line utilities

#2648 2015-09-27 07:41:58

Re: Post your handy self made command line utilities

#2649 2015-09-28 01:06:45

Re: Post your handy self made command line utilities

#2650 2015-10-04 06:55:47

Re: Post your handy self made command line utilities

Board footer