You are not logged in.
I've just started playing with Linux again for the upteenth time, and I think this time it might stick. One of my favourite things is the powerful bash shell and the ease with which you can manipulate data using it.
It's really easy to make scripts to manipulate data using pipes and whatnot, here are a few of mine:
pyline
Take a statement as a parameter (enclosed in quotes), and execute it for each line on STDIN. Can use this for basic text manipulation. Could be extended to create additional context for processing. I know there we have awk and sed for this kind of thing but eh, I know python
#!/usr/bin/python
import sys
i = 0
for line in sys.stdin:
line = line[:-1]
exec sys.argv[1]
i += 1
eg print every third line:
cat txtfile | pyline "if i%3: print line"
try
Try a command until it executes successfully. Optionally takes the number of times to try until it is successful.
#!/bin/bash
COUNT=-1
if [[ $1 =~ ^[0-9]+$ ]]; then
COUNT=$1
shift
fi
STATUS=0
while [ "$COUNT" -ne 0 ]; do
let COUNT-=1
$*
STATUS=$?
if [ $STATUS -eq 0 ]; then
exit $STATUS
fi
done
exit $STATUS
eg to attempt to connect to a wireless network 5 times (sometimes dhcp times out):
try 5 netcfg mywireless
Post some of yours!
edit: bugfix
Last edited by Mashi (2008-10-10 01:11:19)
Offline
Very nice idea. If I find anything I've got that may be useful to anyone, I'll join in.
eg to attempt to connect to a wireless network 5 times (sometimes dhcp times out):
Then you should increase your DHCP timeout in your profile
Last edited by rson451 (2008-10-09 19:43:22)
archlinux - please read this and this — twice — then ask questions.
--
http://rsontech.net | http://github.com/rson
Offline
In my crontab on my machines - checks all mounted partitions for free disk space
#!/bin/sh
### Monitor free disk space
# Send an email to $ADMIN, if the free percentage of space is >= $ALERT
#
# Static Config Variables
#
ADMIN="<<<YOUR@EMAIL.COM>>>" # email to recieve alerts
ALERT=80 # free space percentage to trigger an alert
#
# Static Binary Paths
#
DF='/bin/df'
GREP='/bin/grep'
AWK='/bin/awk'
CUT='/bin/cut'
MAIL='/usr/bin/mail'
HOSTNAME='/bin/hostname'
DATE='/bin/date'
#
# Static System Variables
#
THIS_HOST=`${HOSTNAME}`
#
# Check CLI Options
#
VERBOSE=false
for ARG in "$@" ; do
case $ARG in
"-v")
VERBOSE=true
;;
esac
done
[ $VERBOSE = true ] && echo "Checking free disk space on ${THIS_HOST}"
[ $VERBOSE = true ] && echo "Threshold for warning is ${ALERT}%"
[ $VERBOSE = true ] && echo "------------------------------------------------------------------"
# Dynamic Variables
DATE_STR=`${DATE} "+%d-%B-%y @ %H%Mhrs"`
# Call df to find the used percentages. Grep for only local disks (not remote mounts like nfs or smb)
# Pipe the output to awk to get the needed columns, then start a while loop to process each line.
$DF -HPl | $GREP -E "^/dev/" | $AWK '{ print $5 " " $6 " " $1 }' | while read OUTPUT ; do
USED_PCENT=$(echo ${OUTPUT} | $AWK '{ print $1}' | $CUT -d'%' -f1 ) # Used space as a percentage
PARTITION=$(echo ${OUTPUT} | $AWK '{ print $2 }' ) # Mount Point (eg, /home)
DEVICE=$(echo ${OUTPUT} | $AWK '{ print $3 }' ) # Device (eg, /dev/sda1 or LABEL or UUID)
if [ $VERBOSE = true ] ; then
echo -e "Checking device ${DEVICE} which is mounted to ${PARTITION} \t${USED_PCENT}% used"
fi
if [ ${USED_PCENT} -ge $ALERT ]; then
if [ $VERBOSE = true ] ; then
echo "ALERT: ${PARTITION} (${DEVICE}) is ${USED_PCENT}% full on ${THIS_HOST} (${DATE_STR})"
else
echo "ALERT: ${PARTITION} (${DEVICE}) is ${USED_PCENT}% full on ${THIS_HOST} (${DATE_STR})" |
$MAIL -s "SERVER DISK SPACE LOW" $ADMIN
fi
fi
done
exit 0
Are you familiar with our Forum Rules, and How To Ask Questions The Smart Way?
BlueHackers // fscanary // resticctl
Offline
pyftp - Very, very simple FTP client. It helps me to upload files faster to my FTP server.
#!/usr/bin/env python
import ftplib
import os
import sys
if len(sys.argv) < 2:
print "Usage: %s file [file2] [file3] ..." % os.path.split(sys.argv[0])[1]
sys.exit()
# Clear the screen?
#os.system('clear')
print "Files to upload:"
print sys.argv[1:], "\n"
ftp = ftplib.FTP('myHost')
login = "myLogin"
passwd = "myPassword"
ftp.login(login, passwd)
ftp.dir()
# Move to the desired upload directory.
dir = raw_input("\nType the directory name (leave empty to use the current dir): ")
if dir:
ftp.cwd(dir)
print "\nCurrently in:", ftp.pwd()
for file in sys.argv[1:]:
name = os.path.split(file)[1]
print "Uploading \"%s\" ..." % name,
f = open(file, "rb")
ftp.storbinary('STOR ' + name, f)
f.close()
print "OK"
print "Quitting..."
ftp.quit()
Offline
Well for one, I wrote pkgfile, which I think many people use now
Hmm... pyline. Boo to that!
#!/usr/bin/perl
use warnings;
use strict;
#while (my $line = <STDIN>) { # Cleaner
while (<>) # Easier, uses $_ instead of $line
exec "$ARGV[0]";
}
__END__
foo | plline 'print if $. % 3'
Or simply
#!/usr/bin/perl -n
exec "$ARGV[0]"
Here we go.. I found a script I wrote to generate some very basic MPD stats
#!/usr/bin/perl
use strict;
use warnings;
open( MPC, "-|", "mpc playlist --format '%time% %artist%'" )
or die "Unable to open mpc: $!\n";
my %stats;
while (<MPC>) {
chomp;
my ($min, $sec, $artist) = /^ \d+\) (\d+):(\d+) (.*)/ or next;
$stats{$artist}->{time} += $min * 60 + $sec; # Time is in seconds
$stats{$artist}->{number}++;
}
foreach my $artist (sort {$stats{$b}->{time} <=> $stats{$a}->{time}} keys %stats) {
my @parts = gmtime($stats{$artist}->{time});
printf ("%0.2d:%0.2d:%0.2d ", $parts[7] * 24 + $parts[2], @parts[1,0]); # HH:MM:SS
printf "%0.3d %s\n", $stats{$artist}->{number}, $artist;
}
Last edited by Daenyth (2008-10-10 02:33:30)
[git] | [AURpkgs] | [arch-games]
Offline
Beware! Here's my amazingly advanced spam-mpd-now-playing script:
#/bin/bash
mpc | head -1 | xclip
Offline
#!/bin/bash
mpc --format "http://my-server.tld/Musicroot/%file%" | head -n 1 | sed 's/ /%20/g'
Previously, I used the above, but rather than adding more seds to make it "safe", I extended it a bit:
#!/usr/bin/perl -w
use strict;
use URI::Escape;
my $filename = `mpc --format %file% | head -n 1`;
chomp $filename;
my $baseurl = 'http://my-server.tld/Musicroot/';
print $baseurl, uri_escape($filename), "\n";
EDIT: Wow I totally meant to make this as a new post, overwrote all my old one that I thought I was quoting... lame
#!/bin/bash
mpc --format "np: [[%artist%] - [%title%] - #[[%album%] ##[%track%]#]]|[%file%]" | head -n 1
I put that in my PATH and then I can use it for irssi, pidgin, etc, through /exec.
Last edited by Daenyth (2008-11-19 03:03:33)
[git] | [AURpkgs] | [arch-games]
Offline
Kill gnome-screensaver before starting mplayer
#!/usr/bin/env python
import os
import os.path
import sys
os.system("killall gnome-screensaver")
command = '/usr/bin/mplayer %s "%s"' % (' '.join(sys.argv[1:-1]), os.path.normpath(sys.argv[-1]))
os.system(command)
os.system("gnome-screensaver")
I still do not understand the "command =" line but it works
Offline
I have a small box that i use for ices live stream encoding on our local studentradio, whenever broadcasting from outside the studio.
It has no graphics card, so in order to figure out which ip addres it gets from dhcp i have this script running when it recieves a new ip address.
It uses a bunch of wav files I found somewhere on the net with some lady pronouncing numbers.
#!/bin/bash
data="/opt/ipread/data"
ip=$1
let size=`expr length $ip`-1
index=0
for i in `seq 0 $size`; do
if [ ${ip:$i:1} = '.' ]; then
array[$index]="$data/$buffer.wav"
let index=$index+1
array[$index]="$data/point.wav"
let index=$index+1
unset buffer
else
buffer=$buffer${ip:$i:1}
fi
done
array[$index]="$data/$buffer.wav"
sox ${array[@]} /tmp/output.wav && play /tmp/output.wav 2> /dev/null && rm /tmp/output.wav
Last edited by divineant (2008-10-10 05:08:40)
Offline
It has no graphics card, so in order to figure out which ip addres it gets from dhcp i have this script running when it recieves a new ip address.
It uses a bunch of wav files I found somewhere on the net with some lady pronouncing numbers.
Wow that looks really fun! Actually what came to mind was number stations, I think it would be really freaky to have your computer spell out it's IP with numbers from a numbers station (which tend to sound scary).
Offline
Here's another one:
ptimer {command}
times a process
#!/bin/bash
STARTTIME=$(date "+%s.%N")
$*
PROCESSTIME=$(echo "$(date +%s.%N)-$STARTTIME" | bc)
echo "Process took $PROCESSTIME seconds."
Offline
A small script for setting CPU scaling governor. It requires /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor to be writable by whoever is going to use this script. I set it writable in rc.local.
#!/bin/bash
if test "$1" = "get"
then
cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor
elif test "$1" = "set"
then
if test "$2" = "powersave" || test "$2" = "conservative" || \
test "$2" = "ondemand" || test "$2" = "performance"
then
echo -n "$2" > /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor
else
echo "Unknown governor."
fi
else
echo "Usage:"
echo " $(basename $0) get"
echo " $(basename $0) set powersave|conservative|ondemand|performance"
fi
I call it "scaling", in that case usage is for instance:
$ scaling get
ondemand
$ scaling set powersave
Offline
tmail.sh --> script for sending mails from terminal with thunderbird. Sort of old school mail.
#!/bin/bash
USAGE="tmail usage: tmail -t TO -c CC -s SUBJECT -b BODY -a ATTACHMENT"
while getopts "t:c:s:b:a:" options; do
case $options in
t ) TO=$OPTARG;;
c ) CC=$OPTARG;;
s ) SUBJECT=$OPTARG;;
b ) BODY=$OPTARG;;
a ) ATTACHMENT=$OPTARG;;
\? ) echo $USAGE
exit 1;;
* ) echo $USAGE
exit 1;;
esac
done
thunderbird -compose to=\'$TO\',cc=\'$CC\',subject=\'$SUBJECT\',body=\'$BODY\',attachment=\'file:///$ATTACHMENT\'
say.sh --> like osx "say" command
#!/bin/bash
string_to_say=""
for i in $@;
do
string_to_say="$string_to_say $i"
done
echo $string_to_say | festival --tts
lamp.sh --> starts lamp . I'm a kind of web developer.
#!/bin/bash
#
#This script manages mysql and httpd deamons
if [ $1 = 'start' -o $1 = 'stop' -o $1 = 'restart' ]
then
sudo /etc/rc.d/httpd $1
sudo /etc/rc.d/mysqld $1
else
echo
echo Please specify action: start, stop or restart
echo
exit 1
fi
I have a bunch of other scripts I use to automate some tasks but they are not so interesting because they are written to do some specific tasks. They are not general purpose scripts.
EDIT: misspelled word...I like English but it's not my mother tongue...sorry for mistakes.
Last edited by ArchArael (2008-10-13 08:08:21)
Offline
Ok, this monster is called 'wine-guard', and is used by some one-liner launcher scripts that launch d3d wine games. It often ( ) restores correct gamma and resolution after the game exits (or crashes).
#!/bin/bash
#usage:
# wine-guard [-m mode] [-d] program
#
# options:
# -m: Use xrandr to switch to the mode given, before invoking wine
# -d: Turn on Wine debugging output
#
#settings
SCREEN="VGA-0"
#this is normally fine, but you can change it to a --mode cmd if you want
RESET_CMD="--preferred"
#the default resolution to run windows games in.
#If you normally dont want to change the resolution, leave this setting blank.
#A per-game resolution can be specified on the command line, using the -m option
MODE=
DEBUG="-all" #no debug output pr default
################
################
################
while getopts m:d o
do
case "${o}" in
m) MODE=${OPTARG} ;;
d) DEBUG="all" ;;
[?]) echo "Unknown option." ;;
esac
done
shift $(($OPTIND-1))
# save gamma value
XGAMMA=$(xgamma 2>&1 | sed -e "s/.*Red \(.*\), Green \(.*\), Blue \(.*\)/\/usr\/bin\/xgamma -rgamma\1 -ggamma\2 -bgamma\3/")
#if specified, set resolution
if [ -n "${MODE}" ]
then
echo "Setting mode $MODE"
xrandr --output "${SCREEN}" --mode "${MODE}"
fi
#this function resets resolution and gamma
reset() {
xrandr --output "${SCREEN}" "${RESET_CMD}"
if [ -n "$XGAMMA" ]
then
exec "${XGAMMA}"
fi
}
trap reset EXIT
WINEDEBUG="${DEBUG}" wine "$@"
I didn't use it for a while, but I think it should still work.
Offline
Here's another one:
ptimer {command}
times a process#!/bin/bash STARTTIME=$(date "+%s.%N") $* PROCESSTIME=$(echo "$(date +%s.%N)-$STARTTIME" | bc) echo "Process took $PROCESSTIME seconds."
You do realize, that there already is the 'time'-command which any shell that I know of provides, for example:
$ > time echo '2^223456' | bc > /dev/null
echo '2^223456' 0.00s user 0.00s system 0% cpu 0.001 total
bc > /dev/null 0.64s user 0.02s system 98% cpu 0.666 total
Offline
My server and another I look after run this backup script via cron job
If you have a file called exclude in the root of your backup folder, it will exclude any files specified in the file.
I usually set cron to send me the log file later in the day using sendmail.
It is very basic, but I'm proud of it
syntax = backup /path/of/backup /path/of/log
#!/bin/bash
#backup script for $1
rm $2/backup*.txt
FN="$2/backup-`date +%d-%m-%y`.txt"
STIME=`date +%s`
export FN STIME
echo Setting backup options > $FN
mt -f /dev/st0 defcompression 1
mt -f /dev/st0 defdrvbuffer 1
mt -f /dev/st0 stoptions buffer-writes read-ahead async-writes scsi2logical can-bsr auto-lock
echo - >> $FN
echo Starting backup of $1 on `date` >> $FN
echo - >> $FN
tar cvpPWf /dev/st0 $1 -X $1/exclude >> $FN
ETIME=`date +%s`
export ETIME
BTIME=`expr $ETIME / 60 - $STIME / 60`
export BTIME
echo Backup of $1 complete and verified in $BTIME minuites>> $FN
#mt -f /dev/st0 offline
Offline
Here's another one.
I needed to send some information from one server to another, and i couldn't use ssh so instead i sent it using HTTP GET to a php script.
That so easy if i had access to curl on the server, but that was not the case, instead I made this telnet script.
#!/bin/bash
if [ -z "$1" ]; then
echo "Usage:"
echo "$0 var1=value1 ... varN=valueN"
exit
fi
host='hostname'
port='80'
reciever='/file/to/recieve/get/request.php'
retry=10
for i in ; do
if [ -z "$params" ]; then
params=$i;
else
params=$params'&'$i
fi
done
cmd="GET $reciever?$params /HTTP/1.1\nHost: \n"
i=0
while [ "$output" != "statusUpdate OK" -a $i -ne $retry ] ; do
let i=+1
output=`( echo open ${host} ${port}
/bin/perl -e 'sleep .1'
echo ${cmd}
/bin/perl -e 'sleep .1'
echo -e "\n"
/bin/perl -e 'sleep .1'
echo exit) | telnet | grep statusUpdate` 2> /dev/null
echo $i;
sleep 1;
done
Last edited by divineant (2008-10-11 17:28:39)
Offline
Here is my backup script. It's designed for daily use and saves the backups in a directory named after the date. It works with hardlinks, so there is a directory for every day, but not that much disk space is used.
#!/bin/bash
# Edit parameters here.
# Important: Don't forget the trailing slashes in SOURCES and TARGET
SOURCES="/etc/ /boot/ /home/rine/"
TARGET="/media/e/archbackup/"
OPTIONS="--exclude-from=/home/rine/documents/scripts/exclude.list"
### Don't edit below ###
LASTBACKUP=`ls -d $TARGET* | sort -r | head -1`
TODAY=`date +%y%m%d`
if [ "$LASTBACKUP" == "$TARGET$TODAY" ]; then
echo "Backup for today is done, if there were errors, delete $TARGET$TODAY first"
exit 1
fi
for src in $SOURCES
do
if [ "$LASTBACKUP" ]; then
LD="--link-dest=$LASTBACKUP$src"
fi
mkdir -p $TARGET$TODAY$src
rsync -auv $OPTIONS $LD $src $TARGET$TODAY$src
done
When you want to exclude files, it's important to start them with a slash in exclude.list. If, for example, you want to exclude your "movies" directory, you type /movies. Otherwise, if you happened to have a file called movies somewhere else, it also would get excluded. Example exclude.list:
/.adobe
/.alsaplayer
/.cache
/.dbus
/.fontconfig
/.gstreamer-0.10
/.java
/.local
/.macromedia
/.mcop
/.mozilla
/.openoffice.org2
/.qt
/.quodlibet
/.session
/.texlive
/.texmf-var
/.thumbnails
/.xine
/serien
/tmp
Comments for improvement welcome.
Offline
This one deletes all files with the some extensions in a directory and its subdirectories:
#!/usr/bin/env python
import os
import sys
types = [".rar",".zip",".ace"]
def recurseDirectory(dirname):
list = os.listdir(dirname);
for item in list:
fn = dirname+os.path.sep+item
if os.path.isfile(fn):
#if item[len(item)-4:] in types:
found = False
for ext in types:
if item[len(item)-len(ext):] == ext:
found = True
break
if found:
try:
#os.remove(fn)
os.system("rm -i \""+fn+"\"")
except OSError, e:
print "Error: "+str(e)
#print "Deleting "+fn
elif os.path.isdir(fn):
recurseDirectory(fn)
return
if len(sys.argv) != 2:
print "USAGE: python "+sys.argv[0]+" directory"
sys.exit(1)
recurseDirectory(sys.argv[1])
(Would probably be shorter as a shell-script, though)
Last edited by Asgaroth (2008-10-11 19:57:06)
Offline
This one deletes all files with the some extensions in a directory and its subdirectories:
...
(Would probably be shorter as a shell-script, but I feel more comfortable with python)
find /path/in/which/to/delete -name '*.ext' -exec rm {} +
should do the trick
Offline
I definitely should read the find manpage, until now I only used to _find_ files.
Last edited by Asgaroth (2008-10-11 20:25:26)
Offline
I usually use (but maybe it is the same)
find /path/in/which/to/delete -name '*.ext' -exec rm -f "{}" \+
and if I do not need the `force' option of rm I use
find /path/in/which/to/delete -name '*.ext' -delete
Edit:
About the topic
http://aur.archlinux.org/packages.php?ID=17934
Last edited by ezzetabi (2008-10-12 09:02:45)
Offline
Change the bitrate of mp3 files. This one can be adapted to do almost everything on multiple files...
#! /usr/bin/env perl
use File::Find;
sub wanted {
if (/.mp3$/)
{
($n) = /(.*).mp3$/ ;
system('lame',"-b 64",$_,"$n"."_64.mp3","-S")
}
}
find(\&wanted, ".");
Massive replacement of a string in text files.
#! /usr/bin/env python
# cw_int_replace by catwell
# USAGE : cw_int_replace.py old_string new_string file1 [file2...]
# advice : for i in `find`; do cw_int_replace.py old_string new_string $i; done
# S : skip ; a : auto ; m : manual
from sys import argv
m_old = argv[1]
m_new = argv[2]
m_red = "\033[31m"+m_old+"\033[0m"
for cible in argv[3:]:
print "\033[1m== "+cible+" ==\033[0m"
lignes = open(cible).readlines()
F = open(cible,'w+')
for L in lignes:
S = L.split(m_old)
while len(S) != 1:
print S[0]+m_red+m_old.join(S[1:]).strip("\n"),
r = raw_input('[S/a/m] ')
if r in ('s',''):
S[0] = S[0]+m_old+S[1]
del S[1]
elif r == 'a':
S[0] = S[0]+m_new+S[1]
del S[1]
elif r == 'm':
S[0] = S[0]+raw_input('Replace with? ')+S[1]
del S[1]
A Perl one, a Python one, everybody's happy.
Last edited by catwell (2008-10-12 13:36:31)
Offline
I'm considering rewriting some of my utilities in Common Lisp as a learning exercise...
Offline
I'm considering rewriting some of my utilities in Common Lisp as a learning exercise...
Sorry for the OT, just an info.
About Common Lisp....are you also Emacs or Stumpwm user? I'm really fascinated about the idea of changing my working environment in real time.
I would like to learn more about it.
Offline