You are not logged in.

#626 2009-11-05 10:48:20

lolilolicon
Member
Registered: 2009-03-05
Posts: 1,722

Re: Post your handy self made command line utilities

@mikesd
nice one.
i'd recommend using X SECONDARY selection:

alias pud='pwd | xsel -s'
alias pod='cd "$(xsel -so)"'

since it is almost never used.
also note the double quotes in case path contains white spaces.

Last edited by lolilolicon (2009-11-05 10:51:21)


This silver ladybug at line 28...

Offline

#627 2009-11-05 10:53:08

mikesd
Member
From: Australia
Registered: 2008-02-01
Posts: 788
Website

Re: Post your handy self made command line utilities

lolilolicon wrote:

@mikesd
nice one.
i'd recommend using X SECONDARY selection:

alias pud='pwd | xsel -s'
alias pod='cd "$(xsel -so)"'

since it is almost never used.
also note the double quotes in case path contains white spaces.

Ah, nice. No file needed. Thanks.

Offline

#628 2009-11-05 13:31:12

Daenyth
Forum Fellow
From: Boston, MA
Registered: 2008-02-24
Posts: 1,244

Re: Post your handy self made command line utilities

Or you could use, you know, pushd and popd.

"cd -" also changes to $OLDPWD. It's useful if you didn't push first.

Last edited by Daenyth (2009-11-05 13:32:13)

Offline

#629 2009-11-05 17:02:05

lolilolicon
Member
Registered: 2009-03-05
Posts: 1,722

Re: Post your handy self made command line utilities

So I've just came up with a function, which tests if a string contains unicode characters.

has_unicode () {
    local string="$1" num_chars num_bytes
    num_chars=$(echo -n "${string}" | wc -m)
    num_bytes=$(echo -n "${string}" | wc -c)
    [ "${num_chars}" != "${num_bytes}" ]
}
has_unicode "$1" && echo 'String contains UNICODE characters' || echo 'ANSI only string'

it works by far (and it came to my mind only 52.6 seconds ago, so i'm asking for correction/confirmation from you guys)


This silver ladybug at line 28...

Offline

#630 2009-11-05 17:09:29

Daenyth
Forum Fellow
From: Boston, MA
Registered: 2008-02-24
Posts: 1,244

Re: Post your handy self made command line utilities

How does it work when you have things like \t or \n in the string?

Offline

#631 2009-11-05 17:13:08

lolilolicon
Member
Registered: 2009-03-05
Posts: 1,722

Re: Post your handy self made command line utilities

\t and \n are both counted as '1' both by `wc -c` and `wc -m`, i think it's no problem.


This silver ladybug at line 28...

Offline

#632 2009-11-05 17:19:46

Daenyth
Forum Fellow
From: Boston, MA
Registered: 2008-02-24
Posts: 1,244

Re: Post your handy self made command line utilities

Cool! I think a great extension would be to test the bytestring using several encodings, and then try to figure out the intended encoding. You could look for common constructs that mis-encoding gives. There's no foolproof way to do it, and there's not a *huge* need for it, but I could imagine it being useful. smile

Offline

#633 2009-11-05 17:34:47

lolilolicon
Member
Registered: 2009-03-05
Posts: 1,722

Re: Post your handy self made command line utilities

Sounds like something enca does (and sometimes it does poorly...)
i've always wanted to find something that will automatically figure out the intended encoding of any string, and optionally converts it to utf8.
seems iconv is missing a brother called iguess, huh?

*...has no clue how to do such thing...*


This silver ladybug at line 28...

Offline

#634 2009-11-05 17:59:52

Procyon
Member
Registered: 2008-05-07
Posts: 1,819

Re: Post your handy self made command line utilities

By the way, wc could use a wcswidth() wrapper for testing for multi-column characters too. And you're right lolilolicon, I remember when I had to find the encoding of some id3tags and I had it run through everything in iconv -l (and firefox is better at decoding than iconv by the way) so you still had to go through several pages of garbage. enca is completely useless for this.

Offline

#635 2009-11-05 20:27:03

mikesd
Member
From: Australia
Registered: 2008-02-01
Posts: 788
Website

Re: Post your handy self made command line utilities

Daenyth wrote:

Or you could use, you know, pushd and popd.

pushd and popd only seem to work in the terminal you are using. I wanted something that would work globally, ie, push in one term and pop in another.

Offline

#636 2009-11-05 21:25:38

Daenyth
Forum Fellow
From: Boston, MA
Registered: 2008-02-24
Posts: 1,244

Re: Post your handy self made command line utilities

Ah, I suppose.

Offline

#637 2009-11-06 04:53:39

lolilolicon
Member
Registered: 2009-03-05
Posts: 1,722

Re: Post your handy self made command line utilities

Procyon wrote:

By the way, wc could use a wcswidth() wrapper for testing for multi-column characters too.

i'm not sure I understand what you were talking about...
i searched for wcswidth(), and this is what i've achieved by far:

#include <stdio.h>
#include <wchar.h>

void main(int argc, char *argv[])
{
  wchar_t *ws = L"test string";
  int count;
  count = wcswidth(ws,100);
  printf("%d\n",count);
}

i'd like to change L"test string" to argv[1]. how do I do it?

Also, Procyon, could you post an example presenting what you were saying? i don't think i get it neutral

Last edited by lolilolicon (2009-11-06 05:41:19)


This silver ladybug at line 28...

Offline

#638 2009-11-06 08:52:14

Procyon
Member
Registered: 2008-05-07
Posts: 1,819

Re: Post your handy self made command line utilities

There's a man page. The function tells you that for instance ア takes 2 columns, while it's 3 bytes and 1 character. It's useful for ncurses, but for ncurses you can use large pads with only a portion of it displayed.

For the iconv -l thing here is an example:

string=$(curl -s 'http://www.itit.co.jp/' | grep 'ebisuya.gif' | sed 's/.*alt=//'); for ENCODING in $(iconv -l | sed 's/..$//'); do echo -n $ENCODING :: ; echo "$string" | iconv -f $ENCODING -t utf8 2>/dev/null; done | less

(that wasn't easy getting that string... just took the first site I saw, but grep/sed won't work at all with these sjis strings.)
The kind of output you have to go through:

RK1048 ::"ѓGѓrѓXѓ„"
ROMAN8 ::"<U+0083>G<U+0083>r<U+0083>X<U+0083><U+0084>"
ROMAN9 ::"<U+0083>G<U+0083>r<U+0083>X<U+0083><U+0084>"
RUSCII ::"ГGГrГXГД"
SE ::"SE2 ::"SEN_850200_B ::"SEN_850200_C ::"SHIFT-JIS ::"エビスヤ"
SHIFT_JIS ::"エビスヤ"
SHIFT_JISX0213 ::"エビスヤ"
SJIS-OPEN ::"エビスヤ"
SJIS-WIN ::"エビスヤ"
SJIS ::"エビスヤ"
SS636127 ::"STRK1048-2002 ::"ѓGѓrѓXѓ„"
ST_SEV_358-88 ::"<U+0083>G<U+0083>r<U+0083>X<U+0083><U+0084>"
T.61-8BIT ::"<U+0083>G<U+0083>r<U+0083>X<U+0083><U+0084>"
T.61 ::"<U+0083>G<U+0083>r<U+0083>X<U+0083><U+0084>"
T.618BIT ::"<U+0083>G<U+0083>r<U+0083>X<U+0083><U+0084>"

Offline

#639 2009-11-07 22:50:39

nh
Member
From: England
Registered: 2008-07-09
Posts: 45

Re: Post your handy self made command line utilities

This is in my .bashrc to give a sorted summary of directory sizes within a directory:

du1 () { du -h --max-depth=1 "$@" | sort -k 1,1hr -k 2,2f; }

And this uses an ed script to do a global search and replace in place (i.e., retaining the inode and permissions) across many files.  If environment variable REPLBACKUP exists, the function backs up all changed files and tells you which ones it has updated.  The grep is there to filter out "No match" and "Newline appended" ed "errors", while reporting any unexpected (and possibly genuine) errors.

die () {
    declare -i r=${2:-9}
    echo "$1" 1>&2
    return $r
}

repl () {
    (( $# >= 3 )) || return $(die "Usage: repl from-string to-string file...")
    declare a="$1" b="$2" t e
    while (( $# >= 3 )); do
        [[ -f "$3" ]] || return $(die "$0: $FUNCNAME: Regular file $3 not found")
        [[ -r "$3" && -w "$3" ]] || return $(die "$0: $FUNCNAME: Need read and write access to file $3")
        if [[ -n "$REPLBACKUP" ]]; then
            { t="$REPLBACKUP/$3_$(date +'%F_%T')_$RANDOM" && cp --preserve=all -i -- "$3" "$t"; } || return $(die "$0: $FUNCNAME: Cannot create backup file $t")
        fi
        e=$({ ed -s -- "$3" <<-EOF
            H
            ,s/$a/$b/g
            w
        EOF
        } 2>&1 | gawk '!(/^\?$/ || /No match$/ || /Newline appended$/)')
        [[ -n "$e" ]] && return $(die "$0: $FUNCNAME: Error in ed while updating $3: $e")
        if [[ -n "$REPLBACKUP" ]]; then
            if cmp -s -- "$3" "$t"; then
                rm "$t"
            else
                echo "$3 updated"
            fi
        fi
        shift
    done
}

Offline

#640 2009-11-08 02:40:04

crouse
Arch Linux f@h Team Member
From: Iowa - USA
Registered: 2006-08-19
Posts: 907
Website

Re: Post your handy self made command line utilities

steve___ wrote:
crouse wrote:

I normally use RCS to edit most files, but sometimes I just want a quick backup of a file.
This is in my .bashrc file.

bu () { cp $1 ~/Backups/${1}-`date +%Y%m%d%H%M`.backup ; }

Called with "bu somefile" it puts a timestamped copy of whatever file in my Backups directory.

I think it is best to quote your variable here. Or else it will break if the filename has a space.

bu () { cp "$1" ~/Backups/"${1}"-`date +%Y%m%d%H%M`.backup ; }

True..... BUT I never need to deal with files with spaces in them.... I live in a UNIX world wink
IE: It's my system, and NONE of my files have spaces in them....

but yes.....if you MUST <yuck>use filenames with spaces</yuck>, then quoting the $1 will allow you to backup files with spaces lol.

bu () { cp "$1" ~/Backups/"$1"-`date +%Y%m%d%H%M`.backup ; }

Offline

#641 2009-11-10 04:53:07

lolilolicon
Member
Registered: 2009-03-05
Posts: 1,722

Re: Post your handy self made command line utilities

First cd into a clean directory. Then run this:

#!/bin/bash
# Downloads "Magic Eye Image of The Week"
day=$(date -d'20091029' +%u) #4
today=$(date +%u)
back=$(( ($today - $day + 7) % 7 ))

# Seems it started July 11th, 2002
until [ "$date" == "071102" ] ; do
    date=$(date -d"-$back day" +%m%d%y) ; [ -f "${date}s.jpg" ] && break
    wget "http://www.magiceye.com/gallery/images/${date}s.jpg"
    ((back+=7))
done

I got 368 Magic Eye images. And they are a total of 24MB. Next time I run it, it just looks for updates.

I hope someday I'll come up with a imagemagick command that actually creates Magic Eye images.

Last edited by lolilolicon (2009-11-10 05:06:03)


This silver ladybug at line 28...

Offline

#642 2009-11-12 07:24:58

mjunx
Member
From: Chicago
Registered: 2009-11-01
Posts: 17
Website

Re: Post your handy self made command line utilities

Going back to the whole YouTube audio extractor thing, if the .flv already uses MP3 for audio (which it usually does), you can simply use `mplayer -dumpaudio -dumpfile foo.mp3 foo.flv`.

And to the guy who wrote the sh function: bad name (sh is the shell normally), and you could simply use git or bzr or anything like that instead.

Offline

#643 2009-11-12 15:20:45

steve___
Member
Registered: 2008-02-24
Posts: 452

Re: Post your handy self made command line utilities

youtube's High Quality videos do not use mp3.  It's aac and I use faad to convert.

Offline

#644 2009-11-12 15:39:54

Procyon
Member
Registered: 2008-05-07
Posts: 1,819

Re: Post your handy self made command line utilities

Procyon wrote:

I recently broke the default playlist size limit of mpd (16000?), and thought something was wrong with python-mpd or my netcat script, so thinking of the other mpd.db scripts in this thread I wrote a script to sort the database to an m3u playlist.

awk 'BEGIN {RS="key:"; FS="\n"} { i=0; while (i++<=NF) { if ( $i ~ /^file:/ ) { file=substr($i,7); }; if ($i ~ /^Album:/) {album=substr($i,8) }; if ( $i ~ /^Artist:/) {artist=substr($i,9) }; if ( $i ~ /^Track:/) { track=substr($i,8) }; if ( $i ~ /^Title:/) { title=substr($i,8) }; }; if ( file != "" ) { print track"\t"album"\t"file"\t"title }; };' ~/.mpd/mpd.db | sort -f -k3,3 | sort -f -s -k4,4 | sort -n -f -s -k1,1 | sort -f -s -k2,2 | cut -f 3 > $(date '+%Y-%M-%d').m3u

I don't sort by artist, but it should be easy to add. The sort pipes go from least significant to most (this works thanks to sort -s). In this case: [filename, title], track, album. The first field must be numeric (track) for sort to work numerically (i.e. you can't sort numerically on fields above 1 due to a coreutils bug). If you add an artist field note that the filename field repeats in cut -f 3.

I just noticed that sort is not working very well with unicode. For instance UTF8 album names that differ in an ASCII number will be brought together and for some reason my tracks were unsorted near the end of the playlist. I have a python one in pyampc, but with a really large database it will complain about lists/dicts being too large (did not look at the error carefully though), so I may rewrite this in another language.

Offline

#645 2009-11-12 16:02:04

mjunx
Member
From: Chicago
Registered: 2009-11-01
Posts: 17
Website

Re: Post your handy self made command line utilities

steve___ wrote:

youtube's High Quality videos do not use mp3.  It's aac and I use faad to convert.

Well, either way, mplayer's -dumpaudio option will dump the audio stream in a separate file. Many MP3 players can do AAC as well, so no need for a loss of quality.

Offline

#646 2009-11-12 16:27:46

Daenyth
Forum Fellow
From: Boston, MA
Registered: 2008-02-24
Posts: 1,244

Re: Post your handy self made command line utilities

Maybe dump and then use "file" to figure out if it should be converted?

Offline

#647 2009-11-12 20:19:30

colbert
Member
Registered: 2007-12-16
Posts: 809

Re: Post your handy self made command line utilities

I've been using this for ages in my .bash_functions to make mp3's of YT vids:

# Convert Flash video to MP3
flvmp3() {
  ffmpeg -i $1 -ar 44100 -ab 192k -ac 2 $2
}

Offline

#648 2009-11-12 20:55:27

simongmzlj
Member
From: Canada
Registered: 2008-11-06
Posts: 135

Re: Post your handy self made command line utilities

Command line power management:

if [[ -n $1 ]]
then
    case $1 in
        shutdown)
            dbus-send --system --print-reply --dest=org.freedesktop.Hal \
                /org/freedesktop/Hal/devices/computer \
                org.freedesktop.Hal.Device.SystemPowerManagement.Shutdown > /dev/null
            ;;
        restart)
            dbus-send --system --print-reply --dest=org.freedesktop.Hal \
                /org/freedesktop/Hal/devices/computer \
                org.freedesktop.Hal.Device.SystemPowerManagement.Reboot > /dev/null
            ;;
        suspend)
            dbus-send --system --print-reply --dest=org.freedesktop.Hal \
                /org/freedesktop/Hal/devices/computer \
                org.freedesktop.Hal.Device.SystemPowerManagement.Suspend \
                int32:0 > /dev/null
            ;;
        hibernate)
            dbus-send --system --print-reply --dest=org.freedesktop.Hal \
                /org/freedesktop/Hal/devices/computer \
                org.freedesktop.Hal.Device.SystemPowerManagement.Hibernate > /dev/null
            ;;
        *)
            echo "Operation not supported"
    esac
else
    echo "Operation not supported"
fi

Offline

#649 2009-11-14 01:46:44

nh
Member
From: England
Registered: 2008-07-09
Posts: 45

Re: Post your handy self made command line utilities

This is handy for viewing information about all disk partitions:

alias partitions='blkid /dev/sd?? /dev/sd??? | sort -V'

Could also be /dev/hd?? /dev/hd?? , of course.
blkid /dev/sd* also works fine; blkid seems to silently ignore disks, such as /dev/sda.
blkid doesn't list USB stick partitions, at least on my system.

Offline

#650 2009-11-14 11:43:15

Procyon
Member
Registered: 2008-05-07
Posts: 1,819

Re: Post your handy self made command line utilities

One more try for the mpd playlist sorting. I wrote a python replacement of sort, called colsort.py

#! /usr/bin/python
#colsort.py COL [-n] [-r]
#COL is which col to sort on
#-n is numeric. Takes the first positive integer found
#-r is reverse
import sys, re
from operator import itemgetter

def firstint(data):
    data=re.sub("^[^0-9]*","",data)
    data=re.sub("[^0-9].*$","",data)
    try:
        data=int(data)
    except:
        data=0
    return data
        
numeric=False
reverse=False
try:
    col=int(sys.argv[1])-1
except:
    sys.stderr.write("Specify what column to sort on.\n")
    sys.exit(1)
if "-n" in sys.argv:
    numeric=True
if "-r" in sys.argv:
    reverse=True

input=sys.stdin.readlines()
for nr in xrange(len(input)):
    input[nr]=input[nr][:-1].split('\t')
    if nr == 0:
        nrcols=len(input[nr])
    else:
        if len(input[nr]) != nrcols:
            sys.stderr.write("Unmatched amount of columns on line "+str(nr+1)+". Data: "+repr(input[nr])+"\n")
            sys.exit(1)
if col+1 > nrcols:
    sys.stderr.write("Can't sort on column "+str(col+1)+". Not enough columns in input. Data: "+repr(input[0])+"\n")
    sys.exit(1)
if numeric:
    input.sort(lambda a,b: cmp(firstint(a.split("/")[0]), firstint(b.split("/")[0])),key=itemgetter(col),reverse=reverse)
else:
    input.sort(key=itemgetter(col),reverse=reverse)

for line in input:
    print "\t".join(line)

The mpd.db script:

awk 'BEGIN {RS="key:"; FS="\n"}
{ i=0
  file="";album="";artist="";track="";title="";disc=""
  while (i++<=NF) { 
   if ( $i ~ /^file:/ ) { file=substr($i,7) }
   if ( $i ~ /^Album:/ ) { album=substr($i,8) }
   if ( $i ~ /^Artist:/ ) { artist=substr($i,9) }
   if ( $i ~ /^Track:/ ) { track=substr($i,8) }
   if ( $i ~ /^Title:/ ) { title=substr($i,8) }
   if ( $i ~ /^Disc:/ ) { disc=substr($i,8) }
 }
 if (file != "") { print file"\t"title"\t"track"\t"disc"\t"album }
}' ~/.mpd/mpd.db | 
colsort.py 1 | 
colsort.py 2 | 
colsort.py 3 -n | 
colsort.py 4 | 
colsort.py 5 |
cut -f 1 > $(date '+%Y-%m-%d').m3u

EDIT: added disc sorting, before album.
EDIT: fixed disc's substr argument.
EDIT: reset variables so if something is missing it isn't influenced by the previous track
EDIT: made numeric colsort more lenient, it takes the first int or else 0.

Last edited by Procyon (2009-11-23 08:54:27)

Offline

Board footer

Powered by FluxBB