You are not logged in.

#1 2013-02-08 16:12:13

moofly
Member
Registered: 2012-09-01
Posts: 11

help with efficiency

This code works but I'd like to make it faster(if it can be) and I'm not sure how to do it. Right now this takes about 10-15 seconds to complete.

#!/bin/bash

sed 's/ (Album)/^/1' '/mnt/files/mysecondhome/Upkeep/Arch.Upkeep/Music/Highest Playcount' | sed 's/(Artist) /^/1' | cut -d "^" -f 2 | sed 's/ /ZZZZZ/g' | sort -uf | grep -v "Playcount" | sed 's+^+00000(Artist) +' | cat -n | sed 's/ZZZZZ/ /g' | sed 's/\./\.\.\.\.\.\.\.\.\.\.\.\.\./g' | sed 's/'\''/'\''\\'\'''\''/g' | sed 's/\"/\\\"/g' > '/mnt/files/mysecondhome/Upkeep/Arch.Upkeep/.hide/artists' ; 
sed 's/ (Album)/^/1' '/mnt/files/mysecondhome/Upkeep/Arch.Upkeep/Music/Highest Playcount' | sed 's/(Artist) /^/1' | cut -d "^" -f 2 | sed 's/ /ZZZZZ/g' | sort -uf | grep -v "Playcount" | sed 's+^+0000100001+' | cat -n | sed 's/ZZZZZ/ /g' | sed 's/'\''/'\''\\'\'''\''/g' | sed 's/\"/\\\"/g' >> '/mnt/files/mysecondhome/Upkeep/Arch.Upkeep/.hide/artists' ; 

sort -n '/mnt/files/mysecondhome/Upkeep/Arch.Upkeep/.hide/artists' | sed 's/00001/^/1' | cut -d "^" -f 2 | sed 'N;s/\n/ /;P;D;' | sed 's/ 00001/'\'' '\''\/mnt\/files\/mysecondhome\/Upkeep\/Arch.Upkeep\/Music\/Highest Playcount'\'' \> '\''\/mnt\/files\/mysecondhome\/Upkeep\/Arch.Upkeep\/.hide\/artists3'\'' ; sed '\''s\/\(Playcount\) \/\(Playcount\)\/g'\'' '\''\/mnt\/files\/mysecondhome\/Upkeep\/Arch.Upkeep\/.hide\/artists3'\'' \| sed '\''s\/ \/YYYYY\/g'\'' | sed '\''s\/\(Playcount\)\/\(Playcount\) \/g'\'' \| awk '\''\{ total \+\= \$3\; count\+\+ \} END \{ print \"/g' | sed 's+$+YYzYY\.\.\.\.\.\.\.\.\.\.Artist Playcount Total\=\" total \}'\'' \>\> '\''/mnt/files/mysecondhome/Upkeep/Arch.Upkeep/.hide/artists4'\'' ;+' | sed 's/00000/^grep '\''/g' | cut -d "^" -f 2 | sed '0,/>>/s//>/' | sed 's/\.\.\.\.\.\.\.\.\.\.\.\.\./\./g' > '/mnt/files/mysecondhome/Upkeep/Arch.Upkeep/.hide/artists2' ; '/mnt/files/mysecondhome/Upkeep/Arch.Upkeep/.hide/artists2' ; 

cat '/mnt/files/mysecondhome/Upkeep/Arch.Upkeep/.hide/artists4' | sed 's/ /yuYuy/g' | sed 's/PlaycountyuYuyTotal=/PlaycountyuYuyTotal= /g' | sort -k2nr -k1n | sed 's/yuYuy/ /g' | sed 's/Playcount Total= /Playcount Total=/g' > '/mnt/files/mysecondhome/Upkeep/Arch.Upkeep/Music/Artist Playcount' ; 

sed 's/ (Title)/^/g' '/mnt/files/mysecondhome/Upkeep/Arch.Upkeep/Music/Highest Playcount' | tail -n +2 | cut -d "^" -f 1 | sed 's/(Artist) /^/g' | cut -d "^" -f 2 | sort -uf | sed 's+^+zzzzz+' | sed 's/\"/\\\"/g' | sed 's/'\''/'\''\\'\'''\''/g' > '/mnt/files/mysecondhome/Upkeep/Arch.Upkeep/.hide/albums' ; 

sed 's/ (Title)/^/g' '/mnt/files/mysecondhome/Upkeep/Arch.Upkeep/Music/Highest Playcount' | tail -n +2 | cut -d "^" -f 1 | sed 's/(Artist) /^/g' | cut -d "^" -f 2 | sort -uf | sed 's+^+zzzzz+' | sed 's/'\''/'\''\\'\'''\''/g' | sed 's/\"/\\\"/g' >> '/mnt/files/mysecondhome/Upkeep/Arch.Upkeep/.hide/albums' ; 

sort -f '/mnt/files/mysecondhome/Upkeep/Arch.Upkeep/.hide/albums' | sed 'N;s/\n/ /;P;D;' | sed 's/zzzzz//1' | sed 's/ zzzzz/'\'' '\''\/mnt\/files\/mysecondhome\/Upkeep\/Arch.Upkeep\/Music\/Highest Playcount'\'' \> '\''\/mnt\/files\/mysecondhome\/Upkeep\/Arch.Upkeep\/.hide\/albums3'\'' ; sed '\''s\/(Playcount) \/(Playcount)\/g'\'' '\''\/mnt\/files\/mysecondhome\/Upkeep\/Arch.Upkeep\/.hide\/albums3'\'' \| sed '\''s\/ \/YYYYY\/g'\'' | sed '\''s\/(Playcount)\/(Playcount) \/g'\'' \| awk '\''\{ total \+\= \$3\; count\+\+ \} END \{ print \"/g' | sed 's+$+\.\.\.\.\.\.\.\.\.\.Album Playcount Total\=\" total \}'\'' \>\> '\''/mnt/files/mysecondhome/Upkeep/Arch.Upkeep/.hide/albums4'\'' ;+' | sed 's+^+grep '\''+' | sed '0,/>>/s//>/' > '/mnt/files/mysecondhome/Upkeep/Arch.Upkeep/.hide/albums2' ; 

'/mnt/files/mysecondhome/Upkeep/Arch.Upkeep/.hide/albums2' ; cat '/mnt/files/mysecondhome/Upkeep/Arch.Upkeep/.hide/albums4' | sed 's/ /yuYuy/g' | sed 's/PlaycountyuYuyTotal=/PlaycountyuYuyTotal= /g' | sort -k1nf -k2nr | sed 's/yuYuy/ /g' | sed 's/Playcount Total= /Playcount Total=/g' > '/mnt/files/mysecondhome/Upkeep/Arch.Upkeep/Music/Album Playcount' ; 

cat '/mnt/files/mysecondhome/Upkeep/Arch.Upkeep/Music/Album Playcount' '/mnt/files/mysecondhome/Upkeep/Arch.Upkeep/Music/Artist Playcount' | sed 's/ /yuYuy/g' | sed 's/PlaycountyuYuyTotal=/PlaycountyuYuyTotal= /g' | sed 's/yuYuy(Album)/ZZyZZZ (Album)/g' | sort -k1f -k2nr | sed 's/yuYuy/ /g' | sed 's/YYzYY//1' | sed 's/ZZyZZZ//1' | sed 's/Playcount Total= /Playcount Total=/g' > '/mnt/files/mysecondhome/Upkeep/Arch.Upkeep/Music/Playcount by Artist and Album' ; 

sed -i 's/YYzYY//g' '/mnt/files/mysecondhome/Upkeep/Arch.Upkeep/Music/Artist Playcount' ; grep -v "Playcount Total=[0-9]" '/mnt/files/mysecondhome/Upkeep/Arch.Upkeep/Music/Playcount by Artist and Album' > '/mnt/files/mysecondhome/Upkeep/Arch.Upkeep/Music/fixPlaycount.Artist.Album' ; 

cat '/mnt/files/mysecondhome/Upkeep/Arch.Upkeep/Music/Album Playcount' | sed '1s+^+Album:\n+' >> '/mnt/files/mysecondhome/Upkeep/Arch.Upkeep/Music/Playcount by Artist and Album' ; cat '/mnt/files/mysecondhome/Upkeep/Arch.Upkeep/Music/Artist Playcount' | sed '1s+^+Artist:\n+' >> '/mnt/files/mysecondhome/Upkeep/Arch.Upkeep/Music/Playcount by Artist and Album' ; rm '/mnt/files/mysecondhome/Upkeep/Arch.Upkeep/Music/Artist Playcount' '/mnt/files/mysecondhome/Upkeep/Arch.Upkeep/Music/Album Playcount'

sample from highest playcount

#120	(Artist) Machine Head (Album) The More Things Change (Title) Spine (Playcount) 4
#284	(Artist) Machine Head (Album) Burn My Eyes (Title) Old (Playcount) 3
#840	(Artist) Machine Head (Album) Burn My Eyes (Title) A Thousand Lies (Playcount) 2
#841	(Artist) Machine Head (Album) Supercharger (Title) Nausea (Playcount) 2
#842	(Artist) Machine Head (Album) Supercharger (Title) Supercharger (Playcount) 2
#843	(Artist) Machine Head (Album) The Blackening (Title) Clenching The Fists Of Dissent (Playcount) 2
#844	(Artist) Machine Head (Album) The Blackening (Title) Wolves (Playcount) 2
#845	(Artist) Machine Head (Album) The More Things Change (Title) Down To None (Playcount) 2
#846	(Artist) Machine Head (Album) The More Things Change (Title) Violate (Playcount) 2
#847	(Artist) Machine Head (Album) Unto The Locust (Title) The Sentinel (Playcount) 2

end product

Machine Head..........Artist Playcount Total=53
Machine Head (Album) Burning Red..........Album Playcount Total=4
Machine Head (Album) Burn My Eyes..........Album Playcount Total=9
Machine Head (Album) Supercharger..........Album Playcount Total=12
Machine Head (Album) The Blackening..........Album Playcount Total=7
Machine Head (Album) The More Things Change..........Album Playcount Total=10
Machine Head (Album) Through The Ashes Of Empires..........Album Playcount Total=5
Machine Head (Album) Unto The Locust..........Album Playcount Total=6

Last edited by moofly (2013-02-08 16:38:14)

Offline

#2 2013-02-08 16:25:05

jakobcreutzfeldt
Member
Registered: 2011-05-12
Posts: 1,041

Re: help with efficiency

Holy carp. Please reformat that so that it's on more than one line. It's impossible to follow the logic while having to scroll to the right.

Offline

#3 2013-02-08 16:28:17

ewaller
Administrator
From: Pasadena, CA
Registered: 2009-07-13
Posts: 19,739

Re: help with efficiency

I think this would be a great opportunity to learn awk.

I am not an awk guru, but at the urging of jasonwryan I learned enough of it to appreciate its power.  It was made for this.


Nothing is too wonderful to be true, if it be consistent with the laws of nature -- Michael Faraday
Sometimes it is the people no one can imagine anything of who do the things no one can imagine. -- Alan Turing
---
How to Ask Questions the Smart Way

Offline

#4 2013-02-08 16:34:19

moofly
Member
Registered: 2012-09-01
Posts: 11

Re: help with efficiency

I'm using awk in there, just probably inefficiently

Offline

#5 2013-02-08 16:37:38

Xyne
Administrator/PM
Registered: 2008-08-03
Posts: 6,963
Website

Re: help with efficiency

moofly wrote:

This code works but I'd like to make it faster(if it can be) and I'm not sure how to do it. Right now this takes about 10-15 seconds to complete.

#!/bin/bash

sed 's/ (Album)/^/1' '/mnt/files/mysecondhome/Upkeep/Arch.Upkeep/Music/Highest Playcount' | sed 's/(Artist) /^/1' | cut -d "^" -f 2 | sed 's/ /ZZZZZ/g' | sort -uf | grep -v "Playcount" | sed 's+^+00000(Artist) +' | cat -n | sed 's/ZZZZZ/ /g' | sed 's/\./\.\.\.\.\.\.\.\.\.\.\.\.\./g' | sed 's/'\''/'\''\\'\'''\''/g' | sed 's/\"/\\\"/g' > '/mnt/files/mysecondhome/Upkeep/Arch.Upkeep/.hide/artists' ; sed 's/ (Album)/^/1' '/mnt/files/mysecondhome/Upkeep/Arch.Upkeep/Music/Highest Playcount' | sed 's/(Artist) /^/1' | cut -d "^" -f 2 | sed 's/ /ZZZZZ/g' | sort -uf | grep -v "Playcount" | sed 's+^+0000100001+' | cat -n | sed 's/ZZZZZ/ /g' | sed 's/'\''/'\''\\'\'''\''/g' | sed 's/\"/\\\"/g' >> '/mnt/files/mysecondhome/Upkeep/Arch.Upkeep/.hide/artists' ; sort -n '/mnt/files/mysecondhome/Upkeep/Arch.Upkeep/.hide/artists' | sed 's/00001/^/1' | cut -d "^" -f 2 | sed 'N;s/\n/ /;P;D;' | sed 's/ 00001/'\'' '\''\/mnt\/files\/mysecondhome\/Upkeep\/Arch.Upkeep\/Music\/Highest Playcount'\'' \> '\''\/mnt\/files\/mysecondhome\/Upkeep\/Arch.Upkeep\/.hide\/artists3'\'' ; sed '\''s\/\(Playcount\) \/\(Playcount\)\/g'\'' '\''\/mnt\/files\/mysecondhome\/Upkeep\/Arch.Upkeep\/.hide\/artists3'\'' \| sed '\''s\/ \/YYYYY\/g'\'' | sed '\''s\/\(Playcount\)\/\(Playcount\) \/g'\'' \| awk '\''\{ total \+\= \$3\; count\+\+ \} END \{ print \"/g' | sed 's+$+YYzYY\.\.\.\.\.\.\.\.\.\.Artist Playcount Total\=\" total \}'\'' \>\> '\''/mnt/files/mysecondhome/Upkeep/Arch.Upkeep/.hide/artists4'\'' ;+' | sed 's/00000/^grep '\''/g' | cut -d "^" -f 2 | sed 'N;s/\n/ /;P;D;' | sed 'N;s/\n/ /;P;D;' | sed 'N;s/\n/ /;P;D;' | sed 'N;s/\n/ /;P;D;' | sed 'N;s/\n/ /;P;D;' | sed 'N;s/\n/ /;P;D;' | sed 'N;s/\n/ /;P;D;' | sed 'N;s/\n/ /;P;D;' | sed 'N;s/\n/ /;P;D;' | sed 'N;s/\n/ /;P;D;' | sed 'N;s/\n/ /;P;D;' | sed 'N;s/\n/ /;P;D;' | sed 'N;s/\n/ /;P;D;' | sed 'N;s/\n/ /;P;D;' | sed 'N;s/\n/ /;P;D;' | sed 'N;s/\n/ /;P;D;' | sed 'N;s/\n/ /;P;D;' | sed 'N;s/\n/ /;P;D;' | sed 'N;s/\n/ /;P;D;' | sed 'N;s/\n/ /;P;D;' | sed 'N;s/\n/ /;P;D;' | sed 'N;s/\n/ /;P;D;' | sed 'N;s/\n/ /;P;D;' | sed 'N;s/\n/ /;P;D;' | sed 'N;s/\n/ /;P;D;' | sed 'N;s/\n/ /;P;D;' | sed 'N;s/\n/ /;P;D;' | sed 'N;s/\n/ /;P;D;' | sed 'N;s/\n/ /;P;D;' | sed 'N;s/\n/ /;P;D;' | sed 'N;s/\n/ /;P;D;' | sed 'N;s/\n/ /;P;D;' | sed 'N;s/\n/ /;P;D;' | sed 'N;s/\n/ /;P;D;' | sed 'N;s/\n/ /;P;D;' | sed 'N;s/\n/ /;P;D;' | sed 'N;s/\n/ /;P;D;' | sed 'N;s/\n/ /;P;D;' | sed '0,/>>/s//>/' | sed 's/\.\.\.\.\.\.\.\.\.\.\.\.\./\./g' > '/mnt/files/mysecondhome/Upkeep/Arch.Upkeep/.hide/artists2' ; '/mnt/files/mysecondhome/Upkeep/Arch.Upkeep/.hide/artists2' ; cat '/mnt/files/mysecondhome/Upkeep/Arch.Upkeep/.hide/artists4' | sed 's/ /yuYuy/g' | sed 's/PlaycountyuYuyTotal=/PlaycountyuYuyTotal= /g' | sort -k2nr -k1n | sed 's/yuYuy/ /g' | sed 's/Playcount Total= /Playcount Total=/g' > '/mnt/files/mysecondhome/Upkeep/Arch.Upkeep/Music/Artist Playcount' ; sed 's/ (Title)/^/g' '/mnt/files/mysecondhome/Upkeep/Arch.Upkeep/Music/Highest Playcount' | tail -n +2 | cut -d "^" -f 1 | sed 's/(Artist) /^/g' | cut -d "^" -f 2 | sort -uf | sed 's+^+zzzzz+' | sed 's/\"/\\\"/g' | sed 's/'\''/'\''\\'\'''\''/g' > '/mnt/files/mysecondhome/Upkeep/Arch.Upkeep/.hide/albums' ; sed 's/ (Title)/^/g' '/mnt/files/mysecondhome/Upkeep/Arch.Upkeep/Music/Highest Playcount' | tail -n +2 | cut -d "^" -f 1 | sed 's/(Artist) /^/g' | cut -d "^" -f 2 | sort -uf | sed 's+^+zzzzz+' | sed 's/'\''/'\''\\'\'''\''/g' | sed 's/\"/\\\"/g' >> '/mnt/files/mysecondhome/Upkeep/Arch.Upkeep/.hide/albums' ; sort -f '/mnt/files/mysecondhome/Upkeep/Arch.Upkeep/.hide/albums' | sed 'N;s/\n/ /;P;D;' | sed 's/zzzzz//1' | sed 's/ zzzzz/'\'' '\''\/mnt\/files\/mysecondhome\/Upkeep\/Arch.Upkeep\/Music\/Highest Playcount'\'' \> '\''\/mnt\/files\/mysecondhome\/Upkeep\/Arch.Upkeep\/.hide\/albums3'\'' ; sed '\''s\/(Playcount) \/(Playcount)\/g'\'' '\''\/mnt\/files\/mysecondhome\/Upkeep\/Arch.Upkeep\/.hide\/albums3'\'' \| sed '\''s\/ \/YYYYY\/g'\'' | sed '\''s\/(Playcount)\/(Playcount) \/g'\'' \| awk '\''\{ total \+\= \$3\; count\+\+ \} END \{ print \"/g' | sed 's+$+\.\.\.\.\.\.\.\.\.\.Album Playcount Total\=\" total \}'\'' \>\> '\''/mnt/files/mysecondhome/Upkeep/Arch.Upkeep/.hide/albums4'\'' ;+' | sed 's+^+grep '\''+' | sed '0,/>>/s//>/' > '/mnt/files/mysecondhome/Upkeep/Arch.Upkeep/.hide/albums2' ; '/mnt/files/mysecondhome/Upkeep/Arch.Upkeep/.hide/albums2' ; cat '/mnt/files/mysecondhome/Upkeep/Arch.Upkeep/.hide/albums4' | sed 's/ /yuYuy/g' | sed 's/PlaycountyuYuyTotal=/PlaycountyuYuyTotal= /g' | sort -k1nf -k2nr | sed 's/yuYuy/ /g' | sed 's/Playcount Total= /Playcount Total=/g' > '/mnt/files/mysecondhome/Upkeep/Arch.Upkeep/Music/Album Playcount' ; cat '/mnt/files/mysecondhome/Upkeep/Arch.Upkeep/Music/Album Playcount' '/mnt/files/mysecondhome/Upkeep/Arch.Upkeep/Music/Artist Playcount' | sed 's/ /yuYuy/g' | sed 's/PlaycountyuYuyTotal=/PlaycountyuYuyTotal= /g' | sed 's/yuYuy(Album)/ZZyZZZ (Album)/g' | sort -k1f -k2nr | sed 's/yuYuy/ /g' | sed 's/YYzYY//1' | sed 's/ZZyZZZ//1' | sed 's/Playcount Total= /Playcount Total=/g' > '/mnt/files/mysecondhome/Upkeep/Arch.Upkeep/Music/Playcount by Artist and Album' ; sed -i 's/YYzYY//g' '/mnt/files/mysecondhome/Upkeep/Arch.Upkeep/Music/Artist Playcount' ; grep -v "Playcount Total=[0-9]" '/mnt/files/mysecondhome/Upkeep/Arch.Upkeep/Music/Playcount by Artist and Album' > '/mnt/files/mysecondhome/Upkeep/Arch.Upkeep/Music/fixPlaycount.Artist.Album' ; cat '/mnt/files/mysecondhome/Upkeep/Arch.Upkeep/Music/Album Playcount' | sed '1s+^+Album:\n+' >> '/mnt/files/mysecondhome/Upkeep/Arch.Upkeep/Music/Playcount by Artist and Album' ; cat '/mnt/files/mysecondhome/Upkeep/Arch.Upkeep/Music/Artist Playcount' | sed '1s+^+Artist:\n+' >> '/mnt/files/mysecondhome/Upkeep/Arch.Upkeep/Music/Playcount by Artist and Album' ; rm '/mnt/files/mysecondhome/Upkeep/Arch.Upkeep/Music/Artist Playcount' '/mnt/files/mysecondhome/Upkeep/Arch.Upkeep/Music/Album Playcount'

I don't think I've ever seen so much pipe abuse.

Here are some points to consider:
1) You should never pipe the output of sed to another instance of sed. Just combine the commands in a single sed invocation to avoid redundant invocations. For example:

sed 's/ (Album)/^/1' '/mnt/files/mysecondhome/Upkeep/Arch.Upkeep/Music/Highest Playcount' | sed 's/(Artist) /^/1'

should be contracted to

sed 's/ (Album)/^/1;s/(Artist) /^/1' '/mnt/files/mysecondhome/Upkeep/Arch.Upkeep/Music/Highest Playcount'

Read the sed man page for other options that would avoid abominations such as

sed 'N;s/\n/ /;P;D;' | sed 'N;s/\n/ /;P;D;' | sed 'N;s/\n/ /;P;D;' | sed 'N;s/\n/ /;P;D;' | sed 'N;s/\n/ /;P;D;' | sed 'N;s/\n/ /;P;D;' | sed 'N;s/\n/ /;P;D;' | sed 'N;s/\n/ /;P;D;' | sed 'N;s/\n/ /;P;D;' | sed 'N;s/\n/ /;P;D;' | sed 'N;s/\n/ /;P;D;' | sed 'N;s/\n/ /;P;D;' | sed 'N;s/\n/ /;P;D;' | sed 'N;s/\n/ /;P;D;' | sed 'N;s/\n/ /;P;D;' | sed 'N;s/\n/ /;P;D;' | sed 'N;s/\n/ /;P;D;' | sed 'N;s/\n/ /;P;D;' | sed 'N;s/\n/ /;P;D;' | sed 'N;s/\n/ /;P;D;' | sed 'N;s/\n/ /;P;D;' | sed 'N;s/\n/ /;P;D;' | sed 'N;s/\n/ /;P;D;' | sed 'N;s/\n/ /;P;D;' | sed 'N;s/\n/ /;P;D;' | sed 'N;s/\n/ /;P;D;' | sed 'N;s/\n/ /;P;D;' | sed 'N;s/\n/ /;P;D;' | sed 'N;s/\n/ /;P;D;' | sed 'N;s/\n/ /;P;D;' | sed 'N;s/\n/ /;P;D;' | sed 'N;s/\n/ /;P;D;' | sed 'N;s/\n/ /;P;D;' | sed 'N;s/\n/ /;P;D;' | sed 'N;s/\n/ /;P;D;' | sed 'N;s/\n/ /;P;D;' | sed 'N;s/\n/ /;P;D;' | sed 'N;s/\n/ /;P;D;' 

2) Bash is not the right tool for extensive text manipulation. Use awk or write a proper script in Perl or Python (or Lua, Ruby, ...).

3) If you really are going to insist on using Bash, make use of Bash arrays and built-in string manipulation along with standard control loops (which would avoid the huge chain of repeating statements while also ensuring that you get the desired result instead of just hoping that you've repeated the command enough times).

4) In general, you should not hard-code paths into your scripts. It probably doesn't matter in your case if they are unlikely to change.

5) Don't use "cat" to send single-file input to the next item in a pipe. For example, instead of

cat foo | cmd ...

use

cmd ... < foo

where "< foo" does the same thing as "cat foo |" but much more efficiently.

In the case of sed, just pass it the file name so it can read the file directly.

Last edited by Xyne (2013-02-08 16:40:53)


My Arch Linux StuffForum EtiquetteCommunity Ethos - Arch is not for everyone

Offline

#6 2013-02-08 16:43:33

moofly
Member
Registered: 2012-09-01
Posts: 11

Re: help with efficiency

Xyne wrote:

2) Bash is not the right tool for extensive text manipulation. Use awk or write a proper script in Perl or Python (or Lua, Ruby, ...).

I don't know any of those, i guess i should learn. But i did create this thread so i could get better.

Offline

#7 2013-02-08 17:17:22

aesiris
Member
Registered: 2012-02-25
Posts: 97

Re: help with efficiency

If you want to follow the bash route, parse the input with =~ and the right regex.

In your case

preparseandsort=$(
        while read line; do
                if [[ $line =~ ^#([0-9]*).*\(Artist\)\ ([^\(]*)\ \(Album\)\ ([^\(]*)\ \(Title\)\ ([^\(]*)\ \(Playcount\)\ *([0-9]*)$ ]]; then
                        id=${BASH_REMATCH[1]}
                        art=${BASH_REMATCH[2]}
                        alb=${BASH_REMATCH[3]}
                        tit=${BASH_REMATCH[4]}
                        pco=${BASH_REMATCH[5]}
                        echo "(1)$id (2)$art (3)$alb (4)$tit (5)$pco"
                else
                        echo "Failed parsing $line"
                        exit 1
                fi
        done | sort -k 2
)

Offline

#8 2013-02-08 18:24:08

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 29,447
Website

Re: help with efficiency

moofly wrote:

I'm using awk in there, just probably inefficiently

No, it shouldn't just be "in there".  This whole thing could probably be boiled down to half a dozen clean lines of awk script.  Are the 2nd and 3rd code boxes in the OP an example input with intended output?  If so, I'm not seeing how those numbers are arrived at.  What do you want this to do?

Last edited by Trilby (2013-02-08 18:26:51)


"UNIX is simple and coherent..." - Dennis Ritchie, "GNU's Not UNIX" -  Richard Stallman

Online

#9 2013-02-08 18:30:57

moofly
Member
Registered: 2012-09-01
Posts: 11

Re: help with efficiency

Trilby wrote:
moofly wrote:

I'm using awk in there, just probably inefficiently

No, it shouldn't just be "in there".  This whole thing could probably be boiled down to half a dozen clean lines of awk script.  Are the 2nd and 3rd code boxes in the OP an example input with intended output?  If so, I'm not seeing how those numbers are arrived at.  What do you want this to do?

yes..... the 2nd code box is just a sample like i said so you can see the format of the input (do you want me to put it all up?). 3rd is the output i have and want.  the numbers aren't equal because there's the sample (2) and the total (3), was just trying to not fill it up text thats not really needed. Also I'm sorry for my horrible coding.

Last edited by moofly (2013-02-08 18:39:28)

Offline

#10 2013-02-08 18:50:22

jasonwryan
Anarchist
From: .nz
Registered: 2009-05-09
Posts: 30,424
Website

Re: help with efficiency

moofly wrote:

Also I'm sorry for my horrible coding.

There is absolutely no need to apologise for showing a willingness to learn. Everyone starts with horrible code; not asking for help only means it stays that way...


Arch + dwm   •   Mercurial repos  •   Surfraw

Registered Linux User #482438

Offline

Board footer

Powered by FluxBB