pasting columns in bash

jnwebb · 2008-07-15 16:34:44

Lets say I have two files

File 1 File 2
Freq. Acc. Freq. Acc.
1 6 1 2
2 4 2 5
3 1 3 7
4 2 4 1
5 1 5 3

And I want to take both columns of file 1 and only column two of file (I have about 20 of these files) and end up w/ something like ....

File 3
Freq. Acc. Acc.
1 6 2
2 4 5
3 1 7
4 2 1
5 1 3

Any help would be appreciated.

dyscoria · 2008-07-15 17:10:51

I was bored! So here goes:

#!/bin/bash

total_lines=$(wc -l file1 | cut -d' ' -f 1)
linenumber=1

until [ $linenumber -gt $total_lines ]
do
    apples=$(sed -n "$linenumber"p file1)
    pears=$(sed -n "$linenumber"p file2 | cut -f 2)
    printf "$apples\t$pears\n"
    linenumber=$(($linenumber+1))
done

jnwebb · 2008-07-15 19:15:03

I ran through the following script...per your suggestion...

total_lines=$(wc -l "GRN180-5.crs" | cut -d' ' -f 1)
linenumber=1
until [ $linenumber -gt $total_lines ]
do
apples=$(sed -n "$linenumber"p "GRN180-5.crs")
pears=$(sed -n "$linenumber"p "KAU078-W-5.crs" | cut -f 2)
printf"$apples\t$pears\n"
linenumber=$(($linenumber+1))
done

I get:

-bash: [: 1: unary operator expected
0.1000 0.75992E-02 0.1000 0.78482E-02
it scrolls through all line numbers w/ this error for each line numbers values
.
.
.
then it reaches the end of the file and continues
-bsh: [: #####: unary operator expected

any help

dyscoria · 2008-07-15 19:31:24

Tried to repeat that error and you must have mistyped one of the 'linenumber' or 'total_lines' entries. Basically the error is caused by the loop test. If there's a typo then you get one of these:
until [ -gt $total_lines]
until [ $linenumber -gt ]
The -gt means greater than and only a unary operator (as opposed to a binary operator like -gt) can function with only one item.

Oh I forgot to mention: http://tldp.org/LDP/abs/html/
Everything I know about bash scripting I learnt from that website, and trust me you'll understand a lot more about what's going on!

Procyon · 2008-07-15 20:09:58

cut and paste might do the trick, but it will be harder to check if freq is aligned.

paste file1 file2 | cut -d'<TABKEY>' -f 1,2,4

jnwebb · 2008-07-15 23:28:20

I ended up using .....awk '{print $2}' File 2.txt | paste -d' ' File1.txt - > File3.ttxt

Now I would like to go through line by line and create a new file that has Freq. in the first column and then the value of column 2 divided by column 3 (a ratio).

Procyon · 2008-07-16 00:02:00

awk can do math.

awk '($1 != "Freq."){print $1, $2/$3}' file3

jnwebb · 2008-07-16 01:54:15

can you do any log or e arguments in bash?

DonVla · 2008-07-16 08:49:41

jnwebb wrote:

can you do any log or e arguments in bash?

you can use bc. http://linuxreviews.org/man/bc/#lbAM

jnwebb · 2008-07-16 16:22:18

thank you ... things are coming together...i will post final when finished

Similar to original question (files in two columns). I have 15 files in two columns... similarly the first column in each file is the same. How can I create a file w/ column 1 and then successive column 2's from each file after that so that I would have a total of 16 columns...some sort of loop.

Procyon · 2008-07-16 17:11:31

Same thing as paste and cut.
paste file{1..15} | cut -d' ' -f1,2,4,6,8,10,12,14...etc

jnwebb · 2008-07-16 18:41:48

beautiful...all of your help is really working...ok now the finale!!

I now have a file w/ 16 columns. The first being Freq. and the rest acc values for different earthquakes

Freq. Acc. Acc. ..... ....... ......
1 6 2
2 4 5
3 1 7
....

I need to end up w/

beautiful...all of your help is really working...ok now the finale!!

I now have a file w/ 16 columns. The first being Freq. and the rest acc values for different bins of data

Freq. Acc. Acc. ..... ....... ...... Avg Stand. Dev. Avg+STD Avg-STD e^Avg e^(avg+std) e^(avg-std)
1 6 2
2 4 5
3 1 7
....

how can I do math across each row of data? Don't so much need help w/ the actual math, just getting across to each number (expect for freq.)

Procyon · 2008-07-16 19:14:20

Note that awk may have functions that you are looking for and you wouldn't need bc. Check the man page under Numeric functions

To get the average in a file4 like
Freq. Avg
1 10
2 10
3 11
4 11

you need to keep up some variables. Initialize it in BEGIN, print it in END

awk 'BEGIN { sum=0;amount=0 } ($1!="Freq."){ sum=$2+sum; amount=amount+1} END {print sum/amount}' file4

I don't know how to get standard deviation, but let's try e^Avg now:
file5
Freq. Avg e^Avg
1 10 0
2 10 1
3 11 2
4 11 3

--> awk 'BEGIN { sumc1=0;amount=0;sumc2;e=2.71828 } ($1!="Freq."){ sumc1=$2+sumc1;sumc2=sumc2+$3; amount=amount+1} END {print "Avg:",sumc1/amo
unt,"\ne^Avg",e^(sumc2/amount)}' file5
Avg: 10.5
e^Avg 4.48168

jnwebb · 2008-07-16 19:19:10

thank you, but I want to sum across the rows not down the columns

Procyon · 2008-07-16 21:10:52

Let's take:
file6
Freq. Acc. Acc. Foo. Bar.
1 0 50 10 5
2 1 70 20 5
3 2 60 10 4
4 10 10 11 11
5 4 20 10 4

And we want the average of all columns except Freq. (and output as Freq. Avg)

--> awk '($1!="Freq.") { sum=0;for (i=2;i<=NF;i++) { sum=sum+$i } print $1, sum/(NF-1)}' file6
1 16.25
2 24
3 19
4 10.5
5 9.5

EDIT: for loop variable i should start with 2, not 1. (first field we want is $2)

Last edited by Procyon (2008-07-16 21:13:52)

jnwebb · 2008-07-21 16:35:11

Thank you...much appreciated.

lets say I have a file:

1019 0.3126E+03 0.4976E+02 -0.1580E-03 -0.4306E-04 0.1638E-03 0.1422E-01
1020 0.3129E+03 0.4980E+02 -0.2277E-04 0.1461E-03 0.1478E-03 0.1958E-01
1021 0.3132E+03 0.4985E+02 0.1256E-03 0.1719E-04 0.1267E-03 0.4580E-02
1022 0.3135E+03 0.4990E+02 0.1576E-04 -0.8535E-04 0.8679E-04 0.9437E-02
1023 0.3139E+03 0.4995E+02 -0.3771E-04 -0.1341E-05 0.3774E-04 0.1490E-01
1024 0.3142E+03 0.5000E+02 0.9472E-05 -0.5867E-15 0.9472E-05 0.5000E-02

i want to write an if loop so that if the numbering on the last line is less than 1025 (some files go out to 4096), it will add the following to the end of file

1025 0000000000 0000000000 000000000 000000000 0000000000 0000000000
1026 0000000000 0000000000 000000000 000000000 0000000000 0000000000
1027 0000000000 0000000000 000000000 000000000 0000000000 0000000000
.
.
.
4096 0000000000 0000000000 000000000 000000000 0000000000 0000000000

Procyon · 2008-07-21 17:48:46

I think this should do:

Be careful that the value in lastfreq will be 1 higher when you check if it should be skipped.

EDIT: BTW getting the lastfreq could be a lot simpler using a line counter, but some of these files have headers right? I thought this would be more secure.

for researchdata in file*; do
lastfreq=$(($(sed -ne '$s/^\([0-9]*\).*/\1/p' "$researchdata")+1))

#error checking
if [[ -z $lastfreq ]]; then echo ERROR ON $researchdata; continue; fi

#is this number right? I think this will still give 1024 the empty freqs.
if [[ $lastfreq -gt 1025 ]]; then continue; fi

#better backup everything, for fear of accidentally using >
for neededfreq in $(seq $lastfreq 4096); do echo $neededfreq 000 000 000 000 etc; done >> "$researchdata"
done

EDIT: slightly simpler substitution

Last edited by Procyon (2008-07-21 17:54:32)

jnwebb · 2008-07-21 19:17:08

for researchdata in "CARES.fc"; do
lastfreq=$(($(sed -ne '$s/^$[0-9]*$.*/\1/p' "$researchdata")+1))
if [[ -z $lastfreq ]]; then echo ERROR ON $researchdata; continue; fi
if [[ $lastfreq -gt 1025 ]]; then continue; fi
for neededfreq in $(seq $lastfreq 4096); do echo $neededfreq 000 000 000 000; done >> "$researchdata"
done

i get a bash error: seq:command not found

Procyon · 2008-07-21 20:48:58

seq is in coreutils. Which you have installed of course. Otherwise even things like cat won't work (or does bash take over for that?)

Can you test with just typing seq 1 10 in bash?

jnwebb · 2008-07-21 21:01:17

same return...but obviously cat, etc work fine

Procyon · 2008-07-21 21:39:37

No seq? That's confusing.

It's simple to replace though:

Also, the error check is stupid. The addition in $((..)) makes it 1 if nothing was found.
I left it in, but added a check to see if it's 1. That means if the last frequency is 0 it will give an error.

And one more thing, it may be that there are empty lines at the end, if that's the case you'll get a lot of errors now. That should be easy to fix though.

for researchdata in "CARES.fc"; do
lastfreq=$(($(sed -ne '$s/^\([0-9]*\).*/\1/p' "$researchdata")+1))
if [[ -z $lastfreq ]]; then echo ERROR ON $researchdata; continue; fi
if [[ $lastfreq -eq 1 ]]; then echo ERROR ON $researchdata; continue; fi
if [[ $lastfreq -gt 1025 ]]; then continue; fi
while [[ $lastfreq -le 4096 ]]; do echo $lastfreq 000 000 000 000; lastfreq=$(($lastfreq+1)); done >> "$researchdata"
done

jnwebb · 2008-07-25 17:04:17

I just want to say thank you to all who helped me put this together...the final results are going to be used in earthquake research...attached is my clusterf**k of a script...but hey...it works!!! Here it is if you care to look at it...it is 14 pages long.

Anyway, thank you

#$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$ FOLDER TREE STRUCTURE  $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$

mkdir 139.P 151.P 158.P
    
#$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$ CALCULATE RESPONSE SPECTRUM  $$$$$$$$$$$$$$$$$$$$$$$$$$$$$
#                                        FOR SEEDS, CARES TH FIT, AND SSI FROM SASSI

for f in *.CTH; do
       d=`basename $f.F`
       mkdir $d
         mv $f $d
         cp endans.txt H1Fit1.inp motionTemplate.inp fort.8 $d
         cd $d
             #            SEEDS RS
            echo 6 > answers1.txt
            echo *.CTH >> answers1.txt
            echo $d-5.srs >> answers1.txt
            echo y >> answers1.txt
            echo 5 >> answers1.txt
            echo 6 >> answers1.txt
            echo *.CTH >> answers1.txt
            echo $d-15.srs>> answers1.txt
            echo y >> answers1.txt
            echo 15 >> answers1.txt
            echo 8 >> answers1.txt
            crsdos4 < answers1.txt
            #              CARES TH to FC and pad zeros to 4096
            echo 3 >> answers2.txt
            echo *.CTH >> answers2.txt
            echo CARES.fc >> answers2.txt
            echo y >> answers2.txt
            echo n >> answers2.txt
            echo y >> answers2.txt
            echo 8 >> answers2.txt
            crsdos4 < answers2.txt
            for researchdata in "CARES.fc"; do                
                lastfreq=$(($(find "CARES.fc" -print0 | xargs -0 wc -l | awk '{print$1}')-19)); 
                if [[ $lastfreq -gt 1025 ]]; then continue; fi; 
                while [[ $lastfreq -le 4096 ]]; do 
                echo  $lastfreq 0000000000  0000000000  0000000000  0000000000  0000000000  0000000000; 
                lastfreq=$(($lastfreq+1)); done >> "$researchdata"; done
    
            #            CARES RS
            echo 2 >> answers3.txt
            echo y >> answers3.txt
            echo H1Fit1.inp >> answers3.txt
            echo y >> answers3.txt
            echo y >> answers3.txt
            echo n >> answers3.txt
            echo y >> answers3.txt
            echo 6 >> answers3.txt
            echo CARES.cth >> answers3.txt
            echo $d-5.crs >> answers3.txt
            echo y >> answers3.txt
            echo 5 >> answers3.txt
            echo 6 >> answers3.txt
            echo CARES.cth >> answers3.txt
            echo $d-15.crs >> answers3.txt
            echo y >> answers3.txt
            echo 15 >> answers3.txt
            echo 8 >> answers3.txt
            crsdos4 < answers3.txt
            #            SASSI RS       
            ctos CARES.cth
            cat motionTemplate.inp > motion.inp
            cat CARES.cth.sth >> motion.inp
            sed '$d' < motion.inp > temp.txt
            mv temp.txt motion.inp
            echo '0' >> motion.inp
            motion <motion.inp>motion.out
        #### a = 5% damped b = 15% damped ####           
            awk 'NR==1231,NR==1433' motion.out > $d-139.ars
            cp $d-139.ars CARES.crs ../139.P
            awk 'NR==2195,NR==2397' motion.out > $d-151.ars
            cp $d-151.ars CARES.crs ../151.P
            awk 'NR==3159,NR==3361' motion.out > $d-158.ars
            cp $d-158.ars CARES.crs ../158.P
            awk 'NR==1443,NR==1645' motion.out > $d-139.brs
            cp $d-139.brs ../139.P
            awk 'NR==2407,NR==2609' motion.out > $d-151.brs
            cp $d-151.brs ../151.P
            awk 'NR==3371,NR==3573' motion.out > $d-158.brs
            cp $d-158.brs ../158.P
            cp *.srs ..
            cp *.crs ..
         cd ..
     done 

########################## MOVE and STRIP EXTENSIONS FROM FILES TO NODE FOLDERS#################

#                                                SSI
cd 139.P
filelist=`ls -1 *.ars`
for fname in $filelist; do
   newfn="$(basename "$fname" .CTH.F-139.ars)-5.RS"
   mv $fname $newfn
done
filelist=`ls -1 *.brs`
for fname in $filelist; do
   newfn="$(basename "$fname" .CTH.F-139.brs)-15.RS"
   mv $fname $newfn
done
cd ..
cp 139-5.txt 139.P
cp 139-15.txt 139.P

cd 151.P
filelist=`ls -1 *.ars`
for fname in $filelist; do
   newfn="$(basename "$fname" .CTH.F-151.ars)-5.RS"
   mv $fname $newfn
done
filelist=`ls -1 *.brs`
for fname in $filelist; do
   newfn="$(basename "$fname" .CTH.F-151.brs)-15.RS"
   mv $fname $newfn
done
cd ..
cp 151-5.txt 151.P
cp 151-15.txt 151.P

cd 158.P
filelist=`ls -1 *.ars`
for fname in $filelist; do
   newfn="$(basename "$fname" .CTH.F-158.ars)-5.RS"
   mv $fname $newfn
done
filelist=`ls -1 *.brs`
for fname in $filelist; do
   newfn="$(basename "$fname" .CTH.F-158.brs)-15.RS"
   mv $fname $newfn
done
cd ..
cp 158-5.txt 158.P
cp 158-15.txt 158.P

#                                            CARES TH

filelist=`ls -1 *'-5.crs'`
for fname in $filelist; do
   newfn="$(basename "$fname" .CTH.F-5.crs)-5.crs"
   mv $fname $newfn
done
filelist=`ls -1 *'-15.crs'`
for fname in $filelist; do
   newfn="$(basename "$fname" .CTH.F-15.crs)-15.crs"
   mv $fname $newfn
done

#                                            SEEDS
filelist=`ls -1 *'-5.srs'`
for fname in $filelist; do
   newfn="$(basename "$fname" .CTH.F-5.srs)-5.srs"
   mv $fname $newfn
done
filelist=`ls -1 *'-15.srs'`
for fname in $filelist; do
   newfn="$(basename "$fname" .CTH.F-15.srs)-15.srs"
   mv $fname $newfn
done


#$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$     GNUPLOT   $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
#                                                      SSI
#                                                
#                                                   5% w/ TARGET
cd 139.P
filelist=`ls -1 *'-5.RS'`
for fname in $filelist; do
echo '"'$fname'" u 8:6 w l lw 2 t "'$fname'" , \'>>139-5.txt
done
sed -n '$p' 139-5.txt>temp.txt
sed -n 's/...$//p' temp.txt>temp2.txt
sed '$d' 139-5.txt>temp3.txt
cat temp2.txt>>temp3.txt
mv temp3.txt 139-5.txt
rm temp.txt temp2.txt
gnuplot < 139-5.txt

cd ..
cd 151.P
filelist=`ls -1 *'-5.RS'`
for fname in $filelist; do
echo '"'$fname'" u 8:6 w l lw 2 t "'$fname'" , \'>>151-5.txt
done
sed -n '$p' 151-5.txt>temp.txt
sed -n 's/...$//p' temp.txt>temp2.txt
sed '$d' 151-5.txt>temp3.txt
cat temp2.txt>>temp3.txt
mv temp3.txt 151-5.txt
rm temp.txt temp2.txt
gnuplot < 151-5.txt

cd ..
cd 158.P
filelist=`ls -1 *'-5.RS'`
for fname in $filelist; do
echo '"'$fname'" u 8:6 w l lw 2 t"'$fname'" , \'>>158-5.txt
done
sed -n '$p' 158-5.txt>temp.txt
sed -n 's/...$//p' temp.txt>temp2.txt
sed '$d' 158-5.txt>temp3.txt
cat temp2.txt>>temp3.txt
mv temp3.txt 158-5.txt
rm temp.txt temp2.txt
gnuplot < 158-5.txt

#                                                15% w/o TARGET
cd ..
cd 139.P
filelist=`ls -1 *'-15.RS'`
for fname in $filelist; do
echo '"'$fname'" u 8:6 w l lw 2 t "'$fname'" , \'>>139-15.txt
done
sed -n '$p' 139-15.txt>temp.txt
sed -n 's/...$//p' temp.txt>temp2.txt
sed '$d' 139-15.txt>temp3.txt
cat temp2.txt>>temp3.txt
mv temp3.txt 139-15.txt
rm temp.txt temp2.txt
gnuplot < 139-15.txt

cd ..
cd 151.P
filelist=`ls -1 *'-15.RS'`
for fname in $filelist; do
echo '"'$fname'" u 8:6 w l lw 2 t "'$fname'" , \'>>151-15.txt
done
sed -n '$p' 151-15.txt>temp.txt
sed -n 's/...$//p' temp.txt>temp2.txt
sed '$d' 151-15.txt>temp3.txt
cat temp2.txt>>temp3.txt
mv temp3.txt 151-15.txt
rm temp.txt temp2.txt
gnuplot < 151-15.txt

cd ..
cd 158.P
filelist=`ls -1 *'-15.RS'`
for fname in $filelist; do
echo '"'$fname'" u 8:6 w l lw 2 t"'$fname'" , \'>>158-15.txt
done
sed -n '$p' 158-15.txt>temp.txt
sed -n 's/...$//p' temp.txt>temp2.txt
sed '$d' 158-15.txt>temp3.txt
cat temp2.txt>>temp3.txt
mv temp3.txt 158-15.txt
rm temp.txt temp2.txt
gnuplot < 158-15.txt
cd ..

#                                                        RATIO 5%/15%
cd 139.P
mkdir RATIO
cp *'.RS' RATIO
cd RATIO
i=0
j=0
declare -a five
declare -a fifteen
for file in *'-5.RS'
do
  five[i]=$file
  i=$i+1
done
for file in *'-15.RS'
do 
  fifteen[j]=$file
  j=$j+1
done
echo 'set term postscript enhanced color' > temp.txt
echo 'set output "SSIRATIO-139.eps"' >> temp.txt
echo 'set logscale x' >> temp.txt
echo 'set key out' >> temp.txt
echo 'set xlabel "Frequency (Hz)"' >> temp.txt
echo 'set ylabel "5%-15% SPECTRAL RATIO"' >> temp.txt
echo 'set title  "5%-15% Damped SSI SPECTRAL RATIO at Basemat of Structure (Node 139) \n WUS Rock - Magnitude 6.5 \n Distance 50-100 km"' >> temp.txt
echo 'plot "final.txt" u 1:2 w l lw 3 lt -1 t "MEAN" , \' >> temp.txt
echo '"final.txt" u 1:3 w l lw 3 lt -1 t "+{/Symbol s}" , \' >> temp.txt
echo '"final.txt" u 1:4 w l lw 3 lt -1 t "-{/Symbol s}", \'>> temp.txt
for (( i=0; i<${#five[@]};i++))
do
echo '"< paste '${five[$i]}' '${fifteen[$i]}'" u 8:($6/$14) w l lw 2 t "'${five[$i]}'" , \' >> temp.txt
sed -e '1,3d' ${five[$i]}>${five[$i]}"a"   #remove first 3 lines of file to strip out text
sed -e '1,3d' ${fifteen[$i]}>${fifteen[$i]}"a"
awk '{print $6}' ${five[$i]}"a">${five[$i]}"x"
awk '{print $6}' ${fifteen[$i]}"a" | paste -d' ' ${five[$i]}"x" - > ${five[$i]}"b"  # put 5% and 15% ACC in one file
awk '{print $1/$2}' ${five[$i]}"b" > ${five[$i]}"c"  # find ratio of 5% and 15%
awk '{print $8}' ${five[0]}"a" > freq.txt # saves freq column to own file for copy of all logs file

## find the log of each value  from the ratio file
     cat ${five[$i]}"c" |
     while read a;
       do
         echo "scale=5; l($a)" | bc -l >> ${five[$i]}"f"
       done 
done
sed -n '$p' temp.txt>temp4.txt
sed -n 's/...$//p' temp4.txt>temp2.txt
sed '$d' temp.txt>temp3.txt
cat temp2.txt>>temp3.txt
mv temp3.txt temp.txt
#     copy all logs of ratios to one file
i=0
declare -a logs
for file in *'f'
do 
    logs[i]=$file
    i=$i+1
done    
paste ${logs[@]}| cut -d' ' -f1 > new.txt
paste freq.txt new.txt > logs.txt
#                                CALCULATING MEAN AND +-1 SIGMA
# AVERAGE
awk '{ sum=0; for (i=2;i<=NF;i++) { sum=sum+$i } print sum/(NF-1)}' logs.txt > avg.txt
paste logs.txt avg.txt >logs2.txt
# STANDARD DEVIATION
awk '{ sum=0; for (i=2;i<=(NF-1);i++) { sum=sum+($i-$(NF))*($i-$(NF)) } print sqrt(sum/(NF-3))}' logs2.txt > std.txt
paste avg.txt std.txt > avgstd.txt
# +/- SIGMA
awk '{print $1+$2}' avgstd.txt > plus.txt
awk '{print $1-$2}' avgstd.txt > minus.txt
# exponent of MEAN, AND =/- 1 SIGMA
awk '{print 2.71828**$1}' avg.txt > eavg.txt
awk '{print 2.71828**$1}' plus.txt > eplus.txt
awk '{print 2.71828**$1}' minus.txt > eminus.txt
paste freq.txt eavg.txt > ratio.txt
paste ratio.txt eplus.txt > ratio2.txt
paste ratio2.txt eminus.txt > final.txt
gnuplot < temp.txt
cd ..
cd ..

cd 151.P
mkdir RATIO
cp *'.RS' RATIO
cd RATIO
i=0
j=0
declare -a five
declare -a fifteen
for file in *'-5.RS'
do
  five[i]=$file
  i=$i+1
done
for file in *'-15.RS'
do 
  fifteen[j]=$file
  j=$j+1
done
echo 'set term postscript enhanced color' > temp.txt
echo 'set output "SSIRATIO-151.eps"' >> temp.txt
echo 'set logscale x' >> temp.txt
echo 'set key out' >> temp.txt
echo 'set xlabel "Frequency (Hz)"' >> temp.txt
echo 'set ylabel "5%-15% SPECTRAL RATIO"' >> temp.txt
echo 'set title  "5%-15% Damped SSI SPECTRAL RATIO at Top of Equipment Stick (Node 151) \n WUS Rock - Magnitude 6.5 \n Distance 50-100 km"' >> temp.txt
echo 'plot "final.txt" u 1:2 w l lw 3 lt -1 t "MEAN" , \' >> temp.txt
echo '"final.txt" u 1:3 w l lw 3 lt -1 t "+{/Symbol s}" , \' >> temp.txt
echo '"final.txt" u 1:4 w l lw 3 lt -1 t "-{/Symbol s}", \'>> temp.txt
for (( i=0; i<${#five[@]};i++))
do
echo '"< paste '${five[$i]}' '${fifteen[$i]}'" u 8:($6/$14) w l lw 2 t "'${five[$i]}'" , \' >> temp.txt
sed -e '1,3d' ${five[$i]}>${five[$i]}"a"   #remove first 3 lines of file to strip out text
sed -e '1,3d' ${fifteen[$i]}>${fifteen[$i]}"a"
awk '{print $6}' ${five[$i]}"a">${five[$i]}"x"
awk '{print $6}' ${fifteen[$i]}"a" | paste -d' ' ${five[$i]}"x" - > ${five[$i]}"b"  # put 5% and 15% ACC in one file
awk '{print $1/$2}' ${five[$i]}"b" > ${five[$i]}"c"  # find ratio of 5% and 15%
awk '{print $8}' ${five[0]}"a" > freq.txt # saves freq column to own file for copy of all logs file

## find the log of each value  from the ratio file
     cat ${five[$i]}"c" |
     while read a;
       do
         echo "scale=5; l($a)" | bc -l >> ${five[$i]}"f"
       done 
done
sed -n '$p' temp.txt>temp4.txt
sed -n 's/...$//p' temp4.txt>temp2.txt
sed '$d' temp.txt>temp3.txt
cat temp2.txt>>temp3.txt
mv temp3.txt temp.txt
#     copy all logs of ratios to one file
i=0
declare -a logs
for file in *'f'
do 
    logs[i]=$file
    i=$i+1
done    
paste ${logs[@]}| cut -d' ' -f1 > new.txt
paste freq.txt new.txt > logs.txt
#                                CALCULATING MEAN AND +-1 SIGMA
# AVERAGE
awk '{ sum=0; for (i=2;i<=NF;i++) { sum=sum+$i } print sum/(NF-1)}' logs.txt > avg.txt
paste logs.txt avg.txt >logs2.txt
# STANDARD DEVIATION
awk '{ sum=0; for (i=2;i<=(NF-1);i++) { sum=sum+($i-$(NF))*($i-$(NF)) } print sqrt(sum/(NF-3))}' logs2.txt > std.txt
paste avg.txt std.txt > avgstd.txt
# +/- SIGMA
awk '{print $1+$2}' avgstd.txt > plus.txt
awk '{print $1-$2}' avgstd.txt > minus.txt
# exponent of MEAN, AND =/- 1 SIGMA
awk '{print 2.71828**$1}' avg.txt > eavg.txt
awk '{print 2.71828**$1}' plus.txt > eplus.txt
awk '{print 2.71828**$1}' minus.txt > eminus.txt
paste freq.txt eavg.txt > ratio.txt
paste ratio.txt eplus.txt > ratio2.txt
paste ratio2.txt eminus.txt > final.txt
gnuplot < temp.txt
cd ..
cd ..

cd 158.P
mkdir RATIO
cp *'.RS' RATIO
cd RATIO
i=0
j=0
declare -a five
declare -a fifteen
for file in *'-5.RS'
do
  five[i]=$file
  i=$i+1
done
for file in *'-15.RS'
do 
  fifteen[j]=$file
  j=$j+1
done
echo 'set term postscript enhanced color' > temp.txt
echo 'set output "SSIRATIO-158.eps"' >> temp.txt
echo 'set logscale x' >> temp.txt
echo 'set key out' >> temp.txt
echo 'set xlabel "Frequency (Hz)"' >> temp.txt
echo 'set ylabel "5%-15% SPECTRAL RATIO"' >> temp.txt
echo 'set title  "5%-15% Damped SSI SPECTRAL RATIO at Top of Containment Structure (Node 158) \n WUS Rock - Magnitude 6.5 \n Distance 50-100 km"' >> temp.txt
echo 'plot "final.txt" u 1:2 w l lw 3 lt -1 t "MEAN" , \' >> temp.txt
echo '"final.txt" u 1:3 w l lw 3 lt -1 t "+{/Symbol s}" , \' >> temp.txt
echo '"final.txt" u 1:4 w l lw 3 lt -1 t "-{/Symbol s}", \'>> temp.txt
for (( i=0; i<${#five[@]};i++))
do
echo '"< paste '${five[$i]}' '${fifteen[$i]}'" u 8:($6/$14) w l lw 2 t "'${five[$i]}'" , \' >> temp.txt
sed -e '1,3d' ${five[$i]}>${five[$i]}"a"   #remove first 3 lines of file to strip out text
sed -e '1,3d' ${fifteen[$i]}>${fifteen[$i]}"a"
awk '{print $6}' ${five[$i]}"a">${five[$i]}"x"
awk '{print $6}' ${fifteen[$i]}"a" | paste -d' ' ${five[$i]}"x" - > ${five[$i]}"b"  # put 5% and 15% ACC in one file
awk '{print $1/$2}' ${five[$i]}"b" > ${five[$i]}"c"  # find ratio of 5% and 15%
awk '{print $8}' ${five[0]}"a" > freq.txt # saves freq column to own file for copy of all logs file

## find the log of each value  from the ratio file
     cat ${five[$i]}"c" |
     while read a;
       do
         echo "scale=5; l($a)" | bc -l >> ${five[$i]}"f"
       done 
done
sed -n '$p' temp.txt>temp4.txt
sed -n 's/...$//p' temp4.txt>temp2.txt
sed '$d' temp.txt>temp3.txt
cat temp2.txt>>temp3.txt
mv temp3.txt temp.txt
#     copy all logs of ratios to one file
i=0
declare -a logs
for file in *'f'
do 
    logs[i]=$file
    i=$i+1
done    
paste ${logs[@]}| cut -d' ' -f1 > new.txt
paste freq.txt new.txt > logs.txt
#                                CALCULATING MEAN AND +-1 SIGMA
# AVERAGE
awk '{ sum=0; for (i=2;i<=NF;i++) { sum=sum+$i } print sum/(NF-1)}' logs.txt > avg.txt
paste logs.txt avg.txt >logs2.txt
# STANDARD DEVIATION
awk '{ sum=0; for (i=2;i<=(NF-1);i++) { sum=sum+($i-$(NF))*($i-$(NF)) } print sqrt(sum/(NF-3))}' logs2.txt > std.txt
paste avg.txt std.txt > avgstd.txt
# +/- SIGMA
awk '{print $1+$2}' avgstd.txt > plus.txt
awk '{print $1-$2}' avgstd.txt > minus.txt
# exponent of MEAN, AND =/- 1 SIGMA
awk '{print 2.71828**$1}' avg.txt > eavg.txt
awk '{print 2.71828**$1}' plus.txt > eplus.txt
awk '{print 2.71828**$1}' minus.txt > eminus.txt
paste freq.txt eavg.txt > ratio.txt
paste ratio.txt eplus.txt > ratio2.txt
paste ratio2.txt eminus.txt > final.txt
gnuplot < temp.txt
cd ..
cd ..



#                                                         CARES TH FIT 
#                                                    5% DAMPED RS W/ TARGET
echo 'set term postscript dashed color' > temp.txt
echo 'set output "CARES-5.ps"' >> temp.txt
echo 'set logscale x' >> temp.txt
echo 'set key out' >> temp.txt
echo 'set style line 1 lt -1 lw 2 ' >> temp.txt
echo 'set xlabel "Frequency (Hz)"' >> temp.txt
echo 'set ylabel "ABS. ACC (g)"' >> temp.txt
echo 'set title  "5% Damped CARES TH FIT ARS \n WUS Rock - Magnitude 6.5 \n Distance 50-100 km"' >> temp.txt
echo 'plot "CARES.crs" u 1:3 w l ls 1 t "TARGET RS", \' >> temp.txt
filelist=`ls -1 *'-5.crs'`
for fname in $filelist; do
echo '"'$fname'" u 1:2 w l lw 2 t "'$fname'", \'>>temp.txt
done
sed -n '$p' temp.txt>temp4.txt
sed -n 's/...$//p' temp4.txt>temp2.txt
sed '$d' temp.txt>temp3.txt
cat temp2.txt>>temp3.txt
mv temp3.txt temp.txt
rm temp4.txt temp2.txt
gnuplot < temp.txt
rm temp.txt

#                                                CARES 15% DAMPED RS W/O TARGET
echo 'set term postscript dashed color' > temp.txt
echo 'set output "CARES-15.ps"' >> temp.txt
echo 'set logscale x' >> temp.txt
echo 'set key out' >> temp.txt
echo 'set xlabel "Frequency (Hz)"' >> temp.txt
echo 'set ylabel "ABS. ACC (g)"' >> temp.txt
echo 'set title  "15% Damped CARES TH FIT ARS \n WUS Rock - Magnitude 6.5 \n Distance 50-100 km"' >> temp.txt
echo 'plot \' >> temp.txt
filelist=`ls -1 *'-15.crs'`
for fname in $filelist; do
echo '"'$fname'" u 1:2 w l lw 2 t "'$fname'", \'>>temp.txt
done
sed -n '$p' temp.txt>temp4.txt
sed -n 's/...$//p' temp4.txt>temp2.txt
sed '$d' temp.txt>temp3.txt
cat temp2.txt>>temp3.txt
mv temp3.txt temp.txt
rm temp4.txt temp2.txt
gnuplot < temp.txt
rm temp.txt

#                                                        CARES RATIO
mkdir CRSRATIO
cp *'.crs' CRSRATIO
cd CRSRATIO
i=0
j=0
declare -a five
declare -a fifteen
for file in *'-5.crs'
do
  five[i]=$file
  i=$i+1
done
for file in *'-15.crs'
do 
  fifteen[j]=$file
  j=$j+1
done
echo 'set term postscript enhanced color' > temp.txt
echo 'set output "CARES-RATIO.eps"' >> temp.txt
echo 'set logscale x' >> temp.txt
echo 'set key out' >> temp.txt
#echo 'set label "MEAN" at 55,1.05' >> temp.txt
#echo 'set label "+{/Symbol s}" at 55, 1.13' >> temp.txt
#echo 'set label "-{/Symbol s}" at 55, 0.98' >> temp.txt 
echo 'set xlabel "Frequency (Hz)"' >> temp.txt
echo 'set ylabel "ABS. ACC (g)"' >> temp.txt
echo 'set title  "5%-15% Damped Spectral Ratio of CARES TH FIT\n WUS Rock - Magnitude 6.5 \n Distance 50-100 km"' >> temp.txt
echo 'plot "final.txt" u 1:2 w l lw 3 lt -1 t "MEAN" , \' >> temp.txt
echo '"final.txt" u 1:3 w l lw 3 lt -1 t "+{/Symbol s}" , \' >> temp.txt
echo '"final.txt" u 1:4 w l lw 3 lt -1 t "-{/Symbol s}", \'>> temp.txt
for (( i=0; i<${#five[@]};i++))
do
echo '"< paste '${five[$i]}' '${fifteen[$i]}'" u 1:($2/$4) w l lw 2 t "'${five[$i]}'" , \' >> temp.txt
sed -e '1,7d' ${five[$i]}>${five[$i]}"a"   #remove first 7 lines of file to strip out text
sed -e '1,7d' ${fifteen[$i]}>${fifteen[$i]}"a"
awk '{print $2}' ${fifteen[$i]}"a" | paste -d' ' ${five[$i]}"a" - > ${five[$i]}"b"  # put 5% and 15% ACC in one file
awk '{print $1,$2/$3}' ${five[$i]}"b" > ${five[$i]}"c"  # find ratio of 5% and 15%
awk '{print $1}' ${five[0]}"c" > freq.txt # saves freq column to own file for copy of all logs file
awk '{print $2}' ${five[$i]}"c" > ${five[$i]}"e"  #strip freq column from ratio file
## find the log of each value  from the ratio file
     cat ${five[$i]}"e" |
     while read a;
       do
         echo "scale=5; l($a)" | bc -l >> ${five[$i]}"f"
       done       
done
sed -n '$p' temp.txt>temp4.txt
sed -n 's/...$//p' temp4.txt>temp2.txt
sed '$d' temp.txt>temp3.txt
cat temp2.txt>>temp3.txt
mv temp3.txt temp.txt

#     copy all logs of ratios to one file
i=0
declare -a logs
for file in *'f'
do 
    logs[i]=$file
    i=$i+1
done    
paste ${logs[@]}| cut -d' ' -f1 > new.txt
paste freq.txt new.txt > logs.txt
#                                CALCULATING MEAN AND +-1 SIGMA
# AVERAGE
awk '{ sum=0; for (i=2;i<=NF;i++) { sum=sum+$i } print sum/(NF-1)}' logs.txt > avg.txt
paste logs.txt avg.txt >logs2.txt
# STANDARD DEVIATION
awk '{ sum=0; for (i=2;i<=(NF-1);i++) { sum=sum+($i-$(NF))*($i-$(NF)) } print sqrt(sum/(NF-3))}' logs2.txt > std.txt
paste avg.txt std.txt > avgstd.txt
# +/- SIGMA
awk '{print $1+$2}' avgstd.txt > plus.txt
awk '{print $1-$2}' avgstd.txt > minus.txt
# exponent of MEAN, AND =/- 1 SIGMA
awk '{print 2.71828**$1}' avg.txt > eavg.txt
awk '{print 2.71828**$1}' plus.txt > eplus.txt
awk '{print 2.71828**$1}' minus.txt > eminus.txt
paste freq.txt eavg.txt > ratio.txt
paste ratio.txt eplus.txt > ratio2.txt
paste ratio2.txt eminus.txt > final.txt
gnuplot < temp.txt
cd ..


#                                                        SEEDS 
#                                                          5% 
echo 'set term postscript dashed color' > temp.txt
echo 'set output "SEEDS-5.ps"' >> temp.txt
echo 'set logscale x' >> temp.txt
echo 'set key out' >> temp.txt
echo 'set xlabel "Frequency (Hz)"' >> temp.txt
echo 'set ylabel "ABS. ACC (g)"' >> temp.txt
echo 'set title  "5% Damped SEEDS ARS \n WUS Rock - Magnitude 6.5 \n Distance 50-100 km"' >> temp.txt
echo 'plot \' >> temp.txt
filelist=`ls -1 *'-5.srs'`
for fname in $filelist; do
echo '"'$fname'" u 1:2 w l lw 2 t "'$fname'", \'>>temp.txt
done
sed -n '$p' temp.txt>temp4.txt
sed -n 's/...$//p' temp4.txt>temp2.txt
sed '$d' temp.txt>temp3.txt
cat temp2.txt>>temp3.txt
mv temp3.txt temp.txt
rm temp4.txt temp2.txt
gnuplot < temp.txt
rm temp.txt

#                                                    15% DAMPED RS W/O TARGET
echo 'set term postscript dashed color' > temp.txt
echo 'set output "SEEDS-15.ps"' >> temp.txt
echo 'set logscale x' >> temp.txt
echo 'set key out' >> temp.txt
echo 'set xlabel "Frequency (Hz)"' >> temp.txt
echo 'set ylabel "ABS. ACC (g)"' >> temp.txt
echo 'set title  "15% Damped SEEDS ARS \n WUS Rock - Magnitude 6.5 \n Distance 50-100 km"' >> temp.txt
echo 'plot \'>> temp.txt
filelist=`ls -1 *'-15.srs'`
for fname in $filelist; do
echo '"'$fname'" u 1:2 w l lw 2 t "'$fname'", \'>>temp.txt
done
sed -n '$p' temp.txt>temp4.txt
sed -n 's/...$//p' temp4.txt>temp2.txt
sed '$d' temp.txt>temp3.txt
cat temp2.txt>>temp3.txt
mv temp3.txt temp.txt
rm temp4.txt temp2.txt
gnuplot < temp.txt

#                                                            SEEDS RATIO                                                    
mkdir SRSRATIO
cp *'.srs' SRSRATIO
cd SRSRATIO
i=0
j=0
declare -a five
declare -a fifteen
for file in *'-5.srs'
do
  five[i]=$file
  i=$i+1
done
for file in *'-15.srs'
do 
  fifteen[j]=$file
  j=$j+1
done
echo 'set term postscript enhanced color' > temp.txt
echo 'set output "SEED-RATIO.eps"' >> temp.txt
echo 'set logscale x' >> temp.txt
echo 'set key out' >> temp.txt
#echo 'set label "MEAN" at 55,1.05' >> temp.txt
#echo 'set label "+{/Symbol s}" at 55, 1.13' >> temp.txt
#echo 'set label "-{/Symbol s}" at 55, 0.98' >> temp.txt 
echo 'set xlabel "Frequency (Hz)"' >> temp.txt
echo 'set ylabel "ABS. ACC (g)"' >> temp.txt
echo 'set title  "5%-15% Damped Spectral Ratio of SEEDS\n WUS Rock - Magnitude 6.5 \n Distance 50-100 km"' >> temp.txt
echo 'plot "final.txt" u 1:2 w l lw 3 lt -1 t "MEAN" , \' >> temp.txt
echo '"final.txt" u 1:3 w l lw 3 lt -1 t "+{/Symbol s}" , \' >> temp.txt
echo '"final.txt" u 1:4 w l lw 3 lt -1 t "-{/Symbol s}", \'>> temp.txt
for (( i=0; i<${#five[@]};i++))
do
echo '"< paste '${five[$i]}' '${fifteen[$i]}'" u 1:($2/$4) w l lw 2 t "'${five[$i]}'" , \' >> temp.txt
sed -e '1,7d' ${five[$i]}>${five[$i]}"a"   #remove first 7 lines of file to strip out text
sed -e '1,7d' ${fifteen[$i]}>${fifteen[$i]}"a"
awk '{print $2}' ${fifteen[$i]}"a" | paste -d' ' ${five[$i]}"a" - > ${five[$i]}"b"  # put 5% and 15% ACC in one file
awk '{print $1,$2/$3}' ${five[$i]}"b" > ${five[$i]}"c"  # find ratio of 5% and 15%
awk '{print $1}' ${five[0]}"c" > freq.txt # saves freq column to own file for copy of all logs file
awk '{print $2}' ${five[$i]}"c" > ${five[$i]}"e"  #strip freq column from ratio file
## find the log of each value  from the ratio file
     cat ${five[$i]}"e" |
     while read a;
       do
         echo "scale=5; l($a)" | bc -l >> ${five[$i]}"f"
       done       
done
sed -n '$p' temp.txt>temp4.txt
sed -n 's/...$//p' temp4.txt>temp2.txt
sed '$d' temp.txt>temp3.txt
cat temp2.txt>>temp3.txt
mv temp3.txt temp.txt

#     copy all logs of ratios to one file
i=0
declare -a logs
for file in *'f'
do 
    logs[i]=$file
    i=$i+1
done    
paste ${logs[@]}| cut -d' ' -f1 > new.txt
paste freq.txt new.txt > logs.txt
#                                CALCULATING MEAN AND +-1 SIGMA
# AVERAGE
awk '{ sum=0; for (i=2;i<=NF;i++) { sum=sum+$i } print sum/(NF-1)}' logs.txt > avg.txt
paste logs.txt avg.txt >logs2.txt
# STANDARD DEVIATION
awk '{ sum=0; for (i=2;i<=(NF-1);i++) { sum=sum+($i-$(NF))*($i-$(NF)) } print sqrt(sum/(NF-3))}' logs2.txt > std.txt
paste avg.txt std.txt > avgstd.txt
# +/- SIGMA
awk '{print $1+$2}' avgstd.txt > plus.txt
awk '{print $1-$2}' avgstd.txt > minus.txt
# exponent of MEAN, AND =/- 1 SIGMA
awk '{print 2.71828**$1}' avg.txt > eavg.txt
awk '{print 2.71828**$1}' plus.txt > eplus.txt
awk '{print 2.71828**$1}' minus.txt > eminus.txt
paste freq.txt eavg.txt > ratio.txt
paste ratio.txt eplus.txt > ratio2.txt
paste ratio2.txt eminus.txt > final.txt
gnuplot < temp.txt
cd ..

mkdir PLOTS
mv `find . -name "*.ps"` PLOTS
mv `find . -name "*.eps"` PLOTS
cd PLOTS
for i in *.ps; do ps2pdf $i; done
for i in *.eps; do epstopdf $i; done
rm *.eps
rm *.ps
exit 0

Arch Linux

#1 2008-07-15 16:34:44

pasting columns in bash

#2 2008-07-15 17:10:51

Re: pasting columns in bash

#3 2008-07-15 19:15:03

Re: pasting columns in bash

#4 2008-07-15 19:31:24

Re: pasting columns in bash

#5 2008-07-15 20:09:58

Re: pasting columns in bash

#6 2008-07-15 23:28:20

Re: pasting columns in bash

#7 2008-07-16 00:02:00

Re: pasting columns in bash

#8 2008-07-16 01:54:15

Re: pasting columns in bash

#9 2008-07-16 08:49:41

Re: pasting columns in bash

#10 2008-07-16 16:22:18

Re: pasting columns in bash

#11 2008-07-16 17:11:31

Re: pasting columns in bash

#12 2008-07-16 18:41:48

Re: pasting columns in bash

#13 2008-07-16 19:14:20

Re: pasting columns in bash

#14 2008-07-16 19:19:10

Re: pasting columns in bash

#15 2008-07-16 21:10:52

Re: pasting columns in bash

#16 2008-07-21 16:35:11

Re: pasting columns in bash

#17 2008-07-21 17:48:46

Re: pasting columns in bash

#18 2008-07-21 19:17:08

Re: pasting columns in bash

#19 2008-07-21 20:48:58

Re: pasting columns in bash

#20 2008-07-21 21:01:17

Re: pasting columns in bash

#21 2008-07-21 21:39:37

Re: pasting columns in bash

#22 2008-07-25 17:04:17

Re: pasting columns in bash

Board footer