calling rows and columns using a script

jnwebb · 2008-11-11 20:50:12

I have a file as such

         1         2 PX 
 0.446E+00 0.867E+00-0.123E+00-0.372E+00 0.285E+00-0.640E+00-0.163E+01-0.658E+00
-0.173E+00-0.845E+00-0.446E+00-0.366E+00-0.194E+01-0.194E+01-0.214E-01-0.400E-01
-0.151E+01-0.167E+01-0.173E+01-0.232E+01-0.173E+01-0.121E+01-0.234E+01-0.277E+01
-0.126E+01 0.940E+00 0.404E+01 0.659E+01 0.513E+01 0.198E+01 0.237E+01 0.518E+01
         2         2 PX 
 0.315E+00 0.909E+00 0.104E+00-0.475E+00 0.141E+00-0.449E+00-0.156E+01-0.751E+00
 0.553E-01-0.626E+00-0.640E+00-0.494E+00-0.189E+01-0.213E+01-0.125E+00 0.285E+00
-0.132E+01-0.180E+01-0.172E+01-0.232E+01-0.187E+01-0.103E+01-0.193E+01-0.268E+01
-0.130E+01 0.974E+00 0.370E+01 0.610E+01 0.516E+01 0.218E+01 0.232E+01 0.522E+01
         3         2 PX 
 0.217E+00 0.116E+01 0.552E+00-0.490E+00-0.892E-01-0.303E+00-0.125E+01-0.512E+00
 0.315E+00-0.672E+00-0.104E+01-0.559E+00-0.162E+01-0.222E+01-0.476E+00 0.166E+00
-0.131E+01-0.167E+01-0.142E+01-0.234E+01-0.225E+01-0.901E+00-0.123E+01-0.218E+01
-0.107E+01 0.994E+00 0.320E+01 0.551E+01 0.516E+01 0.225E+01 0.193E+01 0.502E+01
         4         2 PX 
 0.110E+01 0.169E+01 0.538E+00-0.103E+01-0.487E+00 0.154E+00 0.134E+00 0.176E+01
 0.325E+01 0.179E+01-0.308E+00-0.155E+01-0.361E+01-0.457E+01-0.233E+01-0.505E+00
-0.111E+01-0.130E+01-0.128E+01-0.288E+01-0.374E+01-0.274E+01-0.232E+01-0.160E+01
 0.147E+01 0.459E+01 0.612E+01 0.683E+01 0.511E+01 0.116E+01 0.203E+00 0.348E+01
.
.
.

Where each section (1,2,3,4...) represents a different spring element. The data is the force in the spring at time t reading from left to right and then on to the next row (i.e. row1 col1 is time t(1), row1 col2 is time t(2), etc going across)

I would like to sum the forces in each spring at time t (ie call the force at time t(1) for all elements and add them together. So that in the end I have the sum of the forces in all elements at each time t. I would then like to find the time t with the greatest sum and return to a file the force at that time for each element.

Any help would be appreciated:)

Procyon · 2008-11-11 21:59:51

Get a sorted list of total forces:

sed -e 's/.*\([0-9][0-9]*\).*[0-9][0-9]* PX.*/\nSPRING: \1/' -e 's/\([^E:]\)\([ -]\)/\1\n\2/g' file | awk 'BEGIN { RS="";FS="\n"} {for (i=2; i<NF+1; i++) { record[(i-1)]=record[(i-1)]+$i}} END {for (times in record) {print record[times] "\tt=" times}}' | sort -nr

Get composing springs of the max:

grep 999t=$(sed -e 's/.*\([0-9][0-9]*\).*[0-9][0-9]* PX.*/\nSPRING: \1/' -e 's/\([^E:]\)\([ -]\)/\1\n\2/g' file | awk 'BEGIN { RS="";FS="\n"} {for (i=2; i<NF+1; i++) { record[(i-1)]=record[(i-1)]+$i} for (i=2; i<NF+1; i++) {data[(i-1)]=data[(i-1)] $i}} END {for (times in record) {print record[times] "\tt=" times} for (times in data) {print "-999t=" times " " data[times]}}' | tee tempfile.txt | sort -nr | head -n 1 | cut -d= -f2) tempfile.txt | sed 's/[^ ]* *//'

NB: second one uses a tempfile.txt
NB2: I started out changing ' 9 2 PX' into SPRING: 9, but it really isn't necessary.
NB3: it uses tee, if you don't have it, then it will take a bit more than 1 line.
NB4: it presumes some stuff about the numbers:
a space or - before each force is never prepended by a : or E. I think this is reasonable. But no other space or minuses must be in the file! (the : takes care of SPRING: 9, which can be removed, and the E for 0.999E-1, and "^ " is alright too)
I checked the results with the file and a calculator for one entry and it seems to work.

EDIT:
I have to add that I only tested with those 4 entries, so a bit more (especially 10+) would be better. But see what it does. I'll duplicate some to 10+ to test in a minute.

EDIT:
off-by-one. NF in the for loop should be NF+1

Last edited by Procyon (2008-11-11 22:09:52)

Procyon · 2008-11-11 22:21:03

And now let's see what every part does (before I forget) and you want this right?

Command 1:

's/.*$[0-9][0-9]*$.*[0-9][0-9]* PX.* <-- matches the PX lines. "....1-inf numbers (Remember)...1-inf numbers space PX ....."

/\nSPRING: \1/' < change to SPRING: the number above and an extra newline to make paragraphs which is useful for awk below.

's/$[^E:]$$[ -]$/ <-- match a space or minus and the character before it (see NB 4 above)

\1\n\2/g' <-- and put a new line between them. This way all the numbers are on a separate line

'BEGIN { RS="";FS="\n"} <-- RS="" will match paragraphs.

{for (i=2; i<NF+1; i++) { record[(i-1)]=record[(i-1)]+$i}} <-- Field 1 is SPRING: 9, otherwise record everything by adding up. awk can handle scientific notation.
There is a i-1 with record because time starts at 1 and i at 2. This could all make more sense without the spring line.

END {for (times in record) {print record[times] "\tt=" times}}' <-- print the totals and the time that belongs with it.

Command 2:

grep 999t= <-- the output of the spring data will be of the form -999 just because it will be at the bottom of sort -rn.

$(sed -e 's/.*$[0-9][0-9]*$.*[0-9][0-9]* PX.*/\nSPRING: \1/' -e 's/$[^E:]$$[ -]$/\1\n\2/g' file | awk 'BEGIN { RS="";FS="\n"} {for (i=2; i<NF+1; i++) { record[(i-1)]=record[(i-1)]+$i} <-- identical as above.

for (i=2; i<NF+1; i++) {data[(i-1)]=data[(i-1)] $i}} <-- this will append the actual string data as opposed to adding it up and losing the original data.

END {for (times in record) {print record[times] "\tt=" times} <-- identical

for (times in data) {print "-999t=" times " " data[times]}}' <-- -999t= prepended because see above

| tee tempfile.txt <-- tee copies the data because we will match the top total force with the data of the spring

| sort -nr | head -n 1 | cut -d= -f2 <-- gives the single highest force

) tempfile.txt <-- Command becomes: grep '999t=high_number' tempfile

| sed 's/[^ ]* *//' <-- get rid of garbage at the start.

jnwebb · 2008-11-11 22:27:48

Thanks, the problem is the that these are time dependant values so they cant be sorted... should look something like:

Force Value at time # spring1 spring2 spring3 ....... sum at time #
1 0.446E+00 0.315E+00 0.217E+00 sum across
2
3
.
.
.

Again, the forces are ordered row 1 (1-8), row 2 (9-16), etc

Procyon · 2008-11-11 22:39:10

EDIT: not a reply to your post.

Here is command 2 again, but then without putting SPRING: n in there. This made the for-loops a lot simpler.

It will print:
"max force TAB t=time
t=time spring data at that time"

and it will put spring data between positive and negative in tempfile.txt

Also added newlines.

grep t=$(sed \
-e 's/.*PX.*//' \
-e 's/\([^E]\)\([ -]\)/\1\n\2/g' file | \
awk '\
BEGIN { \
  RS=""
  FS="\n" } 
{
  for (i=1; i<NF; i++) {
    record[i]=record[i]+$i
  }
  for (i=1; i<NF; i++) {
    data[i]=data[i] $i
  }
}
END {
  for (times in record) {
    print record[times] "\tt=" times
  }
  for (times in data) {
    print "t=" times " " data[times]
  }
}' | sort -nr | tee tempfile.txt | head -n 1 | cut -d= -f2) tempfile.txt

Last edited by Procyon (2008-11-11 22:39:44)

Procyon · 2008-11-11 22:44:36

jnwebb wrote:

Thanks, the problem is the that these are time dependant values so they cant be sorted... should look something like:
Force Value at time # spring1 spring2 spring3 ....... sum at time #
1 0.446E+00 0.315E+00 0.217E+00 sum across
2
3
.
.
.
Again, the forces are ordered row 1 (1-8), row 2 (9-16), etc

Isn't that exactly what command 2 produces in tempfile.txt between the positive and negative sums? Except for the sums themselves then. If you want this format it is also possible.

EDIT: And it is possible to sort from e.g. the 34th column.

Last edited by Procyon (2008-11-11 22:48:32)

Procyon · 2008-11-11 22:56:06

Actually sorting from a distant column is impossible.

You both don't know which one it is which is necessary, and I don't think numeric sorting works for anything but column 1 (I think it's a bug)

Try this:

sed -e 's/.*PX.*//' -e 's/\([^E]\)\([ -]\)/\1\n\2/g' file | awk '
BEGIN {
  RS=""
  FS="\n"
}
{
  for (i=1; i<NF; i++) {
    record[i]=record[i]+$i
    data[i]=data[i] " " $i
  }
}
END {
  for (times in record) {
    print record[times] " <sum " "t=" times " " data[times]
  }
}' | sort -rn

EDIT: You can pipe this to sed to move column 1 to the back of course (sed 's/$[0-9.]*$ <sum $.*$/\2 \1/')

Last edited by Procyon (2008-11-11 22:59:59)

Procyon · 2008-11-12 14:45:42

OK mistake in the last bit there.

The sum can be negative, so add a - to the brackets.

So append:

 | sed 's/\([0-9.-]*\) <sum \(.*\)/\2 \1/'

klixon · 2008-11-13 16:02:01

I like these puzzles... Here my way too complicated awkness:

#!/bin/awk -f

/2 PX/ {
    line = 0
    spring = $1
    springs[numspring++] = spring
}

interested {
    for (i = 1; i <= NF; i++) {
        time = line * 8 + i
        force[time, spring] = $i
        sumforce[time] += $i
    }
    ++line
}

!interested {
    interested = !interested
    FIELDWIDTHS = "10 10 10 10 10 10 10 10"
}

line == 4 {
    interested = !interested
    FS = FS
}

END {
    for (time in sumforce) {
        if (! maxforce || sumforce[time] == maxforce) {
            maxtime[count++] = time
        }
        if (sumforce[time] > maxforce) {
            maxforce = sumforce[time]
            count = 0
            maxtime[count++] = time
        }
    }
    print "The maximum force of " maxforce " was found " count " times at these timepoints:"
    for (i = 0; i < count; i++) {
        time = maxtime[i]
        print count ": " time " (row " int(time / 8) + 1 ", col " time % 8 ")"
        for (numspring in springs) {
            spring = springs[numspring]
            print "   Force in spring " spring " was " force[time, spring]
        }
    }
}

output:

The maximum force of 25.03 was found 1 times at these timepoints:
1: 28 (row 4, col 4)
   Force in spring 1 was  0.659E+01
   Force in spring 2 was  0.610E+01
   Force in spring 3 was  0.551E+01
   Force in spring 4 was  0.683E+01

EDIT: save it to a file, chmod u+x it and type "<saved_file> input_file"

Last edited by klixon (2008-11-13 17:14:42)

Arch Linux

#1 2008-11-11 20:50:12

calling rows and columns using a script

#2 2008-11-11 21:59:51

Re: calling rows and columns using a script

#3 2008-11-11 22:21:03

Re: calling rows and columns using a script

#4 2008-11-11 22:27:48

Re: calling rows and columns using a script

#5 2008-11-11 22:39:10

Re: calling rows and columns using a script

#6 2008-11-11 22:44:36

Re: calling rows and columns using a script

#7 2008-11-11 22:56:06

Re: calling rows and columns using a script

#8 2008-11-12 14:45:42

Re: calling rows and columns using a script

#9 2008-11-13 16:02:01

Re: calling rows and columns using a script

Board footer