more elegant code to pull some values from an output [SOLVED]

graysky · 2010-10-23 14:41:00

I'd like to capture the GPU load and memory usage of my gfx card from the following output:

$ nvidia-smi -a

==============NVSMI LOG==============


Timestamp            : Sat Oct 23 10:39:53 2010

Driver Version            : 260.19.12

GPU 0:
    Product Name        : GeForce 8400 GS
    PCI Device/Vendor ID    : 6e410de
    PCI Location ID        : 0:1:0
    Display            : Connected
    Temperature        : 41 C
    Utilization
        GPU            : 0%
        Memory        : 8%

I came up with the following but it's kind of clunky. Can other more adept in bash suggest a more simplistic approach?

#!/bin/bash

gpu=`nvidia-smi -a | grep GPU | tail -n+2 | awk '/:/{print $NF}'`
mem=`nvidia-smi -a | grep Memory |  awk '/:/{print $NF}'`

echo "GPU load: "${gpu}
echo "VRAM use: "${mem}

Last edited by graysky (2010-10-23 22:40:59)

juster · 2010-10-23 17:09:17

In my opinion if you are piping grep output into awk you are not using awk properly. awk has grepping bultin. The awk command below sets the field separator to a regexp. This regexp matches a colon char and the surrounding white-space. This helps to automatically split up the field name and value into $1 and $2. You could keep track of state in more complex ways but this seems to work on your example data. This runs nvidia-smi and scans through it once instead of twice and creates two processes instead of seven.

nvidia-smi -a | awk -F ' *: *' '
/GPU [0-9]+:/ { print; }
$1 ~ /GPU$/   { print "GPU load:", $2; }  # Use $, don't match the top GPU
$1 ~ /Memory/ { print "VRAM use:", $2; }
'

graysky · 2010-10-23 17:28:01

I am very green with it comes to awk. I tried your code in the following script but got these errors:

$ cat ~/bin/gpu2 && gpu2
#!/bin/bash
nvidia-smi -a | awk -F ' *: *' '
/GPU [0-9]+:/ { print; }
$1 ~ /GPU$/   { print "GPU load:", $2; }  # Use $, don't match the top GPU
$1 ~ /Memory/ { print "VRAM use:", $2; }
'

awk: cmd. line:3: fatal: cannot open file `match' for reading (No such file or directory)
~/bin/gpu2: line 5: syntax error near unexpected token `}'
~/bin/gpu2: line 5: `$1 ~ /Memory/ { print "VRAM use:", $2; }'

kachelaqa · 2010-10-23 18:02:37

graysky wrote:

I am very green with it comes to awk. I tried your code in the following script but got these errors:

$ cat ~/bin/gpu2 && gpu2
#!/bin/bash
nvidia-smi -a | awk -F ' *: *' '
/GPU [0-9]+:/ { print; }
$1 ~ /GPU$/   { print "GPU load:", $2; }  # Use $, don't match the top GPU
$1 ~ /Memory/ { print "VRAM use:", $2; }
'

awk: cmd. line:3: fatal: cannot open file `match' for reading (No such file or directory)
~/bin/gpu2: line 5: syntax error near unexpected token `}'
~/bin/gpu2: line 5: `$1 ~ /Memory/ { print "VRAM use:", $2; }'

remove the comment ("# Use $, don't match the top GPU") and it should work.

graysky · 2010-10-23 18:58:31

@kach - did it, now output is:

GPU 0:
VRAM use: 8%

skanky · 2010-10-23 20:11:06

Can you show what you have now and double check the output of nvidia-smi again, please?

juster · 2010-10-23 20:23:11

Sorry I copy/pasted from an .awk file (script). I also don't have nvidia-smi installed to check with. If the GPU line is not matching it could be because the FS needs to include other whitespace like tabs which were lost in translation. For example, both spaces and tabs: awk -F '[ \t]*:[ \t]*' ... Or with tabs only: awk -F '\t*:\t*' ...

For the $1 ~ /GPU$/ line to succeed, $1 must end in GPU. So there can be no whitespace after it. the field separator must absorb all the whitespace.

Edit: From the jagged lines in your copy/pasted output it looks like tabs come before the colon, and there is one space after. Not that I can guarantee it, but that would mean awk -F '\t*: *' should match.

Last edited by juster (2010-10-23 20:25:10)

skunktrader · 2010-10-23 20:28:12

The line of interest is actually "\t GPU\t\t\t: 0%"

graysky · 2010-10-23 21:54:45

Sorry gang, here are the data you asked for...

$ nvidia-smi -a

==============NVSMI LOG==============


Timestamp            : Sat Oct 23 17:43:07 2010

Driver Version            : 260.19.12

GPU 0:
    Product Name        : GeForce 8400 GS
    PCI Device/Vendor ID    : 6e410de
    PCI Location ID        : 0:1:0
    Display            : Connected
    Temperature        : 48 C
    Utilization
        GPU            : 0%
        Memory        : 8%

Looks like it goes:

<TAB><space><space><space><space>GPU<TAB><TAB>:<space>##%
<TAB><space><space><space><space>Memory<TAB><TAB>:<space>##%

-----------

If I remove the $ from the original script it actually works but it returns too many "GPU" matches:

The modified script:

#!/bin/bash
nvidia-smi -a | awk -F ' *: *' '
/GPU [0-9]+:/ { print; }
$1 ~ /GPU/   { print "GPU load:", $2; }
$1 ~ /Memory/ { print "VRAM use:", $2; }
'

And its output:

GPU 0:
GPU load: 
GPU load: 0%
VRAM use: 8%

It would seem I just need to surpress the output of the first two GPU matches?

skunktrader · 2010-10-23 21:58:08

nvidia-smi -a | hexdump -c

graysky · 2010-10-23 22:08:07

$ nvidia-smi -a | hexdump -c
0000000  \n   =   =   =   =   =   =   =   =   =   =   =   =   =   =   N
0000010   V   S   M   I       L   O   G   =   =   =   =   =   =   =   =
0000020   =   =   =   =   =   =  \n  \n  \n   T   i   m   e   s   t   a
0000030   m   p  \t  \t  \t   :       S   a   t       O   c   t       2
0000040   3       1   8   :   0   7   :   5   1       2   0   1   0  \n
0000050  \n   D   r   i   v   e   r       V   e   r   s   i   o   n  \t
0000060  \t  \t   :       2   6   0   .   1   9   .   1   2  \n  \n   G
0000070   P   U       0   :  \n  \t   P   r   o   d   u   c   t       N
0000080   a   m   e  \t  \t   :       G   e   F   o   r   c   e       8
0000090   4   0   0       G   S  \n  \t   P   C   I       D   e   v   i
00000a0   c   e   /   V   e   n   d   o   r       I   D  \t   :       6
00000b0   e   4   1   0   d   e  \n  \t   P   C   I       L   o   c   a
00000c0   t   i   o   n       I   D  \t  \t   :       0   :   1   :   0
00000d0  \n  \t   D   i   s   p   l   a   y  \t  \t  \t   :       C   o
00000e0   n   n   e   c   t   e   d  \n  \t   T   e   m   p   e   r   a
00000f0   t   u   r   e  \t  \t   :       4   8       C  \n  \t   U   t
0000100   i   l   i   z   a   t   i   o   n  \n  \t                   G
0000110   P   U  \t  \t  \t   :       2   %  \n  \t                   M
0000120   e   m   o   r   y  \t  \t   :       9   %  \n                
000012c

graysky · 2010-10-23 22:12:35

What exactly does the 3rd line do (I have commented it out in the example below - as far as I can tell it doesn't do anything:

#!/bin/bash
nvidia-smi -a | awk -F '\t\t: ' '
#/GPU [0-9]+:/ { print; }
$1 ~ /Temperature/   { print "GPU temp:", $2; }
$1 ~ /GPU/   { print "GPU load:", $2; }
$1 ~ /Memory/ { print "VRAM use:", $2; }
'

Now produces:

GPU load: 
GPU temp: 48 C
GPU load: 0%
VRAM use: 8%

skunktrader · 2010-10-23 22:17:24

reinstate the commented out line and replace it with

/GPU [0-9]+:/ { next; }

graysky · 2010-10-23 22:20:36

That works...

Script:

#!/bin/bash
nvidia-smi -a | awk -F '\t\t: ' '
/GPU [0-9]+:/ { next; }
$1 ~ /Memory/ { print "VRAM use:", $2; }
$1 ~ /Temperature/   { print "GPU temp:", $2; }
$1 ~ /GPU/   { print "GPU load:", $2; }
'

Output:

GPU temp: 48 C
GPU load: 0%
VRAM use: 8%

Can you explain? The syntax of this is confusing to me. Perhaps say what the script does in plain english:

From what I can tell:
Line 2 runs nvidia-smi and pipes it into awk defining the field sep. as \t\t:<space>
Line 3 = ?
Line 4 1st field something matches Memory then prints "VRAM use: 2nd field
What does the ~ do?

Last edited by graysky (2010-10-23 22:23:11)

skunktrader · 2010-10-23 22:32:59

Awk processes each line in the input sequentially.

/GPU [0-9]+:/ { next; }

just says that if the input line matches the regular expression /GPU [0-9]+:/ then execute next = stop trying to process that line and read the next one.
Each of the other lines in your awk program say if the first token $1 matches ~ the regular expression then execute whatever appears in the braces.
Repeat until end of input

skunktrader · 2010-10-23 22:35:59

graysky wrote:

What does the ~ do?

man awk wrote:

A regular expression can be matched against a specific field or string by using one of the two regular expression matching operators, '~' and "!~" . These operators shall interpret their right-hand operand as a regular expression and their left-hand operand as a string. If the regular expression matches the string, the '~' expression shall evaluate to a value of 1, and the "!~" expression shall evaluate to a value of 0.

Last edited by skunktrader (2010-10-23 22:37:26)

graysky · 2010-10-23 22:40:45

I think I get it. Thanks all!

skunktrader · 2010-10-23 22:44:06

If you wanted to be pedantic, you could rewrite

/GPU [0-9]+:/ { next; }

as

$0 ~ /GPU [0-9]+:/ { next; }

Where $0 refers to the entire line

juster · 2010-10-24 06:18:53

graysky wrote:

Script:

#!/bin/bash
nvidia-smi -a | awk -F '\t\t: ' '
/GPU [0-9]+:/ { next; }
$1 ~ /Memory/ { print "VRAM use:", $2; }
$1 ~ /Temperature/   { print "GPU temp:", $2; }
$1 ~ /GPU/   { print "GPU load:", $2; }
'

I put line 3 there in order to print the GPU number but I see you don't want that. You can just remove line 3, it's unnecessary. If you do you will have to put the $ back where you found it in line 6, so that it says '/GPU$/'. The dollar sign has special meaning in a regular expression. It stands for the end of the string (in this case).

When awk reads the GPU 0: line of the input, $1 will be 'GPU 0'. $1 ~ /GPU$/ will not be true in this case, because this will only be true when the string in $1 ends with 'GPU'. Later on, when awk reads the "\t GPU: 2%" line $1 will be "\t GPU" and $1 ~ /GPU$/ will match, running the code in the brackets.

For some quick background: awk reads the text file separating it into records. Records in this case are lines. After it reads a record it separates the record into fields. awk separates the fields based on the field separator. This is a regular expression or string specified by the -F option or stored in the FS variable. So anything that matches the field separator, splits the record into fields. These fields are $1 and $2 for first and second field, and so on.

The field separator (specified with -F) above does not work with the data you gave using hexdump previously. awk will not print the GPU load because there are a different number of tabs (\t) for that line. I changed it to '\t+: '. This means each field is separated by 1 or more tabs, a colon, and a space.

#!/bin/bash
nvidia-smi -a | awk -F '\t+: ' '                                                                 
$1 ~ /Temperature/ { print "GPU temp:", $2; }                                    
$1 ~ /GPU$/        { print "GPU load:", $2; }                                    
$1 ~ /Memory/      { print "VRAM use:", $2; }                                    
'

Arch Linux

#1 2010-10-23 14:41:00

more elegant code to pull some values from an output [SOLVED]

#2 2010-10-23 17:09:17

Re: more elegant code to pull some values from an output [SOLVED]

#3 2010-10-23 17:28:01

Re: more elegant code to pull some values from an output [SOLVED]

#4 2010-10-23 18:02:37

Re: more elegant code to pull some values from an output [SOLVED]

#5 2010-10-23 18:58:31

Re: more elegant code to pull some values from an output [SOLVED]

#6 2010-10-23 20:11:06

Re: more elegant code to pull some values from an output [SOLVED]

#7 2010-10-23 20:23:11

Re: more elegant code to pull some values from an output [SOLVED]

#8 2010-10-23 20:28:12

Re: more elegant code to pull some values from an output [SOLVED]

#9 2010-10-23 21:54:45

Re: more elegant code to pull some values from an output [SOLVED]

#10 2010-10-23 21:58:08

Re: more elegant code to pull some values from an output [SOLVED]

#11 2010-10-23 22:08:07

Re: more elegant code to pull some values from an output [SOLVED]

#12 2010-10-23 22:12:35

Re: more elegant code to pull some values from an output [SOLVED]

#13 2010-10-23 22:17:24

Re: more elegant code to pull some values from an output [SOLVED]

#14 2010-10-23 22:20:36

Re: more elegant code to pull some values from an output [SOLVED]

#15 2010-10-23 22:32:59

Re: more elegant code to pull some values from an output [SOLVED]

#16 2010-10-23 22:35:59

Re: more elegant code to pull some values from an output [SOLVED]

#17 2010-10-23 22:40:45

Re: more elegant code to pull some values from an output [SOLVED]

#18 2010-10-23 22:44:06

Re: more elegant code to pull some values from an output [SOLVED]

#19 2010-10-24 06:18:53

Re: more elegant code to pull some values from an output [SOLVED]

Board footer