[SOLVED]Get all variable names from bash script

alphaniner · 2012-06-01 16:02:24

Does anyone know of a tool that can return the names of all variables in a bash script?

Last edited by alphaniner (2012-06-04 18:41:08)

ewaller · 2012-06-01 16:04:58

set ??

alphaniner · 2012-06-01 16:12:36

I don't want to run the script (it has too many branches) and I can't see how set could be used in this case.

Trilby · 2012-06-01 18:02:48

Interesting. I suppose you could search the file for strings of legal variable-name characters following a "$". This would show most uses of variables. But this would miss anything like "${varname...}". So this is probably out.

Alternately, you could search for all strings of legal variable-name characters that are not inside quotes, then have a for loop test each one with '[[ which "$symbol" ]] || echo $symbol' to get just the strings that are variables. This would also return anything inside a here-document though - an improved version could skip all lines between "<<something" and "something".

So in short: nope, don't know of anything, but depending on how reliable it needs to be, a couple lines of awk script could do the job.

alphaniner · 2012-06-01 18:19:48

Am I the only one who thinks this would be valuable? I'm rather surprised this functionality isn't built into bash itself. I figured that since vim syntax highlighting can do it there must be other reliable ways out there. Normally I find it educational to reinvent the wheel, but in this case I was hoping for a quick solution. Anyway, thanks for the interest. Looks like I'm in for some awk education...

Trilby · 2012-06-01 18:48:19

Hell, if that's all you need, this will do as well as vim's highlighting:

awk '/[A-Za-z0-9]+=/ { gsub(/=.*/,"",$0); print $NF; }' filename

Edit: explanation:
1) find all lines with legal variable characters preceding an equals sign.
2) delete the equals sign and everything after it.
3) print the last field/word on the line.

Last edited by Trilby (2012-06-01 18:49:32)

alphaniner · 2012-06-01 19:12:09

I was just using vim's highlighting as an example that something could do it. And not just assignment lines (if I had assigned all my vars this would be trivial), but anywhere in the script. I actually have a decent method because most of my variables are all caps. My intention is to rename them to lowercase. I was using a regex substitution in vim until I came upon a few instances such as DATET and DATEt. I used the same regex with grep|sort|uniq to make a list; there were a few false matches, but I'm more worried about misses. Anyway, maybe it's just not worth it for the sake of renaming variables. I really thought there would be something already in existence to do this.

Last edited by alphaniner (2012-06-01 19:13:38)

/dev/zero · 2012-06-02 00:21:55

I would consider using (f)lex. It would be fairly easy to echo regexes matching variables.

Trilby · 2012-06-02 00:23:40

+1 on flex. I thought of recommending that as well if a more reliable tool was needed.

Last edited by Trilby (2012-06-02 00:23:50)

falconindy · 2012-06-02 00:36:57

Until you encounter something like:

read
declare "${REPLY//[^[:alnum:]]}=foo"

And that's a simple example... Parsing shell is only possible via the interpreter itself.

Last edited by falconindy (2012-06-02 00:37:09)

/dev/zero · 2012-06-02 00:40:17

Here we go:
bashvar.l

%%
$[a-zA-Z0-9]+       ECHO;
$\{[a-zA-Z0-9]+\}   ECHO;
.   
%%
int yywrap() {
    return 1;
}

int main() {
    yylex();
    return 0;
}

Compile:

lex bashvar.l && gcc lex.yy.c

Sample usage:

$ cat bashrc
#! /bin/bash

# If not running interactively, don't do anything
[ -n "${PS1}" ] || return

# Run scripts in bash.d
if [ -d "$HOME/bash.d" ]; then
    for f in $HOME/bash.d/*; do
        . "$f"
    done
fi
$ a.out < bashrc | sort -u

$f
$HOME
${PS1}

Trilby · 2012-06-02 00:49:18

/d/z, you should add (at least) underscores to the character lists.

Also this would only detect usages of variables (which may be all that's relevant) but it would miss many other bash built in uses with any string substitution ${variable##ext}

Perhaps the regex could be

${?[A-Za-z0-9_]+

/dev/zero · 2012-06-02 04:25:06

I've played around with this a bit more. I'm not sure what falconindy is referring to, and I'm not familiar with this notion of string substitutions. I've now brought yacc into it, so anyone interested could probably expand the grammar to deal with such oddities.

bashvar.l

%{
#include "y.tab.h"
%}
VAR [a-zA-Z0-9_]+
%%
{VAR} yylval.strval = strdup( yytext ); return VARIABLE;
[${}=]   return *yytext;
\n  
.     
%%
int yywrap() {
    return 1;
}

bashvar.y

%{
#include <fcntl.h>
#include <stdio.h>
#include <errno.h>
#include <sys/stat.h>

extern FILE *yyin;
%}
%token <strval> VARIABLE
%union{
    char* strval;
};
%%
prog:
    | prog variable
    ;

variable:
    '$' VARIABLE     { printf( "%s\n", $2 ); }
    | '$' '{' VARIABLE '}' { printf( "%s\n", $3 ); }
    | VARIABLE          
    | assignment
    ;

assignment:
    VARIABLE '='        { printf( "%s\n", $1 ); }
     ;

%%
int yyerror( char *s ) {
    fprintf( stderr, "%s\n", s );
    return 0;
}

int main( int argc, char *argv[] ) {
    if( argc > 1 ) {
        yyin = fopen( argv[ 1 ], "r" );
        if( yyin == NULL ) {
            perror( "Cannot open input file" );
            printf( "Error number: %d\n", errno );
            return 1;
        }
    }
    
    yyparse();
    return 0;
}

Makefile

NAME=bashvar

.PHONY: all clean
all: lex.yy.c y.tab.h y.tab.c
	gcc lex.yy.c y.tab.c -o $(NAME)

lex.yy.c:
	flex $(NAME).l

y.tab.h:
	yacc -d $(G_FLAG) $(NAME).y

clean:
	rm -f lex.yy.c y.tab.h y.tab.c $(NAME)

Sample usage:

$ make clean && make
rm -f lex.yy.c y.tab.h y.tab.c bashvar
flex bashvar.l
yacc -d  bashvar.y
gcc lex.yy.c y.tab.c -o bashvar
$ cat variables 
foo="bar"
echo nothing to see here ${except_this} and $this.
$ bashvar variables 
foo
except_this
this
$ bashvar < variables 
foo
except_this
this
$ cat variables | bashvar
foo
except_this
this

EDIT: improved some code. Made it possible to pass a filename argument as well as use pipelines/redirection.

Last edited by /dev/zero (2012-06-02 05:27:13)

/dev/zero · 2012-06-03 07:44:19

Solved or unsolved?

alphaniner · 2012-06-04 13:55:28

Unfortunately, your tool didn't make it far into my script. It quits at function declarations, "$$", "$(( $VAR ...))", and worst of all "$VAR = string" (such as in [[ "$VAR" = string ]] ). It also picks up false positives from "$( grep ...)", "$(( digit ...))", and "foo = bar". I assume the latter is related to the equality test crap out.

Last edited by alphaniner (2012-06-04 14:02:45)

Trilby · 2012-06-04 17:41:47

How about my awk oneliner from post #6. It's worked on all my scripts.

debdj · 2012-06-04 18:04:30

@Trilby, tried that on the makepkg script
almost perfect, but needs some refinement.

Last edited by debdj (2012-06-04 18:12:31)

Trilby · 2012-06-04 18:25:21

By all means refine. Or let me know where it fails as I cannot refine it without knowing what's wrong.

By makepkg script, you you mean a PKGBUILD? PKGBUILDS use many evironment variables that they don't define. It seemed the goal was to find variables that were defined or set in a script.

alphaniner · 2012-06-04 18:40:32

Trilby wrote:

How about my awk oneliner from post #6. It's worked on all my scripts.

Sorry. I kind of wrote it off because it only catches 'declarations'. I realize now that was stupid. With my knowledge of bash there's not much it would miss ('read var' being the only practical one I can think of).

After a sort|uniq 102 items were returned, so it may be a while until I can say how precise it is. But in any case, thanks. It's much better than my grep and it looks like it will be very helpful. I'm going to go ahead and mark the thread solved.

debdj · 2012-06-04 20:39:02

Trilby wrote:

By makepkg script, you you mean a PKGBUILD?

No.
/usr/bin/makepkg

Arch Linux

#1 2012-06-01 16:02:24

[SOLVED]Get all variable names from bash script

#2 2012-06-01 16:04:58

Re: [SOLVED]Get all variable names from bash script

#3 2012-06-01 16:12:36

Re: [SOLVED]Get all variable names from bash script

#4 2012-06-01 18:02:48

Re: [SOLVED]Get all variable names from bash script

#5 2012-06-01 18:19:48

Re: [SOLVED]Get all variable names from bash script

#6 2012-06-01 18:48:19

Re: [SOLVED]Get all variable names from bash script

#7 2012-06-01 19:12:09

Re: [SOLVED]Get all variable names from bash script

#8 2012-06-02 00:21:55

Re: [SOLVED]Get all variable names from bash script

#9 2012-06-02 00:23:40

Re: [SOLVED]Get all variable names from bash script

#10 2012-06-02 00:36:57

Re: [SOLVED]Get all variable names from bash script

#11 2012-06-02 00:40:17

Re: [SOLVED]Get all variable names from bash script

#12 2012-06-02 00:49:18

Re: [SOLVED]Get all variable names from bash script

#13 2012-06-02 04:25:06

Re: [SOLVED]Get all variable names from bash script

#14 2012-06-03 07:44:19

Re: [SOLVED]Get all variable names from bash script

#15 2012-06-04 13:55:28

Re: [SOLVED]Get all variable names from bash script

#16 2012-06-04 17:41:47

Re: [SOLVED]Get all variable names from bash script

#17 2012-06-04 18:04:30

Re: [SOLVED]Get all variable names from bash script

#18 2012-06-04 18:25:21

Re: [SOLVED]Get all variable names from bash script

#19 2012-06-04 18:40:32

Re: [SOLVED]Get all variable names from bash script

#20 2012-06-04 20:39:02

Re: [SOLVED]Get all variable names from bash script

Board footer