[SOLVED] Parsing simple config files in pure bash

chimeric · 2007-11-04 21:00:36

Hi everyone,

I'd like to implement a bash function to parse a simple config file for a script I've wrote. The conifg file contains different sections for different scenarios. Here's an example how it should look like:

# configuration file

# global settings
global  {
DATE_PREFIX="-I"
SSHFS_OPTS="-C"
EXT_FULL="full"
EXT_DIFF="diff"
EXT_CATALOGUE="catalogue"
DAR_OPTS="-v -m 256 -y -s 600M -D"
DAR_NOCOMPR="-Z '*.gz' -Z '*.bz2' -Z '*.zip' -Z '*.png'"
}

# system settings
system  {
DAR_OPTS="-v -m 256 -y -s 200M -D"
}

It contains some settings which are global for the whole script and special settings groups which can be used to overwrite global settings.

The function to parse this config file looks as follows:

#!/bin/sh

CONFIG="/home/chi/.daruprc"

function readconf() {
    match=0

    while read line; do
        # skip comments
        [[ ${line:0:1} == "#" ]] && continue

        # still no match? lets check again
        if [ $match == 0 ]; then

            # do we have an open tag ?
            if [[ ${line:$((${#line}-1))} == "{" ]]; then

                # strip "{"
                group=${line:0:$((${#line}-1))}

                # do we have a match ?
                if [[ ${group} -eq $1 ]]; then
                    match=1
                    continue
                fi

                continue
            fi

        # found close tag but still no match - continue
        elif [[ ${line:0} == "}" && $match == 0 ]]; then
            continue

        # found close tag after config was read - exit loop
        elif [[ ${line:0} == "}" && $match == 1 ]]; then
            break

        # got a config line return it
        else
            echo $line
        fi

    done < "$CONFIG"
}

for line in $(readconf "system"); do
    echo $line
done

As you can see I try to just echo the lines at the moment. And here's the problem: It seems I have some " escape issues, if I run it I get the following output:

./parseconfig.sh
DATE_PREFIX="-I"
SSHFS_OPTS="-C"
EXT_FULL="full"
EXT_DIFF="diff"
EXT_CATALOGUE="catalogue"
DAR_OPTS="-v
-m
256
-y
-s
600M
-D"
DAR_NOCOMPR="-Z
'*.gz'
-Z
'*.bz2'
-Z
'*.zip'
-Z
'*.png'"

As you can see there are way to much newlines - which leads to problems when I try to use "eval" to initialize the variables.

I am sure I am missing something simple here, however I am looking for a solution for a few days now - maybe one of the bash gurus on this board can enlighten me ;-).

Thanks in advance

PS.: I am trying to do this in pure bash because the script also needs to run on embedded machines with very low memory like my router on which I don't have sed or awk or the like. I also know that I could for example just use different config files to circumvent the whole parsing - but I am just curious if this could work the way I like ;-).

Last edited by chimeric (2007-11-04 23:20:58)

pelle.k · 2007-11-04 21:52:36

Eh, excuse me if i'm missing something really basic in your approach, but wouldn't sourcing the config, and using functions to override, works just as well?

#config.rc

settings_global() {
value1=1
value2=1
}

settings_system() {
value1=2
}

#application.sh

. config.rc
global_settings
echo $value1 #outputs 1
system_settings
echo $value1 #outputs 2

chimeric · 2007-11-04 21:59:45

Hi pelle.k,

no - of course you're right - that would work and I've thought about that too - but my question is rather if what I'd like to do is possible at all (as I wrote before - I am just curious if it could work the way I like ).

MrWeatherbee · 2007-11-04 22:11:21

chimeric wrote:

Hi pelle.k,
no - of course you're right - that would work and I've thought about that too - but my question is rather if what I'd like to do is possible at all (as I wrote before - I am just curious if it could work the way I like ).

You are trying to read each line of the function with the function call itself when your intention is to read each line of the config file (which the function itself already does).

Change:

for line in $(readconf "system"); do
    echo $line
done

to simply:

readconf "system"

and the output is this:

DATE_PREFIX="-I"
SSHFS_OPTS="-C"
EXT_FULL="full"
EXT_DIFF="diff"
EXT_CATALOGUE="catalogue"
DAR_OPTS="-v -m 256 -y -s 600M -D"
DAR_NOCOMPR="-Z '*.gz' -Z '*.bz2' -Z '*.zip' -Z '*.png'"

chimeric · 2007-11-04 22:15:49

Hi MrWeatherbee,

well yes, but I want to assign (eval) these lines to get variables out of them. Since the while loop is trapped in its' own subshell I need to return the lines into the parent one to be able to make these variables "visible" in the script itself.

kumico · 2007-11-04 22:18:09

you can get around it by removing the spaces and re-interpret them later

in readconf() << echo $line to >> echo ${line// /'\040'}
in for loop      << echo $line      >> echo -e $line

Last edited by kumico (2007-11-04 22:23:07)

pelle.k · 2007-11-04 22:20:26

Did you understand that? If you are passing

DAR_NOCOMPR="-Z '*.gz' -Z '*.bz2' -Z '*.zip' -Z '*.png'"

with the echo in your function, you pass, in effect;

for line in $(echo DAR_NOCOMPR="-Z '*.gz' -Z '*.bz2' -Z '*.zip' -Z '*.png'"); do
    echo $line
done

which would be like doing this

for line in 1 2 3 4 5 6; do
    echo $line
done

and that is 6 events, not one. Thats just how "for" works. So you get that one line, split in to parts....

Last edited by pelle.k (2007-11-04 22:21:06)

MrWeatherbee · 2007-11-04 22:35:56

chimeric wrote:

Hi MrWeatherbee,
well yes, but I want to assign (eval) these lines to get variables out of them. Since the while loop is trapped in its' own subshell I need to return the lines into the parent one to be able to make these variables "visible" in the script itself.

Right. But I was trying to show you that the way you were going about it wouldn't work at a very fundamental level.

Without changing too much code just for a quick and dirty demo,

change:

        # got a config line return it
        else
            echo $line
        fi

    done < "$CONFIG"

 
         # got a config line return it
        else
            line_array=( "${line_array[@]}" "$line" )  
        fi

    done < "$CONFIG"
    numarrayelements=${#line_array[@]}

and then in the main program code, do this:

readconf "system"

for index in $( seq $numarrayelements ); do
    line=${line_array[(($index-1))]}
    echo $line
done

Using an array to collect the good lines from the function allows you to reuse them later for whatever purpose. Although other ways exist, you certainly don't want to do what you originally did.

Edit:
I posted the code with an incorrectly referenced variable and had to fix it.

Last edited by MrWeatherbee (2007-11-04 22:46:34)

chimeric · 2007-11-04 22:42:52

@pelle.k, aaahh - I somehow thought that could be the problem - thanks for the clarification.

@kumico - thanks a lot for your tip! That solves my problem, now everything works the way I like (hmm - sometimes the easiest solutions are those you never think about yourself).

chimeric · 2007-11-04 22:53:57

MrWeatherbee wrote:

change:

        # got a config line return it
        else
            echo $line
        fi

    done < "$CONFIG"

 
         # got a config line return it
        else
            line_array=( "${line_array[@]}" "$line" )  
        fi

    done < "$CONFIG"
    numarrayelements=${#line_array[@]}

I am not quite sure I understand this - how does the main script know about $line_array when it gets the lines assigned inside the function which runs in its' own subshell? I've already tried to work with arrays but didn't succeed - yet.

Edit: Ok - MrWeatherbee - that works perfect - thanks again! But I think I have to learn more about functions and subshells - because at the moment I simply don't understand how $line_array can be visible in the main script allthough it gets its' values assigned in the function . I thought a function call always runs in its' own subshell?!?

Last edited by chimeric (2007-11-04 23:07:06)

MrWeatherbee · 2007-11-04 23:09:01

chimeric wrote:

I am not quite sure I understand this - how does the main script know about $line_array when it gets the lines assigned inside the function which runs in its' own subshell? I've already tried to work with arrays but didn't succeed - yet.

Run the code I provided and you will have achieved success.

But seriously, once a function is run, the only way to keep the scope of the variable within the code block of the function is to declare it as local, something that has not been done in the code I posted.

Add this inside the function:

local line_array

and you won't get any output from the echo statement.

Edit:

Add link to local variable info:

http://tldp.org/LDP/abs/html/localvar.html

Last edited by MrWeatherbee (2007-11-04 23:10:11)

chimeric · 2007-11-04 23:20:25

MrWeatherbee wrote:

Run the code I provided and you will have achieved success.

As written before - runs perfect - you saved me from banging my head against my desk for another week .

I also found my error in reasoning - I simply got the subshell thing wrong. I thought every function call opens his own subshell. But I called the function in the for loop like $(readconf "system") which is why it opens a subshell.

Many thanks again!

Arch Linux

#1 2007-11-04 21:00:36

[SOLVED] Parsing simple config files in pure bash

#2 2007-11-04 21:52:36

Re: [SOLVED] Parsing simple config files in pure bash

#3 2007-11-04 21:59:45

Re: [SOLVED] Parsing simple config files in pure bash

#4 2007-11-04 22:11:21

Re: [SOLVED] Parsing simple config files in pure bash

#5 2007-11-04 22:15:49

Re: [SOLVED] Parsing simple config files in pure bash

#6 2007-11-04 22:18:09

Re: [SOLVED] Parsing simple config files in pure bash

#7 2007-11-04 22:20:26

Re: [SOLVED] Parsing simple config files in pure bash

#8 2007-11-04 22:35:56

Re: [SOLVED] Parsing simple config files in pure bash

#9 2007-11-04 22:42:52

Re: [SOLVED] Parsing simple config files in pure bash

#10 2007-11-04 22:53:57

Re: [SOLVED] Parsing simple config files in pure bash

#11 2007-11-04 23:09:01

Re: [SOLVED] Parsing simple config files in pure bash

#12 2007-11-04 23:20:25

Re: [SOLVED] Parsing simple config files in pure bash

Board footer