Best practice for frequently needed config settings?

ibrunton · 2012-07-01 15:05:07

I have a command-line tool I wrote to keep track of (primarily) everything I eat and drink in the course of the day. Obviously, therefore, I run this program many times every day.

The program reads a keyfile and parses the options defined therein. It strikes me that this may be awfully inefficient to open the file every time, read it, parse options, etc., before even doing anything with command-line input. My computer is pretty powerful so it's not actually a problem, per se, but I do always want to become a better programmer, so I'm wondering whether there's a "better" way to do this, for example some way of keeping settings available without having to read them every single time. A daemon, maybe? I suspect that simply defining a whole bunch of environment variables would not be a best practice.

The program is written in Perl, but I have no objection to porting it to something else; Perl just happens to be very easy to use for handling a lot of text, as the program relies heavily on regexes. I don't think the actual code of the thing is important to my question, but if you're curious, it's at my github. (Keep in mind I'm strictly an amateur, so there are probably some pretty silly things in my code.)

Thanks for any input and ideas.

Trilby · 2012-07-01 15:30:19

There are some ways around this, but it really depends on the type of data.

Options I can think of are the following:
1) read a file at every startup as you are already doing. This is extremely common - look around at the tools you have installed, many of them have an rc file. You can always strive to make this reading more efficient, but under most circumstances reading a file at startup is perfectly legitimate.
2) run in the background or as a daemon which you also note.
3) similar to #1, save the data in a file, but instead of parsing it each time save it instead as a binary. If you're data can all be stored in some nice data structure in the code, in most languages you can just write the block of memory occuppied by that data structure to a file, then on startup you just transfer the contents of the file to a block of allocated memory. This is quiet do-able - but for a vast majority of situations this would be a bad approach (IMHO). The data have to be structured in such a way they will occupy one continuous memory block, and depending on the size of the data block this in itself may be impractical or impossible. Also, you'd need a good amount of error checking or you'd simply have to "trust" that nothing could ever go wrong in your binary file.

So, all in all, I'd say go with #1, but spend time tuning your file read/write proceedures to be efficient. Sometimes a lexer (gnu flex) is good for this, but often times it is also overkill and a well written series of if(strncmp(...)) statements will be better*.

Bear in mind though, this is from another amateur. I c ode for fun - and some of my code has found use - but it is far from my day job.

edit: *note - that is a C example, and flex library is easily used in C. I'd be surprised if there are not perl bindings for flex, but I very rarely use perl. As an after thought, I'd be surprised if flex is even all that useful in perl, given perl's built-in regex abilites. After-after-thought, I would not be surprised if perl itself were built on some version of flex.

edit2: also, I doubt environment variables would be a good way to go. That seems to require more system calls and more overhead than just reading from a config file. Environment variables are a handy way for several programs to be able to access/change the same setting - but for a single program they don't make much sense to me.

Last edited by Trilby (2012-07-01 15:34:43)

ibrunton · 2012-07-01 16:06:03

Trilby wrote:

So, all in all, I'd say go with #1, but spend time tuning your file read/write proceedures to be efficient. Sometimes a lexer (gnu flex) is good for this, but often times it is also overkill and a well written series of if(strncmp(...)) statements will be better*.

Originally I just read the file in and processed it with a series of if statements using regexes. The current version uses Perl's interface to glib to read a more neatly organised rc file with simpler variable names.

I'm tempted to at least try rewriting it as a daemon, if only for the opportunity to learn how.

Roken · 2012-07-01 22:55:55

Unless your config file is extensive, you are unlikely to gain anything at all by changing what you have. The read time (including seek time) for a tiny, few Kb file, even on an old HD, is negligible. You would likely impact resources (again, albeit in a negligible manner) by daemonising it, simply because you would have it permanently resident in memory.

rowdog · 2012-07-01 22:58:25

You might want to look at using Storable to dump and restore the memory structure rather than reconstructing your options every run. If you, you might also want to track config file change time so you know when to rebuild the structure again. I'm not sure this will work with Glib::KeyFile seeing as it's C (XS) rather than Perl. Myself, I usually use JSON or YAML for my config files and they load quite fast.

Daemonizing seems like massive overkill but daemons are fun to write so don't let that stop you if that's what you want to do.

Don't worry about using %ENV because it's slow, it's not, those values are already filled in when your program runs.

See also...
http://search.cpan.org/~ams/Storable-2.35/Storable.pm

perldoc -q daemon

Arch Linux

#1 2012-07-01 15:05:07

Best practice for frequently needed config settings?

#2 2012-07-01 15:30:19

Re: Best practice for frequently needed config settings?

#3 2012-07-01 16:06:03

Re: Best practice for frequently needed config settings?

#4 2012-07-01 22:55:55

Re: Best practice for frequently needed config settings?

#5 2012-07-01 22:58:25

Re: Best practice for frequently needed config settings?

Board footer