Compiling init scripts as C

xentalion · 2010-01-07 00:34:54

Hello!

A while back, I read that Blice (owner of http://its.alrig.ht/) had converted their init scripts to C to help the computer boot faster.

Unfortunately, Blice's site seems to have disappeared, which is a shame since I'd like to read some of the information on his webpage.

Anywho, my question is: anybody here have experience in converting startup scripts to C?

I've rewritten the apache, ssh, and postfix startup scripts so far. I've put the code up on GitHub at http://github.com/ColinJones/binary-startup
Any thoughts?

tavianator · 2010-01-07 00:43:46

A major point would be that system() will probably be more inefficient than a bash script, because system() invokes "/bin/sh -c <argument>" anyway. For any kind of gain you want fork() and exec(). Also it's bad practice to put function definitions in .h files.

The thing is, there's almost no time spent in the /etc/rc.d scripts themselves anyway. For a decent speedup, you'd want to rewrite the /etc/rc.sys* scripts in C. But I doubt you'll get a good speed/simplicity tradeoff.

PJ · 2010-01-07 01:42:26

If you are not going to reconfigure your init scripts a lot, it would probably speed up the boot sequence by turning the complete rc.sysinit and services into one precompiled monolithic program. Pretty much create or use something that can compile bash scripts into binary program. A simpler approach is to just parse rc.conf and then generate the program with everything statically defined, variables, modules to load, services to start, ip-adresses and so on.

The reason why the image should be monolithic by design is that it takes time to load modules, but that does not mean that the design for the source code can't be modularized. During compile time it reads which services it should start from rc.conf and include those into the program image. That means that the program will be small and only contain code to start each services and of course everything that is initialized in rc.sysinit, nothing else. All the parsing stuff should be made during compile time, this will speed up the boot sequence a bit since there is no need to parse several files anymore during boot. In case there is a problem with the developing tools (for example gcc is broken) I think a good idea should be to use the current method (bash scripts) as a backup.

But this is of course just an idea how it could be made to speed up the boot sequence.

<edit>
You might find finit interesting:
http://helllabs.org/finit/

(It used to be a variant of finit for arch linux called finit-arc but I can't find the source code any longer.)
</edit>

Last edited by PJ (2010-01-07 02:03:54)

xentalion · 2010-01-07 18:23:34

@tavianator: I've noticed a fair speedup when using the binary versions as opposed to the regular versions. You are right about storing functions in the headers, it's something I will have to fix.

@PJ: That's a very good idea, I hadn't thought of that. I could adapt my current code to do that fairly easily. I've also been looking at finit-arc, and seeing how they do that. I notice that the developer(s) suggest that this not be used for desktop PCs, but since I'm planning on applying what I learn here to my OpenPandora, it's all good!

PJ · 2010-01-08 18:11:27

I kind of like the idea myself. I would like to help with the development.

After rethinking a bit, I don't think the hardest part to develop is the actually code, the part that is going to be challenging is the build script. I mean, if it is going to be according to my original idea, the program code itself is going to be simple but to be able to have simple program code when parsing rc.conf it means that the build script has to be a bit more complex, otherwise it will mean that the program code itself has to parse rc.conf. Well, that means if the rc.conf is going to be parsed in the first place. I think it should be parsed during build time, it will be a bit like mkinitcpio but for the initscripts but that is of course in my opinion.

One of the first thing that needs to be done is probably do a prof-of-concept of a program + build script that can depending on the content of the config file build a program with different functionality. This is pretty much to simulate the DAEMONS array. A traditional way of doing this with statically linking is to including different files with the same interfaces and in case a functionality is not going to be present, use interface functions that are empty. This pretty much works when there isn't a lot of different functionality but I think there are to many possible DAEMONS to make this practical. I think a better way is to generate a config.h (or use a different name for the file) which contains a list of functional pointers to all the "modules" that should be included. In short, make the program aware of the function and include the file that contains the function to the linker and make the complete process dynamically.

I might look in this prof-of-concept during the weekend.

Last edited by PJ (2010-01-08 18:18:09)

PJ · 2010-01-09 20:10:28

I have created a basic proof-of-concept which is pretty easy to extend.
http://github.com/PeterJohansson/speedy

<edit>
Updated the URL
</edit>

Last edited by PJ (2010-01-10 21:49:01)

xentalion · 2010-01-09 20:20:03

Looks great! What license do you think this should be under? Do you want to use some of my code, such as the code for creating daemons in /var/run/daemons?
Now, your script is for starting and stopping right? So separate rc.d scripts would still be necessary for starting new daemons, and you'd still need to check /var/run/daemons for running daemons right?

PJ · 2010-01-09 20:35:39

xentalion wrote:

Looks great! What license do you think this should be under?

I haven't thought of it yet. I am open for suggestions since you actually started developing this. My code was mostly a proof-of-concept.

xentalion wrote:

Do you want to use some of my code, such as the code for creating daemons in /var/run/daemons?

Well, I was hoping to do that but I haven't figured out everything yet.

xentalion wrote:

Now, your script is for starting and stopping right? So separate rc.d scripts would still be necessary for starting new daemons, and you'd still need to check /var/run/daemons for running daemons right?

That's correct. My code is just to be able to make one monolithic image of several daemons implemented in C. The daemons that I have implemented is just to see that those are linked in to the image, in other words, they are just there to print out a message that the linking worked.

Last edited by PJ (2010-01-09 20:36:39)

PJ · 2010-01-10 18:50:47

I have done some minor changes, mostly to the structure to make it a bit more serious. It is still far from being something useful but I think the code is pretty easy to extend so I might as well continue working on it. The next thing that I am going to do is to use the current bash scripts until those are implemented in C. Pretty much call those from the C program to make it possible to bootup.

I have looked at your code a bit more and what I could use of it is actually the lib part. The daemons in your code base aren't that useful to me since I am going to include those into a monolithic program image which means that there is not going to be any parsing of arguments and the interfaces are going to be different. I don't want to call those as external C programs either since that could probably make the bootup slower.

I have thought a bit on the license and I would like to license my code under ISC, if that is okay since it was actually your idea that made me start writing code.

I was also thinking about rename it since I don't think binary-startup-proof-of-concept is a good name for the repository/program. I haven't figured out a good name for it.

To those that think that this is a waste of time since I am just going to earn a couple seconds for work that might take several days/weeks. Well, I am actually not that interested in earning a couple of seconds, I am just doing this to see if it is possible and just for fun. Sure, I am interested in how much faster it might be possible to make the init process but that is not my main focus.

xentalion · 2010-01-10 19:23:00

@PJ: Now, just a question, right now, this is just for the startup scripts right? Not replacing init yet?

I have to do a bit of fixing for my library, I need to move all of the functions out of the headers. Of course, it makes perfect sense not to run the binaries from in the program.

ISC sounds like a good license, I was going to suggest a BSD license (which this is) or the GPL.

How about arch-init? or arch-startup?

Anyone who thinks that speeding up one's computer is a waste of time is foolish!

PJ · 2010-01-10 20:08:32

xentalion wrote:

@PJ: Now, just a question, right now, this is just for the startup scripts right? Not replacing init yet?

Not really yet, but that is something I was thinking on to do in the long run. It's far more complex to do that so I think it could be a good idea to just concentrate on the daemons for now.

xentalion wrote:

I have to do a bit of fixing for my library, I need to move all of the functions out of the headers. Of course, it makes perfect sense not to run the binaries from in the program.

I need to do a lot of stuff for the core, mainly create a bash script that can generate wrapper code in C for daemons that don't exist as C code. This script will be probably be implemented in bash or (g)awk, I haven't decided yet.

xentalion wrote:

ISC sounds like a good license, I was going to suggest a BSD license (which this is) or the GPL.

Then it is ISC.

xentalion wrote:

How about arch-init? or arch-startup?

Why not. First I need to use google to see if there are already projects named arch-init or arch-startup. I don't think it is a good idea to use a a name that another project is already using.

xentalion · 2010-01-10 20:17:38

@PJ: The only thing about using arch-init or arch-startup is that the Arch linux people may not enjoy their name being associated with it. :S I dunno.

PJ · 2010-01-10 20:56:08

I don't really want to upset any arch developers so I don't think any of those are an alternative.
How about speedy? It is the characteristics of the pink ghost from pacman.

Barrucadu · 2010-01-10 21:00:26

I've converted my /etc/rc.sysinit file to C today (inspired by this thread).

My /etc/rc.sysinit script was already a bit customised anyway, so it's missing some things (eg: LVM and RAID), it's currently got no error handling, and I don't really have a huge amount of experience with C so the code is awful… but if you want to have a look despite all that: http://github.com/Barrucadu/home/tree/m … nary-init/

PJ · 2010-01-10 21:24:28

Cool, I might look into binary-init. It's not exactly what I had in mind, but I guess that is probably expected since my code is a bit more complex. Most of my code so far is just to be able to dynamically chose which "modules" to include during compile time.

PJ · 2010-01-10 21:38:04

I renamed my repository to speedy.

Barrucadu · 2010-01-10 23:21:22

PJ wrote:

Cool, I might look into binary-init. It's not exactly what I had in mind, but I guess that is probably expected since my code is a bit more complex. Most of my code so far is just to be able to dynamically chose which "modules" to include during compile time.

The next thing I plan to do is convert the rest of the /etc/rc.* scripts, then the /etc/rc.d/* scripts. My goal is to have a monolithic program to handle booting, and (perhaps) another to handle shutting down.

xentalion · 2010-01-11 04:57:10

Well, I don't feel so crazy for wanting to do this!
Edited so that I don't sound as much of an idiot as I actually am.

I'm about to test Barracadu's binary-init! With regular rc.sysinit and rc.d scripts, it takes my laptop 24 seconds to boot. Let's see what the difference between your code and the normal one...

Edit:
I've just rebooted my computer, using Barracadu's binary-init, and found this:
It takes an identical amount of time to boot my laptop with the regular sysinit as it does to boot with binary-init. Which makes me conclude that the biggest bottlenecks I have are my kernel and my rc.d scripts. That or my harddrive speed. (this is a 2.2ghz Acer Aspire 5050, 1GB ram)

Probably, once the rc.sysinit and rc.d/* are merged into one file (for booting only), then there will be more substantial speed increases.

I'll post the bootchart pictures once I get them uploaded to my server.

Last edited by xentalion (2010-01-11 05:11:58)

Arch Linux

#1 2010-01-07 00:34:54

Compiling init scripts as C

#2 2010-01-07 00:43:46

Re: Compiling init scripts as C

#3 2010-01-07 01:42:26

Re: Compiling init scripts as C

#4 2010-01-07 18:23:34

Re: Compiling init scripts as C

#5 2010-01-08 18:11:27

Re: Compiling init scripts as C

#6 2010-01-09 20:10:28

Re: Compiling init scripts as C

#7 2010-01-09 20:20:03

Re: Compiling init scripts as C

#8 2010-01-09 20:35:39

Re: Compiling init scripts as C

#9 2010-01-10 18:50:47

Re: Compiling init scripts as C

#10 2010-01-10 19:23:00

Re: Compiling init scripts as C

#11 2010-01-10 20:08:32

Re: Compiling init scripts as C

#12 2010-01-10 20:17:38

Re: Compiling init scripts as C

#13 2010-01-10 20:56:08

Re: Compiling init scripts as C

#14 2010-01-10 21:00:26

Re: Compiling init scripts as C

#15 2010-01-10 21:24:28

Re: Compiling init scripts as C

#16 2010-01-10 21:38:04

Re: Compiling init scripts as C

#17 2010-01-10 23:21:22

Re: Compiling init scripts as C

#18 2010-01-11 04:57:10

Re: Compiling init scripts as C

Board footer