UPX and Linux: a fascinating (but possibly disastrous) combination

dav7 · 2008-12-09 05:50:16

I just compressed a program I was working on with UPX, and as usual, the results were pretty good: ~54% compression reducing my binary from 10712 bytes to 5626 bytes.

As I was musing over the whole compression concept, I thought "UPX produces really good results, but to do so, it has to inject a tiny loader into the binary that first decompresses the binary then jumps to where the actual operating code starts. If only that loader were moved elsewhere, there would probably be a tiny speedup when launching some programs."

Then two and two became four, and I had this idea: move the UPX launch code into the Linux dynamic linker ld.so, and propose that ELF create a new 'compressed' structure in the standard. In this way, one might do:

$ cc -o test test.c
$ file test.c
editor: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), dynamically linked (uses shared libs), not stripped
$ upx -9 test
                       Ultimate Packer for eXecutables
                          Copyright (C) 1996 - 2008
UPX 3.03        Markus Oberhumer, Laszlo Molnar & John Reiser   Apr 27th 2008

        File size         Ratio      Format      Name
   --------------------   ------   -----------   -----------
     10712 ->      5627   52.53%    linux/386    test                          

Packed 1 file.
$ file test.c
editor: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), dynamically linked (uses shared libs), not stripped, compressed

The first thing I'll say here is that yes, working to obtain a speedup of a few milliseconds for a procedure that might run 10 times a day is worthless. But that may increase slightly when the binary is, instead of 10KB, maybe 10MB. Or 60MB. Having the decompression code already in memory - even in the kernel, perhaps, where it could probably be optimized a fair bit because of its placement - might be useful.

And there's the political ramifications: if the support is in the kernel and/or the dynamic linker, people are going to have a higher chance of using it than they would right now. For a distribution like Ubuntu which has quite a lot of bloat from what I've heard, UPXing the entire system would save on a lot of diskspace.

At least on machines with capable processors, anyway. There's always the downside that programs would have to be decompressed before they could run, and this would inevitably take up CPU cycles, so maybe distros like Ubuntu wouldn't benefit from it. Maybe this sort of thing would suit minimalistic distros or embedded systems with an extremely small amount of RAM, notwithstanding the fact that there are already compressed filesystems out there.

-dav7

Last edited by dav7 (2008-12-09 05:53:42)

Hohoho · 2008-12-09 17:04:42

This is pointless on normal desktop machines and even normal laptops, there is already more than enough storage available and core system takes up 10gb tops (full gnome, open office, apps). Furthermore, the additional processing power required might add up to something noticeable and that will kill off the idea completely.
Embedded systems and netbooks are a different story, but I doubt compression ain't already implemented there in some way or another.

string · 2008-12-09 17:56:07

60 MB executables? /me phears

Daenyth · 2008-12-09 18:00:05

Interesting concept, but I don't know if it's really practical. I'm really annoyed, I had read an article a few weeks ago about how to make a binary literally as small as possible. I mean handcoded ASM, putting code inside the ELF header where possible. I can't remember the damn name or site where I read it though...

freakcode · 2008-12-09 19:45:11

60MB raw binary? That's what I call a binary blob. Never saw anything like that.

bluewind · 2008-12-09 20:17:40

Just had a look at /usr/bin and most stuff is 50-200-1000KB. Biggest is inkscape with 13MB.

Ranguvar · 2008-12-14 05:14:51

Yeah... most apps these days are quite small in reality, they just for whatever reason (protection against 'pirates'?) stick data into the binary. Thankfully, Linux apps tend to be sane

Still, this is indeed an interesting concept.

funkyou · 2008-12-14 11:42:05

Daenyth wrote:

I had read an article a few weeks ago about how to make a binary literally as small as possible. I mean handcoded ASM, putting code inside the ELF header where possible. I can't remember the damn name or site where I read it though...

This one could be interesting: http://www.pouet.net/topic.php?which=5392&page=1

EDIT, forgot something:
http://in4k.untergrund.net/index.php?title=Linux
http://www.muppetlabs.com/~breadbox/sof … eensy.html

Last edited by funkyou (2008-12-14 13:19:51)

dav7 · 2008-12-14 11:51:27

Daenyth: I know which one you're talking about. "linux smallest binary asm elf header" turns up [url=]the page you mentioned[/url]; while refining my search (I originally tried "linux smallest binary asm not there yet") I found another page which I'd taken a poke around a while ago; it may interest you, as might the asmutils project, which includes a 532 byte HTTP server, and the ll (linux-logo) project: a logo/stats display package written in 13 different assembly languages for various architectures and even different platforms, some with rather amusing and interesting results.

PS. Writing this post was a URL headache hit quote if you're bored and/or want to see what it looks like

-dav7

Last edited by dav7 (2008-12-14 12:00:27)

Arch Linux

#1 2008-12-09 05:50:16

UPX and Linux: a fascinating (but possibly disastrous) combination

#2 2008-12-09 17:04:42

Re: UPX and Linux: a fascinating (but possibly disastrous) combination

#3 2008-12-09 17:56:07

Re: UPX and Linux: a fascinating (but possibly disastrous) combination

#4 2008-12-09 18:00:05

Re: UPX and Linux: a fascinating (but possibly disastrous) combination

#5 2008-12-09 19:45:11

Re: UPX and Linux: a fascinating (but possibly disastrous) combination

#6 2008-12-09 20:17:40

Re: UPX and Linux: a fascinating (but possibly disastrous) combination

#7 2008-12-14 05:14:51

Re: UPX and Linux: a fascinating (but possibly disastrous) combination

#8 2008-12-14 11:42:05

Re: UPX and Linux: a fascinating (but possibly disastrous) combination

#9 2008-12-14 11:51:27

Re: UPX and Linux: a fascinating (but possibly disastrous) combination

Board footer