You are not logged in.

#1 2008-12-09 05:50:16

dav7
Member
From: Australia
Registered: 2008-02-08
Posts: 674

UPX and Linux: a fascinating (but possibly disastrous) combination

I just compressed a program I was working on with UPX, and as usual, the results were pretty good: ~54% compression reducing my binary from 10712 bytes to 5626 bytes.

As I was musing over the whole compression concept, I thought "UPX produces really good results, but to do so, it has to inject a tiny loader into the binary that first decompresses the binary then jumps to where the actual operating code starts. If only that loader were moved elsewhere, there would probably be a tiny speedup when launching some programs."

Then two and two became four, and I had this idea: move the UPX launch code into the Linux dynamic linker ld.so, and propose that ELF create a new 'compressed' structure in the standard. In this way, one might do:

$ cc -o test test.c
$ file test.c
editor: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), dynamically linked (uses shared libs), not stripped
$ upx -9 test
                       Ultimate Packer for eXecutables
                          Copyright (C) 1996 - 2008
UPX 3.03        Markus Oberhumer, Laszlo Molnar & John Reiser   Apr 27th 2008

        File size         Ratio      Format      Name
   --------------------   ------   -----------   -----------
     10712 ->      5627   52.53%    linux/386    test                          

Packed 1 file.
$ file test.c
editor: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), dynamically linked (uses shared libs), not stripped, compressed

The first thing I'll say here is that yes, working to obtain a speedup of a few milliseconds for a procedure that might run 10 times a day is worthless. But that may increase slightly when the binary is, instead of 10KB, maybe 10MB. Or 60MB. Having the decompression code already in memory - even in the kernel, perhaps, where it could probably be optimized a fair bit because of its placement - might be useful.

And there's the political ramifications: if the support is in the kernel and/or the dynamic linker, people are going to have a higher chance of using it than they would right now. For a distribution like Ubuntu which has quite a lot of bloat from what I've heard, UPXing the entire system would save on a lot of diskspace.

At least on machines with capable processors, anyway. There's always the downside that programs would have to be decompressed before they could run, and this would inevitably take up CPU cycles, so maybe distros like Ubuntu wouldn't benefit from it. Maybe this sort of thing would suit minimalistic distros or embedded systems with an extremely small amount of RAM, notwithstanding the fact that there are already compressed filesystems out there.

-dav7

Last edited by dav7 (2008-12-09 05:53:42)


Windows was made for looking at success from a distance through a wall of oversimplicity. Linux removes the wall, so you can just walk up to success and make it your own.
--
Reinventing the wheel is fun. You get to redefine pi.

Offline

#2 2008-12-09 17:04:42

Hohoho
Member
Registered: 2007-06-23
Posts: 222

Re: UPX and Linux: a fascinating (but possibly disastrous) combination

This is pointless on normal desktop machines and even normal laptops, there is already more than enough storage available and core system takes up 10gb tops (full gnome, open office, apps). Furthermore, the additional processing power required might add up to something noticeable and that will kill off the idea completely.
Embedded systems and netbooks are a different story, but I doubt compression ain't already implemented there in some way or another.

Offline

#3 2008-12-09 17:56:07

string
Member
Registered: 2008-11-03
Posts: 286

Re: UPX and Linux: a fascinating (but possibly disastrous) combination

60 MB executables? /me phears

Offline

#4 2008-12-09 18:00:05

Daenyth
Forum Fellow
From: Boston, MA
Registered: 2008-02-24
Posts: 1,244

Re: UPX and Linux: a fascinating (but possibly disastrous) combination

Interesting concept, but I don't know if it's really practical. I'm really annoyed, I had read an article a few weeks ago about how to make a binary literally as small as possible. I mean handcoded ASM, putting code inside the ELF header where possible. I can't remember the damn name or site where I read it though...

Offline

#5 2008-12-09 19:45:11

freakcode
Member
From: São Paulo - Brazil
Registered: 2007-11-03
Posts: 410
Website

Re: UPX and Linux: a fascinating (but possibly disastrous) combination

60MB raw binary? That's what I call a binary blob. Never saw anything like that.

Offline

#6 2008-12-09 20:17:40

bluewind
Administrator
From: Austria
Registered: 2008-07-13
Posts: 165
Website

Re: UPX and Linux: a fascinating (but possibly disastrous) combination

Just had a look at /usr/bin and most stuff is 50-200-1000KB. Biggest is inkscape with 13MB.


"Everything should be made as simple as possible, but not simpler."
-- Albert Einstein

Offline

#7 2008-12-14 05:14:51

Ranguvar
Member
From: Our collective subconscious
Registered: 2008-08-12
Posts: 2,515
Website

Re: UPX and Linux: a fascinating (but possibly disastrous) combination

Yeah... most apps these days are quite small in reality, they just for whatever reason (protection against 'pirates'?) stick data into the binary. Thankfully, Linux apps tend to be sane smile

Still, this is indeed an interesting concept.

Offline

#8 2008-12-14 11:42:05

funkyou
Member
From: Berlin, DE
Registered: 2006-03-19
Posts: 848
Website

Re: UPX and Linux: a fascinating (but possibly disastrous) combination

Daenyth wrote:

I had read an article a few weeks ago about how to make a binary literally as small as possible. I mean handcoded ASM, putting code inside the ELF header where possible. I can't remember the damn name or site where I read it though...

This one could be interesting: http://www.pouet.net/topic.php?which=5392&page=1

EDIT, forgot something:
http://in4k.untergrund.net/index.php?title=Linux
http://www.muppetlabs.com/~breadbox/sof … eensy.html

Last edited by funkyou (2008-12-14 13:19:51)


want a modular and tweaked KDE for arch? try kdemod

Offline

#9 2008-12-14 11:51:27

dav7
Member
From: Australia
Registered: 2008-02-08
Posts: 674

Re: UPX and Linux: a fascinating (but possibly disastrous) combination

Daenyth: I know which one you're talking about. "linux smallest binary asm elf header" turns up [url=]the page you mentioned[/url]; while refining my search (I originally tried "linux smallest binary asm not there yet") I found another page which I'd taken a poke around a while ago; it may interest you, as might the asmutils project, which includes a 532 byte HTTP server, and the ll (linux-logo) project: a logo/stats display package written in 13 different assembly languages for various architectures and even different platforms, some with rather amusing and interesting results.

PS. Writing this post was a URL headache roll hit quote if you're bored and/or want to see what it looks like tongue

-dav7

Last edited by dav7 (2008-12-14 12:00:27)


Windows was made for looking at success from a distance through a wall of oversimplicity. Linux removes the wall, so you can just walk up to success and make it your own.
--
Reinventing the wheel is fun. You get to redefine pi.

Offline

Board footer

Powered by FluxBB