[solved] memory consumption

bamdad · 2010-05-25 20:43:22

hi everybody,

i've recently given fedora 13 with btrfs a try. well.. it's fedora, so it's 'bleeding edge' and a bit buggy, although aside from the usual firefox-crashing-X-randomly bug (which i've even had on opensolaris), it's fairly good.

after some use, i've noticed something that's awesome. i have only 512 megs of ram. i know, that's crap, and i'm waiting for some money to buy a new computer, or at least some more ram, but that's beside the point.

my regular arch setup uses a lot of memory with gnome. i thought this is to be expected, since ubuntu used about the same amount before. if i start a browser, and play some flash, i can't even multitask without massive swapping. i've set swappiness to 10, but that only made things a little better.

so fedora reports (with a full-blown gnome setup) around 200 megs of memory being used. my similar arch setup uses around 350 on a fresh boot, and after that it only gets worse. i initially thought that this is because it's a new install, but almost every program (gmail checker, gnome-do etc) is installed. then i thought it's because somehow it reports memory incorrectly, but since it's a _lot_ snappier too, i really have no idea.

can this be because of btrfs? if not, what can i do to reduce the memory usage of my arch installation? i have no intenton of switching to fedora.. i've tried commenting out daemons in rc.conf, stopping some programs etc., so please don't suggest that.

thanks in advance,
bamdad

Last edited by bamdad (2010-06-15 22:56:42)

karol · 2010-05-25 21:19:09

I don't think that the fs has much o do w/ RAM usage.

> so fedora reports (with a full-blown gnome setup) around 200 megs of memory being used
I'm not using any DE only dwm and when I open firefox it's about 250 MB.

[karol@black pics]$ free -m
             total       used       free     shared    buffers     cached
Mem:           996        588        408          0         19        328
-/+ buffers/cache:        240        755
Swap:          258          0        258

Can you please say how does fedora 'report' - what app do you use?

bamdad · 2010-05-25 21:30:17

karol wrote:

Can you please say how does fedora 'report' - what app do you use?

sure, in gnome-power-manager and in top. i haven't tried 'free -m'

karol · 2010-05-25 21:35:41

Could you please run 'free -m' on both your fedora and arch setup and post them here?
I'd say that 350MB RAM isn't bad when running Gnome and Firefox. Avoid flash if you can (download the movie to disk).
You can drop Gnome and switch to a lighter browser, don't run too many apps at the same time (no compiling and watching a move and browsing the web in 20 tabs and ...).

Last edited by karol (2010-05-25 21:41:51)

bamdad · 2010-05-25 21:48:02

karol wrote:

don't run too many apps at the same time (no compiling and watching a move and browsing the web in 20 tabs and ...).

yeah, i can really hate that. :) i've started to learn c and java, and.. you know.. i'll just have to get some more memory soon.

i'll post the results of 'free -m' here shortly.

Last edited by bamdad (2010-05-25 21:48:49)

bamdad · 2010-05-25 21:49:52

arch:

(bamdad@Y.U.R.I:~) free -m 
             total       used       free     shared    buffers     cached
Mem:           499        489          9          0          7        111
-/+ buffers/cache:        370        128
Swap:          956          1        955

karol · 2010-05-25 21:53:04

I'm using my 7yo P4 2 GHz 1 GB RAM as my main box and I don't have any problems doing all of the above at the same time, only switching tags in dwm or tabs in firefox takes 2-3 seconds (I'm using the stock kernel and default settings).

Yep, your Arch is using 370M.

bamdad · 2010-05-25 21:53:48

fedora:

(bamdad@Y.U.R.I:~) free -m
             total       used       free     shared    buffers     cached
Mem:           497        446         50          0          0        287
-/+ buffers/cache:        158        338
Swap:          956          0        956

so.. i have to conclude that fedora is bluntly lying to me.. well that doesn't change the fact that it's a hell of a lot snappier..

karol · 2010-05-25 21:55:23

I hope you did a restart before taking those measurements.
Ummm, 158M - are you *sure* that's Gnome w/ a running firefox? If it is, it's quite impressive.

512M RAM is around $20 these days. It will keep you off swap.

Last edited by karol (2010-05-25 22:00:19)

tavianator · 2010-05-25 22:01:18

Well, Fedora's probably not lying. But it's likely running a different kernel and version of X, as well as lots of other stuff, than Arch.

bamdad · 2010-05-25 22:18:16

karol wrote:

I hope you did a restart before taking those measurements.

sure.

karol wrote:

Ummm, 158M - are you *sure* that's Gnome w/ a running firefox? If it is, it's quite impressive.

no, that's just gnome with my addons, just after a fresh reboot.

karol wrote:

512M RAM is around $20 these days. It will keep you off swap.

well, at least in my country, they're charging ridiculous amounts for plain DDR. ever since the advent of DDR2, DDR prices just went up. can you believe that? anyway, i'll get some soon, since my other OS is opensolaris, and zfs tends to eat a *lot* of ram.

tavianator wrote:

Well, Fedora's probably not lying. But it's likely running a different kernel and version of X, as well as lots of other stuff, than Arch

yeah, that's probably the case. sadly, arch doesn't tell the kernel subversion, only '2.6.33', while fedora does. also, the cause of the snappier behaviour might indeed be the file system. i've read somewhere that ext4 is slow on IDE drives..

karol · 2010-05-25 22:23:59

> well, at least in my country, they're charging ridiculous amounts for plain DDR
Same here. Sometimes you can get them cheap on e-bay-like market but otherwise it's better to save more money and buy a new computer w/ DDR2/3.

> arch doesn't tell the kernel subversion, only '2.6.33'
http://www.archlinux.org/download/
Included Kernel: 2.6.33.4

[karol@black pics]$ pacman -Q kernel26
kernel26 2.6.33.4-1

> i've read somewhere that ext4 is slow on IDE drives.
IDE? Ugh, man, that's bad.

Last edited by karol (2010-05-25 22:25:53)

bamdad · 2010-05-25 22:26:32

update: fedora's 'System Monitor' (it was gnome-system-monitor, not gnome-power-manager, sorry) is indeed 'lying', or it uses some other method than 'free -m'. see here: http://www.fw.hu/mutt/fedora.png

karol wrote:

IDE? Ugh, man, that's bad.

tell me about it.. it's a thinkpad T42. nice machine, but i'll have to get a new one soon.

Last edited by bamdad (2010-05-25 22:28:07)

karol · 2010-05-25 22:31:53

> it was gnome-system-monitor, not gnome-power-manager
I figured out that was a typo but had a chuckle anyway :-)

> is indeed 'lying', or it uses some other method
I think it's the latter, but 512M is not much and I assure you that 1G is quite a different experience. I also have another box w/ 512M but I dread to fire up X on it ;P

bamdad · 2010-05-29 00:30:19

HUGE UPDATE:

okay, so i've been against KMS since it first became default with the 2.6.30 (?) kernel. i have a crappy old mobility radeon 7500 (m7) video card, and i've noticed that with KMS enabled, some programs (mainly games) just refuse to work in full screen (produce a backlit black screen). with KMS disabled, they work just fine. examples are dosbox, armagetronad (i use it as a simple 3d test). others partially work (like smc, which freezes at a certain level where there are lots of objects on the screen). some work just fine (like alephone). now i'm not much of a gamer (especially with this laptop), but i miss dosbox in full screen. i have a hunch that this is related to mesa.

my main point is that after recently trying KMS again, i've noticed that the system became much snappier (just like fedora, as mentioned recently), and actually uses less memory. i can multitask and open lots of windows in firefox, even the menus and nautilus windows load much faster.

now i'm faced with a dilemma: a significantly slower system versus a partially broken fullscreen programs..

can someone please give me some advice on this?

karol · 2010-05-29 00:43:18

> i've noticed that the system became much snappier
That's the idea biehing KMS.

> can someone please give me some advice on this?
Two systems: one for games the other for work.

bamdad · 2010-05-29 01:09:21

karol wrote:

That's the idea biehing KMS.

i know, but i never thought it would improve this much on memory usage.

karol wrote:

Two systems: one for games the other for work.

obvious, but that's just circumventing the problem instead of fixing it. most of the time there is solution, and somebody might know about it. that's why i'm asking, because i didn't find it with standard google/wiki/forum searches.

blamed · 2010-05-30 04:11:16

blamed ~ % free -m
             total       used       free     shared    buffers     cached
Mem:          1503        749        754          0         58        428
-/+ buffers/cache:        262       1241
Swap:         4094          0       4094

that's gnome+skype+pidgin+dropbox.

dropbox+skype+pidgin ~= 120-130mb of ram. Clean gnome: 262-120=142.
so.. i don't see any difference with fedora.

Last edited by blamed (2010-05-30 04:12:31)

Square · 2010-05-30 06:36:03

Fresh Arch install on a box with full blown gnome + gdm going:

             total       used       free     shared    buffers     cached
Mem:           435        381         53          0         34        211
-/+ buffers/cache:        135        299
Swap:          517          1        515

That is xorg18 without HAL... privoxy and sshd running, as well as a gnome-terminal.

Last edited by Square (2010-05-30 06:36:31)

bamdad · 2010-06-13 16:22:12

okay, so i've figured it out (sort of).

the distros i've tried and seem to use less memory are ubuntu (10.04), fedora (13) and opensuse (11.3 milestone 7). all of these use upstart as the init replacement. i've tried others, mostly earlier versions of opensuse and debian, which use the traditional sysv init. these consume more or less the same amount of memory as arch.

on starting additional programs, i've noticed that upstart-based distros tend to manage memory in a different way. (according to 'gnome-system-monitor' and 'top', at least).

on upstart-based distros, programs apparently use considerably less memory, and the system stays more responsive while doing multitasking, but after quitting said programs, it takes a while for the memory usage to decrease, and sometimes it does not, at all.

on sysvinit-based distros, e.g. debian, and on arch too, programs eat larger chunks of memory, but immediately release it upon quitting them.

i don't know why this is so, since i'm not that well versed on how memory management works in linux.

sadly, it seems that my options are to switch to an upstart-based distro until i get more ram/a new laptop, or to use an extremely lightweight window manager in arch.

i hope this helps someone out there who's puzzled by similar issues.

Last edited by bamdad (2010-06-13 16:24:21)

dmz · 2010-06-13 16:50:48

free doesnt really tell anyone much. Try this: http://codepad.org/jSwz2XeW

Arch on laptop:

 Private  +   Shared  =  RAM used Program 

128.0 KiB +  20.5 KiB = 148.5 KiB init
204.0 KiB +  22.0 KiB = 226.0 KiB dhcpcd
192.0 KiB +  63.5 KiB = 255.5 KiB hald-addon-acpi
196.0 KiB +  63.5 KiB = 259.5 KiB xinit
192.0 KiB +  82.0 KiB = 274.0 KiB hald-addon-generic-backlight
196.0 KiB +  86.0 KiB = 282.0 KiB hald-addon-rfkill-killswitch
212.0 KiB +  89.0 KiB = 301.0 KiB hald-addon-input
236.0 KiB +  77.0 KiB = 313.0 KiB crond (2)
260.0 KiB +  84.5 KiB = 344.5 KiB hald-runner
292.0 KiB +  57.5 KiB = 349.5 KiB xclip
296.0 KiB +  54.0 KiB = 350.0 KiB dbus-launch
296.0 KiB + 110.5 KiB = 406.5 KiB unclutter
408.0 KiB +  66.0 KiB = 474.0 KiB ssh-agent
428.0 KiB +  95.5 KiB = 523.5 KiB login
352.0 KiB + 180.0 KiB = 532.0 KiB su (2)
444.0 KiB + 112.5 KiB = 556.5 KiB agetty (5)
384.0 KiB + 196.0 KiB = 580.0 KiB hald-addon-storage (2)
316.0 KiB + 350.5 KiB = 666.5 KiB udevd (3)
416.0 KiB + 286.0 KiB = 702.0 KiB sh
484.0 KiB + 286.0 KiB = 770.0 KiB startx
728.0 KiB +  44.5 KiB = 772.5 KiB wpa_supplicant
572.0 KiB + 316.0 KiB = 888.0 KiB dbus-daemon (2)
532.0 KiB + 424.0 KiB = 956.0 KiB gconfd-2
952.0 KiB + 453.0 KiB =   1.4 MiB syslog-ng (2)
  1.1 MiB +   1.2 MiB =   2.3 MiB sshd (3)
  2.5 MiB + 140.5 KiB =   2.7 MiB vim
  2.6 MiB + 189.5 KiB =   2.8 MiB hald
  2.9 MiB + 707.0 KiB =   3.6 MiB ratpoison
  4.3 MiB + 234.0 KiB =   4.5 MiB rmshit
  4.6 MiB + 817.0 KiB =   5.4 MiB urxvt
  4.8 MiB +   1.2 MiB =   6.0 MiB ssh (3)
  7.5 MiB +   1.4 MiB =   8.9 MiB zsh (6)
  8.1 MiB +   1.2 MiB =   9.4 MiB urxvtd
 14.5 MiB + 281.0 KiB =  14.8 MiB sshfs (3)
 29.4 MiB + 766.0 KiB =  30.1 MiB Xorg
 76.7 MiB +   1.4 MiB =  78.1 MiB firefox
---------------------------------
                        180.6 MiB
=================================

Arch on desktop:

 Private  +   Shared  =  RAM used Program 

 88.0 KiB +  14.0 KiB = 102.0 KiB init
108.0 KiB +  17.0 KiB = 125.0 KiB dhcpcd
120.0 KiB +  20.5 KiB = 140.5 KiB crond
140.0 KiB +  28.0 KiB = 168.0 KiB xinit
148.0 KiB +  24.0 KiB = 172.0 KiB dbus-daemon
184.0 KiB +  47.0 KiB = 231.0 KiB unclutter
220.0 KiB +  15.5 KiB = 235.5 KiB gpg-agent
228.0 KiB +  31.0 KiB = 259.0 KiB dbus-launch
228.0 KiB +  40.0 KiB = 268.0 KiB login
272.0 KiB +  22.0 KiB = 294.0 KiB startx
296.0 KiB +  45.5 KiB = 341.5 KiB ssh-agent
300.0 KiB +  75.0 KiB = 375.0 KiB agetty (5)
260.0 KiB + 255.5 KiB = 515.5 KiB udevd (3)
344.0 KiB + 196.0 KiB = 540.0 KiB su (2)
364.0 KiB + 216.5 KiB = 580.5 KiB gconfd-2
676.0 KiB +  24.0 KiB = 700.0 KiB sh
564.0 KiB + 144.5 KiB = 708.5 KiB ssh
604.0 KiB + 204.5 KiB = 808.5 KiB syslog-ng (2)
  1.2 MiB + 100.0 KiB =   1.3 MiB sftp-server (3)
  2.0 MiB +  72.5 KiB =   2.1 MiB rpd
  2.5 MiB + 127.5 KiB =   2.6 MiB perl
  3.0 MiB +  85.0 KiB =   3.1 MiB mutt
  3.3 MiB + 153.0 KiB =   3.5 MiB vim (2)
  3.5 MiB + 101.0 KiB =   3.6 MiB screen-4.0.3 (2)
  3.1 MiB + 624.5 KiB =   3.7 MiB urxvt
  2.2 MiB +   1.7 MiB =   3.9 MiB sshd (7)
 12.8 MiB + 310.0 KiB =  13.1 MiB mpdmon-daemon
 17.0 MiB + 781.5 KiB =  17.7 MiB urxvtd
 19.3 MiB +   1.5 MiB =  20.8 MiB zsh (20)
 22.1 MiB + 967.0 KiB =  23.1 MiB stumpwm
 33.8 MiB + 308.5 KiB =  34.1 MiB Xorg
 19.5 MiB +  26.0 MiB =  45.4 MiB mplayer (3)
 89.8 MiB + 948.0 KiB =  90.8 MiB firefox
119.0 MiB + 175.0 KiB = 119.2 MiB mpd
126.4 MiB + 332.0 KiB = 126.7 MiB rtorrent
---------------------------------
                        521.0 MiB

schen · 2010-06-13 17:29:41

What video driver are you using? I tried using the xf86-video-radeonhd with KMS, and full-screen things didn't work.

bamdad · 2010-06-13 17:40:10

schen wrote:

What video driver are you using? I tried using the xf86-video-radeonhd with KMS, and full-screen things didn't work.

i have a crappy mobility radeon 7500, so i was using xf86-video-ati. i needed to have an xorg.conf, too, because by default it was using XAA instead of EXA, which resulted in a garbled screen sometimes, and my video chip used to overheat.

on a side note, i'm currently trying to configure ubuntu to my liking (which is a lot harder than arch), but it seems to be the only way to go with 512M of ram. arch is completely backed up, and if ubuntu is as buggy and disorganised as it was before, i won't hesitate to go back to arch.

broch · 2010-06-13 23:54:20

this is all your config, nothing to do with distro.

example: my KDE 4.4 after boot:
free -m
total used free shared buffers cached
Mem: 2016 294 1722 0 0 205
-/+ buffers/cache: 88 1927
Swap: 4000 0 4000

example 2: with firefox running
free -m
total used free shared buffers cached
Mem: 2016 397 1618 0 0 275
-/+ buffers/cache: 122 1893
Swap: 4000 0 4000

so Fedora after boot takes more memory than Arch with KDE and firefox running.

bamdad · 2010-06-14 00:08:43

i don't see how this is my config.. the only difference could have been the modules and daemons (which are autoloaded under fedora). gnome was the same (even its version), and i had installed and started all the stuff i use regularly (dropbox, gnome-do etc.).

and the most striking thing was not that +/-30 MB's or so, but the fact that it fedora (and now ubuntu) was still responsive after starting firefox, thunderbird and some other apps, while on arch, it's swapping hell. and yes, i've set the swappiness on both to 10.

now i'm not saying that there's a problem with arch. no, it's still the best distro out there if you ask me. i think this might be related to gnome, xorg, or - as i've stated a few posts back - the init style differences. the latter is the most logical, because my 512 megs of memory is handled much better on *every* distro that uses upstart, and about the same way as arch on sysvinit-based ones (e.g. debian lenny).

i don't know about kde, never used it since kde 4 came out.

Arch Linux

#1 2010-05-25 20:43:22

[solved] memory consumption

#2 2010-05-25 21:19:09

Re: [solved] memory consumption

#3 2010-05-25 21:30:17

Re: [solved] memory consumption

#4 2010-05-25 21:35:41

Re: [solved] memory consumption

#5 2010-05-25 21:48:02

Re: [solved] memory consumption

#6 2010-05-25 21:49:52

Re: [solved] memory consumption

#7 2010-05-25 21:53:04

Re: [solved] memory consumption

#8 2010-05-25 21:53:48

Re: [solved] memory consumption

#9 2010-05-25 21:55:23

Re: [solved] memory consumption

#10 2010-05-25 22:01:18

Re: [solved] memory consumption

#11 2010-05-25 22:18:16

Re: [solved] memory consumption

#12 2010-05-25 22:23:59

Re: [solved] memory consumption

#13 2010-05-25 22:26:32

Re: [solved] memory consumption

#14 2010-05-25 22:31:53

Re: [solved] memory consumption

#15 2010-05-29 00:30:19

Re: [solved] memory consumption

#16 2010-05-29 00:43:18

Re: [solved] memory consumption

#17 2010-05-29 01:09:21

Re: [solved] memory consumption

#18 2010-05-30 04:11:16

Re: [solved] memory consumption

#19 2010-05-30 06:36:03

Re: [solved] memory consumption

#20 2010-06-13 16:22:12

Re: [solved] memory consumption

#21 2010-06-13 16:50:48

Re: [solved] memory consumption

#22 2010-06-13 17:29:41

Re: [solved] memory consumption

#23 2010-06-13 17:40:10

Re: [solved] memory consumption

#24 2010-06-13 23:54:20

Re: [solved] memory consumption

#25 2010-06-14 00:08:43

Re: [solved] memory consumption

Board footer