2.6.33 using 300% more memory than 2.6.32

nfm · 2010-03-21 00:00:53

Hey guys,

I noticed that 2.6.33 is using roughly 3x more memory 'free -m' when booting up to the vc/1.
I get about 110MB on x86_64 while with 2.6.32.10 I get 35MB. What is going on here? Both kernel
are plain vanilla.

Army · 2010-03-21 10:03:49

Maybe it's because of KMS, which makes udev load the full display driver, even when X isn't being started? That's what I noticed here with my ATI card, when I only boot into tty, my system uses more RAM than before, but then after going into X, it needs less than before. Did you also compare the values when booting into X?

nfm · 2010-03-21 17:51:22

I'm booting both kernels with uvesafb, and they are both loading the same modules. Once I enter the X I'm still missing the memory that kernel 2.6.33 seems to eat up. Maybe I have a memory leak in one of my modules.

smakked · 2010-03-21 21:41:22

COmplete opposite for me 2.6.33 dropped memory use ??

skottish · 2010-03-22 01:08:44

I'm also seeing a large increase of reserved RAM on 64 bit and I doubt that KMS is a factor; This machine is running a nVidia card. I'm seeing this not only with X, but with most apps too. I see no evidence of memory leaks though. It seems that the kernel is just being more aggressive with how it uses memory. That's certainly not a problem if it releases it gracefully.

nfm · 2010-03-22 04:32:10

Interesting ,

I do happen to own nvidia card but I'm sure the blob or drm drivers are not the cause. I have another x86_64 machine with same setup and don't have such a problem. I tried unplugging unneeded PCI cards and USB devices, but the memory usage dropped slightly and still was around 100MB when booting to vc/1. The only thing that it could be is the PCIe SCSI controller that I'm using and the SCSI HDD. There could have been regression introduced to SCSI stack, but this would be correct only if my speculation is true . I might try installing base archlinux on separate SATA HDD and compare, and if true, I could probably bisect and find the offending commit.

nfm · 2010-03-22 05:53:43

Hmm I guess it is just 2.6.33 in general using more memory. My laptop went from 25MB to 56MB, not as drastic as my main desktop which went 3x. I guess I'll have to live with this.

smakked · 2010-03-22 08:04:56

Edit by me

I just realized i did have the same issue till i pacthed the kernel with CK patchset.
then i noticed it uses less memory.

pawels64 · 2010-03-23 10:26:00

nfm wrote:

Hmm I guess it is just 2.6.33 in general using more memory.

It shouldn't use more memory. Report to lkml.

nfm · 2010-03-24 02:31:28

pawels64 wrote:

It shouldn't use more memory. Report to lkml.

Seems like a logical thing to do. I'm not a unix newbie but personally I'm not willing to spend extra time debugging this issue since I don't know where to start. If somebody decides to write to lkml I can chime in.

pawels64 · 2010-03-24 08:42:38

nfm wrote:

pawels64 wrote:
It shouldn't use more memory. Report to lkml.
Seems like a logical thing to do. I'm not a unix newbie but personally I'm not willing to spend extra time debugging this issue since I don't know where to start. If somebody decides to write to lkml I can chime in.

I'll try myself when 2.6.33 will hit the stable repository and if my system will be affected I will report. Btw. I tried 2.6.34-rc2 and memory usage seemed to be lower then with *.32, but my measure was far from being accurate.

skottish · 2010-03-26 02:45:29

I started looking into this a little closer because my system with xmonad, urxvt, and vimprobable2 running shouldn't come anywhere near the 303MB of RAM that I'm showing right now. For fun I added up all the stuff in the reserved column (I'm too lazy right now to do it smartly) of htop and it's roughly 150MB (about what I would expect for 64bit). I glanced over all of the stuff in the shared column and it looks like it adds up to about 50-60% of that. free -m is showing 0 shared but the same 303MB used +/- buffers/cache. Am I missing something here or is something else confused?

Anton · 2010-04-02 11:55:07

pawels64 wrote:

I'll try myself when 2.6.33 will hit the stable repository and if my system will be affected I will report.

Is this kernel likely to hit stable soon? Where can I find information about this?

ngoonee · 2010-04-03 00:47:28

Anton wrote:

pawels64 wrote:
I'll try myself when 2.6.33 will hit the stable repository and if my system will be affected I will report.
Is this kernel likely to hit stable soon? Where can I find information about this?

[arch-dev-public] mailing list.

And no, there's some bugs holding it up.

Anton · 2010-04-03 13:31:54

Thank you! I will subscribe to that mailing list.

broch · 2010-04-03 15:31:48

I don't see any memory usage variations: 18MB from .31 to 34-rc2 as estimated by free -m.

free -m is not very adequate by the way, it gives only crude estimate of memory usage.

@skottish
unless you boot kernel only (without any sort of GUI/exit GUI) you can't tell what is going on.

Last edited by broch (2010-04-03 15:35:15)

skottish · 2010-04-03 19:33:49

broch wrote:

I don't see any memory usage variations: 18MB from .31 to 34-rc2 as estimated by free -m.
free -m is not very adequate by the way, it gives only crude estimate of memory usage.
@skottish
unless you boot kernel only (without any sort of GUI/exit GUI) you can't tell what is going on.

That's a good point. I'm just going off of the fact that free, htop, and friends are now reading about 33 to 50% higher readings with the same basic setup as before. Where as before what they said seemed somewhat reliable, now I don't trust the information at all.

broch · 2010-04-03 20:27:24

skottish wrote:

broch wrote:
I don't see any memory usage variations: 18MB from .31 to 34-rc2 as estimated by free -m.
free -m is not very adequate by the way, it gives only crude estimate of memory usage.
@skottish
unless you boot kernel only (without any sort of GUI/exit GUI) you can't tell what is going on.
That's a good point. I'm just going off of the fact that free, htop, and friends are now reading about 33 to 50% higher readings with the same basic setup as before. Where as before what they said seemed somewhat reliable, now I don't trust the information at all.

someone here at Arch forums published very nice script for memory check, detailing actual memory use. Script is called ps_mem.py. It lists exact memory usage with split for each app running. Very nice.

If you can find it, then maybe try to boot first kernel only and check mem usage, the boot to GUI and this way you may be able to figure out if this is kernel or something affected by kernel or something else.

skottish · 2010-04-03 20:49:04

broch wrote:

skottish wrote:
broch wrote:
I don't see any memory usage variations: 18MB from .31 to 34-rc2 as estimated by free -m.
free -m is not very adequate by the way, it gives only crude estimate of memory usage.
@skottish
unless you boot kernel only (without any sort of GUI/exit GUI) you can't tell what is going on.
That's a good point. I'm just going off of the fact that free, htop, and friends are now reading about 33 to 50% higher readings with the same basic setup as before. Where as before what they said seemed somewhat reliable, now I don't trust the information at all.
someone here at Arch forums published very nice script for memory check, detailing actual memory use. Script is called ps_mem.py. It lists exact memory usage with split for each app running. Very nice.
If you can find it, then maybe try to boot first kernel only and check mem usage, the boot to GUI and this way you may be able to figure out if this is kernel or something affected by kernel or something else.

It's in AUR. Cool.

See, now I'm even more convinced that something, somewhere is going nutty on my system.

Out of X:

ps_mem: 32.6 MiB
free: 249.9 MiB (assuming that I did the conversion correctly)

In X:

ps_mem: 131.0 MiB
free: 293.7 MiB

lucke · 2010-04-03 21:35:44

Hm, you can use exmap and smem too.

broch · 2010-04-03 21:52:53

maybe there is something wrong with the way free calculates memory? Shared mem is not easy to estimate.
for example after booting to console free reads 18MB while ps_mem calculates 10.5MB. Than is 60% difference.

However look at your free -m readings:
without X or with X difference is minimal, which would suggest some problem with calculations?

skottish · 2010-04-03 22:02:57

broch wrote:

maybe there is something wrong with the way free calculates memory? Shared mem is not easy to estimate.
for example after booting to console free reads 18MB while ps_mem calculates 10.5MB. Than is 60% difference.
However look at your free -m readings:
without X or with X difference is minimal, which would suggest some problem with calculations?

There's no question that there's a problem with the calculations. It's easiest to see in htop. The values are way off. It seems that ps_mem is accurate. The numbers it's producing are what I would expect.

On the subject of shared memory:

man_free wrote:

free displays the total amount of free and used physical and swap memory in the system, as well as the buffers used by the kernel.
The shared memory column should be ignored; it is obsolete.

Errr... remove it from the program then?

pawels64 · 2010-04-09 09:12:27

In KDE I don't see a difference between 2.6.32.x and 2.6.33.2. Maybe there's some memory leak, but not in my configuration?

Arch Linux

#1 2010-03-21 00:00:53

2.6.33 using 300% more memory than 2.6.32

#2 2010-03-21 10:03:49

Re: 2.6.33 using 300% more memory than 2.6.32

#3 2010-03-21 17:51:22

Re: 2.6.33 using 300% more memory than 2.6.32

#4 2010-03-21 21:41:22

Re: 2.6.33 using 300% more memory than 2.6.32

#5 2010-03-22 01:08:44

Re: 2.6.33 using 300% more memory than 2.6.32

#6 2010-03-22 04:32:10

Re: 2.6.33 using 300% more memory than 2.6.32

#7 2010-03-22 05:53:43

Re: 2.6.33 using 300% more memory than 2.6.32

#8 2010-03-22 08:04:56

Re: 2.6.33 using 300% more memory than 2.6.32

#9 2010-03-23 10:26:00

Re: 2.6.33 using 300% more memory than 2.6.32

#10 2010-03-24 02:31:28

Re: 2.6.33 using 300% more memory than 2.6.32

#11 2010-03-24 08:42:38

Re: 2.6.33 using 300% more memory than 2.6.32

#12 2010-03-26 02:45:29

Re: 2.6.33 using 300% more memory than 2.6.32

#13 2010-04-02 11:55:07

Re: 2.6.33 using 300% more memory than 2.6.32

#14 2010-04-03 00:47:28

Re: 2.6.33 using 300% more memory than 2.6.32

#15 2010-04-03 13:31:54

Re: 2.6.33 using 300% more memory than 2.6.32

#16 2010-04-03 15:31:48

Re: 2.6.33 using 300% more memory than 2.6.32

#17 2010-04-03 19:33:49

Re: 2.6.33 using 300% more memory than 2.6.32

#18 2010-04-03 20:27:24

Re: 2.6.33 using 300% more memory than 2.6.32

#19 2010-04-03 20:49:04

Re: 2.6.33 using 300% more memory than 2.6.32

#20 2010-04-03 21:35:44

Re: 2.6.33 using 300% more memory than 2.6.32

#21 2010-04-03 21:52:53

Re: 2.6.33 using 300% more memory than 2.6.32

#22 2010-04-03 22:02:57

Re: 2.6.33 using 300% more memory than 2.6.32

#23 2010-04-09 09:12:27

Re: 2.6.33 using 300% more memory than 2.6.32

Board footer