You are not logged in.

#1 2010-03-21 00:00:53

nfm
Member
Registered: 2008-06-13
Posts: 66

2.6.33 using 300% more memory than 2.6.32

Hey guys,

I noticed that 2.6.33 is using roughly 3x more memory 'free -m' when booting up to the vc/1.
I get about 110MB on x86_64 while with 2.6.32.10 I get 35MB. What is going on here? Both kernel
are plain vanilla.

Offline

#2 2010-03-21 10:03:49

Army
Member
Registered: 2007-12-07
Posts: 1,784

Re: 2.6.33 using 300% more memory than 2.6.32

Maybe it's because of KMS, which makes udev load the full display driver, even when X isn't being started? That's what I noticed here with my ATI card, when I only boot into tty, my system uses more RAM than before, but then after going into X, it needs less than before. Did you also compare the values when booting into X?

Offline

#3 2010-03-21 17:51:22

nfm
Member
Registered: 2008-06-13
Posts: 66

Re: 2.6.33 using 300% more memory than 2.6.32

I'm booting both kernels with uvesafb, and they are both loading the same modules. Once I enter the X I'm still missing the memory that kernel 2.6.33 seems to eat up. Maybe I have a memory leak in one of my modules.

Offline

#4 2010-03-21 21:41:22

smakked
Member
From: Gold Coast , Australia
Registered: 2008-08-14
Posts: 420

Re: 2.6.33 using 300% more memory than 2.6.32

COmplete opposite for me  2.6.33 dropped memory use ??


Certified Android Junkie
Arch 64

Offline

#5 2010-03-22 01:08:44

skottish
Forum Fellow
From: Here
Registered: 2006-06-16
Posts: 7,942

Re: 2.6.33 using 300% more memory than 2.6.32

I'm also seeing a large increase of reserved RAM on 64 bit and I doubt that KMS is a factor; This machine is running a nVidia card. I'm seeing this not only with X, but with most apps too. I see no evidence of memory leaks though. It seems that the kernel is just being more aggressive with how it uses memory. That's certainly not a problem if it releases it gracefully.

Offline

#6 2010-03-22 04:32:10

nfm
Member
Registered: 2008-06-13
Posts: 66

Re: 2.6.33 using 300% more memory than 2.6.32

Interesting hmm,

I do happen to own nvidia card but I'm sure the blob or drm drivers are not the cause. I have another x86_64 machine with same setup and don't have such a problem. I tried unplugging unneeded PCI cards and USB devices, but the memory usage dropped slightly and still was around 100MB when booting to vc/1. The only thing that it could be is the PCIe SCSI controller that I'm using and the SCSI HDD. There could have been regression introduced to SCSI stack, but this would be correct only if my speculation is true tongue. I might try installing base archlinux on separate SATA HDD and compare, and if true, I could probably bisect and find the offending commit.

Offline

#7 2010-03-22 05:53:43

nfm
Member
Registered: 2008-06-13
Posts: 66

Re: 2.6.33 using 300% more memory than 2.6.32

Hmm I guess it is just 2.6.33 in general using more memory. My laptop went from 25MB to 56MB, not as drastic as my main desktop which went 3x. I guess I'll have to live with this.

Offline

#8 2010-03-22 08:04:56

smakked
Member
From: Gold Coast , Australia
Registered: 2008-08-14
Posts: 420

Re: 2.6.33 using 300% more memory than 2.6.32

Edit by me

I just realized i did have the same issue till i pacthed the kernel with CK patchset.
then i noticed it uses less memory.


Certified Android Junkie
Arch 64

Offline

#9 2010-03-23 10:26:00

pawels64
Member
Registered: 2009-04-07
Posts: 55

Re: 2.6.33 using 300% more memory than 2.6.32

nfm wrote:

Hmm I guess it is just 2.6.33 in general using more memory.

It shouldn't use more memory. Report to lkml.

Offline

#10 2010-03-24 02:31:28

nfm
Member
Registered: 2008-06-13
Posts: 66

Re: 2.6.33 using 300% more memory than 2.6.32

pawels64 wrote:

It shouldn't use more memory. Report to lkml.

Seems like a logical thing to do. I'm not a unix newbie but personally I'm not willing to spend extra time debugging this issue since I don't know where to start. If somebody decides to write to lkml I can chime in.

Offline

#11 2010-03-24 08:42:38

pawels64
Member
Registered: 2009-04-07
Posts: 55

Re: 2.6.33 using 300% more memory than 2.6.32

nfm wrote:
pawels64 wrote:

It shouldn't use more memory. Report to lkml.

Seems like a logical thing to do. I'm not a unix newbie but personally I'm not willing to spend extra time debugging this issue since I don't know where to start. If somebody decides to write to lkml I can chime in.

I'll try myself when 2.6.33 will hit the stable repository and if my system will be affected I will report. Btw. I tried 2.6.34-rc2 and memory usage seemed to be lower then with *.32, but my measure was far from being accurate.

Offline

#12 2010-03-26 02:45:29

skottish
Forum Fellow
From: Here
Registered: 2006-06-16
Posts: 7,942

Re: 2.6.33 using 300% more memory than 2.6.32

I started looking into this a little closer because my system with xmonad, urxvt, and vimprobable2 running shouldn't come anywhere near the 303MB of RAM that I'm showing right now. For fun I added up all the stuff in the reserved column (I'm too lazy right now to do it smartly) of htop and it's roughly 150MB (about what I would expect for 64bit). I glanced over all of the stuff in the shared column and it looks like it adds up to about 50-60% of that. free -m is showing 0 shared but the same 303MB used +/- buffers/cache. Am I missing something here or is something else confused?

Offline

#13 2010-04-02 11:55:07

Anton
Member
From: Lund, Sweden
Registered: 2009-12-12
Posts: 17

Re: 2.6.33 using 300% more memory than 2.6.32

pawels64 wrote:

I'll try myself when 2.6.33 will hit the stable repository and if my system will be affected I will report.

Is this kernel likely to hit stable soon? Where can I find information about this?

Offline

#14 2010-04-03 00:47:28

ngoonee
Forum Fellow
From: Between Thailand and Singapore
Registered: 2009-03-17
Posts: 7,354

Re: 2.6.33 using 300% more memory than 2.6.32

Anton wrote:
pawels64 wrote:

I'll try myself when 2.6.33 will hit the stable repository and if my system will be affected I will report.

Is this kernel likely to hit stable soon? Where can I find information about this?

[arch-dev-public] mailing list.

And no, there's some bugs holding it up.


Allan-Volunteer on the (topic being discussed) mailn lists. You never get the people who matters attention on the forums.
jasonwryan-Installing Arch is a measure of your literacy. Maintaining Arch is a measure of your diligence. Contributing to Arch is a measure of your competence.
Griemak-Bleeding edge, not bleeding flat. Edge denotes falls will occur from time to time. Bring your own parachute.

Offline

#15 2010-04-03 13:31:54

Anton
Member
From: Lund, Sweden
Registered: 2009-12-12
Posts: 17

Re: 2.6.33 using 300% more memory than 2.6.32

Thank you! I will subscribe to that mailing list.

Offline

#16 2010-04-03 15:31:48

broch
Banned
From: L.A. California
Registered: 2006-11-13
Posts: 975

Re: 2.6.33 using 300% more memory than 2.6.32

I don't see any memory usage variations: 18MB from .31 to 34-rc2 as estimated by free -m.

free -m is not very adequate by the way, it gives only crude estimate of memory usage.

@skottish
unless you boot kernel only (without any sort of GUI/exit GUI) you can't tell what is going on.

Last edited by broch (2010-04-03 15:35:15)

Offline

#17 2010-04-03 19:33:49

skottish
Forum Fellow
From: Here
Registered: 2006-06-16
Posts: 7,942

Re: 2.6.33 using 300% more memory than 2.6.32

broch wrote:

I don't see any memory usage variations: 18MB from .31 to 34-rc2 as estimated by free -m.

free -m is not very adequate by the way, it gives only crude estimate of memory usage.

@skottish
unless you boot kernel only (without any sort of GUI/exit GUI) you can't tell what is going on.

That's a good point. I'm just going off of the fact that free, htop, and friends are now reading about 33 to 50% higher readings with the same basic setup as before. Where as before what they said seemed somewhat reliable, now I don't trust the information at all.

Offline

#18 2010-04-03 20:27:24

broch
Banned
From: L.A. California
Registered: 2006-11-13
Posts: 975

Re: 2.6.33 using 300% more memory than 2.6.32

skottish wrote:
broch wrote:

I don't see any memory usage variations: 18MB from .31 to 34-rc2 as estimated by free -m.

free -m is not very adequate by the way, it gives only crude estimate of memory usage.

@skottish
unless you boot kernel only (without any sort of GUI/exit GUI) you can't tell what is going on.

That's a good point. I'm just going off of the fact that free, htop, and friends are now reading about 33 to 50% higher readings with the same basic setup as before. Where as before what they said seemed somewhat reliable, now I don't trust the information at all.

someone here at Arch forums published very nice script for memory check, detailing actual memory use. Script is called ps_mem.py. It lists exact memory usage with split for each app running. Very nice.

If you can find it, then maybe try to boot first kernel only and check mem usage, the boot to GUI and this way you may be able to figure out if this is kernel or something affected by kernel or something else.

Offline

#19 2010-04-03 20:49:04

skottish
Forum Fellow
From: Here
Registered: 2006-06-16
Posts: 7,942

Re: 2.6.33 using 300% more memory than 2.6.32

broch wrote:
skottish wrote:
broch wrote:

I don't see any memory usage variations: 18MB from .31 to 34-rc2 as estimated by free -m.

free -m is not very adequate by the way, it gives only crude estimate of memory usage.

@skottish
unless you boot kernel only (without any sort of GUI/exit GUI) you can't tell what is going on.

That's a good point. I'm just going off of the fact that free, htop, and friends are now reading about 33 to 50% higher readings with the same basic setup as before. Where as before what they said seemed somewhat reliable, now I don't trust the information at all.

someone here at Arch forums published very nice script for memory check, detailing actual memory use. Script is called ps_mem.py. It lists exact memory usage with split for each app running. Very nice.

If you can find it, then maybe try to boot first kernel only and check mem usage, the boot to GUI and this way you may be able to figure out if this is kernel or something affected by kernel or something else.

It's in AUR. Cool.

See, now I'm even more convinced that something, somewhere is going nutty on my system.

Out of X:

ps_mem:  32.6 MiB
free:          249.9 MiB (assuming that I did the conversion correctly)

In X:

ps_mem:  131.0 MiB
free:          293.7 MiB

Offline

#20 2010-04-03 21:35:44

lucke
Member
From: Poland
Registered: 2004-11-30
Posts: 4,018

Re: 2.6.33 using 300% more memory than 2.6.32

Hm, you can use exmap and smem too.

Offline

#21 2010-04-03 21:52:53

broch
Banned
From: L.A. California
Registered: 2006-11-13
Posts: 975

Re: 2.6.33 using 300% more memory than 2.6.32

maybe there is something wrong with the way free calculates memory? Shared mem is not easy to estimate.
for example after booting to console free reads 18MB while ps_mem calculates 10.5MB. Than is 60% difference.

However look at your free -m readings:
without X or with X difference is minimal, which would suggest some problem with calculations?

Offline

#22 2010-04-03 22:02:57

skottish
Forum Fellow
From: Here
Registered: 2006-06-16
Posts: 7,942

Re: 2.6.33 using 300% more memory than 2.6.32

broch wrote:

maybe there is something wrong with the way free calculates memory? Shared mem is not easy to estimate.
for example after booting to console free reads 18MB while ps_mem calculates 10.5MB. Than is 60% difference.

However look at your free -m readings:
without X or with X difference is minimal, which would suggest some problem with calculations?

There's no question that there's a problem with the calculations. It's easiest to see in htop. The values are way off. It seems that ps_mem is accurate. The numbers it's producing are what I would expect.

On the subject of shared memory:

man_free wrote:

free  displays  the total amount of free and used physical and swap memory in the system, as well as the buffers used by the kernel.
       The shared memory column should be ignored; it is obsolete.

Errr... remove it from the program then?

Offline

#23 2010-04-09 09:12:27

pawels64
Member
Registered: 2009-04-07
Posts: 55

Re: 2.6.33 using 300% more memory than 2.6.32

In KDE I don't see a difference between 2.6.32.x and 2.6.33.2. Maybe there's some memory leak, but not in my configuration?

Offline

Board footer

Powered by FluxBB