Kernel supposedly allocates more memory than sum of phys+virt mem

likytau · 2014-01-15 12:48:24

Just encountered this rather bizarre bug:

http://i.imgur.com/WhbhaQQ.png

I know memory usage is somewhat difficult to pin down in Linux, but .. you know.. having an application that supposedly is using all of my real ram, all of my swap (yet only 80k of my swap is actually used), and more besides that... Something is hilariously wrong here.

No actual ill effects, BTW, just blatant lies. Am posting mainly for curiosity's sake.

-- mod edit: read the Forum Etiquette and only post thumbnails http://wiki.archlinux.org/index.php/For … s_and_Code [jwr] --

Last edited by likytau (2014-01-15 23:50:03)

Runiq · 2014-01-15 13:16:22

likytau wrote:

No actual ill effects, BTW, just blatant lies. Am posting mainly for curiosity's sake.

No blatant lies at all. What you see in VIRT is the virtual memory allocated by that process. By default, Linux doesn't care how much memory you have. Instead, it grants all requests for however much memory a process wants. If a runaway process consumes so much memory that actual physical memory is at a limit—due to a memory leak, for example—, the OOM killer kicks in and hopefully kills the culprit.

likytau · 2014-01-15 14:37:20

Runiq wrote:

likytau wrote:
No actual ill effects, BTW, just blatant lies. Am posting mainly for curiosity's sake.
No blatant lies at all. What you see in VIRT is the virtual memory allocated by that process. By default, Linux doesn't care how much memory you have. Instead, it grants all requests for however much memory a process wants. If a runaway process consumes so much memory that actual physical memory is at a limit—due to a memory leak, for example—, the OOM killer kicks in and hopefully kills the culprit.

The point is that that didn't actually happen. If you look at the figures, the Firefox instance is supposedly consuming 13 gb of memory. Well, I only have 12gb, and that's counting swap (which is practically unused.). Where is the rest of the memory that it has supposedly allocated come from, and if it had actually in reality allocated it, shouldn't it have already been killed?

EDIT: Unless you are saying that the kernel will happily allocate more than the maximum possible virtual memory to a process, in which case, yes, it -is- on drugs.

More recently, it has arrived back in the land of the sane, and currently claims to be occupying 2.1gb of virtual memory.

Last edited by likytau (2014-01-15 14:46:06)

Runiq · 2014-01-15 14:46:14

likytau wrote:

Runiq wrote:
likytau wrote:
No actual ill effects, BTW, just blatant lies. Am posting mainly for curiosity's sake.
No blatant lies at all. What you see in VIRT is the virtual memory allocated by that process. By default, Linux doesn't care how much memory you have. Instead, it grants all requests for however much memory a process wants. If a runaway process consumes so much memory that actual physical memory is at a limit—due to a memory leak, for example—, the OOM killer kicks in and hopefully kills the culprit.
The point is that that didn't actually happen. If you look at the figures, the Firefox instance is supposedly consuming 13 gb of memory. Well, I only have 12gb, and that's counting swap (which is practically unused.). Where is the rest of the memory that it has supposedly allocated come from?

That's what I've been saying: There's no limit on how much processes can allocate. Whether they actually use the allocated memory is another story.

jasonwryan · 2014-01-15 19:13:08

Please edit your first post and choose a title that accurately reflects the content of the thread:
https://wiki.archlinux.org/index.php/Fo … ow_to_Post

MrCode · 2014-01-15 23:41:13

Runiq wrote:

Whether they actually use the allocated memory is another story.

The RES field typically shows this, non?

likytau · 2014-01-15 23:55:37

Runiq wrote:

That's what I've been saying: There's no limit on how much processes can allocate. Whether they actually use the allocated memory is another story.

So you're saying it's only in danger of getting OOM-killed when it actually accesses that memory (and that's why my swap was practically unused, because that memory isn't actually considered used until it accesses it)?

MrCode wrote:

The RES field typically shows this, non?

I usually take the RES field as the actual useful value, predicting whether the swapping from hell is about to happen.

Runiq · 2014-01-16 07:26:07

likytau wrote:

Runiq wrote:
That's what I've been saying: There's no limit on how much processes can allocate. Whether they actually use the allocated memory is another story.
So you're saying it's only in danger of getting OOM-killed when it actually accesses that memory (and that's why my swap was practically unused, because that memory isn't actually considered used until it accesses it)?

Yep. As an alternative, you can disallow overcommitting, which will throw an ENOMEM towards the first application that reaches the limit. However, since that's not the default policy, few applications handle that gracefully.

MrCode wrote:

The RES field typically shows this, non?

Not exactly, I believe. RES shows what's actually used in your RAM, but it doesn't take swap into account. Your entire virtual memory consists of RAM+swap.2112

Last edited by Runiq (2014-01-16 07:26:16)

Arch Linux

#1 2014-01-15 12:48:24

Kernel supposedly allocates more memory than sum of phys+virt mem

#2 2014-01-15 13:16:22

Re: Kernel supposedly allocates more memory than sum of phys+virt mem

#3 2014-01-15 14:37:20

Re: Kernel supposedly allocates more memory than sum of phys+virt mem

#4 2014-01-15 14:46:14

Re: Kernel supposedly allocates more memory than sum of phys+virt mem

#5 2014-01-15 19:13:08

Re: Kernel supposedly allocates more memory than sum of phys+virt mem

#6 2014-01-15 23:41:13

Re: Kernel supposedly allocates more memory than sum of phys+virt mem

#7 2014-01-15 23:55:37

Re: Kernel supposedly allocates more memory than sum of phys+virt mem

#8 2014-01-16 07:26:07

Re: Kernel supposedly allocates more memory than sum of phys+virt mem

Board footer