Arch Linux

wil93

Member

From: Italy

Registered: 2013-09-15

Posts: 72

Why can't I use all my memory when vm.overcommit_memory = 2 ?

From what I understand, vm.overcommit_memory can be either 0, 1 or 2:

2 = never overcommit
1 = always overcommit
0 = decide heuristically

Now, I wrote this to test the different options:

#include <iostream>
#include <cstring>

int main() {
    for (int i=1;;i++) {
        char* c = new char[10 * 1024 * 1024];
        memset(c, 0xAB, 10 * 1024 * 1024);
        std::cout << i * 10 << " MiB" << std::endl;
    }
}

This is the current output of free -h

              total        used        free      shared  buff/cache   available
Mem:           7.7G        1.3G        5.8G        211M        564M        5.9G
Swap:            0B          0B          0B

When overcommit_memory is either 0 or 1, I get the correct behavior (that is: the program tries to allocate until around 6000 MiB == 5.94 GiB and then it gets killed):

....
6040 MiB
6050 MiB
6060 MiB
6070 MiB
6080 MiB
6090 MiB
[1]    10015 killed     ./a.out

However, when I set overcommit_memory to 2, that little program gives this output:

10 MiB
20 MiB
30 MiB
40 MiB
50 MiB
60 MiB
70 MiB
80 MiB
90 MiB
100 MiB
110 MiB
120 MiB
130 MiB
140 MiB
150 MiB
160 MiB
170 MiB
180 MiB
190 MiB
200 MiB
210 MiB
220 MiB
230 MiB
240 MiB
250 MiB
terminate called after throwing an instance of 'std::bad_alloc'
  what():  std::bad_alloc
[1]    7867 abort (core dumped)  ./a.out

Notice that it was not killed: it actually thinks that there's no more memory!

In fact, I can't even start a web browser when overcommit_memory is 2, because I don't have a browser that takes less that 250 MiB of memory.

What could be the reason?

Lone_Wolf

Member

From: Netherlands, Europe

Registered: 2005-10-04

Posts: 11,911

Re: Why can't I use all my memory when vm.overcommit_memory = 2 ?

2    -    Don't overcommit. The total address space commit
        for the system is not permitted to exceed swap + a
        configurable amount (default is 50%) of physical RAM.
        Depending on the amount you use, in most situations
        this means a process will not be killed while accessing
        pages but will receive errors on memory allocation as
        appropriate.

        Useful for applications that want to guarantee their
        memory allocations will be available in the future
        without having to initialize every page.

Your swap is set to zero, what does /proc/meminfo show for CommitLimit ?

---

Disliking systemd intensely, but not satisfied with alternatives so focusing on taming systemd.

(A works at time B)  && (time C > time B ) ≠  (A works at time C)

wil93

Member

From: Italy

Registered: 2013-09-15

Posts: 72

Re: Why can't I use all my memory when vm.overcommit_memory = 2 ?

CommitLimit says 8046704 kB

However, I tried setting also vm.overcommit_ratio=100 (I don't know what is the default value), and now it gives the same error but around 4000 MiB, which is much more reasonable (and I can open a web browser). So probably, the issue was overcommit_ratio... I'm not sure why I would need to specify it in order to say "don't overcommit"...