You are not logged in.
I have a memory node with 16GB of RAM that I would like to allocate as 16 x 1GB hugepages. This is usually impossible after runtime due to fragmentation, and I end up with approximately 12 hugepages depending on how good a day the machine is having. So I'd like the hugepages to be allocated automatically during boot time, but I cannot find a way to specify an allocation node either using kernel boot parameters (hugepages=N) or the sysctl interface (vm.nr_hugepages).
Is there any way to allocate hugepages on a specific memory node besides writing on the /sys/devices/system/node/nodeX/ interface after boot?
Thanks!
Offline
If you want to allocate hugepages on a specific NUMA node instead of basically random, you HAVE to write that into the specific /sys/.../nodeX/hugepages/hugepages-1048576kB/nr_hugepages file on the node you wish to have these.
To not run into the memory fragmentation problem or memory already being used thus blocking allocation of full 1G pages, you'll want to do this as early as possible in the boot sequence which you can do using a systemd service file.
Red Hat has a nice instruction on how to set that up. Read Procedure 7.1 under section 7.3.1.
https://access.redhat.com/documentation … huge-pages
Last edited by Omar007 (2020-07-11 12:43:44)
Offline
If you want to allocate hugepages on a specific NUMA node instead of basically random, you HAVE to write that into the specific /sys/.../nodeX/hugepages/hugepages-1048576kB/nr_hugepages file on the node you wish to have these.
To not run into the memory fragmentation problem or memory already being used thus blocking allocation of full 1G pages, you'll want to do this as early as possible in the boot sequence which you can do using a systemd service file.
Red Hat has a nice instruction on how to set that up. Read Procedure 7.1 under section 7.3.1.
https://access.redhat.com/documentation … huge-pages
Thanks! Yes this is a reasonable approach, but even with this "early" reservation it is not possible to reserve 16 x 1GB hugepages, I get at most 15. I was hoping for a kernel parameter or an "earlier" mechanism in the boot sequence, but this might very well not exist.
Offline
How many memory does that node have in total? If 16GB is all that is connected to that node, the full 16GB will not be possible afaik. 15GB will be the most you can get.
Initializing a CPU and hardware in general will already use memory even before any OS is touched at all.
Offline
How many memory does that node have in total? If 16GB is all that is connected to that node, the full 16GB will not be possible afaik. 15GB will be the most you can get.
Initializing a CPU and hardware in general will already use memory even before any OS is touched at all.
This is a computer with two memory domains, "node0" with 192GB and "node1" with 16GB. Default allocations go to node0, whereas node1 has to be managed with numactl or memkind. I want these 16GB in node1 to be fully "hugepaged". Maybe it is not possible, even at boottime.
Offline
Maybe someone can correct me if I'm wrong but in that case, as far as I'm aware, you're at the maximum possible with those 15 pages when using 1GB pages due to the aforementioned reason; hardware initialization will already have taken up some of the memory, making the last 1GB unavailable for this.
I am not aware of any method to prevent that from happening and keeping the whole set completely untouched and available for hugepages. So you'll need to have slightly more memory on that node than you want to assign for hugepages so add some or accept the 15 I suppose.
Last edited by Omar007 (2020-07-12 12:40:16)
Offline
Maybe someone can correct me if I'm wrong but in that case, as far as I'm aware, you're at the maximum possible with those 15 pages when using 1GB pages due to the aforementioned reason; hardware initialization will already have taken up some of the memory, making the last 1GB unavailable for this.
I am not aware of any method to prevent that from happening and keeping the whole set completely untouched and available for hugepages. So you'll need to have slightly more memory on that node than you want to assign for hugepages so add some or accept the 15 I suppose.
Ok. Thanks a lot for your help!
Offline