You are not logged in.
Hi,
I'm owning a server with 64GB of Ram, and I'm trying to allocate 48 of them as Hugepages/Hugetlb.
Tried the "echo 24576 > /proc/sys/vm/nr_hugepages", it doesn't apply.
Tried to add the kernel parameter at boot with the same value, it doesn't apply.
The maximum I can get is apparently the default one, which is stuck at 8200 huge pages.
Dmesg shows that HugeTLB allocated them.
[ 0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-linux-lts root=/dev/sda2 rw quiet hugepages=24576
[ 0.000000] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-linux-lts root=/dev/sda2 rw quiet hugepages=24576
[ 0.793975] HugeTLB registered 2 MB page size, pre-allocated 24576 pages
Is there a soft limit that I am missing ?
Thanks for any advice.
Linux pallet-town.com 3.14.29-1-lts #1 SMP Fri Jan 16 20:57:17 CET 2015 x86_64 GNU/Linux
AnonHugePages: 120832 kB
HugePages_Total: 8200
HugePages_Free: 8200
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB
kernel.shmmax=51539607552
kernel.shmall=51539607552
vm.nr_hugepages=24576
vm.hugetlb_shm_group=0
root soft memlock 50331648
root hard memlock 50331648
PS: I'm doing, FOR NOW, everything under root. Since I can't even get it to work for root, I'm not trying to do it with any other user, but that will be done afterwards.
EDIT:
Marking this as solved. I don't know exactly what I did to have it working, but it seems to me that hugepages, especially when you ask a big quantity, take a lot of time to initialize upon system boot. I simply spam "echo 24576 > /proc/sys/vm/nr_hugepages" until it is correctly set (displaying the contents of the file every once in a while shows it growing slowly). It seems ""hacky"" but at least it works, and since it's on a server that reboots maybe twice a year tops, it's not too big of an issue for me.
If someone comes with a better solution, I'm all ears.
Last edited by Webbeh (2015-09-12 15:18:05)
Offline
Hey, but I suppose it was related to deprecation of
/etc/sysctl.confin favor of
/etc/sysctl.d/*.confDeprecation of /etc/sysctl.conf
(yes, I know it's 3+ years later, but still)
Last edited by murlakatamenka (2018-06-01 05:10:59)
Offline
Hi.
I actually migrated the file to its own .conf in the sysctl.d folder just after my last answer, but it didn't help.
The system was trying to allocate the hugepages upon boot but just took an awful lot of time to do so.
All I needed to do was wait before starting the softwares that needed the hugepages. I ended up creating a loop in my software that was just getting the current nr_hugepages value and setting it to the one I wanted, every 2 seconds, until it reached the correct value.
If anyone encounters the same issue and doesn't want to edit the sources, you can simply write a small bash script that does that and starts your software when the correct nr_hugepages is reached.
Offline