Indefinite Instability ~2 years in [SOLVED]

V1del · 2022-11-09 10:28:42

Is "your super basic patch" on top of the latest repo grub release? That issue should be fixed in the latest GRUB release from the repos.

covid19 · 2022-12-19 07:10:27

I've stopped applying the patch and just have a copy of a patched /boot/grub/grub.cfg with entries in the correct order. Been playing around messing with this for awhile now; have gathered a couple answers.
The frequent "freezing" that I was guessing had something to do with VMware, as well as stability overall, is fixed via feeding the kernel "transparent_hugepage=never" at boot.
This eliminated the piss poor stability I was seeing, but yet it still seemed something was not quite right in terms of performance. Wasn't really seeing much related to my specific situation searching around, but somehow came upon the "real deal". Once I gave it "echo 0 > /proc/sys/vm/compaction_proactiveness" it's a day and night difference. Absolutely no lag, stuttering, keyboard strokes delaying, nada; buttery smooth.
I see what the disabling of transparent hugepages does, but no clue what the "compaction_proactiveness" has to do with. The original value was 20. Wiki does mention the transparent hugepages, but not the latter. Literal night and day difference. All of a sudden I can run 4 VMs all at once and have some odd 50 tabs open in Chromium, without any lag whatsoever.

As for the grub_debug_malloc, still seeing that on boot if I don't mash F9 and load GRUB via the system EFI boot menu. Not holding back any updates other than linux and linux-headers

Last edited by covid19 (2022-12-19 16:23:39)

cfr · 2022-12-19 09:21:58

covid19 wrote:

As for the grub_debug_malloc, still seeing that on boot if I don't mash F9 and load GRUB via the system EFI boot menu. Not holding back any updates other than linux and linux-headers

Something is still wonky, then. Are you sure you have the latest grub and that that grub is correctly installed with grub-install and configured with grub-config etc.? What's the output of efibootmgr?

Lone_Wolf · 2022-12-19 10:55:40

https://www.kernel.org/doc/html/latest/ … tl/vm.html has details and a note :

Note that compaction has a non-trivial system-wide impact as pages belonging to different processes are moved around, which could also lead to latency spikes in unsuspecting applications.

seth · 2022-12-19 11:07:16

https://bbs.archlinux.org/viewtopic.php … 2#p2063182

covid19 · 2022-12-23 06:59:48

cfr wrote:

Something is still wonky, then. Are you sure you have the latest grub and that that grub is correctly installed with grub-install and configured with grub-config etc.? What's the output of efibootmgr?

$ efibootmgr 
BootCurrent: 0001
Timeout: 0 seconds
BootOrder: 0001,0000
Boot0000* GRUB	HD(1,GPT,0d024583-35fa-0f4f-a699-0fca3161e548,0x800,0x64000)/File(\EFI\GRUB\grubx64.efi)
Boot0001* arch	HD(1,GPT,0d024583-35fa-0f4f-a699-0fca3161e548,0x800,0x64000)/File(\EFI\arch\grubx64.efi)
Boot0080* Mac OS X	PciRoot(0x0)/Pci(0x1f,0x2)/Sata(0,0,0)/HD(3,GPT,3a77ec51-107a-4093-929c-2a5f14

I took the 6.1.1 kernel today and so had an opportunity to mess about with the grub-install process, it's added a new entry @ "Boot0001 arch", however still does not boot without me manually going into that boot menu, Mac OS X is surprisingly on this list from about 3-4 years ago or so, I was using this HP Elitebook as a Hackintosh for about a year or two; until 10.11 lost security support. "macOS" is a pathetic joke of an OS. [/rant]

I do believe this is the most recent update, I've removed grub from my IgnorePkg @ /etc/pacman.conf

$ p -Qi grub
Name            : grub
Version         : 2:2.06.r403.g7259d55ff-1
Description     : GNU GRand Unified Bootloader (2)
Architecture    : x86_64
URL             : https://www.gnu.org/software/grub/
Licenses        : GPL3
Groups          : None
Provides        : grub-common  grub-bios  grub-emu  grub-efi-x86_64
Depends On      : sh  xz  gettext  device-mapper
Optional Deps   : freetype2: For grub-mkfont usage [installed]
                  fuse2: For grub-mount usage [installed]
                  dosfstools: For grub-mkrescue FAT FS and EFI support [installed]
                  lzop: For grub-mkrescue LZO support
                  efibootmgr: For grub-install EFI support [installed]
                  libisoburn: Provides xorriso for generating grub rescue iso using grub-mkrescue [installed]
                  os-prober: To detect other OSes when generating grub.cfg in BIOS systems
                  mtools: For grub-mkrescue FAT FS support [installed]
Required By     : archiso
Optional For    : None
Conflicts With  : grub-common  grub-bios  grub-emu  grub-efi-x86_64  grub-legacy
Replaces        : grub-common  grub-bios  grub-emu  grub-efi-x86_64
Installed Size  : 33.13 MiB
Packager        : Christian Hesse <eworm@archlinux.org>
Build Date      : Thu 08 Dec 2022 11:20:08 PM PST
Install Date    : Fri 09 Dec 2022 11:24:04 PM PST
Install Reason  : Explicitly installed
Install Script  : Yes
Validated By    : Signature

seth wrote:

https://bbs.archlinux.org/viewtopic.php … 2#p2063182

Not sure if all of it applies to me, I have no swap (16GB ram, rarely exceed 50%+)

I'm absolutely thrilled with how snappy and responsive my laptop now is. Will never ever allow transparent hugepages again, this has been a miracle it's as if I'm using a different, much more powerful machine. Not a single stutter or any type of lagginess, wish I had found this sooner. I probably didn't need to, but even my Linux VMs I've set the transparent hugepages to never. All this time I thought it was a resources thing, even reducing the CPU count in half and decreasing RAM by 2-3GB per every VM, still 10/10 performance. I couldn't be happier. Is this an isolated case? If not, can't fathom why the default behavior is THP enabled. It's been incredibly frustrating and a horrendous un-ending experience for quite awhile now tbh. Felt like I was running Windows Vista on a cheap underpowered emachine or something. I'm over the moon about this solution ^_^

Was giving the 6.1 kernel release notes a quick glance, saw a mentioning of performance impacts and specifically THP related. Not sure if it's the same thing as what I've been experiencing. Apparently it came down to some 3 lines, which they've reverted. Thought that was interesting

Last edited by covid19 (2022-12-23 07:14:27)

seth · 2022-12-23 09:41:52

You'll still have zswap, the pattern fits and thp are related, so you might want to check whether it's the direct cause to +1 the other reporter.

covid19 · 2022-12-23 20:53:53

seth wrote:

You'll still have zswap, the pattern fits and thp are related, so you might want to check whether it's the direct cause to +1 the other reporter.

Oh okay, I see, thanks for the heads up! Will do some further experimentation

covid19 · 2023-07-18 03:26:13

I know I already marked this solved, just wanted to chime in again now that its been multiple months.
This laptop has never ever had stability the way it does now. I can actually trust it when I walk away from my desk I won't come back to a screwup
I've only restarted once since these posts @ 100 days, and it was on my terms, updating my kernel, instead of system screwup as usual

19:20:06 up 73 days, 20:30,  3 users,  load average: 1.04, 0.83, 1.15

I'm sure there's cases where THP is a desirable thing, but wow... that steaming pile of dung took so much time away from me pulling my hair out. Must be something significant I guess to have it enabled by default, just kinda in disbelief how far-fetching the implications of dumping it have been. I'll never forget this, any and all new machines I setup now, the very first thing I do now at the earliest point possible is tell THP to piss off go where the sun doesn't shine
I hope someone finds this useful in the future, months after months of Windows ME like stability, it was very frustrating.
Still have the same IgnorePkg, taking in everything except for 'linux' and 'linux-headers'
Current versions 6.4.x, I'm on 6.2.13, everything smooooth

For good health this can be put in /etc/systemd/system/disable_transparent_hugepages.service, don't forget to systemctl enable and start

[Unit]
Description=Friends do NOT let friends use transparent huge pages

[Service]
Type=oneshot
ExecStart=/bin/sh -c "/usr/bin/echo never | /usr/bin/tee /sys/kernel/mm/transparent_hugepage/enabled"
ExecStart=/bin/sh -c "/usr/bin/echo never | /usr/bin/tee /sys/kernel/mm/transparent_hugepage/defrag"
ExecStart=/bin/sh -c "/usr/bin/echo 0 | /usr/bin/tee /proc/sys/vm/compaction_proactiveness"

[Install]
WantedBy=multi-user.target]

Can also edit your boot/kernel flags, the line containing "rw quiet loglevel=3", I've changed it to "rw loglevel=3 mitigations=off transparent_hugepage=never"
I use both the edit and the systemd service just to be sure. The last thing I want is even a small chance that THP not be told to piss off at the soonest possible point

TLDR: Friends don't let friends use transparent hugepages

And as far as the GRUB issue went, I didn't apply any patches or customize in any fashion, it just drops me into a rescue, requiring me to boot indirectly via efi boot. I solved this by disposing of GRUB altogether, and going with systemd-boot...Are the majority of people not using uefi capable hardware in 2023?
Can GRUB not be considered obsolete for the most part?(all I've ever used) Life has become much simpler booting via uefi, works reliably, consistently, every time

Arch Linux

#26 2022-11-09 10:28:42

Re: Indefinite Instability ~2 years in [SOLVED]

#27 2022-12-19 07:10:27

Re: Indefinite Instability ~2 years in [SOLVED]

#28 2022-12-19 09:21:58

Re: Indefinite Instability ~2 years in [SOLVED]

#29 2022-12-19 10:55:40

Re: Indefinite Instability ~2 years in [SOLVED]

#30 2022-12-19 11:07:16

Re: Indefinite Instability ~2 years in [SOLVED]

#31 2022-12-23 06:59:48

Re: Indefinite Instability ~2 years in [SOLVED]

#32 2022-12-23 09:41:52

Re: Indefinite Instability ~2 years in [SOLVED]

#33 2022-12-23 20:53:53

Re: Indefinite Instability ~2 years in [SOLVED]

#34 2023-07-18 03:26:13

Re: Indefinite Instability ~2 years in [SOLVED]

Board footer