You are not logged in.

#1 2019-04-17 05:45:17

magicloud
Member
Registered: 2014-06-17
Posts: 48

[SOLVED] Huge IO load when memory is full.

Observed this a few times. And due to system freezing from the high IO load, I could not figure out what actually happened.

Symptom:
With/without physical swap, or zswap, when some progress gets crazy and keeps allocating memory until memory is (nearly) full (observed on memory usage monitor), a huge IO is triggered (observed by disk light). The "freezing" lasts a couple of hours and I have to force shutdown.

What I tried:
Set following value with sysctl. Not helping.
vm.swappiness = 10
vm.min_free_kbytes = 65536

The disk is SSD, and I have tried a totally another computer, same symptom.\

So my questions are, what was happening? How to prevent it?

Last edited by magicloud (2019-04-21 09:30:11)

Offline

#2 2019-04-17 06:40:07

seth
Member
Registered: 2012-09-03
Posts: 49,992

Re: [SOLVED] Huge IO load when memory is full.

swapping, maybe thrashing.
Lowering swappiness will only prefer to drop file caches over swapping out anon pages (what is ultimately prone to thrashing, because required files, includiding binaries and libraries, now have to be constantly re-read from disk)

https://bbs.archlinux.org/viewtopic.php … 9#p1763139
https://unix.stackexchange.com/question … -frozen-os

If a single process has a tendency to go wild, DON'T USE CRAPPY SOFTWARE. If you cannot avoid that, pass it a hard quota and get it kill if it want's more than n xB of RAM.

Offline

#3 2019-04-17 08:28:25

Mr.Elendig
#archlinux@freenode channel op
From: The intertubes
Registered: 2004-11-07
Posts: 4,092

Re: [SOLVED] Huge IO load when memory is full.

What hardware is this/how much ram do you have?


Evil #archlinux@libera.chat channel op and general support dude.
. files on github, Screenshots, Random pics and the rest

Offline

#4 2019-04-21 04:55:42

magicloud
Member
Registered: 2014-06-17
Posts: 48

Re: [SOLVED] Huge IO load when memory is full.

seth wrote:

swapping, maybe thrashing.

I see. It is a building and testing host. Both tasks could require too much memory. I have some tools running killing those huge RSS processes. But before that, which bugs me is that, I cannot save anything.

Mr.Elendig wrote:

What hardware is this/how much ram do you have?

Regular Intel CPU PC, 8 GiB memory.

Offline

#5 2019-04-21 06:04:21

seth
Member
Registered: 2012-09-03
Posts: 49,992

Re: [SOLVED] Huge IO load when memory is full.

If "building" includes one of the major browser engines (especially chromium, whether for a browser or Electron):
8GB RAM is the bare minium, 32GB is reasonable. See https://chromium.googlesource.com/chrom … uctions.md

Ensure you've a sufficiently large swap device, ie. eg. 24GB in such case. Building chromium will still take ages, but should not thrash (given you rise the swappiness again)

Even better: get more RAM ;-)

Offline

#6 2019-04-21 09:25:44

magicloud
Member
Registered: 2014-06-17
Posts: 48

Re: [SOLVED] Huge IO load when memory is full.

seth wrote:

Even better: get more RAM ;-)

I have a "more RAM" host, but too lousy, and too hot (physically)....

Anyway, I set the vm.overcommit_memory to 2. Let me see if that, at least, works on those progresses with memory leakage.

Offline

Board footer

Powered by FluxBB