Arch Linux

ugjka

Member

From: Latvia

Registered: 2014-04-01

Posts: 1,806

Website

Discussion about vm.swappiness variable

Here is a discussion that I started in the "Grrr" thread and it got a bit heated, so I'm "splitting it off" here, comments about your experience are welcome

So setting swappiness to low value is bullshit advice it seem from my testing. I have 4 gigs of ram and I had swappiness set to 10. When I ran out of ram what happenned was that my system would lock up and my swap partition would not get any use and I had to hard reset my system because I couldn't do anything. So I had some beers and decided to experiment with swappiness value. First I set swappiness to 50 so I expected that system would start to swap when my ram would hit 50% of usage. Nope, I exhausted my ram and still had barely any swap activity. Then I set it to 60 which is the default apparently, still same stuff swap activity was low. Now I have it set to 100 which is maximum and guess what it still only starts to swap when my ram is almost full. BUT my system is not locking up anymore and I don't have to hard reset my system anymore because swapping actually starts to happen when it needs to happen (hey I'm low on memory, pls swap something instead of locking up on my system).

I suppose someone should look into the kernel code and see how swap stuff actually works because it seems the common belief that setting swappiness to low value is completely wrong from my experience

swappiness is *NOT* about "OMG, only 80% RAM left, let's slow down the system by invoking the swap partition"
https://unix.stackexchange.com/question … by-default

swappiness is *NOT* about "OMG, only 80% RAM left, let's slow down the system by invoking the swap partition"
https://unix.stackexchange.com/question … by-default

Can you give tl;dr or eli5 because that was way above my head

RAM is occupied by files read into it and "anonymous" data (stuff that's generated on the fly and not available on disk)

If you get short on RAM the kernel can either sacrifice re-readable data or (if available) swap out anon pages to the swap partition/file.
100 means evenly kick file data and swap out anon pages, 20 means to preferably sacrifice re-readable data, 180 would be the opposite (swapping out anon pages and leave file data mostly untouched)

Depending on the way the memory is stressed (pace, type) things can go wrong with a "bad" swappiness.
If you set the swappiness to 20 and then fill RAM with random data as fast as you can you might end up in a situation where the kernel needs to constantly re-read files from disk (and kicks file data which it will re-read in the next cycle ....) - while the (intentionally) rogue process won't get killed before RAM and SWAP are (nearly) consumed.

The extreme values (0/200) got special treatment, but idk if that's still the case.

Memory management isn't simple - sorry :-(

...
The extreme values (0/200) got special treatment, but idk if that's still the case.
...

I think it is 0 to 100 now, cause I can't set it above 100 (I got error on boot when I tried)

So setting swappiness to low value is bullshit advice it seem from my testing.

it depends entirely on your use case. when I had a 32bit system, ages ago, I would still game a lot and would end up with swap issues if I didn't have a low swap setting, my PC would almost lock up briefly as the drive was thrashed. with a low swap setting, I was able to play with relatively little inconvenience due to drive thrashing. it all depends on the type of data you're using

So setting swappiness to low value is bullshit advice it seem from my testing.

it depends entirely on your use case. when I had a 32bit system, ages ago, I would still game a lot and would end up with swap issues if I didn't have a low swap setting, my PC would almost lock up briefly as the drive was thrashed. with a low swap setting, I was able to play with relatively little inconvenience due to drive thrashing. it all depends on the type of data you're using

Mind me, but I have vm.page-cluster=16 in my sysctl which make swap be written out in 64K chunks, maybe that's improving things. But really having swapiness set to 100 I see no downside since swapping seems to only happen on low ram. I also have a ssd with bfq enabled. I do remember I had problems with swapping years ago which is why bought into the whole thing of setting swapping to low value because it did improve the situation. I'm pretty sure something has changed in the kernel recently because it is obvious from my testing that things work differently now. ¯\_(ツ)_/¯

Edit: I'll report back in a week how's these new changes are going

I suppose someone should look into the kernel code and see how swap stuff actually works because it seems the common belief that setting swappiness to low value is completely wrong from my experience

The wiki says "Using a low value on sufficient memory is known to improve responsiveness on many systems."  Obviously, when you can afford to avoid swapping the system will be faster due to doing less work. Note that an otherwise unused swap may still be useful for hibernation.

My grrr is that whenever some system setting (such as vm.swappiness) takes values between 0 and 100, people immediately interpret it as a percentage of something.

Will see, from the little stress testing I did this evening it seems the swappiness value has no meaning unless you are really running out of ram. On low value my system starts to experience high io activity and locks up, on high value it swaps and everything is smooth.

If you have a lot of ram you probably never ran in low memory condition and thus never experieced what I described.

the swappiness value has no meaning unless you are really running out of ram

This.

As indicated: where would be the point in going for disk I/O otherwise?
You can drop caches and preload any stuff into RAM if you feel like this gets you somewhere; but generally preferring (dead slow) swap space over RAM would just be silly waste of resources.

the swappiness value has no meaning unless you are really running out of ram

This.

As indicated: where would be the point in going for disk I/O otherwise?
You can drop caches and preload any stuff into RAM if you feel like this gets you somewhere; but generally preferring (dead slow) swap space over RAM would just be silly waste of resources.

I reread the entire stuff again, basically swappiness value is just a ratio between file cache and anonymous pages which I understand correctly are program file assets vs programs stack.
So when you run oom the kernel will evict one of those depending on what ratio is set.

it seems the swappiness value has no meaning unless you are really running out of ram.

Of course it doesn't.  Swapiness settings are not to tell your computer to use swap despite there being no reason to use swap.  Swap is used when you are running low on memory.  Period.  The swapiness settings influence how it is used: which pages are swapped out first.

Will see, from the little stress testing I did...

How did you do this?  Some artificial process written to consume memory?  That will not give meaningful results at all.

In any case, this should probably split off to a support thread.

(edit: oops, I missed the last page of comments including the two above, my post is mostly redundant now)

How did you do this?  Some artificial process written to consume memory?  That will not give meaningful results at all.

Opening trice as many tabs in my browser with memory hungry web apps and 10 instances of visual studio code with different projects of mine, and bunch of other apps that I have installed, pretty much simulating heavy multitasking that I usually couldn't do because my system would go berzerk with io activity and everything would come to halt

In any case, this should probably split off to a support thread.

(edit: oops, I missed the last page of comments including the two above, my post is mostly redundant now)

No need for a support thread I've made up my mind, I'll leave my swappiness to 100

My grr is that most guides on the internet are just set this to X to do Y without explaining anything.

Thanks for the explanation.  That sounds like a good way to test swap use.

Thanks for the explanation.  That sounds like a good way to test swap use.

The morale of the story is that you god damn want your system to start to swap when you are getting near OOM condition!

Edit: I really think the whole "set swappiness to low value" thing is perpetuated by peaple who have plenty of ram and have never encountered a near OOM condition. And I wanna see benchmarks that it somehow improves perforance lol because my system now having swappines set to 100 is more performant then ever. Really I no more need to worry how many apps I have open or how many chrome tabs I can handle.

The morale of the story is that you god damn want your system to start to swap when you are getting near OOM condition!

I want my system to kill the offending process(es) when it gets near OOM. (I have no swap at the moment, but even when I did I would prefer it to be used only for hibernation because every single time the system started swapping, there was something wrong with the program(s) running and the system would run out of swap eventually, so there was no point in prolonging the period before crash.) But most of the time, the system freezes for like 30 minutes before the OOM killer kicks in. There seems to be a difference based on how the program allocates memory: does it allocate many small chunks or one big chunk? I guess in the first case identifying the offending process takes some time, but 30 minutes, really?

Anyway, if you want a stress test to get into OOM situation regardless of how much memory (including swap) you have, try to compile the python-graph-tool package using as many parallel threads as is the number of cores on your system. Note that all compilers I've seen allocate memory by small chunks, so I think that's actually a very good test.

I really think the whole "set swappiness to low value" thing is perpetuated by peaple who have plenty of ram and have never encountered a near OOM condition. And I wanna see benchmarks that it somehow improves perforance lol because my system now having swappines set to 100 is more performant then ever. Really I no more need to worry how many apps I have open or how many chrome tabs I can handle.

Right now this sounds like you've made a preconceived opinion on swappiness and you're actually promoting the highest value for everybody. I fail to see how this is better than "perpetuating" the opposite...

oom_kill_allocating_task=1

This could however have unwanted outcome (see https://www.kernel.org/doc/Documentation/sysctl/vm.txt)

You can also make a particular process more or less likely to be killed in OOM conditions altering /proc/$PID/oom_adj - happy reading ;-)

I guess in the first case identifying the offending process takes some time, but 30 minutes, really?

There is always Alt-SysCtrl-F for that

oom_kill_allocating_task=1

This could however have unwanted outcome (see https://www.kernel.org/doc/Documentation/sysctl/vm.txt)

Nope. I just tried it with a leaking program and the oom killer still killed postgres instead, which was sitting quietly and inactively in the background. And even though the oom killing algorithm was much less complex in this case, the system got still frozen for a couple of minutes.

You can also make a particular process more or less likely to be killed in OOM conditions altering /proc/$PID/oom_adj - happy reading ;-)

I'm aware of this, but it seems that the only program on my system which uses this is chromium, which is usually not the problem. And adjusting it manually after the process has been started is basically unapplicable to quick-running processes or processes which spawn subprocesses.

There is always Alt-SysCtrl-F for that

That's actually the only effective workaround I've seen so far. Thanks!

I guess in the first case identifying the offending process takes some time, but 30 minutes, really?

There is always Alt-SysCtrl-F for that

You meant SysRq instead of SysCtrl, right?

You meant SysRq instead of SysCtrl, right?

Yes, of course, sorry. I only have PrtScr written on my key, so I quoted it from my head.

The morale of the story is that you god damn want your system to start to swap when you are getting near OOM condition!

I want my system to kill the offending process(es) when it gets near OOM. (I have no swap at the moment, but even when I did I would prefer it to be used only for hibernation because every single time the system started swapping, there was something wrong with the program(s) running and the system would run out of swap eventually, so there was no point in prolonging the period before crash.)

Not everything can be swapped out and you'll have OOM situation probably with your swap half full at least that's what happened on my system

Right now this sounds like you've made a preconceived opinion on swappiness and you're actually promoting the highest value for everybody. I fail to see how this is better than "perpetuating" the opposite...

the highest value actually means that priorities are equal when it comes to evicting anonimous pages vs file cache and it is reccommended for I/O heavy processes

Given a vm.swappiness of 100, the priorities would be equal (file_prio=200-100=100, anon_prio=100). This would make sense for an I/O heavy system if it is not wanted that pages from the file cache being evicted in favour of anonymous pages.

We should put this discussion to rest, this is not the right place. What matters is that it works for my workload and I'm happy with it. Don't listen to me do your own testing, your workloads may require different settings depending how much ram you have

Last edited by ugjka (2017-10-26 10:21:39)

---

https://ugjka.net
paru > yay | webcord > discord
pacman -S spotify-launcher
mount /dev/disk/by-...

ugjka

Member

From: Latvia

Registered: 2014-04-01

Posts: 1,806

Website

Re: Discussion about vm.swappiness variable

But most of the time, the system freezes for like 30 minutes before the OOM killer kicks in. There seems to be a difference based on how the program allocates memory: does it allocate many small chunks or one big chunk? I guess in the first case identifying the offending process takes some time, but 30 minutes, really?

Is there high IO activity when your system freezes? Like is your HDD constantly lit up when it happens? Then what happens really is that  file cache has been completely pushed out of ram and your system starts to read everything directly from your disk. Which is utterly slow and that's why file cache exists to make it fast. I had this problem when I had swappiness set to 10. The system would favour anonymous pages to be kept in ram instead file cache.

Last edited by ugjka (2017-10-26 09:37:30)

---

https://ugjka.net
paru > yay | webcord > discord
pacman -S spotify-launcher
mount /dev/disk/by-...

ewaller

Administrator

From: Pasadena, CA

Registered: 2009-07-13

Posts: 19,774

Re: Discussion about vm.swappiness variable

I approve of the split -- but I did not see the discussion as heated.  Lively, and good debate - yes.  But I thought it quite civil.

---

Nothing is too wonderful to be true, if it be consistent with the laws of nature -- Michael Faraday
Sometimes it is the people no one can imagine anything of who do the things no one can imagine. -- Alan Turing
---
How to Ask Questions the Smart Way

lahwaacz

Wiki Admin

From: Czech Republic

Registered: 2012-05-29

Posts: 749

Re: Discussion about vm.swappiness variable

Is there high IO activity when your system freezes? Like is your HDD constantly lit up when it happens? Then what happens really is that  file cache has been completely pushed out of ram and your system starts to read everything directly from your disk. Which is utterly slow and that's why file cache exists to make it fast. I had this problem when I had swappiness set to 10. The system would favour anonymous pages to be kept in ram instead file cache.

I doubt that there is an IO problem. The LED indicator on my laptop is normally constantly on and I think that it would start blinking on high IO activity, but I can't remember when was the last time I noticed that because there is an SSD inside.