You are not logged in.

#1 2019-11-08 10:38:57

clupus
Member
From: Saarbrücken
Registered: 2017-01-31
Posts: 22

Problem with non-working swap partition

Hello,

I have a problem with my system as it seems swap is not working as expected. I am not pretty sure, in which forum this question is better located, I think here it matches but I am unsure. So please forgive me if got it wrong.

First, I am pretty sure, I know what swap is (not) for.
I do have a machine with 16GB of RAM installed. No problem there. Nevertheless since arch uses tmpfs for /tmp, the data written there is put in RAM/Swap. In older days this data would have handled in its own partition on HDD. Nowadays, small pieces of data (KB to MB) are well located in RAM for quick access. However I had multiple times the problem, that bigger files or directories were generated in /tmp causing it to fill up. I expect the kernel to start using the swap partition as an extension to the RAM.
I do not request that all the data from programs and all the cache (crazy) would be put to swap. I just want to have a way to transparently switch to the HDD/SSD backed storage as soon as the files grow bigger. I thought this to be the major benefit of e.g. tmpfs over ramfs for /tmp.

This whole problem gets more critical as soon as I run 1-3 virtual machines each eating away some amount of RAM (that is not swappable).

The problem I see is that as soon as RAM approaches ~95% the machnine starts to work (CPU fans on 100%). Part of my programs gets killed or freezes completely. I do not know why. Normally all I can do is to cut the power and thus force a hard reset. Recently I was lucky that I had to leave the machine alone for some time (in the range of hours) during which this happend and the CPU afterwards droped again to a more healty value. Most of my programs was not working anymore, so I had to reboot, but I was able to dump the output of dmsg.
The OOM-killer seems not to have kicked in. I put the (selected) output here: https://gist.github.com/christianlupus/ … d223ec1250. There are some messages that some tasks are waiting long for carrying out the saving of certain data.

As I have a SSD I reduces vm.swappiness to 5 in the past. I assume this was only to protect the SSD from wearing out. I just increaed now to 30 after reading https://unix.stackexchange.com/question … by-default. I have not yet tested it out.

Do I miss something? Why is the kernel not working as assumed to save the large data blocks to swap?
Of course I could add a partition /tmp-static that is a normal partition containing larger temporary files and work out a solution to clean it on boot as well. However, I doubt this to be the best solution....

Thanks
Christian

Offline

#2 2019-11-08 11:17:24

Wild Penguin
Member
Registered: 2015-03-19
Posts: 319

Re: Problem with non-working swap partition

What is filling up your /tmp? Under normal circumstances, there shouldn't be too much in there. If there really are gigabytes of stuff there, consider moving them elsewhere (and/or rethink your configuration). EDIT: If this is not intentional (not caused by something odd you did to your setup), I'd consider anything eating more than a few tens of megabytes from /tmp a bug. It just "shouldn't happend". I haven't looked into my /tmp that often, but when I have, IIRC I've seen at max 100-200MB used (total, ever - and even that I'd consider a lot). Right now, after a fresh boot, with an Office Suite (Libreoffice), Desktop (Plasma), Firefox and other stuff running, I see 24K used in /tmp.

A wild guess: Kernel sees an open file in /tmp as "in use" and does not want to swap it. Do you have very large files in there which are also open?

If there are very large files in there which are closed, well then I believe Kernel should swap them. If it doesn't, well, maybe there is a setting which can work around this (hmm, or perhaps some bug with tmpfs).

However if your RAM hits 95%, I think you just have too little RAM. SWAP is (in these days) mainly useful for allowing more RAM for cache/buffers, if some unused applications (and their data) is moved into SWAP (AFAIK Kernel will never move cache/buffers into SWAP; that would be stupid indeed. If there is too little RAM, cache/buffers are just discarded).

FWIW I did use to compile on a computer with little RAM and with a huge swap file (well, partition), with the compilation directory in the tmpfs (on Gentoo). I had no issues there, things worked as they were supposed to (compilation succeeded and the resultant package was installed in the end). Hence, I believe your Kernel is not swapping stuff (you would like it to) because in the eyes of the Kernel, in your situation there is nothing suitable to be swapped in RAM (for one reason or another).

Last edited by Wild Penguin (2019-11-08 11:29:00)

Offline

#3 2019-11-08 11:26:38

xse
Member
Registered: 2019-01-08
Posts: 36

Re: Problem with non-working swap partition

Hi,

The kernel will prefer RAM over swap, as you've seen you can influence that with vm.swappiness

The default value of vm.swappiness is 60 and represents the percentage of the free memory before activating swap. The lower the value, the less swapping is used and the more memory pages are kept in physical memory.

Nowadays most systems don't really use/need swap, i mean some do for hibernating and stuff, but i don't even create swap partitions anymore on servers and stuff.

To me the question is more "wtf is in /tmp for it to take that much space ?"

So, what's using all that space in /tmp ?
systemd ? I really doubt logs are taking all that space, if they do that's an easy fix.

If you are building in tmp a lot you should also be able to clean, so i don't get what's taking that much space.
I have in front of me a system with 16G ram and an uptime of 18 days that has nothing in /tmp.


Carefully explaining your problem is half the solution.

Offline

#4 2019-11-08 12:48:03

seth
Member
Registered: 2012-09-03
Posts: 49,991

Re: Problem with non-working swap partition

The default value of vm.swappiness is 60 and represents the percentage of the free memory before activating swap. The lower the value, the less swapping is used and the more memory pages are kept in physical memory.

While it's a popular misconception, it's still utter bullshit.
The value essentially controls whether anon pages are swapped out or file caches are dropped - when you're running OOM!
The system will not just ignore your vast amounts of free RAM and use slow swap space to safe the RAM "for later" or so.

I'll just link a former discussion… https://bbs.archlinux.org/viewtopic.php?id=231265

Online

#5 2019-11-08 13:00:34

CarbonChauvinist
Member
Registered: 2012-06-16
Posts: 412
Website

Re: Problem with non-working swap partition

This link is a really good explanation of swap and works well to dispell some common misconceptions.


"the wind-blown way, wanna win? don't play"

Offline

#6 2019-11-08 16:36:10

clupus
Member
From: Saarbrücken
Registered: 2017-01-31
Posts: 22

Re: Problem with non-working swap partition

Just to make things clear: For me, I consider /tmp the scratch storage. If I need to do some quick work on a file, I do not want to store permanently, I put it there. Or I install a new VM. Then the ISO goes there. Just examples. All in all it will not ly there for days more minutes and hours. But it needs to be saved somewhere. So it is temporary data but from my work with the machine instead of the machine's own behaviour.

Regarding the OOM reaction: I assume the kernel to drop both caches (can be reread if need arises from SSD) and store free pages from RAM (especially those from the files in /tmp) to swap in order to get more "air" on /tmp to breathe.

I skimmed the discussion, seth mentioned but I was able to understand 100% of the detailed technical discussion from it. Maybe I should work my way through it once more. But just from understanding, how the kernel decides what pages to put to swap, I do not see a way to avoid the freeze in my system. Sorry I don't want to sound ignorant but I do not get the point. No offence!

Offline

#7 2019-11-08 16:38:09

clupus
Member
From: Saarbrücken
Registered: 2017-01-31
Posts: 22

Re: Problem with non-working swap partition

PS: One other thing I forgot (and I remember during my searches on the net): The swap partition is a LV that is encrypted using LUKS. Within the LUKS container the swap formatted storage is located. I found someone mentioning a bug but I did not find any more information that helps me to understand the bug's nature and how this might affect me (or not).

Offline

#8 2019-11-08 17:17:42

seth
Member
Registered: 2012-09-03
Posts: 49,991

Re: Problem with non-working swap partition

I previously wasn't addressing you or your issue, just debunking that statement (which gets repeated a lot and still remains false)

The swap partition is a LV that is encrypted using LUKS

Yeah, that's the problem. More the lvm than the encryption, it seems.
https://bbs.archlinux.org/viewtopic.php … 5#p1827575

Online

#9 2019-11-08 21:32:06

Wild Penguin
Member
Registered: 2015-03-19
Posts: 319

Re: Problem with non-working swap partition

FWIW I always create folders labeled "DELETE" (or something similar / obvious) for the stuff I know I won't probably need later. I got ample HD space so if I forget them that is not an issue. And, after all, you never know when you might need some ISO (etc.) you downloaded again, when you thought it is a one-shot need. Then sometimes (umm., which is quite seldom, actually) the file cleaner person inside me activates and I look out for these folders and remove them.

However hope you got it working. I know it is possible (tmpfs should be put into swap) since I've had that kind of setup (for huge compiles). Maybe there indeed is an LV related bug...

Last edited by Wild Penguin (2019-11-09 09:18:45)

Offline

Board footer

Powered by FluxBB