[SOLVED] How to register swap if it is not auto-enabled

bertulli · 2022-10-21 17:27:42

Hi all!

I need to compile LLVM+Clang, and apparently for a debug release my 12GB of memory are not enough. So I enabled a swapfile (I have an SSD).

Now, to save unnecessary writing to the disk (to reduce the SSD wear), I though I can make swap disabled by default, and if I need that (for instance for this compilations) activate it. Now:

What should I put in the fstab? Specifically, I was told to use the 'noauto' option. But then, I lose the settings generated by 'defaults' (as specified in the wiki), right? So what should I use? A combination of defaults' and 'noauto'?
When I want to enable the swapfile, having booted with the fstab of the previous point, is activating it with
```
sudo swapon /swapfile
```
enough?
It is also said in the wiki:

Compression needs to be disabled as well but the No_COW attribute (previously set with chattr +C) takes care of that.

However, in the previous part of the wiki that is not mentioned. Moreover, issuing

[alessandro@commodoroII ~]$ sudo lsattr /
[sudo] password for alessandro: 
[...]
--------------e------- /swapfile
--------------e------- /home
[...]

shows that 'C' is, effectively, not set, but neither is 'c'. Does this mean the partition is ok? Should I add '+C'? Do you think it's a typo of the wiki?

Thanks!

Last edited by bertulli (2022-10-25 22:28:21)

ugjka · 2022-10-21 17:33:03

try zramswap before anything else

astralc · 2022-10-21 18:31:49

1+2: you don't need to add it to fstab, as you can use 'swapon/swapoff <swapfile>' without it being listed there.
3: are you using CoW filesystem like btrfs?

Slithery · 2022-10-21 19:58:55

bertulli wrote:

Now, to save unnecessary writing to the disk (to reduce the SSD wear)...

What SSD do you have?
Unnecessary wear is usually only an issue on early 1st generation drives. Anything recent will last much longer than spinning rust and therefore isn't worth worrying about...

bertulli · 2022-10-21 21:20:13

Slithery wrote:

What SSD do you have?
Unnecessary wear is usually only an issue on early 1st generation drives. Anything recent will last much longer than spinning rust and therefore isn't worth worrying about
..

I have a 500GB WD Blue (WDC WDS500G2B0A-00SM50), bought last year. According to what you said, it shouldn't be a problem then?

Btw, I think another possibility is to set the swappiness to a very low value (5), and keep the swap on: this way, it is automatically used but only if truly necessary.

ugjka wrote:

try zramswap before anything else

It's a possibility, however, won't this slow down the system considerably? I want to preserve the disk but I can consider compromises in favor of keeping the speed acceptable

astralc wrote:

1+2: you don't need to add it to fstab, as you can use 'swapon/swapoff <swapfile>' without it being listed there.
3: are you using CoW filesystem like btrfs?

1+2:Thanks! 3: no, I'm on ext4.

Trilby · 2022-10-21 21:25:39

Is your issue actually a shortage of RAM, or just of tmpfs? I suspect the latter is far more likely, especially if there's 12GB of physical ram.

If the issue really is just tmpfs size you can (temporarily) mount a disk-based /tmp, or better yet get the build system to use a tmp storage somewhere other than the default /tmp. Or as a bit of a hybrid solution, if all the tmpfs use tends to be under a given directory just mount some physical device space to /tmp/<whatever>/ right before the build (e.g., a bind mount of a directory housed on the SSD rather than in RAM).

Last edited by Trilby (2022-10-21 21:28:25)

bertulli · 2022-10-21 21:46:42

Trilby wrote:

Is your issue actually a shortage of RAM, or just of tmpfs? I suspect the latter is far more likely, especially if there's 12GB of physical ram.

Uhm, I actually don't know, to be honest. Compilation stopped (in the linking phase) with a simple

collect2: fatal error: ld terminated with signal 9 [Killed]

Trilby wrote:

If the issue really is just tmpfs size you can (temporarily) mount a disk-based /tmp, or better yet get the build system to use a tmp storage somewhere other than the default /tmp. Or as a bit of a hybrid solution, if all the tmpfs use tends to be under a given directory just mount some physical device space to /tmp/<whatever>/ right before the build (e.g., a bind mount of a directory housed on the SSD rather than in RAM).

I'm not acquainted with this (and honestly I only partially understood), but I can investigate. In the meantime, thanks!

Trilby · 2022-10-21 22:18:40

Big project builds can, and often do, create huge amounts of temporary files. But I highly doubt any would stress your actual RAM if you have even a fraction of the 12GB available.

Per default on arch (and common on many *nixes) the /tmp filesystem does not exist on disk but uses a chunk of RAM. So the content of /tmp is limited in size to the tmpfs that is created in RAM (generally 50% of the physical RAM, or 6GB in your case). This can quite easily be filled by large project builds due to the above-mentioned temporary files.

You will find many discussions on websites about building LLVM saying it "ate all ram" and that they needed obscene amounts of RAM. But in any case where it's actually been intelligently investigated, the only problem was the /tmp usage outpacing the ram-mounted tmpfs. For example, here's one recent discussion on this noting that 4GB of ram was sufficient as long as /tmp wasnt a ram-mounted tmpfs (while 10G RAM + 47G swap was insufficient as it expanded memory space but not /tmp space). So your 12GB should be plenty, and adding swap space will not likely help at all.

As always, don't start by throwing more resources at a computational challenge. Start by identifying the actual bottleneck / issue.

Last edited by Trilby (2022-10-21 22:24:11)

bertulli · 2022-10-23 16:39:52

Thanks!
I'll investigate your suggestions. I have never interacted with such big codebases, so I just trusted huge RAM usage was natural

Trilby wrote:

You will find many discussions on websites about building LLVM saying it "ate all ram" and that they needed obscene amounts of RAM.

That's true, but the suggestions I found were to use a different linker ('gold' or 'lld'), and switching to just one linking thread (which I haven't managet to do, but that's off topic for this thread).

Anyway, I also partially received an answer to my original question from an acquaintance: listing the swapfile in fstab using 'noauto' is useful only if you are using systemd-swap: otherwise, there's actually no point, since you need to specify the swapfile as argument of 'swapon' anyway. So the answers to my first two questions are:
1. No need to: use 'defaults' to automount, and simply comment the line otherwise
2. Yes

Anyway, I am still in doubt for the third one: do you have any suggestion?

V1del · 2022-10-23 18:13:57

The third one is only relevant if you run a CoW capable filesystem e.g. btrfs

bertulli · 2022-10-25 21:49:45

Thanks, I mark the thread as solved then. Still, do you think the wiki needs editing (given that this is not explained)?

Edit: crap, I can't add "[SOLVED]" to the title, as it exceeds the max length. What is the custom in these cases?

Last edited by bertulli (2022-10-25 21:51:58)

2ManyDogs · 2022-10-25 22:12:53

bertulli wrote:

Edit: crap, I can't add "[SOLVED]" to the title, as it exceeds the max length. What is the custom in these cases?

Shorten the title enough to add [SOLVED]. "How to register swap if it is not auto-enabled" should be short enough to add [SOLVED].

bertulli · 2022-10-25 22:28:46

Done, thanks!

Arch Linux

#1 2022-10-21 17:27:42

[SOLVED] How to register swap if it is not auto-enabled

#2 2022-10-21 17:33:03

Re: [SOLVED] How to register swap if it is not auto-enabled

#3 2022-10-21 18:31:49

Re: [SOLVED] How to register swap if it is not auto-enabled

#4 2022-10-21 19:58:55

Re: [SOLVED] How to register swap if it is not auto-enabled

#5 2022-10-21 21:20:13

Re: [SOLVED] How to register swap if it is not auto-enabled

#6 2022-10-21 21:25:39

Re: [SOLVED] How to register swap if it is not auto-enabled

#7 2022-10-21 21:46:42

Re: [SOLVED] How to register swap if it is not auto-enabled

#8 2022-10-21 22:18:40

Re: [SOLVED] How to register swap if it is not auto-enabled

#9 2022-10-23 16:39:52

Re: [SOLVED] How to register swap if it is not auto-enabled

#10 2022-10-23 18:13:57

Re: [SOLVED] How to register swap if it is not auto-enabled

#11 2022-10-25 21:49:45

Re: [SOLVED] How to register swap if it is not auto-enabled

#12 2022-10-25 22:12:53

Re: [SOLVED] How to register swap if it is not auto-enabled

#13 2022-10-25 22:28:46

Re: [SOLVED] How to register swap if it is not auto-enabled

Board footer