Arch Linux

ondoho

Member

Registered: 2013-04-30

Posts: 692

Website

[SOLVED] high RAM usage - tmpfs has 0 bytes available - swap unused

I am currently transcoding some video to x265 codec.
Ffmpeg has been chugging away on all 4 cores for hours now - that is as expected.

Physical memory usage is at 70-80% (~5.2G).
The system is mostly usable, I can watch videos, browse the web and the filesystem etc., even do some light gaming.

The problem is that several commands fail because they cannot create even the smallest file in /tmp.
One example is makepkg:

/usr/share/makepkg/util/parseopts.sh: line 119: cannot create temp file for here-document: No space left on device

Thunar reports /tmp having 0 bytes free space although 'du' tells me it only contains:

du -h --summarize /tmp
200K /tmp

Nevertheless, 'df' claims that it is 100% full:

df | grep -w tmp
tmpfs 3538528 3538528 0 100% /tmp

That's 100% of ~3.5G. Weird.

I also have a 4GB swap partition, only a very small part of which is being used:

free -h
total used free shared buff/cache available
Mem: 6.7Gi 1.6Gi 787Mi 3.6Gi 4.3Gi 1.1Gi
Swap: 4.0Gi 42Mi 4.0Gi

I am guessing that ffmpeg's high memory usage prevents /tmp from utilizing physical memory, correct?

Can I configure /tmp to use swap space if there isn't enough physical memory?

This is a very new install following standard procedures.
/etc/fstab does not contain a swap entry, it's automatically configured & mounted by systemd.

Last edited by ondoho (2020-05-08 04:37:37)

Lone_Wolf

Administrator

From: Netherlands, Europe

Registered: 2005-10-04

Posts: 15,392

Re: [SOLVED] high RAM usage - tmpfs has 0 bytes available - swap unused

You could try overrriding the max size used for tmpfs , https://wiki.archlinux.org/index.php/Tmpfs

A better solution may be to stop using tmp to build something.

If your hdd is slow, try adding a sata ssd.
They have dropped so fast in price that you now can get a small one in 2.5 / 3.5 " casing (making them fit in standard hdd bays ) for less than 50 euros.

You may also be interested in https://wiki.archlinux.org/index.php/Im … M_handling

Last edited by Lone_Wolf (2020-05-03 11:52:42)

---

Disliking systemd intensely, but not satisfied with alternatives so focusing on taming systemd.

clean chroot building not flexible enough ?
Try clean chroot manager by graysky

WorMzy

Administrator

From: Scotland

Registered: 2010-06-16

Posts: 13,644

Website

Re: [SOLVED] high RAM usage - tmpfs has 0 bytes available - swap unused

Swap isn't being used (much) because your actual memory isn't exhausted yet. Tmpfs will overflow into swapspace if it needs to. Make your /tmp bigger.

---

Sakura:-
Mobo: MSI MAG X570S TORPEDO MAX // Processor: AMD Ryzen 9 5950X @4.9GHz // GFX: AMD Radeon RX 5700 XT // RAM: 32GB (4x 8GB) Corsair DDR4 (@ 3000MHz) // Storage: 1x 3TB HDD, 6x 1TB SSD, 2x 120GB SSD, 1x 275GB M2 SSD

Making lemonade from lemons since 2015.

ondoho

Member

Registered: 2013-04-30

Posts: 692

Website

Re: [SOLVED] high RAM usage - tmpfs has 0 bytes available - swap unused

A better solution may be to stop using tmp to build something...If your hdd is slow, try adding a sata ssd.

This is not the issue here.
Slowness is not the issue either.

The makepkg-related error quoted above is actually a bash error.

Neither ffmpeg nor makepkg are using /tmp.
/tmp uses only 200k but appears to be 100% full.
I can create an empty file there, but not write anything into it.

I think that this has something to do with ffmpeg using most of my physical memory.

Tmpfs will overflow into swapspace if it needs to.

I think it needs to - but it doesn't!

Make your /tmp bigger.

I will try after ffmpeg finished, but I don't see how that can help: /tmp appears to be 3.4G, uses only 200k but reports as being 100% full.
Also, if I make it close to or even largerthan physical RAM, won't it fill up all of my physical memory before swapping out? That can't be good.

ondoho

Member

Registered: 2013-04-30

Posts: 692

Website

Re: [SOLVED] high RAM usage - tmpfs has 0 bytes available - swap unused

After ffmpeg finished transcoding I still couldn't write to /tmp.

I rebooted without making /tmp bigger and now everything is OK again; ffmpeg is again happily transcoding (but it's using way less memory now, possibly because of different encoder settings), /tmp is writeable again, bash can create here-documents.

But I remembered something else: before I started transcoding a broken script had filled up /tmp with a log file. I removed that log file, then didn't pay any attention to /tmp until I started getting weird errors.
Maybe it wasn't ffmpeg after all then.

So the scenario could be: some file fills up /tmp, but even after that file is deleted the underlying tmpfs sees itself as 100% full?
Some coincidence, ffmpeg using 80% of RAM after that.

sabroad

Member

Registered: 2015-05-24

Posts: 242

Re: [SOLVED] high RAM usage - tmpfs has 0 bytes available - swap unused

So the scenario could be: some file fills up /tmp, but even after that file is deleted the underlying tmpfs sees itself as 100% full?

From Why is space not being freed from disk after deleting a file?

deleting a file [...] will unlink the file from the file system's directory structure; however, if the file is still open (in use by a running process) it will still be accessible to this process and will continue to occupy space on disk.

---

--
saint_abroad

ondoho

Member

Registered: 2013-04-30

Posts: 692

Website

Re: [SOLVED] high RAM usage - tmpfs has 0 bytes available - swap unused

deleting a file [...] will unlink the file from the file system's directory structure; however, if the file is still open (in use by a running process) it will still be accessible to this process and will continue to occupy space on disk.

That explains it.
Thanks!