Arch Linux

nomme

Member

Registered: 2016-04-20

Posts: 7

[Solved] systemd-nspawn: prevent guest from mounting /tmp as tmpfs

Hi

Background:
I have a host running Arch with a container running Ubuntu. One difference between these two distros is that Arch mounts /tmp with tmpfs where as Ubuntu uses whatever is mounted on / for /tmp. Systemd-nspawn, I guess, clones some of the mount setup of the host i.e. /tmp is mounted with tmpfs in the Ubuntu container. This creates some problems as I use the container to compile a project which puts some large files in /tmp. The files are large enough to exhaust the 8GB of space in /tmp.

Problem:
I cannot find a way of specifying that I don't want /tmp mounted with tmpfs in the Ubuntu container in systemd-nspawn. Is there a way? Can i specify it in Ubuntu somehow?
I have tried to manually unmount /tmp in the container and one resort could be to automate that unmount at boot. I'm out of ideas how to do that though.
A final resort, which I would like to avoid, is to extend the tmpfs size. I need at least 12GB of free space.

Any suggestions for how to do this?

Thanks

Last edited by nomme (2017-05-23 06:42:25)

nomme

Member

Registered: 2016-04-20

Posts: 7

Re: [Solved] systemd-nspawn: prevent guest from mounting /tmp as tmpfs

It seems I made some incorrect assumptions on what systemd-nspawn does.
The Ubuntu machine runs systemd as well and thus seems to mount /tmp with tmp.mount. However, it is still mounted at boot after masking tmp.mount:

● tmp.mount
   Loaded: masked (/dev/null; bad)
   Active: active (mounted) since Mon 2017-05-22 10:36:56 CEST; 4min 52s ago
    Where: /tmp
     What: tmpfs

Stopping tmp.mount unmounts /tmp though. So I'll keep looking in that area.

Last edited by nomme (2017-05-22 09:20:48)

tom.ty89

Member

Registered: 2012-11-15

Posts: 897

Re: [Solved] systemd-nspawn: prevent guest from mounting /tmp as tmpfs

https://github.com/systemd/systemd/blob/v219/NEWS#L256

https://github.com/systemd/systemd/blob … awn.c#L647
https://github.com/systemd/systemd/blob … awn.c#L810
https://github.com/systemd/systemd/blob … unt.c#L373

Not sure if can be avoided by any means.

Last edited by tom.ty89 (2017-05-22 10:58:23)

nomme

Member

Registered: 2016-04-20

Posts: 7

Re: [Solved] systemd-nspawn: prevent guest from mounting /tmp as tmpfs

Sorry, should have mentioned that fstab i empty in the container.
Adding this made no change:

tmpfs /tmp tmpfs noauto 0 0

Also:

ubuntedx # systemctl cat tmp.mount
# /dev/null 

tom.ty89

Member

Registered: 2012-11-15

Posts: 897

Re: [Solved] systemd-nspawn: prevent guest from mounting /tmp as tmpfs

Sorry, should have mentioned that fstab i empty in the container.
Adding this made no change:

tmpfs /tmp tmpfs noauto 0 0

Also:

ubuntedx # systemctl cat tmp.mount
# /dev/null 

Never mind. See my edited post.

nomme

Member

Registered: 2016-04-20

Posts: 7

Re: [Solved] systemd-nspawn: prevent guest from mounting /tmp as tmpfs

Wow, that's some nice digging right there.
Disappointing at the same time though... Why would nspawn take it upon itself to force this on the guest.

Thanks a lot Tom!

tom.ty89

Member

Registered: 2012-11-15

Posts: 897

Re: [Solved] systemd-nspawn: prevent guest from mounting /tmp as tmpfs

Well, here is the commit message from poettering, if you are interested

nomme

Member

Registered: 2016-04-20

Posts: 7

Re: [Solved] systemd-nspawn: prevent guest from mounting /tmp as tmpfs

Well, that wouldn't be the first time someone has opinions which clash with Poettering's
I'm sure there are good reasons for nspawn to behave this way.

But since I cannot have it like that I searched for ways of umounting /tmp after boot. I tried different variations in tmp.mount such as  LazyUmount, ForceUmount, StopWhenUnneeded but without success. The idea I had was that systemd in the container could unmount /tmp after nspawn had mounted it.

One solution I found is to unmount /tmp in /etc/rc.local:

 # /etc/rc.local
# Either
systemctl stop tmp.mount
# Or
umount /tmp

I'm not sure which option is better or if using rc.local is a crappy idea to begin with.
But if someone has a better idea or input on why this is not a good idea I'm glad to hear it. Otherwise I can mark this as solved.

Thanks again Tom.

Last edited by nomme (2017-05-23 06:45:34)

Ropid

Member

Registered: 2015-03-09

Posts: 1,069

Re: [Solved] systemd-nspawn: prevent guest from mounting /tmp as tmpfs

About extending the size of /tmp, I did that with a config that adds to the original config:

# /etc/systemd/system/tmp.mount.d/override.conf
[Mount]
Options=mode=1777,strictatime,nosuid,nodev,size=12G

You can see the original file when you do "systemctl cat tmp.mount". That's where I got the rest of that stuff you see on that "Options=..." line. After you create a config like this, you need to run "sudo systemctl daemon-reload", can then check with "systemctl cat tmp.mount" if systemd found your config.

I think you are also supposed to be able to change the size through an entry in /etc/fstab, but I didn't try that.

tom.ty89

Member

Registered: 2012-11-15

Posts: 897

Re: [Solved] systemd-nspawn: prevent guest from mounting /tmp as tmpfs

@nomme, well I don't really know about nspawn (almost zero experience with it), so I don't know if rc.local is the only possible "implementation" (maybe you can do it with a systemd service as well?). But I do think that a manual stop / umount is the only way, because of how nspawn does the mount.

@Ropid you can't do that for nspawn machines, see this pull request I made after I dug for this post.