lxc container in user namespace

warm · 2014-08-21 02:27:13

I would like to run unprivileged archlinux template container in arch-host system. Usual lxc container runs with no problem (at this moment) but with user namespace does not want to run.

I've recompiled kernel for host system with user namespace enabled, then I've created container with 'lxc-create -n c1 -t archlinux' command, then 'container-userns-convert c1 100000' (bash script from AUR's nsexec-bzr package). After that I get errors when trying to start c1:

[root@localhost]# lxc-start --logpriority=DEBUG --logfile=/tmp/c1.log -n c1
newuidmap: uid range [0-10000) -> [100000-110000) not allowed
lxc-start: failed to set up id mapping
lxc-start: failed to spawn 'c1'
lxc-start: The container failed to start.
lxc-start: Additional information can be obtained by setting the --logfile and --log-priority options.
[root@localhost]#

logfile contains the same error.

I've read that unprivileged container should use special temlate 'download' but it does not work for me and I would like to use arch.

Is it possible to run unprivileged container with arch-template ?

opotonil · 2014-08-30 12:22:40

I am testing with a Debian container.

Config subuid and subgid:

# cat /etc/sub*
username:100000:65536
username:100000:65536

Config container:

# cat /etc/lxc/container.conf | grep -i id_map
lxc.id_map = u 0 100000 65536
lxc.id_map = g 0 100000 65536

Start container as user ("username") and you will get new errors XD:

$ lxc-start -l DEBUG -o lxc.log -n container -f /etc/lxc/container.conf
lxc-start: Permission denied - Could not create cgroup '/container' in '/sys/fs/cgroup/perf_event'.                                                             
lxc-start: Permission denied - cgroup_rmdir: failed to delete /sys/fs/cgroup/perf_event//lxc                                                                 
lxc-start: Read-only file system - cgroup_rmdir: failed to delete /sys/fs/cgroup/perf_event/                                                                 
lxc-start: Permission denied - cgroup_rmdir: failed to delete /sys/fs/cgroup/blkio//lxc                                                                      
lxc-start: Read-only file system - cgroup_rmdir: failed to delete /sys/fs/cgroup/blkio/                                                                      
lxc-start: Permission denied - cgroup_rmdir: failed to delete /sys/fs/cgroup/net_cls//lxc                                                                    
lxc-start: Read-only file system - cgroup_rmdir: failed to delete /sys/fs/cgroup/net_cls/                                                                    
lxc-start: Permission denied - cgroup_rmdir: failed to delete /sys/fs/cgroup/freezer//lxc                                                                    
lxc-start: Read-only file system - cgroup_rmdir: failed to delete /sys/fs/cgroup/freezer/                                                                    
lxc-start: Permission denied - cgroup_rmdir: failed to delete /sys/fs/cgroup/devices/user.slice                                                              
lxc-start: Permission denied - cgroup_rmdir: failed to delete /sys/fs/cgroup/memory//lxc                                                                     
lxc-start: Read-only file system - cgroup_rmdir: failed to delete /sys/fs/cgroup/memory/                                                                     
lxc-start: Permission denied - cgroup_rmdir: failed to delete /sys/fs/cgroup/cpu,cpuacct//lxc                                                                
lxc-start: Read-only file system - cgroup_rmdir: failed to delete /sys/fs/cgroup/cpu,cpuacct/                                                                
lxc-start: Permission denied - cgroup_rmdir: failed to delete /sys/fs/cgroup/cpuset//lxc                                                                     
lxc-start: Read-only file system - cgroup_rmdir: failed to delete /sys/fs/cgroup/cpuset/                                                                     
lxc-start: failed creating cgroups                                                                                                                           
lxc-start: failed to spawn 'container'
lxc-start: The container failed to start.
lxc-start: Additional information can be obtained by setting the --logfile and --log-priority options.

I need time to continue reading this thread:
https://lists.linuxcontainers.org/piper … 08040.html

More specifically from here onwards:
https://lists.linuxcontainers.org/piper … 08055.html

Last edited by opotonil (2014-08-30 12:34:16)

opotonil · 2014-08-31 16:27:30

Using the correct paths (At this moment, in my case, are symbolic links):

/etc/lxc/lxc.conf => ~/.config/lxc/lxc.conf
/etc/lxc/default.conf => ~/.config/lxc/default.conf
/var/lib/lxc => ~/.local/share/lxc
/var/lib/lxcsnaps => ~/.local/share/lxcsnaps
/var/cache/lxc => ~/.cache/lxc

And setting /sys/fs/cgroup/cpuset/cgroup.clone_children

# echo 1 > /sys/fs/cgroup/cpuset/cgroup.clone_children

The result is better but not is good:

$ lxc-start -n container
lxc_container: Permission denied - Could not create cgroup '/container' in '/sys/fs/cgroup/perf_event'.                                                         
lxc_container: Read-only file system - cgroup_rmdir: failed to delete /sys/fs/cgroup/perf_event/                                                             
lxc_container: Read-only file system - cgroup_rmdir: failed to delete /sys/fs/cgroup/blkio/                                                                  
lxc_container: Read-only file system - cgroup_rmdir: failed to delete /sys/fs/cgroup/net_cls/                                                                
lxc_container: Read-only file system - cgroup_rmdir: failed to delete /sys/fs/cgroup/freezer/                                                                
lxc_container: Permission denied - cgroup_rmdir: failed to delete /sys/fs/cgroup/devices/user.slice                                                          
lxc_container: Read-only file system - cgroup_rmdir: failed to delete /sys/fs/cgroup/memory/                                                                 
lxc_container: Read-only file system - cgroup_rmdir: failed to delete /sys/fs/cgroup/cpu,cpuacct/                                                            
lxc_container: Read-only file system - cgroup_rmdir: failed to delete /sys/fs/cgroup/cpuset/                                                                 
lxc_container: failed creating cgroups                                                                                                                       
lxc_container: failed to spawn 'container'
lxc_container: The container failed to start.
lxc_container: Additional information can be obtained by setting the --logfile and --log-priority options.

Last edited by opotonil (2014-08-31 16:28:00)

teekay · 2014-08-31 18:01:18

warm wrote:

[...]then 'container-userns-convert c1 100000'[...]

[root@localhost]# lxc-start --logpriority=DEBUG --logfile=/tmp/c1.log -n c1
newuidmap: uid range [0-10000) -> [100000-110000) not allowed
lxc-start: failed to set up id mapping
[...]

[...]

I didn't look at lxc source code, but I suspect they limit the max uid number to 65534 (16bit) for backwards compatibility with older kernels and/or filesystems. So an offset of 100000 might be supported by your kernel, but not by lxc obviously.
Try a lower offset, like 10000.

My arch containers work fine unprivileged.

@opotonil: yours seems to be a permission problem of the container's root fs.

Last edited by teekay (2014-08-31 18:07:15)

opotonil · 2014-09-01 11:57:51

teekay wrote:

@opotonil: yours seems to be a permission problem of the container's root fs.

Thanks @teekay but I think I have another problem.

I created all directories, without using symbolic links, all owned by my user and with "drwxr-xr-x" including my home. And the error persist:

$ lxc-create -t download -n prueba -- -d debian -r wheezy -a armhf
Setting up the GPG keyring
Downloading the image index
Downloading the rootfs
Downloading the metadata
The image cache is now ready
Unpacking the rootfs

---
You just created a Debian container (release=wheezy, arch=armhf, variant=default)
The default root password is: root

$ lxc-start -n prueba                                                                                                  
lxc-start: Permission denied - Could not create cgroup '/prueba' in '/sys/fs/cgroup/perf_event'.                                                             
lxc-start: Read-only file system - cgroup_rmdir: failed to delete /sys/fs/cgroup/perf_event/                                                                 
lxc-start: Read-only file system - cgroup_rmdir: failed to delete /sys/fs/cgroup/blkio/                                                                      
lxc-start: Read-only file system - cgroup_rmdir: failed to delete /sys/fs/cgroup/net_cls/
lxc-start: Read-only file system - cgroup_rmdir: failed to delete /sys/fs/cgroup/freezer/
lxc-start: Permission denied - cgroup_rmdir: failed to delete /sys/fs/cgroup/devices/user.slice
lxc-start: Read-only file system - cgroup_rmdir: failed to delete /sys/fs/cgroup/memory/
lxc-start: Read-only file system - cgroup_rmdir: failed to delete /sys/fs/cgroup/cpu,cpuacct/
lxc-start: Read-only file system - cgroup_rmdir: failed to delete /sys/fs/cgroup/cpuset/
lxc-start: failed creating cgroups
lxc-start: failed to spawn 'prueba'
lxc-start: The container failed to start.
lxc-start: Additional information can be obtained by setting the --logfile and --log-priority options.

PS: I am using Arch Linux ARM as host (Kernel config: https://github.com/archlinuxarm/PKGBUIL … v7/config).

Last edited by opotonil (2014-09-01 12:07:26)

progandy · 2014-09-01 12:41:36

@warm: If you want to use UID and GID mapping as an unprivileged user, you have to assign some ids to that user with usermod --add-sub-uids and --add-sub-gids.
This blogpost by Stéphane Graber should give a rough introduction: https://www.stgraber.org/2014/01/17/lxc … ontainers/

Last edited by progandy (2014-09-01 12:43:12)

aya · 2014-09-09 05:48:41

Hello opotonil,

I guess the problem is that your unprivileged user does not have her own cgroup within the following controllers.
By default unprivileged user is assigned to the root cgroup and of course any action on behalf of such user on /sys/fs/cgroup/blkio/ will fail, because only root has permissions to make changes there

You can check it like this:

$ cat /proc/self/cgroup 
8:blkio:/
7:net_cls:/
6:freezer:/
5:devices:/user.slice
4:memory:/
3:cpu,cpuacct:/
2:cpuset:/
1:name=systemd:/user.slice/user-1000.slice/session-c2.scope

So the workaround would be to create directories under /sys/fs/cgroup/blkio/ etc. for your user, chown those to your unprivileged user and move there the $$ process of your unprivileged user. Then the permission problems should disappear.

This is what Serge Hallyn suggested here and it worked for me.
https://lists.linuxcontainers.org/piper … 08088.html

But I would strongly think that systemd can do it automatically on boot given the proper configuration, but I still have not found the way.

Maybe teekay could share how he got it working and the output of /proc/self/cgroup for the unprivileged user.

opotonil · 2014-11-16 19:42:18

Thanks @aya, I think that is the problem.

But I would strongly think that systemd can do it automatically on boot given the proper configuration, but I still have not found the way.

I think "systemd" require be patched with: https://code.launchpad.net/~stgraber/ub … ontrollers

aya · 2014-11-21 02:33:09

@opotonil
the latest systemd will not do it for us
https://www.mail-archive.com/systemd-de … 24592.html

Surely you can workaround with a unit file that would create the needed directories under each controller, assign proper user ownership to those and start the CT with this user privileges. I believe this is possible.

Arch Linux

#1 2014-08-21 02:27:13

lxc container in user namespace

#2 2014-08-30 12:22:40

Re: lxc container in user namespace

#3 2014-08-31 16:27:30

Re: lxc container in user namespace

#4 2014-08-31 18:01:18

Re: lxc container in user namespace

#5 2014-09-01 11:57:51

Re: lxc container in user namespace

#6 2014-09-01 12:41:36

Re: lxc container in user namespace

#7 2014-09-09 05:48:41

Re: lxc container in user namespace

#8 2014-11-16 19:42:18

Re: lxc container in user namespace

#9 2014-11-21 02:33:09

Re: lxc container in user namespace

Board footer