You are not logged in.

#1 2018-11-12 13:22:47

Harey
Member
From: Bavaria, Germany
Registered: 2007-03-24
Posts: 359

docker container fails on kernel 4.19.1

There seems to be a kernel problem with docker and 4.19.1

I have 2 docker container I want to test (postgres:latest and mayan-edms:latest )

The process involves the use of 'apt install' inside the container to install some additional tesseract language data files for the german language. This is accomplished by the docker run command:

docker run -d --name mayan-edms --restart=always -p 81:8000 \
-e MAYAN_DATABASE_ENGINE=django.db.backends.postgresql \
-e MAYAN_DATABASE_HOST=172.17.0.1 \
-e MAYAN_DATABASE_NAME=mayan \
-e MAYAN_DATABASE_PASSWORD=mysupersecret \
-e MAYAN_DATABASE_USER=mayan \
-e MAYAN_DATABASE_CONN_MAX_AGE=60 \
-e MAYAN_APT_INSTALLS="tesseract-ocr-deu tesseract-ocr-deu-frak" \
-v /home/docker-volumes/mayan-edms/media:/var/lib/mayan \
-v /srv/scanned-for-EDMS:/srv/watch_folder \
mayanedms/mayanedms:latest

While the process works on the LTS kernel 4.14 and the current 4.18.16 in core it fails with kernel 4.19.1 from testing.

I have already posted on the mayan-edms forum but I think that this is not a problem of the container but of the kernel konfiguration.

The use of vsyscall=emulate as a kernel parameter is not needed to accomplish the installation on 4.14-lts and core/4.18.16 but setting it does not help either on kernel 4.19.1

The conainer shell is bailing out like this:

root@7cfe63f9d46d:/opt/mayan-edms# apt install tesseract-ocr-deu
E: dpkg was interrupted, you must manually run 'dpkg --configure -a' to correct the problem. 
root@7cfe63f9d46d:/opt/mayan-edms# dpkg --configure -a
dpkg: error: error creating new backup file '/var/lib/dpkg/status-old': Invalid cross-device link

I have read a lot about the use of vsyscall on the different kernels, maybe 4.19.1 introduces an new build parameter that has to be set?

hmm

Greetings
Harvey


Linux is like a wigwam: No Gates, no Windows and an Apache inside

Offline

#2 2018-11-12 21:07:01

willy9
Member
Registered: 2009-04-14
Posts: 56

Re: docker container fails on kernel 4.19.1

I can confirm this on testing... Very basic example:

Dockerfile:

FROM    debian:stretch 
RUN      apt-get -y update && apt-get -y install nano

Result:

sudo docker build .
Sending build context to Docker daemon  2.048kB
Step 1/2 : FROM    debian:stretch
 ---> f2aae6ff5d89
Step 2/2 : RUN     apt-get -y update && apt-get -y install nano
 ---> Running in 64138311bb3e
Ign:1 http://cdn-fastly.deb.debian.org/debian stretch InRelease
Get:3 http://cdn-fastly.deb.debian.org/debian stretch-updates InRelease [91.0 kB]
Get:2 http://security-cdn.debian.org/debian-security stretch/updates InRelease [94.3 kB]
Get:4 http://cdn-fastly.deb.debian.org/debian stretch Release [118 kB]
Get:5 http://security-cdn.debian.org/debian-security stretch/updates/main amd64 Packages [454 kB]
Get:6 http://cdn-fastly.deb.debian.org/debian stretch Release.gpg [2434 B]
Get:7 http://cdn-fastly.deb.debian.org/debian stretch-updates/main amd64 Packages [5152 B]
Get:8 http://cdn-fastly.deb.debian.org/debian stretch/main amd64 Packages [7089 kB]
Fetched 7855 kB in 7s (1086 kB/s)
Reading package lists...
Reading package lists...
Building dependency tree...
Reading state information...
Suggested packages:
  spell
The following NEW packages will be installed:
  nano
0 upgraded, 1 newly installed, 0 to remove and 5 not upgraded.
Need to get 485 kB of archives.
After this operation, 2092 kB of additional disk space will be used.
Get:1 http://cdn-fastly.deb.debian.org/debian stretch/main amd64 nano amd64 2.7.4-1 [485 kB]
debconf: delaying package configuration, since apt-utils is not installed
Fetched 485 kB in 0s (622 kB/s)
Selecting previously unselected package nano.
(Reading database ... 6498 files and directories currently installed.)
Preparing to unpack .../nano_2.7.4-1_amd64.deb ...
Unpacking nano (2.7.4-1) ...
dpkg: error: error creating new backup file '/var/lib/dpkg/status-old': Invalid cross-device link
E: Sub-process /usr/bin/dpkg returned an error code (2)
The command '/bin/sh -c apt-get -y update && apt-get -y install nano' returned a non-zero code: 100

Would appreciate any hint smile


Skrooge, a personal finances manager for the Plasma Desktop
http://skrooge.org

Offline

#3 2018-11-12 22:54:42

bresilla
Member
Registered: 2018-11-12
Posts: 1

Re: docker container fails on kernel 4.19.1

I can confirm that too.
I was getting this error while building a Ubuntu docker image on Kernel 4.19.1-1 (Manjaro)

 dpkg: error: error creating new backup file '/var/lib/dpkg/status-old': Invalid cross-device link 

I just rolled back to Kernel 4.18.17-1 and now Docker works fine.

Offline

#4 2018-11-13 02:43:44

jbenoist
Member
Registered: 2018-11-13
Posts: 1

Re: docker container fails on kernel 4.19.1

Confirming the issue, suspecting this is coming from this change:

commit a725356b6659469d182d662f22d770d83d3bc7b5
Author: Amir Goldstein <amir73il@gmail.com>
Date:   Tue Sep 18 16:34:34 2018 +0300

    vfs: swap names of {do,vfs}_clone_file_range()

    Commit 031a072a0b8a ("vfs: call vfs_clone_file_range() under freeze
    protection") created a wrapper do_clone_file_range() around
    vfs_clone_file_range() moving the freeze protection to former, so
    overlayfs could call the latter.

    The more common vfs practice is to call do_xxx helpers from vfs_xxx
    helpers, where freeze protecction is taken in the vfs_xxx helper, so
    this anomality could be a source of confusion.

    It seems that commit 8ede205541ff ("ovl: add reflink/copyfile/dedup
    support") may have fallen a victim to this confusion -
    ovl_clone_file_range() calls the vfs_clone_file_range() helper in the
    hope of getting freeze protection on upper fs, but in fact results in
    overlayfs allowing to bypass upper fs freeze protection.

    Swap the names of the two helpers to conform to common vfs practice
    and call the correct helpers from overlayfs and nfsd.

Offline

#5 2018-11-13 10:14:10

Harey
Member
From: Bavaria, Germany
Registered: 2007-03-24
Posts: 359

Re: docker container fails on kernel 4.19.1

There already seems to be a bug report on docker github


Linux is like a wigwam: No Gates, no Windows and an Apache inside

Offline

#6 2018-11-14 10:37:39

Harey
Member
From: Bavaria, Germany
Registered: 2007-03-24
Posts: 359

Re: docker container fails on kernel 4.19.1

Still valid for 4.19.2


Linux is like a wigwam: No Gates, no Windows and an Apache inside

Offline

#7 2018-11-14 16:24:06

Netizen29
Member
Registered: 2018-11-14
Posts: 1

Re: docker container fails on kernel 4.19.1

The problem appears only, afaik, if you use overlay2 as storage driver, which is the default for modern docker installations.
If you need to build images locally, as a workaround, you can change the storage driver to "devicemapper", which is not recommended in production. Further information: Docker Archlinux documentation.

Offline

#8 2018-11-14 20:33:10

loqs
Member
Registered: 2014-03-06
Posts: 17,195

Re: docker container fails on kernel 4.19.1

Is the issue present in 4.20-rc2 ?

Offline

#9 2018-11-14 21:45:29

filotek
Member
Registered: 2018-08-15
Posts: 19

Re: docker container fails on kernel 4.19.1

Ran into this issue today after upgrading kernel to 4.19.1.  Besides the change that Netizen29 suggested, I found a mailing list where Ted Ts'o ran into the same issue...

Someone had suggested that...

echo N | sudo tee /sys/module/overlay/parameters/metacopy

was sufficient to stop the error from happening (https://www.spinics.net/lists/linux-uni … 06313.html)

My understanding is that the /metacopy parameter allows only file attribute info to be copied in an overlayfs when its metadata changes (instead of copying the whole file).

Turning this off might not be ideal, but for my use case it's good enough for the short term until a proper fix can be applied.

/EDIT: grammar

Last edited by filotek (2018-11-14 21:46:08)

Offline

#10 2018-11-14 22:06:05

progandy
Member
Registered: 2012-05-17
Posts: 5,184

Re: docker container fails on kernel 4.19.1

In the default kernel configuration it is disabled according to https://www.spinics.net/lists/linux-uni … 06319.html

The arch kernel changed that in 4.19.arch1-1


| alias CUTF='LANG=en_XX.UTF-8@POSIX ' |

Online

#11 2018-11-15 13:54:50

Harey
Member
From: Bavaria, Germany
Registered: 2007-03-24
Posts: 359

Re: docker container fails on kernel 4.19.1

@Netizen29:
But it worked with overlay2 in the 4.18x series... The error has been introduced with the change to 4.19.x.


Linux is like a wigwam: No Gates, no Windows and an Apache inside

Offline

#12 2018-11-15 15:36:45

jetm
Member
From: Costa Rica
Registered: 2010-12-19
Posts: 34

Re: docker container fails on kernel 4.19.1

There is a fix for the kernel: https://www.spinics.net/lists/linux-uni … 06316.html. Hope it's merged in 4.19.3.

Offline

#13 2018-11-16 08:03:50

yohanaizraeli
Member
Registered: 2016-09-01
Posts: 3

Re: docker container fails on kernel 4.19.1

I also ran into this problem yesterday. I just downgraded the kernel to linux-4.18.16 as explained in the docs here: https://wiki.archlinux.org/index.php/do … the_kernel

I'll stick to this kernel version while I wait for a fix for sometime.

Offline

#14 2018-11-16 12:27:40

epinephrine
Member
From: Frankfurt
Registered: 2012-10-18
Posts: 92

Re: docker container fails on kernel 4.19.1

filotek wrote:

Someone had suggested that...

echo N | sudo tee /sys/module/overlay/parameters/metacopy

was sufficient to stop the error from happening (https://www.spinics.net/lists/linux-uni … 06313.html)

I can confirm that this fixes the issue for me! (4.19.1)

Offline

#15 2018-11-16 15:33:21

modnoob
Member
Registered: 2017-10-17
Posts: 28

Re: docker container fails on kernel 4.19.1

Same Problem for me:
FROM debian:stretch
RUN apt-get update && apt-get install -y postfix dovecot-mysql

Anyone reported it on Arch-Bugtracker?

Offline

#16 2018-11-17 12:21:29

jocke-l
Member
Registered: 2018-11-17
Posts: 1

Re: docker container fails on kernel 4.19.1

epinephrine wrote:
filotek wrote:

Someone had suggested that...

echo N | sudo tee /sys/module/overlay/parameters/metacopy

was sufficient to stop the error from happening (https://www.spinics.net/lists/linux-uni … 06313.html)

I can confirm that this fixes the issue for me! (4.19.1)

This works for me as well with 4.19.1-arch1-1-ARCH

Offline

#17 2018-11-18 19:39:29

wopfel
Member
Registered: 2013-11-13
Posts: 16

Re: docker container fails on kernel 4.19.1

jocke-l wrote:
epinephrine wrote:
filotek wrote:

Someone had suggested that...

echo N | sudo tee /sys/module/overlay/parameters/metacopy

was sufficient to stop the error from happening (https://www.spinics.net/lists/linux-uni … 06313.html)

I can confirm that this fixes the issue for me! (4.19.1)

This works for me as well with 4.19.1-arch1-1-ARCH

Same here, works for me in 4.19.1. Thank you for this hint, filotek.

Offline

#18 2018-12-12 12:04:58

bodolsog
Member
Registered: 2016-03-28
Posts: 2

Re: docker container fails on kernel 4.19.1

echo N | sudo tee /sys/module/overlay/parameters/metacopy
wopfel wrote:
jocke-l wrote:
epinephrine wrote:

I can confirm that this fixes the issue for me! (4.19.1)

This works for me as well with 4.19.1-arch1-1-ARCH

Same here, works for me in 4.19.1. Thank you for this hint, filotek.

Just want to confirm - solution works also for: 4.19.6.

Last edited by bodolsog (2018-12-12 12:05:47)

Offline

#19 2018-12-18 15:45:38

loqs
Member
Registered: 2014-03-06
Posts: 17,195

Re: docker container fails on kernel 4.19.1

Offline

Board footer

Powered by FluxBB