Arch Linux

Zack7777

Member

From: Germany

Registered: 2012-04-06

Posts: 25

Docker issues after docker-1:23.0.1-1 update

After updating from 1:20.0 to 1:23.0.1-1 a bunch of containers started to experience crashes.
The app inside catches SIGTERM out of the blue.
I thought it to be the problem with a particular container, so I opened a bug on their gh: https://github.com/linuxserver/docker-w … issues/243
But later on I started to notice that other containers are behaving weird and restarting randomly, too. It would seem that the package is broken.

benvosper

Member

Registered: 2019-01-28

Posts: 8

Re: Docker issues after docker-1:23.0.1-1 update

I'm having similar issues with 1:23.0.1, but in my case, the docker service is unable to start at all:

> journalctl -xeu docker.service
Feb 13 10:39:26 beta systemd[1]: docker.service: Failed with result 'exit-code'.
░░ Subject: Unit failed
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░
░░ The unit docker.service has entered the 'failed' state with result 'exit-code'.
Feb 13 10:39:26 beta systemd[1]: Failed to start Docker Application Container Engine.
░░ Subject: A start job for unit docker.service has failed
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░
░░ A start job for unit docker.service has finished with a failure.
░░
░░ The job identifier is 1229 and the job result is failed.

Rolling back to 1:20.10.23 seems to work fine.

Zack7777

Member

From: Germany

Registered: 2012-04-06

Posts: 25

Re: Docker issues after docker-1:23.0.1-1 update

Opened a bug report: https://bugs.archlinux.org/task/77481

programmador

Member

Registered: 2020-04-19

Posts: 12

Re: Docker issues after docker-1:23.0.1-1 update

For me it causes 100% cpu load for rabbitmq containers. They look as started but erl_child_setup process just eats cpu and rabbitmq server actually does not start. Rolling back to 1.20 helped.
Strange thing is that only reboot helps after downgrading.
This DOES NOT HELP:
* systemctl stop docker
* downgrade
* systemctl daemon-reload
* systemctl start docker
Erlang is still broken in docker after performing these actions. Only rebooting after a downgrade helps.

Humandoodlebug

Member

Registered: 2023-02-15

Posts: 1

Re: Docker issues after docker-1:23.0.1-1 update

Same here, on two different devices. Based on this thread, I suspect rabbitmq is to blame (which is running in the container in question).

TantricShades

Member

Registered: 2023-02-15

Posts: 2

Re: Docker issues after docker-1:23.0.1-1 update

Does anyone know where I can find the older version of this package? I have a fresh install of arch and I am fairly certain I don't have the old version in my cache to rollback to. I could be misunderstanding how that works. When I go to github for the source, I can't find an older version.  Any help and or advice would be appreciated, thank you.

Zack7777

Member

From: Germany

Registered: 2012-04-06

Posts: 25

Re: Docker issues after docker-1:23.0.1-1 update

Does anyone know where I can find the older version of this package? I have a fresh install of arch and I am fairly certain I don't have the old version in my cache to rollback to. I could be misunderstanding how that works. When I go to github for the source, I can't find an older version.  Any help and or advice would be appreciated, thank you.

There you go: https://archive.archlinux.org/repos/202 … kg.tar.zst
Bookmark this: https://wiki.archlinux.org/title/Arch_Linux_Archive

TantricShades

Member

Registered: 2023-02-15

Posts: 2

Re: Docker issues after docker-1:23.0.1-1 update

Does anyone know where I can find the older version of this package? I have a fresh install of arch and I am fairly certain I don't have the old version in my cache to rollback to. I could be misunderstanding how that works. When I go to github for the source, I can't find an older version.  Any help and or advice would be appreciated, thank you.

There you go: https://archive.archlinux.org/repos/202 … kg.tar.zst
Bookmark this: https://wiki.archlinux.org/title/Arch_Linux_Archive

GEATLY appreciated, thank you

gileri

Member

Registered: 2018-09-07

Posts: 3

Re: Docker issues after docker-1:23.0.1-1 update

For me it causes 100% cpu load for rabbitmq containers. They look as started but erl_child_setup process just eats cpu and rabbitmq server actually does not start. Rolling back to 1.20 helped.
Erlang is still broken in docker after performing these actions. Only rebooting after a downgrade helps.

Same here. One thing that alleviate the issue is setting lower nofile ulimits when running the rabbitmq container (by default there was 1073741816):

docker run --rm --ulimit nofile=65536:65536 rabbitmq:3.6.6-management

Similar issue here.

Last edited by gileri (2023-02-16 10:11:28)

iMon

Member

From: Germany

Registered: 2013-01-23

Posts: 2

Re: Docker issues after docker-1:23.0.1-1 update

What fixed the issue for me was to add the following configurations:

/etc/systemd/system/docker.service.d/override.conf:

[Service]                                                                                                                     
LimitNOFILE=1048576

/etc/systemd/system/containerd.service.d/override.conf:

[Service]                                                                                                                     
LimitNOFILE=1048576

Now my containers run like before the update to Docker 23.0.

To make it take effect, I had to restart my computer. Restarting both services was not enough.

Last edited by iMon (2023-02-19 09:21:56)

MB4E

Member

Registered: 2023-02-18

Posts: 1

Re: Docker issues after docker-1:23.0.1-1 update

@iMon : Didn't work for me.

iMon

Member

From: Germany

Registered: 2013-01-23

Posts: 2

Re: Docker issues after docker-1:23.0.1-1 update

I had a typo in the path above. It should be "/etc/systemd/system/docker.service.d/override.conf" not "/etc/systemd/system/dockerd.service.d/override.conf".

bennypt

Member

Registered: 2023-02-20

Posts: 1

Re: Docker issues after docker-1:23.0.1-1 update

What fixed the issue for me was to add the following configurations:

/etc/systemd/system/docker.service.d/override.conf:

[Service]                                                                                                                     
LimitNOFILE=1048576

/etc/systemd/system/containerd.service.d/override.conf:

[Service]                                                                                                                     
LimitNOFILE=1048576

Now my containers run like before the update to Docker 23.0.

To make it take effect, I had to restart my computer. Restarting both services was not enough.

@iMon that worked for me

Thank you for sharing.

Zarch

Member

Registered: 2023-03-01

Posts: 1

Re: Docker issues after docker-1:23.0.1-1 update

I had a similar issue after updating to 1:23.0.1. My CPU would spike to 100% and repeatedly freeze my screen after running the Docker containers, so it's not just RabbitMQ. Also, the MySQL container would exit with the message: "mysqld failed while attempting to check config".

I had a typo in the path above. It should be "/etc/systemd/system/docker.service.d/override.conf" not "/etc/systemd/system/dockerd.service.d/override.conf".

Thanks, this solved the problem for me.

rogedelgado

Member

Registered: 2023-03-14

Posts: 1

Re: Docker issues after docker-1:23.0.1-1 update

Hello,

I can confirm that after the upgrade to docker 1:23.0.1-1 I had a problem while building my images. The error was:

library initialization failed - unable to allocate file descriptor table - out of memory

Using the fix that @iMon mention fix the problem also. Thanks iMon

Last edited by rogedelgado (2023-03-14 00:10:54)

marrs

Member

Registered: 2022-12-29

Posts: 4

Re: Docker issues after docker-1:23.0.1-1 update

I had a different issue.  My configuration had been using the deprecated (presumably now removed) devicemapper by default and I had to configure my docker daemon to use a different storage driver instead.  I went with overlay2.

To do this, I updated my /etc/docker/daemon.json config as follows

{
    "storage-driver": "overlay2"
}

More info here: https://wiki.archlinux.org/title/Docker#Storage_driver

Last edited by marrs (2023-03-18 23:54:24)