You are not logged in.

#1 2019-01-23 00:25:55

jernst
Member
From: Silicon Valley
Registered: 2014-03-04
Posts: 296
Website

mysql_install_db fails from within .service file

This used to work (until maybe a few weeks ago?)

mariadb.service ships with this line commented out:

# Needed to create system tables etc.
# ExecStartPre=/usr/bin/mysql_install_db -u mysql

I used to uncomment those (and add the flags to make it work), like this:

ExecStartPre=/usr/bin/mysql_install_db --user=mysql --basedir=/usr --datadir=/var/lib/mysql

and voila, mariadb would create its own bootstrap tables when first started.

But it stopped working. Now, when I execute "systemctl start mariadb", the system starts getting unresponsive, CPU and memory usage go to about 90% each, and the service fails with no files being created in /var/lib/mysql (not even a log file), and journal reporting that mysql_install_db was killed. Unclear by whom or for what.

If I execute the same command manually, it works just fine. In both cases, it's executed as root (PermissionsStartOnly=true).

I figured maybe there are some permissions or capabilities that aren't available to the process when run from the .service file, as it is tightened down somewhat, but I systematically uncommented things and the behavior remained.

I came across this problem first when booting a container with systemd-nspawn whose file system was created a few weeks ago, and I'm quite certain it worked at the time. Nothing has changed in this container (I run it with -x, so it should be just like it was originally created). What has changed is that I have upgraded the Arch host system. The behavior is now the same, however, whether I run mariadb.service it in the container or on the host system. This makes me suspect that it might come from a systemd upgrade, or such, although that sounds a bit unlikely. Maybe systemd tightened some default limits? But then I should not be seeing 90% resource usage?

How would I even debug this? The journal doesn't have anything useful that I can find other than that the process was killed.

Offline

#2 2019-01-23 00:47:12

loqs
Member
Registered: 2014-03-06
Posts: 18,180

Re: mysql_install_db fails from within .service file

Possibly related https://bugs.archlinux.org/task/61433 you could test by building systemd with -Dbump-proc-sys-fs-file-max=no and -Dbump-proc-sys-fs-nr-open=no.

Offline

#3 2019-01-23 01:01:01

jernst
Member
From: Silicon Valley
Registered: 2014-03-04
Posts: 296
Website

Re: mysql_install_db fails from within .service file

Mine is not a galera recovery problem, but the bug you found is just as ugly, and smells similar. Chances are they have the same cause.

Thanks @loqs for finding this. Note to self: check bug reports first next time.

Offline

Board footer

Powered by FluxBB