Arch Linux

syco

Member

Registered: 2008-11-06

Posts: 20

mariadb not starting from systemd

Good morning,
I've upgraded mariadb from 10.1.14 to 10.1.16 and it won't start anymore.
I've already done everything in the wiki under the Troubleshooting section, including check and repair and repair all tables, but nothing changed.
Here's my situation:
/var/lib/mysql is a symlink to /home/DATA/MYSQL
I've already chowned /var/lib/mysql and /home/DATA/MYSQL to mysql.mysql
use mysql can navigate to the folder and create files

[root@man mysql]# systemctl start mariadb
Job for mariadb.service failed because the control process exited with error code.
See "systemctl status mariadb.service" and "journalctl -xe" for details.
[root@man mysql]# systemctl status mariadb.service
● mariadb.service - MariaDB database server
   Loaded: loaded (/usr/lib/systemd/system/mariadb.service; disabled; vendor preset: disabled)
   Active: failed (Result: exit-code) since Mon 2016-07-25 09:50:54 BST; 4s ago
  Process: 3324 ExecStart=/usr/sbin/mysqld $MYSQLD_OPTS $_WSREP_NEW_CLUSTER $_WSREP_START_POSITION (code=exited, status=1/FAILURE)
  Process: 3237 ExecStartPre=/bin/sh -c VAR=`/usr/bin/galera_recovery`; [ $? -eq 0 ] &&   systemctl set-environment _WSREP_START_POSITION=$VAR || exit 1 (code=exited, status=0/SUCCESS)
  Process: 3228 ExecStartPre=/bin/sh -c systemctl unset-environment _WSREP_START_POSITION (code=exited, status=0/SUCCESS)
 Main PID: 3324 (code=exited, status=1/FAILURE)
   Status: "MariaDB server is down"

Jul 25 09:50:53 man systemd[1]: Starting MariaDB database server...
Jul 25 09:50:54 man mysqld[3324]: 2016-07-25  9:50:54 140241961297984 [Note] /usr/sbin/mysqld (mysqld 10.1.16-MariaDB) starting as process 3324 ...
Jul 25 09:50:54 man mysqld[3324]: 2016-07-25  9:50:54 140241961297984 [Warning] Can't create test file /var/lib/mysql/man.lower-test
Jul 25 09:50:54 man mysqld[3324]: [90B blob data]
Jul 25 09:50:54 man systemd[1]: mariadb.service: Main process exited, code=exited, status=1/FAILURE
Jul 25 09:50:54 man systemd[1]: Failed to start MariaDB database server.
Jul 25 09:50:54 man systemd[1]: mariadb.service: Unit entered failed state.
Jul 25 09:50:54 man systemd[1]: mariadb.service: Failed with result 'exit-code'.

mysqld_safe on the other hand works perfectly.

[root@man mysql]# mysqld_safe --user=mysql
160725 09:52:45 mysqld_safe Logging to '/var/lib/mysql/man.err'.
160725 09:52:45 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql

Any idea what's wrong with it? Thanks everybody

Awebb

Member

Registered: 2010-05-06

Posts: 6,610

Re: mariadb not starting from systemd

Remember, that you can inspect unit journals with journalctl more closely.

adrianx

Member

From: South Africa

Registered: 2010-03-22

Posts: 70

Re: mariadb not starting from systemd

I've noticed the following in /usr/lib/systemd/system/mariadb.service

# Prevent accessing /home, /root and /run/user
ProtectHome=true

Could that be causing problems for you?

P.S. I start mariadb mannually as needed, that's why I do not have a symlink in /etc/systemd/system/multi-user.target.wants/

syco

Member

Registered: 2008-11-06

Posts: 20

Re: mariadb not starting from systemd

Hi,
I think adrianx might be right, my symlink point to /home/... after all.
Instead of changing that setting, however, I've removed the symlink and binded the folder.
It is working fine now.

ioquatix

Member

Registered: 2014-03-30

Posts: 18

Re: mariadb not starting from systemd

Just thought I'd add I also had this problem.. It seems like this should be documented somewhere, or perhaps a warning given when upgrading if the data directory is in /home?

WorMzy

Administrator

From: Scotland

Registered: 2010-06-16

Posts: 12,410

Website

Re: mariadb not starting from systemd

If you're doing something daft like storing your system databases in /home, then you're on your own.

---

Sakura:-
Mobo: MSI MAG X570S TORPEDO MAX // Processor: AMD Ryzen 9 5950X @4.9GHz // GFX: AMD Radeon RX 5700 XT // RAM: 32GB (4x 8GB) Corsair DDR4 (@ 3000MHz) // Storage: 1x 3TB HDD, 6x 1TB SSD, 2x 120GB SSD, 1x 275GB M2 SSD

Making lemonade from lemons since 2015.

ukhippo

Member

From: Non-paged pool

Registered: 2014-02-21

Posts: 366

Re: mariadb not starting from systemd

Don't symlink /var/lib/mysql to /where/ever -- if you want your databases elsewhere specify datadir in my.cnf

ruidc

Member

Registered: 2014-10-16

Posts: 9

Re: mariadb not starting from systemd

what's daft about storing datadir in a subdirectory of /home?

mrunion

Member

From: Jonesborough, TN

Registered: 2007-01-26

Posts: 1,938

Website

Re: mariadb not starting from systemd

Why do it? I must admit I don't understand why you would put your data in "home". That's supposed to be your personal stuff that nother else has access to. Why put data that is accessed from another system user (mysql) in your home folder?

---

Matt

"It is very difficult to educate the educated."

Alad

Wiki Admin/IRC Op

From: Bagelstan

Registered: 2014-05-04

Posts: 2,418

Website

Re: mariadb not starting from systemd

All I can think of is that someone miscalculated the size of their /var partition.

---

Mods are just community members who have the occasionally necessary option to move threads around and edit posts. -- Trilby

ukhippo

Member

From: Non-paged pool

Registered: 2014-02-21

Posts: 366

Re: mariadb not starting from systemd

I wouldn't say it was daft, but does go against the Filesystem Hierarchy Standard. Software developers need to follow the standard, but as the owner of the system it's your choice to follow or not the standard.

ruidc

Member

Registered: 2014-10-16

Posts: 9

Re: mariadb not starting from systemd

If it's my database for my purposes, why shouldn't it reside in /home/me/data ? I wanted it there as i'd set that up as it's own large drive.
If it was a shared database on a database server, would the convention be for it to reside in /srv ? or default /var/lib

rx

Member

Registered: 2012-06-19

Posts: 6

Re: mariadb not starting from systemd

Hi there.

I have another problem with starting MariaDB. Since version 10.1.16 it refuses to start with the following error:

Aug 17 08:59:54 ... systemd[426]: mariadb.service: Failed at step NAMESPACE spawning /bin/sh: Too many levels of symbolic links
Aug 17 08:59:54 ... systemd[1]: mariadb.service: Control process exited, code=exited status=226
Aug 17 08:59:54 ... systemd[1]: Failed to start MariaDB database server.
Aug 17 08:59:54 ... systemd[1]: mariadb.service: Unit entered failed state.
Aug 17 08:59:54 ... systemd[1]: mariadb.service: Failed with result 'exit-code'.

For some reason it just can't run ExecStartPre= command sequences:

[Service]
...
ExecStartPre=/bin/sh -c "systemctl unset-environment _WSREP_START_POSITION"
ExecStartPre=/bin/sh -c "VAR=`/usr/bin/galera_recovery`; [ $? -eq 0 ] && systemctl set-environment _WSREP_START_POSITION=$VAR || exit 1"
...

Update: Solved with the following service drop-in:

[Service]
ProtectHome=read-only

It looks like systemd cannot stand when directories /home, /root or /run/user are symbolic links, and ProtectHome is set to true.

Last edited by rx (2016-08-23 12:39:35)