[SOLVED] postgreSQL won't start with systemctl

dwfa · 2022-01-19 18:45:47

Hi,

After upgrading to v13.4 using pacman and following the instructions to upgrade/migrate the DB all went well (i.e. no errors). I am having an issue starting postgreSQL using systemctl. However, I am able to start and stop postgreSQL using pg_ctl. The error I am getting:

I issue ->

systemctl start postgresql

I get this error back

Job for postgresql.service failed because the control process exited with error code.
See "systemctl status postgresql.service" and "journalctl -xeu postgresql.service" for details.
linux: systemctl status postgresql.service
× postgresql.service - PostgreSQL database server
     Loaded: loaded (/usr/lib/systemd/system/postgresql.service; disabled; vendor preset: disabled)
     Active: failed (Result: exit-code) since Wed 2022-01-19 13:25:25 EST; 14s ago
    Process: 10040 ExecStartPre=/usr/bin/postgresql-check-db-dir ${PGROOT}/data (code=exited, status=0/SUCCESS)
    Process: 10042 ExecStart=/usr/bin/postgres -D ${PGROOT}/data (code=exited, status=1/FAILURE)
   Main PID: 10042 (code=exited, status=1/FAILURE)
        CPU: 71ms

Jan 19 13:25:25 ds9 systemd[1]: Starting PostgreSQL database server...
Jan 19 13:25:25 ds9 postgres[10042]: 2022-01-19 13:25:25.707 EST [10042] FATAL:  could not create lock file "postmaster.pid": Read-only file system
Jan 19 13:25:25 ds9 systemd[1]: postgresql.service: Main process exited, code=exited, status=1/FAILURE
Jan 19 13:25:25 ds9 systemd[1]: postgresql.service: Failed with result 'exit-code'.
Jan 19 13:25:25 ds9 systemd[1]: Failed to start PostgreSQL database server.

And this is the output of journalctl

Jan 19 13:25:25 ds9 systemd[1]: Starting PostgreSQL database server...
░░ Subject: A start job for unit postgresql.service has begun execution
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░
░░ A start job for unit postgresql.service has begun execution.
░░
░░ The job identifier is 168373.
Jan 19 13:25:25 ds9 postgres[10042]: 2022-01-19 13:25:25.707 EST [10042] FATAL:  could not create lock file "postmaster.pid": Read-only file system
Jan 19 13:25:25 ds9 systemd[1]: postgresql.service: Main process exited, code=exited, status=1/FAILURE
░░ Subject: Unit process exited
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░
░░ An ExecStart= process belonging to unit postgresql.service has exited.
░░
░░ The process' exit code is 'exited' and its exit status is 1.
Jan 19 13:25:25 ds9 systemd[1]: postgresql.service: Failed with result 'exit-code'.
░░ Subject: Unit failed
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░
░░ The unit postgresql.service has entered the 'failed' state with result 'exit-code'.
Jan 19 13:25:25 ds9 systemd[1]: Failed to start PostgreSQL database server.

What I want to know is what directory is this failing on when using systemctl? How do I find this out? I have verified that the data directory and /var/run/postgresql ore owned by the appropriate users (dba in my case).

Here is my service file.

[Unit]
Description=PostgreSQL database server
After=network.target

[Service]
Type=notify
TimeoutSec=120
User=dba
Group=dba

Environment=PGROOT=/usr/local/data/db/postgreSQL

SyslogIdentifier=postgres
PIDFile=/usr/local/data/db/postgreSQL/data/postmaster.pid
RuntimeDirectory=postgresql
RuntimeDirectoryMode=755

ExecStartPre=/usr/bin/postgresql-check-db-dir ${PGROOT}/data
ExecStart=/usr/bin/postgres -D ${PGROOT}/data
ExecReload=/bin/kill -HUP ${MAINPID}
KillMode=mixed
KillSignal=SIGINT

# Due to PostgreSQL's use of shared memory, OOM killer is often overzealous in
# killing Postgres, so adjust it downward
OOMScoreAdjust=-200

# Additional security-related features
PrivateTmp=true
ProtectHome=true
ProtectSystem=full
NoNewPrivileges=true
ProtectControlGroups=true
ProtectKernelModules=true
ProtectKernelTunables=true
PrivateDevices=true
RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6
RestrictNamespaces=true
RestrictRealtime=true
SystemCallArchitectures=native

[Install]
WantedBy=multi-user.target

Last edited by dwfa (2022-01-20 04:03:14)

Trilby · 2022-01-19 19:58:52

Note the error output is saying the filesystem is read only. What's the output of `mount` (at least for the filesystem containing that data directory).

EDIT: I'd also suspect some of the security settings in the service file might be relevant, particularly ProtectSystem, but I'm not familiar with these and a quick check of `man systemd.service` and `man systemd.unit` didn't include the documentation for this setting. I did find some descriptions from a web search that indicate ProtectSystem=full would make /usr (and everything under it including your specified directory) read only. That looks to be the cause.

EDIT: Apparently this is covered in `man systemd.exec` which includes the following note for this setting:

man systemd.exec wrote:

... If this option is used, ReadWritePaths= may be used to exclude specific directories from being made read-only.

Last edited by Trilby (2022-01-19 21:01:00)

dwfa · 2022-01-20 02:56:57

Thanks for this - I would have nvr thought of investigating the .service file attributes, the older service file (for v12) worked with no issues. I will look into this a bit further, thx again for the help. As for the mount question, the FS is not mounted as read-only. I can start postgreSQL and use it normally when using pg_ctl.

dwfa · 2022-01-20 04:02:24

I read the man page (a view times) and as suggested I added ReadWritePaths= attribute and it worked. Thx a bunch for the quick help ...

Arch Linux

#1 2022-01-19 18:45:47

[SOLVED] postgreSQL won't start with systemctl

#2 2022-01-19 19:58:52

Re: [SOLVED] postgreSQL won't start with systemctl

#3 2022-01-20 02:56:57

Re: [SOLVED] postgreSQL won't start with systemctl

#4 2022-01-20 04:02:24

Re: [SOLVED] postgreSQL won't start with systemctl

Board footer