You are not logged in.

#1 2020-10-10 16:04:09

amo
Member
Registered: 2016-01-05
Posts: 10

Need advise packaging "papermerge" django web app (python3)

Dear arch community

I am trying to package papermerge. The packaging itself works fine, the package installs, but I run into issues when trying to run it:
Papermerge is a python3 app based on the django framework. All dependencies are installed with pacman (or from AUR and a few additional PKGBUILDs I prepared). Before the initial start of the server, papermerge needs to do a few migrations: first using

django-admin makemigrations

(to my understanding, these are migrations in the libraries which papermerge depends on) and

django-admin migrate

(these are migrations in papermerge itself).
The problem is now that the first migration call fails to run successfully because it is run with the papermerge user (created by the package) who has no write access inside the directory containing the migrations (owned by root since it was installed by pacman).
My question is, how do I go about this? I would like to prevent running stuff as root as I think it is not intended to be used that way.

For reference and help: here is a short discussion about the exact problem and here are the files I use to build and install the package (credits to Lucki who initially made these files):
PKGBUILD

# Contributor: Lucki <https://aur.archlinux.org/account/Lucki>
# shellcheck disable=SC2034,2154,2148
# Maintainer: amo <https://aur.archlinux.org/account/amo>

pkgname=papermerge
pkgver=1.4.5
pkgrel=1
pkgdesc="An open source document management system (DMS) primarily designed for archiving and retrieving your digital documents."
arch=('any')
url="https://github.com/ciur/papermerge"
license=('Apache')
backup=("etc/$pkgname.conf.py" "usr/lib/python3.8/site-packages/config/settings/production.py")
install=$pkgname.install
depends=('imagemagick' 'pdftk' 'poppler' 'python-celery' 'python-django' 'python-django-allauth' 'python-django-celery-results' 'python-django-dynamic-preferences'
      'python-django-modelcluster' 'python-django-mptt' 'python-django-polymorphic-tree' 'python-django-rest-framework' 'python-django-rest-knox' 'python-django-taggit'
      'python-imapclient' 'python-lxml' 'python-mgclipboard' 'python-mglib' 'python-psycopg2' 'python-pyaml' 'tesseract')
optdepends=('gunicorn: run papermerge in a production environment with nginx'
          'nginx: run papermerge through gunicorn in a production environment')
source=("$pkgname-$pkgver.src.tar.gz::https://github.com/ciur/$pkgname/archive/v$pkgver.tar.gz"
      "$pkgname.install"
      "$pkgname.sysusers"
      "$pkgname.tmpfiles"
      "$pkgname-gunicorn.service"
      "$pkgname-worker.service"
      "$pkgname.sh")
sha512sums=('e2f2227d9e57ef38efbc70d0a64c99134907a92a84b31d54adbe643bd69bd0f16a65251eddef61e402b20213ad2baf047f570d1609c948cb320d4505dd2ede8c'
          '22a83a9f26c6991ef6710c08157ff24e11f246a45ccac4b88406fb4f4b57d07608f5fa75bc42f28e7db920ce7ff05617cd03f9febc476f8d654e9fa7d6eb9019'
          'eb408d6d6c0072b4883cae9b6aa591935b910585e93aa6fdacbb543dbc4233588678d396c06d0f10b7c74b4d393af19b8be526d094821de23792c29df593d4a5'
          '07726d47deb2438dbeb3f5dc6ebc2218a4d5f25dacb7e0304102deddfb914d4d6d649643e26cb844ac4d02c7f21b58835edd4026ff91a6481fee59f786b90925'
          '58cd8936ccd5700361f01af2c01af5645ef08c2b1045efd28eb4697af60e49230615b0b0a57dcca1f9813b5658d5244bee8f7315fc62aa5c903cf9586e9a8cc8'
          'fcdd49e0b6b93ade84ddafd77111f28bdea75a4f12829ec249d414d46a14399198be7bf29d0ce5166de6dba1d6386eda4ab9b39a1f5376b7c7758c8966a53ce3'
          'b838d54dca56e45f99690ddaae1b058e3f72b314adf9d689869f14bb141a5bbf209df974eea20cb2305b127d5119d0b8d11293494d47350f6974edc6fffc374f')

prepare() {
  {
      echo 'DBDIR = "/var/lib/papermerge/database"';
      echo 'MEDIA_DIR = "/var/lib/papermerge/media"';
      echo 'STATIC_DIR = "/var/lib/papermerge/static"';
      echo 'TASK_QUEUE_DIR = "/var/tmp/papermerge/queue"'
  } >> "$pkgname-$pkgver/$pkgname.conf.py.example"

  # https://papermerge.readthedocs.io/en/latest/setup/server_configurations.html#step-1-install-gunicorn
  {
      echo 'from .base import *  # noqa';
      echo 'DEBUG = False';
      echo "ALLOWED_HOSTS = ['*']"
  } > "$pkgname-$pkgver/config/settings/production.py"

  # The admin package is missing somehow so let's move it plain stupid directly in place
  mkdir -p "$pkgname-$pkgver/build/lib/$pkgname/"
  cp -dpr --no-preserve=ownership "$pkgname-$pkgver/$pkgname/contrib" "$pkgname-$pkgver/build/lib/$pkgname/"
}

build() {
  cd "$pkgname-$pkgver" || exit
  python setup.py build
}

check() {
  cd "$pkgname-$pkgver" || exit
  ./run_tests.sh
}

package() {
  # tmpfiles.d
  install -Dm 644 "$pkgname.tmpfiles" \
      "$pkgdir/usr/lib/tmpfiles.d/$pkgname.conf"

  # sysusers.d
  install -Dm 644 "$pkgname.sysusers" \
      "$pkgdir/usr/lib/sysusers.d/$pkgname.conf"

  # service files
  install -Dm 644 "$pkgname-gunicorn.service" \
      "$pkgdir/usr/lib/systemd/system/$pkgname-gunicorn.service"
  install -Dm 644 "$pkgname-worker.service" \
      "$pkgdir/usr/lib/systemd/system/$pkgname-worker.service"

  # wrapper
  install -Dm 755 "$pkgname.sh" \
      "$pkgdir/usr/bin/$pkgname-manage"

  cd "$pkgname-$pkgver" || exit

  # config
  install -Dm 644 "$pkgname.conf.py.example" \
      "$pkgdir/etc/$pkgname.conf.py"

  export PYTHONHASHSEED=0
  python setup.py install --root="$pkgdir" --optimize=1 --skip-build
}

papermerge.tmpfiles

d /var/lib/papermerge/media - papermerge papermerge -
d /var/lib/papermerge/static - papermerge papermerge -
d /var/lib/papermerge/database - papermerge papermerge -
d /run/papermerge - papermerge papermerge -
d /var/log/papermerge - papermerge papermerge -

papermerge.sh

#!/bin/sh
export DJANGO_SETTINGS_MODULE=config.settings.dev
cd /tmp || exit
exec /usr/bin/django-admin "$@"

papermerge.install

post_install() {
	printf '\n --> Run papermerge with "systemctl start papermerge-gunicorn papermerge-worker". Point your reverse proxy (e.g. nginx) to 127.0.0.1:9001.\n'
	printf '\n --> If this is a new Papermerge installation, please run "sudo -u papermerge papermerge-manage createsuperuser" to create the admin user.\n\n'
}

papermerge.sysusers

u papermerge - "open source document management system" -

papermerge-gunicorn.service

[Unit]
Description=Gunicorn providing papermerge
Documentation=https://papermerge.readthedocs.io/en/latest/setup/server_configurations.html,https://docs.gunicorn.org/en/latest/deploy.html#systemd
After=network.target
Wants=papermerge-worker.service

[Service]
Type=notify
User=papermerge
Group=papermerge
RuntimeDirectory=papermerge
WorkingDirectory=/var/lib/papermerge
Environment="DJANGO_SETTINGS_MODULE=config.settings.production"
ExecStartPre=/usr/bin/papermerge-manage makemigrations
ExecStartPre=/usr/bin/papermerge-manage migrate
ExecStartPre=/usr/bin/papermerge-manage collectstatic --noinput
ExecStart=/usr/bin/gunicorn config.wsgi:application --bind 127.0.0.1:9001 --workers 2 --access-logfile "/var/log/papermerge/gunicorn.access" --error-logfile "/var/log/papermerge/gunicorn.error"
ExecReload=/bin/kill -s HUP $MAINPID
KillMode=mixed
TimeoutStopSec=5
PrivateTmp=true
Restart=on-failure

[Install]
WantedBy=multi-user.target

papermerge-worker.service

[Unit]
Description=Papermerge Worker
Documentation=https://papermerge.readthedocs.io/en/latest/setup/server_configurations.html
After=network.target
PartOf=papermerge-gunicorn.service

[Service]
Type=simple
User=papermerge
Group=papermerge
WorkingDirectory=/tmp
Environment="DJANGO_SETTINGS_MODULE=config.settings.production"
ExecStart=/usr/bin/django-admin worker --pidfile /var/run/papermerge/papermerge-worker.pid
Restart=on-failure

[Install]
WantedBy=multi-user.target

Maybe somebody with experience packaging python applications can help here. Any advise or hint is greatly appreciated!

Offline

#2 2020-10-29 09:02:21

amo
Member
Registered: 2016-01-05
Posts: 10

Re: Need advise packaging "papermerge" django web app (python3)

Unfortunately, no better idea came to my mind in the meantime than just to set the

/usr/lib/python3.8/site-packages/papermerge/core/migrations

directory to be owned by the papermerge user.
Is there anything to be said against it?

If I am not mistaken, nothing other than papermerge would need the content of that dir anyway.
Also: would it be a better "packaging practice" to have this migrations dir owned by the papermerhe user reside somewhere else in the filesystem and be symlinked there?

Offline

#3 Yesterday 00:34:06

eschwartz
Trusted User/Bug Wrangler
Registered: 2014-08-08
Posts: 3,712

Re: Need advise packaging "papermerge" django web app (python3)

The symlink method is better, let the actual files be edited in e.g. /var/lib/papermerge/migrations

This does seem like a stupid limitation though.


Managing AUR repos The Right Way -- aurpublish (now a standalone tool)

Offline

#4 Yesterday 07:59:48

amo
Member
Registered: 2016-01-05
Posts: 10

Re: Need advise packaging "papermerge" django web app (python3)

Thank you a lot for your reply.

How would I go about creating the symlink with the PKGBUILD?
My idea would be to add

d /var/lib/papermerge/migrations - papermerge papermerge -

to papermerge.tmpfiles in order to have the directory created for the package. But where would I put the ln -s statement? To my understanding it would need to be after the creation of the link's parent directory but before the extraction of files from the package into the linked destination. Is that possible at all?

Last edited by amo (Yesterday 08:00:39)

Offline

#5 Yesterday 12:05:58

Lone_Wolf
Member
From: Netherlands, Europe
Registered: 2005-10-04
Posts: 8,497

Re: Need advise packaging "papermerge" django web app (python3)

man ln wrote:

When creating hard links, each TARGET must exist.
       Symbolic links can hold arbitrary text

Just create the symbolic link in the package function.
for an example check package_mesa() in https://github.com/archlinux/svntogit-p … k/PKGBUILD


Multi-init booting with apg Openrc and systemd coexisting
Automounting : not needed, i prefer pmount
Aur helpers : makepkg + my own local repo === rarely need them

Offline

#6 Yesterday 14:21:48

amo
Member
Registered: 2016-01-05
Posts: 10

Re: Need advise packaging "papermerge" django web app (python3)

Unfortunately, it is not that easy:
If I put the ln -s statement into the package() function, the packaging procedure fails at that point with a "file exists" error. I guess the problem is that it tries to extract files to the not-yet-existing target folder of the link.
Now, first thing that comes to mind is: create the target folder beforehand. But this approach would not work because the target folder would then be owned by root, but it explicitely need to be owned by the papermerge user that, at this point, does not yet exist either. This is what I hoped to work around when adding "d /var/lib/papermerge/migrations - papermerge papermerge -" to papermerge.tmpfiles, but these folder are only created later.

What I also tried was to chown the folders in a ExecPre statement of the systemd service, but this approach also fails because the service is to be run as the papermerge user which does not have the privileges to do that.

Any further ideas would be greatly appreciated!

Offline

Board footer

Powered by FluxBB