PKGBUILD for a python script (need reviews)

alfalfa · 2018-01-22 22:35:17

I wrote this PKGBUILD to package a simple python script. It clone the repository from github, and then generate the shortcut script and the desktop file. It works fine but I figured that it would need improvements before submitting it to the AUR, as the method provide no md5sum or version control.

I was wondering if I should rather do something like in that example, and add an archive folder to my repository that would include the pre-packaged file. If it is done this way, should the .BUILDINFO be removed from the archive?

Also, is it bad practice to generate the system shortcut and the desktop file from the PKGBUILD?

Last edited by alfalfa (2018-01-22 22:36:08)

Omar007 · 2018-01-23 00:04:06

Some quick tips.

1) Don't use git yourself directly. Put the repo in the PKGBUILD 'source' array.
2) To create desktop entries, you could use `gendesk` instead. Alternatively, you could include it as a file and just `install -Dm...` it.
3) Do preparative stuff, such as creating desktop files and executables, in the 'prepare' function instead of inside the 'package' function.
4) If this is a package that just tracks the git head instead of a specific version (e.g. a release tarball/tag/commit), suffix the package name with '-git' and add the 'provides' and 'conflicts' array with the package name without said suffix. See also: https://wiki.archlinux.org/index.php/VC … guidelines

Also, the `msg` calls inside a PKGBUILD are quite unusual. Haven't seen those much (if at all), though it's not a problem afaik.

For further reference:
https://wiki.archlinux.org/index.php/PKGBUILD
https://wiki.archlinux.org/index.php/Creating_packages
https://wiki.archlinux.org/index.php/VC … guidelines
https://wiki.archlinux.org/index.php/Ar … g_packages

Last edited by Omar007 (2018-01-23 00:16:53)

alfalfa · 2018-01-23 01:05:54

Thank you. Following your suggestions I came up with this. But how can I set pkgname to qtpad-git without breaking all the installation paths? Should I also add a _pkgname="qtpad" ?

# Maintainer: William Belanger <echo d2lsbGlhbS5iZWxyQGdtYWlsLmNvbQ== | base64 -d>

pkgname=qtpad
#provides=('qtpad')
#conflicts=('qtpad')
pkgdesc="Modern and customizable sticky note application"
url="https://github.com/willbelr/$pkgname"
pkgver=1.0
pkgrel=1
arch=('any')
license=('GPL3')
source=('https://github.com/willbelr/qtpad/archive/master.zip')
depends=('python>=3' 'python-pyqt5' 'qt5-svg' 'python-requests')
md5sums=('3f0d2a64cef18883b2eac5d6835abbcb')

prepare()
{
  echo "#!/bin/bash\npython /usr/share/$pkgname/$pkgname.py \"\$@\"" > $pkgname
}

package()
{
  install -Dm755 $pkgname "$pkgdir/usr/bin/$pkgname"
  cd $pkgname"-master"
  install -Dm755 $pkgname.desktop "$pkgdir/usr/share/applications/$pkgname.desktop"
  install -Dm755 $pkgname.py "$pkgdir/usr/share/$pkgname/$pkgname.py"
  install -Dm644 gui_child.ui "$pkgdir/usr/share/$pkgname/gui_child.ui"
  install -Dm644 gui_profile.ui "$pkgdir/usr/share/$pkgname/gui_profile.ui"
  install -Dm644 gui_preferences.ui "$pkgdir/usr/share/$pkgname/gui_preferences.ui"
  for file in icons/* ; do
      install -Dm644 $file $pkgdir/usr/share/$pkgname/$file
  done
  install -Dm644 README.md "$pkgdir/usr/share/$pkgname/README.md"
  install -Dm644 LICENSE "$pkgdir/usr/share/licenses/$pkgname/LICENSE"
}

Last edited by alfalfa (2018-01-23 01:06:32)

eschwartz · 2018-01-23 01:09:49

Omar007, the msg calls are a "problem" because msg is not part of makepkg's public API, so why rely on it? Also, because it isn't *useful* information. If you want to tell the user what is happening, Makefiles print the commands being run to stdout, and you can have the install command do the same thing via the -v flag.

alfalfa, as you seem to be the upstream developer, I would like to encourage you to

1) use a setup.py to install the modules directly to python's sys.path in /usr/lib/python3.6/site-packages, taking advantage of byte-compiled pyc,
2) use pyuic to generate python files from the ui files, as they load faster than uic.loadUi(), this can be done with setuptools via a custom build command,
3) encapsulating the stuff following "if __name__ == '__main__':" in a global main() function, and then just calling main() -- this is required to use setuptools console_scripts to run qtpad.qtpad:main rather than directly exec'ing qtpad.py

setuptools allows you to automatically install everything where it needs to go, including the desktop file you just added to github. You can use the setuptools "data_files" keyword to install the desktop file to "share/applications/qtpad.desktop".

The final process for installing the entire project, would be as simple as `python setup.py build` and `python setup.py install --root="$pkgdir"`
See the Python package guidelines in our wiki for details.

alfalfa · 2018-01-29 21:27:12

So this is how I've finally done it;

- qtpad.py was renamed to __init__.py and placed into "qtpad" directory
- content under "if __name__ == '__main__'" was moved into main() function, and the variables set as global
- at the top level directory, I created a setup.py installer and I added a custom function to generate the desktop file.
- the setup file also check if PyQt5 is already installed outside of PyPi. It is only added when PyQt5 library is not present on the system, as else pip ignore the package and program refuse to launch after install.

I chose not to add a custom compiling for ui file since it rely on Qt libs, which might not be installed at build time. The ui would likely need to be 'compiled' to python beforehand.

Hopefully, that will do it

PKGBUILD:

# Maintainer: William Belanger <echo d2lsbGlhbS5iZWxyQGdtYWlsLmNvbQ== | base64 -d>

pkgname=qtpad-git
provides=("qtpad")
conflicts=("qtpad")
pkgdesc="Modern and customizable sticky note application"
url="https://github.com/willbelr/qtpad"
pkgver=1.0.0
pkgrel=1
arch=("any")
license=("GPL3")
depends=("python>=3" "python-pyqt5" "qt5-svg" "python-requests")
source=("git+https://github.com/willbelr/qtpad.git")
md5sums=("SKIP")

package()
{
  cd qtpad
  python setup.py install --root="$pkgdir"
}

setup.py:

import os
import setuptools
import setuptools.command.build_py
here = os.path.abspath(os.path.dirname(__file__))


class CreateDesktopFile(setuptools.command.build_py.build_py):
  def run(self):
    path = here + "/qtpad.desktop"
    with open(os.path.join(path), 'w') as f:
        f.write("[Desktop Entry]\n")
        f.write("Name=qtpad\n")
        f.write("Exec=qtpad\n")
        f.write("Terminal=false\n")
        f.write("Type=Application\n")
        f.write("Icon=view-compact-symbolic")
    setuptools.command.build_py.build_py.run(self)


# Workaround in case PyQt5 was installed without pip
install_requires=['requests']
try:
    import PyQt5
except ImportError:
    install_requires.append("pyqt5")

setuptools.setup(
    name='qtpad',
    version='1.0.0',
    description='Modern and customizable sticky note application',
    url='https://github.com/willbelr/qtpad',
    keywords='sticky note text editor note-taking',

    # From https://pypi.python.org/pypi?%3Aaction=list_classifiers
    classifiers=[
        'Development Status :: 4 - Beta',
        'License :: OSI Approved :: GNU General Public License v3 (GPLv3)',
        'Programming Language :: Python :: 3',
        'Programming Language :: Python :: 3.4',
        'Programming Language :: Python :: 3.5',
        'Programming Language :: Python :: 3.6',
    ],
    cmdclass={'build_py': CreateDesktopFile},
    data_files=[('share/applications/', ['qtpad.desktop'])],
    include_package_data=True,
    package_data={'': ['*.ui', 'icons/*.svg']},
    packages=setuptools.find_packages(),
    install_requires=install_requires,
    entry_points={
        'console_scripts': [
            'qtpad=qtpad:main',
        ],
    },
)

Slithery · 2018-01-29 21:34:15

As you are pulling from git master you need a pkgver function...
https://wiki.archlinux.org/index.php/VC … guidelines

eschwartz · 2018-01-29 21:58:01

alfalfa wrote:

So this is how I've finally done it;
- qtpad.py was renamed to __init__.py and placed into "qtpad" directory
- content under "if __name__ == '__main__'" was moved into main() function, and the variables set as global
- at the top level directory, I created a setup.py installer and I added a custom function to generate the desktop file.
- the setup file also check if PyQt5 is already installed outside of PyPi. It is only added when PyQt5 library is not present on the system, as else pip ignore the package and program refuse to launch after install.
I chose not to add a custom compiling for ui file since it rely on Qt libs, which might not be installed at build time. The ui would likely need to be 'compiled' to python beforehand.
Hopefully, that will do it

Thanks for considering my suggestions!

There is no need for a custom function to generate the desktop file, you can just go back to how it was before and keep the file in git. You're not actually doing anything special in setup.py that requires you to change the file...

Regarding pyuic, that is of course a personal choice, I suppose it depends on whether you want to assume that PyQt5.uic.pyuic() is available at the time setup.py is run. This obviously wouldn't be a problem for people using the Arch Linux PKGBUILD...
But you did opt for dynamically adding a pyqt5 dependency for pip users.

alfalfa · 2018-01-31 14:30:00

slithery wrote:

As you are pulling from git master you need a pkgver function...
https://wiki.archlinux.org/index.php/VC … guidelines

My bad. However this is good news as it will make maintenance easier. The examples in the wiki did not work for me so instead I use the commit count along with the hash.

PKGBUILD:

# Maintainer: William Belanger <echo d2lsbGlhbS5iZWxyQGdtYWlsLmNvbQ== | base64 -d>

pkgname=qtpad-git
_pkgname=qtpad
pkgver=0.0.0
pkgrel=1
provides=("$_pkgname")
conflicts=("$_pkgname")
pkgdesc="Modern and customizable sticky note application"
url="https://github.com/willbelr/$_pkgname"
arch=("any")
license=("GPL3")
depends=("python>=3" "python-pyqt5" "qt5-svg" "python-requests")
source=("git+https://github.com/willbelr/$_pkgname.git")
md5sums=("SKIP")

pkgver()
{
  cd "$_pkgname"
  echo $(git rev-list --count HEAD).$(git rev-parse --short HEAD)
}

package()
{
  cd "$_pkgname"
  python setup.py install --root="$pkgdir"
}

Eschwartz wrote:

There is no need for a custom function to generate the desktop file, you can just go back to how it was before and keep the file in git. You're not actually doing anything special in setup.py that requires you to change the file...

If it is allowed, I rather prefer to generate the desktop file dynamically, since it has no use in developpement. Just like for the executable file in /usr/bin, I prefer to avoid having them in my projects folders to reduce the amount of 'garbage'. Also it makes it easier to reuse setup.py, as all I have to do is to replace all 'qtpad' to another project name.

Eschwartz wrote:

Regarding pyuic, that is of course a personal choice, I suppose it depends on whether you want to assume that PyQt5.uic.pyuic() is available at the time setup.py is run. This obviously wouldn't be a problem for people using the Arch Linux PKGBUILD...
But you did opt for dynamically adding a pyqt5 dependency for pip users.

Here is the compromise I went for. If PyQt5 is available, the ui files are compiled during build;

setup.py:

...
# Workaround in case PyQt5 was installed without pip
# Pre-compile ui files if PyQt5 is already installed
install_requires=['requests']
package_data={'': ['icons/*.svg']}
try:
    import PyQt5
    import subprocess
    path = here + "/qtpad/"
    for f in os.listdir(path):
        ext = f.lower().rsplit('.', 1)[-1]
        if ext == "ui":
            cmd = "pyuic5 " + path + f + " -o " + path  + f[:-3] + ".py"
            subprocess.run(cmd.split())
except ImportError:
    install_requires.append("pyqt5")
    package_data[''].append("*.ui")

setuptools.setup(
...

Then I try to import the ui modules within the script:

...
try:
    # Load pre-compiled Ui files if available
    import qtpad.gui_child
    import qtpad.gui_preferences
    import qtpad.gui_profile
except ImportError:
    pass
...

And the modules are loaded from memory if available;

class PreferencesDialog(QtWidgets.QDialog):
    def __init__(self, parent):
        super().__init__()
        if "qtpad.gui_preferences" in sys.modules:
            self.ui = qtpad.gui_preferences.Ui_Dialog()
            self.ui.setupUi(self)
        else:
            self.ui = uic.loadUi(LOCAL_DIR + 'gui_preferences.ui', self)
...

Last edited by alfalfa (2018-01-31 18:54:35)

eschwartz · 2018-01-31 15:37:39

alfalfa wrote:

My bad. However this is good news as it will make maintenance easier. The examples in the wiki did not work for me so instead I use the commit count along with the hash.
PKGBUILD:
pkgver()
{
  cd "$_pkgname"
  echo $(git rev-list --count HEAD).$(git rev-parse --short HEAD)
}

The wiki lists several options, depending on whether the upstream repository contains git tagged releases. The one you wanted was the last one in the list:

printf "r%s.%s" "$(git rev-list --count HEAD)" "$(git rev-parse --short HEAD)"

The advantage of this is that the "r" in the beginning, ensures if you do ever tag a release you can update to using that without having to use an epoch. Say you have 50 commits by the time you tag release 1.0 -- well, r50 is a lower version number than 1.0, but 50 is a higher version number than 1.0

Eschwartz wrote:
There is no need for a custom function to generate the desktop file, you can just go back to how it was before and keep the file in git. You're not actually doing anything special in setup.py that requires you to change the file...
If it is allowed, I rather prefer to generate the desktop file dynamically, since it has no use in developpement. Just like for the executable file in /usr/bin, I prefer to avoid having them in my projects folders to reduce the amount of 'garbage'. Also it makes it easier to reuse setup.py, as all I have to do is to replace all 'qtpad' to another project name.

I guess that would fall under "personal difference of opinions", there is certainly no AUR rule against it.

I will try not to command you on how to develop your project once you've stated that you had specific intentions.

Eschwartz wrote:
Regarding pyuic, that is of course a personal choice, I suppose it depends on whether you want to assume that PyQt5.uic.pyuic() is available at the time setup.py is run. This obviously wouldn't be a problem for people using the Arch Linux PKGBUILD...
But you did opt for dynamically adding a pyqt5 dependency for pip users.
Here is the compromise I went for. If PyQt5 is available, the ui files are compiled during build;
setup.py:
...
# Workaround in case PyQt5 was installed without pip
# Pre-compile ui files if PyQt5 is already installed
install_requires=['requests']
package_data={'': ['icons/*.svg']}
try:
    import PyQt5
    import subprocess
    path = here + "/qtpad/"
    for f in os.listdir(path):
        ext = f.lower().rsplit('.', 1)[-1]
        if ext == "ui":
            cmd = "pyuic5 " + path + f + " -o " + path  + f[:-3] + ".py"
            subprocess.run(cmd.split())
except ImportError:
    install_requires.append("pyqt5")
    package_data[''].append("*.ui")

setuptools.setup(
...
[...]

Interesting workaround!

However, you may wish to avoid the use of subprocess by using

from PyQt5 import uic
uic.compileUiDir('qtpad')

alfalfa · 2018-01-31 18:45:23

Eschwartz wrote:

However, you may wish to avoid the use of subprocess by using
from PyQt5 import uic
uic.compileUiDir('qtpad')

Thank you, I wished to do it directly through python but I was not aware of that method. It look much cleaner;

install_requires=['requests']
try:
    from PyQt5 import uic
    uic.compileUiDir('qtpad')
except:
    install_requires.append("pyqt5")

edit: when some dependancies are unsatisfied, setup.py is ran a second time AFTER their installation. Therefore package_data can be set to a fixed value, and the line 'package_data[''].append("*.ui")' can be removed. This way the Ui files are compiled, whether the app is installed from pip or from the AUR.

Last edited by alfalfa (2018-02-01 01:14:52)

eschwartz · 2018-01-31 19:10:12

Always worth looking at the docs: http://pyqt.sourceforge.net/Docs/PyQt5/ … uic-module

pyuic5 is actually a shellscript which runs `exec /usr/bin/python -m PyQt5.uic.pyuic ${1+"$@"}` so it should be assumed there is some way, somewhere in the docs.

alfalfa · 2018-02-01 15:33:36

There seems to be a problem regarding the VCS. When testing the package I noticed that even when pkgver() function is present, makepkg ask for the pkgver variable, and else return "ERROR: pkgver is not allowed to be empty". So to allow the installation to proceed I set the pkgver variable to an arbitrary value (such as pkgver=0.0.0) as shown above. It install correctly but the package always appear as if it is not up to date, even if it correspond to the lastest release;

cower -u
:: qtpad-git r101.a210678-1 -> 0.0.0-1

What would be the correct way to fix this? Do I have to update the PKGBUILD at every commit?

Scimmia · 2018-02-01 15:36:27

The way that was explained to you on IRC twice. Let makepkg set the pkgver and stop screwing with it.

Trilby · 2018-02-01 16:07:27

alfalfa wrote:

It install correctly but the package always appear as if it is not up to date

That is correct, it is a vcs PKGBUILD. All is as intended.

eschwartz · 2018-02-01 17:31:02

The problem is that pacman's version comparison logic considers "0" to be newer than "r1". I wonder if that should be considered a bug, or whether we should consider versions evaluating to "there is no version" to be a bug.

I vote for the latter... upload a PKGBUILD to the AUR that contains the pkgver you built at the time you uploaded the PKGBUILD...
I don't understand why a *number* of VCS package maintainers over the years have decided to override the pkgver created by pkgver() as though "0" is somehow a holy concept.

Last edited by eschwartz (2018-02-01 17:33:00)

Trilby · 2018-02-01 19:02:33

Eschwartz wrote:

The problem is that pacman's version comparison logic considers "0" to be newer than "r1".

I disagree. While your statement is factually correct - it is not the source of the problem. If instead of a "0" the PKGBUILD variable had a "s1" instead, the version comparision logic (not pacman's, just whatever is in aur-helper-of-the-day) will say there is no update available. This would also often be wrong.

There is no comparison of the installed version and the version listed on the AUR web interface that will give reliable information about when there is an update available. So like I've said before, there is only one sane answer for a check of whether there is an update available for a vcs package: "yes".

"pkgver=0.0.0" may or may not be silly, but it's definitely not relevant ... for anything.

eschwartz · 2018-02-01 19:30:04

There is another sane answer for that question: "I don't care, I'll do a more thorough check next week".

That's why, for example, no AUR helper -- even the good ones -- defaults to always attempting to build VCS packages and see if an updated version can be built.

Trilby · 2018-02-01 20:32:38

I'm not implying that the "yes" answer means an update should be built (and certainly not be default by any helper) - I'm only saying that the pkgver variable is meaningless in vcs PKGBUILDs. It does need to be there, but it's content is irelevant: so it really doesn't matter if it is "r1.867.fd43ec" or "0.0.0" or "Nothing.To.See.Here".

Arch Linux

#1 2018-01-22 22:35:17

PKGBUILD for a python script (need reviews)

#2 2018-01-23 00:04:06

Re: PKGBUILD for a python script (need reviews)

#3 2018-01-23 01:05:54

Re: PKGBUILD for a python script (need reviews)

#4 2018-01-23 01:09:49

Re: PKGBUILD for a python script (need reviews)

#5 2018-01-29 21:27:12

Re: PKGBUILD for a python script (need reviews)

#6 2018-01-29 21:34:15

Re: PKGBUILD for a python script (need reviews)

#7 2018-01-29 21:58:01

Re: PKGBUILD for a python script (need reviews)

#8 2018-01-31 14:30:00

Re: PKGBUILD for a python script (need reviews)

#9 2018-01-31 15:37:39

Re: PKGBUILD for a python script (need reviews)

#10 2018-01-31 18:45:23

Re: PKGBUILD for a python script (need reviews)

#11 2018-01-31 19:10:12

Re: PKGBUILD for a python script (need reviews)

#12 2018-02-01 15:33:36

Re: PKGBUILD for a python script (need reviews)

#13 2018-02-01 15:36:27

Re: PKGBUILD for a python script (need reviews)

#14 2018-02-01 16:07:27

Re: PKGBUILD for a python script (need reviews)

#15 2018-02-01 17:31:02

Re: PKGBUILD for a python script (need reviews)

#16 2018-02-01 19:02:33

Re: PKGBUILD for a python script (need reviews)

#17 2018-02-01 19:30:04

Re: PKGBUILD for a python script (need reviews)

#18 2018-02-01 20:32:38

Re: PKGBUILD for a python script (need reviews)

Board footer