You are not logged in.

#1 2021-01-11 17:12:07

schard
Member
From: Hannover
Registered: 2016-05-06
Posts: 1,932
Website

[SOLVED] Python packaging with setuptools - easy_install vs. pip

A user has encountered a problem with one of my python pacakges in the AUR.
After some tinkering, I finally was able to reproduce this issue in my build environment.
It seems, that, if python-pip is not installed, distutils falls back to easy_install to setup the python package and builds it just fine:

$ makechrootpkg -cr /var/build
==> Synchronizing chroot copy [/var/build/root] -> [build]...done
==> Making package: speculum 1.7.0-1 (Mon Jan 11 17:55:55 2021)
==> Retrieving sources...
  -> Updating speculum git repo...
Fetching origin
remote: Enumerating objects: 16, done.
remote: Counting objects: 100% (16/16), done.
remote: Compressing objects: 100% (8/8), done.
remote: Total 16 (delta 7), reused 16 (delta 7), pack-reused 0
Unpacking objects: 100% (16/16), 47.85 KiB | 374.00 KiB/s, done.
From https://github.com/coNQP/speculum
   ee8fe5c..33339b0  master     -> master
==> Validating source files with sha256sums...
    speculum ... Skipped
==> Making package: speculum 1.7.0-1 (Mon 11 Jan 2021 05:56:00 PM CET)
==> Checking runtime dependencies...
==> Installing missing dependencies...
resolving dependencies...
looking for conflicting packages...

Packages (2) libnsl-1.3.0-1  python-3.9.1-1

Total Installed Size:  79.58 MiB

:: Proceed with installation? [Y/n] 
(2/2) checking keys in keyring                                                                                     [####################################################################] 100%
(2/2) checking package integrity                                                                                   [####################################################################] 100%
(2/2) loading package files                                                                                        [####################################################################] 100%
(2/2) checking for file conflicts                                                                                  [####################################################################] 100%
(2/2) checking available disk space                                                                                [####################################################################] 100%
:: Processing package changes...
(1/2) installing libnsl                                                                                            [####################################################################] 100%
(2/2) installing python                                                                                            [####################################################################] 100%
Optional dependencies for python
    python-setuptools
    python-pip
    sqlite [installed]
    mpdecimal: for decimal
    xz: for lzma [installed]
    tk: for tkinter
:: Running post-transaction hooks...
(1/1) Arming ConditionNeedsUpdate...
==> Checking buildtime dependencies...
==> Installing missing dependencies...
resolving dependencies...
looking for conflicting packages...

Packages (11) perl-error-0.17029-2  perl-mailtools-2.21-4  perl-timedate-2.33-2  python-appdirs-1.4.4-3  python-ordered-set-4.0.2-3  python-packaging-20.8-2  python-pyparsing-2.4.7-3
              python-six-1.15.0-3  git-2.30.0-1  python-setuptools-1:51.1.2-1  python-setuptools-git-1.2-6

Total Installed Size:  35.56 MiB

:: Proceed with installation? [Y/n] 
(11/11) checking keys in keyring                                                                                   [####################################################################] 100%
(11/11) checking package integrity                                                                                 [####################################################################] 100%
(11/11) loading package files                                                                                      [####################################################################] 100%
(11/11) checking for file conflicts                                                                                [####################################################################] 100%
(11/11) checking available disk space                                                                              [####################################################################] 100%
:: Processing package changes...
( 1/11) installing perl-error                                                                                      [####################################################################] 100%
( 2/11) installing perl-timedate                                                                                   [####################################################################] 100%
( 3/11) installing perl-mailtools                                                                                  [####################################################################] 100%
( 4/11) installing git                                                                                             [####################################################################] 100%
Optional dependencies for git
    tk: gitk and git gui
    perl-libwww: git svn
    perl-term-readkey: git svn and interactive.singlekey setting
    perl-mime-tools: git send-email
    perl-net-smtp-ssl: git send-email TLS support
    perl-authen-sasl: git send-email TLS support
    perl-mediawiki-api: git mediawiki support
    perl-datetime-format-iso8601: git mediawiki support
    perl-lwp-protocol-https: git mediawiki https support
    perl-cgi: gitweb (web interface) support
    python: git svn & git p4 [installed]
    subversion: git svn
    org.freedesktop.secrets: keyring credential helper
    libsecret: libsecret credential helper [installed]
( 5/11) installing python-appdirs                                                                                  [####################################################################] 100%
( 6/11) installing python-pyparsing                                                                                [####################################################################] 100%
( 7/11) installing python-six                                                                                      [####################################################################] 100%
( 8/11) installing python-packaging                                                                                [####################################################################] 100%
( 9/11) installing python-ordered-set                                                                              [####################################################################] 100%
(10/11) installing python-setuptools                                                                               [####################################################################] 100%
(11/11) installing python-setuptools-git                                                                           [####################################################################] 100%
:: Running post-transaction hooks...
(1/4) Creating system user accounts...
Creating group git with gid 977.
Creating user git (git daemon user) with uid 977 and gid 977.
(2/4) Reloading system manager configuration...
  Skipped: Current root is not booted.
(3/4) Arming ConditionNeedsUpdate...
(4/4) Warn about old perl modules
==> Retrieving sources...
==> WARNING: Skipping all source file integrity checks.
==> Extracting sources...
  -> Creating working copy of speculum git repo...
Cloning into 'speculum'...
done.
Switched to a new branch 'makepkg'
==> Entering fakeroot environment...
==> Starting package()...
WARNING: The pip package is not available, falling back to EasyInstall for handling setup_requires/test_requires; this is deprecated and will be removed in a future version.
zip_safe flag not set; analyzing archive contents...

Installed /build/speculum/src/speculum/.eggs/setuptools_git_version-1.0.3-py3.9.egg
running install
running build
running build_py
creating build
creating build/lib
creating build/lib/speculum
copying speculum/io.py -> build/lib/speculum
copying speculum/main.py -> build/lib/speculum
copying speculum/limiting.py -> build/lib/speculum
copying speculum/mirrors.py -> build/lib/speculum
copying speculum/config.py -> build/lib/speculum
copying speculum/logging.py -> build/lib/speculum
copying speculum/filtering.py -> build/lib/speculum
copying speculum/sorting.py -> build/lib/speculum
copying speculum/parsers.py -> build/lib/speculum
copying speculum/__init__.py -> build/lib/speculum
copying speculum/argparse.py -> build/lib/speculum
running install_lib
creating /build/speculum/pkg/speculum/usr
creating /build/speculum/pkg/speculum/usr/lib
creating /build/speculum/pkg/speculum/usr/lib/python3.9
creating /build/speculum/pkg/speculum/usr/lib/python3.9/site-packages
creating /build/speculum/pkg/speculum/usr/lib/python3.9/site-packages/speculum
copying build/lib/speculum/io.py -> /build/speculum/pkg/speculum/usr/lib/python3.9/site-packages/speculum
copying build/lib/speculum/main.py -> /build/speculum/pkg/speculum/usr/lib/python3.9/site-packages/speculum
copying build/lib/speculum/limiting.py -> /build/speculum/pkg/speculum/usr/lib/python3.9/site-packages/speculum
copying build/lib/speculum/mirrors.py -> /build/speculum/pkg/speculum/usr/lib/python3.9/site-packages/speculum
copying build/lib/speculum/config.py -> /build/speculum/pkg/speculum/usr/lib/python3.9/site-packages/speculum
copying build/lib/speculum/logging.py -> /build/speculum/pkg/speculum/usr/lib/python3.9/site-packages/speculum
copying build/lib/speculum/filtering.py -> /build/speculum/pkg/speculum/usr/lib/python3.9/site-packages/speculum
copying build/lib/speculum/sorting.py -> /build/speculum/pkg/speculum/usr/lib/python3.9/site-packages/speculum
copying build/lib/speculum/parsers.py -> /build/speculum/pkg/speculum/usr/lib/python3.9/site-packages/speculum
copying build/lib/speculum/__init__.py -> /build/speculum/pkg/speculum/usr/lib/python3.9/site-packages/speculum
copying build/lib/speculum/argparse.py -> /build/speculum/pkg/speculum/usr/lib/python3.9/site-packages/speculum
byte-compiling /build/speculum/pkg/speculum/usr/lib/python3.9/site-packages/speculum/io.py to io.cpython-39.pyc
byte-compiling /build/speculum/pkg/speculum/usr/lib/python3.9/site-packages/speculum/main.py to main.cpython-39.pyc
byte-compiling /build/speculum/pkg/speculum/usr/lib/python3.9/site-packages/speculum/limiting.py to limiting.cpython-39.pyc
byte-compiling /build/speculum/pkg/speculum/usr/lib/python3.9/site-packages/speculum/mirrors.py to mirrors.cpython-39.pyc
byte-compiling /build/speculum/pkg/speculum/usr/lib/python3.9/site-packages/speculum/config.py to config.cpython-39.pyc
byte-compiling /build/speculum/pkg/speculum/usr/lib/python3.9/site-packages/speculum/logging.py to logging.cpython-39.pyc
byte-compiling /build/speculum/pkg/speculum/usr/lib/python3.9/site-packages/speculum/filtering.py to filtering.cpython-39.pyc
byte-compiling /build/speculum/pkg/speculum/usr/lib/python3.9/site-packages/speculum/sorting.py to sorting.cpython-39.pyc
byte-compiling /build/speculum/pkg/speculum/usr/lib/python3.9/site-packages/speculum/parsers.py to parsers.cpython-39.pyc
byte-compiling /build/speculum/pkg/speculum/usr/lib/python3.9/site-packages/speculum/__init__.py to __init__.cpython-39.pyc
byte-compiling /build/speculum/pkg/speculum/usr/lib/python3.9/site-packages/speculum/argparse.py to argparse.cpython-39.pyc
writing byte-compilation script '/tmp/tmp3ce9p4b_.py'
/usr/bin/python /tmp/tmp3ce9p4b_.py
removing /tmp/tmp3ce9p4b_.py
running install_data
creating /build/speculum/pkg/speculum/etc
copying files/speculum.conf -> /build/speculum/pkg/speculum/etc
creating /build/speculum/pkg/speculum/usr/lib/systemd
creating /build/speculum/pkg/speculum/usr/lib/systemd/system
copying files/speculum.service -> /build/speculum/pkg/speculum/usr/lib/systemd/system
copying files/speculum.timer -> /build/speculum/pkg/speculum/usr/lib/systemd/system
running install_egg_info
running egg_info
creating speculum.egg-info
writing speculum.egg-info/PKG-INFO
writing dependency_links to speculum.egg-info/dependency_links.txt
writing entry points to speculum.egg-info/entry_points.txt
writing top-level names to speculum.egg-info/top_level.txt
writing manifest file 'speculum.egg-info/SOURCES.txt'
writing manifest file 'speculum.egg-info/SOURCES.txt'
Copying speculum.egg-info to /build/speculum/pkg/speculum/usr/lib/python3.9/site-packages/speculum-1.7.0-py3.9.egg-info
running install_scripts
Installing speculum script to /build/speculum/pkg/speculum/usr/bin
==> Tidying install...
  -> Removing libtool files...
  -> Purging unwanted files...
  -> Removing static library files...
  -> Stripping unneeded symbols from binaries and libraries...
  -> Compressing man and info pages...
==> Checking for packaging issues...
==> Creating package "speculum"...
  -> Generating .PKGINFO file...
  -> Generating .BUILDINFO file...
  -> Generating .MTREE file...
  -> Compressing package...
==> Leaving fakeroot environment.
==> Finished making: speculum 1.7.0-1 (Mon 11 Jan 2021 05:56:05 PM CET)

However, when python-pip is installed, it will also require python-wheel to run the setup:

$ sudo systemd-nspawn -D /var/build/build pacman -Syu python-pip
[sudo] Passwort für rne: 
Spawning container build on /var/build/build.
Press ^] three times within 1s to kill container.
:: Synchronizing package databases...
 core                                                                                  132.3 KiB  2.44 MiB/s 00:00 [####################################################################] 100%
 extra                                                                                1646.7 KiB  5.83 MiB/s 00:00 [####################################################################] 100%
 community                                                                               5.3 MiB  5.83 MiB/s 00:01 [####################################################################] 100%
 rne is up to date
:: Starting full system upgrade...
resolving dependencies...
looking for conflicting packages...

Packages (25) archlinux-keyring-20210110-1  python-asn1crypto-1.4.0-3  python-cachecontrol-0.12.6-3  python-cffi-1.14.4-1  python-chardet-3.0.4-7  python-colorama-0.4.4-3
              python-contextlib2-0.6.0.post1-3  python-cryptography-3.3.1-1  python-distlib-0.3.1-2  python-distro-1.5.0-3  python-html5lib-1.1-5  python-idna-2.10-3  python-msgpack-1.0.2-2
              python-pep517-0.9.1-3  python-ply-3.11-7  python-progress-1.5-5  python-pycparser-2.20-3  python-pyopenssl-20.0.1-1  python-requests-2.25.1-1  python-resolvelib-0.5.4-1
              python-retrying-1.3.3-9  python-toml-0.10.2-3  python-urllib3-1.26.1-1  python-webencodings-0.5.1-6  python-pip-20.2.4-1

Total Download Size:    3.68 MiB
Total Installed Size:  16.04 MiB
Net Upgrade Size:      14.74 MiB

:: Proceed with installation? [Y/n] 
:: Retrieving packages...
 archlinux-keyring-20210110-1-any                                                      950.2 KiB  4.99 MiB/s 00:00 [####################################################################] 100%
 python-urllib3-1.26.1-1-any                                                           182.1 KiB  6.59 MiB/s 00:00 [####################################################################] 100%
 python-chardet-3.0.4-7-any                                                            169.8 KiB  5.03 MiB/s 00:00 [####################################################################] 100%
 python-idna-2.10-3-any                                                                 88.1 KiB  8.60 MiB/s 00:00 [####################################################################] 100%
 python-requests-2.25.1-1-any                                                           90.8 KiB  8.86 MiB/s 00:00 [####################################################################] 100%
 python-pep517-0.9.1-3-any                                                              29.6 KiB  0.00   B/s 00:00 [####################################################################] 100%
 python-retrying-1.3.3-9-any                                                            10.9 KiB  0.00   B/s 00:00 [####################################################################] 100%
 python-resolvelib-0.5.4-1-any                                                          22.0 KiB  0.00   B/s 00:00 [####################################################################] 100%
 python-ply-3.11-7-any                                                                  76.6 KiB  7.48 MiB/s 00:00 [####################################################################] 100%
 python-pycparser-2.20-3-any                                                           181.1 KiB  6.80 MiB/s 00:00 [####################################################################] 100%
 python-cffi-1.14.4-1-x86_64                                                           227.3 KiB  6.73 MiB/s 00:00 [####################################################################] 100%
 python-asn1crypto-1.4.0-3-any                                                         176.1 KiB  5.73 MiB/s 00:00 [####################################################################] 100%
 python-cryptography-3.3.1-1-x86_64                                                    379.6 KiB  5.88 MiB/s 00:00 [####################################################################] 100%
 python-pyopenssl-20.0.1-1-any                                                          82.9 KiB  5.78 MiB/s 00:00 [####################################################################] 100%
 python-pip-20.2.4-1-any                                                               411.8 KiB  6.09 MiB/s 00:00 [####################################################################] 100%
 python-msgpack-1.0.2-2-x86_64                                                          74.4 KiB  7.26 MiB/s 00:00 [####################################################################] 100%
 python-cachecontrol-0.12.6-3-any                                                       30.7 KiB  9.99 MiB/s 00:00 [####################################################################] 100%
 python-colorama-0.4.4-3-any                                                            24.9 KiB  8.10 MiB/s 00:00 [####################################################################] 100%
 python-contextlib2-0.6.0.post1-3-any                                                   15.5 KiB  0.00   B/s 00:00 [####################################################################] 100%
 python-distlib-0.3.1-2-any                                                            244.3 KiB  6.63 MiB/s 00:00 [####################################################################] 100%
 python-distro-1.5.0-3-any                                                              34.0 KiB  0.00   B/s 00:00 [####################################################################] 100%
 python-webencodings-0.5.1-6-any                                                        18.6 KiB  0.00   B/s 00:00 [####################################################################] 100%
 python-html5lib-1.1-5-any                                                             200.6 KiB  5.94 MiB/s 00:00 [####################################################################] 100%
 python-toml-0.10.2-3-any                                                               34.8 KiB  11.3 MiB/s 00:00 [####################################################################] 100%
 python-progress-1.5-5-any                                                              12.3 KiB  0.00   B/s 00:00 [####################################################################] 100%
(25/25) checking keys in keyring                                                                                   [####################################################################] 100%
(25/25) checking package integrity                                                                                 [####################################################################] 100%
(25/25) loading package files                                                                                      [####################################################################] 100%
(25/25) checking for file conflicts                                                                                [####################################################################] 100%
(25/25) checking available disk space                                                                              [####################################################################] 100%
:: Processing package changes...
( 1/25) installing python-msgpack                                                                                  [####################################################################] 100%
( 2/25) installing python-urllib3                                                                                  [####################################################################] 100%
Optional dependencies for python-urllib3
    python-pysocks: SOCKS support
    python-brotli: Brotli support
    python-pyopenssl: security support [pending]
( 3/25) installing python-chardet                                                                                  [####################################################################] 100%
( 4/25) installing python-idna                                                                                     [####################################################################] 100%
( 5/25) installing python-requests                                                                                 [####################################################################] 100%
Optional dependencies for python-requests
    python-pysocks: SOCKS proxy support
( 6/25) installing python-cachecontrol                                                                             [####################################################################] 100%
Optional dependencies for python-cachecontrol
    python-lockfile: for the FileCache
( 7/25) installing python-colorama                                                                                 [####################################################################] 100%
( 8/25) installing python-contextlib2                                                                              [####################################################################] 100%
( 9/25) installing python-distlib                                                                                  [####################################################################] 100%
(10/25) installing python-distro                                                                                   [####################################################################] 100%
(11/25) installing python-webencodings                                                                             [####################################################################] 100%
(12/25) installing python-html5lib                                                                                 [####################################################################] 100%
Optional dependencies for python-html5lib
    python-lxml: lxml treebuilder
    python-genshi: genshi treewalker
(13/25) installing python-toml                                                                                     [####################################################################] 100%
(14/25) installing python-pep517                                                                                   [####################################################################] 100%
(15/25) installing python-progress                                                                                 [####################################################################] 100%
(16/25) installing python-retrying                                                                                 [####################################################################] 100%
(17/25) installing python-resolvelib                                                                               [####################################################################] 100%
(18/25) installing python-ply                                                                                      [####################################################################] 100%
(19/25) installing python-pycparser                                                                                [####################################################################] 100%
(20/25) installing python-cffi                                                                                     [####################################################################] 100%
(21/25) installing python-asn1crypto                                                                               [####################################################################] 100%
(22/25) installing python-cryptography                                                                             [####################################################################] 100%
(23/25) installing python-pyopenssl                                                                                [####################################################################] 100%
(24/25) installing python-pip                                                                                      [####################################################################] 100%
(25/25) upgrading archlinux-keyring                                                                                [####################################################################] 100%
==> Appending keys from archlinux.gpg...
gpg: Warning: using insecure memory!
==> Locally signing trusted keys in keyring...
  -> Locally signing key AB19265E5D7D20687D303246BA1DFB64FFF979E7...
  -> Locally signing key DDB867B92AA789C165EEFA799B729B06A680C281...
  -> Locally signing key 0E8B644079F599DFC1DDC3973348882F6AC6A4C2...
  -> Locally signing key D8AFDDA07A5B6EDFA7D8CCDAD6D055F927843F1C...
  -> Locally signing key 91FFE0700E80619CEB73235CA88E23E377514E00...
==> Importing owner trust values...
gpg: Warning: using insecure memory!
==> Disabling revoked keys in keyring...
  -> Disabling key 4A8B17E20B88ACA61860009B5CED81B7C2E5C0D2...
  -> Disabling key 684148BB25B49E986A4944C55184252D824B18E8...
  -> Disabling key 5357F3B111688D88C1D88119FCF2CB179205AC90...
  -> Disabling key 50F33E2E5B0C3D900424ABE89BDCF497A4BBCC7F...
  -> Disabling key 39F880E50E49A4D11341E8F939E4F17F295AFBF4...
  -> Disabling key F5A361A3A13554B85E57DDDAAF7EF7873CFD4BB6...
  -> Disabling key 40440DC037C05620984379A6761FAD69BA06C6A9...
  -> Disabling key FB871F0131FEA4FB5A9192B4C8880A6406361833...
  -> Disabling key 487EACC08557AD082088DABA1EB2638FF56C0C53...
  -> Disabling key 76B4192E902C0A52642C63C273B8ED52F1D357C1...
  -> Disabling key 40776A5221EF5AD468A4906D42A1DB15EC133BAD...
  -> Disabling key 0B20CA1931F5DA3A70D0F8D2EA6836E1AB441196...
  -> Disabling key 07DFD3A0BC213FA12EDC217559B3122E2FA915EC...
  -> Disabling key 34C5D94FE7E7913E86DC427E7FB1A3800C84C0A5...
  -> Disabling key B1F2C889CB2CCB2ADA36D963097D629E437520BD...
  -> Disabling key D4DE5ABDE2A7287644EAC7E36D1A9E70E19DAA50...
  -> Disabling key 44D4A033AC140143927397D47EFD567D4C7EA887...
  -> Disabling key 8F76BEEA0289F9E1D3E229C05F946DED983D4366...
  -> Disabling key 27FFC4769E19F096D41D9265A04F9397CDFD6BB0...
  -> Disabling key 4FCF887689C41B09506BE8D5F3E1D5C5D30DB0AD...
  -> Disabling key 5A2257D19FF7E1E0E415968CE62F853100F0D0F0...
  -> Disabling key 7FA647CD89891DEDC060287BB9113D1ED21E1A55...
  -> Disabling key 5E7585ADFF106BFFBBA319DC654B877A0864983E...
  -> Disabling key E7210A59715F6940CF9A4E36A001876699AD6E84...
  -> Disabling key 5559BC1A32B8F76B3FCCD9555FA5E5544F010D48...
  -> Disabling key BFA1ECFEF1524EE4099CDE971F0CD4921ECAA030...
  -> Disabling key 4D913AECD81726D9A6C74F0ADA6426DD215B37AD...
  -> Disabling key 8840BD07FC24CB7CE394A07CCF7037A4F27FB7DA...
  -> Disabling key BC1FBE4D2826A0B51E47ED62E2539214C6C11350...
  -> Disabling key 9515D8A8EAB88E49BB65EDBCE6B456CAF15447D5...
  -> Disabling key 779CD2942629B7FA04AB8F172E89012331361F01...
  -> Disabling key D921CABED130A5690EF1896E81AF739EC0711BF1...
  -> Disabling key 5696C003B0854206450C8E5BE613C09CB4440678...
  -> Disabling key 8CF934E339CAD8ABF342E822E711306E3C4F88BC...
  -> Disabling key 1A60DC44245D06FEF90623D6EEEEE2EEEE2EEEEE...
  -> Disabling key 81D7F8241DB38BC759C80FCE3A726C6170E80477...
  -> Disabling key 63F395DE2D6398BBE458F281F2DBB4931985A992...
  -> Disabling key 65EEFE022108E2B708CBFCF7F9E712E59AF5F22A...
  -> Disabling key 66BD74A036D522F51DD70A3C7F2A16726521E06D...
==> Updating trust database...
gpg: Warning: using insecure memory!
gpg: next trustdb check due at 2021-03-21
:: Running post-transaction hooks...
(1/1) Arming ConditionNeedsUpdate...
Container build exited successfully.
$ makechrootpkg -r /var/build
==> Making package: speculum 1.7.0-1 (Mon Jan 11 17:57:51 2021)
==> Retrieving sources...
  -> Updating speculum git repo...
Fetching origin
==> Validating source files with sha256sums...
    speculum ... Skipped
==> Making package: speculum 1.7.0-1 (Mon 11 Jan 2021 05:57:55 PM CET)
==> Checking runtime dependencies...
==> Checking buildtime dependencies...
==> Retrieving sources...
==> WARNING: Skipping all source file integrity checks.
==> Extracting sources...
  -> Creating working copy of speculum git repo...
Cloning into 'speculum'...
done.
Switched to a new branch 'makepkg'
==> Entering fakeroot environment...
==> Starting package()...
WARNING: The wheel package is not available.
  ERROR: Command errored out with exit status 1:
   command: /usr/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-wheel-c4grf1ve/setuptools-git-version/setup.py'"'"'; __file__='"'"'/tmp/pip-wheel-c4grf1ve/setuptools-git-version/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-qod7ge6f
       cwd: /tmp/pip-wheel-c4grf1ve/setuptools-git-version/
  Complete output (6 lines):
  usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
     or: setup.py --help [cmd1 cmd2 ...]
     or: setup.py --help-commands
     or: setup.py cmd --help
  
  error: invalid command 'bdist_wheel'
  ----------------------------------------
  ERROR: Failed building wheel for setuptools-git-version
ERROR: Failed to build one or more wheels
Traceback (most recent call last):
  File "/usr/lib/python3.9/site-packages/setuptools/installer.py", line 126, in fetch_build_egg
    subprocess.check_call(cmd)
  File "/usr/lib/python3.9/subprocess.py", line 373, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/usr/bin/python', '-m', 'pip', '--disable-pip-version-check', 'wheel', '--no-deps', '-w', '/tmp/tmpclsha1pc', '--quiet', 'setuptools-git-version']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/build/speculum/src/speculum/setup.py", line 8, in <module>
    setup(
  File "/usr/lib/python3.9/site-packages/setuptools/__init__.py", line 152, in setup
    _install_setup_requires(attrs)
  File "/usr/lib/python3.9/site-packages/setuptools/__init__.py", line 147, in _install_setup_requires
    dist.fetch_build_eggs(dist.setup_requires)
  File "/usr/lib/python3.9/site-packages/setuptools/dist.py", line 673, in fetch_build_eggs
    resolved_dists = pkg_resources.working_set.resolve(
  File "/usr/lib/python3.9/site-packages/pkg_resources/__init__.py", line 764, in resolve
    dist = best[req.key] = env.best_match(
  File "/usr/lib/python3.9/site-packages/pkg_resources/__init__.py", line 1049, in best_match
    return self.obtain(req, installer)
  File "/usr/lib/python3.9/site-packages/pkg_resources/__init__.py", line 1061, in obtain
    return installer(requirement)
  File "/usr/lib/python3.9/site-packages/setuptools/dist.py", line 732, in fetch_build_egg
    return fetch_build_egg(self, req)
  File "/usr/lib/python3.9/site-packages/setuptools/installer.py", line 128, in fetch_build_egg
    raise DistutilsError(str(e)) from e
distutils.errors.DistutilsError: Command '['/usr/bin/python', '-m', 'pip', '--disable-pip-version-check', 'wheel', '--no-deps', '-w', '/tmp/tmpclsha1pc', '--quiet', 'setuptools-git-version']' returned non-zero exit status 1.
==> ERROR: A failure occurred in package().
    Aborting...
==> ERROR: Build failed, check /var/build/build/build

If I also install python-wheel, the package will built again:

$ sudo systemd-nspawn -D /var/build/build pacman -Syu python-wheel
[sudo] Passwort für rne: 
Spawning container build on /var/build/build.
Press ^] three times within 1s to kill container.
:: Synchronizing package databases...
 core is up to date
 extra is up to date
 community is up to date
 rne is up to date
:: Starting full system upgrade...
resolving dependencies...
looking for conflicting packages...

Packages (1) python-wheel-0.36.2-1

Total Download Size:   0.04 MiB
Total Installed Size:  0.16 MiB

:: Proceed with installation? [Y/n] 
:: Retrieving packages...
 python-wheel-0.36.2-1-any                                                              45.8 KiB  1695 KiB/s 00:00 [####################################################################] 100%
(1/1) checking keys in keyring                                                                                     [####################################################################] 100%
(1/1) checking package integrity                                                                                   [####################################################################] 100%
(1/1) loading package files                                                                                        [####################################################################] 100%
(1/1) checking for file conflicts                                                                                  [####################################################################] 100%
(1/1) checking available disk space                                                                                [####################################################################] 100%
:: Processing package changes...
(1/1) installing python-wheel                                                                                      [####################################################################] 100%
Optional dependencies for python-wheel
    python-xdg: for wheel.signatures
:: Running post-transaction hooks...
(1/1) Arming ConditionNeedsUpdate...
Container build exited successfully.
$ makechrootpkg -r /var/build
==> Making package: speculum 1.7.0-1 (Mon Jan 11 18:08:37 2021)
==> Retrieving sources...
  -> Updating speculum git repo...
Fetching origin
==> Validating source files with sha256sums...
    speculum ... Skipped
==> Making package: speculum 1.7.0-1 (Mon 11 Jan 2021 06:08:41 PM CET)
==> Checking runtime dependencies...
==> Checking buildtime dependencies...
==> Retrieving sources...
==> WARNING: Skipping all source file integrity checks.
==> Extracting sources...
  -> Creating working copy of speculum git repo...
Cloning into 'speculum'...
done.
Switched to a new branch 'makepkg'
==> Entering fakeroot environment...
==> Starting package()...
running install
running build
running build_py
creating build
creating build/lib
creating build/lib/speculum
copying speculum/io.py -> build/lib/speculum
copying speculum/main.py -> build/lib/speculum
copying speculum/limiting.py -> build/lib/speculum
copying speculum/mirrors.py -> build/lib/speculum
copying speculum/config.py -> build/lib/speculum
copying speculum/logging.py -> build/lib/speculum
copying speculum/filtering.py -> build/lib/speculum
copying speculum/sorting.py -> build/lib/speculum
copying speculum/parsers.py -> build/lib/speculum
copying speculum/__init__.py -> build/lib/speculum
copying speculum/argparse.py -> build/lib/speculum
running install_lib
creating /build/speculum/pkg/speculum/usr
creating /build/speculum/pkg/speculum/usr/lib
creating /build/speculum/pkg/speculum/usr/lib/python3.9
creating /build/speculum/pkg/speculum/usr/lib/python3.9/site-packages
creating /build/speculum/pkg/speculum/usr/lib/python3.9/site-packages/speculum
copying build/lib/speculum/io.py -> /build/speculum/pkg/speculum/usr/lib/python3.9/site-packages/speculum
copying build/lib/speculum/main.py -> /build/speculum/pkg/speculum/usr/lib/python3.9/site-packages/speculum
copying build/lib/speculum/limiting.py -> /build/speculum/pkg/speculum/usr/lib/python3.9/site-packages/speculum
copying build/lib/speculum/mirrors.py -> /build/speculum/pkg/speculum/usr/lib/python3.9/site-packages/speculum
copying build/lib/speculum/config.py -> /build/speculum/pkg/speculum/usr/lib/python3.9/site-packages/speculum
copying build/lib/speculum/logging.py -> /build/speculum/pkg/speculum/usr/lib/python3.9/site-packages/speculum
copying build/lib/speculum/filtering.py -> /build/speculum/pkg/speculum/usr/lib/python3.9/site-packages/speculum
copying build/lib/speculum/sorting.py -> /build/speculum/pkg/speculum/usr/lib/python3.9/site-packages/speculum
copying build/lib/speculum/parsers.py -> /build/speculum/pkg/speculum/usr/lib/python3.9/site-packages/speculum
copying build/lib/speculum/__init__.py -> /build/speculum/pkg/speculum/usr/lib/python3.9/site-packages/speculum
copying build/lib/speculum/argparse.py -> /build/speculum/pkg/speculum/usr/lib/python3.9/site-packages/speculum
byte-compiling /build/speculum/pkg/speculum/usr/lib/python3.9/site-packages/speculum/io.py to io.cpython-39.pyc
byte-compiling /build/speculum/pkg/speculum/usr/lib/python3.9/site-packages/speculum/main.py to main.cpython-39.pyc
byte-compiling /build/speculum/pkg/speculum/usr/lib/python3.9/site-packages/speculum/limiting.py to limiting.cpython-39.pyc
byte-compiling /build/speculum/pkg/speculum/usr/lib/python3.9/site-packages/speculum/mirrors.py to mirrors.cpython-39.pyc
byte-compiling /build/speculum/pkg/speculum/usr/lib/python3.9/site-packages/speculum/config.py to config.cpython-39.pyc
byte-compiling /build/speculum/pkg/speculum/usr/lib/python3.9/site-packages/speculum/logging.py to logging.cpython-39.pyc
byte-compiling /build/speculum/pkg/speculum/usr/lib/python3.9/site-packages/speculum/filtering.py to filtering.cpython-39.pyc
byte-compiling /build/speculum/pkg/speculum/usr/lib/python3.9/site-packages/speculum/sorting.py to sorting.cpython-39.pyc
byte-compiling /build/speculum/pkg/speculum/usr/lib/python3.9/site-packages/speculum/parsers.py to parsers.cpython-39.pyc
byte-compiling /build/speculum/pkg/speculum/usr/lib/python3.9/site-packages/speculum/__init__.py to __init__.cpython-39.pyc
byte-compiling /build/speculum/pkg/speculum/usr/lib/python3.9/site-packages/speculum/argparse.py to argparse.cpython-39.pyc
writing byte-compilation script '/tmp/tmparmh0bwo.py'
/usr/bin/python /tmp/tmparmh0bwo.py
removing /tmp/tmparmh0bwo.py
running install_data
creating /build/speculum/pkg/speculum/etc
copying files/speculum.conf -> /build/speculum/pkg/speculum/etc
creating /build/speculum/pkg/speculum/usr/lib/systemd
creating /build/speculum/pkg/speculum/usr/lib/systemd/system
copying files/speculum.service -> /build/speculum/pkg/speculum/usr/lib/systemd/system
copying files/speculum.timer -> /build/speculum/pkg/speculum/usr/lib/systemd/system
running install_egg_info
running egg_info
creating speculum.egg-info
writing speculum.egg-info/PKG-INFO
writing dependency_links to speculum.egg-info/dependency_links.txt
writing entry points to speculum.egg-info/entry_points.txt
writing top-level names to speculum.egg-info/top_level.txt
writing manifest file 'speculum.egg-info/SOURCES.txt'
writing manifest file 'speculum.egg-info/SOURCES.txt'
Copying speculum.egg-info to /build/speculum/pkg/speculum/usr/lib/python3.9/site-packages/speculum-1.7.0-py3.9.egg-info
running install_scripts
Installing speculum script to /build/speculum/pkg/speculum/usr/bin
==> Tidying install...
  -> Removing libtool files...
  -> Purging unwanted files...
  -> Removing static library files...
  -> Stripping unneeded symbols from binaries and libraries...
  -> Compressing man and info pages...
==> Checking for packaging issues...
==> Creating package "speculum"...
  -> Generating .PKGINFO file...
  -> Generating .BUILDINFO file...
  -> Generating .MTREE file...
  -> Compressing package...
==> Leaving fakeroot environment.
==> Finished making: speculum 1.7.0-1 (Mon 11 Jan 2021 06:08:44 PM CET)

In the Wiki, however, the guidelines suggest to refrain from using pip.
So I am uncertain how to correctly specify the build dependencies in the package?
PKGBUILD does not provide something like "makeconflicts" where I could add python-pip to force distutils to fall back to easy_install.
Consequentially I suspect, that I am doing something wrong regadring my package setup, but I cannot see what it might be.
Am I to add python-pip and python-wheel as dependencies or do I somehow need to rewrite my package() function?
What's the recommended way to go here?

Last edited by schard (2021-01-12 18:44:26)

Offline

#2 2021-01-12 01:59:04

eschwartz
Fellow
Registered: 2014-08-08
Posts: 4,097

Re: [SOLVED] Python packaging with setuptools - easy_install vs. pip

This is an easy one to solve. Your setup.py requires the use of this pypi module: https://pypi.org/project/setuptools-git-version/

But you have a makedepends on the Arch community package for this one: https://pypi.org/project/setuptools-git/

This is obviously terribly broken -- the PKGBUILD does not install the needed makedepends, so the python build system tries to download it unchecksummed over the internet in order to make use of it. In various circumstances it tries different things, too...

The number of different modules performing the exact same "get a version number by executing git describe" is astounding. They breed like rabbits.

What's even the point of all this? Much better to use tarball sources and have a trivial release rule to update setup.py with the version. Or even better, migrate to setup.cfg: https://setuptools.readthedocs.io/en/la … ing-values (note "attr:" and "file:" directives. Then you could set the version number from the common convention of the program's internal __version__ value in __init__.py (unless of course your program doesn't know its version when running progname --help of course).


Managing AUR repos The Right Way -- aurpublish (now a standalone tool)

Offline

#3 2021-01-12 08:04:37

schard
Member
From: Hannover
Registered: 2016-05-06
Posts: 1,932
Website

Re: [SOLVED] Python packaging with setuptools - easy_install vs. pip

Thank you for your review and the suggestions.
The discrepancy between the git package required and actually used, made me actually facepalm.

The reason I'm using such a thing in the first place is, that I have had bad experience with manual version tracking.
Manually updating the version has proven to be error prone, since I frequently forget to update it in all necessary places on a version bump i.e. new release.
I found that just adding an appropriate git tag and let software do the magic for me is more reliable, but I am open for new suggestions.

My software also does not know about its version. I never found that to be useful, since this was one of those places, where I frequently forgot to bump the version number, so that there would be an actual discrepancy between the package version and the version the program reports.
If a user wants to know the version of my python packages, they can ask pacman or pip respectively.

Another reason that I use the versioning automation is, that I use the same PKGBUILD base for all my python packages, some of which I also publish on pypi, since they are not Arch-specific (unlike speculum, of course).
So to update a package on PyPI I can just run "make pypi" (see Makefile) and after providing my credentials, it will also update the package on the PyPI to the latest tagged version.

If you think that this is an unfeasible approach, I am open to suggestions. I however would like to have the option to just update the package version in one place only, preferably in the git tags.

PS: The community package setuptools-git does not provide the versioning magic, that setuptools-git-version provides.
Luckily I just found python-setuptools-scm, which seems to work nicely on first tests.

Last edited by schard (2021-01-12 08:29:14)

Offline

#4 2021-01-12 12:15:58

eschwartz
Fellow
Registered: 2014-08-08
Posts: 4,097

Re: [SOLVED] Python packaging with setuptools - easy_install vs. pip

Ah, well, there's definitely a bunch of modules to do it even if setuptools-git isn't one of them. tongue

Anyway, the nice thing about setup.cfg attributes is, like I said, you can use one file in your repository to track this info for multiple uses. I honestly don't see the point of deferring to git describe though. Just add a Makefile rule that bumps both at the same time. There are also a few different python projects to do this for you specific to bumping version strings everywhere in your codebase.


Managing AUR repos The Right Way -- aurpublish (now a standalone tool)

Offline

Board footer

Powered by FluxBB