[Solved] Resolve dependencies using pip in AUR package?

kryptxy · 2017-08-06 07:27:48

Hello,

I am trying to build AUR package for a simple tool.
Project: https://github.com/kryptxy/torrench

I am stuck at a point, and confused as to how I should proceed.

The tool requires following additional packages to work:
requests, bs4, lxml, tabulate, termcolor.
Out of these, one package is not available in arch repos - tabulate. Tabulate is present in AUR though.

Can I install this remaining package using pip? Or should I explicitly mention user to install tabulate from AUR?
I'd personally prefer if all the dependencies were able to be installed using pip.

Here is the PKGBUILD I came up with till now (Please correct me wherever I go wrong/inappropriate):

# Maintainer: kryptxy <kryptxy@protonmail.com>
pkgname=torrench
pkgver=1.0.0.20170806
pkgrel=1
pkgdesc="Command-line torrent search tool"
arch=('any')
url="https://github.com/kryptxy/torrench"
license=('GPL')
depends=('python' ) #Packages available in arch repos will be added here
provides=("torrench")
conflicts=("torrench")
source=('git+https://github.com/kryptxy/torrench.git')
md5sums=('SKIP')

#prepare() {
#	cd "$pkgname-$pkgver"
#	patch -p1 -i "$srcdir/$pkgname-$pkgver.patch"  
#}


build() {
	cd "$pkgname"
	python setup.py build
}

#check() {
#cd "$pkgname-$pkgver"
#	make -k check
#}

package() {
	cd "$srcdir/$pkgname"
	python setup.py install --root="${pkgdir}/"
}

When "python setup.py install" is run as standalone, it installs all required dependencies from pip, as mentioned in setup.py file.
But here in package() , only the tool is being installed in pkg/ directory. Dependencies are not being installed.

How can I resolve pip dependencies?

Thank you.

Last edited by kryptxy (2017-08-06 20:15:52)

mpan · 2017-08-06 08:12:47

Unless one searches for problems, one should not install anything¹ without pacman. Since pacman requires a package file, one has to make one.

Just list “python{2,3}-tabulate” among your dependencies and also manage such package in AUR (separately). Also see that thread.

A side note: it would be wiser to explicitly specify if you use python2 or python3.

____
¹ Except if you do so in “/usr/local”, but then don’t expect non-local things to work with it nicely. Plus the package manager should not touch “/usr/local”, so it’s out of discussion in this topic.

Last edited by mpan (2017-08-06 08:13:41)

kryptxy · 2017-08-06 08:50:31

Using python3.

Tabulate is already present in AUR (python-tabulate). Can you please elaborate Just list “python{2,3}-tabulate” among your dependencies and also manage such package in AUR (separately).
If I mention python-tabulate in dependencies, I get dependency error when building (which is obvious).

This means tabulate needs to be installed separately from AUR right?

Also, one thing I don't understand is why doesn't python setup.py install install dependencies mentioned in setup.py file? When run as standalone, it installs all dependencies.

Lone_Wolf · 2017-08-06 14:36:53

This means tabulate needs to be installed separately from AUR right?

yes.

Also, one thing I don't understand is why doesn't python setup.py install install dependencies mentioned in setup.py file? When run as standalone, it installs all dependencies.

makepkg doesn't use the user envirnoment to build, build supplies a different one.
setup.py might detect this and switch from standalone install to systemwide install.
Look into setup.py if you want to be sure.

eschwartz · 2017-08-06 15:28:14

1) Arch users are expected to install AUR dependencies for their AUR packages. If users get dependency errors and don't know what to do, this means nothing other than that they do not know how to use the AUR, which is not something that should be fixed by the PKGBUILD.

2) setup.py defines the needed dependent python modules via setuptools metadata. pip auto-installs modules while resolving their dependencies through PyPI. It's not quite the same thing...
EDIT: Possibly also worth noting that setuptools itself will act like pip does in pulling dependencies, but *only* if you do not install with --root which is typcially used for distro packaging purposes where autoinstalled dependencies are anathema.

Last edited by eschwartz (2017-08-06 15:47:03)

Awebb · 2017-08-06 15:39:33

kryptxy wrote:

Or should I explicitly mention user to install tabulate from AUR?

By putting the package into the dependency array, you already explicitly mentioned the package to AUR users. Everybody uses their own dependency resolution methods, ranging from manually resolving all non-repo deps to PKGBUILD parsers.

eschwartz · 2017-08-06 15:51:35

Lone_Wolf wrote:

makepkg doesn't use the user envirnoment to build, build supplies a different one.
setup.py might detect this and switch from standalone install to systemwide install.
Look into setup.py if you want to be sure.

makepkg doesn't define its own environment, what do you think we are, Debian?
What a filthy idea.

Setuptools is intelligent enough to recognize when you are installing to a DESTDIR (or --root in this case), and switches to "distro packaging mode".

...

What do you even mean by "standalone install to systemwide install"? pip and setuptools usually install to the system, the exception is when you use a PKGBUILD...

Lone_Wolf · 2017-08-06 16:05:50

You really mean there are installers that don't have a user install option that's chosen by default and very hard to convince to install systemwide ?

/me has spend to much time with proprietary stuff on that other OS lately.

In that case, i hereby apologize to python devs.
I'm very glad you wrote a package manager for python instead of an installer.

As for makepkg not defining it's own environment :
If you want to convince me that makepkg uses the exact same environment as the user it was started as in everything except the package() function,
go ahead.

eschwartz · 2017-08-06 16:10:14

I don't really care about convincing you.

But I've read the makepkg source code, and I am certainly able to assert that other than e.g. exporting the CFLAGS defined in makepkg.conf it is pretty hands off about your environment. If you want rigidly-defined build environments, use makechrootpkg. And that doesn't even define garbage "helpful" environment stuff either, it just runs in a chroot without your preexisting environment.

Aside: I'd actually prefer if pip *did* do `pip install --user` as default, rather than encouraging people to scribble all over / with sudo...

Last edited by eschwartz (2017-08-06 16:14:48)

kryptxy · 2017-08-06 17:41:02

Thank you all for helping me out. Package dependencies issue has been resolved

Here is the final PKGBUILD:

# Maintainer: kryptxy <kryptxy@protonmail.com>

pkgname=torrench
pkgver=1.0.1.20160806
pkgrel=1
pkgdesc="Command-line torrent search tool"
arch=('any')
url="https://github.com/kryptxy/torrench"
license=('GPL')
depends=("python" "python-beautifulsoup4" "python-lxml" "python-requests" "python-termcolor" "python-tabulate")
makedepends=("python-setuptools")
provides=("torrench")
conflicts=("torrench")
source=('git+https://github.com/kryptxy/torrench.git')
md5sums=('SKIP')

build() {
	cd "$pkgname"
	python setup.py build	
}

package() {
	cd "$srcdir/$pkgname"
	python setup.py install --prefix=/usr --root="${pkgdir}" --optimize=1
}

Hope this is fine? Builds and installs correctly.

eschwartz · 2017-08-06 17:49:24

https://wiki.archlinux.org/index.php/Py … setuptools

You use console_scripts, therefore setuptools is a runtime requirement...
That being said, you depend on python-requests ==> python-chardet ==> python-setuptools anyway.

Also, it isn't strictly necessary to list both python and python-$modulename as depends, since python-$modulename will itself depend on python. But there is nothing wrong with it either.

Your package should not clone from git, rather it should download the archive from "${pkgname}-${pkgver}.tar.gz::${url}/archive/v${pkgver}.tar.gz".
Alternatively you can create a torrench-git package that follows the https://wiki.archlinux.org/index.php/VC … guidelines

--prefix is unnecessary, packages do not need to provide and conflict themselves, and it is a bit odd to see you use $var most of the time, then use ${pkgdir} so please stick to one style throughout.

Last edited by eschwartz (2017-08-06 17:52:01)

Slithery · 2017-08-06 17:50:16

As you are pulling the source from the current git master then the package name needs to be torrench-git and you need to include a pkgver function.

https://wiki.archlinux.org/index.php/VC … guidelines

kryptxy · 2017-08-06 18:28:08

Thanks.

Here:

# Maintainer: kryptxy <kryptxy@protonmail.com>

pkgname=torrench
pkgver=1.0.0.20170806
pkgrel=1
pkgdesc="Command-line torrent search tool"
arch=('any')
url="https://github.com/kryptxy/torrench"
license=('GPL')
depends=("python-setuptools" "python-beautifulsoup4" "python-lxml" "python-requests" "python-termcolor" "python-tabulate")
provides=("torrench")
conflicts=("torrench")
source=("${pkgname}-${pkgver}.tar.gz::${url}/archive/v${pkgver}.tar.gz")
md5sums=('SKIP')

build() {
	cd "${pkgname}-${pkgver}"
	python setup.py build	
}

package() {
	cd "${srcdir}/${pkgname}-${pkgver}"
	python setup.py install --root="${pkgdir}" --optimize=1
}

Since I am not pulling from master, do I need to include pkgver() ?

eschwartz · 2017-08-06 18:29:46

Nope, it's an either or. Which is why I said "alternatively you can make a torrench-git package".

kryptxy · 2017-08-06 18:37:26

Yes okay thanks

kryptxy · 2017-08-06 18:43:05

Eschwartz wrote:

2) setup.py defines the needed dependent python modules via setuptools metadata. pip auto-installs modules while resolving their dependencies through PyPI. It's not quite the same thing...
EDIT: Possibly also worth noting that setuptools itself will act like pip does in pulling dependencies, but *only* if you do not install with --root which is typcially used for distro packaging purposes where autoinstalled dependencies are anathema.

Eschwartz wrote:

What do you even mean by "standalone install to systemwide install"? pip and setuptools usually install to the system, the exception is when you use a PKGBUILD...

It's still not clear to me why ``python setup.py install`` didn't install dependencies
`python setup.py install` installs the torrench package in pkg/ but skips the install_requires.
Is this because of absence of system-wide environment?

eschwartz · 2017-08-06 19:33:36

You even quoted the second paragraph of my post where I explained that setuptools itself does not install dependencies when --root is used.

kryptxy · 2017-08-06 20:15:32

Got it. Thanks

Arch Linux

#1 2017-08-06 07:27:48

[Solved] Resolve dependencies using pip in AUR package?

#2 2017-08-06 08:12:47

Re: [Solved] Resolve dependencies using pip in AUR package?

#3 2017-08-06 08:50:31

Re: [Solved] Resolve dependencies using pip in AUR package?

#4 2017-08-06 14:36:53

Re: [Solved] Resolve dependencies using pip in AUR package?

#5 2017-08-06 15:28:14

Re: [Solved] Resolve dependencies using pip in AUR package?

#6 2017-08-06 15:39:33

Re: [Solved] Resolve dependencies using pip in AUR package?

#7 2017-08-06 15:51:35

Re: [Solved] Resolve dependencies using pip in AUR package?

#8 2017-08-06 16:05:50

Re: [Solved] Resolve dependencies using pip in AUR package?

#9 2017-08-06 16:10:14

Re: [Solved] Resolve dependencies using pip in AUR package?

#10 2017-08-06 17:41:02

Re: [Solved] Resolve dependencies using pip in AUR package?

#11 2017-08-06 17:49:24

Re: [Solved] Resolve dependencies using pip in AUR package?

#12 2017-08-06 17:50:16

Re: [Solved] Resolve dependencies using pip in AUR package?

#13 2017-08-06 18:28:08

Re: [Solved] Resolve dependencies using pip in AUR package?

#14 2017-08-06 18:29:46

Re: [Solved] Resolve dependencies using pip in AUR package?

#15 2017-08-06 18:37:26

Re: [Solved] Resolve dependencies using pip in AUR package?

#16 2017-08-06 18:43:05

Re: [Solved] Resolve dependencies using pip in AUR package?

#17 2017-08-06 19:33:36

Re: [Solved] Resolve dependencies using pip in AUR package?

#18 2017-08-06 20:15:32

Re: [Solved] Resolve dependencies using pip in AUR package?

Board footer