You are not logged in.

#1 2017-08-17 13:53:04

WhittlesJr
Member
Registered: 2017-08-04
Posts: 6

Splitting python3 and python2 packages?

So whenever I've had to submit a python package, I've followed the convention of using one PKGBUILD and splitting it into two packages (python-* and python2-*).

However, one user has complained that his system only has python 3 installed, and doesn't like that he has to install python 2 to build the thing. I'm sympathetic to the problem, even though it doesn't impact me and I don't think it's a very typical case.

This made me question the convention. It's definitely easier to maintain but this is a real drawback. It's unfortunate that we can't split the build() step like we can split the package() step, and direct makepkg to build only one or the other.

I've seen similar discussions complaining about long build times for multiple versions of a given package, but never about python2 vs python3 specifically.

Here's the package and the thread: https://aur.archlinux.org/pkgbase/python-p4python/

Thoughts?

Offline

#2 2017-08-17 14:22:34

fsckd
Forum Fellow
Registered: 2009-06-15
Posts: 4,173

Re: Splitting python3 and python2 packages?

My thoughts run on a few possibilities.

You can make a second python2- package. That would be more work for you. This would be the way I would go.

You can have a variable in your PKGBUILD which when set enables building of a python2 package as well as a python one.

You can drop python2 altogether. This is not ideal.

You can make a feature request to allow split build(). This may have been done already and I expect it would be rejected.


aur S & M :: forum rules :: Community Ethos
Resources for Women, POC, LGBT*, and allies

Offline

#3 2017-08-17 14:32:33

eschwartz
Trusted User/Bug Wrangler
Registered: 2014-08-08
Posts: 2,853

Re: Splitting python3 and python2 packages?

Do not, under any circumstances, "submit a spinoff package", that is in violation of the AUR guidelines and will be deleted as soon as anyone notices it exists.

There are only two valid ways to handle this, either a split package as you are doing now or a python3-only package and a python2-only package.
And using a split package makes it easier to maintain both at once, makes it easier to install both if you need it, keeps AUR comments together, etc. The only downside, if it can actually be called a downside, is that e.g. users who don't have any python2-dependent software on their system *at all*, then need to install python2 in order to build it. But `makepkg -sr` will uninstall python2 afterwards...

As a general rule, Arch Linux users/package maintainers prefer split packages, to the point where separate packages tend to get merged rather than the other way around. wink That being said, I don't think anyone will actually arrest you for crimes against Arch Linux if you do separate packages...

The only time I have ever recommended separate packages for multiple packages that get built from a single source, is when someone made the convincing argument that the software in question took a long time to build *and* most users will not want both packages. Take that as you will.

EDIT: Or, well, yes. As fsckd says, you could essentially reimplement the Gentoo USE flags, and allow people to modify a variable at the top of the PKGBUILD that toggles on/off the python2 make/depends, pkgname=() and part of the build().
Which of course does nothing for the people who only have python2 and don't want python3. tongue

Last edited by eschwartz (2017-08-17 14:35:40)


Managing AUR repos The Right Way -- aurpublish (now a standalone tool)

Offline

#4 2017-08-17 14:56:24

WhittlesJr
Member
Registered: 2017-08-04
Posts: 6

Re: Splitting python3 and python2 packages?

Mm I see, thank you both for the input. So I'm definitely leaning towards "leave it as-is" at this point, and users not wanting python2 can run `makepkg -sr` afterwards. Not an ideal solution, sadly.

The extra flags seems to be a bit overkill, but I didn't realize it was an option previously, so thanks for pointing that out.

If vostok4 would like to chime in with any further points, he is welcome. Otherwise, here's what I'm willing to do:

  1. Leave it as a split package and continue to maintain it myself, or...

  2. Happily orphan the package and let vostok4 do the package separating and maintaining (or awkward build flag implementing) if he feels like it

Offline

#5 2017-08-17 15:15:43

Slithery
Forum Moderator
From: Norfolk, UK
Registered: 2013-12-01
Posts: 3,507

Re: Splitting python3 and python2 packages?

Tell the user to build in a clean chroot and then only install the package they want. No cluttering up the host system by installing python2 (it will get installed to the chroot but this is ephemeral anyway).


No, it didn't "fix" anything. It just shifted the brokeness one space to the right. - jasonwryan
Closing -- for deletion; Banning -- for muppetry. - jasonwryan

aur - dotfiles

Offline

#6 2017-08-17 15:25:26

WorMzy
Forum Moderator
From: Scotland
Registered: 2010-06-16
Posts: 9,184
Website

Re: Splitting python3 and python2 packages?

Clean chroots would also be my recommendation if the user simply doesn't want to "pollute" their host system (even transiently).

https://wiki.archlinux.org/index.php/De … ean_Chroot

Alternatively, they can simply modify the PKGBUILD themselves before building.


Sakura:-
Mobo: MSI X299 TOMAHAWK ARCTIC // Processor: Intel Core i7-7820X 3.6GHz // GFX: nVidia GeForce GTX 970 // RAM: 32GB (4x 8GB) Corsair DDR4 (@ 3000MHz) // Storage: 1x 3TB HDD, 5x 1TB HDD, 2x 120GB SSD, 1x 275GB M2 SSD

Making lemonade from lemons since 2015.

Offline

#7 2017-08-17 15:35:27

WhittlesJr
Member
Registered: 2017-08-04
Posts: 6

Re: Splitting python3 and python2 packages?

Ah yes, good old chroot. Excellent suggestion.

Offline

#8 2017-08-21 05:34:37

severach
Member
Registered: 2015-05-23
Posts: 192

Re: Splitting python3 and python2 packages?

I make all my python packages separate, not split. To make maintenance easier I use the same PKGBUILD for both. A sed script changes one variable which changes the whole PKGBUILD from python3 to python2. Since that wasn't good enough for me the same script also changes another variable to change the PKGBUILD to a git package. I modify only the python3 non git version and the script fixes and pushes all 3 versions. It's easy to maintain and everyone is happy except for those who don't think that's the simple way.

Offline

#9 2017-08-25 13:15:26

vostok4
Member
Registered: 2010-12-16
Posts: 70

Re: Splitting python3 and python2 packages?

Sorry, I was away for some bit and didn't have time to check in here.

The conversation is very interesting to me. I've been adopting a python3 only approach to all my development and haven't required python2 for some time, I naively guessed this would be the default case for new development but that's no excuse to force things to bend to my will smile

For me editing the PKGBUILD is OK as the final solution to this specific case.

Philosophically, I don't think that split package is a straightforward and clear ideology:
1) Every package in the official repos have separate packages, python- and python2-. This seems to me that the lack of a python2 requirement to have a python3 environment is wholly support (this makes complete sense). For users who never would install python3, they are happy to always use the python2 prefix.
2) The idea that AUR packages should build both is at odds with this. I'd be wasting cycles/bandwidth/power building a package I never required (and will not require or use in any possible way). The chroot suggesting is nice for keeping clean environments, but still embraces the idea that "its ok to do something completely unrequired".

My proposal would be that the AUR strives to maintain uniformity with the core-repo naming patterns: If I see a python- package, it means python3 to me. If I see a python2- package, it means python2. That does require two different packages, increasing the workload for the maintainer. Do the core packages (python/python2 variants) not have some tooling to improve maintenance overhead?

As said, I'd prefer to just keep editing the PKGBUILD to delete python2 references, as to me it is the lowest overhead and keeps the most efficient workflow.

Thank you for opening this thread, and appreciate all the insight to the underlying reasoning.

Offline

#10 2017-08-25 13:30:36

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 22,411
Website

Re: Splitting python3 and python2 packages?

vostok4 wrote:

1) Every package in the official repos have separate packages, python- and python2-.

2) The idea that AUR packages should build both is at odds with this.

1) False.  Those are almost always if not always from a single split-package PKGBUILD.  You don't see that as you only need to install the built package.  But to build any of those packages from the main repos, you'd need both python2 and python3.

2) Definitely false.  You may have good reasons to prefer python3-only, I'm totally with you there, but your reasoning here is faulty: AUR packages building both a python- and python2- version is not at all at odds with what is in the official repos as that is exactly what happens in the official repos.  If anything is "at odds with" the main repos it would be the idea of having separate PKGBUILDs or a python3-only PKGBUILD.

Personally I do think the AUR is sufficiently different from the repos that it is fair to assess whether a different approach is justified.  And (also personally) I'd prefer the python3-only approach.  But there is no way to justify these views by appealing to how packages in the official repos work: quite the opposite, one would need to argue that the AUR should be handled differently than the official repos.


"UNIX is simple and coherent..." - Dennis Ritchie, "GNU's Not UNIX" -  Richard Stallman

Online

#11 2017-08-25 13:40:45

eschwartz
Trusted User/Bug Wrangler
Registered: 2014-08-08
Posts: 2,853

Re: Splitting python3 and python2 packages?

vostok4 wrote:

1) Every package in the official repos have separate packages, python- and python2-. This seems to me that the lack of a python2 requirement to have a python3 environment is wholly support (this makes complete sense). For users who never would install python3, they are happy to always use the python2 prefix.
2) The idea that AUR packages should build both is at odds with this. I'd be wasting cycles/bandwidth/power building a package I never required (and will not require or use in any possible way). The chroot suggesting is nice for keeping clean environments, but still embraces the idea that "its ok to do something completely unrequired".

Well, building packages is different from installing them, and on the maintainer side, in the official repositories, one almost has to have a both a python3 and a python2 environment for possibly several reasons.

But again,

Eschwartz wrote:

`makepkg -sr` will uninstall python2 afterwards...

So you don't need a python2 environment, you just need a python2 makedepends environment -- and many packages have makedepends that aren't needed for running the package, so I don't think this is unusually burdensome. smile

If Arch was philosophically opposed to the idea of using resources to build a package you don't require/use, then why does Arch explicitly not cater to the idea of similarly preserving resources when installing packages? We automatically include development headers whether you plan to build packages or not, we include docs even though you probably won't read them and might be using the online docs anyway, compile in support for everything possible even if that means hard dependencies on a bunch of stuff...

Do the core packages (python/python2 variants) not have some tooling to improve maintenance overhead

Um, yes we do. The tooling is split packages. big_smile

...

Anytime I edit an AUR package before building, I clone it to /var/aur and then rebase my changes on top of the AUR  maintainer's changes. This makes it much less effortful to apply my own changes going forward.
If the patches won't cleanly apply, you can also override things using standard bash, e.g. depends=("${depends[@]//removeme/}") at the very end, or variable+=(), unset package_idontwantthis, etc.
Though extra package_* functions don't matter as long as you remove the name from the pkgname=() array.

Last edited by eschwartz (2017-08-25 13:43:34)


Managing AUR repos The Right Way -- aurpublish (now a standalone tool)

Offline

#12 2017-08-26 08:04:50

vostok4
Member
Registered: 2010-12-16
Posts: 70

Re: Splitting python3 and python2 packages?

And this is the benefit of open dialog smile I stand corrected about the official repo path, and in general am happy that the AUR does support client side modifications as a first class citizen, so I can avoid the extra build with a tiny bit of leg work.

Again, thank you for the insight to how things actually work, assumptions from the final view of built packages are not enough.

Offline

Board footer

Powered by FluxBB