Splitting package

Tahvok · 2017-02-15 13:13:51

Hello,

I'm maintaining the following package: https://aur.archlinux.org/pkgbase/gridcoinresearch/
Which builds 2 separate packages: gridcoinresearch-daemon and gridcoinresearch-qt

Some users requested that I split the package into to. One package does *not* require the other, you either run one or the other. So there really is no reason to build 2 packages with one PKGBUILD.

Is it possible to rename the remote git repo? Currently as I have one PKGBUILD, I have the following repo: aur.archlinux.org/gridcoinresearch.git - I would like to rename it to aur.archlinux.org/gridcoinresearch-qt.git which will build only gridcoinresearch-qt and I will create a new AUR package which will build gridcoinresearchd (not -daemon as before - as I would like to change the name as well). So I will have 2 separate PKGBUILDS - gridcoinresearch-qt and gridcoinresearchd

Is it even possible? If not, how would you suggest to proceed with the split?

Slithery · 2017-02-15 13:19:00

The forums aren't the correct place for this, you need to discuss on the AUR mailing list then file a merge request using the AUR web interface.

Trilby · 2017-02-15 13:45:06

I don't see a reason to worry about renaming. Just create your two new package repos and delete the old one. Or you could leave the old one around for a bit with comments indicating users should switch to the other package(s).

Tahvok · 2017-02-15 18:07:00

slithery wrote:

The forums aren't the correct place for this, you need to discuss on the AUR mailing list then file a merge request using the AUR web interface.

according to the wiki: https://wiki.archlinux.org/index.php/Ar … Repository
The forums are one of the places for asking this questions..

Anyway, I am going to follow @Trilby suggestion, and just create 2 new packages.

eschwartz · 2017-02-16 00:12:12

Tahvok wrote:

Hello,
I'm maintaining the following package: https://aur.archlinux.org/pkgbase/gridcoinresearch/
Which builds 2 separate packages: gridcoinresearch-daemon and gridcoinresearch-qt
Some users requested that I split the package into to. One package does *not* require the other, you either run one or the other. So there really is no reason to build 2 packages with one PKGBUILD.
Is it possible to rename the remote git repo? Currently as I have one PKGBUILD, I have the following repo: aur.archlinux.org/gridcoinresearch.git - I would like to rename it to aur.archlinux.org/gridcoinresearch-qt.git which will build only gridcoinresearch-qt and I will create a new AUR package which will build gridcoinresearchd (not -daemon as before - as I would like to change the name as well). So I will have 2 separate PKGBUILDS - gridcoinresearch-qt and gridcoinresearchd
Is it even possible? If not, how would you suggest to proceed with the split?

Except that this really is exactly the case where split packages are supposed to be used.

Tell your users that they don't have to install both of the created packages...

Trilby · 2017-02-16 00:27:32

This is an ambiguous case. Split packages are for when there is one source repo, and one build, but the results of the build can be in two different packages.

Here there are two separate build processes.

eschwartz · 2017-02-16 03:35:30

Not really. It still uses the same source archive and has a tightly-coupled package version.

All the PKGBUILDs I have seen do that as split packages, regardless of whether the build process allows them to be built entirely separately or not.

Tahvok · 2017-02-16 11:09:59

Eschwartz wrote:

Tahvok wrote:
Hello,
I'm maintaining the following package: https://aur.archlinux.org/pkgbase/gridcoinresearch/
Which builds 2 separate packages: gridcoinresearch-daemon and gridcoinresearch-qt
Some users requested that I split the package into to. One package does *not* require the other, you either run one or the other. So there really is no reason to build 2 packages with one PKGBUILD.
Is it possible to rename the remote git repo? Currently as I have one PKGBUILD, I have the following repo: aur.archlinux.org/gridcoinresearch.git - I would like to rename it to aur.archlinux.org/gridcoinresearch-qt.git which will build only gridcoinresearch-qt and I will create a new AUR package which will build gridcoinresearchd (not -daemon as before - as I would like to change the name as well). So I will have 2 separate PKGBUILDS - gridcoinresearch-qt and gridcoinresearchd
Is it even possible? If not, how would you suggest to proceed with the split?
Except that this really is exactly the case where split packages are supposed to be used.
Tell your users that they don't have to install both of the created packages...

Well, that is the problem. Both packages has a separate build process, and the problem is that on slow machines the build of both packages can take up to an hour. I heard some SoC devices can take a few hours...
Having 2 separate PKGBUILDS, will reduce the build process by half, as people don't need both of them.

@Eschwartz can't this be enough to justify the separation?

Lone_Wolf · 2017-02-16 14:42:52

There are simple ways to prevent duplicate downloads outside of PKGBUILDs.

If both packages share the same source but have separate build processes, my personal choice would be 2 separate packages.

eschwartz · 2017-02-16 15:01:50

Tahvok wrote:

Well, that is the problem. Both packages has a separate build process, and the problem is that on slow machines the build of both packages can take up to an hour. I heard some SoC devices can take a few hours...
Having 2 separate PKGBUILDS, will reduce the build process by half, as people don't need both of them.
@Eschwartz can't this be enough to justify the separation?

Hmm, maybe. Just because something isn't "more correct" doesn't mean you can't do it anyway, it just means you should take into serious consideration why you want to do it.

Does the average gridcoinresearch user only want one or the other?
If not, then splitting it has no real gain (just for a minority of users???).
If yes, and a significant number of people are using older machines that simply take way too long to build the extraneous package, then I guess practicality would trump correctness. You'll have to manage updates to 2 PKGBUILDs for the same update though, and thereby complicate both maintenance and end-user tracking... which is why it should never be your first choice.

eschwartz · 2017-02-16 15:03:10

Lone_Wolf wrote:

There are simple ways to prevent duplicate downloads outside of PKGBUILDs.

That is not even what I was talking about, but okay.

Lone_Wolf wrote:

If both packages share the same source but have separate build processes, my personal choice would be 2 separate packages.

I don't have the actual numbers, but by that logic I suppose the significant minority, perhaps even the majority, of split packages should not exist? For example, the entire python ecosystem uses split packages to build completely independent python2/python3 versions of modules.

Trilby · 2017-02-16 15:20:35

Gcc makes 6 packages, but there is only one build command (well 2, one for documentation) and all resulting packages need that same build. The results of that build are separated into 6 packages. I can't think of a single example that has multiple builds. You refer to the whole python ecosystem: can you give a specific example? I just checked python-pygments: that is a split package for python 2/3 but there is no unneccessary build for either one, in fact there is no build function at all - the same is true of python-dnslib.

Python-jinja seems to have a 2 "builds", but it's not actually compiling anything.

eschwartz · 2017-02-16 15:30:11

That is because `setup.py build` is an internal setuptools dependency of `setup.py install` (in much the same way that most Makefiles have `make install` a dependency of `make (all|default|first|build)`), and it runs during package(). Many `./configure; make; make install` PKGBUILDs can have their build() functions removed without ill effect too.

Regardless, the fact that there is no build function could only reinforce my point; making separate PKGBUILDs as a matter of policy for anything which *doesn't* explicitly use a unified build process (thus forcing you to do all the work for both packages even if you only want one of them), very much goes against the community standards (and in fact, the intention behind split packages).

Trilby · 2017-02-16 15:48:32

I'm not referring to build functions, I'm referring to build processes. With all split packages I'm familiar with (e.g. gcc) if you were to build any one of the 6 split components, you've basically built them all. The incremental ammount of work for actually bundling up those other 5 packages is trivial, while the duplicated work if they were built separately is massive.

The case in this thread is the opposite. The shared work between the two resulting packages is *only* the source download. The build->package process diverges immediately after the download. So the incremental ammount of work to build the second unneed package is substantial, and the duplicated work if they are separate is virtually nonexistent.

I'm still looking for a split package that has a split build process, but am not finding any.

eschwartz · 2017-02-17 15:28:17

So the incremental ammount of work to build the second unneed package is substantial, and the duplicated work if they are separate is virtually nonexistent.

Yes, I did say that on balance this seems to be a case where multiple PKGBUILDs is warranted...

Also, not really sure what you think is the difference between a build function and a build process, where either one can be implied or explicit. They serve the same role, and it is work that must be done either way.

(Is the problem here, the perception of "building" as "anything that involves the GNU Compiler Collection"? Okay, fine. Look at, say, python{,2}-regex.)

Scimmia · 2017-02-17 15:33:16

Trilby wrote:

I'm still looking for a split package that has a split build process, but am not finding any.

I could give you dozens.

https://git.archlinux.org/svntogit/comm … /doublecmd
https://git.archlinux.org/svntogit/comm … galculator
https://git.archlinux.org/svntogit/comm … ages/libfm

For a start

Scimmia · 2017-02-17 15:39:47

Eschwartz wrote:

Not really. It still uses the same source archive and has a tightly-coupled package version.
All the PKGBUILDs I have seen do that as split packages, regardless of whether the build process allows them to be built entirely separately or not.

This doesn't make it right. A PKGBUILD is built around a single build, which is why there's a single build function. It doesn't matter if two packages use the same source, they should only do one build. Copying the source to build twice is a nasty hack that people use to reduce maintenance; it's not right, but it will work.

Trilby · 2017-02-17 15:56:12

Thanks for the examples Scimmia, those are fitting examples. Eschwartz, the difference between a build process and a build function was my attempt to deal with what felt like a semantic argument. You were saying that building might actually happen in the build or package function - and notwithstanding the problems of that itself, I wanted to highlight that that point was tangential to my arguement: PKBGUILDs that I was familiar with and that you had given examples of, did not have invocations of `make` or the equivalent that would be unnecessary for some of the split packages.

This is why I gave the example of GCC. Even if you only need gcc-fortran, there are no invocations of `make` that are solely for packages other than gcc-fortran*. Scimmia's examples are the suitable counter: there is an example there that builds a qt version, then separately builds a gtk version. In that case, if you only want the qt version, you are forced to do a superflouous build.

In my opinion, I'd prefer a different approach were taken (where split packages were just those more like gcc) but given the above examples, there certainly is precident for keeping it all in one PKGBUILD. If I'm understanding correctly, the former seems to be what Scimmia is advocating for in his second post as well.

EDIT: *to be fair, there might be configure options that would streamline the process a bit if one really only did need one of the resulting packages. But there is nothing like:

cd gcc-fortran
make
cd ../gcc-libs
make
cd ../gcc-other
make

There is one build that produces results which can be packaged separately. To me that is what a split package is for.

Lone_Wolf · 2017-02-18 11:09:52

Trilby wrote:

There is one build that produces results which can be packaged separately. To me that is what a split package is for.

I completely agree with that.

@Tahvok :
as can you see opinions on this differ, i suggest you use what feels best for you.

Tahvok · 2017-02-19 12:29:00

Thanks for the great discussion here everyone!

I'm going to split the package into two, as it would ease on the users of the package.

The end result of the split would be the same as current packages, only one package will be renamed, but it will ship the same file.
So, do I place `replaces` on the new packages? Or conflicts? Or ignore the old packages completely?

Trilby · 2017-02-19 13:47:15

I'd only use 'replaces' if you were going to create two new packages and (eventually) delete the current one. One of the two new packages should be the "default" that 'replaces' the old one, and both new packages should 'conflict' with each other.

But it sounds like you are only creating a new one and revising the current one. In that case 'replaces' should not be used. The current package will be revised to only make the one that you feel is a reasonable default. The new package will make the other version. Still both of these should 'conflict' with each other.

Arch Linux

#1 2017-02-15 13:13:51

Splitting package

#2 2017-02-15 13:19:00

Re: Splitting package

#3 2017-02-15 13:45:06

Re: Splitting package

#4 2017-02-15 18:07:00

Re: Splitting package

#5 2017-02-16 00:12:12

Re: Splitting package

#6 2017-02-16 00:27:32

Re: Splitting package

#7 2017-02-16 03:35:30

Re: Splitting package

#8 2017-02-16 11:09:59

Re: Splitting package

#9 2017-02-16 14:42:52

Re: Splitting package

#10 2017-02-16 15:01:50

Re: Splitting package

#11 2017-02-16 15:03:10

Re: Splitting package

#12 2017-02-16 15:20:35

Re: Splitting package

#13 2017-02-16 15:30:11

Re: Splitting package

#14 2017-02-16 15:48:32

Re: Splitting package

#15 2017-02-17 15:28:17

Re: Splitting package

#16 2017-02-17 15:33:16

Re: Splitting package

#17 2017-02-17 15:39:47

Re: Splitting package

#18 2017-02-17 15:56:12

Re: Splitting package

#19 2017-02-18 11:09:52

Re: Splitting package

#20 2017-02-19 12:29:00

Re: Splitting package

#21 2017-02-19 13:47:15

Re: Splitting package

Board footer