Suggestion for new package management scheme

contrasutra · 2004-05-01 04:09:40

As was talked about in the INCOMING thread, Arch is growing and the devs are having trouble keeping up with maintaining packages. Arch is becoming disorganized and messy.

Well, Xentac and I were discussing this for a while, and we both seem to like a voting system. Arch shouldn't try to maintain every package out there, it's simply a waste of effort. If only one or two people want a package, they can build it themselves, it's not hard with ABS.

Arch devs should focus on keeping packages high quality, up to date, etc. They can't do this if there's 10,000 packages. So the solution would be that a package won't be included in the official repos (current, extra, unstable) unless it gets a certain number of votes, showing that lots of people really want the package, and it's "essential".

I don't know how many votes a package would need, but that's a little thing. I would suggest all packages in Extra/Unstable initially go up for this voting (once we implement it) and we make the vote quota be a little lower for this initial slimming, and that'll cut down the repos right there. All those packages that no one uses (that wont even get one vote) can be cut away. People can always vote them back if they become popular.

I don't want to have to do things like KDE, GNOME, etc myself, but I really don't mind maintaining my own little customizations, like Rhythmbox w/ XINE support (instead of Gstreamer). This vote system simply makes sure that a package is worth the developers time.

This goes with my idea of Arch as the perfect "Base" distro. People can build what they want on top of it.

This idea would also remove STAGING, as Arch wouldn't be accepting outside packages (though you're free to run your own repo or post PKGBUILDs on the forum). TESTING would be still used of course.

Suggestions?

rasat · 2004-05-01 04:37:58

The vote should start from the incoming packages. A voted package goes to TUR. With more votes including TU's recommendation, moves to stagging. Popular / usefull stagging packages become official (extra/current).

contrasutra · 2004-05-01 04:49:09

There would be no INCOMING, STAGING or TU(R)s. Package acceptance isn't open.

It would be the devs job to make the PKGBUILDs, but they only have to do that once a package gets enough votes. And the package would go into TESTING for a while before it gets into extra/current/unstable.

Though people are welcome to put PKGBUILDs on the forum/mailing list, or run their own repos. But all supported packages should be well tested and up to date (that's why I use a binary distro after all), and that can't happen w/ 10,000 packages.

Xentac · 2004-05-01 06:02:06

How is it that you can take an ok idea and masacre it like that?

I think that most packages would stay in the repos, but a method of removing them should be defined (voting might work for that).

Incoming, staging, etc would be gone. The TURs would probably stay, but only to serve purpose 2 (read the page if you don't know what that is).

I saw a new submittal system that stored PKGBUILDs, let people reference them (and was thus easier to organize), and also to vote for them. Once a package got enough votes it would be added to extra.

contrasutra · 2004-05-01 06:14:57

I saw a new submittal system that stored PKGBUILDs, let people reference them (and was thus easier to organize), and also to vote for them. Once a package got enough votes it would be added to extra.

Well, my idea kind of got messed up in my crappy explanation. This is basically what I want too. Some sort of PKGBUILD submission scheme w/ a voting mechanism. And please don't say the AIR. :

I just think the idea should change from "include as many packages as we can in Extra" to "Include only popular enough packages that warrent a dedicated maintainer".

Eh, I was just throwing ideas out there (its really late). Its just that it makes sense to slim down Extra a little if we go through with this. Im not talking about removing 50% of the packages or anything, but it can't hurt to remove some of the dead or really niche apps.

So yeah, I still want to keep Arch community driven, in fact I want it more community driven. I want to ease the burden on the Arch developers, so they can focus on creating a really nice "core" or "base", and let other people add stuff on (though the core would still have pkgs like kde, et al, so it's not really that minimal).

Dusty · 2004-05-01 06:40:45

contrasutra wrote:

I just think the idea should change from "include as many packages as we can in Extra" to "Include only popular enough packages that warrent a dedicated maintainer".

Excellent! Exactly! Perfect.

This is how we Keep It Simple!

Have these cleaned up repositories. Have a level for TURs to maintain selected unsupported packages. Finally have a section where users can share PKGBUILDs (even just wiki). This last section can be (even should be) quite messy. Get the mess out of incoming and away from the devs. However, a mechanism would have to be in place so they know when to move packages from this mess to TUR or even extra.

Dusty

i3839 · 2004-05-01 13:52:17

No matter what the final construction will be, it would be nice if all custom, non-official packages + pkgbuild are centralized so that they can be found easily. Otherwise they will be scattered around on the net on personal webpages. Something like the current TUR, but totaly independent from the official package system. Otherwise no one will probably contribute pgkbuilds anymore, because there is simply no place for them (and why bothering to make a pgkbuild if ./configure; make; make install does the trick?). If you switch to a closed package system then at least give people another way of sharing their packages.

Zephirias · 2004-05-01 13:57:04

It makes sense. Keep what's most important updated the most frequently, and anything else can just go to EXTRA or whatnot.

beniro · 2004-05-01 15:32:08

I'm glad to see that this issue is being addressed. This idea sounds great if done correctly and I hope that it will bring Arch's quality back up to what it always has been, except for these past few rough weeks.

Arch is just growing and some sort of infrastructure needs to be put in place to guide ARch through this stage of growth and the idea proposed above sounds great!

I started a thread last night about the need for some way to avoid installing buggy packages and I think this solution may be better: don't have any buggy package!

Zephirias · 2004-05-01 15:59:37

Yeah, but then you'd need to do a lot more testing.

contrasutra · 2004-05-01 16:28:53

OK, here's an idea:

We bring back an "unofficial" repository (and remove the TURs, purpose 2 isn't worth it IMO).

The unofficial repo will ONLY have PKGBUILDs(& .installs, etc), and will be available via ABS (hosted by Archlinux). There are no vote requirements to get in, but there's no guarantee of correctness or up-to-dateness. Its something the devs do on their free time.

But because nothing needs to be rebuilt in it, it'll still enable niche apps to be easily available, but the devs dont have to spend nearly as much time on them.

Basically, we want to make sure PACKAGES are well built, so we only support a smaller number of packages. But PKGBUILDs aren't nearly as big of a deal. And yes, this idea is similar to Gentoo's new system, but presumably we'll have to modify it to fit Arch.

EDIT: The PKGBUILD voting/submitting system (whatever it will be) still goes along with this. This idea is in addition to help compromise with people who don't like building packages as much as others.

Here's the problem:

No easy way to update to new versions from ABS if you have more than a few pkgs from Unofficial. I guess this'll require some changing of pacman, so consider this a "far out" idea.

Vision:

I see Arch's official pkgs as a "core" that people customize on top of (Ive said this before). So this kind of thing would encourage people to make custom versions, but instead of doing the usual forking, they could do it in the form of a repository. Because they know what the base packages that are available to people are, and that they'll be stable and well built (and up to date), they could easily make a [mythtv] or [router] or [pro_audio] repo that'll transform the core into whatever they want.

kakabaratruskia · 2004-05-01 18:25:24

I think arch can handle having 10.000 packages, it's just a matter of time. FreeBSD has tons of packages an ports, and I think maybe the repositories should grow slowier, just as the community and support grows, but they should never become smaller.
It's just my opinion...

Zephirias · 2004-05-01 18:45:55

kakabaratruskia wrote:

I think arch can handle having 10.000 packages, it's just a matter of time. FreeBSD has tons of packages an ports, and I think maybe the repositories should grow slowier, just as the community and support grows, but they should never become smaller.
It's just my opinion...

It's not that it couldn't handle the packages, it's just the amount of work to maintain them. That means that you'd need to find more developers, but that also leads to more people starting to use Arch. Though the stability factor does remain important, and the quality, too. FreeBSD does do good work maintaining everything, it's just that you need quite a large amount of people to do so.

kakabaratruskia · 2004-05-01 19:36:18

That's exactly what I mean, sorry if I didn't explain it fine. I think the repositorioes should grow just as the community does, and as the number of developers do. Maybe it's growing to fast now, but I don't think it should cease growing. Maybe it should be just a little slower.

Dusty · 2004-05-01 20:42:16

I'm feeling annoyed so maybe this isn't in the best taste...

I think the community should be closed off. Nobody else can use Arch besides those who already use it, anybody who complains (suggestions are good, complaints are not) should not be allowed to use Arch, and the package base should be made smaller and stable. No need for incoming, TURS, unofficial. etc.

:evil:

Dusty

Zephirias · 2004-05-01 20:49:11

Dusty wrote:

I'm feeling annoyed so maybe this isn't in the best taste...
I think the community should be closed off. Nobody else can use Arch besides those who already use it, anybody who complains (suggestions are good, complaints are not) should not be allowed to use Arch, and the package base should be made smaller and stable. No need for incoming, TURS, unofficial. etc.
:evil:
Dusty

What do you mean, like a lockdown to a closed community? :?

Note that I haven't complained at all, I'm just stating what needs to happen for certain things other people mentioned to happen.

Dusty · 2004-05-01 21:25:08

I wasn't complaining about you, it was a hint of irony that this discussion is still going nowhere fast and has been since long before I joined here...

I think a lockdown would be perfect. A secret society where nobody knows what OS we use. We could all brainwash ourselves to like only the packages that apeiro likes; that way we wouldn't need extra, incoming, or anything like that, just base and current. it has promise...

Dusty

Zephirias · 2004-05-01 21:32:20

Heh heh, I could picture that...like a small cyber-city.

Though I think more people would be better...maybe a couple more thousand or so, just so there's more talk around here.

contrasutra · 2004-05-01 21:33:04

Dusty wrote:

I wasn't complaining about you, it was a hint of irony that this discussion is still going nowhere fast and has been since long before I joined here...
I think a lockdown would be perfect. A secret society where nobody knows what OS we use. We could all brainwash ourselves to like only the packages that apeiro likes; that way we wouldn't need extra, incoming, or anything like that, just base and current. it has promise...
Dusty

You obviously misread something in this thread.

This isn't about picking only the packages that apeiro likes, this is about making sure the developers only have to maintain "popular" (relative term) packages. There's nothing stopping you from getting other, odd apps, through the "unofficial" idea. It's just that binary packages SHOULD be high quality, and this makes it easier.

This isn't about being a closed society, it's about knowing Arch's purpose, and not just trying to recreate the efforts of Debian or Gentoo. Seriously, do you really think we can do it better than they can? They have people working FULL TIME, we're all volunteers.

Xentac · 2004-05-01 22:07:04

contrasutra wrote:

You obviously misread something in this thread.
This isn't about picking only the packages that apeiro likes, this is about making sure the developers only have to maintain "popular" (relative term) packages. There's nothing stopping you from getting other, odd apps, through the "unofficial" idea. It's just that binary packages SHOULD be high quality, and this makes it easier.

Dusty's point about apeiro's packages relates to the fact that current is Judd's favourite packages (one of each type). It's the way it is and the way it will probably be for a while.

beniro · 2004-05-01 22:23:35

I haven't really seen a single post in this thread that is without some merit, or at least an attempt at input.

The great thing about not being a larger distro is that the situation isn't as pressurized. Still, though, we (meaning the community AND developers) still obviously want Arch to be the best it can be and that's what this thread is about.

So...are any of the ideas mentioned here feasible? What do you think would be the best solution to creating a package release system that can handle Arch's current growth?

What about implementing a vote-based system for everything not in current? Would this be done with a web-based interface by using "submitrequest/vote" or something...obviously only registered users should be voting, right?

Anyway, I think a great job is being done and this thread is simply a positive indication of the vitality of the community. I think most of us fully recognize the resources of our developers and are simply wanting to find a way for them to bring us the best possible product. Arch rules!

Zephirias · 2004-05-01 23:06:24

beniro wrote:

I haven't really seen a single post in this thread that is without some merit, or at least an attempt at input.
The great thing about not being a larger distro is that the situation isn't as pressurized. Still, though, we (meaning the community AND developers) still obviously want Arch to be the best it can be and that's what this thread is about.
So...are any of the ideas mentioned here feasible? What do you think would be the best solution to creating a package release system that can handle Arch's current growth?
What about implementing a vote-based system for everything not in current? Would this be done with a web-based interface by using "submitrequest/vote" or something...obviously only registered users should be voting, right?
Anyway, I think a great job is being done and this thread is simply a positive indication of the vitality of the community. I think most of us fully recognize the resources of our developers and are simply wanting to find a way for them to bring us the best possible product. Arch rules!

I agree. I think what we should do is just take this time that we have while Arch is still young and not as well known among the Linux community, and try to make it great, relieving the pressures that we'd get a lot of later (not to say that we wouldn't get any, just less, hopefully).

As for the package setup, isn't there some way that we could write some system to do this? I had mentioned it before. And, I just came up with a small system. It would work like this:

We get one main server, whose "job" would be to create and store packages, and users could download from that server to install, etc. Now, whenever someone on the package management team finds an updated version of a package, they could log in to a special area of the site meant specifically for those developers. Then, using the special program on the website/server, they could link to the .tar or .bz2 that contains the source and submit it, along with the file's information that would be included with the package when you install it (file name, version, etc.) into the web-based form. Then, they click submit, and the server processes that file after downloading it, runs a makepkg-type thing on it, and stores the produced package in a "Fresh" repository. Then, the developer can download the file and opt to test it. If they're sure that it's okay, then they can replace the new file over the old one.

Just an idea, tell me what you think... :?

Dusty · 2004-05-01 23:54:24

Sorry about me earlier, that was my nasty half... no, about a 1/4, only 1/4 nasty. I'm improving.

Zephirias wrote:

Anyway, I think a great job is being done and this thread is simply a positive indication of the vitality of the community.

This thread is a recurring issue that should be solved... perhaps sooner or later it will be though.

As for the package setup, isn't there some way that we could write some system to do this? I had mentioned it before. And, I just came up with a small system.

It has merit. The problem with new systems is that its hard to reuse old components. ftp is such an elegant solution to many a problem...

Can we figure out what it is that people agree on so far, before proposing new systems?

a. there needs to be a way for users to submit packages
b. incoming isn't working

things people mostly agree on:
c. submission should only be package builds
d. packages should make it into official repositories based on some sort of popularity system such as counting dls or voting

things people disagree on:
e. how many official repositories there should be
f. how many unofficial repositories there should be
g. how many packages should be in official repositories

anybody wanna add to that?

Dusty

Zephirias · 2004-05-02 00:10:33

Well, I figure that we should just form a "Board" of Users that'll be like a miniature senate for Arch, and we vote on the big decisions, and the majority rules (or whenever we get to an agreement). However, we can also be influenced by the people, obviously, since they're most important.

I can come up with lots of ideas, so I'd be glad to be a part of the Board, should we decide to make one. I just think that organization is key in the success of Arch. The system itself has the potential to work; we just need to make that possibility become a reality.

Dusty · 2004-05-02 00:34:03

Zephirias wrote:

Well, I figure that we should just form a "Board" of Users that'll be like a miniature senate for Arch, and we vote on the big decisions, and the majority rules (or whenever we get to an agreement). However, we can also be influenced by the people, obviously, since they're most important.

They already have something like that, its called the Arch developers.

Arch Linux

#1 2004-05-01 04:09:40

Suggestion for new package management scheme

#2 2004-05-01 04:37:58

Re: Suggestion for new package management scheme

#3 2004-05-01 04:49:09

Re: Suggestion for new package management scheme

#4 2004-05-01 06:02:06

Re: Suggestion for new package management scheme

#5 2004-05-01 06:14:57

Re: Suggestion for new package management scheme

#6 2004-05-01 06:40:45

Re: Suggestion for new package management scheme

#7 2004-05-01 13:52:17

Re: Suggestion for new package management scheme

#8 2004-05-01 13:57:04

Re: Suggestion for new package management scheme

#9 2004-05-01 15:32:08

Re: Suggestion for new package management scheme

#10 2004-05-01 15:59:37

Re: Suggestion for new package management scheme

#11 2004-05-01 16:28:53

Re: Suggestion for new package management scheme

#12 2004-05-01 18:25:24

Re: Suggestion for new package management scheme

#13 2004-05-01 18:45:55

Re: Suggestion for new package management scheme

#14 2004-05-01 19:36:18

Re: Suggestion for new package management scheme

#15 2004-05-01 20:42:16

Re: Suggestion for new package management scheme

#16 2004-05-01 20:49:11

Re: Suggestion for new package management scheme

#17 2004-05-01 21:25:08

Re: Suggestion for new package management scheme

#18 2004-05-01 21:32:20

Re: Suggestion for new package management scheme

#19 2004-05-01 21:33:04

Re: Suggestion for new package management scheme

#20 2004-05-01 22:07:04

Re: Suggestion for new package management scheme

#21 2004-05-01 22:23:35

Re: Suggestion for new package management scheme

#22 2004-05-01 23:06:24

Re: Suggestion for new package management scheme

#23 2004-05-01 23:54:24

Re: Suggestion for new package management scheme

#24 2004-05-02 00:10:33

Re: Suggestion for new package management scheme

#25 2004-05-02 00:34:03

Re: Suggestion for new package management scheme

Board footer