Suggestion for new package management scheme

Zephirias · 2004-05-02 00:43:27

Yeah, well I specifically meant a general group, including document writers, package managers, and of course Judd and the rest of those people, so it's not one specific group that has an opinion.

Dusty · 2004-05-02 02:58:28

Zephirias wrote:

Yeah, well I specifically meant a general group, including document writers, package managers, and of course Judd and the rest of those people, so it's not one specific group that has an opinion.

"The developers" generally refers to document writers, package managers, and of course Judd and the rest of those people:

http://www.archlinux.org/people.php

Sections include Administration, Documentation, Packages, Programming, Release, and Security. That's "developers".

It doesn't include any 'normal" users, but I doubt anybody would be a developer that didn't *use* Arch.

Its ok for users to make suggestions and such, but ultimately Judd created Arch and the philosophy behind Arch. If just anybody could influence that philosophy without first knowing what it really is, Arch would be something very different.

But this is off topic. God help people who have to read my posts. *sigh*

Dusty

jak · 2004-05-02 03:31:18

Xentac wrote:

current is Judd's favourite packages (one of each type). It's the way it is and the way it will probably be for a while

I have no complaint about that. The barebones nature of arch is what appeals to me. As far as I care, you could drop extra altogether. Anything else I need, like sendmail for instance, I compile from source and install in /usr/local. More people should try /usr/local. That's why it's called LOCAL.

Dusty · 2004-05-02 03:39:37

jak wrote:

Anything else I need, like sendmail for instance, I compile from source and install in /usr/local. More people should try /usr/local. That's why it's called LOCAL.

That's sensible in some ways, but makes both pacman and abs quite unnecessary in your case. Updating stuff in /usr/local isn't much fun. If you're compiling from scratch, you lose the advantage of binary pacman packages; having to compile everything takes time. You have to sort out dependencies by ourself.

Wouldn't you be just as content using Linux From Scratch or Gentoo as Arch?

Dusty

rasat · 2004-05-02 03:50:29

Dusty wrote:

They already have something like that, its called the Arch developers.

This true. But Arch have indirectly established, for a practical reason, another system as well.

In Arch there is not only one side (devs) but also the "user contributions". Incoming, TUR, Wiki and posts in "Users Contributions" forum, all "belong" to the users. It has been agreed that devs are not obliged to make the contributions official.... up to the devs to decide. Also its upto users if they want to use official or non-official products. This style of cooperation / understanding is the base of Arch, thereby having a "prefect/high" quality nucleus with non-prefect but useful satellites.

Incoming is users' concern, not devs. The issue we are facing, devs have difficulties to coop with the package contributions, so how will the users fix it? Does it require working in a Borad.... I would prefer the "user contributions" side to remain less noticeable, indirect entity. But can it sustain as such without frustrating the users when no decision can be made?

Zephirias · 2004-05-02 03:56:21

This'll probably be my last post for the night. Anyway, I think a Board-type thing for the Packages would be a good idea, but we'd have to work out the details, though like I said, organization is key.

Later for now.

sweiss · 2004-05-02 06:04:08

To me it sounds as if Arch is slowly turning into a source-based distro.

I don't know, I don't think limiting packages is the solution really. As the community grows, the packages grow, and TUs are also being added.

So ok, the popular packages will remain popular and will recieve frequent upgrades and fixes. If a package is getting old/outdated/corrupted and it is not so popular, the damage is minimal anyway. This is why we have the bug tracking system, so everybody can have what they want.

I think the community always played an important part in anything related to Linux and open source, as it should, after all.

Zephirias · 2004-05-02 14:04:28

sweiss wrote:

To me it sounds as if Arch is slowly turning into a source-based distro.
I don't know, I don't think limiting packages is the solution really. As the community grows, the packages grow, and TUs are also being added.
So ok, the popular packages will remain popular and will recieve frequent upgrades and fixes. If a package is getting old/outdated/corrupted and it is not so popular, the damage is minimal anyway. This is why we have the bug tracking system, so everybody can have what they want.
I think the community always played an important part in anything related to Linux and open source, as it should, after all.

The community is important, though the problem isn't that, it's just that if we eventually get to a few thousand plus packages, it's going to start turning to hell to maintain, especially since this is a volunteer project, unlike that of other organizations. What I think we need to do, other than organize a Board that really gets a run on this project, is to go out on the web and start promoting this Distro. If we can get more users, some of them are bound to be able to program/develop, and we may get some more people that would be willing to help.

sweiss · 2004-05-02 14:11:47

I think that'll happen on its own as we advance. After all it has been lately, surely some developers will also appear.

contrasutra · 2004-05-02 14:29:12

What I want to know is why everyone seems to want to clone Debian and Gentoo. They have this huge, intricate maintainership system with super strict rules. Gen and Deb already overlap on lots of things, and now you guys want to just repeat their work AGAIN.

Arch isn't Debian/Gentoo and it really can never be. The key is to maximize our resources. We have very little, and we can't just get by through "throwing more man power at it".

1. A Message Board is not even a possible solution for this. Message boards are messy and inefficient. They don't do ANY of the things we need. We're probably going to need something like Bugzilla (if that can do voting in some way) or modify Archstats.

2. You guys seem to be forgetting that the developers are already overworked. Right now their load is too big (I can name a bunch of major pkgs that haven't received updates in months).

With a voting system, Arch can grow even without gaining many developers. If the developers are overworked, all they need to do is raise the voting requirement. So instead of needing 30 votes (example), a package would need 50.

3. Xentac and I also discussed having something like Crux's CLC. They're user made repositories (like the TURs). I think this would work if Arch hosted special "category" repositories (unofficial/unsupported). Someone could provide a [Pro Video] or [SVN Server] or [Games] Repository. Like I said earlier, Arch would maintain the "Core" (which remember, is still rather complete), and people would create add-on repositories that Arch could host, but not support.

Hell, these could be in the default pacman.conf, just commented out. It lets a person create a custom Arch workstation for whatever their needs just by running a pacman -Syu.

EDIT: Now this thread has been going around and around, so unless someone has any NEW ideas (I sure dont), this debate is going nowhere.

Zephirias · 2004-05-02 14:38:35

contrasutra wrote:

What I want to know is why everyone seems to want to clone Debian and Gentoo. They have this huge, intricate maintainership system with super strict rules. Gen and Deb already overlap on lots of things, and now you guys want to just repeat their work AGAIN.
Arch isn't Debian/Gentoo and it really can never be. The key is to maximize our resources. We have very little, and we can't just get by through "throwing more man power at it".

1. A Message Board is not even a possible solution for this. Message boards are messy and inefficient. They don't do ANY of the things we need. We're probably going to need something like Bugzilla (if that can do voting in some way) or modify Archstats.
2. You guys seem to be forgetting that the developers are already overworked. Right now their load is too big (I can name a bunch of major pkgs that haven't received updates in months).
With a voting system, Arch can grow even without gaining many developers. If the developers are overworked, all they need to do is raise the voting requirement. So instead of needing 30 votes (example), a package would need 50.
3. Xentac and I also discussed having something like Crux's CLC. They're user made repositories (like the TURs). I think this would work if Arch hosted special "category" repositories (unofficial/unsupported). Someone could provide a [Pro Video] or [SVN Server] or [Games] Repository. Like I said earlier, Arch would maintain the "Core" (which remember, is still rather complete), and people would create add-on repositories that Arch could host, but not support.
Hell, these could be in the default pacman.conf, just commented out. It lets a person create a custom Arch workstation for whatever their needs just by running a pacman -Syu.

EDIT: Now this thread has been going around and around, so unless someone has any NEW ideas (I sure dont), this debate is going nowhere.

I still think that my web-based idea could work, if gone about correctly...though the voting/bugzilla idea's sounding good now, too.

rasat · 2004-05-02 16:16:34

About the voting place, why not use "New Package" forum? Currently most of the incoming and TUR packages are already there. The poll questions can be added by the first author and moderators.

If the poll is added, then the forum will do all the suggestions presented: (1) announcement of new package, (2) poll, (3) feedback, and (4) resolution.

By the way, what are the poll questions?

Dusty · 2004-05-02 16:57:24

rasat wrote:

About the voting place, why not use "New Package" forum? Currently most of the incoming and TUR packages are already there. The poll questions can be added by the first author and moderators.
If the poll is added, then the forum will do all the suggestions presented: (1) announcement of new package, (2) poll, (3) feedback, and (4) resolution.
By the way, what are the poll questions?

This solution is super simple and super elegant!

ditch incoming. Store PKGBUILDS as text files in new packages board. If there are other files for the package, they are invariable text files that can also be stored in forum (ie: shell script to run the program).

Add poll to each package added:
Do you think this package should be added to the official repositories?
* yes
* no

A lot of people won't bother to vote no, so if yes becomes proprotionally high enough (choose a number. ) add it.

In the long run, it would be better to have this stuff done in a separate web area, as several people have suggested. In the forum, things can get messy for the moderators to clean up, it can be better organized and easier to search (though AIR will help) in a separate area. However, this solution is quick and simple and could be implemented immediatly as a first step.

The second question is what to do with certain not-too-popular packages already in the TURs and extra. I think we should leave it up to the TUs and package maintainers to clean this up on their own. Take unneeded packages out and post the PKGBUILDs to New Packages.

This isn't a perfect solution, but please can we start on *something*? Better to do something interim and fix it up than to discuss until we have something we think is perfect and find out after implementing it that it has flaws. This is how Open Source works.

please?

Dusty

Zephirias · 2004-05-02 17:07:32

Not a bad idea, Dusty, though I think that a specific Board just for that purpose would be better. However, let me re-post what I had said earlier about my idea:

We get one main server, whose "job" would be to create and store packages, and users could download from that server to install, etc. Now, whenever someone on the package management team finds an updated version of a package, they could log in to a special area of the site meant specifically for those developers. Then, using the special program on the website/server, they could link to the .tar or .bz2 that contains the source and submit it, along with the file's information that would be included with the package when you install it (file name, version, etc.) into the web-based form. Then, they click submit, and the server processes that file after downloading it, runs a makepkg-type thing on it, and stores the produced package in a "Fresh" repository. Then, the developer can download the file and opt to test it. If they're sure that it's okay, then they can replace the new file over the old one.

I mean, it's just an idea, but I think it could definitely work. Tell me what you all think.

jak · 2004-05-02 17:22:25

Dusty wrote:

makes both pacman and abs quite unnecessary in your case ... If you're compiling from scratch, you lose the advantage of binary pacman packages ... Wouldn't you be just as content using Linux From Scratch

I've never tried Linux from scratch. I don't want to spend time building a base system. I'm not interested in customizing bash, gcc, perl, xfree, etc. I'm glad for someone else to build a base, so I can focus on customizing local needs: bind, sendmail, apache, php, etc. A tight, well tested base is what I want from a distro. I'll do the rest myself, and install my local customizations to /usr/local. AFAICT, arch comes closer than anything else to fitting my requirements.

contrasutra · 2004-05-02 17:41:49

Well, we're all going around in circles again.

The voting function on the forum is not secure AT ALL. We have to find a way to prevent people from voting more than once (it has to be somewhat secure at least). Remember, guests can post on the board, so I assume they can vote.

Also, there'll be no way to list "Packages with Most Votes", etc. Presumably this is what the developers will want to do. Packages with the most votes get the highest priority. Also there's no download/working stats among other things. A forum doesn't have the features this needs.

The forum will simply be messy. There's no organization, just one big section with a bunch of posts that may or may not be a PKGBUILD. We shouldn't have to rely on moderators.

The word "elegant" and "forum" should never be used in the same sentence. Look how horribly the forum has worked for announcing INCOMING pkgs.

Also, we have to have some way to get working PKGBUILDs. This forum thing is all assuming that the first PKGBUILD a person posts will be perfect. What happens when there PKGBUILD has to be modified 3-4 times to get it to work? A forum doesn't have any sort of facilities to aid in group collaboration/version control. At that rate, a Wiki would work better (and I'm not suggesting we use that either).

sweiss · 2004-05-02 17:43:00

I have to say that I think the voting system would cause a pretty weird situation. If it is adopted, PKGBUILDs will be posted in polls. The users who will want that package wouldn't want to wait for it to go official and will compile them themselves. I don't think there will be much of a point to that really, as by the time it gets to the official repos everyone will already have it.

I was thinking of another method, involving the testpkg idea.

The main idea is as follows:

testpkg will be able to list the packages submitted within the last seven days, with their name, description, amount of times it was downloaded and a voting mechanism for working/not working status.

Someone who would like a package will be able to grab it and test it, and will be able to add votes to either the working or not-working counters.

Developers will be able to see what packages have been downloaded and how many times, and their current working/not-working status. This will allow them to see if a package is wanted or not and wether it's working or not, and will give them a chance to look at the pkgbuild and fix some things if needed. These packages will go to staging or so, and from there to the official repos when the time comes.

As for maintaining, this does not necessarily have to be done by the developers. It is obvious that the one who has built the package is also one of those who are using it, otherwise, why building it?

Thus, that user will be aware if a new version or a fix for this specific application has been released. So, that user will probably upload an updated version of the package once available.

That updated package will undergo exactly the same process. Also, if a lot of users downloaded it at first it is likely they will download an updated version. And so the package will regenerate in the staging repo in its new version.

I'm not sure how difficult it is to implement it, but I'm guessing it's not too difficult, and I believe this could supply a valid working solution. I also find it a lot more snappy to vote through a terminal rather than a forum.

As for dealing with the current mess, simply lock incoming and set a public date for whiping. This will allow anyone to download something from incoming if he needs it, and then the system can start taking its place.

Just a thought..

Dusty · 2004-05-02 18:09:19

jak wrote:

Dusty wrote:
makes both pacman and abs quite unnecessary in your case ... If you're compiling from scratch, you lose the advantage of binary pacman packages ... Wouldn't you be just as content using Linux From Scratch
I've never tried Linux from scratch. I don't want to spend time building a base system. I'm not interested in customizing bash, gcc, perl, xfree, etc. I'm glad for someone else to build a base

You could also try alfs (automated linux from scratch). The point I was making is not so much that you shouldn't use Arch as a base (anybody can use it), but that your philosophy is different from the Arch philosophy in some ways. This doesn't make either philosophy wrong (philosophies never are ), just that you won't be able to change the philosophy here.

What you suggest (only a few packages, all in current) is the way I understood Arch originally was, but its had to grow from there, or would have died.

jak · 2004-05-02 18:16:21

Dusty wrote:

What you suggest (only a few packages, all in current) is the way I understood Arch originally was, but its had to grow from there, or would have died.

Maybe arch would do better sticking to its roots. If that was a bad idea, why did it ever start growing in the first place?

Dusty · 2004-05-02 18:18:02

contrasutra wrote:

Well, we're all going around in circles again.

yeah, and they don't seem to be getting smaller or converging. Too many opinions, no consensus...

The voting function on the forum is not secure AT ALL. We have to find a way to prevent people from voting more than once (it has to be somewhat secure at least). Remember, guests can post on the board, so I assume they can vote.

Is there a problem with trust? Again, this is a temporary suggestion, so we can at least get our butts (avatars) moving.

Also, there'll be no way to list "Packages with Most Votes", etc. Presumably this is what the developers will want to do.

I'm betting AIR can do it with not much modification. This is a temporary solution, blah blah.

The forum will simply be messy. There's no organization, just one big section with a bunch of posts that may or may not be a PKGBUILD. We shouldn't have to rely on moderators.

Absolutely true, but if the moderators can control it while a better system is being discussed and built, things won't look so overwhelming. I was taught all this stuff about modular design and top-down-design in school too, but I'm a huge fan of prototyping. Start with something, find out what doesn't work, improve it.

The word "elegant" and "forum" should never be used in the same sentence. Look how horribly the forum has worked for announcing INCOMING pkgs.

two excellent points...

At that rate, a Wiki would work better (and I'm not suggesting we use that either).

A wiki might be a good second step. First we set up a forum system in about 15 minutes. Then we work out a more functional wiki system in a couple of weeks. After that we will have learned from the mistakes and might be able to set up some sort of web based or testpkg based script.

Dusty

Dusty · 2004-05-02 18:21:08

sweiss wrote:

As for dealing with the current mess, simply lock incoming and set a public date for whiping. This will allow anyone to download something from incoming if he needs it, and then the system can start taking its place.

excellent suggestion. If we lock and wipe incoming right now, it will at least force us to look for some other way to work. I do believe that almost everybody agrees that incoming isn't working...

Dusty

Zephirias · 2004-05-02 18:26:16

Dusty, I'm just curious. What was you opinion on my web-based system? I think it's a good idea, I just want to know what someone like you thinks about it.

Dusty · 2004-05-02 19:06:43

It's interesting and has some advantages, but these disadvantages:

doesn't interface well with pacman
devels love commandline, hate gui/web interfaces
repositories are mirrored and distributed

advantage is that everything is in one place.

I'm done with this issue, I believe I'm confusing the issue, rather than simplifying it as I intended.

Zephirias · 2004-05-02 19:44:44

Well, is there anyway that we could just design a terminal/standalone app to deal with that? I'm just wondering. That way, there'd essentially be no need for the web interface part instead.

rasat · 2004-05-02 20:15:03

contrasutra wrote:

The voting function on the forum is not secure AT ALL. We have to find a way to prevent people from voting more than once (it has to be somewhat secure at least).

The poll system in Forum is one vote per user. If a guest can vote in "New Package" forum depends on the permission. If not I can ask apeiro to change.

The Forum will simply be messy. There's no organization, just one big section with a bunch of posts that may or may not be a PKGBUILD. We shouldn't have to rely on moderators.

Wherever we place the poll and feedback will be messy and someone has to maintain it. Minimum requirement for a place are "edit" and "delete" functions. The forum have them. The moderators will clean all the post and add the missing PKGBUILD. I will do it if we agree on this arrangement.

Also, we have to have some way to get working PKGBUILDs. This forum thing is all assuming that the first PKGBUILD a person posts will be perfect. What happens when there PKGBUILD has to be modified 3-4 times to get it to work?

If we look in "New Package" forum there are several topics with improved PKGBUILDs. Moreover, the final PKGBUILD will be in the ftp server replacing the first upload by the test_pkg script. I hope we still plan to use the script system. The forum is only for communication.

The point here as Dusty said, this is a temporary arrangement giving scope to come up with better facilities. We need some concreted to get started. The "New Package" forum is not an ideal place but at least it fulfills the main ideas what we have been discussing.

Arch Linux

#26 2004-05-02 00:43:27

Re: Suggestion for new package management scheme

#27 2004-05-02 02:58:28

Re: Suggestion for new package management scheme

#28 2004-05-02 03:31:18

Re: Suggestion for new package management scheme

#29 2004-05-02 03:39:37

Re: Suggestion for new package management scheme

#30 2004-05-02 03:50:29

Re: Suggestion for new package management scheme

#31 2004-05-02 03:56:21

Re: Suggestion for new package management scheme

#32 2004-05-02 06:04:08

Re: Suggestion for new package management scheme

#33 2004-05-02 14:04:28

Re: Suggestion for new package management scheme

#34 2004-05-02 14:11:47

Re: Suggestion for new package management scheme

#35 2004-05-02 14:29:12

Re: Suggestion for new package management scheme

#36 2004-05-02 14:38:35

Re: Suggestion for new package management scheme

#37 2004-05-02 16:16:34

Re: Suggestion for new package management scheme

#38 2004-05-02 16:57:24

Re: Suggestion for new package management scheme

#39 2004-05-02 17:07:32

Re: Suggestion for new package management scheme

#40 2004-05-02 17:22:25

Re: Suggestion for new package management scheme

#41 2004-05-02 17:41:49

Re: Suggestion for new package management scheme

#42 2004-05-02 17:43:00

Re: Suggestion for new package management scheme

#43 2004-05-02 18:09:19

Re: Suggestion for new package management scheme

#44 2004-05-02 18:16:21

Re: Suggestion for new package management scheme

#45 2004-05-02 18:18:02

Re: Suggestion for new package management scheme

#46 2004-05-02 18:21:08

Re: Suggestion for new package management scheme

#47 2004-05-02 18:26:16

Re: Suggestion for new package management scheme

#48 2004-05-02 19:06:43

Re: Suggestion for new package management scheme

#49 2004-05-02 19:44:44

Re: Suggestion for new package management scheme

#50 2004-05-02 20:15:03

Re: Suggestion for new package management scheme

Board footer