Suggestions for INCOMING

Dusty · 2004-04-28 04:07:30

Xentac, you try too hard.

rasat · 2004-04-28 04:30:48

test_pkg

options:
-submit Tar and upload the necessary build scripts to incoming
-list List the available packages from incoming
-get <package> Download the package from incoming
-build <package> Download and build the package from incoming

If the submitted package only includes PKGBUILD, I am suggesting not to tar it but upload as a text file. If a user don't want to download, can read in ftp server.

linfocito · 2004-04-28 04:48:55

If the submitted package only includes PKGBUILD, I am suggesting not to tar it but upload as a text file. If a user don't want to download, can read in ftp server.

The problem with sumitting just thePKGBUILD (instead of a tarball) is that some packages have $install and some local $source files.
Xentac´s script is aware of that, and put all the necessary scrips/files into the generated tarball.

EDITING (28/04/2004)
:oops: Oh!, I missed the "If the submitted package only includes PKGBUILD...." above.

Xentac · 2004-04-28 04:50:37

If you can offer code to make -get, -list, and -build work with it, I'll include it.

I suppose I'm looking for a patch

I wrote this to automate things better, not to make it easier for the user.

Dusty · 2004-04-28 15:58:58

also a lot of packages need small non-PKGBUILD files besides install, like a bash script to execute a java jar, for example. Some don't even have download sources, I made one that included the c file itself.

Dusty

sarah31 · 2004-04-29 20:16:13

is this still being discussed? i mean really incoming has been inadequate for well over a year.

incoming could just be PKGBUILDs and necessary files but that stuill does not solve anything. the problem is that the people who should be managing incoming never do or they go through spates of removing what they like or know and then leave the rest.

incoming should be done away with altogether. if it stays around then it will continue to collect dust.

Dusty · 2004-04-29 20:51:24

sarah31 wrote:

is this still being discussed? i mean really incoming has been inadequate for well over a year.

Hey hey Sarah, welcome back, missed you... things are getting complacent around here.

rasat · 2004-04-30 04:20:04

sarah31 wrote:

the problem is that the people who should be managing incoming never do or they go through spates of removing what they like or know and then leave the rest.

To leave packages unmanaged when not knowing how to test / fix a particular type of package, is one of the reasons. The other, what I said earlier, large number of packages are left when not knowing what they are. The name of packages often don't say much. No doubt, surely there are situations when a package maintainer get frustrated when many pkgs are poorly build, so "why testing unknown packages".

The list of incoming packages MUST include the description. Example, if the test_pkg script lists the package names only (currently around 300) will not do. Also many packages take time to move in TUR or may not be moved at all, will keep the list long. It would be helpful to be able to list e.g. latest uploaded packages or type of apps (select a category as per pkg maintainer's skill). Most convenient if the script can do the sorting. Otherwise some kind of database / search engine has to be included.

sarah31 · 2004-04-30 05:31:13

well descriptions only go so far. there was alot of stuff in incoming that had descriptions that i had no cllue as to how to utilize even if it built. there were others that would not build and no one else was helping so they sat there.

incoming should be eliminated and PKGBUILDs should be sent to TURs or some other set of developers committed to doing something with them. If they do not know how to utilize them then they should seek help. but as it stands now incoming just serves as a "lets cure my boredom" farm.

incoming is inadequate as it is or even if it remains as a non proactive repository. if there is no pressing obligation to work on it then it will never get emptied. it is too big and intimidating. unfortunately this attitude gets viewed negatively by users.

anyway i will stop now as it si obvious the point is rather beaten to death and a year or more later i have no confidence that a solution is at hand.

Mork II · 2004-04-30 12:40:56

Xentac: Thanks for your effort. Hope it isn't wasted

Sarah 31: Yes, there are problems with the current system. The chain incoming-staging-testing-official is at least one step too long. There should be a single respitory where all packages (new packages, new versions of old packages) are tested. We have Testing, wich is good. The problem is getting new packages into the system with minimal effort. I can see a couple of alternatives:

A. My intial suggestion. A strict no-binaries policy is inforced on Incoming (erasing old packages). Testpkg is further developed to assist with submitting and initial testing. When a package is done here it is sent to testing.
- Requires writing testpkg
- Does not make discussion easier

B. Remould Incoming and Staging into a respitory that only contains PKBUILD:s and other buildfiles. Buildfiles can be submitted and built with testpkg (using abs and makepkg). If a package builds it goes to testing.
- Requires writing testpkg.
- Requires creating a new respitory
- Does not make diskussion easier
+ Automatation

C. Ditch incoming. New packages should be sent to TU:s who test them and submit them directly to testing.
+ Simple
+/- Discussion is easy / Discussion only includes the sender and the TU
- No automatation. You have to cut and paste
* What if the TU is too busy?

D. Forget Incoming. Forget Staging. A mailinglist is created and users are urged to send PKGBUILD:s to it. Developers cut and paste and try to build. If a build fails (or the the description is too vague etc.) it can be discussed directly in the mailinglist. When a developer manages to build a package it's sent to testing without further ado.
+ Simple
+ Discussion is easy
- No automatation. You have to cut and paste

I prefer option D since it's simple and does not introduce new softare to scare newbies with. There is also a educational value in getting critique on your PKGBUILD:s.

Mork II · 2004-04-30 12:44:46

Another point entirely: Wichever method is chosen, it should be easy (for developers at least) to check how many times a package in testing has been downloaded. The reason beeing that there are to possibilities why there are no bugreports on a package:

1. No testing has been done. The package should not be included in Base/Extra/Unstable.
2. Testing has been done. The package may be included in Base/Extra/Unstable.

Dusty · 2004-04-30 15:54:24

sarah31 wrote:

anyway i will stop now as it si obvious the point is rather beaten to death and a year or more later i have no confidence that a solution is at hand.

au contraire...

I think a solution finally is at hand. A lot of other stuff had to get organized before incoming could be. You were just ahead of your time, Sarah, someday Arch will be what you visualized.

MorkII: excellent suggestions/summary of possible recourse.

Dusty

apeiro · 2004-04-30 16:32:50

Mork II wrote:

C. Ditch incoming. New packages should be sent to TU:s who test them and submit them directly to testing.
+ Simple
+/- Discussion is easy / Discussion only includes the sender and the TU
- No automatation. You have to cut and paste
* What if the TU is too busy?
D. Forget Incoming. Forget Staging. A mailinglist is created and users are urged to send PKGBUILD:s to it. Developers cut and paste and try to build. If a build fails (or the the description is too vague etc.) it can be discussed directly in the mailinglist. When a developer manages to build a package it's sent to testing without further ado.
+ Simple
+ Discussion is easy
- No automatation. You have to cut and paste
I prefer option D since it's simple and does not introduce new softare to scare newbies with. There is also a educational value in getting critique on your PKGBUILD:s.

Agreed. I like the "+ Simple" options for that reason. It turns the package submittal phase into more of an open forum where people get peer help and critique (kinda like the New Packages forum, but email is still more prevalent and universal), and with option D, any TU can pick up an interesting PKGBUILD off the list if they're not too overloaded.

That's not final say or anything, just my 2 cents. I like the simple things.

Dusty · 2004-04-30 16:36:35

apeiro wrote:

Agreed. I like the "+ Simple" options for that reason. It turns the package submittal phase into more of an open forum where people get peer help and critique (kinda like the New Packages forum, but email is still more prevalent and universal), and with option D, any TU can pick up an interesting PKGBUILD off the list if they're not too overloaded.

What if no TU finds a particular PKGBUILD interesting... is it lost?

That's not final say or anything, just my 2 cents. I like the simple things.

Hence Arch.

rasat · 2004-04-30 18:51:15

Mork II wrote:

<b>C</b>. Ditch incoming. New packages should be sent to TU:s who test them and submit them directly to testing.
+ Simple
+/- Discussion is easy / Discussion only includes the sender and the TU
- No automatation. You have to cut and paste
* What if the TU is too busy?

The end result with this system, when less people involved, Arch will have plenty of so called "personal" packages... good for the contributor but may not be used by other users. Do Arch want quantity or quality/useful packages?

Dusty · 2004-04-30 19:12:55

rasat wrote:

The end result with this system, when less people involved, Arch will have plenty of so called "personal" packages... good for the contributor but may not be used by other users. Do Arch want quantity or quality/useful packages?

This is true. Perhaps incoming is serving a purpose then, after all; it stores PKGBUILDS for packages that are very uncommon, but might be useful to a few people. It makes more sense to use somebody else's PKGBUILD if it exists.

How about yet another repo breakdown:
contrib: PKGBUILDS (not packages) contributed by users that are not going to be needed by many people
tur: PKGBUILDS (not packages) contributed by users and deemed by those users to be useful to Arch as a community. TUs will be expected to pull PKGBUILDS off this repo, or move them to contrib.
staging: packages built from PKGBUILDS in tur that have been tested and probably edited by TUs.
testing/current/extra/unstable: same as already

contrib should have some sort of automatic accounting system that notes how often PKGBUILDS are dl'd. If a lot of users are using a PKGBUILD, it should be moved to tur. If a PKGBUILD is not downloaded, it can be removed.

I know I've personally built a few packages that I didn't think anybody would use, so I kept them local; now they are "lost" to anybody else in the community. I have also uploaded a couple packages to incoming that I did not think would be useful, but one or two other people have expressed an interest in; cluttering incoming.

Putting stuff in incoming is perhaps too easy; it doesn't have enough quality control. Its like having a free hospital emergency room, people go there to have slivers removed.

Dusty

contrasutra · 2004-04-30 19:47:23

OK, I think STAGING should stay how it is. It seems to work fine, and I promise it'll be more active when developers start testing pkgs in it more (this is happening) and when INCOMING gets cleaned up.

Now, on INCOMING, I don't like the Mailing List idea. It's just too hard to navigate. It's much harder to search a mailing list archive then to look at an FTP directory.

I also don't want people to be mailing things right to the TUs. Not every TU can test everything, and it's better to keep things public.

I think INCOMING should stay as an FTP, with all discussion remaining on the forum and mailing list. But, that'll give us a messy INCOMING again. Since no one likes my separate TU folder idea, why not have folders for package categories? This solves rasat's problem too. We could use the official category names, and just go with that. Presumably the TUs will focus (though not be exclusive) on certain categories they have experience/interest in.

Now, I think we should completely wipe INCOMING when we change systems. So many of those packages are out of date and broken. If the package builders still know the package works and are still interested in it, he can reupload the PKGBUILD. This will instantly remove all "dead" packages.

Also, remember this decision doesn't have to be final. We should try something out for a month or so and see how it works.

I've written a Python version of testpkg by the way. The actual client isn't hard to make, so we shouldn't be focusing on that. If an FTP is chosen, testpkg is definitely needed though. It prevents people from uploading messed up tarballs to INCOMING. As long as they use testpkg, everything will be uploaded in a standard way.

Xentac · 2004-04-30 23:24:19

If we use ftp, accounting is much more difficult. With a custom/integrated system, accounting of number of downloads is a lot easier.

Xentac · 2004-04-30 23:28:58

Related to the developers taking packages, there's a very real problem there. There is some limit where all developers won't be able to maintain more packages. Then what do we do? Add more devs? The more devs we add, the more difficult organization becomes.

I've heard some people mention splitting off into seperate community repos. The problem with that is management. The way we have it now ugly upgrades are less ugly, because we can do everything at once in testing. If we split into community repos (much like the personal TURs are right now) updates will get out of sync.

On another note, I don't like having brainstorming like this on forums. In fact, I don't like forums in general. Suggestions for better methods of communication?

Dusty · 2004-04-30 23:47:11

Xentac wrote:

On another note, I don't like having brainstorming like this on forums. In fact, I don't like forums in general. Suggestions for better methods of communication?

Smoke rings!

contrasutra · 2004-05-01 00:02:14

Well, Debian seems to maintain things well, and they have over 8000 packages.

Honestly, I have no idea how to solve the growing repo problem. We should look closely at organizations like Debian and Gentoo though, because maybe we can modify their ideas to get a solution.

dp · 2004-05-01 00:05:57

contrasutra wrote:

Well, Debian seems to maintain things well, and they have over 8000 packages.
Honestly, I have no idea how to solve the growing repo problem. We should look closely at organizations like Debian and Gentoo though, because maybe we can modify their ideas to get a solution.

exactly!

Xentac · 2004-05-01 00:08:25

Yes, they also have about 5000 developers and a hell of a lot more resources than us.

rasat · 2004-05-01 03:59:25

Xentac wrote:

I don't like having brainstorming like this on forums. In fact, I don't like forums in general. Suggestions for better methods of communication?

Mailing lists / email is for faster communication.... to finalize a decision. Chat for specific detail. Brainstorming and feedback a forum is ideal giving users time to think and retrace what have been said.

The problem with this topic its time consuming / takes energy when no decision (happens when all active Archers are involved ). I suggest Xentac and Contrasutra to make the decision, to get started.

PS.
About communication, to know what packages (PKGBUILD) are uploaded and for feedback, the "New Package" forum has done / does well. Would be good if the test_pkg script or whatever will be used, telling the contributor to post a note in the forum.

Dusty · 2004-05-01 06:19:15

I think perhaps this major decision needn't be discussed so publicly, there's too many opinions to be voiced, but honestly, most don't matter, as it's Judd and the devs that make the final decision.

I don't *feel* like a dev, so that doesn't count even my opinion.

Dusty

Arch Linux

#26 2004-04-28 04:07:30

Re: Suggestions for INCOMING

#27 2004-04-28 04:30:48

Re: Suggestions for INCOMING

#28 2004-04-28 04:48:55

Re: Suggestions for INCOMING

#29 2004-04-28 04:50:37

Re: Suggestions for INCOMING

#30 2004-04-28 15:58:58

Re: Suggestions for INCOMING

#31 2004-04-29 20:16:13

Re: Suggestions for INCOMING

#32 2004-04-29 20:51:24

Re: Suggestions for INCOMING

#33 2004-04-30 04:20:04

Re: Suggestions for INCOMING

#34 2004-04-30 05:31:13

Re: Suggestions for INCOMING

#35 2004-04-30 12:40:56

Re: Suggestions for INCOMING

#36 2004-04-30 12:44:46

Re: Suggestions for INCOMING

#37 2004-04-30 15:54:24

Re: Suggestions for INCOMING

#38 2004-04-30 16:32:50

Re: Suggestions for INCOMING

#39 2004-04-30 16:36:35

Re: Suggestions for INCOMING

#40 2004-04-30 18:51:15

Re: Suggestions for INCOMING

#41 2004-04-30 19:12:55

Re: Suggestions for INCOMING

#42 2004-04-30 19:47:23

Re: Suggestions for INCOMING

#43 2004-04-30 23:24:19

Re: Suggestions for INCOMING

#44 2004-04-30 23:28:58

Re: Suggestions for INCOMING

#45 2004-04-30 23:47:11

Re: Suggestions for INCOMING

#46 2004-05-01 00:02:14

Re: Suggestions for INCOMING

#47 2004-05-01 00:05:57

Re: Suggestions for INCOMING

#48 2004-05-01 00:08:25

Re: Suggestions for INCOMING

#49 2004-05-01 03:59:25

Re: Suggestions for INCOMING

#50 2004-05-01 06:19:15

Re: Suggestions for INCOMING

Board footer