Incoming organization system

netkrash · 2003-08-31 22:34:46

Hello archers,

As some of you have noticed (sarah) i'm back to help out testing / fixing incoming pkgs to be a kind of filter to the pkgs that will made it to official/unofficial/unstable repos.

I realize that this proccess needs some kind of system that help us to work on this user-submited pkgs.

I'm planning to write a little app in php that help us (sarah, xentac, apeiro if he has some free time , me and others) with this..

I need to know if you guys will like this type of application, currently i have a little feature list in my mind that i want to share with you and of course recive some feedback.

this is what i think we need to help us testing incoming stuff faster and with almost no waste of time.

Features:

- Read the incoming directory and filter the file list by extension (.gz and .bz2) this will discard the .jpg PKGBUILD and some other non-useful files.

- Be able to have a mantainer db with user/pass.

- Mantainers can self-assign a pkg and the download it.

- After the mantainer do his work it can report some comments about the pkgs and change the state of pkg. E.g FREE - ASSIGNED - BROKEN - RELEASED, maybe others.

- The system needs to show the mantainers the list of pkgs that are into the directory and its state, of course you can filter the list by state.

- Another useful thing is to delete the file when the state change to RELEASED and the move the db pkg info to a history table.

- dunno what ever we need to.

ok im able to this if you agreed with the idea..

commets?

contrasutra · 2003-08-31 23:20:51

That sounds great.

I think /incoming really seperates Arch from Debian/Gentoo/etc.

With Arch, anyone can submit packages into one central location. You dont have to become a maintainer or any other hoops. It really builds a community.

It also helps keep the standard repositories full. With Debian, there are APT repositories all over the place. When Arch gets bigger, everything can stay in the official repositories.

rasat · 2003-09-01 04:18:24

Good idea.

My concern is more on the result of creating more packages for Arch Linux. Example, what I didn't like with SuSE and Debian overwhelming the user with hundreds of applications. Confusing the user with too many similar applications not knowing what to use. I think it was Redmond Linux (today's Lycoris Linux), mid 2001, who first came up with the idea to "keep the product lean by trimming off the excessive, rarely-used programs".

Selection of applications on Redmond Linux was excellent..... one application for: photo viewer, graphic, digital camera link, scanner, DVD and VCD player, accounting, word processor, spreadsheet, good games, web browser and e-mail. So far Arch Linux offical has the same idea. Here I am concern about unofficial... not the number of packages but how to systematize, maintain, without confusing the user.

From my experience with people I have helped to install Linux, once they are on their own later they come and ask what programs (packages) and functions are availabe. What packages are needed for such and such work. What I mean by systematizing the unofficial packages coming from incoming, is to help user with the selection of packages and what are available. Not only the current unofficial list of packages categorized into main categories but also divided in to sub-categories and popularity. In long term there could be a page giving hints what packages to select to make a simple office, multimedia or developer station.

netkrash · 2003-09-01 04:55:26

rasat wrote:

In long term there could be a page giving hints what packages to select to make a simple office, multimedia or developer station.

that sounds great!

perhaps we could start a doc about that kind of hints.

dp · 2003-09-01 04:58:23

rasat wrote:

Good idea.
My concern is more on the result of creating more packages for Arch Linux. Example, what I didn't like with SuSE and Debian overwhelming the user with hundreds of applications. Confusing the user with too many similar applications not knowing what to use. I think it was Redmond Linux (today's Lycoris Linux), mid 2001, who first came up with the idea to "keep the product lean by trimming off the excessive, rarely-used programs".
Selection of applications on Redmond Linux was excellent..... one application for: photo viewer, graphic, digital camera link, scanner, DVD and VCD player, accounting, word processor, spreadsheet, good games, web browser and e-mail. So far Arch Linux offical has the same idea. Here I am concern about unofficial... not the number of packages but how to systematize, maintain, without confusing the user.
From my experience with people I have helped to install Linux, once they are on their own later they come and ask what programs (packages) and functions are availabe. What packages are needed for such and such work. What I mean by systematizing the unofficial packages coming from incoming, is to help user with the selection of packages and what are available. Not only the current unofficial list of packages categorized into main categories but also divided in to sub-categories and popularity. In long term there could be a page giving hints what packages to select to make a simple office, multimedia or developer station.

good point ... and i think my idea with "lists" for pacman is not that bad in organizing what to install to have a specific installation

-> http://bbs.archlinux.org/viewtopic.php?t=955

about "keep it clean by trimming off": do you like shopping? ... well, i think people like to go shopping, searching for new interessant, funny, usefull, ... stuff --- so if you try to stop the repository from growing, you actually force users to build packages by their own from the beginning (why inventing the wheel x times? || do you know how long it takes to compile all kde, qt and all dependendances? not longer than 22hours on 2GHz :-) --- but i agree fully with you that there must be a clear structure in the repositories

sarah31 · 2003-09-01 05:17:12

netkrash wrote:

- Read the incoming directory and filter the file list by extension (.gz and .bz2) this will discard the .jpg PKGBUILD and some other non-useful files.

well i don't think filtering out PKGBUILDs that are sitting in incoming is what i would want.

- Be able to have a mantainer db with user/pass.
- Mantainers can self-assign a pkg and the download it.
- After the mantainer do his work it can report some comments about the pkgs and change the state of pkg. E.g FREE - ASSIGNED - BROKEN - RELEASED, maybe others.
- The system needs to show the mantainers the list of pkgs that are into the directory and its state, of course you can filter the list by state.
- Another useful thing is to delete the file when the state change to RELEASED and the move the db pkg info to a history table.

i just don't know it seems alot of work to set something up like this when it is very easy to download untar, build and test the packages then delete them as they are added to the repositories. then notify the users via the forum in the new packages section by answering the package posts there with something like "added to unofficial/unstable".

about the only thing i would like to know is whether a package is a networking or system package because i would then ask someone with far more knowledge or a larger network setup to test those packages. i have a small network set up but i do not nor ever have run apache or samba so i have very little clue how to test packages that use either. i also have little interest in learning it to be honest networking stuff bores me to tears.

i honestly don't know if i would use something like this.

dp · 2003-09-01 05:25:07

sarah31 wrote:

[...-quoted-...]
about the only thing i would like to know is whether a package is a networking or system package because [...]

-> what about writing in the forum in "New Packages" each time something like this

e.g. "samba(networking) in incoming" or "gpm(daemons) in incoming"? would that help?

netkrash · 2003-09-01 05:42:31

sarah31 wrote:

netkrash wrote:
- Read the incoming directory and filter the file list by extension (.gz and .bz2) this will discard the .jpg PKGBUILD and some other non-useful files.
well i don't think filtering out PKGBUILDs that are sitting in incoming is what i would want.
- Be able to have a mantainer db with user/pass.
- Mantainers can self-assign a pkg and the download it.
- After the mantainer do his work it can report some comments about the pkgs and change the state of pkg. E.g FREE - ASSIGNED - BROKEN - RELEASED, maybe others.
- The system needs to show the mantainers the list of pkgs that are into the directory and its state, of course you can filter the list by state.
- Another useful thing is to delete the file when the state change to RELEASED and the move the db pkg info to a history table.
i just don't know it seems alot of work to set something up like this when it is very easy to download untar, build and test the packages then delete them as they are added to the repositories. then notify the users via the forum in the new packages section by answering the package posts there with something like "added to unofficial/unstable".
about the only thing i would like to know is whether a package is a networking or system package because i would then ask someone with far more knowledge or a larger network setup to test those packages. i have a small network set up but i do not nor ever have run apache or samba so i have very little clue how to test packages that use either. i also have little interest in learning it to be honest networking stuff bores me to tears.
i honestly don't know if i would use something like this.

One of the goals i had in mind was to know exactly what pkgs are builded/tested by someone else..

actually i don't know if the pkg im building/testing is ready to go by someone else.

this app serve as a "tracking" system to know the state of the submited pkg, and i think that helps a lot..

rasat · 2003-09-03 16:26:45

netkrash wrote:

rasat wrote:
In long term there could be a page giving hints what packages to select to make a simple office, multimedia or developer station.
perhaps we could start a doc about that kind of hints.

Or AL could make an "Arch Linux Package Database". AL website has a small database but maybe it could be extending to something like this demo website:

http://bliss-solutions.org/amlug/al-pkg … -demo.html

apeiro · 2003-09-03 17:12:55

I don't want to quell your enthusiasms or anything, but I don't really see the need for either application.

Arch already pulls package information from a database, but the advantage of our current setup is that almost all information can be re-generated from the CVS repository, so we don't have to rely on the web database to keep us going.

As for the incoming app, I'm not really the one to comment since I don't peruse /incoming all that often. But the maintainers seem to have a good hold on it. If someone wants to test/modify an incoming package for addition to a repository, all they need to do is delete/move it (if they're worried about another maintainer snatching it up simultaneously), download it, untar it, and go to work.

Rather than flag a package as BROKEN or somesuch, I think it would be better to just email the creator and notify him/her that the package doesn't work, and delete it.

Once the package is added, the maintainer can then assign/adopt the package as one of their own.

Xentac · 2003-09-03 20:14:53

I agree with apeiro on incoming. While a little better organizing would be nice, all we really need is someone to define some sort of structure (something like apeiro suggested, move the tars when you start testing them). Incoming isn't supposed to be an official repository, it's just there for packages to be tested and then put into an official repository.

rasat · 2003-09-04 01:44:31

apeiro wrote:

.....I don't really see the need for either application.
Arch already pulls package information from a database, but the advantage of our current setup is that almost all information can be re-generated from the CVS repository, so we don't have to rely on the web database to keep us going.

From a developer and experienced AL user view I can agree with apeiro when knowing AL packages and the description of most of the names. But from a common user view I don't agree. It can be a time consuming / frustrating effort trying to browse through package names which are often max. 8 characters names. Not to say, first going through the official and then unofficial. Once you find what you want among several alternatives and then click PKGBUILDs one by one.

Let us say, CVS is for developers and AL website for common users. On website they can browse through all lists (official and unofficial) within a category. But again its a time consuming effort going through 50 or 100 packages within a category. In future maybe 150 packages or more.

Example, user wants to find a cd burner package. First he/she may not be sure if its office or multimedia or maybe its system. Lets say knows multimedia and finds a list of 110 packages for sound and graphic. Names starting with "cd" is easy to find but cannot find a cd burner. Then browse description column and hopefully finds k3b or arson .... strange names . Cd burner package works with cdrdao, cdrecord, readcd and mikisofs. This is an other headage. Only k3b does a scan telling the user about these packages (if correct).

CVS search method related to the above example is to check k3b's PKGBUILD "depends=('arts' 'audiofile' 'fam' 'freetype2' 'gcc' 'glibc' 'kdelibs' 'libart-lgpl' 'libogg' 'libpng' 'libvorbis' 'qt' 'xfree86' 'zlib')". Does it help?

It would definitely help if user could simply enter in a database "cd burner" to get a list. Or specify a purpose by entering few additional criteria "develop", "multimedia" and "KDE", thereby having a complete set of related packages to run a cd burner with KDE.

What I am here trying to suggest with an extended database and what I also understand about netkrash's package test script, is to fasten the work and get it done easier where both developers and users can benefit.

sarah31 · 2003-09-04 03:40:12

rasat...we are talking about an interface for incoming which is not a repository nor do the developers guarantee anything from it. by the time we download, build, test and upload the need for flagging it in a databse would be pointless.

as for searching on the current web interface it is not very hard. use key words for example try "cd burning" you get quite a few hits many are not cd burning but ALL of the burning apps are in there. i really don't know why more is needed. it is just the same as searching google. there is no way we can make a tool that will read a users mind and pull exactly what they are looking for out of their mind. in fact searching on arch's db is far easier than searching debian's package lists (especially online)i certainly do not see the need for such a database on incoming.

rasat · 2003-09-04 04:25:54

sarah31 wrote:

as for searching on the current web interface it is not very hard. use key words for example try "cd burning" you get quite a few hits many are not cd burning but ALL of the burning apps are in there. i really don't know why more is needed.

As I said, its for the experienced AL users knowing the names, key words, etc. Database I am suggesting has fixed criteria / key words..... no need to read anyone's mind. The efficiency depends on the database developer. It requires some planning.

About incoming packages, would be nice to have a database knowing what packages are available if wanting to download or having an idea about future packages. I agree with contrasutra "I think /incoming really seperates Arch from Debian/Gentoo/etc". How this will be applied is part of this discussion.

rasat · 2003-09-09 03:32:00

I have received several requests to have a software database. Also this was in my mind before joining Arch Linux. Instead of holding AL packages database alone I also want to include tarballs what works well will Arch Linux.

I have put this database under AMLUG as an experimental project. It will be under development (and may not work at beginning stage). If anyone is interested they can take a look once in while and suggestion & feedback are welcome.

Feedback:
rasat@pacific.net.sg

Software Database:
http://bliss-solutions.org/archlinux/incoming/

Arch Linux

#1 2003-08-31 22:34:46

Incoming organization system

#2 2003-08-31 23:20:51

Re: Incoming organization system

#3 2003-09-01 04:18:24

Re: Incoming organization system

#4 2003-09-01 04:55:26

Re: Incoming organization system

#5 2003-09-01 04:58:23

Re: Incoming organization system

#6 2003-09-01 05:17:12

Re: Incoming organization system

#7 2003-09-01 05:25:07

Re: Incoming organization system

#8 2003-09-01 05:42:31

Re: Incoming organization system

#9 2003-09-03 16:26:45

Re: Incoming organization system

#10 2003-09-03 17:12:55

Re: Incoming organization system

#11 2003-09-03 20:14:53

Re: Incoming organization system

#12 2003-09-04 01:44:31

Re: Incoming organization system

#13 2003-09-04 03:40:12

Re: Incoming organization system

#14 2003-09-04 04:25:54

Re: Incoming organization system

#15 2003-09-09 03:32:00

Re: Incoming organization system

Board footer