TUR guidelines

Xentac · 2003-12-09 21:33:23

Ok. This is how I want the TURs to be run. This message doesn't just go to the TUs, it goes to everyone.

(ED: I use 'we' a lot here... sometimes I'm talking about the developers and other times I'm just talking about me)

The problem that the developers saw with incoming was that packages piled up. If we didn't look at incoming for a while it became that much harder to keep up to date. We tried throwing more bodies at it and organizing it more but none of it seemed to work as well as we'd like.

The point of having incoming was to let users who created packages and wanted them included in the repos upload them. It wasn't for users to share files or packages. It wasn't meant for a permanent storage. It was created so that developers didn't have to do all of the work.

As we got more users, incoming got larger and larger (people like creating packages... imagine that...). As incoming got larger, older packages were being superceeded by newer packages (and we'd have two copies in incoming). We needed to figure out some way to manage it. One of the ideas was to let users run their own repos: user-repos. We thought it would be nice if users could just keep their repos up to date and anyone could access them. After some thought we found this would be a huge security risk. When someone wanted to install something from incoming, they had the PKBUILD and install files right in front of them... most likely they read them. People trust repos and don't usually verify PKGBUILDs from them.

Then the idea of a trusted user repo came about. The difference between the two was that only certain users would be allowed to run repos. We would select people who had their respect and honour on the line. If they made good packages, people would like them and add their repos to their systems. If they made bad packages, people wouldn't like them and wouldn't add their repos.

We hoped that having trusted users manage some packages that we'd be able to make incoming smaller, because regular users would trust the trusted users enough to offer their packages to the trusted user's repos, instead of uploading them to incoming. Then popular and often updated packages could be shared by the general populace, kept up to date, and still considered for inclusion into the official repos.

Now to the present. A number of trusted users have been selected (maybe I chose the wrong ones...) and are currently running their own repos. Packages are still only being included in incoming and then the TUs are snatching them and telling everyone that they snatched them. Some people hate the ideas of TURs, others love them.

A request for the users. Figure out whether you like the Trusted User Repo idea or not. If you do, then start talking to the Trusted Users. Get to know them. I'm sure some of you will get along. When you create packages, offer them to your favourite Trusted User to manage and keep them up to date, for a time when they could be included in the official repos.

A request for the Trusted Users. Only include packages in your repos that the creators want. Don't just take them and tell someone, instead work something out. If you like a package that's been added to incoming, email the person who uploaded it and ask if you could include it in your repo. Be courteous. Also, keep your repos as clean as possible and take as much care as you can to make quality package upgrades and run a quality repo. If your repo isn't up to someone's standards, they won't want to add it and then you're just running a repo for yourself.

A request for developers. We're probably going to have to start looking at the TURs for possible packages to be included in the official repos. Reason being, all of the Trusted User's packages are only in their TURs and not in incoming, to reduce clutter. By not looking at their repos we're telling the Trusted Users that their work isn't really worth it and not giving them an incentive to continue keeping their repos up to date.

Comments?

Xentac · 2003-12-09 22:19:08

To elaborate more on my request to the trusted users. The repo you are running is for packages that could eventually be included in the repos. Customized/optimized packages really shouldn't be included in there. Try not to replace other packages or generally make your user's lives miserable. Your repo should do its best to follow the Archlinux package guidelines. It should be an extension to the official repos, not a replacement.

andy · 2003-12-10 00:41:37

I have another suggestion :

what about the TU's make a list of packages/PKGBUILDS which are best suited for inclusion in AL ?

Hopefully, they get feedback which PKGBUILDS/packages won't work (or even which ones work), and hence can prepare the PKGBUILDS to make them as guideline conform as possible... or at least they know which ones are good already.
That way, the developers can (with such help of TU's) decide more easily and quicker which packages go into AL.

Although I'm not a developer I could imagine the following :
Now (or just before the TUR's): the developer stares at incoming and goes "hmmm ... oh well ... the oldest is next ... whatever it may be ..."
In the future : the developer can go to a TU and go : "which ones would you take next ?" And the work shouldn't be too much for developers.

I hope I got the idea right ;-)

contrasutra · 2003-12-10 01:02:04

Well, I don't like all the work we have to do to get a package from incoming to a TUR.

If you put a package in incoming, you are ASKING people to include it in a repository. Why would someone be upset if we actually do? I do email every person when I add one of their packages, but I don't know why I should have to ask their permission first.

Of course, if someone asked me to remove one of their packages, I would. But you act like we are stealing something.

Xentac · 2003-12-10 01:23:55

contrasutra wrote:

Well, I don't like all the work we have to do to get a package from incoming to a TUR.
If you put a package in incoming, you are ASKING people to include it in a repository. Why would someone be upset if we actually do? I do email every person when I add one of their packages, but I don't know why I should have to ask their permission first.
Of course, if someone asked me to remove one of their packages, I would. But you act like we are stealing something.

Maybe they don't like you?

The idea is to have incoming and the TURs compliment each other. Having the same stuff in both of them is sort of pointless. Suddenly you have to check if a package is in incoming and in the TURs. In the end I'd like packages either going to one or the other.

Hopefully the users will decide whether they want their packages included in the TURs or in incoming. Hopefully there's enough good feedback for the TURs to continue successfully. People have to trust the TUs or everything breaks down.

I have more to say about this but can't put the words together...

deepfreeze · 2003-12-10 02:03:44

Well I've said this before in another thread but I'll repeat it here because I think it bears repeating. I am a TU. However, if you look at my repository, it's quite threadbare. That's because I'm only putting things in there that a) I make myself or b) someone else explicitly asks me to. I don't think that the TUs should go around and raid /incoming to pad their own repos. If someone wants their package in a TU, they'll ask. Otherwise, I assume (rightly) that if someone puts their package in /incoming, that they want it evaluated and added to the Arch supported repos.

Contrasutra, it's true that people that put a package in incoming are asking for it to be included in a repo. More to the point, they're asking for it to be included in unofficial, not your TU just because you feel like adding it there.

Maybe I'm the "anti-TUR" TU, but the more I think about TURs, the more I get the feeling that they aren't really going to accomplish what I thought they would, which was accelerate the process moving a package from incoming to unofficial.

Xentac · 2003-12-10 02:15:56

deepfreeze wrote:

Maybe I'm the "anti-TUR" TU, but the more I think about TURs, the more I get the feeling that they aren't really going to accomplish what I thought they would, which was accelerate the process moving a package from incoming to unofficial.

Currently, what do you think they will accomplish? As there get to be more packages in general things will take longer, that's just the way it goes.

I also wouldn't discount the TURs just because of these initial bumps. If anyone had bothered to ask me, I'd have said that the TURs weren't ready to be relied on until this afternoon. Before then it was just beta. I told everyone about it to see initial reaction and how many people would like it, but the software wasn't there to do what it was intended. Only now do we have enough software to be considered a viable alternative to incoming.

deepfreeze · 2003-12-10 02:28:07

Perhaps this is just my perception, but it seems that the only thing that the TURs have served to do thus far is to have a race to see which TUs can plunder incoming fast enough to build their own massive repos. I really hope that doesn't continue.

I guess I am just a little confused about what, ultimately, the end purpose of TURs is. Is it to have these TURs as a separate entity from unofficial, with no packages from TURs ever being added to official Arch repos, or are the TURs a way for the developers of Arch to more easily add packages to unofficial, etc. Please don't get me wrong. I like the idea of assisting the developers and maintainers. That's why I signed up to have a TUR in the first place. I hope it works out and, if this was all beta until now, then I will wait to have a final judgement on the system. What I am saying is that I hope this does not ultimately stop people from contributing packages because they're afraid that 2 minutes after it hits incoming, a TU will come along and snatch it up, possibly "customize" it to their liking and put it in their TUR with no shot of it ever getting to unofficial.

Xentac · 2003-12-10 02:40:06

deepfreeze wrote:

Perhaps this is just my perception, but it seems that the only thing that the TURs have served to do thus far is to have a race to see which TUs can plunder incoming fast enough to build their own massive repos. I really hope that doesn't continue.

Hehehe... it's like trying to get first post on Slashdot. That's a good point and it's something that should really be stopped.

deepfreeze wrote:

I guess I am just a little confused about what, ultimately, the end purpose of TURs is. Is it to have these TURs as a separate entity from unofficial, with no packages from TURs ever being added to official Arch repos, or are the TURs a way for the developers of Arch to more easily add packages to unofficial, etc. Please don't get me wrong. I like the idea of assisting the developers and maintainers. That's why I signed up to have a TUR in the first place. I hope it works out and, if this was all beta until now, then I will wait to have a final judgement on the system. What I am saying is that I hope this does not ultimately stop people from contributing packages because they're afraid that 2 minutes after it hits incoming, a TU will come along and snatch it up, possibly "customize" it to their liking and put it in their TUR with no shot of it ever getting to unofficial.

I think that the TURs do need more structure that way. I envisioned co-ordination between everyone so that one person wouldn't be left maintaining everything (I hope that all the TUs realize that the packages in their repos need to be updated, not just left to get bit rot).

You do bring up good points. I thought I explained my position better in this initial post. Does it seem more like what you thought and not what it's turning out to be?

On the side of getting packages into the repos, we (the developers) still haven't figured out a method for looking at packages in the TURs for inclusion. That really should be figured out along with incoming. Not sure exactly how we'll do it... Maybe the TURs should be smaller until things get figured out. We definitely need to co-ordinate better. That's why I'm setting up the mailing list.

contrasutra · 2003-12-10 02:49:10

Just for the record, I made almost all of the packages in my repos.

Most were made before the TUR even started, I just hadn't gotten around to uploading them to incoming.

Here's what I think. We should make it extremely clear what you should expect to happen to your package when you upload it to incoming.
Put a disclaimer: "These may be added to an official repo or TUR at any time. But if you contact the maintainer, he/she will remove the package if you so desire".

Something like that.

red_over_blue · 2003-12-10 02:54:17

I'm going to throw my 0.02 in here quick. However, not specifically about the TURs, but about packages making it from incoming to the official repos faster.

1. I don't see the TURs accomplishing this unless they adhere STRICTLY to the Arch guidelines for building packages. Most of the TUs are probably thinking, "But that's no fun." However, I haven't been a maintainer for very long, but I can tell you, taking care of packages is not a race to see who has the most. It is the meticulous building of packages, with pedantic attention to their correctness.

2. If people want to contribute to Arch by getting packages into the official repos sooner, I have only two words for you.... peer review:

a. When adding a package to incoming, post the PKGBUILD along with your post stating the package has been added.

b. If you want to help, not for popularity, but to truely help, take the PKGBUILDs from the posts that others have submitted and build them yourself. Using your knowledge and the tools available to you, ensure their correctness and post back any changes that should be made to the PKGBUILD.

Thanks for taking the time to read my post. Also, let's please remember that we are all friends here. I sincerely believe that everyone here is acting with the best interests of Arch Linux in mind.

red_over_blue · 2003-12-10 04:21:52

sarah31 wrote:

ROB: i dunno if i would say that TURs will not succeed if such and such is not done. BUT how successful they are in acting as a desired buffer between incoming...

right... that is exactly what I was saying. I think the TURs are already "successful".

red_over_blue wrote:

not specifically about the TURs, but about packages making it from incoming to the official repos faster.

1. I don't see the TURs accomplishing this unless they...

farphel · 2003-12-10 12:59:15

Xentac wrote:

On the side of getting packages into the repos, we (the developers) still haven't figured out a method for looking at packages in the TURs for inclusion. That really should be figured out along with incoming. Not sure exactly how we'll do it... Maybe the TURs should be smaller until things get figured out. We definitely need to co-ordinate better. That's why I'm setting up the mailing list.

Not to sound like I'm picking a fight or anything, but this just doesn't make any sense to me at all! I thought the original concept of the TURs was to alleviate the problem of packages sitting in /incoming so long they became stale. In other words, the TUR was a way to improve the process of getting packages tested and included into "unofficial". If that's true, how could you invent the TUR and implement it without knowing how it was going to solve the original problem?

Of course, you did say in this thread that the TURs took off faster than you'd like; that everything up to now was supposed to be a beta test. However, I just don't see (and never have) how the TURs were supposed to improve the process of packages getting into "unofficial".

I'm really in agreement with deepfreeze's concerns with how things have progressed so far.

deepfreeze · 2003-12-10 14:10:50

sarah: I also think that TURs are and can continue to be a good thing for Arch. I just think we need to define things a little bit better (which we are doing with this very thread) so there's less confusion about the overall scope of the TURs and what they are meant to accomplish. Like I said, if I didn't like the idea, I wouldn't be running one myself. But I think Xentac has done a very good job thus far in setting this up and now redefining things so everyone is on the same page.

ROB: I completely agree with you regarding adhering to a set of standards for building packages if they are going to be included in repos. If a TU decides that they want to build ultra-customized packages or things that aren't "standard" then we should have a way of advertising that fact so users aren't confused or accidentally hose their box.

Contra: I think you're still missing my point about incoming. Putting a disclaimer on there that "a TU may take your package and put it in their TUR unless you explicitly ask them not to" is, honestly, backwards. If a TU wants to add a package they find in incoming, then they should ask permission first, not second. A particular contributor may not want his work in your (or my) TUR and the burden should not be on him to say so, and track you down to make sure you are doing what he requests.

I think this is a very good, necessary and helpful conversation. I hope we can continue this so everyone can be on the same page and make TURs the success I think they can be

Xentac · 2003-12-10 15:09:26

farphel wrote:

Not to sound like I'm picking a fight or anything, but this just doesn't make any sense to me at all! I thought the original concept of the TURs was to alleviate the problem of packages sitting in /incoming so long they became stale. In other words, the TUR was a way to improve the process of getting packages tested and included into "unofficial". If that's true, how could you invent the TUR and implement it without knowing how it was going to solve the original problem?

Mwahaha. Now I'm being accused of not being thorough enough (Accused is probably the wrong word... I'm not saying it in a negative way). I had an idea, my idea, of how the TURs would help incoming become less stale but that doesn't mean that's the way it'll actually work. If I impose exactly what I believe is right, maybe the whole thing will fail because I didn't look at all the options? I have ideas how to get packages from the TURs and incoming into the repos but maybe it's not the most efficient way. Maybe other developers, or users, have better ideas than I do. I never claimed to be smart .

Just because I developed some software doesn't mean I know the best way to use it.

farphel wrote:

Of course, you did say in this thread that the TURs took off faster than you'd like; that everything up to now was supposed to be a beta test. However, I just don't see (and never have) how the TURs were supposed to improve the process of packages getting into "unofficial".
I'm really in agreement with deepfreeze's concerns with how things have progressed so far.

I'm starting to see the different sides of this very controversial topic. You're of the it-ain't-broke-so-don't-fix-it school of thought. That's just fine. I don't know how exactly I can show you what I see. My thought is that it is broke, because, when we first thought it up, incoming was very large and no one was available to look at it. It just kept getting larger. Then it was so big that no one wanted to look at it.

Incoming has had many casualties. Sarah doesn't want to work on it anymore, I don't really want to work on it anymore... some people have stepped up because they have more time but I figured having a few more people doing sub-developer work (checking packages and keeping them up to date for eventual inclusion) would help things out. I see now that I need a lot more order within these sub-developers and I'm working on that.

A little trivia for you. Did you know what Roberto (netkrash) started out doing exactly that? He started out checking over packages in incoming and making sure they followed the Archlinux guidelines. Then he fell off the earth for a while and things got messy again.

There are a lot of kinks to be ironed out, but I believe that the TURs can be a help more than a hinderance in the end.

red_over_blue · 2003-12-10 15:17:36

One of the things I love about Arch Linux is centralized package repositories. I love the fact that I don't have to hunt down repos to find a specific package, it's either there or it isn't.

Analogously, I would like the TURs to be centralized as well. This is the way I would like to see the TURs operate:

1. One single repository: I think the argument that users will pick and choose packages dependending on the TU is not that significant. I surely don't determine which maintainer is responsible for a package before installing it from the official repos.

2. No overlap. What I mean by this, is if a package is in the TUR, then there is only one version of that package, and one TU responsible for it.

3. Strict adherence to the Arch guidelines for building packages. I don't think this is an option. I wouldn't want to have to start double-checking everything I install to see if it is a proper Arch package or some uber-optimized non-conforming package. I know it sounds harsh, but if someone wants to share some soup-sandwich package they invented, let them find a way to host it themselves, and if someone wants it, they can download it manually and install it with pacman -A.

4. Packages should not be taken from incoming without first getting the permission of the person who submitted it there.

What I can see happening if tight controls are not placed on the TURs is that Arch will morph into something debian-like, where there are many repositories, each holding different packages, at different versions, and to me it just becomes a big mess.

NOTE - I think the thread on "packages in TURs removed from incoming" should be closed so that we aren't having the same conversation in two places.

farphel · 2003-12-10 15:35:42

Xentac wrote:

I'm starting to see the different sides of this very controversial topic. You're of the it-ain't-broke-so-don't-fix-it school of thought. That's just fine. I don't know how exactly I can show you what I see. My thought is that it is broke, because, when we first thought it up, incoming was very large and no one was available to look at it. It just kept getting larger. Then it was so big that no one wanted to look at it.

I understand completely!

Xentac wrote:

A little trivia for you. Did you know what Roberto (netkrash) started out doing exactly that? He started out checking over packages in incoming and making sure they followed the Archlinux guidelines. Then he fell off the earth for a while and things got messy again.

This is EXACTLY what I propose would work if we switched over to just ONE TUR with many TU's managing it. Rather than relying on one person (netkrash), the TUR is managed by several developer-appointed TUs so if one person leaves, another is there to keep up with the load.

The way I see the process working is:
1) Jane Doe submits a package that she built to /incoming.
2) A TU grabs the package, tests it and make sure it conforms to Arch guidelines and then puts it into the one-and-only TUR.
3) After it has been there for a while and other users and/or TUs have used/tested it, it can migrated into "unofficial".

So really, my spin on your idea is to make /incoming into a repository and rather than just having one person (netkrash) review it, it is managed by several developer-appointed TUs.

I think this solves the problem of new packages being submitted by any user, and a clear migration path into "unofficial". It is an Arch supported repo, but only in the sense that users understand that this is use-at-your-own-risk software (just like the existing TURs).

My $.02... I think I'm up to about $1.50 worth of opinions by now...

Xentac · 2003-12-10 16:23:51

red_over_blue and farphel, this is a good point. Having only one repo does get rid of the problem of duplicates and can concert (concerte?) all effort into one place... if it's true that the users don't care (after all, if they are using the TUR, they want certain packages and nothing will be forced on them anyway) then it's looking like I'll have to change the software I really don't care about that, it'll just take more time...

Hmmm... I'm gonna have to think about this more, because having one repo like that creates a certain problem that I'd rather not deal with... it will be discussed though.

andy · 2003-12-10 21:18:34

After digging through all the posts I want to make two more comments (I hope I wasn't too quick with my reading ;-) .. )

- about the TU's "raiding" incoming : I don't see this as a problem. One of my packages was taken by a TU, the TU mailed me, and I was (even before the mail) completely ok with it ! After all, I wanted my package to be in AL sometime, and I reagrd the TUR's as a pre-stage for inclusion.

- about that pre-stage : I think it would help if the TU's could maintain something like a top-10 list of packages. But the criteria is not popularity or age, but "works best with AL". This list does not need to be public or official or anything like that - just something a TU can come up with if a package maintainer asks. Otherwise, the situation may not be that much better for you developers. You want to and need to have easy work, don't you ? It is better to pick a designated package than a random package.

Arch Linux

#1 2003-12-09 21:33:23

TUR guidelines

#2 2003-12-09 22:19:08

Re: TUR guidelines

#3 2003-12-10 00:41:37

Re: TUR guidelines

#4 2003-12-10 01:02:04

Re: TUR guidelines

#5 2003-12-10 01:23:55

Re: TUR guidelines

#6 2003-12-10 02:03:44

Re: TUR guidelines

#7 2003-12-10 02:15:56

Re: TUR guidelines

#8 2003-12-10 02:28:07

Re: TUR guidelines

#9 2003-12-10 02:40:06

Re: TUR guidelines

#10 2003-12-10 02:49:10

Re: TUR guidelines

#11 2003-12-10 02:54:17

Re: TUR guidelines

#12 2003-12-10 04:21:52

Re: TUR guidelines

#13 2003-12-10 12:59:15

Re: TUR guidelines

#14 2003-12-10 14:10:50

Re: TUR guidelines

#15 2003-12-10 15:09:26

Re: TUR guidelines

#16 2003-12-10 15:17:36

Re: TUR guidelines

#17 2003-12-10 15:35:42

Re: TUR guidelines

#18 2003-12-10 16:23:51

Re: TUR guidelines

#19 2003-12-10 21:18:34

Re: TUR guidelines

Board footer