You are not logged in.

#1 2003-09-10 22:52:38

_JeffG_
Member
Registered: 2003-08-04
Posts: 61
Website

package dependencies and autodetection during building

OK, I updated my Arch box and pulled in the latest xmms package.  It installed fine but faild to run due to a missing library.  I think it was libasound or something.  At any rate, I rebuilt the package and reinstalled it with my built package and now xmms works. 

Now, here is the point.  This has been discussed a little with respect to other packages.  However, it does bring up a flaw in the abs system or packages in general.

The issue is depending on a ton of dependencies so that each install of the same package gets the same results.   Or each build of a package gets the same resulting package built unless modified prior to building.

So, in my xmms case, libasound or whatever must have existed on the machine where the latest xmms package was built and does not exist on mine but since it did exist at build time it got depended on yet it did not have the package providing libasound or whatever as a dependency since it never triggered my system to install it.

I think this is an issue that needs to be addressed because it was not the first time it has been mentioned. 

So that begs the question, should packages have large lists of dependencies so that when installed they get all the features working or should they be stripped down to the essentials. 

IMHO, I think package based systems need to depend on all possible popular features so that a working generic package can run correctly each time regardless of what was already installed. 

I mean xmms should not have broke by upgrading it via pacman -Syu but it did.

When the build process of a distributed package like xmms or such starts, there should be no guessing/auto detecting/probing  for libraries and stuff that are needed. 

That is why each package needs to be built in a clean environment before it is distributed.

Jeff

Offline

#2 2003-09-10 23:54:20

jlowell
Member
Registered: 2003-08-10
Posts: 270

Re: package dependencies and autodetection during building

_JeffG_

So that begs the question, should packages have large lists of dependencies so that when installed they get all the features working or should they be stripped down to the essentials.

It seem to me that a dependency is a dependency and that the installation of a package that advertises itself as including dependencies should include all of them without exception. If this is what you're saying, I would concur. I haven't noticed an inbox brimming with requests for my opinion about questions of this kind lately, so having me in your corner might be something of a dubious honor. smile

jlowell

Offline

#3 2003-09-11 00:52:52

_JeffG_
Member
Registered: 2003-08-04
Posts: 61
Website

Re: package dependencies and autodetection during building

Yes, that is what I am saying in one point.

The other was picking on the build system which some packages do not have their configure options tightly controlled to maintain consistency.

So, in a nutshell, the downloaded xmms package was linked to a library that was not on my system even though all the dependencies were met according to pacman

And that snuck in probably on a configure script checking for stuff which were not listed in the dependency list but got linked in anyway.

Same thing happened with KDE and CUPS.  Cups needed to be installed before kde was built so that dependency was compiled in. 

Strange stuff like that.

Jeff

Offline

#4 2003-09-11 00:52:55

sarah31
Member
From: Middle of Canada
Registered: 2002-08-20
Posts: 2,975
Website

Re: package dependencies and autodetection during building

_JeffG_ wrote:

OK, I updated my Arch box and pulled in the latest xmms package.  It installed fine but faild to run due to a missing library.  I think it was libasound or something.  At any rate, I rebuilt the package and reinstalled it with my built package and now xmms works.

this will happen from time to time with ANY distro. that is what bugtrackers are for. 

Now, here is the point.  This has been discussed a little with respect to other packages.  However, it does bring up a flaw in the abs system or packages in general.

The issue is depending on a ton of dependencies so that each install of the same package gets the same results.   Or each build of a package gets the same resulting package built unless modified prior to building.

if a package NEEDS another package then it WILL be in the dependency list UNLESS it is provided by a package ALREADY in the list. we try and build our packages very very minimalist...at least i do.

so what exactly is the problem with the build system?

So, in my xmms case, libasound or whatever must have existed on the machine where the latest xmms package was built and does not exist on mine but since it did exist at build time it got depended on yet it did not have the package providing libasound or whatever as a dependency since it never triggered my system to install it.

the user who built the xmms had alsa installed on their system and likely the xmms build source is set to detect whether a user has alsa or OSS installed on their sytem and builds accordingly. or perhaps it is now necessary to add options to the configure line of the build to compile it without alsa support in any case without furhter inspection of the xmms source don't assume this error was some sort of plot by the xmms package maintainer to screw you up.

just think if you think this small error and EASY means of fixing the package via abs is a fault then i wonder what you think about the trouble you would go through at one of the other precompiled distros to get it fixed. the fix you utilized is one of the things that make arch unique.....if its broke you can fix it yourself and then provide feedback to the community.

yes upgrading xmms should not have broke it but sometimes "errors" we make aren't even noticed until released. (i had issues building some of the sdl stuff without alsa. in their source your sound setup is autodetected and there is no optioning around it)

I think this is an issue that needs to be addressed because it was not the first time it has been mentioned.

sorry it merits discussion but to be absolutely honest there is no solution (see below). 

So that begs the question, should packages have large lists of dependencies so that when installed they get all the features working or should they be stripped down to the essentials.

 

without question minimalist is the ONLY way to build.

IMHO, I think package based systems need to depend on all possible popular features so that a working generic package can run correctly each time regardless of what was already installed.

well in that case your xmms will not work for you the next time either because alsa IS a popular choice. or what about xft? lots of people like it but i don't need fancy fonts and i know many many other serious arch users that give no regard for the extra bloat they add to packages. not too mention adding a popular feature adds dependencies which add to the number of the packages on the iso and your installed system. arch is minimalist we don't want to have a two disc install we want everybody to be able to use most apps in as wide a range of environments as possible. if you don't want alsa and i don't want xft (or some other gui crud) then we should not be forced to swallow it.

I mean xmms should not have broke by upgrading it via pacman -Syu but it did.

no it should not have but sometimes, as mentioned again, packages can be compiled in a way we had not anticipated, known, or be able to do anything about and yet still worked when we had tested it.

When the build process of a distributed package like xmms or such starts, there should be no guessing/auto detecting/probing  for libraries and stuff that are needed.

so true and a developer of source should notify the community of changes like sound system detection that they have made. we should not have to build test release only to find that there were changes not mentioned in the logs. 

That is why each package needs to be built in a clean environment before it is distributed

even a "clean" system cannot avoid build problems. besides a clean system goes against exactly the point you made earlier that every package should have every popular feature enabled.

bug will happen and will always happen on the whole though i do believe we have no more problems than other precompiled distros. remember forums are place where people bring their issues you rarely have people that post here saying the product is perfect (even i won't say it is perfect). i asume there are many more out there that have little or no issues with arch .

to sum up i and the other developers do indeed try and make package work in a way that almost everyone can use any application. we even sacrifice certain features that may make an app even more popular just so it can be used by a wider audience. we never try and deliberately break something for a certain group of users. in the case of xmms, again i have not looked into it, either the xmms developers made hidden changes and did not inform users or the detection of sound systems changes were missed by the maintainer.

personally i do like that you brought this bug to attention and were willing to express your opinions on dependencies. i am sure it will stimulate much discussion.


AKA uknowme

I am not your friend

Offline

#5 2003-09-11 01:09:55

apeiro
Daddy
From: Victoria, BC, Canada
Registered: 2002-08-12
Posts: 771
Website

Re: package dependencies and autodetection during building

_JeffG_ wrote:

The other was picking on the build system which some packages do not have their configure options tightly controlled to maintain consistency.

True, we don't explicitly --disable every possible option that we purposefully mean to exclude.  I tried that for a bit, and it turns out that a surprisingly large amount of configure scripts don't pay attention to --disable-yourfeaturehere and --without-yourotherfeaturehere switches.  Instead, they happily go on and detect the libraries anyway.

So for those of us w/o a dedicated build box, we have to shuffle packages and libraries around everytime we do a build.

_JeffG_ wrote:

I think this is an issue that needs to be addressed because it was not the first time it has been mentioned. 

That is why each package needs to be built in a clean environment before it is distributed.

Also true.  Now all we need are some clean, build-dedicated environments.  Care to donate any hardware?  I'm a student who works part time to pay tuition.

Offline

#6 2003-09-11 01:30:11

_JeffG_
Member
Registered: 2003-08-04
Posts: 61
Website

Re: package dependencies and autodetection during building

The minimalist aspect I like because it does keep things simple. 

I actually like the build system it was easy to pick up and learn.

I am not saying there is a plot or anything.  Quit taking criticism so negatively.

All I was saying that software these days tends to auto detect stuff during build time and that should be controlled as much as possible.

However, back to the xmms alsa thing.  When you do that ldd command to check to see what libs are depended on, then things like this could be seen and noticed and when alsa was detected then maybe the deps be updated or something.. 

I do not have the answers... just only I got annoyed when I started xmms today and it did not start.

Yes these things are easy to fix or maybe not so easy... and bugs do happen... but I was just bringing up the point so that these types of issues can be a minimum in arch.

regarding being forced to swallow stuff... why be forced...   alsa just did not work for me... or at least i could not get it to produce any sound even though the driver appeared to detect the card and all.. 

regarding a clean build.  that is to make sure the correct deps get installed so you no exactly what a particular package needs /wants...  my guess (correct me if I am wrong) is that if a package can not find something it absolutely requires that does not exist in a base install then the build process will fail at some point.  then you fix and repeat.  then you get resulting package that has the minimum dependancies...    as you said there is no real fix so... i guess that is why we have package versions.

I guess when pacman 3 or whatever comes out that has the provides option so we can have multiple packages doing the same task will hopefully cure this since xmms-alsa providing xmms or xmms-oss providing xmms will both fulfill that dependency.

At any rate, not much else to say about this.

Jeff

Offline

#7 2003-09-11 01:39:51

_JeffG_
Member
Registered: 2003-08-04
Posts: 61
Website

Re: package dependencies and autodetection during building

Also true. Now all we need are some clean, build-dedicated environments. Care to donate any hardware? I'm a student who works part time to pay tuition.

Well, I am a former student who can not find a good job and is stuck working part time at Target.  So, I am strapped for cash.  Otherwise, I would be using that cash to buy servers to maybe start my own business or something.

Don't get me wrong.  I like Arch Linux the best.  However, I have been living in the land of source based FreeBSD for a couple years and where everything got rebuilt all the time so this was a non-event.  Only true way to provide packages that are not going to complain in all cases are to provide static binaries... yeek... I do not even want to think of those. 

Small stuff is easy to rebuild and fix.  But big stuff like kde missing cups for example takes much more time to rebuild that package. 

Jeff - who just wants to have Arch be the best Linux

Offline

#8 2003-09-11 02:17:51

Toth
Member
Registered: 2002-12-04
Posts: 82

Re: package dependencies and autodetection during building

I can deal with broken packages from time to time. It happens. We're all only human.

However, dependencies should probably be double checked when packages are upgraded, just in case. In the case of the ALSA dependency for XMMS, I didn't think it was that big a deal. I figured libasound.so probably belonged to ALSA given the package I upgraded and the name of the library, so I just pulled down the ALSA packages and haven't had a problem. ALSA probably should have just been added as a dependency for the package.

Same with CUPS. A bug report should probably be filed for it and it should become a dependency of whatever component of KDE was built against it. I see the merit of not compiling packages with every known option, but in most cases, it's better to compile in support, IMO. But that's up to the maintainers. Look at the CUPS/KDE case. If KDE were not compiled against CUPS, then those that use CUPS and KDE would find themselves recompiling KDE. Not quick task. If CUPS is made a dependency of KDE, then those that use KDE will have to waste 8MB of disk space on CUPS. A program that they will never use. But, a program that will never slow their computer down either, since they don't use it. It's all about compromise, and I think non-CUPS users can deal with the 8MB of space CUPS takes so that others do not need to recompile KDE.

In short, packages should be checked for dependencies after being upgraded. If the package depends on something you hadn't anticipated, add it as a dependency or recompile (your choice...whatever you think is best).

That said, package maintainers do a thankless job by helping the rest of us stay up to date without the need to compile our own software. Thanks.

Offline

#9 2003-09-11 02:27:32

apeiro
Daddy
From: Victoria, BC, Canada
Registered: 2002-08-12
Posts: 771
Website

Re: package dependencies and autodetection during building

_JeffG_ wrote:

Well, I am a former student who can not find a good job and is stuck working part time at Target.  So, I am strapped for cash.

Well then, we're stuck in similar situations.  We need to find someone with a rich uncle.  smile

Small stuff is easy to rebuild and fix.  But big stuff like kde missing cups for example takes much more time to rebuild that package. 

Jeff - who just wants to have Arch be the best Linux

You're right about both package bugs.  They are errors, and we've made them before.  All the packagers (as far as I know) use their personal boxes to build packages for Arch, and this can cause problems with the build configuration.

Xentac has written a good package inspection utility that can catch some of these errors.  It's called namcap and there's a package for it in unofficial.  I haven't looked at it much, but it could be very useful in Arch's package build process.

Offline

#10 2003-09-11 02:48:41

_JeffG_
Member
Registered: 2003-08-04
Posts: 61
Website

Re: package dependencies and autodetection during building

I had one of those... rich uncles that is.  But lately he also has not been getting as much work as before.

And I also want to show my appreciation to the developers / packagers / maintainers / and anybody else that has contributed to Arch.  It would not be as good as it is without you all.  I just want to make it better.  Not crying.  Because I had fixed xmms before this thread was started.  I suspected it was alsa, but since alsa does not like me, I rebuilt xmms package and let it not detect alsa.   

At any rate,   keep up the good work.

Jeff

Offline

#11 2003-09-11 03:55:04

sarah31
Member
From: Middle of Canada
Registered: 2002-08-20
Posts: 2,975
Website

Re: package dependencies and autodetection during building

_JeffG_ wrote:

Quit taking criticism so negatively.

i will when people stop wording their comment in negative tones. do you think i respond to every negative post? hell no i don't i do respond to every post that leans towards a tone that we are not doning what we can to make the best arch linux we can.

just as it is frustrating for you to fire up xmms and it not work it is for me when i read user comments here that imply that we do not do what we should do. i spend alot of my time working on arch so it gets irritating.

All I was saying that software these days tends to auto detect stuff during build time and that should be controlled as much as possible.

yes it does and sometime it is not even documented so nor are options available to customize  the build. thankfully strict detect source is rare.

However, back to the xmms alsa thing.  When you do that ldd command to check to see what libs are depended on, then things like this could be seen and noticed and when alsa was detected then maybe the deps be updated or something..

i do and i am sure that maintaner of xmms does but when you spend months with having a build one way sometimes you get complacent and miss the small stuff such as ldd. of course your test work and you release it but little did you know ldd was a big thing this time.

accidents happen and bug reports should result is all i am saying. 

I do not have the answers... just only I got annoyed when I started xmms today and it did not start.

bah if you ran mp3blaster instead you would not have had that error tongue j/k i know how you feel that was what gentoo was like but instead of the occasional bug with an easy fix gentoo was frequent bugs with obscure fixes.

Yes these things are easy to fix or maybe not so easy... and bugs do happen... but I was just bringing up the point so that these types of issues can be a minimum in arch.

usually easy. and yes i recognize exactly why you brought up the point and so you know...to make sure that i do not miss dependency checking on the finished package i have added the namcap command to may makepkg alias.

see i do listen AND learn

regarding being forced to swallow stuff... why be forced...   alsa just did not work for me... or at least i could not get it to produce any sound even though the driver appeared to detect the card and all..

yeah alsa, like anything, can be hit and miss. my point here is that i really do thik minimalist or low extra features style f building should be the standard always. i think it is silly to have one bloated package or lots of little feature added packages. arch does this for the most part, but this does not stop some user for requesting features that add more than just a popular feature to a package. 

regarding a clean build.  that is to make sure the correct deps get installed so you no exactly what a particular package needs /wants...  my guess (correct me if I am wrong) is that if a package can not find something it absolutely requires that does not exist in a base install then the build process will fail at some point.  then you fix and repeat.  then you get resulting package that has the minimum dependancies...    as you said there is no real fix so... i guess that is why we have package versions.

jst what i was thinking but to be honest for most of the existing package i maintain would be a real hassle to clean build all the time because of the inter mingled dependencies. onthe other hand i should switch to building/testing incoming packages on my other mostly clean box...well it could be lightened alot easier than this one anyway it has some "'fat" .

I guess when pacman 3 or whatever comes out that has the provides option so we can have multiple packages doing the same task will hopefully cure this since xmms-alsa providing xmms or xmms-oss providing xmms will both fulfill that dependency.

i believe the current version of pacman has this feature. now don't get me wrong i like it but i really really really really hope that it does not result in a debian style package repo where there can be up to ten packages for one app but all built in a different manner (ie xmms-oss, xmms-alsa, etc (that is not a stab at you just i could not think of one of those multiformat packages debian is famous for...wait mysql?) ). that is just messy and i reall woud not want to spend all day making many versions of one package when other need attention too.

(now for kernels i am a little more forgiving but..;) )

thats about it for me. too


AKA uknowme

I am not your friend

Offline

#12 2003-09-11 17:35:28

apeiro
Daddy
From: Victoria, BC, Canada
Registered: 2002-08-12
Posts: 771
Website

Re: package dependencies and autodetection during building

_JeffG_ wrote:

And I also want to show my appreciation to the developers / packagers / maintainers / and anybody else that has contributed to Arch.  It would not be as good as it is without you all.  I just want to make it better.  Not crying.  Because I had fixed xmms before this thread was started.  I suspected it was alsa, but since alsa does not like me, I rebuilt xmms package and let it not detect alsa.   

At any rate,   keep up the good work.

Thanks, Jeff.  And please do continue to post any bugs that you see within Arch Linux.  Don't get me wrong, we want/need to be notified of issues like this, otherwise they slip right under the radar.

For the record though, namcap would have caught this error for us.

[root@saturn pkg]# namcap xmms-1.2.8-1.pkg.tar.gz
xmms       E: Dependency detected and not included (glib)
xmms       E: Dependency detected and not included (xfree86)
xmms       E: Dependency detected and not included (alsa-lib)

Handy little utility.  Some missing dependencies are okay, as they're covered farther up in the dependency tree.  For example, xfree86 and glib don't need to be depended on by xmms, since it depends on gtk which in turn requires those two.  But it caught alsa-lib right away.

I'll start using namcap on all my freshly-built packages.

Offline

Board footer

Powered by FluxBB