Maintaining self compiled packages

bananabrain · 2010-05-26 20:50:58

Hello folks,

I used the lovely abs~makepkg system to compile a couple of applications that I thought would benefit from being built on my own hardware.

Having done so, is there a preferred method of keeping these packages up to date? All I can think of is to keep syncing the abs tree and looking to see if there's a later version of the source available to recompile, but this would quickly become tedious with a large number of self-built packages. What does everyone else do?

Regardless of the above question, I'm a bit surprised to see that the source I downloaded and compiled is an older version than the binary in arch extra.
For example, after building vlc...

pacman -Qi vlc

Version          :  1.0.6-4
Install Reason   :  Explicitly installed



pacman -Si vlc

Repository       :  extra
Version          :  1.0.6-5

And because of this I have to add a line to /etc/pacman.conf...

IgnorePkg   =  vlc vlc-plugin

Was this the correct thing to do? Is there a preferred way to deal with these issues?

Thanks in advance.

Willie (devoted to Arch after barely a week...)

Last edited by bananabrain (2010-05-26 20:53:51)

brisbin33 · 2010-05-26 21:07:07

the abs tree is only updated once a day IIRC, you'll likely see the -5 version in abs tomorrow. i recommend you keep the IgnorePkg and when you see the warning about skipping upgrade just know that within a day or two you can rebuild via abs to get up to date.

you could also use a helper scripts (bauerbill, clyde, etc) but i'm unfamiliar with them or if they handle auto-abs-updates in any way.

jasonwryan · 2010-05-26 21:18:32

You could also look at adding these packages to a group: ataraxia's 'modified' seems an elegant solution that works for me: http://bbs.archlinux.org/viewtopic.php? … 41#p623841

wonder · 2010-05-26 21:23:42

if you do the above solution then you got a broken vlc. i'm saying that because -5 was recompiled against new x264. you will have to do it manually.

if you want to maintain your own package, i suggest to change pkgname to vlc-bananabrain or something like that. this is the best solution because:
1) it will not be updated from repo when we bump it
2) you will not report useless bugs because of what i said about about x264.
3) look for changes in svn. we got a nice web interface for that

Last edited by wonder (2010-05-26 22:28:45)

demian · 2010-05-26 22:54:29

Usually it's enough to just change the source array in the PKGBUILD.
How about a script that checks the urls for new sources? You could even integrate an automatic update of your PKGBUILDs.
You could also integrate an abs sync and diff so you get notified if for instance more than 2 lines changed so you know that there's been changes to the build process.

Last edited by demian (2010-05-26 22:58:25)

ngoonee · 2010-05-26 22:59:33

You could use bauerbill, it can be configured to automatically build some packages from source.

I doubt VLC benefits from being compiled though. Would it play your videos faster?

bananabrain · 2010-05-26 23:02:07

Thanks for the replies boys.

The abs vlc entry was indeed updated as brisbin said - just after I'd built the bugger

diff /home/willie/Build/abs/vlc/PKGBUILD /var/abs/extra/vlc/PKGBUILD

9c9
< pkgrel=4
---
> pkgrel=5

I'm still slightly confused though.

wonder wrote:

if you do the above solution then you got a broken vlc. i'm saying that because -5 was recompiled against new x264.

I think you mean that my new vlc build is broken simply because "-5" has better codec capabilities?

wonder wrote:

you will have to do it manually.

Do what manually? Remove my vlc and rebuild it from the updated abs tree?
If so, I have no problem with that, I was just wondering if there was some way to trap the situation and/or semi-automate the response.

wonder wrote:

if you want to maintain your own package, i suggest to change pkgname to vlc-bananabrain or something like that. this is the best solution because:
1) it will not be updated from repo when we bump it

I think I may have missed something fundamental. If I intend to always use a self-compiled version of any given package, does it follow that I must rename that package as you describe to avoid any interference from the standard repos when I `pacman -Syu`?
And if that's the case, doesn't the same apply to anyone building any package using abs?

Lastly, how do I give a package swiped from the abs tree a new name? Is it just an edit to PKGBUILD...?

from:
pkgname=('vlc'
         'vlc-plugin')

to:
pkgname=('bananabrain-vlc'
         'bananabrain-vlc-plugin')

Many thanks for all your help.

demian · 2010-05-26 23:02:33

Btw: Where you can make a real difference is compiling the kernel from source with for instance ck-patchset and 1000HZ timer frequency (and maybe optimized cpu architecture et al).

bananabrain · 2010-05-26 23:10:14

ngoonee wrote:

You could use bauerbill, it can be configured to automatically build some packages from source.
I doubt VLC benefits from being compiled though. Would it play your videos faster?

hmm... odd that you say that - although I'm always receptive to the experience of others.
I read somewhere just yesterday that video software is an ideal candidate to optimise for maximum pokiness.

Last edited by bananabrain (2010-05-26 23:12:03)

wonder · 2010-05-26 23:25:27

pkgrel=4 -> pkgrel=5 doesn't mean anything in abs. but a reason exists and that is in commit message. when i said that is rebuilt against new x264, that means that x264 had a soname bump and vlc needed a rebuild to link against this new one.

about my suggestion. i only give you those because i've seen that you wanted to add it to ignore and because of that you will end up with broken packages(see my first paragraph from this post) and vlc will not start. i've seen a lot of cases like you and this ended up on bugtracker which in final was not a bug at all.

ngoonee · 2010-05-27 00:24:28

bananabrain wrote:

ngoonee wrote:
You could use bauerbill, it can be configured to automatically build some packages from source.
I doubt VLC benefits from being compiled though. Would it play your videos faster?
hmm... odd that you say that - although I'm always receptive to the experience of others.
I read somewhere just yesterday that video software is an ideal candidate to optimise for maximum pokiness.

My opinion is that it doesn't matter except for software with algorithms which are CPU-limited. While video software does fall into this category, as I understand the issue is more a scheduler issue than any 5-10% gains you can make by self-compiling (especially if you're using default flags which then means you're getting NO benefit).

Please pay attention to wonder as well. When a library is updated, ALL packages which depend on it must be recompiled. The issue is not -4 to -5, its that your self-compiled package expects version .x of a library when its actually at version .y, and will die a horrible death if its not recompiled. Which is another reason NOT to self-compile packages without knowing what you're doing and actually having a need.

brisbin33 · 2010-05-27 01:54:14

wonder wrote:

pkgrel=4 -> pkgrel=5 doesn't mean anything in abs. but a reason exists and that is in commit message. when i said that is rebuilt against new x264, that means that x264 had a soname bump and vlc needed a rebuild to link against this new one.
about my suggestion. i only give you those because i've seen that you wanted to add it to ignore and because of that you will end up with broken packages(see my first paragraph from this post) and vlc will not start. i've seen a lot of cases like you and this ended up on bugtracker which in final was not a bug at all.

regardless of what the op does or did, i think the moral here should be any packages you build by hand either via abs or from the aur must be rebuilt when there's a soname bump

how you find out about or deal with these soname bumps is kind of up to you. if it were me, (as i said) i'd keep the ignorepkg and use the pacman warning as a signal to rebuild the package. as long as you understand that abs will lag slightly and your stuff might be broken until you rebuild* i think you'd be fine.

if you were to rename the package to foo-vlc, you'd never know when the repo vlc was upgraded (possibly due to a soname bump) and you'd never be nagged to rebuild against a new lib, i see this as a more breakage-prone scenario.

*also note that in this soname bump scenario, even rebuilding an out of date -4 vlc would fix a broken package.

/sidebar

Last edited by brisbin33 (2010-05-27 01:55:54)

bananabrain · 2010-05-27 08:15:10

I see this morning vlc is at "-6"!

I'm a bit slow, but I think I'm getting it.

It it the case then that when I run a system upgrade, pacman is looking at the version numbers of every package dependency and thereby marks for upgrade any target package who's library dependencies have been updated?

I'm doing this on an aspire one netbook:

CFLAGS="-O2 -march=atom -mtune=atom -msse3 -mfpmath=sse -fomit-frame-pointer -pipe"
CXXFLAGS="${CFLAGS}"
MAKEFLAGES="-j3"

...I wasn't sure about O2/O3.

Thanks especially for the last two posts which have helped my understanding of what's going on here.

brisbin33 · 2010-05-27 13:28:59

bananabrain wrote:

It it the case then that when I run a system upgrade, pacman is looking at the version numbers of every package dependency and thereby marks for upgrade any target package who's library dependencies have been updated?

bananabrain,

what's happened in this case is that there was a "soname bump", meaning a shared library was updated. this means that any packages (like vlc) that are using the current version of this shared library need to be "rebuilt" to link against the updated library.

there's no change whatsoever to the vlc application itself (meaning no upstream upgrade), it was just recompiled by the maintainer into a new package that's linked against the updated library; then this package is shared with you via the repos. this is why pkgrel is incremented, and not pkgver

now, pacman simply looks at "pkgver-pkgrel" and sees if the repo is > the currently installed. if so, it notifies you of an available upgrade. it's up to you how to handle that given your specific situation.

make sense?

Last edited by brisbin33 (2010-05-27 13:32:34)

bananabrain · 2010-05-28 00:09:06

brisbin33 wrote:

make sense?

It makes perfect sense now.
That was an ace explanation that also helped me understand wonder's posts, the rest of this thread, and probably the most important aspect of how pacman does its thing - thanks for taking the time to write it.

Now to decide on the best way to proceed. I'll take a look at bauerbill, being very slightly idle and all...

Arch Linux

#1 2010-05-26 20:50:58

Maintaining self compiled packages

#2 2010-05-26 21:07:07

Re: Maintaining self compiled packages

#3 2010-05-26 21:18:32

Re: Maintaining self compiled packages

#4 2010-05-26 21:23:42

Re: Maintaining self compiled packages

#5 2010-05-26 22:54:29

Re: Maintaining self compiled packages

#6 2010-05-26 22:59:33

Re: Maintaining self compiled packages

#7 2010-05-26 23:02:07

Re: Maintaining self compiled packages

#8 2010-05-26 23:02:33

Re: Maintaining self compiled packages

#9 2010-05-26 23:10:14

Re: Maintaining self compiled packages

#10 2010-05-26 23:25:27

Re: Maintaining self compiled packages

#11 2010-05-27 00:24:28

Re: Maintaining self compiled packages

#12 2010-05-27 01:54:14

Re: Maintaining self compiled packages

#13 2010-05-27 08:15:10

Re: Maintaining self compiled packages

#14 2010-05-27 13:28:59

Re: Maintaining self compiled packages

#15 2010-05-28 00:09:06

Re: Maintaining self compiled packages

Board footer