pacman wrapper to allow source builds (srcpac)

Xentac · 2004-05-29 08:49:00

This is mostly just something that I did because I had never written a wrapper for pacman before and wanted to see if it worked with the current pacman feature set.

It's 1:30ish in the morning, so my eyes are a little blurry.

The wrapper is called srcpac. It's version 0.1 at this point

The code is quite ugly because I threw it together in the past 3 hours, but my inital tests have been successes.

Now, for the pitch. Imagine if you could use pacman to install binary packages as well as build those same packages from source (with your own optimizations). Not only that, but upgrades remembered which packages were precompiled and which were compiled on the spot.

Now, imagine that pacman didn't actually handle that, because the hard-core KISS people would complain too much and you wouldn't be able to think while you watch gcc output flash by. Imagine instead a wrapper around pacman that handled it for you.

Enter srcpac. Lights brighten, music comes to a peak, close up!

srcpac acts exactly the same as pacman (well, it should anyway), except you get this little extra flag -b (or --build) tacked on to -S (--sync) to build from source instead of install from a binary package.

Let's see an example:

srcpac -Sb w3m
will compile w3m, install it, and remember that it was compiled

srcpac -Sybu
will upgrade all the packages that need upgrading by building them and remember that they were compiled (be careful with this one, it will include packages that weren't previously compiled).

srcpac -Syu
will upgrade all binary packages by installing binaries and upgrade all source packages by compiling from source.

srcpac -Qi w3m adds an extra line too, "Source", with a simple yes or no.

Which packages are source and which aren't are stored in /var/lib/srcpac just as files. If the file exists, then the package was a source package (so it's easy to fix it if srcpac gets confused, also it doesn't touch anything pacman uses).

While I can guarantee that this wrapper won't break your packages (because it uses pacman and makepkg to do all of it's package related stuff), I can't guarantee it'll work 100% correctly (it's 1:45 now!).

If anyone is interested, the script is available here: http://xentac.net/svn/arch-tools/srcpac/tags/0.1/srcpac

Yes, it's in bash.

Feel free to tell me problems you run into using this tool. Remember, if you use pacman instead of using srcpac, then all the packages will continue installing as binary packages.

BEWARE: the -r option isn't fully implemented and may install to / in some cases, I'm pretty sure I don't ever write to / when -r is specified, but I can't be sure.

.:edit - changed title to show srcpac name - dibble:.

Mr Green · 2004-05-29 08:58:54

If this proves popular ... could this feature be added to pacman in future versions... --build option :?:

Get some sleep

Mr Green....

tehdely · 2004-05-29 12:43:16

I have wanted something like this for a long time.

Thank you, Xentac.

Xentac · 2004-05-29 17:30:00

Mr Green wrote:

If this proves popular ... could this feature be added to pacman in furture versions... --build option :?:
Get some sleep
Mr Green....

If it proves popular, why not just leave it the way it is? The design works fine the way it is.

Mr Green · 2004-05-29 17:39:56

me & my big mouth.....

Dusty · 2004-05-29 18:12:19

Wow.

Bows to Xentac with great respect.

Xentac · 2004-05-29 18:22:51

I've been talking to contrasutra about support for custom package configuration changes. So that your gaim package can be compiled with spell support, for example. As long as the PKGBUILD format doesn't change a lot, it'll keep working across upgrades.

I'll probably implement this sometime when school isn't so overbearing.

It'll use sed, so you'll have to have basic knowledge about how a sed substitute works.

Zephirias · 2004-05-29 18:42:31

So...I'm assuming that by what you're saying, we could just host source tar.gz files in a repository, and the command could download it and configure it, like a PKGBUILD? That's what I'm getting out of it... :?

So, if the above was true...it'd be like a Pacman/ABS type-system integrated into one, with the ability to use binary packages or source packages with ease...that's be sweet.

Xentac · 2004-05-29 19:04:25

Zephirias wrote:

So...I'm assuming that by what you're saying, we could just host source tar.gz files in a repository, and the command could download it and configure it, like a PKGBUILD? That's what I'm getting out of it... :?
So, if the above was true...it'd be like a Pacman/ABS type-system integrated into one, with the ability to use binary packages or source packages with ease...that's be sweet.

No... more like, you store how you want the official PKGBUILD to change so that it's configured the way you want it to. Before building, the PKGBUILD is updated to reflect your changes. No source .tar.gz files.

srcpkg only works with abs right now. It'll need a bunch of re-thinking to work with other repositories.

rls · 2004-05-30 18:26:16

For building packages from source, there is a magnificent distribution named Gentoo. For a distro based on binaries, we have the magnificent Arch. Heaven would be a (meta) distribution using Gentoo if one wanted to build from source and Arch if one wanted to use binaries. Yes, one can use binaries with Gentoo, but it is nowhere near as nice as Arch. Seems like something of an afterthought. I reserve judgment on building packages from source in Arch (other than the old fashioned way in /usr/local).

Final thought. I couldn't image using Arch with non-broadband access, just too slow to download. With broadband, downloading is no longer a problem. However compiling still is with one box. My box requires over a day to build KDE. However, if you have several boxes, distcc could be the white knight. If one could build KDE in an hour......

Xentac · 2004-05-30 18:33:17

rls wrote:

For building packages from source, there is a magnificent distribution named Gentoo. For a distro based on binaries, we have the magnificent Arch. Heaven would be a (meta) distribution using Gentoo if one wanted to build from source and Arch if one wanted to use binaries. Yes, one can use binaries with Gentoo, but it is nowhere near as nice as Arch. Seems like something of an afterthought. I reserve judgment on building packages from source in Arch (other than the old fashioned way in /usr/local).
Final thought. I couldn't image using Arch with non-broadband access, just too slow to download. With broadband, downloading is no longer a problem. However compiling still is with one box. My box requires over a day to build KDE. However, if you have several boxes, distcc could be the white knight. If one could build KDE in an hour......

That's.... uhhhh... great...

How it relates to srcpac, I'm not quite sure.

Zephirias · 2004-05-30 18:52:25

Xentac, aside from his mention his compiling sluggishness (I know of that, too :cry: ), I think what's he's saying is that a combination of a Gentoo-type system and an Arch type system with the full features of both into one would be great...but as a "meta" distro, whcih I assume he's defining as "popular".

Dusty · 2004-05-30 18:57:01

rls wrote:

For building packages from source, there is a magnificent distribution named Gentoo. For a distro based on binaries, we have the magnificent Arch.

Bull. Arch is both binary oriented (pacman) and source oriented (ABS). Most people use binary only, a few people do it source only. People that need unavailable packages use both methods.

I reserve judgment on building packages from source in Arch (other than the old fashioned way in /usr/local).

in other words you don't know about, or don't like ABS and its simplicity??? :-$

Final thought. I couldn't image using Arch with non-broadband access, just too slow to download.

So funny, I never had any problem with it. I complain about dialup a lot, but I'm having no trouble with Arch on 33kbps.

Dusty

Dusty · 2004-05-30 18:57:49

Zephirias wrote:

Xentac, aside from his mention his compiling sluggishness (I know of that, too :cry: ), I think what's he's saying is that a combination of a Gentoo-type system and an Arch type system with the full features of both into one would be great...but as a "meta" distro, whcih I assume he's defining as "popular".

Uh... excellent summary? why not just say "me too"?

Zephirias · 2004-05-30 19:31:27

Dusty wrote:

Zephirias wrote:
Xentac, aside from his mention his compiling sluggishness (I know of that, too :cry: ), I think what's he's saying is that a combination of a Gentoo-type system and an Arch type system with the full features of both into one would be great...but as a "meta" distro, whcih I assume he's defining as "popular".
Uh... excellent summary? why not just say "me too"?

Bah, I'm sorry that I analyze things.

rls · 2004-05-31 01:19:07

First, one definition of meta is beyond, transcending, higher as in metaphysics (although I seriously doubt metaphysics transcends physics in reality.) Anyway, that was my rational for using "meta" distribution.

My reference to the /usr/local method was the sequence download-untar/unzip-configure-make make install. That is a fine methodology as long as nothing bombs in the configure or make stages that you can't diagnose. Perhaps it is "child's play" for all you CS types.

ABS is fine, but unless I am mistaken, it does nothing to ensure the configure and make stages go smoothly. It is a good way to integrate "home-rolled" packages into the Arch system.

Regards, Rick

Dusty · 2004-05-31 02:17:21

rls wrote:

ABS is fine, but unless I am mistaken, it does nothing to ensure the configure and make stages go smoothly. It is a good way to integrate "home-rolled" packages into the Arch system.

hmmmm... I could be wrong because I've never used Gentoo, but if you make a package that doesn't already exist for Gentoo, does it do anything to make sure the compilation goes smoothly? If the package exists, then there is a way to build the package that has been tested by somebody else. This is how ABS works too; if a PKGBUILD exists, you can be reasonably sure it will work.

I can't imagine a program that can automatically fix or recover from compiler or Makefile errors. If it does, then... wow.

I assume that Gentoo has a larger package base than Arch, but let's not get into that discussiong again!

Xentacs script is basically designed to allow you to choose whether you are going to install from source or binary. Assuming the PKGBUILDS are in order (which for arch repository programs they are, because the binaries were built from them!), this should work as flawlessly as installing from binaries.

Dusty

rls · 2004-05-31 02:46:04

To the best of my knowledge, if you want a package not in the Gentoo repository you are on your own. If it is in the repository, customization/fine-tuning is easy. Perhaps it is easy with srcpac.

Egil.B · 2004-06-01 07:33:57

Dusty wrote:

hmmmm... I could be wrong because I've never used Gentoo, but if you make a package that doesn't already exist for Gentoo, does it do anything to make sure the compilation goes smoothly? If the package exists, then there is a way to build the package that has been tested by somebody else. This is how ABS works too; if a PKGBUILD exists, you can be reasonably sure it will work.
Dusty

When I removed Gentoo to try Arch, I had around 70 updates waiting because one shitty dependency wouldnt compile.

Mr Green · 2004-06-01 17:25:40

you had just 70 ... thats why I wiped it from my HD

Mr Green..

iotc247 · 2004-06-02 19:34:08

What do you do with that script. If you need to install it how do you... Im new to arch as you can see... And ive always done binary packages.. Just hoping this will get kde to work for me by building it.

jlvsimoes · 2004-06-04 19:02:12

cant i do something like that using my repo with my tweaked PKGBUIDLS
and using makepkg -si ?

Xentac · 2004-06-04 20:17:38

You could, but every upgrade, you'd have to tweak some more. This way should be more automatic.

Xentac · 2004-06-06 05:35:17

I actually created a package. Anyone who wants to use srcpac to actually manage source packages can install it.

I've also added the support for package configs. There's no documentation on it, but it uses sed and is decently easy to understand. Have a look at the example /etc/srcpac.conf entry. I know it works because that's what I used to test with

Version 0.2 also includes the -o option to -S. This will apply config changes and less the PKGBUILDs without doing any building. I figured this would be a good way to test configs without a lot of manual labour.

I'll get to a manpage sometime later, maybe in version 0.3.

kakabaratruskia · 2004-06-06 20:18:34

I've been using it, and I have to say that it's a wonderful app. I know that you don't want to put it in pacman, because KISS people would complain, but I say, screw them, this tool is the best. I say "Revolution.........."

Arch Linux

#1 2004-05-29 08:49:00

pacman wrapper to allow source builds (srcpac)

#2 2004-05-29 08:58:54

Re: pacman wrapper to allow source builds (srcpac)

#3 2004-05-29 12:43:16

Re: pacman wrapper to allow source builds (srcpac)

#4 2004-05-29 17:30:00

Re: pacman wrapper to allow source builds (srcpac)

#5 2004-05-29 17:39:56

Re: pacman wrapper to allow source builds (srcpac)

#6 2004-05-29 18:12:19

Re: pacman wrapper to allow source builds (srcpac)

#7 2004-05-29 18:22:51

Re: pacman wrapper to allow source builds (srcpac)

#8 2004-05-29 18:42:31

Re: pacman wrapper to allow source builds (srcpac)

#9 2004-05-29 19:04:25

Re: pacman wrapper to allow source builds (srcpac)

#10 2004-05-30 18:26:16

Re: pacman wrapper to allow source builds (srcpac)

#11 2004-05-30 18:33:17

Re: pacman wrapper to allow source builds (srcpac)

#12 2004-05-30 18:52:25

Re: pacman wrapper to allow source builds (srcpac)

#13 2004-05-30 18:57:01

Re: pacman wrapper to allow source builds (srcpac)

#14 2004-05-30 18:57:49

Re: pacman wrapper to allow source builds (srcpac)

#15 2004-05-30 19:31:27

Re: pacman wrapper to allow source builds (srcpac)

#16 2004-05-31 01:19:07

Re: pacman wrapper to allow source builds (srcpac)

#17 2004-05-31 02:17:21

Re: pacman wrapper to allow source builds (srcpac)

#18 2004-05-31 02:46:04

Re: pacman wrapper to allow source builds (srcpac)

#19 2004-06-01 07:33:57

Re: pacman wrapper to allow source builds (srcpac)

#20 2004-06-01 17:25:40

Re: pacman wrapper to allow source builds (srcpac)

#21 2004-06-02 19:34:08

Re: pacman wrapper to allow source builds (srcpac)

#22 2004-06-04 19:02:12

Re: pacman wrapper to allow source builds (srcpac)

#23 2004-06-04 20:17:38

Re: pacman wrapper to allow source builds (srcpac)

#24 2004-06-06 05:35:17

Re: pacman wrapper to allow source builds (srcpac)

#25 2004-06-06 20:18:34

Re: pacman wrapper to allow source builds (srcpac)

Board footer