You are not logged in.
Hi there,
Sorry if this is a double post.
I know of srcpac, but have it been discussed if pacman should have an option to compile packages? It doesn't seam like a particularly big feature---I would even be willing to make it myself. If this has been discussed, what were the main arguments pro/con?
Jacob
Offline
Errr, why not use makepkg for that?
Edit: And wrt to srcpac: https://bugs.archlinux.org/task/36140
Last edited by karol (2013-08-04 14:35:27)
Offline
If you're not familiar with ABS, you should read up on it, once your abs tree is up to date just `cp /var/abs/${repo}/$pkg/* ./ && makepkg -si`
If that is too much, you can make a tiny bash function that will - given a package name - find which repo it is in, make sure the abs tree is up to date, make a suitable build dir, and run the above commands. I have such a function in one of my bashrc's.
Last edited by Trilby (2013-08-04 14:45:13)
"UNIX is simple and coherent..." - Dennis Ritchie, "GNU's Not UNIX" - Richard Stallman
Offline
Errr, why not use makepkg for that?
For several reasons---Primarily ease of use. I don't really follow your logic. makepkg is not a package manager. It doesn't handle updates etc.
You could make the argument that it is not the package manager's job to compile the packages. Yet, a similar argument could be made that it is not the package manager's job to fetch the packages, though most would agree that this is a nice feature to have in pacman.
Ease of use:
pacman --build -S package
pacman --build -Syu
That second line could take some time with makepkg and pacman -U!
Jacob.
Last edited by jacobgrevald (2013-08-04 14:57:21)
Offline
The authors specifically don't want to obfuscate that part of it; if you're going to be building packages, you need to know how to use makepkg, not just a frontend.
Offline
If you're not familiar with ABS, you should read up on it, once your abs tree is up to date just `cp /var/abs/${repo}/$pkg/* ./ && makepkg -si`
If that is too much, you can make a tiny bash function that will - given a package name - find which repo it is in, make sure the abs tree is up to date, make a suitable build dir, and run the above commands. I have such a function in one of my bashrc's.
Yes. And it would be just as easy to make a bash script which uses wget to fetch packages. Yet this functionality is part of pacman.
The ability to do without, does not alone justify leaving the feature out of pacman. My question was not how to do this, but rather if it has been discussed to make it a part of pacman.
Offline
I don't know how about you, but I use aliases, like 'msi' for 'makepkg -si' and 'psyu' for 'pacman -Syu'.
If you want to rebuild many packages, you need a working srcpac or a similar tool.
Offline
Offline
Yaourt has exactly this feature, 'yaourt -Sb <package>' or 'yaourt -Syub' work just fine. I use it only occasionally, but I agree with OP that having such thing in pacman would be a good thing.
I don't see any problem with separating responsibilities, as long as pacman just calls makepkg to do the built. Just as 'makepkg -i' calls pacman and nobody has problem with that. If the idea would be to reimplement the package building in its own code, then of course it would be breaking the responsibility separation, but the way OP puts it, it seems OK to me.
I second his question, whether someone actually thought about this and/or discussed it before? Nobody in this thread has even tried to answer that yet...
Offline
I still don't understand the ease of use thing: if you can use an alias, why does it matter if it's makepkg, pacman, yaourt or something else?
Offline
I second his question, whether someone actually thought about this and/or discussed it before? Nobody in this thread has even tried to answer that yet...
Allan posted on pacman-dev some time ago about the idea of adding a -B option to build via pacman. It didn't really go anywhere. Neither will this thread/idea if it isn't brought up with the right people, cogent reasons presented, and patches offered.
Offline
As far as I know (which may not be all that far in that case - so correct me if I'm wrong) pacman doesn't have a currently built in mechanism to get PKGBUILDS. Built pkg.tar.gz files which are downloaded from repo mirrors do not include the PKGBUILD. It seems it would be trivial to have pacman call makepkg as they are part of the same package: if you have pacman, you have makepkg. This is not the case for ABS though. It seems such an option would *require* abs, or a substitute for it. So while a command line switch would be easy to add, this would also be adding another dependency and the requirement of maintaining (at least a partial) abs tree on the user's system.
I'm quite happy to have this package, and the abs tree - but I'm also happy that I chose to install it as a separate tool. I can't see what possible advantage merging the two packages (pacman + abs) would serve. The *only* thing mentioned so far is the alleged ease of use, but this has been answered with the suggestion of an alias. Or I'll go one further, a function:
pacman() {
if [[ "$@" =~ "--build" ]]; then
# insert previously mentioned abs function code
else
shift
/usr/bin/pacman $@
fi
}
"UNIX is simple and coherent..." - Dennis Ritchie, "GNU's Not UNIX" - Richard Stallman
Offline
What is the advantage of building ABS packages if you aren't customising them at all? (I assume a --build option for pacman would not customise them if it worked in the way suggested i.e. updating abs, copying the relevant build recipe, compiling and installing it.)
CLI Paste | How To Ask Questions
Arch Linux | x86_64 | GPT | EFI boot | refind | stub loader | systemd | LVM2 on LUKS
Lenovo x270 | Intel(R) Core(TM) i5-7200U CPU @ 2.50GHz | Intel Wireless 8265/8275 | US keyboard w/ Euro | 512G NVMe INTEL SSDPEKKF512G7L
Offline
It depends what you mean by "customizing". One can define compiler flags and machine-specific optimizations in makepkg.conf which would be applied.
"UNIX is simple and coherent..." - Dennis Ritchie, "GNU's Not UNIX" - Richard Stallman
Offline
It seems such an option would *require* abs, or a substitute for it.
The dependency on abs could be optional, using -B without having abs installed would result in error stating that abs should be installed first...
Also, each PKGBUILD can be downloaded separately from the mirrors, e.g.: http://ftp5.gwdg.de/pub/linux/archlinux … pkg.tar.xz. This could be used as a fallback in case abs is not installed.
Offline
Yaourt has exactly this feature, 'yaourt -Sb <package>' or 'yaourt -Syub' work just fine. I use it only occasionally, but I agree with OP that having such thing in pacman would be a good thing.
I don't see any problem with separating responsibilities, as long as pacman just calls makepkg to do the built. Just as 'makepkg -i' calls pacman and nobody has problem with that. If the idea would be to reimplement the package building in its own code, then of course it would be breaking the responsibility separation, but the way OP puts it, it seems OK to me.
I second his question, whether someone actually thought about this and/or discussed it before? Nobody in this thread has even tried to answer that yet...
First, if you want a robust implementation of source-installs (download src, compile, install), ditch yaourt and go for gentoo because it implements the procedure the right way. However, even their approach has problems IMHO:
1. Make dependencies. Package X needs pkgs M_1, M_2, ... to build which has to be installed on your system. That's either unnecessary bloat or disk writes if you remove M_x after each source-install.
2. Building as root. To avoid this, pacman will have to somehow drop priviliges for building...
Have you guys given thought to addressing these issues? Simply put, building and installing packages in a single step is messy. Even so that at some point FreeBSD people tried to fix this in their ports system,
http://forums.freebsd.org/showthread.php?t=6975 (don't know how the story ended though).
Last edited by Leonid.I (2013-08-04 18:49:19)
Arch Linux is more than just GNU/Linux -- it's an adventure
pkill -9 systemd
Offline
It depends what you mean by "customizing". One can define compiler flags and machine-specific optimizations in makepkg.conf which would be applied.
Oh, I see. So is the idea to compile everything rather than just particular packages?
CLI Paste | How To Ask Questions
Arch Linux | x86_64 | GPT | EFI boot | refind | stub loader | systemd | LVM2 on LUKS
Lenovo x270 | Intel(R) Core(TM) i5-7200U CPU @ 2.50GHz | Intel Wireless 8265/8275 | US keyboard w/ Euro | 512G NVMe INTEL SSDPEKKF512G7L
Offline
If you're not familiar with ABS, you should read up on it, once your abs tree is up to date just `cp /var/abs/${repo}/$pkg/* ./ && makepkg -si`
If that is too much, you can make a tiny bash function that will - given a package name - find which repo it is in, make sure the abs tree is up to date, make a suitable build dir, and run the above commands. I have such a function in one of my bashrc's.
I usually use the SVN tree. The main problem with ABS is that you don't usually know why a given PKGBUILD has a particular entry. So, you have go to svntogit (web interface) and read packager's comments. I guess ABS could include SVN history via a "changelog" but that'll be reinventing the wheel
Last edited by Leonid.I (2013-08-04 19:01:05)
Arch Linux is more than just GNU/Linux -- it's an adventure
pkill -9 systemd
Offline
Hi there,
...has it been discussed if pacman should have an option to compile packages? It doesn't seam like a particularly big feature.....
I think that it is not only a difficult feature to add, but from my experience it would end up making something other than "Archlinux". The idea of building our packages from source code on the user's system breaks the Arch development model. The binary packages that are released as Archlinux are built on developer's machines against the libraries and tools that the developer happens to have installed at that moment. There is a huge discipline required for the dev's to stage some changes that impact one another and to serialize changes that involve soname bumps in libraries, but that does not guarantee that the day after a binary package is released by Arch you can successfully build it on your machine using a currently up-to-date system and the released PKGBUILD for that package. For example a recently released dependency may have removed some macros and headers, but not broken the library interface (no soname bump). The Arch devs will eventually have to correct (patch) all other packages that build against that dependency, if upstream doesn't fix it first, but they don't do it immediately. They do it the slacker way, "eventually when it's really necessary".
In other words, there is no Archlinux build server and packages are not routinely rebuilt (Allan routinely rebuilds core and ensures that it builds). Really that's the price you pay for continuously bleeding-edge packages. I don't think there is enough time in the day for Arch devs to also ensure that all of their packages are continuously buildable. It's actually a big task.
That's one of the things that differentiates Arch from the Big Boys like Fedora and OpenSUSE. When they put out a 6-month or 8-month release they check that everything in the release is rebuildable. In a rolling-release the only way to guarantee that all of the packages are rebuildable from source code would be to rebuild everything continuously (with a continuous build server or force users to do it like Gentoo does).
I'm not arguing against working from ABS and building packages from source -- I don't have any Arch binaries on my Archlinux system. Everything on my system is built from abs/aur pkgbuilds but achieving that requires that I do little releases of my own with periods of "freeze". It forces me to develop my system more like Debian (but faster with my "releases" coming every couple of months).
Summary: pacman should not have a build option because it's bad factoring and it would promote a false expectation.
Offline
So is the idea to compile everything rather than just particular packages?
Not necessarily everything. If/when compiler optimizations make a noteworthy difference, they'd make that difference even on single packages. So some users may opt for compiling "everything" if they are patient and/or have a fast system (though if their system is that fast, I'd ask why they need the optimizations), others may compile just select packages: either those that are used most frequently, most likely to benefit from machine specific optimizations, or otherwise are strategically chosen.
For example, I use urxvt *a lot* - I've compiled my own urxvt with optimizations for my system. I don't know if it really makes a difference though, and this was not really the purpose of recompiling urxvt, I was actually making configuration changes.
"UNIX is simple and coherent..." - Dennis Ritchie, "GNU's Not UNIX" - Richard Stallman
Offline
As falconindy pointed out above, I want this in pacman. The initial parts would be quite easy to add and would replace the steaming pile of crap that is ABS.
Patches we need:
1) Option in repo-add to add details of the source package (if the coreesponding foo-1-1-i686.src.pkg.tar.gz is in the directory with the binary package, its details including signature get added to the repo db)
2) Add -B option to pacman, initially only supporting -Bw to download the source package.
Those two steps set us up to be able to build packages, but more importantly provide a superior system than the current ABS. Actual building etc (and patching the files before that build) are all features that require some actual thought.
But as always, we need someone to do the work. I barely get through reviewing patches these days and this is not in my high priority queue of things to implement. So if someone steps up and does the work....
Offline
Hmm... Maybe I should try it for kile/okular. kile/okular are much slower on my Arch machine than my Fedora box at work even though my Arch machine is generally faster and has generally higher specs (better cpu, more memory etc.). Then again, I wonder how much time I would spend to save each second!
CLI Paste | How To Ask Questions
Arch Linux | x86_64 | GPT | EFI boot | refind | stub loader | systemd | LVM2 on LUKS
Lenovo x270 | Intel(R) Core(TM) i5-7200U CPU @ 2.50GHz | Intel Wireless 8265/8275 | US keyboard w/ Euro | 512G NVMe INTEL SSDPEKKF512G7L
Offline
This question seems to crop up every few months, for some odd reason. srcpac has been mentioned already; pacbuilder was another attempt at this, allowing the user to recompile the system and run updates according to the settings in makepkg.conf. I think srcpac has a configuration file that applies package-specific sed magic to "automatically" apply customizations to PKGBUILDs, and there was some other script that could be paired with pacbuilder to allow for the same thing. I don't remember that script's name, though.
I'm not sure how much benefit adding such a feature to Pacman would have beyond making things easier on the developers; Arch was never intended to be a source-based distro, pacman is an excelent binary package manager, the and the repositories are very well-maintained to minimize the installation footprint. I'm with Trillby on this: The ability to compile from the ABS is great, but I doubt the average user customizes more than a few packages, or that recompiling everything would be of great benefit.
Offline
This question seems to crop up every few months, for some odd reason.
Because of the ABS and PKGBUILDS it is natural to expect some tool that makes automated compilation, in spite Arch isn't supposed to be source-based by default. But heck, how many binary distros have a ports/ebuild-like system? Arch is unique in that regard and I think it's being underrated; this could become the first officially supported hybrid distribution ever. Nice to hear directly from Allan and the opposed opinions, for instance, sitquietly's argument.
I coincidently found this fresh thread searching the web because I cannot make up my mind with building a GNU/Linux system from source. Given that I'm a happy Arch user I would like to try out the ABS before experimenting with Gentoo, and I also think Arch possesses certain advantages over the latter. So here I am to ask you: what method would you recommend for massively building and installing packages from its PKGBUILDS? srcpac is not even mentioned in the wiki and hasn't been version-bumped in a while, yet has an official package; whereas pacbuilder has a development entry on the AUR. Maybe raw makepkg and shell scripting, or maybe yaourt?
Edited: didn't know about Sabayon until now
Last edited by isacdaavid (2013-08-13 13:36:57)
Offline
As far as I know (which may not be all that far in that case - so correct me if I'm wrong) pacman doesn't have a currently built in mechanism to get PKGBUILDS. ..... I'm quite happy to have this package, and the abs tree - but I'm also happy that I chose to install it as a separate tool. I can't see what possible advantage merging the two packages (pacman + abs) would serve. .....
I don't see any advantage of adding a -B option to pacman. The advantage of separating the "build from source" functionality is (imo) greatly increased flexibility. As others have noticed there is not much reason to just build packages from source with custom CFLAGS. The real motivation for using the source is to control dependencies and options, and possibly to integrate mutliple upstream sources (abs, aur, ...).
For example I build everything from source using my own "build" command. It integrates abs and aur (in fact, it optionally integrates the unity, antergos, and manjaro source repos). So for example I can "build firefox-kde-opensuse" and build an aur package. Or "build unity" and build the unity meta-package. All "build" knows is that it should follow a search path in a collection of "upstream buildscripts" until it finds the desired package. A "pacman -B" option is probably not going to be that simple or allow for that integration. ("build" also needs access to a database of packages so that it knows the "pkgbase" corresponding to any "pkgname").
To further the example: the function for fetching the pkgbuild is
FROMDIR=""
function from() { # dir -- copy dir to Buildfolder and set FROMDIR
cp -r --no-clobber $1 "${Buildfolder}"
FROMDIR=$1
}
# get the Pkgbase directory into $Buildfolder and initialize the dirty Pkgbase_build
function get_pkgbuild() {
cd "${Buildfolder}"
[ -d "${Pkgbase}_build" ] && rm -rf "${Pkgbase}_build"
if [[ $REBUILD == 1 && -d $MASTER/$Pkgbase ]]; then
from "$MASTER/$Pkgbase"
cp -r "${Buildfolder}/$Pkgbase" "${Buildfolder}/${Pkgbase}_build"
return 0
fi
for repo in ${UPSTREAM[*]}; do
if [ -d $repo ]; then
cd $repo
for folder in */; do
if [[ $folder == $Pkgbase/ ]]; then
from "${repo}/${Pkgbase}"
cp -r "${Buildfolder}/$Pkgbase" "${Buildfolder}/${Pkgbase}_build"
return 0
fi
done
fi
done
return 1
}
Just adding directories to the UPSTREAM array gives access to abs/core, abs/extra, abs/community, aur, unity, ... whatever.
The help message for "build" may give an idea of where you may end up by following this kind of factoring that I'm recommending:
└─> build --help
------------------------------------------------------------------------------
A utility for Archlinux to build packages from source code.
Syntax: build options packages
Build the packages named on the command line (and packages listed in the file
using the -f option). Packages are built in the order listed. Packages named
on the command line are built after any given in a file.
Use control-c to interrupt the build then "build --resume" when ready to continue.
Options:
--rebuild Copy pkgbuilds from the master repository, ignore upstream
-c, --chroot Build packages in $MYROOT using makechrootpkg, initializing
the root with packages from the mylinux current repo (or from
Archlinux upstream if the package is not in the mylinux repo).
-f file Read package names from file. Only one file can be used.
-m, --merge Interactively merge in changes master repository.
-e, --edit Edit each PKGBUILD before building the package.
-k, --keep-going Do not stop to show namcap errors.
-L, --log Enable makepkg build logging (do not use for interactive builds).
-r, --resume Resume the previously interrupted build in this directory.
-s, --save Save the buildscript for each package (see build.conf).
-S, --src Also save the unpacked and patched source code (./src)
-h, --help Print this message.
Examples:
Compile the kernel, headers, and docs:
build -mes linux
Upgrade all packages except those in groups that are in FREEZE:
pacman -Qq | versions -qu | freeze | order | unify >upgrades
build -msLf upgrades
Best practice:
build is intentionally dumb and builds just what you ask for in the way you specify.
* always use the -s option to accumulate pkgbuilds into a local master collection
* always use the -m option to merge back into upstream any local changes
* always use the order command to get packages in correct build order
See also the help messages for:
broken freeze group need order packages pkgtree release requires
unify use versions update-package-database
------------------------------------------------------------------------------
I know that that is way too much info but maybe the details of an example can stimulate folks who think similarly.
Last edited by sitquietly (2013-08-05 23:59:46)
Offline