Package Manager to compile all packages as in Gentoo

gopichand · 2011-08-26 20:50:56

Hello,

I love Arch and was looking to get my hands dirty, I there a way where I can compile all the packages that I install like in Gentoo rather than using only the pre-compiled i686 packages? I looked at ABS but was not sure how to keep track of the changes, I believe I simply cant do '-Syu' with ABS. I am just looking for something like Yaourt or Packer AUR helpers but for all the packages from core/extra/community.

Thank You
Gopi

karol · 2011-08-26 20:55:58

https://wiki.archlinux.org/index.php/Pacbuilder ?

gopichand · 2011-08-27 16:29:53

Pacbuilder!

Can it perform the same as [pacman -Syu] ?
I need something like emerge of Gentoo for Arch linux.

karol · 2011-08-27 16:38:01

https://bbs.archlinux.org/viewtopic.php … 35#p939535
https://bbs.archlinux.org/viewtopic.php … 82#p973882

You can use ABS or vABS. https://aur.archlinux.org/packages.php?ID=51856

There's also customizepkg.

Last edited by karol (2011-08-27 16:42:00)

RichAustin · 2011-08-27 17:39:18

I'm perhaps being a bit dumb here but why not just use Gentoo instead? I am actually quite serious here since I wouldn't have thought there was a major difference between the two except you compile everything in Gentoo. I can't really see why you would want to find a workaround to do what you can already do in Gentoo.

megadriver · 2011-08-28 15:05:17

I see an important difference between Arch and Gentoo: PKGBUILDs are much, much, much easier to create/modify/update than ebuilds (at least for me!).

A source-based distro combining the Arch Build System and Gentoo's USE flags would be a real dream come true!

RichAustin · 2011-08-28 18:59:15

megadriver wrote:

I see an important difference between Arch and Gentoo: PKGBUILDs are much, much, much easier to create/modify/update than ebuilds (at least for me!).
A source-based distro combining the Arch Build System and Gentoo's USE flags would be a real dream come true!

That's an interesting point. PKGBUILDS, as far as I understand them, are essentially a script which compiles the application. Does Gentoo expect you to build everything manually? That seems pretty arduous to me, whereas I have found Arch is actually quite enjoyable when building a package. I'm a programmer though so I suppose I would say that!

sitquietly · 2011-08-28 19:04:55

RichAustin wrote:

I'm perhaps being a bit dumb here but why not just use Gentoo instead? I am actually quite serious here since I wouldn't have thought there was a major difference between the two except you compile everything in Gentoo. I can't really see why you would want to find a workaround to do what you can already do in Gentoo.

I recently spent many days relearning the Gentoo way, did a stage3 Gentoo install alongside my Arch system, compiled a custom kernel from pf-sources, updated world, spent time reading the forum and web pages. I won't try to replay "what I found wrong in Gentoo" except for the refrain "Gentoo is broken." That was my take. I found important things broken for long periods with no attention. People rude to each other. I won't do that scene...

Arch may have a smaller developer team but I've experienced a generally high level of cooperation and discipline here. I've seen complaints about the forum here being moderated -- to me that seems to work for our good. Thanks to the moderators who keep this place civil and rational!

There seems to be a good knowledge base here and the principle of always keeping up-to-date with the upstream releases forces the developers to constantly parse upstream information to roll a coherent stream of Arch packages. The knowledge here is as good as anywhere. I just don't expect Arch Linux to be an integrated operating system, like Debian stable or Ubuntu. In my experience it gets almost as much stability as Debian Testing by virtue of quickly taking in the upstream releases, which then fix the bugs that Arch never has to fix ourselves. We trust upstream I didn't see a clear policy in Gentoo -- some upstream releases are held back forever as Gentoo devs argue over it.

I have found it possible to use Arch as the base for a customized, locally compiled system; it's possible to do that with abs/AUR and makepkg. I've developed my own style. In place of USE flags I use meld to carry forward my local changes from my local PKGBUILD to the newly released Arch PKGBUILD. I don't modify many pkgbuilds and mostly just remember which ones I change, for example to exclude gconf from my system. (It's a KDE / Xmonad desktop with a lot of audio tools).

I haven't found it possible or desirable to do bulk builds on Arch. I encounter quite a few pkgbuilds from abs that do not build straight away, sometimes because the package won't build against the library that is in abs at the same time, sometimes because patches are missing or checksums are incorrect: it becomes clear that the released binary packages are not built from the daily abs update. That would be the job of a build server that takes in abs and spits out a complete set of binaries. That is the missing step that I do myself: take in abs and compile a (daily) coherent set of binaries from it (with local mods). It is not like Gentoo. I don't think that the Arch team tries to guarantee that the their binaries can be re-built perfectly from abs, so if you try to use abs rather than pacman -S you need to understand that and accept the responsibility. It works well enough for me.

falconindy · 2011-08-28 19:41:34

megadriver wrote:

I see an important difference between Arch and Gentoo: PKGBUILDs are much, much, much easier to create/modify/update than ebuilds (at least for me!).
A source-based distro combining the Arch Build System and Gentoo's USE flags would be a real dream come true!

At some point you'll realize that it's because Arch doesn't get into the shitty business of USE flags that we're able to provide such an easy and usable build system.

sitquietly wrote:

I don't think that the Arch team tries to guarantee that the their binaries can be re-built perfectly from abs, so if you try to use abs rather than pacman -S you need to understand that and accept the responsibility. It works well enough for me.

What appears in the ABS tree is exactly what is used to build the packages released into the repos. If there's a problem with building a package as is in ABS, then it's bug report worthy.

karol · 2011-08-28 19:50:25

ABS is updated once a day, so there may a problem here and there https://bugs.archlinux.org/task/24537

sitquietly · 2011-08-29 01:04:52

falconindy wrote:

At some point you'll realize that it's because Arch doesn't get into the shitty business of USE flags that we're able to provide such an easy and usable build system.
sitquietly wrote:
I don't think that the Arch team tries to guarantee that the their binaries can be re-built perfectly from abs, so if you try to use abs rather than pacman -S you need to understand that and accept the responsibility. It works well enough for me.
What appears in the ABS tree is exactly what is used to build the packages released into the repos. If there's a problem with building a package as is in ABS, then it's bug report worthy.

Thanks for saying that. I've been submitting an occasional bug report about failures to build from abs and I'm glad to hear that such reports are appreciated. I don't mean to irritate anyone and I know that very few of us (or am I wrong?) compile packages from source using abs. Having repositories of well-engineered binaries AND the Arch Build System is a killer combination for me.

Anyone interested in using abs extensively might consider trying my process as a set of cron jobs:
0. do an incremental backup up of the local system at 8:00pm every night --
for me that is /etc and pkgsrc/{AUR, abs, packages, sources}
1. sync the pacman database at 10:00pm every night (pacman -Sy)
2. sync to abs at midnight (this timing works for me; every upgrade known to pacman is in the local abs tree)
3. generate the todo-list every morning;
echo "n" | pacman -Su >rebuild
4. during the day work through the "rebuild" list and compile each package using your copy of abs

gopichand · 2011-08-29 04:34:12

sitquietly and others! Thank You for your replies.
I was wondering? if AUR helpers can download the PKGBUILDs from AUR and compile them, why cant we do it with the PKGBUILDS of the core/extra?
I mean can we actually do that by creating a package manager which can handle the PKGBUILDS of core, extra, community, AUR etc??
for example, Packer is a AUR helper which can download the PKGBUILDS from AUR and compile the sources and creates package. Cant we change it to do the same with the core or extra repos? Will we face a problem because the PKGBUILDS in core and extra change too often and will break the system?

I tried Gentoo, it took soo much time and I thought Its not worth it after it broke. I like Arch's philosophy and the way it works.

ngoonee · 2011-08-29 04:38:58

gopichand wrote:

sitquietly and others! Thank You for your replies.
I was wondering? if AUR helpers can download the PKGBUILDs from AUR and compile them, why cant we do it with the PKGBUILDS of the core/extra?
I mean can we actually do that by creating a package manager which can handle the PKGBUILDS of core, extra, community, AUR etc??
for example, Packer is a AUR helper which can download the PKGBUILDS from AUR and compile the sources and creates package. Cant we change it to do the same with the core or extra repos? Will we face a problem because the PKGBUILDS in core and extra change too often and will break the system?
I tried Gentoo, it took soo much time and I thought Its not worth it after it broke. I like Arch's philosophy and the way it works.

You know, actually taking a look at the webpages/man pages of the AUR helpers in question is a good idea... your question indicates you haven't done that. Some helpers are intentionally AUR-only, some work identically for the repos and the AUR.

karol · 2011-08-29 04:42:58

ngoonee wrote:

gopichand wrote:
sitquietly and others! Thank You for your replies.
I was wondering? if AUR helpers can download the PKGBUILDs from AUR and compile them, why cant we do it with the PKGBUILDS of the core/extra?
I mean can we actually do that by creating a package manager which can handle the PKGBUILDS of core, extra, community, AUR etc??
for example, Packer is a AUR helper which can download the PKGBUILDS from AUR and compile the sources and creates package. Cant we change it to do the same with the core or extra repos? Will we face a problem because the PKGBUILDS in core and extra change too often and will break the system?
I tried Gentoo, it took soo much time and I thought Its not worth it after it broke. I like Arch's philosophy and the way it works.
You know, actually taking a look at the webpages/man pages of the AUR helpers in question is a good idea... your question indicates you haven't done that. Some helpers are intentionally AUR-only, some work identically for the repos and the AUR.

I think he meant he wants the official PKGBUILDs so he can rebuild the official packages - most AUR helpers don't do that.

gopichand · 2011-08-29 04:44:57

I am sorry, I was looking exactly at the AUR helpers and found that Packer actually do support core/extra. I just need to find the way compiling every time there is an update.
Thank You.

gopichand · 2011-08-29 04:47:40

The wiki says, Packer has support for core/extra. I went to the Packer page and found there is no parameter to specify that we want to compile packages from core and extra aswell along with AUR. I only got the usual packer -Syu etc...

Here is the link:
https://github.com/bruenig/packer/wiki

Thank You

ethail · 2011-08-29 07:13:23

"support for core and extra" in packer means that can work with the repos in the way the repos work. How is that way? binary packages.

Why does the AUR usually build packages from source? Because there is no "Arch Developer" that has built it before.

Pacbuilder is the best choice here, I think. I can also suggest that you instead of using the ABS make your own "ABS on demand" using the svntogit. :

https://projects.archlinux.org/svntogit/

You clone them once, then everytime you want to build something because of updates, you pull.

Also, there could be another idea, interfacing directly with the svn and using some kind of script that checks only for the packages that you have installed in order to not overload the SVN:

(little quick writing get-the-job-done script that does the SVN job after you've created the folders the way is said here: https://wiki.archlinux.org/index.php?ti … S_From_SVN )

BIG DISCLAIMER: It's not clean or well written in terms of someone who has experience with bash scripts, furthermore, this was written yesterday before going to sleep, what adds a bonus of warning to the reader.

http://pastie.org/2447319

EDIT: Checking from the SVN only the packages that you has less disk usage for you, but takes more time to update if I'm not mistaken. Git is faaaaaaast.

Last edited by ethail (2011-08-29 07:45:30)

tomk · 2011-08-29 09:30:24

gopichand wrote:

I love Arch and was looking to get my hands dirty

Just my €0.02, but there are far more productive ways to get dirty with Arch. You will spend a lot of time setting up your source-based package management deal, and when it's done, you will gain little if any benefit from it.

Some alternative ideas here for all that spare time.

gopichand · 2011-08-29 13:26:41

tomk and ethail, Thank You for the replies.

May be you are right tomk, I am looking for the thing thats not needed. Actually, I wanted to build my system with Hex/Quad core and wanted to compile all the packages so that I can get an optimized performance. I have read a suggestion in the Arch forums that Gentoo provides very less gain in speed and its not worth it when looking at the times to compile. But I thought since Arch's makepkg is really nice and easy to perform, I thought I would try to do that.

I would be very happy to contribute to the Arch projects, I need a mentor, if someone can adopt me I will be able to spend a lot of time on the Arch projects.

Thank You

karol · 2011-08-29 13:49:45

Doesn't really matter if you recompile your system using Gentoo or Arch, speed benefits will usually be minimal.

ethail · 2011-08-29 14:37:23

I don't know how many times I think this have been written, but Gentoo "I will build them all" motto is not for speed, is for customization. You build because of the USE flags, if that reports you some kind of speed because of the GCC flags is one thing, but the main goal is less bloat on software because you throw code that supports things you're not going to use.

Gentoo is customization.

EDIT: the post got sent at half writting >,<

Obviously, you can achieve the same in Arch, using PKGBUILDS and changing what gets configured before building the package, but as said, the goal of that is customization and not speed.

Some example of that: I build my own kernel without lots of drivers that are related to hardware I do not use or have and don't intend to have on this laptop. What do I achieve? A better fitting kernel for this laptop, but that barely means a noticeable increase on speed, even with flags march=native.

There are some packages where the speed could get noted. Firefox build with PGO is one of them, but because of the PGO fitting the code for the hardware, not the compiler doing the machine code aimed to that architecture.

Last edited by ethail (2011-08-29 14:49:55)

karol · 2011-08-29 14:40:25

ethail wrote:

I don't know how many times I think this have been written, but Gentoo "I will build them all" motto is not for speed, is for customization. You build because of the USE flags, if that reports you some kind of speed because of the GCC flags is one thing, but the main goal is less bloat on software because you throw code that supports things you're not going to use.
Gentoo is customization.

And Arch doesn't use USE flags so you won't get the kind of customization (or at least, there's no framework available), right?

karol · 2011-09-08 15:21:09

Has anyone tried using makeworld from abs to rebuild the whole system?

ANOKNUSA · 2011-09-08 19:36:43

tomk wrote:

<snip> Some alternative ideas here for all that spare time.

Arch Wiki wrote:

In biology, mutualism is an interaction between two organisms from which each derives benefit. This principle can also apply to proactive and motivated members of the Arch community wanting to get involved and contribute to their favorite GNU/Linux distribution. This participation benefits not only the community member and their fellow Archers, but potentially all fans of free and open source software.

An idea after my own heart...

@karol: It's my understanding that the only customization one would get would be architecture optimization from the cflags in makepkg.conf, which might get you a couple percentage points on some benchmarks. I've thought about looking into makeworld (and pacbuilder, at one point) but found the tedium of editing potentially dozens of PKGBUILDs for little benefit off-putting.

karol · 2011-09-08 19:40:19

I think you can customize the PKGBUILDs to add / remove features

--include-X
--disable-Y

but it would indeed involve quite a bit editing.

Arch Linux

#1 2011-08-26 20:50:56

Package Manager to compile all packages as in Gentoo

#2 2011-08-26 20:55:58

Re: Package Manager to compile all packages as in Gentoo

#3 2011-08-27 16:29:53

Re: Package Manager to compile all packages as in Gentoo

#4 2011-08-27 16:38:01

Re: Package Manager to compile all packages as in Gentoo

#5 2011-08-27 17:39:18

Re: Package Manager to compile all packages as in Gentoo

#6 2011-08-28 15:05:17

Re: Package Manager to compile all packages as in Gentoo

#7 2011-08-28 18:59:15

Re: Package Manager to compile all packages as in Gentoo

#8 2011-08-28 19:04:55

Re: Package Manager to compile all packages as in Gentoo

#9 2011-08-28 19:41:34

Re: Package Manager to compile all packages as in Gentoo

#10 2011-08-28 19:50:25

Re: Package Manager to compile all packages as in Gentoo

#11 2011-08-29 01:04:52

Re: Package Manager to compile all packages as in Gentoo

#12 2011-08-29 04:34:12

Re: Package Manager to compile all packages as in Gentoo

#13 2011-08-29 04:38:58

Re: Package Manager to compile all packages as in Gentoo

#14 2011-08-29 04:42:58

Re: Package Manager to compile all packages as in Gentoo

#15 2011-08-29 04:44:57

Re: Package Manager to compile all packages as in Gentoo

#16 2011-08-29 04:47:40

Re: Package Manager to compile all packages as in Gentoo

#17 2011-08-29 07:13:23

Re: Package Manager to compile all packages as in Gentoo

#18 2011-08-29 09:30:24

Re: Package Manager to compile all packages as in Gentoo

#19 2011-08-29 13:26:41

Re: Package Manager to compile all packages as in Gentoo

#20 2011-08-29 13:49:45

Re: Package Manager to compile all packages as in Gentoo

#21 2011-08-29 14:37:23

Re: Package Manager to compile all packages as in Gentoo

#22 2011-08-29 14:40:25

Re: Package Manager to compile all packages as in Gentoo

#23 2011-09-08 15:21:09

Re: Package Manager to compile all packages as in Gentoo

#24 2011-09-08 19:36:43

Re: Package Manager to compile all packages as in Gentoo

#25 2011-09-08 19:40:19

Re: Package Manager to compile all packages as in Gentoo

Board footer