Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

graysky · 2010-11-28 00:09:28

@karol - I thought about it but given the warnings decided against it ...is that my only option? Is there not a way to get bauerbill to build all, then install rather than:

1) build a
2) install a
3) build b
4) install b, etc.

This behavior would be helpful for users who wish to rebuild ALL installed packages from an existing system from source in an overnight fashion.

Another question: is it best to rebuild base and base-devel first, then all other packages on my system (again wanting to use the CFLAGS in /etc/makepkg.conf). Basically, I'd want to subtract out out the package names from both the\ base and base-devel packages from a pacman -Qq or...?

Last edited by graysky (2010-11-28 00:36:46)

milomouse · 2010-11-28 00:57:07

@graysky: Isn't the unsupervised "overnight" fashion blindly trusting, anyway? I think you might just want to use the ABS program and write a simple shell script for makepkg. If so, run abs and then do something like..

yourbuilddir=$HOME/abs/ ; packages=(blah blah) ; for i in $packages ; blah blah.. for d in $(find /var/abs/*/$i -type d); do ; cp -rp $d $yourbuildir ; done ; cd $yourbuilddir/$i ; makepkg -scf --noconfirm && cp -p *.pkg.tar* ../ ; done

and to install after all are done: for post in $packages*pkg.tar* ; blah blah.. if [[ -f $post ]]; then ; pacman -U --noconfirm $post ; done

Something along those lines (much more refined [and working], of course). I guess the $packages variable could be a list you manually type in or scrape from pacman for base and base-devel. Then you can check $yourbuilddir in the morning and, if you added the code to install, see your pacman.log. It should be really easy to do although a little "blind" as well. If this is a horrible idea someone please delete this. I've used similar methods before and they work fine so..

Last edited by milomouse (2010-11-28 00:59:10)

graysky · 2010-11-28 08:47:47

@milomouse - dude, that's a great idea! Let's take this discussion out of this thread to help keep the signal to noise for bauerbill up. Continued here.

Last edited by graysky (2010-11-28 09:05:32)

Xyne · 2010-11-28 20:00:20

@graysky
The "--pacman-noconfirm" option is the only way to do that. If you need to build AUR packages then you can use the "--trusted-users" option to bypass inspection prompts. I definitely don't recommend using the "--blindly-trust..." option.

Make sure that you check the output or pacman log after an upgrade if you suppress the prompts with "--pacman-noconfirm" (you should always check the output, but even more so in this case to make sure that you don't miss something critical).

As for the "-w" option, the packages still need to be installed in most cases so that other packages can be built against them. I don't even know offhand whether I added checks to avoid such installations when they're unnecessary.

graysky · 2010-11-30 00:48:59

Thanks for the info.

Last edited by graysky (2010-11-30 23:02:48)

ferda · 2010-12-09 20:17:16

I have a question about the search functionality in bauerbill. Maybe I didn't search hard enough but I couldn't find anything...

In pacman

pacman -Ss

with multiple arguments only shows results that match all of them, whereas bauerbill with -Ss shows results that contain any of the search terms.

Is there an option to get only the packages that match all search terms (especially in AUR...)?

Xyne · 2010-12-11 23:36:24

@ferda
The AUR search api doesn't support that. Try using grep to filter the results, e.g.

baurbill -Ss --aur foo bar | grep -C 2 foo | grep -C 2 bar

ferda · 2010-12-11 23:58:31

thx

jowilkin · 2010-12-12 17:36:50

Ran into what I would call a bug in bauerbill. Seems to be an issue in most pacman/libalpm wrappers. It seems they do not handle package replacement very well. More info here: https://bbs.archlinux.org/viewtopic.php?id=109766

Last edited by jowilkin (2010-12-12 17:39:31)

Xyne · 2010-12-12 19:37:23

Can you post the output starting from "switching to pacman"? Pacman handles all installation and I want to see exactly what command pacman is getting to cause that error. It might be that bauerbill and others need to check for replacements first, but it seems that pacman should handle this gracefully itself as it's just a regular command.

jowilkin · 2010-12-12 23:04:14

Xyne wrote:

Can you post the output starting from "switching to pacman"? Pacman handles all installation and I want to see exactly what command pacman is getting to cause that error. It might be that bauerbill and others need to check for replacements first, but it seems that pacman should handle this gracefully itself as it's just a regular command.

Unfortunately I already used pacman to fix the error, I should have saved the output. I just ran pacman -Syu and it prompted me if I wanted to replace libusb1 with libusb, I said yes and everything went through fine. I would guess that the way you are calling pacman is somehow skipping past the step where it checks if packages need to be replaced.

I'm not sure under what conditions pacman will check for replacements. It seems you specify the packages to install in your call to pacman, perhaps that prevents it from checking for replacements?

Edit: I went through my pacman.log and found the place where it happened:

[2010-12-07 20:45] Running '/usr/bin/pacman --config /etc/pacman.conf -S --cachedir /var/cache/pacman/pkg gstreamer0.10 gstreamer0.10-base poppler libusb polkit gstreamer0.10-good upower libftdi lirc-utils libmtp cdrdao xfsprogs usbutils cairomm polkit-gnome obex-data-server usbmuxd libcups poppler-data poppler-glib gnupg gmime udev gstreamer0.10-base-plugins hal xorg-xhost xterm openobex telepathy-idle libdc1394 freetype2 gnupg2 gstreamer0.10-good-plugins'
[2010-12-07 20:48] Running '/usr/bin/pacman --config /etc/pacman.conf -S --cachedir /var/cache/pacman/pkg gstreamer0.10 gstreamer0.10-base poppler polkit gstreamer0.10-good libftdi lirc-utils libmtp cdrdao xfsprogs usbutils cairomm polkit-gnome obex-data-server libcups poppler-data poppler-glib gnupg gmime udev gstreamer0.10-base-plugins hal xorg-xhost xterm openobex telepathy-idle libdc1394 freetype2 gnupg2 gstreamer0.10-good-plugins'
[2010-12-07 20:49] Running '/usr/bin/pacman --config /etc/pacman.conf -S --cachedir /var/cache/pacman/pkg gstreamer0.10 gstreamer0.10-base poppler polkit gstreamer0.10-good libftdi lirc-utils libmtp cdrdao xfsprogs usbutils cairomm polkit-gnome obex-data-server libcups poppler-data poppler-glib gnupg gmime udev gstreamer0.10-base-plugins hal xorg-xhost xterm openobex telepathy-idle libdc1394 freetype2 gnupg2 gstreamer0.10-good-plugins'
[2010-12-07 20:49] Running 'pacman -Syu'
[2010-12-07 20:49] synchronizing package lists
[2010-12-07 20:49] starting full system upgrade
[2010-12-07 20:50] removed libusb1 (1.0.8-1)
[2010-12-07 20:50] upgraded cairomm (1.8.4-1 -> 1.8.6-1)
[2010-12-07 20:50] upgraded cdrdao (1.2.3-4 -> 1.2.3-5)
[2010-12-07 20:50] upgraded freetype2 (2.4.3-1 -> 2.4.4-1)
[2010-12-07 20:50] upgraded gmime (2.4.19-1 -> 2.4.21-1)
[2010-12-07 20:50] upgraded libusb (0.1.12-4 -> 1.0.8-1)
[2010-12-07 20:50] installed libusb-compat (0.1.3-1)
[2010-12-07 20:50] upgraded gnupg (1.4.11-1 -> 1.4.11-2)

I believe the first few lines are the failed calls initiated by bauerbill and then the 4th line is when I just ran pacman directly. You can see that it first removed libusb1 and then later libusb was upgraded.

Last edited by jowilkin (2010-12-12 23:11:29)

pho · 2010-12-13 13:54:25

Is there a way to determine the new md5 automatically?
Cause when a AUR-build is outdated(mostly the lib32pkgs), because there is allready a new Version, I have to edit the AUR-pkg and after downloading the latest version I dont have the new md5sum and bauerbill wants to Abort.

It were nice if after a wrong md5sum bauerbill gives the current md5sum of the downloaded pkg and then give us the chance to update the PKGBUILD again.

mhertz · 2010-12-14 18:35:40

I've just discovered bauerbill(and powerpill!) and it is indeed awesome; can't believe it took me so long! Thanks alot Xyne!

Just one quick question; you have "bypass prompts" options for both pacman installations and pkgbuilds so as to automate the operation, but I couldn't find one for disabling the "proceed with download?"...

I hate such prompts when upgrading my system, as I just want to hit my alias and then the job starts/finishes + I have scripts to setup new installs/image-restores which I also like completelly unattended...

Could you kindly consider the above feature request, thanks ?

Last edited by mhertz (2010-12-14 18:40:02)

karol · 2010-12-14 18:40:44

@ mhertz

Have you tried

       --pacman-noconfirm
              [Powerpill option]

              Pass the "--noconfirm" flag to Pacman to bypass installation confirmation. This is potentially danger‐
              ous  and  not  recommended unless you really know what you're doing. If you use this option, make sure
              that you at least review the messages afterwards.

Last edited by karol (2010-12-14 18:41:03)

mhertz · 2010-12-14 18:56:53

Hi karol!

Thanks for your reply, mate! Yeah, I actually always use that parameter in my aliases/scripts and it's great for disabling pacman's "Proceed with installation?" prompt, but still i'm left with the previous bauerbill supplied prompt of "Proceed with download?". Btw, it obviously only comes when the packages to download aren't allready cached locally.

Actually I just came to think that I of course just can set my aliases/scripts to have a prepended 'echo y | sudo bauerbill.....' which would fix my issue(dooh, i'm a GNU/Linux noob still unfortunetly ), but still I think that it would be nicer with a real parameter for e.g. others thinking in the same manner.

CU, Martin.

Last edited by mhertz (2010-12-14 18:59:44)

karol · 2010-12-14 19:00:05

'echo y | ...' is a popular trick :-)

You can suggest that feature in the pacman bugtracker.

mhertz · 2010-12-15 09:26:00

Thanks, mate!

Btw, pacman's --noconfirm parameter overrides everything fine, but the prompt I am bypassing with the piping 'y' to it, is from bauerbill itself.

Xyne · 2010-12-16 05:02:10

@jowilkin
I would argue that pacman should be able to detect the replacement when explicitly installing packages. I don't think I can work around that in bauerbill because it needs to specify the packages. I don't remember why but I'm sure that I had to make it do that at some point.

It's not that big of a deal considering how rarely it occurs.

@pho
The point of the md5sum is to ensure download integrity. It would be pointless if bauerbill just updated the md5sum to match whatever it downloaded. Even if there is a new version of a package, the md5sum should not change for the old version unless either the package or upstream is doing something wrong.

Of course, if you're really sure that you want to ignore the md5sum, you can use bauerbill's "--makepkg-args" option to pass "--skipinteg" to makepkg.

@mhertz
Bauerbill recognizes pacman's "--noconfirm" option. You can use it to bypass bauerbill's prompts.

jowilkin · 2010-12-16 21:39:08

Xyne wrote:

@jowilkin
I would argue that pacman should be able to detect the replacement when explicitly installing packages. I don't think I can work around that in bauerbill because it needs to specify the packages. I don't remember why but I'm sure that I had to make it do that at some point.
It's not that big of a deal considering how rarely it occurs.

Well it seems the same problem just happened to me again, so not so rare. pycairo was replaced with python2-cairo. My first run of bauerbill -Syu failed to detect this. Running the same command again immediately after the first call it was detected. So this time there was no error due to file conflicts, but I needed to run bauerbill twice to get a full upgrade. I could change my alias to run bauerbill twice every time I want to upgrade, but this seems far from ideal (and will not deal with the case when a file conflict occurs).

I would guess the problem is that on the first call there was no 'u' flag supplied to pacman. So it seems the problem is that replacements are not checked for unless the u flag is supplied. As a result, the user will not be prompted to install replacements unless they run bauerbill -Syu when no other packages have upgrades available. Shouldn't it be possible to explicitly install packages with pacman -Su instead of just pacman -S? It seems like that would likely fix it, although I haven't tested it.

Last edited by jowilkin (2010-12-16 21:40:38)

Xyne · 2010-12-16 22:53:53

Amusing (imo) anecdote:
I decided to implement the search for replacements in powerpill/bauerbill. After I wrote the function to find replacements, I got a "function redefined" error. I had actually written the function before (verbatim). I then found that I had commented out the call to the function when checking for upgrades. I quizzically uncommented that section, ran bauerbill, and quickly understood why I had removed the code... it was agonizingly slow.

So, I have refactored the code to optimize it and it is now reasonably fast.

I tested the code by downgrading to pycairo then running "-Su":

--> Searching for upgradable repo packages...
--> resolving python2... installed
--> resolving cairo... installed
--> Reflecting... retrieved 50 mirrors.

:: The following packages will be downloaded:
   [extra]
     python2-cairo-1.8.10-1

:: Number of Packages:                       1
:: Total Download Size:               0.02 MiB
:: Binary Package Install Size:       0.16 MiB

:: Proceed with download? [Y/n]

Upgrade to the latest version of perl-xyne-arch from the repo on my site. I will push the update to [community] in a few days.

mhertz · 2010-12-17 14:16:45

Xyne wrote:

@mhertz
Bauerbill recognizes pacman's "--noconfirm" option. You can use it to bypass bauerbill's prompts.

Thanks alot for your reply and solution.

I would suggest that you add that '--noconfirm' also is a valid bauerbill command in the bauerbill man. Nomally, seeing as bauerbill is a pacman wrapper, then I would also have tried to use the '--noconfirm' option, but as the bauerbill man specifically mentions a '--pacman-noconfirm'(and no '--noconfirm'), then I thought that --pacman-noconfirm was the only "noconfirm" option available in bauerbill.

Thanks again!

Edit: Dohh, sorry about the above, as I now by again rereading the man see specifically mentioned:

OPTIONS
Bauerbill supports the following superset of Powerpill's arguments in
addition to Pacman's arguments:

Thanks, and sorry for being so thick

Last edited by mhertz (2010-12-17 14:25:00)

karol · 2010-12-20 19:45:47

[karol@black test]$ sudo bauerbill -Ssq -- -mt | wc -l
5850
[karol@black test]$ sudo pacman -Ssq -- -mt | wc -l
3
[karol@black test]$ sudo bauerbill -Ssq mt$ | wc -l
3
[karol@black test]$ sudo bauerbill -Ssq -mt$ | wc -l
0
[karol@black test]$ sudo pacman -Ssq mt$ | wc -l
3
[karol@black test]$ sudo pacman -Ssq -mt$ | wc -l
pacman: invalid option -- '$'
0
[karol@black test]$ sudo pacman -Ssq -- -mt$ | wc -l
1

Me needs help with this one.

Edit: Some more discrepancies, this time because of stripping the quotes.

[karol@black test]$ sudo bauerbill -Ssq '(-mt$|abc)'
sh: -c: line 0: syntax error near unexpected token `('
sh: -c: line 0: `/usr/bin/pacman --config /etc/pacman.conf -S -s -q (-mt$|abc) 2>/dev/null'
[karol@black test]$ /usr/bin/pacman --config /etc/pacman.conf -S -s -q (-mt$|abc)
bash: syntax error near unexpected token `('
[karol@black test]$ /usr/bin/pacman --config /etc/pacman.conf -S -s -q '(-mt$|abc)'
abcde
ypbind-mt

Last edited by karol (2010-12-20 20:01:31)

Xyne · 2010-12-20 20:43:42

It would help if you explained what you're expecting.

Are you trying to do this?

 bauerbill --aur -Ssq mt | grep -- '-mt$'

karol · 2010-12-20 20:54:24

Xyne wrote:

It would help if you explained what you're expecting.
Are you trying to do this?
 bauerbill --aur -Ssq mt | grep -- '-mt$'

Initially I was searching for multithreaded packages. This

bauerbill -Ssq --aur mt | grep -E '(-mt$|-mt-)'

works, but I found out that bauerbill needs grep where pacman does not - and it has nothing to do with searching AUR.
For some reason I expected bauerbill to be drop-in replacement. It's not a problem, but I was curious if I'm missing something or is it just the way it is.

Xyne · 2010-12-21 23:56:27

I've updated perl-xyne-common and perl-xyne-arch to fix an error in the way that "--" was handled. This fixes several of the cases that you posted and it should make bauerbill a drop-in replacement for pacman searches for most cases.

The other discrepancies are indeed due to the way it searches the AUR. The AUR does not understand regular expressions so searching for e.g. "-mt$" will not return anything because no package contains "$" in its name. You have to pass the results through grep to find the ones that end in "-mt".

The other problem is that the AUR only accepts one search term at a time. Because of this, I thought it made sense to return all the results from multiple search terms rather than implement internal filtering. The consequence is that pacman search terms are combined into alternations, e.g. "foo bar" becomes "(foo|bar)". It's not a perfect solution but it gives the user more control by enabling further filtering through grep.

I may rework the way it searches though now that I'm thinking about it. In the meantime, please update to the latest version of perl-xyne-* from my site and let me know if it works more as expected.

Also note that you do not need to use "sudo" for searches.

Arch Linux

#901 2010-11-28 00:09:28

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#902 2010-11-28 00:57:07

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#903 2010-11-28 08:47:47

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#904 2010-11-28 20:00:20

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#905 2010-11-30 00:48:59

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#906 2010-12-09 20:17:16

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#907 2010-12-11 23:36:24

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#908 2010-12-11 23:58:31

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#909 2010-12-12 17:36:50

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#910 2010-12-12 19:37:23

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#911 2010-12-12 23:04:14

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#912 2010-12-13 13:54:25

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#913 2010-12-14 18:35:40

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#914 2010-12-14 18:40:44

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#915 2010-12-14 18:56:53

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#916 2010-12-14 19:00:05

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#917 2010-12-15 09:26:00

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#918 2010-12-16 05:02:10

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#919 2010-12-16 21:39:08

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#920 2010-12-16 22:53:53

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#921 2010-12-17 14:16:45

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#922 2010-12-20 19:45:47

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#923 2010-12-20 20:43:42

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#924 2010-12-20 20:54:24

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#925 2010-12-21 23:56:27

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

Board footer