Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

ngoonee · 2010-04-23 02:09:25

Nope, no error messages. Here's the direct copy-paste (missing colours of course):-

$ rm -R ~/temp/testbauerbill/* ; bauerbill -b ~/temp/testbauerbill -Sy ; ls ~/temp/testbauerbill
--> Rebasing...

+ xyne-any/aurploader 2010.03.17.1-1
+ xyne-any/bash-xyne-common_functions 1.3-1
+ xyne-any/bauerbill 2010.03.14.1-1
+ xyne-any/cronwhip 2.0.1-3
+ xyne-any/facadefs 0.2.3-1
+ xyne-any/makerepo 2010.04.16.1-1
+ xyne-any/metapax 3.0.1-3
+ xyne-any/mimeman 2010.03.03.1-1
+ xyne-any/mimeo 2010.03.03.2-1
+ xyne-any/obfilebrowser 2010.04.17.1-1
+ xyne-any/obtheme 2010.03.01.1-1
+ xyne-any/pacnew_scripts 2010.03.21.1-2
+ xyne-any/paconky 2.4-1
+ xyne-any/pacpal 3.1-1
+ xyne-any/pacpan 2010.04.11.1-1
+ xyne-any/pbget 2010.02.12.2-1
+ xyne-any/perl-xyne-arch 2010.04.16.1-1
+ xyne-any/perl-xyne-common 2010.04.01.2-3
+ xyne-any/pkgd 2010.03.05.1-1
+ xyne-any/powerpill 17.1-1
+ xyne-any/pychrom 2010.04.19.2-1
+ xyne-any/pystopwatch 2010.04.15.1-1
+ xyne-any/quickserve 2010.02.15.1-1
+ xyne-any/quixand 3.0-9
+ xyne-any/rebase 4.0-1
+ xyne-any/reflector 5.0-2
+ xyne-any/reposearch 2.0-10
+ xyne-any/svg2ggr 1.0-3
+ xyne-any/svn-export 3.0-12
+ xyne-any/tiv 1.1-1
+ xyne-any/unibrow 0.3-1
+ xyne-any/unisel 0.1-3
+ xyne-any/voracious 2010.02.15.1-1
--> updated [xyne-any] +33 ^0 -0
sync

I thought I'd add a specific server to /etc/pacman.conf (instead of including the mirrorlist) to check whether that's the problem, but no joy. Anything else you'd like me to check?

MindTooth · 2010-04-23 07:05:58

Does Bauerbill support an ignore option with a command, rather then a hard edit in the config file?

Xyne · 2010-04-24 00:53:20

@ngoonee
Does the order of the commands matter? (bauerbill -> rebase, rebase -> bauerbill)
Does "bauerbill -Syy -b /path/to/empty/dir" work?

@MindTooth
It supports (or should support) of all Pacman's options, so yes. Use "--ignore" or "--ignoregroup".

ngoonee · 2010-04-26 01:40:46

@Xyne, -Syy works, -Sy does not. And am not sure what is meant by order of commands, of course if I run rebase first bauerbill uses the updated database.

Xyne · 2010-04-26 05:33:30

I had forgotten that rebase ignores the database timestamp by default. That explains why "bauerbill -Sy" didn't work but both "rebase" and "bauerbill -Syy" did.

Try a different mirror. If it's correctly updating [xyne-any] then it's almost certainly the server's fault. Either it's ignoring the HTTP HEAD request or it's reporting the wrong time (or it's just out of sync).

MindTooth · 2010-04-27 10:36:05

I tried this command:
sudo bauerbill -Syu --rebase --aur --devel --ignore cairo-compmgr-git

But still it tries to download that package.

Xyne · 2010-04-28 00:07:54

The latest version of perl-xyne-arch (>=2010.04.27.1) should filter ignored packages when using "--devel".

MindTooth · 2010-04-28 13:49:34

Thank you

jdarnold · 2010-04-28 15:26:03

Not sure if it is a bauerbill or the way the package is set up, but there's a problem with the 'brscan3' package in the AUR when run on Arch64. One of the Sources listed is an i386 package which I guess bauerbill automagically downloads. But then the PKGBUILD downloads the x64 version of it, and runs ar *.deb, which expands to be 'ar -x ...x64.deb ...i386.deb' and thus you get a mysterious 'no entry ../brscan3-0.2.9-1.i386.deb in archive' error. If I delete the i386 file and run mkpkg by hand, all is well.

Should I ask the package maintainer to either remove the i386 from the Sources or perhaps not do an 'ar *' in the build()?

Xyne · 2010-04-28 20:55:05

@jdarnold
The problem is caused by the conditional statements in the PKGBUILD (i.e. "if [ $(uname -m) = "x86_64" ]; then..."). The parser simply can't handle them. To avoid the unnecessary download, just remove them from the PKGBUILD.

As for the build function, I would say that it's a bit sloppy to glob for the deb file. I would either add another conditional statement to the build function, or do the following:

if [ $(uname -m) = "x86_64" ]; then
  source=("http://pub.brother.com/pub/com/bsc/linux/dlf/${pkgname}-${pkgver}-${pkgrel}.amd64.deb" brscan3.rules)
  md5sums=('6e2d43295ca01804cae13b6b62eeaf36' '76537a0eb5b68c48b57b4409397a4fa5')
  deb="${pkgname}-${pkgver}-${pkgrel}.amd64.deb"
else
  source=("http://pub.brother.com/pub/com/bsc/linux/dlf/${pkgname}-${pkgver}-${pkgrel}.i386.deb" brscan3.rules)
  md5sums=('7660b3f3e8d5e128f04ed1a79946d4a9' '76537a0eb5b68c48b57b4409397a4fa5')
  deb="${pkgname}-${pkgver}-${pkgrel}.i386.deb"
fi

build() {
  ...
  ar -x "../$deb" || return 1
  ...
}

There are actually quite a few things that should be changed in that PKGBUILD (e.g. removal of "$startdir" from the build function).

gav616 · 2010-04-29 10:54:08

Hi Xyne, I've requested this feature before, yu said maybe in the future ( with time issues and all..)

but I've just noticed 'makepkg' has a simple cleanup option that I think could be added to bauerbill (in config or/and at cmdline) without too much hassle (fatal final words).
example from makepkg 'cleanup'

##
# Clean up function. Called automatically when the script exits.
##
clean_up() {
    local EXIT_CODE=$?

    if [ "$INFAKEROOT" -eq 1 ]; then
        # Don't clean up when leaving fakeroot, we're not done yet.
        return
    fi

    if [ $EXIT_CODE -eq 0 -a "$CLEANUP" -eq 1 ]; then
        # If it's a clean exit and -c/--clean has been passed...
        msg "$(gettext "Cleaning up...")"
        rm -rf "$pkgdir" "$srcdir"
        if [ -n "$pkgname" ]; then
            # Can't do this unless the BUILDSCRIPT has been sourced.
            rm -f "${pkgbase}-${pkgver}-${pkgrel}-${CARCH}-build.log"*
            if [ "$PKGFUNC" -eq 1 ]; then
                rm -f "${pkgbase}-${pkgver}-${pkgrel}-${CARCH}-package.log"*
            elif [ "$SPLITPKG" -eq 1 ]; then
                for pkg in ${pkgname[@]}; do
                    rm -f "${pkg}-${pkgver}-${pkgrel}-${CARCH}-package.log"*
                done
            fi
        fi
    fi

    remove_deps
}


##

basically, if I was building a stable AUR package releases from one month to the next I would use the cleanup option to clear the src/pkg dirs,
but obviously if I was building/debugging my favourite program that I use git/ccache with, I would what not to clean the directory's.

So that said, I think a clean by default would work well, and clean off on a 'need to' basis (cmdline).

p.s.

I've uninstalled 'rebase' in favour of bauerbill's internal rebase in a cron.hourly.

Is...

/usr/bin/bauerbill -Sy --rebase --arch i686 --aur --pacman-bin /usr/bin/pacman --no-color >/dev/null 2>&1

...the same as?

/usr/bin/rebase --arch i686 --by-lastupdate --no-color

Last edited by gav616 (2010-04-29 10:55:58)

Xyne · 2010-04-30 02:38:01

@gav616
Why don't you just create an alias for that? For example:

alias bbc="bauerbill --makepkg-args '--clean'"

Otherwise you could easily write a script to run bauerbill then remove the build directory afterwards, e.g.:

#!/bin/bash
bauerbill "$@" && rm -fr /tmp/bauerbill

I suppose I could add an option to remove the build directory after the operation.

As for the second question, yes those are effectively the same, but you have some unnecessary options. You can remove the following options:
*) "--aur": has no effect on database updates
*) "--arch i686": it is automatically detected (the option is only for special cases such as building in chroots or downloading packages for a different architecture)
*) "--pacman-bin": unnecessary unless you specify a different binary in the configuration file
*) "--no-color": the output is redirected to /dev/null, so it doesn't matter

ngoonee · 2010-05-14 08:18:13

Hi Xyne,

I want a single-line update to do the following:-

1. Check for updates to every package including those from the AUR.
2. Update every package from the repos except for a few that I have my own custom patches for.
3. Build from ABS the packages I have my own custom patches for and update. Build directory (with sources) to be saved in a location of my choice.
4. Update from AUR. Build directory (with sources) should also be saved in the same place.
5. All packages (downloaded or built, repo or AUR) to be saved to the same cache.
6. Reinstalling already-built packages (whether with my custom patches or from AUR) can be done based on cached .pkg.tar.xz without having to manually -U

If I'm not wrong, this can be achieved by the following options is /etc/bauerbill.conf
- Autobuild and Autopatch listing the packages in 3.
- CacheBuilt (for 5.)
- PatchDir (what sort of heirarchy is expected, simply the package name?)

I don't know how to achieve 6 though. Any advise? Do I have to run a local repo listing those packages? I've got 40+ AUR/custom packages now, so am looking to really streamline my updates.

Runiq · 2010-05-14 08:26:27

6 is already done automatically IIRC, when makepkg starts and finds an already built package in the cache directory, it installs that package right away. The rest can be done in bauerbill.conf or makepkg.conf (package cache).

ngoonee · 2010-05-14 09:58:25

Any problems with pointing the package cache for makepkg/bauerbill to /var/cache/pacman/pkg?

It seems only PKGBUILD patches work... I patch mpd's "mpd" rc.d start script, is there anyway to have that patch applied as well? Or do I need to add a section to the PKGBUILD patch to sed in my changes?

Last edited by ngoonee (2010-05-14 10:52:17)

Xyne · 2010-05-14 16:04:19

Runiq is right. If a package has already been built then bauerbill should install it without rebuilding it. It should be able to detect packages in the build directory or the PKGDEST directory if the latter has been set in makepkg.conf.

ngoonee wrote:

Any problems with pointing the package cache for makepkg/bauerbill to /var/cache/pacman/pkg?

The default cache is /var/cache/pacman/pkg. You should be able to set PKGDEST in makepkg.conf to /var/cache/pacman/pkg without any problems too.

ngoonee wrote:

It seems only PKGBUILD patches work... I patch mpd's "mpd" rc.d start script, is there anyway to have that patch applied as well? Or do I need to add a section to the PKGBUILD patch to sed in my changes?

Currently only PKGBUILD and install script patches are supported although adding general patch support is still on the todo list. I hadn't even considered using the PKGBUILD to apply other patches but it should work. You can keep the additional patches in the PatchDir without affecting anything else.

ngoonee · 2010-05-15 01:10:15

Xyne wrote:

Runiq is right. If a package has already been built then bauerbill should install it without rebuilding it. It should be able to detect packages in the build directory or the PKGDEST directory if the latter has been set in makepkg.conf.
ngoonee wrote:
Any problems with pointing the package cache for makepkg/bauerbill to /var/cache/pacman/pkg?
The default cache is /var/cache/pacman/pkg. You should be able to set PKGDEST in makepkg.conf to /var/cache/pacman/pkg without any problems too.

ngoonee wrote:
It seems only PKGBUILD patches work... I patch mpd's "mpd" rc.d start script, is there anyway to have that patch applied as well? Or do I need to add a section to the PKGBUILD patch to sed in my changes?
Currently only PKGBUILD and install script patches are supported although adding general patch support is still on the todo list. I hadn't even considered using the PKGBUILD to apply other patches but it should work. You can keep the additional patches in the PatchDir without affecting anything else.

Thanks Xyne, have that pretty much set-up here. What's the 'correct' way to create the patches? I'm used to 'diff -Naur' but then Autopatch doesn't seem to work.

Side-note: Is pbget abandoned? Any suggestions what else I should use?

Xyne · 2010-05-15 02:43:50

The PKGBUILD and install file patches should be created at the prompt (back up -> edit -> diff). Other patches should be created and saved wherever you want. How you apply them in the PKGBUILD is also up to you.

ngoonee · 2010-05-15 04:59:25

Ah, thanks. Seems to work. Anyway to force the package to build even though its already in the cache (rebuilding the package)? My workaround would be to specify a different PKGDEST, but that's hacky.

Xyne · 2010-05-15 16:03:08

Just use makepkg's "-f" option, e.g.

--makepkg-args "-f"

If you only want to force certain packages then (re)move them first, e.g. "rm /var/cache/pacman/pkg/{foo,bar,baz}-*-pkg.* && bauerbill ...".

ngoonee · 2010-05-15 16:34:17

Ah, was afraid it'd be something of that sort. If I already have makepkg arguments in bauerbill.conf will --makepkg-args overwrite?

Xyne · 2010-05-15 19:21:50

ngoonee wrote:

If I already have makepkg arguments in bauerbill.conf will --makepkg-args overwrite?

Yeah, command-line options take precedence. Just create an alias or write a script that contains the arguments that you want. You could even use something like

--makepkg-args "$(grep MakepkgArgs /etc/bauerbill.conf | cut -d= -f2)"

if you don't want to update the alias every time you update arguments in the configuration file.

smradlev · 2010-05-20 22:22:45

I think there is a problem with the file validation checks when installing from aur. Trying to install tp_smapi I get the error below. I get a similar error when trying to install bin32-google-earth. I tried installing with yaourt, and the file checks pass and the installation finishes successfully.

==> Validating source files with md5sums...
tp_smapi-0.40.tgz ... FAILED
tp_smapi.patch ... Passed
==> ERROR: One or more files did not pass the validity check!
ERROR: makepkg exited with an error (256)
WARNING: expected package does not exist: /tmp/bauerbill/build/aur/tp_smapi/tp_smapi-0.40-2-x86_64.pkg.tar.xz
--> scanning /tmp/bauerbill/build/aur/tp_smapi for matching packages...
ERROR: no package found: /tmp/bauerbill/build/aur/tp_smapi/tp_smapi-0.40-2-x86_64.pkg.tar.xz

Last edited by smradlev (2010-05-20 22:23:13)

ngoonee · 2010-05-20 23:20:22

Most likely is an aria2 problem actually, I've been getting md5sum fails from aria2 downloads which are obtained properly on wget. Haven't checked to see which mirrors exactly are/aren't affected though.

And smradlev, the file validation checks aren't even done by bauerbill. The fact that they fail doesn't mean file validation is done wrongly, but that the downloaded file is corrupted.

Xyne · 2010-05-21 15:26:27

If makepkg is exiting with an error then it shouldn't be a bauerbill issue. The PKGBUILD parsers limitation have no effect on makepkg so makepkg is always able to grab the correct sources if bauerbill missed them.

What happens when you run "makepkg" yourself in the build directory?

Arch Linux

#676 2010-04-23 02:09:25

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#677 2010-04-23 07:05:58

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#678 2010-04-24 00:53:20

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#679 2010-04-26 01:40:46

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#680 2010-04-26 05:33:30

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#681 2010-04-27 10:36:05

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#682 2010-04-28 00:07:54

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#683 2010-04-28 13:49:34

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#684 2010-04-28 15:26:03

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#685 2010-04-28 20:55:05

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#686 2010-04-29 10:54:08

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#687 2010-04-30 02:38:01

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#688 2010-05-14 08:18:13

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#689 2010-05-14 08:26:27

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#690 2010-05-14 09:58:25

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#691 2010-05-14 16:04:19

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#692 2010-05-15 01:10:15

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#693 2010-05-15 02:43:50

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#694 2010-05-15 04:59:25

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#695 2010-05-15 16:03:08

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#696 2010-05-15 16:34:17

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#697 2010-05-15 19:21:50

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#698 2010-05-20 22:22:45

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#699 2010-05-20 23:20:22

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#700 2010-05-21 15:26:27

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

Board footer