paCPAN: tool for building Perl packages for Pacman from CPAN

Xyne · 2009-01-07 10:59:54

http://xyne.archlinux.ca/info/pacpan
*edit*

Pacpan functionality has evolved as of version 4.0. CPAN distribution and metadata handling has improved and there are now functions which can query installed packages to determine their adherence to the packaging guidelines.

Building packages with dependency resolution etc has been move to bauerbill, which can now build CPAN packages using standard pacman options, e.g. "bauerbill -S perl-foo-bar" or "bauerbill -S Foo::Bar".

Last edited by Xyne (2010-02-01 22:00:32)

Allan · 2009-01-07 11:03:08

I hate to be the bearer of bad news....

pacman -Si perl-cpanplus-pacman

Xyne · 2009-01-07 11:09:21

LOL

*sigh*

*walks away*

*edit*

*walks back in*

Actually, I just tried building the same thing with both cpan4pacman and pacpan... the former took much longer then failed... I'm naturally biased, but I think I'm sticking with pacpan.

Last edited by Xyne (2009-01-07 11:24:33)

Allan · 2009-01-07 11:18:34

Maybe you and Firmicus should get together and create one super script...

peets · 2009-01-07 12:37:55

I think this is a great idea. Will try in the coming week.

skottish · 2009-01-08 04:23:33

Allan wrote:

Maybe you and Firmicus should get together and create one super script...

+100. Yes. Please.

No matter what they say about you Allan, you're easily in the top ten greatest current forum mods.

Dusty · 2009-01-08 14:22:41

Allan's also one of the top devs.

Not quite on topic: Xyne, do you still get DNS errors with archlinux.ca? I'm trying to decide if I need to harass somebody or if it went away by itself.

Dusty

u_no_hu · 2009-01-08 14:33:21

I got errors while trying to access xyne's archlinux page today

Dusty · 2009-01-08 14:53:00

u_no_hu wrote:

I got errors while trying to access xyne's archlinux page today

Hmm, can you also not access archlinux.ca (nothing to see here IS the correct response ) or schwag.archlinux.ca?

Also, if anyone with DNS problems on archlinux.ca can supply me with ping and/or traceroute results it might help.

Dusty

Edit: I hate it when I hit quote instead of edit.

Last edited by Dusty (2009-01-08 14:56:10)

Xyne · 2009-01-08 15:18:44

Yeah, I still get periodic errors. It's probably time to harass someone. I'll try to remember to email you a traceroute the next time it comes up.

For what it's worth, at some point I discovered that some of the opendns servers worked while others (opendns) didn't. I thought that was strange.

*edit*
Before this gets completely threadjacked, has anyone actually tried pacpan? I honestly think that it's quite elegant and even if there is another package which supposedly does the same thing (not for me though), I would really like some feedback (I was all proud of myself before Allan's revelation brought my enthusiasm to crashing halt).

As for collaborating with the Firmicus, I have nothing against the idea, but I'm not sure how that would work practically, especially with something written in Perl
Also, aside from making a few things configurable and adding some documentation (which is minimal anyway, as this is straight-forward), this is essentially finished (I say that now, but I thought that with powerpill v2.0, so meh).

Last edited by Xyne (2009-01-08 15:23:57)

Dusty · 2009-01-08 15:49:52

Xyne wrote:

Yeah, I still get periodic errors. It's probably time to harass someone. I'll try to remember to email you a traceroute the next time it comes up.
For what it's worth, at some point I discovered that some of the opendns servers worked while others (opendns) didn't. I thought that was strange.

Do you happen to know "which" opendns server(s) is/are broken? I need something I can duplicate before I can actually accomplish anything.

For what it's worth, I've had the DNS break on both my ISP DNS (Rogers /choke) and on opendns, but both are working today. Bleh.

Before this gets completely threadjacked

Sorry 'bout dat. ;-) Your tools are getting to be quite popular, I figure I should get your site fixed. :-D

Dusty

Edit: I did it again. WTF?

Last edited by Dusty (2009-01-08 15:50:40)

Xyne · 2009-01-08 19:09:30

Hmmm, my reply was lost with the server reboot.

I was mistaken about using opendns server. I was using vnsc-pri.sys.gtei.net servers from this page: http://theos.in/windows-xp/free-fast-pu … rver-list/
Some of them worked, some of them didn't. but I don't remember which nor if they were consistent.

Btw, if there are server stats online, email me a link so that I can get an idea of usage. I'd like to get an idea of aur vs repo usage, etc.

Xyne · 2009-01-29 13:42:13

I'm bumping this, partly to get it back on topic and partly to let everyone know that I've cleaned up the code a bit and improved the way it discovers pkgs.

I'd still really like to get some feedback on this because I think it works very well.

*edit*
Also, it works quite well with makerepo, which was the main incentive to go though the code again.

Last edited by Xyne (2009-01-29 13:43:38)

Xiong Chiamiov · 2009-02-12 22:07:32

Xyne wrote:

I'd still really like to get some feedback on this because I think it works very well.

You don't know how glad I was to find this. I've been struggling with some CPAN modules that I really shouldn't be, and I've gotten the whole thing into quite a mess. I was starting to update an orphaned module package, and realizing that I'd need to create/update quite a few more, and getting a little discouraged. So, I'll try this out this weekend, when I'll actually have time (hopefully) for messing around with this kind of stuff.

As a side note, do you know if there's an easy way to remove all CPAN modules? I've done a lot of mucking around, and I'd rather like to just start fresh.

Oh, and another question: from what I understand, if a package for the appropriate module isn't available in the repos (I assume this includes AUR?), it will generate a pkgbuild for it and install it. Is this meant to be a replacement for all of the perl-foo-bar packages in the AUR, or as a tool to generate pkgbuilds easily so that you can add new perl-foo-bar packages with little effort? Also, is there any sort of update functionality, or was the 2nd part of my last question correct and it's up to the package maintainer to keep things up-to-date?

Xyne · 2009-02-12 22:48:36

I haven't installed anything from CPAN without using pacman so I don't know for sure how to remove them (the main reason that I wrote this was because I was reluctant to install directly).
This is what I might try to remove all the manually installed modules, but I can't guarantee that this would work or that it's safe to do:
1) use pacpal's --list-unpkgd /path/to/perldirs option to find which files are not in a package and remove them
2) re-install all of my perl packages with "pacman -S $(pacman -Qqs ^perl)"

The main reason for writing this was to just generate all of the necessary module pkgs automatically. It checks the repos known to pacman but it doesn't check the AUR. I don't think I will add AUR support either because it's dangerous and somewhat redundant because a perl-foobar pkg in the AUR should be functionally identical to perl-foobar generated by this. With the AUR, you would still need to download and build the sources anyway, so there's nothing to be gained from that, plus I trust pacpan far more than random users on the AUR .

It's not really mean to replace the perl-foo packages in the AUR, but it doesn't guarantee a certain degree of uniformity and dependency handling (dep handing only fails when the CPAN module maintainer hasn't listed all of the deps, but then it's as easy as "pacpan Missing::Dep").

There is no update function yet although I will probably add one*. "makerepo" (on my site) could be used to accomplish this in the meantime by creating a REPOBUILD which lists the modules that should be kept up-to-date.

*I'll try to get to that soon, but I'm easily sidetracked and have a few things I want to get to right now.

Daenyth · 2009-03-06 02:19:59

I second the suggestion that you and the other author collaborate... Looking forward to the result. Hard to keep track of you Xyne, seems like you bring up a new program every month...

vociferous · 2009-03-19 12:42:37

Imagine going looking for a perl cpan pacman solution and finding out you wrote one ofc i had to give it a spin

Works really well but i did run into trouble when installing Spreadsheet::WriteExcel something went wrong with one of the dependencies.

REQUIREMENTS

This module requires Perl 5.005 (or later), Parse::RecDescent
and File::Temp:

Works fine as long as i removed, 'perl-' from depends=

[vociferous@avatar ~]$ pacpan Spreadsheet::WriteExcel
==> retrieving info for Spreadsheet::WriteExcel
Use of uninitialized value $module in hash element at /usr/bin/pacpan line 961, <GEN1> line 298.
Use of uninitialized value $p_dep in string eq at /usr/bin/pacpan line 789, <GEN1> line 298.
Use of uninitialized value $p_dep in hash element at /usr/bin/pacpan line 790, <GEN1> line 298.
==> FOUND: found pkg: File::Temp -> perl
==> retrieving info for OLE::Storage_Lite
==> FOUND: found pkg: Parse::RecDescent -> perl-parse-recdescent
Use of uninitialized value in concatenation (.) or string at /usr/bin/pacpan line 546, <GEN1> line 1098.
Use of uninitialized value $pkgname in concatenation (.) or string at /usr/bin/pacpan line 559, <GEN1> line 1098.
Use of uninitialized value $pkgver in concatenation (.) or string at /usr/bin/pacpan line 559, <GEN1> line 1098.
Use of uninitialized value $url in concatenation (.) or string at /usr/bin/pacpan line 559, <GEN1> line 1098.
Use of uninitialized value $name in concatenation (.) or string at /usr/bin/pacpan line 325, <GEN1> line 1098.
Use of uninitialized value $ver in concatenation (.) or string at /usr/bin/pacpan line 325, <GEN1> line 1098.
Use of uninitialized value $name in concatenation (.) or string at /usr/bin/pacpan line 340, <GEN1> line 1098.
Use of uninitialized value $name in concatenation (.) or string at /usr/bin/pacpan line 340, <GEN1> line 1098.
Use of uninitialized value $name in concatenation (.) or string at /usr/bin/pacpan line 340, <GEN1> line 1098.
Use of uninitialized value $name in concatenation (.) or string at /usr/bin/pacpan line 340, <GEN1> line 1098.
==> SUCCESS: generated build script: build_cpan

 pacpan Spreadsheet::WriteExcel --get-pkgbuild
Use of uninitialized value $module in hash element at /usr/bin/pacpan line 961, <GEN1> line 298.
Use of uninitialized value $p_dep in string eq at /usr/bin/pacpan line 789, <GEN1> line 298.
Use of uninitialized value $p_dep in hash element at /usr/bin/pacpan line 790, <GEN1> line 298.
# PKGBUILD generated by pacpan
pkgname=perl-spreadsheet-writeexcel
_realname=Spreadsheet-WriteExcel
pkgver=2.25
pkgrel=1
pkgdesc="no description available"
arch=(i686 x86_64)
license=('unknown, check site')
url="http://search.cpan.org/~jmcnamara/Spreadsheet-WriteExcel-2.25/lib/Spreadsheet/WriteExcel.pm"
depends=('perl' 'perl-parse-recdescent' 'perl-' 'perl-ole-storage_lite')

options=(!emptydirs)
#provides=('Spreadsheet-WriteExcel')
provides=('spreadsheet-writeexcel')
source=(http://search.cpan.org/CPAN/authors/id/J/JM/JMCNAMARA/Spreadsheet-WriteExcel-2.25.tar.gz)
md5sums=('ad70e5b031c0af60c50a72d8b09240c9')
build() {
  cd ${srcdir}/${_realname}-${pkgver}
  # install module in vendor directories.
  PERL_MM_USE_DEFAULT=1 perl Makefile.PL INSTALLDIRS=vendor || return 1
  make  || return 1
  make install DESTDIR=${pkgdir} || return 1

  # remove perllocal.pod and .packlist
  find ${pkgdir} -name perllocal.pod -delete
  find ${pkgdir} -name .packlist -delete
}
# END OF PACPAN PKGBUILD

Xyne · 2009-03-19 16:15:23

Hey Voci,

I've found the problem and I think I've fixed it (a blank line was getting through a "split"). I'm able to build Spreadsheet::WriteExcel without any issues now at least, so give it another try.

Anyway, I would have told you about this before but you were sick of me mentioning my repo etc.
I'm glad you've found this anyway and I hope you'll get some good use out of it.

Last edited by Xyne (2009-03-19 16:16:03)

vociferous · 2009-03-20 14:56:58

Confirmed, it now builds without problems. I'll start building packages of all the modules i need over the weekend and let you know if there are any other problems

ido50 · 2009-03-24 19:18:38

Some feedback on installation with pacpan:

pacpan would fail when trying to install a module such as Catalyst::Action::REST, and one of its dependencies URI::Find, for several reasons:
1. URI::Find does not use the traditional Makefile.PM build process, instead providing a Build.pm file.
2. Even if it would have supported this proccess, it would still fail complaining Module::Install and Test::More (which are provided by the perl arch package) are old, so one has to rebuild them with --prefer-cpan --skip-deps (which did work!)
3. Installing URI::Find with makepkg (After creating an appropriate PKGBUILD file) _as root_, since it fails as normal user.
4. Proceeding to install Catalyst::Action::REST with pacpan will fail with the same mysterious error I have faced a few times before:

pacman -U perl-catalyst-action-rest-0.66-1-i686.pkg.tar.gz
loading package data...
checking dependencies...
error: failed to prepare transaction (could not satisfy dependencies)
:: perl-catalyst-action-rest: requires

5. Installing Catalyst::Action::REST with makepkg normally will work.

Thanks!

Xiong Chiamiov · 2009-03-24 19:35:16

I have to apologize, Xyne, because installing Catalyst was the reason I was trying out pacpan earlier. I had lots of issues with different things, and ended up installing things through a combination of pacpan, cpan, and manual building, since I had problems with each on different modules. I didn't keep track of what went wrong, though, because I was really tired and just wanted to get it done, else I could have provided some of the information ido50 did, but earlier.

ido50 · 2009-03-24 20:23:20

Xiong Chiamiov wrote:

I have to apologize, Xyne, because installing Catalyst was the reason I was trying out pacpan earlier. I had lots of issues with different things, and ended up installing things through a combination of pacpan, cpan, and manual building, since I had problems with each on different modules. I didn't keep track of what went wrong, though, because I was really tired and just wanted to get it done, else I could have provided some of the information ido50 did, but earlier.

I would like to add, due to your reply, that even after installing Catalyst::Devel, which is the minimum set of modules needed to run Catalyst, I am still in the process of installing yet more and more Catalyst modules, and like Xiong, it's through a combination of pacpan, makepkg, etc. At least I'm not installing anything manually and create arch packages for every module, but considering all the file conflicts and new vs. old modules, I'm not sure how clean my installations are.

Obviously cpan is a complex and intertwined structure, but I'm sure there's some way to make this all easier. If they managed to do it in Debian, there has to be some way.

Xyne · 2009-03-24 20:48:49

Thanks for the feedback, both of you.

@ido50
I"ll look at the specific modules mentioned later (I'm so tired that I can barely see straight right now ), but here are my thoughts right now:
*nvm, looked at them anyway*
1. I'll look into detecting alternative build methods... I really wish that CPAN was more homogeneous though.
2. At least it works, even if it's not as smooth as I'd like.
3. I don't understand what you mean by installing it with makepkg (I'm admittedly tired and halfway expect to facepalm at my own stupidity tomorrow when I read this, but afaik all installations are handled by pacman and require root privileges).
4. Is that the full output? What I would suggest anyway is that when you encounter an error that requires a manual intervention in building the package, once you've installed rerun pacpan (i.e. not the build script) to see if it can resolve more deps using the currently installed modules (sorry if that's what you've done... I wasn't sure).
5. That makes me think that running pacpan again would have fixed it.

As for Catalyst... I had no problems with that today, but I did update some meta-data handling so that almost definitely affects it.

As a general note to all the readers looking at this and possibly thinking "what a buggy piece of shit that seems to be"... I don't expect pacpan to ever be capable of building absolutely anything and everything on CPAN until they formalize it, which they won't because rigor is anti-perl I know there are still some things to do, but I think I'm at the "handling exceptions" stage. It seems to be able to build nearly all simple modules and only tends to die on modules with large dependency trees. Even when it dies though, it's often possible to manually work around the problematic module and use it to build the rest. If you compare the PKGBUILDs generated by pacpan to the ones in the AUR and in Community I think you'll notice that the PKGBUILDs generated by pacpan are much more comprehensive, especially for the provides data which is quite extensive.

I'll try to make the build script more resilient and interactive, with better indicators of what might fix issues as they arise. Really though, for most modules it should just work.

p.s. I ramble/waffle when I'm tired... disregard all inanity in this post.

*edit*

ido50 wrote:

Obviously cpan is a complex and intertwined structure, but I'm sure there's some way to make this all easier. If they managed to do it in Debian, there has to be some way.

Have you looked into how they do it? Is it just a case of all the pre-built packages being available in their repos?

Last edited by Xyne (2009-03-24 20:51:50)

ido50 · 2009-04-02 11:11:40

As I wrote before pacpan seems to fail a lot with the error message: ":: some-module: requires"

I think I know what's causing this. I've been browsing through the PKGBUILD files pacpan is creating, and I've noticed that those modules that fail to build with pacpan have an empty string in the depends variable in PKGBUILD. For example, for Catalyst::Action::REST:

depends=('perl>=5.8.9' 'perl-yaml-syck' '' 'perl-data-serializer' 'perl-http-headers' 'perl-params-validate' 'perl-catalyst-runtime' 'perl-class-inspector')

Notice the empty string '' between 'perl-yaml-syck' and 'perl-data-serializer'. Don't know what's causing this, but obviously this could be fixed easily in pacpan.

Last edited by ido50 (2009-04-02 14:15:39)

Xyne · 2009-04-02 23:19:08

Thanks, ido50.
That should be easy to fix. I'll see what I can do as soon as I'm in front of my computer again.

Does removing that from the PKGBUILD fix the problem?

Arch Linux

#1 2009-01-07 10:59:54

paCPAN: tool for building Perl packages for Pacman from CPAN

#2 2009-01-07 11:03:08

Re: paCPAN: tool for building Perl packages for Pacman from CPAN

#3 2009-01-07 11:09:21

Re: paCPAN: tool for building Perl packages for Pacman from CPAN

#4 2009-01-07 11:18:34

Re: paCPAN: tool for building Perl packages for Pacman from CPAN

#5 2009-01-07 12:37:55

Re: paCPAN: tool for building Perl packages for Pacman from CPAN

#6 2009-01-08 04:23:33

Re: paCPAN: tool for building Perl packages for Pacman from CPAN

#7 2009-01-08 14:22:41

Re: paCPAN: tool for building Perl packages for Pacman from CPAN

#8 2009-01-08 14:33:21

Re: paCPAN: tool for building Perl packages for Pacman from CPAN

#9 2009-01-08 14:53:00

Re: paCPAN: tool for building Perl packages for Pacman from CPAN

#10 2009-01-08 15:18:44

Re: paCPAN: tool for building Perl packages for Pacman from CPAN

#11 2009-01-08 15:49:52

Re: paCPAN: tool for building Perl packages for Pacman from CPAN

#12 2009-01-08 19:09:30

Re: paCPAN: tool for building Perl packages for Pacman from CPAN

#13 2009-01-29 13:42:13

Re: paCPAN: tool for building Perl packages for Pacman from CPAN

#14 2009-02-12 22:07:32

Re: paCPAN: tool for building Perl packages for Pacman from CPAN

#15 2009-02-12 22:48:36

Re: paCPAN: tool for building Perl packages for Pacman from CPAN

#16 2009-03-06 02:19:59

Re: paCPAN: tool for building Perl packages for Pacman from CPAN

#17 2009-03-19 12:42:37

Re: paCPAN: tool for building Perl packages for Pacman from CPAN

#18 2009-03-19 16:15:23

Re: paCPAN: tool for building Perl packages for Pacman from CPAN

#19 2009-03-20 14:56:58

Re: paCPAN: tool for building Perl packages for Pacman from CPAN

#20 2009-03-24 19:18:38

Re: paCPAN: tool for building Perl packages for Pacman from CPAN

#21 2009-03-24 19:35:16

Re: paCPAN: tool for building Perl packages for Pacman from CPAN

#22 2009-03-24 20:23:20

Re: paCPAN: tool for building Perl packages for Pacman from CPAN

#23 2009-03-24 20:48:49

Re: paCPAN: tool for building Perl packages for Pacman from CPAN

#24 2009-04-02 11:11:40

Re: paCPAN: tool for building Perl packages for Pacman from CPAN

#25 2009-04-02 23:19:08

Re: paCPAN: tool for building Perl packages for Pacman from CPAN

Board footer