Python+ALPM+Me = pyalpmm a Python High-Level alpm Library now v0.3

meissna · 2008-12-11 10:48:58

Hi all,

The pyalpmm library is usable and stable for the most every-day package-manager tasks there also is a pacman clone called "mmacman" built on top of the pyalpmm lib, but I don't want to write to much, if anyone is interested in reading more:

project homepage

Release 0.3 (includes PKGBUILDs)
Changelog
Board (post ideas/feedback/flame/bugs here)

Quickstart, just wget one of those PKGBUILDs and "makepkg -i" it, to get mmacman and pyalpmm
Release PKGBUILD
svn PKGBUILD

SVN-browser
SVN: svn://infolexikon.de/pyalpmm/trunk

The latest release 0.3 has some really shiny features:

- Transparent/Seamless AUR handling, means dependency resolving and searching and anything you know from your core-repos is now also available for AUR. An absolute seamless integration of AUR!
- The new System class allows you to manipulate system-packages in the highest possible level, means sync/upgrade/remove/build packages from AUR/RegularRepos in one line of code!

This makes pyalpmm not only attractive to developers, also the regular user can benefit from the AUR integration.
There is a Demonstration/Test site, where you can see how useful mmacman can be...

Last edited by meissna (2010-08-18 16:00:21)

MindTooth · 2008-12-11 11:25:40

Written with 2.6.1 in mind?

Birger

meissna · 2008-12-11 11:39:10

of course, even with python 3.0 in mind to avoid porting hell with the python 3.0 release

ornitorrincos · 2008-12-11 19:53:53

I'm the one from the sourceforge project

but got stuck with a segfault, should resume activity(now that I got some new ideas of how to resolve it, it got personal) when I get to reinstall arch(computer started doing crazy things)

anyway, good luck

killsto · 2008-12-12 00:15:58

ornitorrincos wrote:

I'm the one from the sourceforge project
but got stuck with a segfault, should resume activity(now that I got some new ideas of how to resolve it, it got personal) when I get to reinstall arch(computer started doing crazy things)
anyway, good luck

I'm getting the same error, how did you resolve it?

andieh · 2008-12-12 08:13:24

killsto wrote:

I'm getting the same error, how did you resolve it?

where did you get a segfault? Did you try the pyalpm project from sourceforge or the pyalpmm project from meissna?

killsto · 2008-12-12 22:47:25

andieh wrote:

killsto wrote:
I'm getting the same error, how did you resolve it?
where did you get a segfault? Did you try the pyalpm project from sourceforge or the pyalpmm project from meissna?

Meissna. Followed the tutorial.

andieh · 2008-12-12 23:11:23

hmm segfault is always bad. could you please post more debug?
strace mmacman, or something like that. this would be very helpful!

killsto · 2008-12-13 05:28:52

andieh wrote:

hmm segfault is always bad. could you please post more debug?
strace mmacman, or something like that. this would be very helpful!

Must be a user error on my part somehow since mmacman works.

meissna · 2008-12-13 08:31:48

please remember, that the tutorial is written for the most recent svn version, until pyalpmm reaches 0.4/0.5 this won't change, because there is still a lot to do and I will not implement workarounds and break consistence just to keep the API frozen.

But when this point is reached, I will clearly announce it and then the releases will have their own tutorial/reference.

If someone really wants to build some app on top of pyalpmm, she/he will have to do this with the svn version as long as the API is not frozen.
My Roadmap is like this:
0.2: mainly bugfixing, configfile and any features that depend on it (you _can_ but don't have to use it - there will be a way to use your own configfile provided by your app)
0.3: aur support(!), yes with pyalpmm it will be possible to transparently use the aur (only if wanted, of course) and the official repositories (is there some way to collect packagedata (name would be enough) for the aur repo without regex-parsing http://aur.archlinux.org/packages/ ? if not, could i encourage someone responsible to provide a simple txt-file with all pkg names (or is there one already?))
0.4 hopefully after 0.3 there won't be much "regular" features missing, so until 0.4 there will be just bugfixing, "stableing" and fixing "bugs" that (from my point of view of course ) violate consistentcy.

Oh and please post bugs/requests/longer feedback on the board but I will also keep an eye on this thread

Last edited by meissna (2008-12-13 11:40:51)

Xyne · 2008-12-16 13:34:58

meissna wrote:

if not, could i encourage someone responsible to provide a simple txt-file with all pkg names

Does this generate what you wanted?

#!/usr/bin/perl
use strict;
use warnings;
my $html = `curl http://aur.archlinux.org/packages/`;
my @lines = split "\n", $html;
foreach my $line (@lines)
{
    if ($line =~ m/href="(.*?)">\1/)
    {
        my $pkg = $1;
        chop $pkg;
        print "$pkg\n";
    }
}

meissna · 2008-12-17 15:58:56

yes, this is quite exactly what I was thinking for. But the problem is not generating such a file. The index.html from http://aur.archlinux.org/ is like 2,5MB big, and I wouldn't like to download it too often. Nice would be to have the generated file in availible as http://aur.archlinux.org/pkglist.txt, which would generate just some KB traffic each time it is downloaded instead of the 2,5MB...

rson451 · 2008-12-17 16:04:57

Why do you need a list of available packages? Couldn't you just search the AUR just like all the other AUR frontends do when the user requests a package? Just incase you don't know this (you very well may) the AUR has a JSON RPC interface you can use to do searches and such. http://aur.archlinux.org/rpc.php

Last edited by rson451 (2008-12-17 16:09:19)

meissna · 2008-12-17 16:46:51

Uhu, great!

exactly this information was needed, thx!

meissna · 2008-12-20 16:35:05

Announcing pyalpmm/mmacman version 0.2

I also updated the first post, check the Changelog to see what happend.

MMacman is now a good pacman replacement, the library is very stable and fast. There still is no "WHAO!" feature, but transparent AUR-support is pending (v0.3 will have it, and the first support will be added to svn the next days), this will include an automated (re)build-process of any package on the system... Anything, of course available through the beauty of python

any feedback is appreciated, check the board

Xyne · 2008-12-25 02:09:30

Hi meissna,

When trying to build the latest version with makepkg on x86_64 architechture, the build stops at this error:

cp: cannot stat `build/lib.linux-i686-2.6/_pyalpmm_raw.so': No such file or directory

The problem can be resolved by loading a variable with the output of "uname -m" and using that in pyalpmm.conf and the Makefile.

*edit*
When trying to run mmacman, I get this error:

Traceback (most recent call last):
  File "/usr/bin/mmacman", line 112, in <module>
    session = Session(config)
  File "/usr/lib/python2.6/site-packages/pyalpmm/session.py", line 32, in __init__
    self.db_man.register(repo, SyncDatabase(repo, url))
  File "/usr/lib/python2.6/site-packages/pyalpmm/database.py", line 139, in __init__
    raise DatabaseError("Could not connect database: %s to server: %s" % (tree, url))
pyalpmm.database.DatabaseError: Could not connect database: compiz-fusion to server: http://compiz.dreamz-box.de/x86_64

Why does it even need to connect back to the site?

Last edited by Xyne (2008-12-25 02:19:57)

meissna · 2008-12-26 01:24:51

hey xyne,

thanks for the feedback,

the quick fix for your second problem is to just delete the "compiz-fusion" line from your /etc/pyalpmm.conf, those are the remains of my personal config, the repository handling will change anyway in the near future, it will use of the mirrorlists pacman uses .... (btw. compiz-fusion-repo seems to be not updated anymore)

I guess you are the first trying mmacman on x86_64 I will add your proposed changes in the next svn commit to the makefile,

Why does it even need to connect back to the site?

It does not try to connect to the compiz-site, this is just the compiz-fusion-git repo...
or am I missunderstanding what you mean?

greets

Last edited by meissna (2008-12-26 01:29:19)

invernomuto · 2009-04-16 15:47:13

Hi, I'm working with your library to rewrite my pacman gui YAPG both for GTK/Gnome and Qt/KDE4.
I only want to say thank you for the excellent work you are doing. At begin is a bit complex api use without a reference, but svn code is clean and simple to understand. Thank you.
:)

meissna · 2009-05-09 13:54:09

There are recent commits in the SVN fixing several problems with the AUR...
Right now I'm thinking about transparently handling AUR and regular repositories, as this would be the only way to have full dependency resolution on AUR packages. This would lead to another higher abstraction class (maybe System) to handle the "transparency". To avoid rumors: This project is still under active development and growing, but I can't spend that much time at the moment, because I am writing my graduation right now, so please don't hit me for beeing a bit late on responses

meissna · 2009-06-06 08:51:51

FYI,
there now is a 0.2.1 Release,
the first post was updated

u_no_hu · 2009-06-06 14:41:55

The dep in Release PKGBUILD should be pacman-mirrorlist.

meissna · 2009-06-07 09:35:44

Thanks for the feedback,

I've fixed it, and as I thought this is an evil error in a release, I made a new one: pyalpmm-0.2.1.1.tgz

first post was also updated

u_no_hu · 2009-07-01 09:00:50

Hi,
I am using this for an ncurses pacman frontend and i am having a minor issue. All searches for packages (get_packages, search_sync_package etc) seem to search aur along with pacman dbs. Is there a way to disable aur from the search ?

meissna · 2009-07-05 03:05:42

hey u_no_hu,

yes, this is a "feature", means no you cannot switch it off at some place, as AUR support is a bit tricky it integrate with the existing repository model. If you have no need for AUR, then just comment out the line

self.db_man.register("aur", AURDatabase())

it should be around line 34 in session.py (likely to be in /usr/lib/python2.6/site-packages/pyalpmm/ on your system)...
this is the hard way, if you don't wan't AUR in general. Alternatively you can just use get_package() like that:

session.db_man.get_package("myrepo/somepackage")

this will only get the package from the specified "myrepo" repository. (like "core", "extra", "community", "aur" ...)
and furthermore, even better: search_sync_package() is just a shortcut for:

session.db_man.search_package(repo=session.db_man.sync_dbs.keys(), name="foo")

this evaluates to a line like:

session.db_man.search_pacakge(repo=["core","extra","community", "aur"], name="foo")

you could just pass ["core","extra","community"] as the keyword argument for "name" but a little more non-hardcoded would be:

used_repos = [ dbname for dbname in session.db_man.sync_dbs.keys() if dbname != "aur" ]
session.db_man.search_package(repo=used_repos, name="foo")

there is a big commit pending on my box with exactly this changes included, because this is covered under "more configuration options" on my todo-list. But this commit will take some time to be ready, as i still do not have much time. But as you seem to need such a configuration option, I'll extract the needed functionality from my local version and commit it into svn, somewhere until wednesday...

meanwhile, if someone with experience with the alpm library is reading this:
Can I feed an arbitrary dependency (strings or pmdepend_t) to the library and it tells me which packages have to be installed to fulfill the dependencies?
This or any equivalent case is an important question for me, as I am writing on transparent AUR support right now, this requires that I can check for a dependency and resolve it correctly. Of course, I could do it "by hand" but before I start to reinvent the wheel I wanted to ask...

P.S: I will try to be in the #archlinux (and #archlinux.de) irc-channels on freenode more often so just look for "meissna" and you'll get pyalpmm support^^

shining · 2009-07-06 18:39:44

meissna wrote:

meanwhile, if someone with experience with the alpm library is reading this:
Can I feed an arbitrary dependency (strings or pmdepend_t) to the library and it tells me which packages have to be installed to fulfill the dependencies?
This or any equivalent case is an important question for me, as I am writing on transparent AUR support right now, this requires that I can check for a dependency and resolve it correctly. Of course, I could do it "by hand" but before I start to reinvent the wheel I wanted to ask...

You are welcome on pacman-dev ML for any questions about alpm.
The few ppl here are also likely to be interested by any existing alpm bindings.

About your question : do you really need to get the list of packages? And packages from where, from sync db or from AUR? alpm does not know about AUR.

If you really need to get that list, you have to do a transaction like in src/pacman/sync.c : trans_init + trans_prepare +
packages = alpm_trans_get_pkgs();

If you only want to check whether a dep is really installed, you can just use the simple code of pacman -T in src/pacman/deptest.c :
alpm_list_t *deps = alpm_deptest(alpm_option_get_localdb(), targets);

Arch Linux

#1 2008-12-11 10:48:58

Python+ALPM+Me = pyalpmm a Python High-Level alpm Library now v0.3

#2 2008-12-11 11:25:40

Re: Python+ALPM+Me = pyalpmm a Python High-Level alpm Library now v0.3

#3 2008-12-11 11:39:10

Re: Python+ALPM+Me = pyalpmm a Python High-Level alpm Library now v0.3

#4 2008-12-11 19:53:53

Re: Python+ALPM+Me = pyalpmm a Python High-Level alpm Library now v0.3

#5 2008-12-12 00:15:58

Re: Python+ALPM+Me = pyalpmm a Python High-Level alpm Library now v0.3

#6 2008-12-12 08:13:24

Re: Python+ALPM+Me = pyalpmm a Python High-Level alpm Library now v0.3

#7 2008-12-12 22:47:25

Re: Python+ALPM+Me = pyalpmm a Python High-Level alpm Library now v0.3

#8 2008-12-12 23:11:23

Re: Python+ALPM+Me = pyalpmm a Python High-Level alpm Library now v0.3

#9 2008-12-13 05:28:52

Re: Python+ALPM+Me = pyalpmm a Python High-Level alpm Library now v0.3

#10 2008-12-13 08:31:48

Re: Python+ALPM+Me = pyalpmm a Python High-Level alpm Library now v0.3

#11 2008-12-16 13:34:58

Re: Python+ALPM+Me = pyalpmm a Python High-Level alpm Library now v0.3

#12 2008-12-17 15:58:56

Re: Python+ALPM+Me = pyalpmm a Python High-Level alpm Library now v0.3

#13 2008-12-17 16:04:57

Re: Python+ALPM+Me = pyalpmm a Python High-Level alpm Library now v0.3

#14 2008-12-17 16:46:51

Re: Python+ALPM+Me = pyalpmm a Python High-Level alpm Library now v0.3

#15 2008-12-20 16:35:05

Re: Python+ALPM+Me = pyalpmm a Python High-Level alpm Library now v0.3

#16 2008-12-25 02:09:30

Re: Python+ALPM+Me = pyalpmm a Python High-Level alpm Library now v0.3

#17 2008-12-26 01:24:51

Re: Python+ALPM+Me = pyalpmm a Python High-Level alpm Library now v0.3

#18 2009-04-16 15:47:13

Re: Python+ALPM+Me = pyalpmm a Python High-Level alpm Library now v0.3

#19 2009-05-09 13:54:09

Re: Python+ALPM+Me = pyalpmm a Python High-Level alpm Library now v0.3

#20 2009-06-06 08:51:51

Re: Python+ALPM+Me = pyalpmm a Python High-Level alpm Library now v0.3

#21 2009-06-06 14:41:55

Re: Python+ALPM+Me = pyalpmm a Python High-Level alpm Library now v0.3

#22 2009-06-07 09:35:44

Re: Python+ALPM+Me = pyalpmm a Python High-Level alpm Library now v0.3

#23 2009-07-01 09:00:50

Re: Python+ALPM+Me = pyalpmm a Python High-Level alpm Library now v0.3

#24 2009-07-05 03:05:42

Re: Python+ALPM+Me = pyalpmm a Python High-Level alpm Library now v0.3

#25 2009-07-06 18:39:44

Re: Python+ALPM+Me = pyalpmm a Python High-Level alpm Library now v0.3

Board footer