python-aur: Python3 modules and scripts for automating AUR interaction

Xyne · 2011-06-27 12:33:32

Project page: http://xyne.archlinux.ca/projects/python3-aur/

Update

This package now contains some utility scripts in addition to the Python 3 modules:

aurploader - upload package to the AUR from the command line, with support for comments, votes and notifications
aurquery - retrieve information about AUR packages
aurtomatic - post comments, vote, unvote, notify, unnotify, flag, unflag, etc.

Original Post

I usually don't announce modules that I release because I generally write them for myself and only release them to support my own applications. I think this one has the potential to be generally useful though, especially when combined with Reflector and Rémy's pyalpm.

The module uses threading to speed up retrievals and caches results in an SQLite3 database.

The source archive contains a directory named "scripts" with some examples. There's also a paconky replacement script.

As always, questions, suggestions and other feedback are welcome.

Last edited by Xyne (2013-03-01 18:35:38)

lolilolicon · 2011-06-27 14:04:10

I suspect you didn't include uploading support because the lack of AUR RPC API support? I wonder when AUR will support this...
There're also the other user actions the RPC isn't currently aware of: vote, flag, notify, adopt, disown, delete, unvote, unflag, unnotify, and comments.

What's your plan regarding these features?

It would be nicer if you implemented a full AUR interface, so both AUR users and package maintainers could benefit greatly.

Daenyth · 2011-06-27 18:03:35

It's a bit of a tangent to this, but can you also post the VCS information so people can easily clone/improve your stuff? It's a little annoying to not have anything except static tarball downloads.

Xyne · 2011-06-28 02:21:20

@lolilolicon
I may implement non-RPC features but I'm reluctant to scrape HTML pages for data. It will introduce long-term volatility and it will increase load on the server. It would be better if the AUR provided a general API for user actions but that is unlikely in the near future.

Still, I see the utility of supporting other user actions so I may end up implementing them.

@Daenyth
Moving all of my projects to Git and updating my release scripts is on my todo list. I hope to get to it soon.

Last edited by Xyne (2011-06-28 02:22:02)

Xyne · 2012-11-05 10:00:28

The latest release includes some important changes which are described here.

See the project and AUR pages for more details.

Xyne · 2013-03-01 18:35:00

The package now contains a new tool: aurtomatic
You can use it to vote, flag, comment and notify from the command line.

Vote for all of your AUR packages with

aurtomatic -a vote -i

ava1ar · 2014-08-13 00:02:05

Looks like python3-aur can't upload splitted packages to AUR, can anybody confirm?

Xyne · 2014-08-13 00:37:08

I haven't tried. Can you give me an example for testing?

ava1ar · 2014-08-13 00:39:27

Check chromium-plugins

Xyne · 2014-08-13 04:32:54

The upload works but it fails to confirm the upload because the RPC interface does not return information when querying a pkgbase. I'll bring this up on the aur-dev list and post an update here later.

<edit>If anyone wants to follow the discussion on the list, here it is.</edit>

Incidentally, the use of $startdir is deprecated and should be removed from the PKGBUILD.

It is also unnecessary (but not wrong) to include entries in the "conflicts" array for derivative packages (e.g. *-dev, *-git). It is the responsibility of the derivative packages to manage "conflicts" and "provides".

Last edited by Xyne (2014-08-13 04:58:03)

Xyne · 2014-12-25 03:56:24

The package now includes AUR.PkgList and aurpkglist to access the full list of AUR packages. See the website for details.

ava1ar · 2014-12-25 05:03:39

Thanks, will check.

Xyne · 2015-07-11 19:16:47

Aurploader users, please see my comments concerning the move to AUR 4.0 here.

ngoonee · 2015-09-29 03:58:42

Hi Xyne, just a poke on a rebuild due to python update to 3.5 (since I use your repo).

Xyne · 2015-10-06 19:27:31

See here.

eschwartz · 2015-12-07 05:01:11

It seems flagging a package out of date via aurtomatic is broken (for a while) -- ever since AUR 4.0 which requires you to (and lets you!) submit a comment when flagging something out-of-date.

Also, AUR.RPC now tries to access the brand-new XCGF without importing it.

Xyne wrote:

@Daenyth
Moving all of my projects to Git and updating my release scripts is on my todo list. I hope to get to it soon.

Ping?

...

Random miscellaneous request: can this automatically check, say, $XDG_CONFIG_HOME/AUR.conf for the user/pass file, rather than requiring me to add an extra option to every invocation of aurtomatic.

Last edited by eschwartz (2015-12-07 05:06:14)

Xyne · 2015-12-07 08:19:55

I have added the missing import statements and fixed package flagging.

Uploading the Git repos is still on my "I should probably do that but meh, not a priority" list. I have a few concerns about uploading them.

I suppose that I could add a central check for a login file which would allow any tool based on this login automatically, but that would also be a security risk. It's a bad idea to have plaintext login credentials in general, and even worse to have them in a predictable location, especially if they would be used by default by any malicious script.

Can't you just use an alias or wrapper script?

alias aurto=/usr/bin/aurtomatic -l /path/to/login/credentials'

Yannick_LM · 2015-12-07 19:42:54

You could also use python package keyring to securely store the password

Xyne · 2015-12-07 21:31:30

Yannick_LM wrote:

You could also use python package keyring to securely store the password

That may avoid the issue of storing the password in plaintext, but it would still be easily accessible via a script.

eschwartz · 2015-12-13 06:56:25

Well, there is such a thing as too many aliases, so I thought I'd mention the idea of a default conf file.

No one has to use it, but I don't really see why it is the job of this tool to tell me what I should do. It's not like this would be the only tool out there that empowers the user to make a dangerous choice.

But hey, I will settle for caching the login. It appear the current setup is broken, because the cookiejar in ~/.cache/AUR/ hasn't been updated in 6 months despite my typing "y" every time it prompts me.

Xyne · 2015-12-14 01:40:46

It works here. Do you have to log in every time? Is the XDG_CACHE_HOME environment variable set for the user that runs the script?

What's the output of the following script?

#!//usr/bin/env python3
from AUR.Aurtomatic import get_default_cookiejar_path
print(get_default_cookiejar_path())

If it points to something other than ~/.cache/AUR/, check there for a cookiejar.

The next time that I play with the code I will reconsider using the keyring package.

eschwartz · 2015-12-14 05:59:02

Yes, that much works fine.

...

In AUR.Aurtomatic.CookieWrapper.__exit__
If you don't explicitly use the option to keep|remove, it prompts you for y/n (only if one already exists from previous runs), and eventually offers to unlink it on "n".

But never does it actually call self.aurploader.save-cookies()

self.aurploader.save-cookies() is only run on action == 'keep'

Or earlier, in

AUR.Aurtomatic.Aurtomatic.initialize(...  , cookiejar_path=None):
    ...
    elif cookiejar_path:
        self.save_cookies(cookiejar_path)

Then initialize gets called with

self.aurploader.initialize(login_file=self.login_file)

...

So I'm not sure how that is supposed to work, but it seems it only gets saved when you actually pass `-j keep`

I think, what was meant to happen is it automatically gets saved and maybe gets deleted if you choose to discard it.

Last edited by eschwartz (2015-12-14 06:03:21)

Xyne · 2015-12-14 20:57:20

It should be fixed now. I'm surprised that no one else has reported that bug. Thanks!

eschwartz · 2015-12-15 02:09:19

Either you have fewer users than you thought, or I am significantly lazier than the rest of them.

Thanks!
(I actually use randomly-generated letters/numbers/symbols passwords for everything, so I hate having to remember more of them than I have to.)

progandy · 2015-12-15 02:28:47

Eschwartz wrote:

Either you have fewer users than you thought, or I am significantly lazier than the rest of them.

I only use the aurquery part. Package upload is done with git and to write comments, the webpage is sufficient.
(Currently I use luajit (ffi) to cut down on the dependency tree for my aur helper, though)

Last edited by progandy (2015-12-15 02:29:51)

Arch Linux

#1 2011-06-27 12:33:32