You are not logged in.

#1 2011-06-27 12:33:32

Xyne
Administrator/PM
Registered: 2008-08-03
Posts: 6,965
Website

python-aur: Python3 modules and scripts for automating AUR interaction

Project page: http://xyne.archlinux.ca/projects/python3-aur/

Update

This package now contains some utility scripts in addition to the Python 3 modules:

  • aurploader - upload package to the AUR from the command line, with support for comments, votes and notifications

  • aurquery - retrieve information about AUR packages

  • aurtomatic - post comments, vote, unvote, notify, unnotify, flag, unflag, etc.

Original Post

I usually don't announce modules that I release because I generally write them for myself and only release them to support my own applications. I think this one has the potential to be generally useful though, especially when combined with Reflector and Rémy's pyalpm.

The module uses threading to speed up retrievals and caches results in an SQLite3 database.

The source archive contains a directory named "scripts" with some examples. There's also a paconky replacement script.

As always, questions, suggestions and other feedback are welcome.

Last edited by Xyne (2013-03-01 18:35:38)


My Arch Linux StuffForum EtiquetteCommunity Ethos - Arch is not for everyone

Offline

#2 2011-06-27 14:04:10

lolilolicon
Member
Registered: 2009-03-05
Posts: 1,722

Re: python-aur: Python3 modules and scripts for automating AUR interaction

I suspect you didn't include uploading support because the lack of AUR RPC API support? I wonder when AUR will support this...
There're also the other user actions the RPC isn't currently aware of: vote, flag, notify, adopt, disown, delete, unvote, unflag, unnotify, and comments.

What's your plan regarding these features?

It would be nicer if you implemented a full AUR interface, so both AUR users and package maintainers could benefit greatly.


This silver ladybug at line 28...

Offline

#3 2011-06-27 18:03:35

Daenyth
Forum Fellow
From: Boston, MA
Registered: 2008-02-24
Posts: 1,244

Re: python-aur: Python3 modules and scripts for automating AUR interaction

It's a bit of a tangent to this, but can you also post the VCS information so people can easily clone/improve your stuff? It's a little annoying to not have anything except static tarball downloads.

Offline

#4 2011-06-28 02:21:20

Xyne
Administrator/PM
Registered: 2008-08-03
Posts: 6,965
Website

Re: python-aur: Python3 modules and scripts for automating AUR interaction

@lolilolicon
I may implement non-RPC features but I'm reluctant to scrape HTML pages for data. It will introduce long-term volatility and it will increase load on the server. It would be better if the AUR provided a general API for user actions but that is unlikely in the near future.

Still, I see the utility of supporting other user actions so I may end up implementing them.



@Daenyth
Moving all of my projects to Git and updating my release scripts is on my todo list. I hope to get to it soon.

Last edited by Xyne (2011-06-28 02:22:02)


My Arch Linux StuffForum EtiquetteCommunity Ethos - Arch is not for everyone

Offline

#5 2012-11-05 10:00:28

Xyne
Administrator/PM
Registered: 2008-08-03
Posts: 6,965
Website

Re: python-aur: Python3 modules and scripts for automating AUR interaction

The latest release includes some important changes which are described here.

See the project and AUR pages for more details.


My Arch Linux StuffForum EtiquetteCommunity Ethos - Arch is not for everyone

Offline

#6 2013-03-01 18:35:00

Xyne
Administrator/PM
Registered: 2008-08-03
Posts: 6,965
Website

Re: python-aur: Python3 modules and scripts for automating AUR interaction

The package now contains a new tool: aurtomatic
You can use it to vote, flag, comment and notify from the command line.

Vote for all of your AUR packages with

aurtomatic -a vote -i

My Arch Linux StuffForum EtiquetteCommunity Ethos - Arch is not for everyone

Offline

#7 2014-08-13 00:02:05

ava1ar
Member
From: NYC
Registered: 2009-10-20
Posts: 24

Re: python-aur: Python3 modules and scripts for automating AUR interaction

Looks like python3-aur can't upload splitted packages to AUR, can anybody confirm?

Offline

#8 2014-08-13 00:37:08

Xyne
Administrator/PM
Registered: 2008-08-03
Posts: 6,965
Website

Re: python-aur: Python3 modules and scripts for automating AUR interaction

I haven't tried. Can you give me an example for testing?


My Arch Linux StuffForum EtiquetteCommunity Ethos - Arch is not for everyone

Offline

#9 2014-08-13 00:39:27

ava1ar
Member
From: NYC
Registered: 2009-10-20
Posts: 24

Re: python-aur: Python3 modules and scripts for automating AUR interaction

Offline

#10 2014-08-13 04:32:54

Xyne
Administrator/PM
Registered: 2008-08-03
Posts: 6,965
Website

Re: python-aur: Python3 modules and scripts for automating AUR interaction

The upload works but it fails to confirm the upload because the RPC interface does not return information when querying a pkgbase. I'll bring this up on the aur-dev list and post an update here later.

<edit>If anyone wants to follow the discussion on the list, here it is.</edit>



Incidentally, the use of $startdir is deprecated and should be removed from the PKGBUILD.

It is also unnecessary (but not wrong) to include entries in the "conflicts" array for derivative packages (e.g. *-dev, *-git). It is the responsibility of the derivative packages to manage "conflicts" and "provides".

Last edited by Xyne (2014-08-13 04:58:03)


My Arch Linux StuffForum EtiquetteCommunity Ethos - Arch is not for everyone

Offline

#11 2014-12-25 03:56:24

Xyne
Administrator/PM
Registered: 2008-08-03
Posts: 6,965
Website

Re: python-aur: Python3 modules and scripts for automating AUR interaction

The package now includes AUR.PkgList and aurpkglist to access the full list of AUR packages. See the website for details.


My Arch Linux StuffForum EtiquetteCommunity Ethos - Arch is not for everyone

Offline

#12 2014-12-25 05:03:39

ava1ar
Member
From: NYC
Registered: 2009-10-20
Posts: 24

Re: python-aur: Python3 modules and scripts for automating AUR interaction

Thanks, will check.

Offline

#13 2015-07-11 19:16:47

Xyne
Administrator/PM
Registered: 2008-08-03
Posts: 6,965
Website

Re: python-aur: Python3 modules and scripts for automating AUR interaction

Aurploader users, please see my comments concerning the move to AUR 4.0 here.


My Arch Linux StuffForum EtiquetteCommunity Ethos - Arch is not for everyone

Offline

#14 2015-09-29 03:58:42

ngoonee
Forum Fellow
From: Between Thailand and Singapore
Registered: 2009-03-17
Posts: 7,358

Re: python-aur: Python3 modules and scripts for automating AUR interaction

Hi Xyne, just a poke on a rebuild due to python update to 3.5 (since I use your repo).


Allan-Volunteer on the (topic being discussed) mailn lists. You never get the people who matters attention on the forums.
jasonwryan-Installing Arch is a measure of your literacy. Maintaining Arch is a measure of your diligence. Contributing to Arch is a measure of your competence.
Griemak-Bleeding edge, not bleeding flat. Edge denotes falls will occur from time to time. Bring your own parachute.

Offline

#15 2015-10-06 19:27:31

Xyne
Administrator/PM
Registered: 2008-08-03
Posts: 6,965
Website

Re: python-aur: Python3 modules and scripts for automating AUR interaction

See here.


My Arch Linux StuffForum EtiquetteCommunity Ethos - Arch is not for everyone

Offline

#16 2015-12-07 05:01:11

eschwartz
Fellow
Registered: 2014-08-08
Posts: 4,097

Re: python-aur: Python3 modules and scripts for automating AUR interaction

It seems flagging a package out of date via aurtomatic is broken (for a while) -- ever since AUR 4.0 which requires you to (and lets you!) submit a comment when flagging something out-of-date.

Also, AUR.RPC now tries to access the brand-new XCGF without importing it. yikes

Xyne wrote:

@Daenyth
Moving all of my projects to Git and updating my release scripts is on my todo list. I hope to get to it soon.

Ping?

...

Random miscellaneous request: can this automatically check, say, $XDG_CONFIG_HOME/AUR.conf for the user/pass file, rather than requiring me to add an extra option to every invocation of aurtomatic. smile

Last edited by eschwartz (2015-12-07 05:06:14)


Managing AUR repos The Right Way -- aurpublish (now a standalone tool)

Offline

#17 2015-12-07 08:19:55

Xyne
Administrator/PM
Registered: 2008-08-03
Posts: 6,965
Website

Re: python-aur: Python3 modules and scripts for automating AUR interaction

I have added the missing import statements and fixed package flagging.

Uploading the Git repos is still on my "I should probably do that but meh, not a priority" list. I have a few concerns about uploading them.

I suppose that I could add a central check for a login file which would allow any tool based on this login automatically, but that would also be a security risk. It's a bad idea to have plaintext login credentials in general, and even worse to have them in a predictable location, especially if they would be used by default by any malicious script.

Can't you just use an alias or wrapper script?

alias aurto=/usr/bin/aurtomatic -l /path/to/login/credentials'

My Arch Linux StuffForum EtiquetteCommunity Ethos - Arch is not for everyone

Offline

#18 2015-12-07 19:42:54

Yannick_LM
Member
Registered: 2008-12-22
Posts: 142

Re: python-aur: Python3 modules and scripts for automating AUR interaction

You could also use python package keyring to securely store the password

Offline

#19 2015-12-07 21:31:30

Xyne
Administrator/PM
Registered: 2008-08-03
Posts: 6,965
Website

Re: python-aur: Python3 modules and scripts for automating AUR interaction

Yannick_LM wrote:

You could also use python package keyring to securely store the password

That may avoid the issue of storing the password in plaintext, but it would still be easily accessible via a script.


My Arch Linux StuffForum EtiquetteCommunity Ethos - Arch is not for everyone

Offline

#20 2015-12-13 06:56:25

eschwartz
Fellow
Registered: 2014-08-08
Posts: 4,097

Re: python-aur: Python3 modules and scripts for automating AUR interaction

Well, there is such a thing as too many aliases, so I thought I'd mention the idea of a default conf file.

No one has to use it, but I don't really see why it is the job of this tool to tell me what I should do. wink It's not like this would be the only tool out there that empowers the user to make a dangerous choice.

But hey, I will settle for caching the login. It appear the current setup is broken, because the cookiejar in ~/.cache/AUR/ hasn't been updated in 6 months despite my typing "y" every time it prompts me. wink


Managing AUR repos The Right Way -- aurpublish (now a standalone tool)

Offline

#21 2015-12-14 01:40:46

Xyne
Administrator/PM
Registered: 2008-08-03
Posts: 6,965
Website

Re: python-aur: Python3 modules and scripts for automating AUR interaction

It works here. Do you have to log in every time? Is the XDG_CACHE_HOME environment variable set for the user that runs the script?

What's the output of the following script?

#!//usr/bin/env python3
from AUR.Aurtomatic import get_default_cookiejar_path
print(get_default_cookiejar_path())

If it points to something other than ~/.cache/AUR/, check there for a cookiejar.


The next time that I play with the code I will reconsider using the keyring package.


My Arch Linux StuffForum EtiquetteCommunity Ethos - Arch is not for everyone

Offline

#22 2015-12-14 05:59:02

eschwartz
Fellow
Registered: 2014-08-08
Posts: 4,097

Re: python-aur: Python3 modules and scripts for automating AUR interaction

Yes, that much works fine.

...

In AUR.Aurtomatic.CookieWrapper.__exit__
If you don't explicitly use the option to keep|remove, it prompts you for y/n (only if one already exists from previous runs), and eventually offers to unlink it on "n".

But never does it actually call self.aurploader.save-cookies()

self.aurploader.save-cookies() is only run on action == 'keep'

Or earlier, in

AUR.Aurtomatic.Aurtomatic.initialize(...  , cookiejar_path=None):
    ...
    elif cookiejar_path:
        self.save_cookies(cookiejar_path)

Then initialize gets called with

self.aurploader.initialize(login_file=self.login_file)

...

...

So I'm not sure how that is supposed to work, but it seems it only gets saved when you actually pass `-j keep`

I think, what was meant to happen is it automatically gets saved and maybe gets deleted if you choose to discard it.

Last edited by eschwartz (2015-12-14 06:03:21)


Managing AUR repos The Right Way -- aurpublish (now a standalone tool)

Offline

#23 2015-12-14 20:57:20

Xyne
Administrator/PM
Registered: 2008-08-03
Posts: 6,965
Website

Re: python-aur: Python3 modules and scripts for automating AUR interaction

It should be fixed now. I'm surprised that no one else has reported that bug. Thanks!


My Arch Linux StuffForum EtiquetteCommunity Ethos - Arch is not for everyone

Offline

#24 2015-12-15 02:09:19

eschwartz
Fellow
Registered: 2014-08-08
Posts: 4,097

Re: python-aur: Python3 modules and scripts for automating AUR interaction

Either you have fewer users than you thought, or I am significantly lazier than the rest of them. big_smile big_smile

Thanks!
(I actually use randomly-generated letters/numbers/symbols passwords for everything, so I hate having to remember more of them than I have to.)


Managing AUR repos The Right Way -- aurpublish (now a standalone tool)

Offline

#25 2015-12-15 02:28:47

progandy
Member
Registered: 2012-05-17
Posts: 5,269

Re: python-aur: Python3 modules and scripts for automating AUR interaction

Eschwartz wrote:

Either you have fewer users than you thought, or I am significantly lazier than the rest of them. big_smile big_smile

I only use the aurquery part. Package upload is done with git and to write comments, the webpage is sufficient.
(Currently I use luajit (ffi) to cut down on the dependency tree for my aur helper, though)

Last edited by progandy (2015-12-15 02:29:51)


| alias CUTF='LANG=en_XX.UTF-8@POSIX ' |

Online

Board footer

Powered by FluxBB