Arch Linux

bartus

Member

Registered: 2013-05-13

Posts: 50

List all AUR packages.

I've used to use this script to gather information and statistics for AUR packages.

This fragment obtain names for all AUR packages:

declare -i i=0
while 
	pkgs=$(curl -s "https://aur.archlinux.org/packages/?SB=n&PP=250&O=$i"|hq "tr td:first-child" text)
	[ -n "$pkgs" ]
do 
	echo "$pkgs"
	((i+=250))
done

But in `aurweb:v5` limits number of listed packages to `2500` those rendering this solution obsolete.

Is there any other way to list all ~70k names of packages hosted on AUR repo?

twelveeighty

Member

From: Alberta, Canada

Registered: 2011-09-04

Posts: 1,115

Re: List all AUR packages.

It probably comes down to understanding *why* there are limits imposed on the AUR RPC interface and then see if you can work within those limits. My guess is that the limits are to preserve general system performance for the regular Web users as well as to guard hosting costs, which are typically tied to the amount of egress data served. I don't maintain the system, so these are pure guesses. Can you change the script to do a 'page-wise' query that stays within the documented limits? It looks like you can do a maximum of 4,000 requests per day, each with a max of 5,000 search results, which sounds like plenty of room to get all 70k, if all you need are the package names?

What are you trying to achieve with downloading all 70k package names that cannot be achieved with the standard AUR Web UI?

WorMzy

Forum Moderator

From: Scotland

Registered: 2010-06-16

Posts: 12,263

Website

Re: List all AUR packages.

https://wiki.archlinux.org/title/Arch_U … _packages?

---

Sakura:-
Mobo: MSI MAG X570S TORPEDO MAX // Processor: AMD Ryzen 9 5950X @4.9GHz // GFX: AMD Radeon RX 5700 XT // RAM: 32GB (4x 8GB) Corsair DDR4 (@ 3000MHz) // Storage: 1x 3TB HDD, 6x 1TB SSD, 2x 120GB SSD, 1x 275GB M2 SSD

Making lemonade from lemons since 2015.

bartus

Member

Registered: 2013-05-13

Posts: 50

Re: List all AUR packages.

https://wiki.archlinux.org/title/Arch_U … _packages?

Exactly what I was looking forヾ(˙❥˙)ﾉ

What are you trying to achieve with downloading all 70k

I've used those to check some linting schemes I was developing for my custom aurutils addons, best dataset to pin point any edge cases. My last "grab" was getting old and I was looking for a way to refresh it.

Last edited by bartus (2021-06-12 18:28:28)

eschwartz

Fellow

Registered: 2014-08-08

Posts: 4,097

Re: List all AUR packages.

my custom aurutils addons

So you're not aware that

man aur-pkglist

which is part of aurutils, implements WorMzy's recommendation?

---

Managing AUR repos The Right Way -- aurpublish (now a standalone tool)