Quick way to see if a list of packages exist in the repos or the AUR

gsingh93 · 2014-12-24 21:11:30

For a script I'm writing, I'd like to check if a list of packages actually exists in the enabled repos or in the AUR. Currently, I'm looping through all packages in the list and runing `pacman -Ss package_name`, but there are two problems with this. The first is pacman can return multiple packages here, whereas I only want it to return the package that exactly matches `package_name`. I can fix this by using grep to find an exact match for the name. The second problem is that this method is very slow, but I want this to take no more than a second for about 100 packages. Is there a faster way I can figure out which packages actually exist?

Trilby · 2014-12-24 21:19:36

gsingh93 wrote:

The first is pacman can return multiple packages here, whereas I only want it to return the package that exactly matches `package_name`. I can fix this by using grep to find an exact match for the name.

A better solution for that problem (as stated) is simply to use a proper regex. The following returns only the single exact match:

pacman -Ss ^firefox$

gsingh93 wrote:

The second problem is that this method is very slow, but I want this to take no more than a second for about 100 packages. Is there a faster way I can figure out which packages actually exist?

Then a much better approach would be something like the following:

comm -23 your_list_file <(pacman -Ssq | sort)

EDIT: I just tested the above command using a random selection of 100 package names from the repos with 40 of the modified (so as not to match) and then I ran the command through `time`:

real	0m0.283s
user	0m0.227s
sys	0m0.030s

If you want this to work for the AUR as well, we'd just need a list of all package names in the AUR. This should be easy to get from the web-interface (standby I'll see if I can get it). Then basically you'd do the following:
1) pacman -Ssq > pkglist
2) cmd_to_get_aur_names >> pkglist
3) comm -23 <(sort your_list_file) <(sort pkglist)

EDIT 2: I don't know of any efficient way to get a list of all AUR packages. The best idea I had was to "simply" wget and parse this page: http://pkgbuild.com/git/aur-mirror.git/tree/ but at 26MB that takes well more than a second to download even on a good connection. I suspect it will be far more efficient to get the `comm` output from above for a list of packages not in the repos, then for each of those search the aur as a fallback (with something like cower).

Last edited by Trilby (2014-12-24 21:38:08)

gsingh93 · 2014-12-24 21:52:52

Thanks for the response. I saw the regex feature in the man page, but it's not working for me. That command you posted gives no results.

In the year I've been using Arch, I never thought to run `pacman -Ss` without any packages. I never thought it'd return everything. That'll work great, thanks!

If anyone thinks of an efficient way to get a list of AUR packages, let me know. I'm going to use the fallback method for now.

jasonwryan · 2014-12-24 21:57:25

gsingh93 wrote:

If anyone thinks of an efficient way to get a list of AUR packages, let me know. I'm going to use the fallback method for now.

https://wiki.archlinux.org/index.php/AurJson

karol · 2014-12-24 22:25:52

Trilby wrote:

The following returns only the single exact match:
pacman -Ss ^firefox$

Nope:

$ pacman -Ss ^firefox$
extra/firefox 34.0.5-1 [installed]
    Standalone web browser from mozilla.org
archlinuxcn/firefox-kde-opensuse 34.0.5-1
    Standalone web browser from mozilla.org with OpenSUSE patch, integrate better with KDE

$ pacman -Si firefox-kde-opensuse | grep firefox$
Conflicts With : firefox

'pacman -Ss' searches all fields, that's why you have to weed out false positives with e.g. grep or use expac:

$ expac "%n" -S firefox
firefox

Edit: For AUR, try https://aur.archlinux.org/packages/aurlist/

Last edited by karol (2014-12-24 22:33:34)

Xyne · 2014-12-24 22:34:53

I've added a script named pkg-simple_search to my pkg_scripts package. It will search for exact matches in the repos and the AUR. You can use the options to either return a simple list of the packages found or a list of packages along with their locations.

jasonwryan · 2014-12-24 22:38:33

Xyne wrote:

I've added a script named pkg-simple_search to my pkg_scripts package. It will search for exact matches in the repos and the AUR. You can use the options to either return a simple list of the packages found or a list of packages along with their locations.

OT: Is there anywhere these are hosted where I can browse the source?

Xyne · 2014-12-24 22:40:40

jasonwryan wrote:

OT: Is there anywhere these are hosted where I can browse the source?

You can download the archives here: http://xyne.archlinux.ca/projects/pkg_scripts/src/

jasonwryan · 2014-12-24 22:43:44

Xyne wrote:

jasonwryan wrote:
OT: Is there anywhere these are hosted where I can browse the source?
You can download the archives here: http://xyne.archlinux.ca/projects/pkg_scripts/src/

Ta: I was being lazy and hoping for a web type thing

Xyne · 2014-12-24 22:52:07

jasonwryan wrote:

Xyne wrote:
jasonwryan wrote:
OT: Is there anywhere these are hosted where I can browse the source?
You can download the archives here: http://xyne.archlinux.ca/projects/pkg_scripts/src/
Ta: I was being lazy and hoping for a web type thing

Yeah, I figured. There is no web type thing because I'm lazy too.

karol · 2014-12-24 23:33:35

Xyne wrote:

I've added a script named pkg-simple_search to my pkg_scripts package.

Would it be possible to mention such changes in the news on your website and the feed?

I don't think xyne spelled backwards says 'lazy' ;P

Xyne · 2014-12-25 00:27:26

karol wrote:

Xyne wrote:
I've added a script named pkg-simple_search to my pkg_scripts package.
Would it be possible to mention such changes in the news on your website and the feed?
I don't think xyne spelled backwards says 'lazy' ;P

http://xyne.archlinux.ca/news/#update

And with that the derailment of this thread is complete.

So, who's tried poutine?

helix · 2015-01-04 21:47:13

Another way:

pacman -Ss package_name | grep -v '    ' | gawk -F '/' '{print $2}' | gawk '{print $1}' > result
grep "^package_name$" result
rm -f result

Last edited by helix (2015-01-04 21:49:17)

karol · 2015-01-05 05:56:41

I have no idea why are you doing it this way, helix ...
Why not just

pacman -Ssq firefox | grep "^firefox$"

?

helix · 2015-01-05 06:01:28

The original line included on a script of my own (debtap) uses package-query instead of pacman to search in AUR as well

Last edited by helix (2015-01-05 06:02:07)

Herk · 2015-01-06 18:02:04

gsingh93 wrote:

If anyone thinks of an efficient way to get a list of AUR packages, let me know. I'm going to use the fallback method for now.

Maybe not needed anymore, but a complete list of AUR packages is available at https://aur.archlinux.org/packages.gz

Seems to be regenerated every hour.

Trilby · 2015-01-06 19:16:14

Nice Herk. Adding that to my previous post works well:

comm -23 <(sort -u needed_pkg_list) <(sort -u <(wget -q -O - https://aur.archlinux.org/packages.gz | gunzip) <(pacman -Ssq))

Time says:

real	0m1.569s
user	0m0.023s
sys	0m0.013s

Of course this time will be highly dependent on your internet connection - I'm on a fairly mediocre connection at the moment. The plus side is it will not be affected noticably by the size of the "needed_pkg_list" file.

kmille · 2021-12-13 10:43:29

I think `pacman -Si` does what you want

kmille@linbox:~ pacman -Si linux            
Repository      : core
Name            : linux
Version         : 5.15.7.arch1-1
Description     : The Linux kernel and modules
Architecture    : x86_64
URL             : https://github.com/archlinux/linux/commits/v5.15.7-arch1
Licenses        : GPL2
Groups          : None
Provides        : VIRTUALBOX-GUEST-MODULES  WIREGUARD-MODULE
Depends On      : coreutils  kmod  initramfs
Optional Deps   : crda: to set the correct wireless channels of your country
                  linux-firmware: firmware images needed for some devices
Conflicts With  : None
Replaces        : virtualbox-guest-modules-arch  wireguard-arch
Download Size   : 130.22 MiB
Installed Size  : 135.23 MiB
Packager        : Jan Alexander Steffens (heftig) <heftig@archlinux.org>
Build Date      : Wed 08 Dec 2021 03:33:16 PM CET
Validated By    : MD5 Sum  SHA-256 Sum  Signature

kmille@linbox:~ pacman -Si yay  
error: package 'yay' was not found

WorMzy · 2021-12-13 11:13:13

Please don't necrobump old topics.

https://wiki.archlinux.org/title/Genera … bumping%22

Closing.

Arch Linux

#1 2014-12-24 21:11:30

Quick way to see if a list of packages exist in the repos or the AUR

#2 2014-12-24 21:19:36

Re: Quick way to see if a list of packages exist in the repos or the AUR

#3 2014-12-24 21:52:52

Re: Quick way to see if a list of packages exist in the repos or the AUR

#4 2014-12-24 21:57:25

Re: Quick way to see if a list of packages exist in the repos or the AUR

#5 2014-12-24 22:25:52

Re: Quick way to see if a list of packages exist in the repos or the AUR

#6 2014-12-24 22:34:53

Re: Quick way to see if a list of packages exist in the repos or the AUR

#7 2014-12-24 22:38:33

Re: Quick way to see if a list of packages exist in the repos or the AUR

#8 2014-12-24 22:40:40

Re: Quick way to see if a list of packages exist in the repos or the AUR

#9 2014-12-24 22:43:44

Re: Quick way to see if a list of packages exist in the repos or the AUR

#10 2014-12-24 22:52:07

Re: Quick way to see if a list of packages exist in the repos or the AUR

#11 2014-12-24 23:33:35

Re: Quick way to see if a list of packages exist in the repos or the AUR

#12 2014-12-25 00:27:26

Re: Quick way to see if a list of packages exist in the repos or the AUR

#13 2015-01-04 21:47:13

Re: Quick way to see if a list of packages exist in the repos or the AUR

#14 2015-01-05 05:56:41

Re: Quick way to see if a list of packages exist in the repos or the AUR

#15 2015-01-05 06:01:28

Re: Quick way to see if a list of packages exist in the repos or the AUR

#16 2015-01-06 18:02:04

Re: Quick way to see if a list of packages exist in the repos or the AUR

#17 2015-01-06 19:16:14

Re: Quick way to see if a list of packages exist in the repos or the AUR

#18 2021-12-13 10:43:29

Re: Quick way to see if a list of packages exist in the repos or the AUR

#19 2021-12-13 11:13:13

Re: Quick way to see if a list of packages exist in the repos or the AUR

Board footer