You are not logged in.

#1 2014-12-24 21:11:30

gsingh93
Member
Registered: 2013-07-19
Posts: 96

Quick way to see if a list of packages exist in the repos or the AUR

For a script I'm writing, I'd like to check if a list of packages actually exists in the enabled repos or in the AUR. Currently, I'm looping through all packages in the list and runing `pacman -Ss package_name`, but there are two problems with this. The first is  pacman can return multiple packages here, whereas I only want it to return the package that exactly matches `package_name`. I can fix this by using grep to find an exact match for the name. The second problem is that this method is very slow, but I want this to take no more than a second for about 100 packages. Is there a faster way I can figure out which packages actually exist?

Offline

#2 2014-12-24 21:19:36

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 29,442
Website

Re: Quick way to see if a list of packages exist in the repos or the AUR

gsingh93 wrote:

The first is  pacman can return multiple packages here, whereas I only want it to return the package that exactly matches `package_name`. I can fix this by using grep to find an exact match for the name.

A better solution for that problem (as stated) is simply to use a proper regex.  The following returns only the single exact match:

pacman -Ss ^firefox$
gsingh93 wrote:

The second problem is that this method is very slow, but I want this to take no more than a second for about 100 packages. Is there a faster way I can figure out which packages actually exist?

Then a much better approach would be something like the following:

comm -23 your_list_file <(pacman -Ssq | sort)

EDIT: I just tested the above command using a random selection of 100 package names from the repos with 40 of the modified (so as not to match) and then I ran the command through `time`:

real	0m0.283s
user	0m0.227s
sys	0m0.030s

If you want this to work for the AUR as well, we'd just need a list of all package names in the AUR.  This should be easy to get from the web-interface (standby I'll see if I can get it).  Then basically you'd do the following:
1) pacman -Ssq > pkglist
2) cmd_to_get_aur_names >> pkglist
3) comm -23 <(sort your_list_file) <(sort pkglist)

EDIT 2: I don't know of any efficient way to get a list of all AUR packages.  The best idea I had was to "simply" wget and parse this page: http://pkgbuild.com/git/aur-mirror.git/tree/ but at 26MB that takes well more than a second to download even on a good connection.  I suspect it will be far more efficient to get the `comm` output from above for a list of packages not in the repos, then for each of those search the aur as a fallback (with something like cower).

Last edited by Trilby (2014-12-24 21:38:08)


"UNIX is simple and coherent..." - Dennis Ritchie, "GNU's Not UNIX" -  Richard Stallman

Offline

#3 2014-12-24 21:52:52

gsingh93
Member
Registered: 2013-07-19
Posts: 96

Re: Quick way to see if a list of packages exist in the repos or the AUR

Thanks for the response. I saw the regex feature in the man page, but it's not working for me. That command you posted gives no results.

In the year I've been using Arch, I never thought to run `pacman -Ss` without any packages. I never thought it'd return everything. That'll work great, thanks!

If anyone thinks of an efficient way to get a list of AUR packages, let me know. I'm going to use the fallback method for now.

Offline

#4 2014-12-24 21:57:25

jasonwryan
Anarchist
From: .nz
Registered: 2009-05-09
Posts: 30,424
Website

Re: Quick way to see if a list of packages exist in the repos or the AUR

gsingh93 wrote:

If anyone thinks of an efficient way to get a list of AUR packages, let me know. I'm going to use the fallback method for now.

https://wiki.archlinux.org/index.php/AurJson


Arch + dwm   •   Mercurial repos  •   Surfraw

Registered Linux User #482438

Offline

#5 2014-12-24 22:25:52

karol
Archivist
Registered: 2009-05-06
Posts: 25,440

Re: Quick way to see if a list of packages exist in the repos or the AUR

Trilby wrote:

The following returns only the single exact match:

pacman -Ss ^firefox$

Nope:

$ pacman -Ss ^firefox$
extra/firefox 34.0.5-1 [installed]
    Standalone web browser from mozilla.org
archlinuxcn/firefox-kde-opensuse 34.0.5-1
    Standalone web browser from mozilla.org with OpenSUSE patch, integrate better with KDE
$ pacman -Si firefox-kde-opensuse | grep firefox$
Conflicts With : firefox

'pacman -Ss' searches all fields, that's why you have to weed out false positives with e.g. grep or use expac:

$ expac "%n" -S firefox
firefox

Edit: For AUR, try https://aur.archlinux.org/packages/aurlist/

Last edited by karol (2014-12-24 22:33:34)

Offline

#6 2014-12-24 22:34:53

Xyne
Administrator/PM
Registered: 2008-08-03
Posts: 6,963
Website

Re: Quick way to see if a list of packages exist in the repos or the AUR

I've added a script named pkg-simple_search to my pkg_scripts package. It will search for exact matches in the repos and the AUR. You can use the options to either return a simple list of the packages found or a list of packages along with their locations.


My Arch Linux StuffForum EtiquetteCommunity Ethos - Arch is not for everyone

Offline

#7 2014-12-24 22:38:33

jasonwryan
Anarchist
From: .nz
Registered: 2009-05-09
Posts: 30,424
Website

Re: Quick way to see if a list of packages exist in the repos or the AUR

Xyne wrote:

I've added a script named pkg-simple_search to my pkg_scripts package. It will search for exact matches in the repos and the AUR. You can use the options to either return a simple list of the packages found or a list of packages along with their locations.

OT: Is there anywhere these are hosted where I can browse the source?


Arch + dwm   •   Mercurial repos  •   Surfraw

Registered Linux User #482438

Offline

#8 2014-12-24 22:40:40

Xyne
Administrator/PM
Registered: 2008-08-03
Posts: 6,963
Website

Re: Quick way to see if a list of packages exist in the repos or the AUR

jasonwryan wrote:

OT: Is there anywhere these are hosted where I can browse the source?

You can download the archives here: http://xyne.archlinux.ca/projects/pkg_scripts/src/


My Arch Linux StuffForum EtiquetteCommunity Ethos - Arch is not for everyone

Offline

#9 2014-12-24 22:43:44

jasonwryan
Anarchist
From: .nz
Registered: 2009-05-09
Posts: 30,424
Website

Re: Quick way to see if a list of packages exist in the repos or the AUR

Xyne wrote:
jasonwryan wrote:

OT: Is there anywhere these are hosted where I can browse the source?

You can download the archives here: http://xyne.archlinux.ca/projects/pkg_scripts/src/

Ta: I was being lazy and hoping for a web type thing smile


Arch + dwm   •   Mercurial repos  •   Surfraw

Registered Linux User #482438

Offline

#10 2014-12-24 22:52:07

Xyne
Administrator/PM
Registered: 2008-08-03
Posts: 6,963
Website

Re: Quick way to see if a list of packages exist in the repos or the AUR

jasonwryan wrote:
Xyne wrote:
jasonwryan wrote:

OT: Is there anywhere these are hosted where I can browse the source?

You can download the archives here: http://xyne.archlinux.ca/projects/pkg_scripts/src/

Ta: I was being lazy and hoping for a web type thing smile

Yeah, I figured. There is no web type thing because I'm lazy too. wink


My Arch Linux StuffForum EtiquetteCommunity Ethos - Arch is not for everyone

Offline

#11 2014-12-24 23:33:35

karol
Archivist
Registered: 2009-05-06
Posts: 25,440

Re: Quick way to see if a list of packages exist in the repos or the AUR

Xyne wrote:

I've added a script named pkg-simple_search to my pkg_scripts package.

Would it be possible to mention such changes in the news on your website and the feed?

I don't think xyne spelled backwards says 'lazy' ;P

Offline

#12 2014-12-25 00:27:26

Xyne
Administrator/PM
Registered: 2008-08-03
Posts: 6,963
Website

Re: Quick way to see if a list of packages exist in the repos or the AUR

karol wrote:
Xyne wrote:

I've added a script named pkg-simple_search to my pkg_scripts package.

Would it be possible to mention such changes in the news on your website and the feed?

I don't think xyne spelled backwards says 'lazy' ;P

http://xyne.archlinux.ca/news/#update smile

And with that the derailment of this thread is complete.

So, who's tried poutine? tongue


My Arch Linux StuffForum EtiquetteCommunity Ethos - Arch is not for everyone

Offline

#13 2015-01-04 21:47:13

helix
Member
Registered: 2013-02-17
Posts: 180

Re: Quick way to see if a list of packages exist in the repos or the AUR

Another way:

pacman -Ss package_name | grep -v '    ' | gawk -F '/' '{print $2}' | gawk '{print $1}' > result
grep "^package_name$" result
rm -f result

Last edited by helix (2015-01-04 21:49:17)

Offline

#14 2015-01-05 05:56:41

karol
Archivist
Registered: 2009-05-06
Posts: 25,440

Re: Quick way to see if a list of packages exist in the repos or the AUR

I have no idea why are you doing it this way, helix ...
Why not just

pacman -Ssq firefox | grep "^firefox$"

?

Offline

#15 2015-01-05 06:01:28

helix
Member
Registered: 2013-02-17
Posts: 180

Re: Quick way to see if a list of packages exist in the repos or the AUR

The original line included on a script of my own (debtap) uses package-query instead of pacman to search in AUR as well

Last edited by helix (2015-01-05 06:02:07)

Offline

#16 2015-01-06 18:02:04

Herk
Member
Registered: 2013-01-20
Posts: 4

Re: Quick way to see if a list of packages exist in the repos or the AUR

gsingh93 wrote:

If anyone thinks of an efficient way to get a list of AUR packages, let me know. I'm going to use the fallback method for now.

Maybe not needed anymore, but a complete list of AUR packages is available at https://aur.archlinux.org/packages.gz

Seems to be regenerated every hour.

Offline

#17 2015-01-06 19:16:14

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 29,442
Website

Re: Quick way to see if a list of packages exist in the repos or the AUR

Nice Herk.  Adding that to my previous post works well:

comm -23 <(sort -u needed_pkg_list) <(sort -u <(wget -q -O - https://aur.archlinux.org/packages.gz | gunzip) <(pacman -Ssq))

Time says:

real	0m1.569s
user	0m0.023s
sys	0m0.013s

Of course this time will be highly dependent on your internet connection - I'm on a fairly mediocre connection at the moment.  The plus side is it will not be affected noticably by the size of the "needed_pkg_list" file.


"UNIX is simple and coherent..." - Dennis Ritchie, "GNU's Not UNIX" -  Richard Stallman

Offline

#18 2021-12-13 10:43:29

kmille
Member
Registered: 2020-10-26
Posts: 5

Re: Quick way to see if a list of packages exist in the repos or the AUR

I think `pacman -Si` does what you want

kmille@linbox:~ pacman -Si linux            
Repository      : core
Name            : linux
Version         : 5.15.7.arch1-1
Description     : The Linux kernel and modules
Architecture    : x86_64
URL             : https://github.com/archlinux/linux/commits/v5.15.7-arch1
Licenses        : GPL2
Groups          : None
Provides        : VIRTUALBOX-GUEST-MODULES  WIREGUARD-MODULE
Depends On      : coreutils  kmod  initramfs
Optional Deps   : crda: to set the correct wireless channels of your country
                  linux-firmware: firmware images needed for some devices
Conflicts With  : None
Replaces        : virtualbox-guest-modules-arch  wireguard-arch
Download Size   : 130.22 MiB
Installed Size  : 135.23 MiB
Packager        : Jan Alexander Steffens (heftig) <heftig@archlinux.org>
Build Date      : Wed 08 Dec 2021 03:33:16 PM CET
Validated By    : MD5 Sum  SHA-256 Sum  Signature

kmille@linbox:~ pacman -Si yay  
error: package 'yay' was not found

Offline

#19 2021-12-13 11:13:13

WorMzy
Forum Moderator
From: Scotland
Registered: 2010-06-16
Posts: 11,787
Website

Re: Quick way to see if a list of packages exist in the repos or the AUR

Please don't necrobump old topics.

https://wiki.archlinux.org/title/Genera … bumping%22

Closing.


Sakura:-
Mobo: MSI MAG X570S TORPEDO MAX // Processor: AMD Ryzen 9 5950X @4.9GHz // GFX: AMD Radeon RX 5700 XT // RAM: 32GB (4x 8GB) Corsair DDR4 (@ 3000MHz) // Storage: 1x 3TB HDD, 6x 1TB SSD, 2x 120GB SSD, 1x 275GB M2 SSD

Making lemonade from lemons since 2015.

Online

Board footer

Powered by FluxBB