You are not logged in.

#1 2020-07-28 17:48:50

loqs
Member
Registered: 2014-03-06
Posts: 11,247

[SOLVED] Comand to search all files in a repository

End result I want to achieve,  find pam config files using pam_tally.so or pam_tally2.so in any package in the official repositories.

Is there already a tool that can do this I looked in devtools,  pacutils,  pacman-contrib and expac but could not find such a tool.

I can use pacman -Fy etc/pam.d/ to produce a list of packages owning a certain path.
From there is there a way to get the current package name how to get the archive name to check if it is in the package cache?
Then bsdtar extract the archive,  grep output,  clean and repeat.

Last edited by loqs (2020-07-28 22:05:23)

Offline

#2 2020-07-28 17:59:36

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 23,953
Website

Re: [SOLVED] Comand to search all files in a repository

pacman -Sp $(pacman -Fq etc/pam.d/)

This will generate the list of remote urls to the packages in question.  If you want to check against the local cache, the filename at the end of the url would/should match what'd be found in the cache - if not cached, curl the url, pipe to bsdtar with appropriate options, and then to grep.

EDIT: Holy awesome!  The -Sp will actually print file:// urls for locally cached files, so you can skip that step.

Here's a start:

#!/bin/sh

pacman -Spdd $(pacman -Fq etc/pam.d/) | while read url; do
	case $url in
		file*)
			bsdtar --include=etc/pam.d/* -xOf ${url#file://} | grep WHATAVER
		;;
		http*)
			curl -s $url | bsdtar --include=etc/pam.d/* -xOf ${url#file://} | grep WHATAVER
		;;
	esac
done

EDIT: added 'dd' flags as per below.

Last edited by Trilby (2020-07-28 18:21:22)


"UNIX is simple and coherent..." - Dennis Ritchie, "GNU's Not UNIX" -  Richard Stallman

Offline

#3 2020-07-28 18:19:13

loqs
Member
Registered: 2014-03-06
Posts: 11,247

Re: [SOLVED] Comand to search all files in a repository

Noticed some unexpected entries in the results:

pacman -Sp community/xpra
https://mirrors.kernel.org/archlinux/community/os/x86_64/python-lz4-2.2.1-4-x86_64.pkg.tar.zst
https://mirrors.kernel.org/archlinux/extra/os/x86_64/xf86-video-dummy-0.3.8-4-x86_64.pkg.tar.zst
https://mirrors.kernel.org/archlinux/community/os/x86_64/python-netifaces-0.10.9-3-x86_64.pkg.tar.xz
file:///var/cache/pacman/pkg/python-rencode-1.0.6-3-x86_64.pkg.tar.xz
https://mirrors.kernel.org/archlinux/extra/os/x86_64/freeglut-3.2.1-2-x86_64.pkg.tar.zst
https://mirrors.kernel.org/archlinux/extra/os/x86_64/python-opengl-3.1.5-1-any.pkg.tar.zst
https://mirrors.kernel.org/archlinux/community/os/x86_64/xpra-3.0.9-3-x86_64.pkg.tar.zst

Offline

#4 2020-07-28 18:20:54

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 23,953
Website

Re: [SOLVED] Comand to search all files in a repository

Ha ha, oops!  This is better:

pacman -Spdd community/xpra

"UNIX is simple and coherent..." - Dennis Ritchie, "GNU's Not UNIX" -  Richard Stallman

Offline

#5 2020-07-28 22:05:06

loqs
Member
Registered: 2014-03-06
Posts: 11,247

Re: [SOLVED] Comand to search all files in a repository

Thank you Trilby.

Added -L to curl invocation for redirections and for http replaced ${url#file://} with - in the bsdtar invocation to use stdin.

Offline

#6 2020-07-28 23:31:18

eschwartz
Trusted User/Bug Wrangler
Registered: 2014-08-08
Posts: 3,537

Re: [SOLVED] Comand to search all files in a repository

Any url handled by pacman must be usable via libcurl:

#!/bin/sh

pacman -Spdd $(pacman -Fq etc/pam.d/) | while read url; do
    curl -s $url | bsdtar --include=etc/pam.d/* -xOf - | grep WHATAVER
done

The http block anyway needed to use "-". But curl can read a file:// url from disk and emit its contents on stdout, and essentially serves as an exotic syntax for `cat`.


Managing AUR repos The Right Way -- aurpublish (now a standalone tool)

Offline

#7 2020-07-28 23:35:12

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 23,953
Website

Re: [SOLVED] Comand to search all files in a repository

Nice.  I suspected curl could handle "file://" but I tried it once and it didn't work as intended.  I didn't investigate further at the time.


"UNIX is simple and coherent..." - Dennis Ritchie, "GNU's Not UNIX" -  Richard Stallman

Offline

#8 2020-07-29 00:03:23

eschwartz
Trusted User/Bug Wrangler
Registered: 2014-08-08
Posts: 3,537

Re: [SOLVED] Comand to search all files in a repository

curl does require full paths, but that's how file:// urls are defined. It's also possible to compile curl with file:// support disabled. I'd be interested to hear what happened when you "tried it once and it didn't work as intended".


Managing AUR repos The Right Way -- aurpublish (now a standalone tool)

Offline

#9 2020-07-29 00:47:38

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 23,953
Website

Re: [SOLVED] Comand to search all files in a repository

Not much to hear: I fucked something up apparently!  I can't replicate it.  At the time it was nested in the script and simply didn't produce any expected output.  I suspect the other glaring error in the very rough draft script was the cause: I copy-pasted the line from the local file case statement to the curl one but didn't replace the ${url...} with a "-" or /dev/stdin in the bsdtar command.  So curl likely worked but the pipeline ignored it's output.  But this is just speculation as it was an isolated incident while I was tinkering for all of 5 minutes to find a rough first pass at an idea of how to accomplish this.


"UNIX is simple and coherent..." - Dennis Ritchie, "GNU's Not UNIX" -  Richard Stallman

Offline

Board footer

Powered by FluxBB