You are not logged in.

#1 2008-02-07 12:56:55

ibendiben
Member
Registered: 2007-10-10
Posts: 519

Howto list files not known by pacman (-Ql) and not in /home

Imagine I want to list all files, except those listed by pacman -Ql, and except those stored in /home.
How would I do that simple? I want to see if I've got many leftovers from previous scripts I made, or uninstalled packages, or installed packages pacman doesn't know of.

Offline

#2 2008-02-07 13:01:32

tomk
Forum Fellow
From: Ireland
Registered: 2004-07-21
Posts: 9,839

Re: Howto list files not known by pacman (-Ql) and not in /home

Roughly like this - run find on your system (except /home), run pacman -Qo on the results, write unowned filenames to a file.

I'll leave the specifics up to you.

Offline

#3 2008-02-07 13:24:08

shining
Pacman Developer
Registered: 2006-05-10
Posts: 2,043

Re: Howto list files not known by pacman (-Ql) and not in /home

Another equivalent way : build a filelist of all files on your file system except /home with find.
build an excludelist (~ output of pacman -Ql)
then :
grep -v -f excludelist filelist


pacman roulette : pacman -S $(pacman -Slq | LANG=C sort -R | head -n $((RANDOM % 10)))

Offline

#4 2008-02-07 13:32:25

Gilneas
Member
From: Netherlands
Registered: 2006-10-22
Posts: 320

Re: Howto list files not known by pacman (-Ql) and not in /home

use find all the way:

find / \( -path '/sys' -o -path '/proc' -o -path '/home' \) -prune -o -type f -a -exec pacman -Qo '{}' \; | less

edit:
probably add /var in there too (and a 2>&1)
find / \( -path '/var' -o -path '/sys' -o -path '/proc' -o -path '/home' \) -prune -o -type f -a -exec pacman -Qo '{}' \; 2>&1 | less

Last edited by Gilneas (2008-02-07 13:39:56)

Offline

#5 2008-02-07 15:35:22

ibendiben
Member
Registered: 2007-10-10
Posts: 519

Re: Howto list files not known by pacman (-Ql) and not in /home

Okay, tongue
This is what I ended up doing:

find /etc /bin /sbin /opt /lib /usr -type f >/tmp/files
pacman -Ql | cut -d' ' -f2 | grep -v '/$' >/tmp/exclude
cat /tmp/files /tmp/exclude | sort | uniq -u >/tmp/orphans

It is pretty interesting really....
6334 results roll lol

EDIT: grep -v -f wouln't work, I don't quite understand

Last edited by ibendiben (2008-02-07 15:36:38)

Offline

#6 2008-02-07 15:43:36

ibendiben
Member
Registered: 2007-10-10
Posts: 519

Re: Howto list files not known by pacman (-Ql) and not in /home

I'm gonna try pacman -Qo now, seems better option, because -Ql doesn't know:
/bin/awk
/bin/bunzip2
/bin/bzcat
/bin/compress
/bin/dnsdomainname
/bin/domainname
/bin/nisdomainname
/bin/pidof
/bin/sh
/bin/ypdomainname

Offline

#7 2008-02-07 16:01:34

Cerebral
Forum Fellow
From: Waterloo, ON, CA
Registered: 2005-04-08
Posts: 3,108
Website

Re: Howto list files not known by pacman (-Ql) and not in /home

Wait, pacman -Ql doesn't list /bin/awk, but pacman -Qo /bin/awk can find an owner for it?  That seems odd to me.

Offline

#8 2008-02-07 16:25:40

ibendiben
Member
Registered: 2007-10-10
Posts: 519

Re: Howto list files not known by pacman (-Ql) and not in /home

I feel the same actually... but try for yourself!
Another question, I can't get

pacman -Qo `cat /tmp/orphans` | grep 'error' >/tmp/orphans2

to work...
tried different things already, but every time the results are displayed in stead of being written to the file.

/maybe it's the newlines, let's see
/nope, still the same mad

Last edited by ibendiben (2008-02-07 16:29:48)

Offline

#9 2008-02-07 16:30:12

tomk
Forum Fellow
From: Ireland
Registered: 2004-07-21
Posts: 9,839

Re: Howto list files not known by pacman (-Ql) and not in /home

pacman -Ql gawk shows /bin/awk for me.

Offline

#10 2008-02-07 16:32:32

ibendiben
Member
Registered: 2007-10-10
Posts: 519

Re: Howto list files not known by pacman (-Ql) and not in /home

hey, yeah it does for me too...
failure somewhere else then srry

Offline

#11 2008-02-07 16:32:54

shining
Pacman Developer
Registered: 2006-05-10
Posts: 2,043

Re: Howto list files not known by pacman (-Ql) and not in /home

ibendiben wrote:

Okay, tongue
This is what I ended up doing:

find /etc /bin /sbin /opt /lib /usr -type f >/tmp/files
pacman -Ql | cut -d' ' -f2 | grep -v '/$' >/tmp/exclude
cat /tmp/files /tmp/exclude | sort | uniq -u >/tmp/orphans

It is pretty interesting really....
6334 results roll lol

EDIT: grep -v -f wouln't work, I don't quite understand

Forget it, it was a bad idea. Apparently, every line needs to be quoted (simple sed line). Then it is still complaining about /usr/bin/[
(can be changed to /usr/bin/\[), but then the complexity explodes. It uses 100% cpu and makes the system swap like hell.
Your method is nicer and WAY more efficient, very nice find (uniq -u). smile

A small note : it should be -f2- instead of -f2 for dealing with filenames with space :
pacman -Ql | cut -d' ' -f2 | grep -v '/$' >/tmp/exclude


pacman roulette : pacman -S $(pacman -Slq | LANG=C sort -R | head -n $((RANDOM % 10)))

Offline

#12 2008-02-07 16:35:59

ibendiben
Member
Registered: 2007-10-10
Posts: 519

Re: Howto list files not known by pacman (-Ql) and not in /home

Have you tried this though?

find /etc /bin /sbin /opt /lib /usr -type f >/tmp/files
pacman -Ql | cut -d' ' -f2 | grep -v '/$' >/tmp/exclude
cat /tmp/files /tmp/exclude | sort | uniq -u >/tmp/orphans

What could be wrong?

Offline

#13 2008-02-07 16:40:36

nj
Member
Registered: 2007-04-06
Posts: 93

Re: Howto list files not known by pacman (-Ql) and not in /home

ibendiben wrote:

Another question, I can't get

pacman -Qo `cat /tmp/orphans` | grep 'error' >/tmp/orphans2

to work...
tried different things already, but every time the results are displayed in stead of being written to the file.

The error lines are printed to stderr.

pacman -Qo `cat /tmp/orphans` 2> /tmp/orphans2

Offline

#14 2008-02-07 16:42:58

ibendiben
Member
Registered: 2007-10-10
Posts: 519

Re: Howto list files not known by pacman (-Ql) and not in /home

aha, learned something again...
can't cope with all that yet thanks!

Offline

#15 2008-02-07 16:50:43

Cerebral
Forum Fellow
From: Waterloo, ON, CA
Registered: 2005-04-08
Posts: 3,108
Website

Re: Howto list files not known by pacman (-Ql) and not in /home

ibendiben wrote:

Have you tried this though?

find /etc /bin /sbin /opt /lib /usr -type f >/tmp/files
pacman -Ql | cut -d' ' -f2 | grep -v '/$' >/tmp/exclude
cat /tmp/files /tmp/exclude | sort | uniq -u >/tmp/orphans

What could be wrong?

Well, first, find out which file /bin/awk is missing from:

$ grep '/bin/awk' /tmp/files
$ grep '/bin/awk' /tmp/exclude

If it's in /tmp/exclude but not in /tmp/files, then somehow your find command isn't seeing /bin/awk.  When you run find, do you have read/execute permissions on /bin?

Offline

#16 2008-02-07 16:58:32

shining
Pacman Developer
Registered: 2006-05-10
Posts: 2,043

Re: Howto list files not known by pacman (-Ql) and not in /home

/bin/awk is a symlink. Your find command doesn't keep symlink.

Here are the commands I just used :

time sudo find /etc /bin /sbin /opt /lib /usr /home/httpd -type f -o -type l >/tmp/files
pacman -Ql | cut -d' ' -f2- | grep -v '/$' >/tmp/exclude
cat /tmp/files /tmp/exclude | sort | uniq -u >/tmp/orphans

Your last command doesn't deal with filename space either :
pacman -Qo `cat /tmp/orphans` 2> /tmp/orphans2


pacman roulette : pacman -S $(pacman -Slq | LANG=C sort -R | head -n $((RANDOM % 10)))

Offline

#17 2008-02-07 17:54:23

ibendiben
Member
Registered: 2007-10-10
Posts: 519

Re: Howto list files not known by pacman (-Ql) and not in /home

time -p (sudo find /etc /bin /sbin /opt /lib /usr -type f -o -type l >/tmp/files
pacman -Ql | cut -d' ' -f2- | grep -v '/$' >/tmp/exclude
cat /tmp/files /tmp/exclude | sort | uniq -u >/tmp/orphans)

real 3.82
user 3.05
sys 0.89

Thanks a lot.
I thought about symlinks at first, only forget about it at second tongue
I like this. Gives me 3203 lines to investigate.
-A lot of .pacsave files of course, and the old manpages (which should have been deleted),
-a lot of fonts (good?),
-also numbers of /gconf/gconf.xml.defaults/%gconf-tree...(what's that??),
-there's a whole range of /share/texmf-dist/tex/.. files too, and that's old leftovers from a wrong install of xemacs -(not enough free space) if I'm correct.
-/usr/bin/... and /usr/lib show A LOT yikes.

How safe would it be to just delete everything?? roll

Last edited by ibendiben (2008-02-07 18:08:15)

Offline

#18 2008-02-07 18:09:00

Gilneas
Member
From: Netherlands
Registered: 2006-10-22
Posts: 320

Re: Howto list files not known by pacman (-Ql) and not in /home

Dangerous!
.install files can produce additional files. Think about /usr/share/fonts/*/fonts.{dir,scale} for instance.

Offline

#19 2008-02-07 18:12:55

ibendiben
Member
Registered: 2007-10-10
Posts: 519

Re: Howto list files not known by pacman (-Ql) and not in /home

you mean those 'additional' files aren't register by pacman?

Offline

#20 2008-02-07 18:18:36

ibendiben
Member
Registered: 2007-10-10
Posts: 519

Re: Howto list files not known by pacman (-Ql) and not in /home

I'd be interested to know how much files other users would have...
Can someone post some of his numbers?

Offline

#21 2008-02-07 18:54:58

shining
Pacman Developer
Registered: 2006-05-10
Posts: 2,043

Re: Howto list files not known by pacman (-Ql) and not in /home

ibendiben wrote:

you mean those 'additional' files aren't register by pacman?

No these files are not tracked (just like any other files created as package installation or at runtime, as opposed to package creation)


pacman roulette : pacman -S $(pacman -Slq | LANG=C sort -R | head -n $((RANDOM % 10)))

Offline

#22 2008-02-07 19:03:12

shining
Pacman Developer
Registered: 2006-05-10
Posts: 2,043

Re: Howto list files not known by pacman (-Ql) and not in /home

ibendiben wrote:

I'd be interested to know how much files other users would have...
Can someone post some of his numbers?

I get 1381 files. But since you didn't include all directories in the find command, there are many legit files there (some are even critical).
For example, there are the kernel and grub files in /boot/, the web files in /home/httpd, etc.
But there are also some files that every user added manually that should stay, like /etc/X11/xorg.conf for example.

Anyway, removing these files will likely break your system. Even if you are careful and don't remove any critical ones, you can still mess up something. And all this for 0 benefit.

A little problem with the cat | uniq -u way is that we don't know which files are only in "files", in which files are only in "exclude".


pacman roulette : pacman -S $(pacman -Slq | LANG=C sort -R | head -n $((RANDOM % 10)))

Offline

#23 2008-02-07 19:56:48

Cerebral
Forum Fellow
From: Waterloo, ON, CA
Registered: 2005-04-08
Posts: 3,108
Website

Re: Howto list files not known by pacman (-Ql) and not in /home

shining wrote:

A little problem with the cat | uniq -u way is that we don't know which files are only in "files", in which files are only in "exclude".

I would think that a diff would be more useful than a uniq:

sudo find /bin /boot /etc /lib /opt /sbin /srv /usr /var -type f -o -type l | sort >/tmp/files
pacman -Ql | cut -d' ' -f2- | grep -v '/$' | sort >/tmp/exclude
diff /tmp/files /tmp/exclude >/tmp/orphans

Last edited by Cerebral (2008-02-07 20:17:08)

Offline

#24 2008-02-07 23:18:37

shining
Pacman Developer
Registered: 2006-05-10
Posts: 2,043

Re: Howto list files not known by pacman (-Ql) and not in /home

That's much better indeed. Seems like this gives a decent result now.
To find all files that should be installed as part of a package, but were not on the filesystem, I then did :

 grep ">" /tmp/orphans

This gave me a lot of files tongue I think my local database was a bit broken (I remember now that I must have messed up something once wink)
After removing / installing some packages, I only have some files from ttf-ms-fonts and zsh packages, because these are modified by their scriptlets.


pacman roulette : pacman -S $(pacman -Slq | LANG=C sort -R | head -n $((RANDOM % 10)))

Offline

#25 2008-02-08 10:13:56

ibendiben
Member
Registered: 2007-10-10
Posts: 519

Re: Howto list files not known by pacman (-Ql) and not in /home

I'm not yet content with the output of the diff command.
I do not like it, I tried using it before (also this time before I decided to use uniq) but it is difficult to understand. There are definitely lines (custom scripts in /usr/bin for example) that aren't displayed with a grep ">".
I see there are some folders, uncluding /usr/bin, displayed though. (what does that mean?)
As I understand it, diff has its own way of understanding whether a line should be added (>) or one should be deleted (<) to sync lines between the two files it compares. But then again I'm quite new to linux and bash, and stuff... so I'd like to be corrected. wink

There's a point though in the folder selection. If I don't select those folders outputted by a pacman -Ql, those folders-->files will be uniq....and therefor the uniq -u isn't perfect either. I think I can make up something though.
I could make a first column containing the names: "files" or "exlude", and then let sort and uniq ignore this first column.

shining wrote:
ibendiben wrote:

you mean those 'additional' files aren't register by pacman?

No these files are not tracked (just like any other files created as package installation or at runtime, as opposed to package creation)

Can you tell me how pacman succeeds in removing those packages/files then? If pacman can find out (when removing a package) which files belonged to a package, including those 'additional' files.... shouldn't there be a way to make a list of those files... including all 'additional' files?
As you see, I don't quite understand big_smile Maybe the simple question is: Doesn't pacman -Ql foo list all files it would remove with pacman -R foo?

Last edited by ibendiben (2008-02-08 10:16:10)

Offline

Board footer

Powered by FluxBB