[SOLVED] Will files owned by packages ever change?

donaastor · 2022-03-23 03:09:23

Do files from the list printed with "pacman -Ql" ever change? I made a c++ program to find all files not owned by any package and to sum their sizes and it (works btw, I excluded tmpfs and symlink) gives a seemingly small value. So I wonder, do files owned by some package grow over time?

Disambiguation: By "change" I meant during runtime, not a package update or changes made by users (for example, config files). So, can the contents of a file that is owned by a package A change during runtime of that package A?

Probable answer: NO.

Oh, and my code is here: https://github.com/donaastor/orphan-memory

Last edited by donaastor (2022-03-24 23:28:14)

Trilby · 2022-03-23 03:15:05

I'm not sure I understand your question. Actually I'm totally sure I don't: I can see it as ambiguous with at least two meanings, but neither one of them are at all related to the comment about your C++ program.

Are you asking whether the list of files could change, or whether the size / content of the files in that list could change? In either case, the answer is of course they change anytime the package owning them is updated (or a new package added, or and old one removed). Other than that they could change ... but only if you change them (by editing or deleting a file). Of course they will not just magically change on their own (though I'm pretty sure I could give the benefit of the doubt that this isn't what you'd be asking about).

So is any of the above related to what you are asking? If so, what does the result of your program that calculates the size of other unowned files have to do with this?

Aside: how do you conclude that your program "works btw" when you express doubt about the result it provides as seeming too small??

Last edited by Trilby (2022-03-23 03:18:02)

ngoonee · 2022-03-23 07:19:56

One example of a drastic (100%) change would be when python updates version, every package which used to have files in /usr/lib/python3.(x)/site-packages will now have files in /usr/lib/python3.(x+1)/site-packages

dogknowsnx · 2022-03-23 08:24:56

Not really a response to your question, but there is a community maintained package, which looks for orphaned files:
https://archlinux.org/packages/community/any/lostfiles/

You could compare its output with yours...

donaastor · 2022-03-23 14:16:25

@Trilby Sorry, I now see that I was ambiguous indeed. I meant to ask "Can the contents of a file that is owned by some package A be changed during runtime of the package A?" So I guess your "magic changes" were the most relevant haha. My program has ~nothing to do with it, it is just how I came up with this wonder. I am 99% sure that it works because it also sums up the sizes of owned files and those 2 sums together sum up to the same value my "du" command prints (minus Shmem, that is: files in tmpfs). Thank you for your response!
@dogknowsnx Thanks! I will compare it later to really see.
@ngoonee Yes, that is true. It falls into my definition of "change"

I added a disambiguation to my question to be more precise.

Trilby · 2022-03-23 14:47:13

donaastor wrote:

Can the contents of a file that is owned by some package A be changed during runtime of the package A.

Thanks, this is much more clear. The answer is technically 'yes', but rarely and only under specific conditions. And for most purposes you could likely work under the assumption that the answer is effectively 'no'.

Logs, or user configuration data which may change regularly while a program is running are not generally owned by the package. So these changes are not relevant. However, you will deliberately change some package-owned configs from time time time - perhaps most often right after installing them and initially setting them up; you would need to be root (and / or use su/sudo) to do this and the relevant files would most likely be under /etc.

I don't know of cases where any package would modify it's own files while it was running. It's certainly possible, but only if that program was run as root. This would be most likely for various services that are launched as root by systemd or other service managers. This type of software does not generally modify packaged/distributed files, but it is possible, in contrast to a program running as a regular user which could not possibly modify files managed by pacman.

donaastor wrote:

@ngoonee Yes, that is true. It falls into my definition of "change"

Wait, does it? He's referring specifically to a package upgrade run by pacman. If these changes count, then all of my above answer is irrelevant, and the true answer is 'yes' the size and content of these files will change quite frequently: every time there is an update. But in your good revision to your initial post, you explicitly excluded changes due to package updates.

Note you can check the output of `pacman -Qkk` to see if any package-owned files have changed since they were installed with the latest version of the package that owns them.

A few files that most of us will have explicitly modified include /etc/locale.gen and /etc/pacman.conf. Files that have been changed by the software (and not explicitly with an editor by us) on most systems would include /etc/{shadow,gshadow,group,passwd} and possibly /etc/shells.

Last edited by Trilby (2022-03-23 14:57:52)

seth · 2022-03-23 15:15:00

The OP in edit wrote:

By "change" I meant during runtime, not a package update or changes made by users (for example, config files). So, can the contents of a file that is owned by a package A change during runtime of that package A?

a) what do you understand as "runtime of that package"?
b) how does any of this relate to "program to find all files not owned by any package and to sum their sizes"?

grep -rhA1 '%SIZE%' --include desc /var/lib/pacman/local/ | grep -v -- '--\n%SIZE%' | awk '{s+=$1} END {print s/1024^3}' | bc -l

and discount that from df?

Edit: though this won't be 100% accurate for the few config changes and eg. kernel depmod database but you should be off by far less than 1% and it's MUCH faster than stat'ing the entire disk and matching that against the output of "pacman -Ql" for each and every package.

Last edited by seth (2022-03-23 15:19:17)

Trilby · 2022-03-23 16:33:02

(non-posix)grep + grep + awk + bc ???

awk '/%SIZE%/{getline;s+=$0;}END{print s/(1024^3);}' /var/lib/pacman/local/*/desc

Last edited by Trilby (2022-03-23 16:36:03)

seth · 2022-03-23 17:38:59

"time" it.
(Though I genuinly though awk would not interpret the print token)

Trilby · 2022-03-23 18:21:39

Well that's crazy. I've been trying to figure out how / why the many grep version could be faster as it's still doing the math in awk (despite an apparent intent otherwise).

EDIT: actually comparing gawk and gnu's grep end up giving identical times for 100 times through each version... and this is the first thing I've found busybox to be slower at

EDIT2: ok, it seems to be the "-r --include desc $path" is what really gives grep the advantage over all other invocations (including grep with the shell-globbed path). So the rate-limiting step is the file globbing done by the shell (well mostly... ash is faster than bash at this).

Last edited by Trilby (2022-03-23 18:31:23)

donaastor · 2022-03-23 20:32:32

seth wrote:

and it's MUCH faster

WOW it is so fast. Thanks for the command. However, I needed ("needed", it's my fancy actually) the list of files not owned by any program so I actually needed to somehow get list of all files on my disk (and their sizes). I used std::filesystem for that and for getting sizes. So, I would not use that hack with desc file, but still use the list from "files" file.

For your questions:
a) when processes are running, when I run the actual program from the package, does it ever touch something that is owned by some (maybe other) package. that's what I meant
b) it doens't relate. That program was the plot of my hysteria

More on the hysteria:
My goal was to more deeply understand how some packages work so that I don't need to have my pc feel dirty when it has 10 GB after months of usage. For some reason I care about knowing every single bit of activity on my pc especially disk usage, otherwise I feel horrible. Hope there is a cure for that Windows 10 related trauma. Windows 10 killed two of my hard drives in the past.

donaastor · 2022-03-23 20:38:43

Trilby wrote:

Wait, does it?

No, it did. I meant to admit that my former definition covered his example. It's not intended to be covered.

Trilby wrote:

pacman -Qkk

Thanks! It's amazing, I wasn't aware of it.

I am happy to hear it doesn't change. I am not running many programs as root. Mostly pacman and some small tools.
I found out that texlive-core likes to do a lot (very little, actually, but it does sometihng) in it's "texmf" folders. I am trying to put those "working folders" as tmpfs.

Trilby · 2022-03-23 20:44:39

donaastor wrote:

I found out that texlive-core likes to do a lot (very little, actually, but it does sometihng) in it's "texmf" folders. I am trying to put those "working folders" as tmpfs.

Are you running *tex binaries as root? If not, they shouldn't be able to modify anything there.

donaastor · 2022-03-23 20:47:08

Trilby wrote:

Are you running *tex binaries as root? If not, they shouldn't be able to modify anything there.

No no, I am not running as root, but it's also not changed. Tex (LyX actually, I guess) just adds more files to those directories. about 80 MB is already there and I didn't even compile a single tex file.
80MB that is not owned, I mean

Last edited by donaastor (2022-03-23 20:48:03)

Trilby · 2022-03-23 21:33:14

Those then are not files owned by packages which is what your question(s) have been all about. If you are really asking whether packaged binaries can create new files while running, yes, they do this all the time.

However, I'm curious which "texmf" folder you are talking about. If it's the one in your home directory, sure. But if it's /var/lib/texmf, that is owned by root and no files could be added to it without root access. Some non-owned files are created there by the post-install scripts (or alpm hooks) of texlive-core/bin packages when executed by pacman (which is running as root), but running texlive programs (or LyX) cannot modify or add to those directories unless you are running them as root too.

Last edited by Trilby (2022-03-23 21:33:40)

donaastor · 2022-03-23 22:51:49

Trilby wrote:

If you are really asking whether packaged binaries can create new files while running...

No; I know that they can. You already understood my question perfetcly. I was just going on to explain why I cared about all that stuff: I saw that that folder contains some owned files and some not owned too. So I thought "If program can modify the owned folders by adding some similar related files, why not modify files themselves". That's why I wanted to know how usual it is. You already answered that: it's not usual.

Specifically about texmf, you are right. I checked and it is owned by root. Then I suppose too that it were some hooks after install that created them. It's not LyX, I was wrong.

Arch Linux

#1 2022-03-23 03:09:23

[SOLVED] Will files owned by packages ever change?

#2 2022-03-23 03:15:05

Re: [SOLVED] Will files owned by packages ever change?

#3 2022-03-23 07:19:56

Re: [SOLVED] Will files owned by packages ever change?

#4 2022-03-23 08:24:56

Re: [SOLVED] Will files owned by packages ever change?

#5 2022-03-23 14:16:25

Re: [SOLVED] Will files owned by packages ever change?

#6 2022-03-23 14:47:13

Re: [SOLVED] Will files owned by packages ever change?

#7 2022-03-23 15:15:00

Re: [SOLVED] Will files owned by packages ever change?

#8 2022-03-23 16:33:02

Re: [SOLVED] Will files owned by packages ever change?

#9 2022-03-23 17:38:59

Re: [SOLVED] Will files owned by packages ever change?

#10 2022-03-23 18:21:39

Re: [SOLVED] Will files owned by packages ever change?

#11 2022-03-23 20:32:32

Re: [SOLVED] Will files owned by packages ever change?

#12 2022-03-23 20:38:43

Re: [SOLVED] Will files owned by packages ever change?

#13 2022-03-23 20:44:39

Re: [SOLVED] Will files owned by packages ever change?

#14 2022-03-23 20:47:08

Re: [SOLVED] Will files owned by packages ever change?

#15 2022-03-23 21:33:14

Re: [SOLVED] Will files owned by packages ever change?

#16 2022-03-23 22:51:49

Re: [SOLVED] Will files owned by packages ever change?

Board footer