Why does pacman lock the database during downloads?

cesura · 2012-03-24 00:34:25

Something that's always irked me about pacman is the fact that it locks the package database during the download stages of package installation, before anything has actually been installed/committed. Is there a reason for this, and if not, would a suggestion/patch be acceptable?

Here's the scenario I'm encountering right now:
* Downloading large package in background
* Need to upgrade flash player (YouTube no longer accepts an older version I suppose, and lolcats are a necessity)
* I cannot do this quick upgrade due to the lock pacman has on the database while downloading said large package

Last edited by cesura (2012-03-24 00:36:31)

krigun · 2012-03-24 00:57:40

Sounds reasonable, but the following scenario may or may not be a problem:

You have package A B and C installed:

Run pacman -Syu (huge download, all three packages updated)
Simultaneously, you execute pacman -R A (basically modify the pacman DB)
Package A updated (reinstalled) by first process

Not sure how pacman is built, but it probably locks and reads the DB when started, and then does it's thing without worries. In any case, it does support resuming downloads, so you could just end the long running process, install whatever, and resume.

I think it's the same with apt-get.

Last edited by krigun (2012-03-24 01:00:45)

hadrons123 · 2012-03-24 01:32:34

What is that large package which you think is that big?

I think locking is present in all package managers in all distributions and it is a good Idea, so that someone can't install and uninstall a package at different terminals at the same time, and end up with a messed up package manager .

cesura · 2012-03-24 02:57:48

krigun wrote:

Sounds reasonable, but the following scenario may or may not be a problem:
You have package A B and C installed:
Run pacman -Syu (huge download, all three packages updated)
Simultaneously, you execute pacman -R A (basically modify the pacman DB)
Package A updated (reinstalled) by first process
Not sure how pacman is built, but it probably locks and reads the DB when started, and then does it's thing without worries. In any case, it does support resuming downloads, so you could just end the long running process, install whatever, and resume.
I think it's the same with apt-get.

That's an interesting point, although I'm guessing the package manager could check to make sure that all packages that it wants to update are present on the system right after it locks the db, and if for some (rare) reason that one is missing, skip it.

hadrons123 wrote:

What is that large package which you think is that big?
I think locking is present in all package managers in all distributions and it is a good Idea, so that someone can't install and uninstall a package at different terminals at the same time, and end up with a messed up package manager .

Well in this case, it is eclipse (>200 MB), but there are a variety of other large packages in the repositories. What may be small to you may seem massive to those with slower connections.

How could it become messed up? I'm not proposing the database lock be eliminated, but rather wait until the actual installation process begins before locking it. It's no skin off anybody's ass if someone simply downloads packages simultaneously.

karol · 2012-03-24 03:01:15

https://bugs.archlinux.org/task/24036

cesura · 2012-03-24 03:47:23

Thanks karol, +10 internets for you!

So I take it from that feature request that the idea isn't currently on the table? I don't know if I can be arsed to write patches for such a thing, I seem to be in more of a complaining mood right now.

R00KIE · 2012-03-24 18:40:22

hadrons123 wrote:

What is that large package which you think is that big?
I think locking is present in all package managers in all distributions and it is a good Idea, so that someone can't install and uninstall a package at different terminals at the same time, and end up with a messed up package manager .

Big enough with a connection slow enough that you have time to open another console and issue another pacman -Rs, the update might fail on the first pacman due to missing dependencies (at best, worst case can grant the user with a few hours of recovery from an inconsistent state) .

I think it is reasonable enough that when one instance of pacman has write access to the package database files that another instance of pacman can't do any changes. I certainly don't want my package database borked, this is one of the cases where all safety checks are welcome (as long as there is some way to override then in case something has gone wrong previously).

I know arch is a bleeding edge distro, and I don't mind that, but what isn't nice is the package manager screwing things up because of something like this.

hokasch · 2012-03-24 18:55:15

If you kill the downloading pacman with ctrl+c, the aborted download is resumed when downloading the same package again (at least here).

cesura · 2012-03-24 19:20:12

If I'm not mistaken, yum does (used to?) not lock the database until after the packages had been downloaded (correct me if I'm wrong). I do know, however, that it waits and polls for the lock to be released, rather than exit out entirely (like pacman). It's really not that difficult of a process to implement:

1) Packages A and D depend on package B, and packages A, B, C, and D are installed.
2) User begins update, and updates for packages A, B, and C are being downloaded.
3) User opens another terminal and removes package A, keeping its dependency intact (because D depends on B as well).
4) Updates are finished downloading, and pacman takes the database lock.
5) It sees that package A, which it was set to update, has been removed. It is assumed, then, that the user removed it in that time span, and it is ignored.

hokasch wrote:

If you kill the downloading pacman with ctrl+c, the aborted download is resumed when downloading the same package again (at least here).

As well as that would work in most cases, it's a workaround, not a solution. The onus should be on the package manager to correctly handle situations like this.

Last edited by cesura (2012-03-24 19:21:40)

ngoonee · 2012-03-26 01:58:33

I'm not sure why package database files are an issue at all, this has been discussed before (1 1/2 years ago maybe) and the conclusion was simply that it would be inefficient. Right now, what pacman does (if I recall correctly) is:-

1. sync mirrors
2. check what packages are needed
3. download them
4. ask for confirmation etc. and then install

The database is locked after checking what is needed. If (for example) the same check is done AFTER the download is completed, then the database does not need to be locked during (3) (meaning (2) is replicated again in between (3) and (4)). This would be doing the same work twice though, just for the benefit of maybe being able to do some other stuff to the package database during downloading.

Best way would be for that sort of behaviour to be opt-in. And, of course, patches welcome to those who want it (bauerbill used to do things that way, which I loved on my slow connection, but I can live without it).

Arch Linux

#1 2012-03-24 00:34:25

Why does pacman lock the database during downloads?

#2 2012-03-24 00:57:40

Re: Why does pacman lock the database during downloads?

#3 2012-03-24 01:32:34

Re: Why does pacman lock the database during downloads?

#4 2012-03-24 02:57:48

Re: Why does pacman lock the database during downloads?

#5 2012-03-24 03:01:15

Re: Why does pacman lock the database during downloads?

#6 2012-03-24 03:47:23

Re: Why does pacman lock the database during downloads?

#7 2012-03-24 18:40:22

Re: Why does pacman lock the database during downloads?

#8 2012-03-24 18:55:15

Re: Why does pacman lock the database during downloads?

#9 2012-03-24 19:20:12

Re: Why does pacman lock the database during downloads?

#10 2012-03-26 01:58:33

Re: Why does pacman lock the database during downloads?

Board footer