pacman -Syu fails (countless No such file & exists in system errors)

wrc1944 · 2014-07-11 16:51:08

This pacman -Syu output run has 28068 lines.
I've excerpted most of it, only showing the important ones to show what happened.
This is just a default 64bit Arch install.

[root@myhost wrc]# pacman -Syu
:: Synchronizing package databases...
 core is up to date
 extra is up to date
 community is up to date
:: Starting full system upgrade...
:: Replace polkit-qt with extra/polkit-qt4? [Y/n] Y
resolving dependencies...
looking for inter-conflicts...

Packages (516):

Name                                               Old Version          New Version          Net Change  Download Size

extra/alsa-lib                                     1.0.27.2-1           1.0.28-1             -0.09 MiB

(SKIPPED 500+ download LINES)

cinnamon-screensaver-2.2.4-1-x86_64                            64.4 KiB   314K/s 00:00 [##################################################] 100%
 cinnamon-2.2.13-2-x86_64                                        2.1 MiB   638K/s 00:03 [##################################################] 100%
 protobuf-2.5.0-4-x86_64                                       661.6 KiB   566K/s 00:01 [##################################################] 100%
 protobuf-c-0.15-3-x86_64                                       96.9 KiB   347K/s 00:00 [##################################################] 100%
(515/515) checking keys in keyring                                                      [##################################################] 100%
(515/515) checking package integrity                                                    [##################################################] 100%
(515/515) loading package files                                                         [##################################################] 100%
(515/515) checking for file conflicts                                                   [##################################################] 100%
error: could not open file /var/lib/pacman/local/alsa-lib-1.0.27.2-1/files: No such file or directory
error: could not open file /var/lib/pacman/local/alsa-plugins-1.0.27-2/files: No such file or directory
error: could not open file /var/lib/pacman/local/alsa-utils-1.0.27.2-1/files: No such file or directory
error: could not open file /var/lib/pacman/local/gcc-libs-4.9.0-3/files: No such file or directory

(SKIPPED coutless No such file LINES)

error: could not open file /var/lib/pacman/local/harfbuzz-icu-0.9.28-1/files: No such file or directory
error: could not open file /var/lib/pacman/local/webkitgtk-2.4.2-1/files: No such file or directory
error: could not open file /var/lib/pacman/local/cinnamon-screensaver-2.2.3-1/files: No such file or directory
error: could not open file /var/lib/pacman/local/cinnamon-2.2.13-1/files: No such file or directory

error: failed to commit transaction (conflicting files) (in red fonts) then it resumes with "exists in filesystem errors.

alsa-lib: /usr/bin/aserver exists in filesystem
alsa-lib: /usr/include/alsa/alisp.h exists in filesystem
alsa-lib: /usr/include/alsa/asound

(SKIPPED countless exists in filesystem LINES)

cinnamon: /usr/share/man/man1/cinnamon-settings.1.gz exists in filesystem
cinnamon: /usr/share/man/man1/cinnamon.1.gz exists in filesystem
cinnamon: /usr/share/man/man1/cinnamon2d.1.gz exists in filesystem
cinnamon: /usr/share/man/man1/gnome-session-cinnamon.1.gz exists in filesystem
cinnamon: /usr/share/man/man1/gnome-session-cinnamon2d.1.gz exists in filesystem
cinnamon: /usr/share/polkit-1/actions/org.cinnamon.settings-users.policy exists in filesystem
cinnamon: /usr/share/xsessions/cinnamon.desktop exists in filesystem
cinnamon: /usr/share/xsessions/cinnamon2d.desktop exists in filesystem
libxft: /usr/lib/libXft.so.2.3.2 exists in filesystem
pango: /usr/lib/libpango-1.0.so.0.3600.5 exists in filesystem
pango: /usr/lib/libpangocairo-1.0.so.0.3600.5 exists in filesystem
pango: /usr/lib/libpangoft2-1.0.so.0.3600.5 exists in filesystem
pango: /usr/lib/libpangoxft-1.0.so.0.3600.5 exists in filesystem
Errors occurred, no packages were upgraded.
[root@myhost wrc]#

If I try and re-run pacman -Syu, nothing happens, but none of the 515 packages were updated???

[root@myhost wrc]# pacman -Syu
:: Synchronizing package databases...
 core is up to date
 extra is up to date
 community is up to date
:: Starting full system upgrade...
 there is nothing to do
[root@myhost wrc]#

This must be something simple to fix, but I've looked around for 3 hours, and so far can't find any clue.

Any ideas? What am I missing? I suspect corrupt pacman, or db or symlink problem, but really don't know.

Last edited by wrc1944 (2014-07-11 17:51:21)

Trilby · 2014-07-11 16:59:32

Please edit your post to use code tags.

My first suspicion on the actual problem may be an issue with the writing to /var. Is this on a separate partition? Is the partition full? Have you run a recent fsck and/or SMART check on the drive?

EDIT: I do suspect that the more proximate cause may be some sort of corruption of pacman database(s), but I think figuring out what lead to that condition (e.g. i/o issues on /var) would be the best starting point.

Last edited by Trilby (2014-07-11 17:01:47)

wrc1944 · 2014-07-11 18:14:56

Thanks Trilby,
Sorry about no code tags- don't know how i missed that.

This Arch install is completely on the sda9 partition (including grub, and of course /var), which I chainload into from my main Gentoo grub.
sda9 is 80% full, but even 95% full never caused huge pacman update to fail over many years. The disk also contains several other distros, all working perfectly on their own partitions (including three gentoos, both ~x86 and ~amd64). Yesterday, while updating my gentoo installs IIRC an fsck was run on sda9 during booting. The disk really is only 1 year old, and apparently in perfect health.

Arch was updated and checked about 2-3 weeks ago, no problems, pacman working perfectly, and no runtime problems after those updates. I haven't booted Arch since then, until today. All was going perfectly on the -Syu, until the final checking for file conflicts bombed out (I didn't see it until later, when I returned to the terminal).

Would it be risky to try a reboot? AFAIK, pacman didn't change any packages on the attempted -Syu.

BTW, all that's in /var/db is a makefile. Isn't there supposed to also be an actual db?

EDIT: The pacman.log apparently says all 500+ packages WERE upgraded, including (as an example of what's in the log:

 [2014-07-11 11:47] [PACMAN] upgraded linux-firmware (20140316.dec41bc-1 -> 20140603.a4f3bc0-1)
[2014-07-11 11:47] [ALPM-SCRIPTLET] >>> Updating module dependencies. Please wait ...
[2014-07-11 11:47] [ALPM-SCRIPTLET] >>> Generating initial ramdisk, using mkinitcpio.  Please wait...
[2014-07-11 11:47] [ALPM-SCRIPTLET] ==> Building image from preset: /etc/mkinitcpio.d/linux.preset: 'default'
[2014-07-11 11:47] [ALPM-SCRIPTLET]   -> -k /boot/vmlinuz-linux -c /etc/mkinitcpio.conf -g /boot/initramfs-linux.img
[2014-07-11 11:47] [ALPM-SCRIPTLET] ==> Starting build: 3.15.5-1-ARCH
[2014-07-11 11:47] [ALPM-SCRIPTLET]   -> Running build hook: [base]
[2014-07-11 11:47] [ALPM-SCRIPTLET]   -> Running build hook: [udev]
[2014-07-11 11:47] [ALPM-SCRIPTLET]   -> Running build hook: [autodetect]
[2014-07-11 11:47] [ALPM-SCRIPTLET]   -> Running build hook: [modconf]
[2014-07-11 11:47] [ALPM-SCRIPTLET]   -> Running build hook: [block]
[2014-07-11 11:47] [ALPM-SCRIPTLET]   -> Running build hook: [filesystems]
[2014-07-11 11:47] [ALPM-SCRIPTLET]   -> Running build hook: [keyboard]
[2014-07-11 11:47] [ALPM-SCRIPTLET]   -> Running build hook: [fsck]
[2014-07-11 11:47] [ALPM-SCRIPTLET] ==> Generating module dependencies
[2014-07-11 11:47] [ALPM-SCRIPTLET] ==> Creating gzip initcpio image: /boot/initramfs-linux.img
[2014-07-11 11:47] [ALPM-SCRIPTLET] ==> Image generation successful
[2014-07-11 11:47] [ALPM-SCRIPTLET] ==> Building image from preset: /etc/mkinitcpio.d/linux.preset: 'fallback'
[2014-07-11 11:47] [ALPM-SCRIPTLET]   -> -k /boot/vmlinuz-linux -c /etc/mkinitcpio.conf -g /boot/initramfs-linux-fallback.img -S autodetect
[2014-07-11 11:47] [ALPM-SCRIPTLET] ==> Starting build: 3.15.5-1-ARCH
[2014-07-11 11:47] [ALPM-SCRIPTLET]   -> Running build hook: [base]
[2014-07-11 11:47] [ALPM-SCRIPTLET]   -> Running build hook: [udev]
[2014-07-11 11:47] [ALPM-SCRIPTLET]   -> Running build hook: [modconf]
[2014-07-11 11:47] [ALPM-SCRIPTLET]   -> Running build hook: [block]
[2014-07-11 11:47] [ALPM-SCRIPTLET] ==> WARNING: Possibly missing firmware for module: aic94xx
[2014-07-11 11:47] [ALPM-SCRIPTLET] ==> WARNING: Possibly missing firmware for module: smsmdtv
[2014-07-11 11:47] [ALPM-SCRIPTLET]   -> Running build hook: [filesystems]
[2014-07-11 11:47] [ALPM-SCRIPTLET]   -> Running build hook: [keyboard]
[2014-07-11 11:47] [ALPM-SCRIPTLET]   -> Running build hook: [fsck]
[2014-07-11 11:47] [ALPM-SCRIPTLET] ==> Generating module dependencies
[2014-07-11 11:47] [ALPM-SCRIPTLET] ==> Creating gzip initcpio image: /boot/initramfs-linux-fallback.img
[2014-07-11 11:47] [ALPM-SCRIPTLET] ==> Image generation successful
[2014-07-11 11:47] [PACMAN] upgraded linux (3.14.4-1 -> 3.15.5-1)
[2014-07-11 11:47] [PACMAN] upgraded linux-docs (3.14.4-1 -> 3.15.5-1)
[2014-07-11 11:47] [PACMAN] upgraded linux-headers (3.14.4-1 -> 3.15.5-1)

This seems to conflict with the error message at the end of the pacman -Syu output:

 cinnamon: /usr/share/xsessions/cinnamon.desktop exists in filesystem
cinnamon: /usr/share/xsessions/cinnamon2d.desktop exists in filesystem
libxft: /usr/lib/libXft.so.2.3.2 exists in filesystem
pango: /usr/lib/libpango-1.0.so.0.3600.5 exists in filesystem
pango: /usr/lib/libpangocairo-1.0.so.0.3600.5 exists in filesystem
pango: /usr/lib/libpangoft2-1.0.so.0.3600.5 exists in filesystem
pango: /usr/lib/libpangoxft-1.0.so.0.3600.5 exists in filesystem
Errors occurred, no packages were upgraded.
[root@myhost wrc]#

I still don't get what's going on here. /boot/vmlinuz-linux says it was modified yesterday (when I hadn't booted Arch in 2+ weeks, and the fallback image is modified today exactly when I ran pacman -Syu. Therefore, vmlinuz-linux must be the new 3.1.5 kernel ( I assume added to the Arch repo yesterday), confirming that pacman -Syu actually did all the ugrades regardless of the "no packages were upgraded" message? These conflicting messages and facts are very confusing. Anyway, it's convincing me a reboot at this point might not be as risky as i thought. Please advise.

Last edited by wrc1944 (2014-07-11 19:42:50)

Trilby · 2014-07-11 19:08:24

My /var/db has just a Makefile and a folder for sudo - so it sounds like your case is normal. The pacman dbs are elsewhere ( /var/{cache,lib}/pacman ). Also it sounds like disk I/O issues can be ruled out.

I agree that it looks like no changes were committed and so a reboot *should* be safe. But the second -Syu saying everything is up to date calls for more caution.

I'd check the ouput from `pacman -Qk` and maybe `-Qkk` first. The first should show lines all ending in " 0 missing files" if everything is in order.

wrc1944 · 2014-07-11 19:12:32

I'm now concluding all the changes WERE made, as /lib/modules does contain the new 3.15.5 modules directory, modified today at 11:47AM. Maybe you posted before reading my edits above to the last post.

wrc1944 · 2014-07-11 19:17:35

pacman -Qk shows 0 missing files on everything.

pacman -Qkk shows Modification time mismatch on maybe 50-60 packages, and the others say 0 altered files.

That time mismatch thing made me think. At the same time i was downloading the package updates for pacman -Syu, I also happen to decide to setup ntp service, as my kde clock was always going wrong, and i decided to just use ntpd, as I do on all other distros. Maybe a foolish mistake that could have caused this problem? I thought I would easily finish setting ntp up before pacman actually started the update, but maybe pacman already had the time wrong because it "logged itself" to the wrong time my clock showed before I made it correct with ntpd? .

Last edited by wrc1944 (2014-07-11 19:50:00)

Trilby · 2014-07-11 19:23:24

Yes, I was posting as you were editing. This is seeming more and more odd - but I'd err on the side of trusting pacman's logs. Also that -Qk(k) output is normal. -Qkk will show mod-time mismatch on a number of packages as these are config files that have been modified after installation.

-- edit: what was noted here as speculation does not pan out - see subsequent posts by apg --

Completely speculatiively, it seem like the "files" files didn't survive the download, but everything else did. This would explain the first erros in pacman's output. Then the absence of this data *might* trigger the conflicting files messages even though everything else was working as intended. But perhaps Allan or pacman devs could have better insight.

Last edited by Trilby (2014-07-12 15:16:01)

wrc1944 · 2014-07-11 19:30:08

Trilby,
Please check my new theory posted above.

Trilby · 2014-07-11 20:27:42

That's at least as reasonable as my speculation - but I just don't know the pacman internals. A quick greping through some of the code does indicate that file mod/access times are used in many of these checks.

That, however, would not have anything to do with the mtime mismatches in the `pacman -Qkk` output. Such mismatches are expected on many files.

apg · 2014-07-11 20:38:06

When pacman encounters file conflicts like you described it aborts immediately and will not commit anything. The only explanation I can think of that makes any sense is that you had two instances of pacman running system upgrades at the same time, one of which successfully upgraded and the other aborted.

wrc1944 · 2014-07-11 20:49:39

Well, I decided to reboot, and it did so normally, and got the new kernel. Ran pacman -Syu to check, and got this:

Linux myhost 3.15.5-1-ARCH #1 SMP PREEMPT Thu Jul 10 07:08:50 CEST 2014 x86_64 GNU/Linux
[wrc@myhost ~]$ su
Password: 
[root@myhost wrc]# pacman -Syu
:: Synchronizing package databases...
 core                                                          111.1 KiB   387K/s 00:00 [##################################################] 100%
 extra                                                        1743.1 KiB   633K/s 00:03 [##################################################] 100%
 community                                                       2.3 MiB   640K/s 00:04 [##################################################] 100%
:: Starting full system upgrade...
resolving dependencies...
looking for inter-conflicts...

Packages (10):

Name                     Old Version  New Version  Net Change  Download Size

core/dbus                1.8.4-1      1.8.6-1      0.00 MiB         0.39 MiB
core/filesystem          2014.06-2    2014.07-1    0.00 MiB         0.01 MiB
extra/harfbuzz           0.9.29-1     0.9.30-1     0.06 MiB         0.23 MiB
extra/harfbuzz-icu       0.9.29-1     0.9.30-1     0.00 MiB         0.01 MiB
core/libdbus             1.8.4-1      1.8.6-1      0.00 MiB         0.13 MiB
core/libsystemd          214-2        215-4        -0.06 MiB        0.11 MiB
extra/libtracker-sparql  1.0.1-2      1.0.2-1      0.00 MiB         0.27 MiB
extra/libxi              1.7.2-1      1.7.3-1      0.06 MiB         0.14 MiB
core/systemd             214-2        215-4        0.03 MiB         3.56 MiB
core/systemd-sysvcompat  214-2        215-4        0.00 MiB         0.01 MiB

Total Download Size:    4.85 MiB
Total Installed Size:   24.61 MiB
Net Upgrade Size:       0.09 MiB

:: Proceed with installation? [Y/n]

I ran the updates, all went well, plus my kde clock automatically used ntpd at boot, as expected- all OK.

My conclusion is that since all seems well now, the entire problem was apparently caused by the concurrent setting of ntp while pacman was running updates. In other words, a self-inflicted and foolish mistake.
I've gotten into the habit on Gentoo of concurrently running an emerge "fetch only" in one terminal, while also starting a full emerge @world or any number of other things in other terminals. I do this in order to be compilng some packages while other packages are still downloading (have slow internet connection).

As long as the fetch packages process in the first terminal keeps ahead of the actual emerge @world packages being installed, all goes well. But even if the emerge of a package starts before it's actually fetched to the cache, emerge doesn't bail out- it either waits on the other terminal, or starts it's own fetch process in its own terminal as if there were no other independent fetch already running. Emerge has the ability to handle such circumstances without getting confused.

Anyway, not being as familiar with pacman as I am emerge, I didn't consider that another concurrent process (such as seting up ntp) might cause pacman to react in an unexpected manner. At least this is my current theory of where I went wrong. Acually, now that I think about it changing the system clock while also emerging might throw off emerge, too.

Thanks much to Trilby for cluing me in the the pacman -Qk and pacman -Qkk commands, which exposed the time mismatch, which led me to realize what most likely happened with the concurrent ntp setup.

Sorry if I wasted everyones time.

wrc1944 · 2014-07-11 21:07:04

Pacman Developer wrote:

The only explanation I can think of that makes any sense is that you had two instances of pacman running system upgrades at the same time, one of which successfully upgraded and the other aborted.

That makes at least as much sense as my theory. I did have a few terminals open when I was doing pacman and systemctl stuff at the same time.

In gentoo, I usually have three emerge terminals open, one is an emerge -upvDN @world to generate a verbose list with USE flags, etc. to refer back to, the 2nd is the fetch only terminal, and the third is my actual emerge -uDNv @world --keep-going terminal where all the gcc output is shown, which I start after the fetch terminal has made enough progress where the third terminal can't catch and pass it.

Anyhow, I do this in gentoo without thinking about it, so I could have easily opened two pacman terminals, not realizing it might abort one of them under certain circumstances. In fact, I think I did open two pacman terminals. The original, with the bail out message, which I kept for reference the list and output, and then a 2nd one where at this point I'm not sure what I did next.

apg · 2014-07-11 21:27:45

Adjusting the time while pacman is running will not cause the errors you described.

Trilby · 2014-07-11 21:35:13

wrc1944 wrote:

Thanks much to Trilby for cluing me in the the pacman -Qk and pacman -Qkk commands, which exposed the time mismatch, which led me to realize what most likely happened with the concurrent ntp setup.

I'm glad the end result was success - but it is not as you describe:

Trilby wrote:

That, however, would not have anything to do with the mtime mismatches in the `pacman -Qkk` output. Such mismatches are expected on many files.

wrc1944 · 2014-07-12 12:48:21

Hmmmmm... So my time mismatch theory is out the window. I guess that leaves either multiple instances of pacman running and one aborting, and/or some files not surviving the download as the likely cause of what happened. Should I conclude from all this that it's a good policy never to be running pacman in more than one terminal at the same time?

On the "files not surviving the download" aspect, wouldn't pacman catch something like that before or during the pre-install checks (checking package integrity, loading package files, checking for file conflicts)?

Trilby · 2014-07-12 13:00:34

wrc1944 wrote:

Should I conclude from all this that it's a good policy never to be running pacman in more than one terminal at the same time?

That is a good conclusion to hold. Whether the content of this thread actually leads to that conclusion is another question.

Under normal conditions it should not be possible to run pacman in more than one terminal at the same time (at least not for any operations that modify the database). When pacman is running anything that might modify the database, a lock file is created, and other pacman commands that would modify the database will fail due to the existence of the lock file.

apg · 2014-07-12 14:36:01

wrc1944 wrote:

Hmmmmm... So my time mismatch theory is out the window. I guess that leaves either multiple instances of pacman running and one aborting, and/or some files not surviving the download as the likely cause of what happened. Should I conclude from all this that it's a good policy never to be running pacman in more than one terminal at the same time?
On the "files not surviving the download" aspect, wouldn't pacman catch something like that before or during the pre-install checks (checking package integrity, loading package files, checking for file conflicts)?

There is nothing to fail to download. The package file lists that pacman complained about being missing are created by pacman, not downloaded. If something had corrupted your database and removed those files pacman would have refused to update those packages due to the filesystem conflicts. Concurrent sysupgrades, which as Trillby points out should be impossible without manually removing pacman's lock file, explain your symptoms perfectly. Did you remove the file '/var/lib/pacman/db.lck'? If not, you should be trying to determine why pacman failed to lock the database.

wrc1944 · 2014-07-12 16:50:42

I think we're getting close to what happened. Now that you mention it, I do recall removing ( IIRC in a response to a prompt) a lockfile during the time I was trying to sort out the problem.

It might have been /var/lib/pacman/db.lck in a failed attempt to re-run pacman -Syu, but to be accurate I can't be absolutely sure it was the db.lck.

Thanks very much for the continued attention and clarifcations. It's been very helpful in increasing my working knowledge of pacman.

Update:
After some more thought, I'm pretty much satisfied I made some sort of mistake, and pacman and/or any Arch updates really had nothing to do with what happen. Since it seems to be resolved, and I can't think of any other point that hasn't been discussed, should I mark this as as "solved"?

Last edited by wrc1944 (2014-07-13 05:00:43)

Arch Linux

#1 2014-07-11 16:51:08

pacman -Syu fails (countless No such file & exists in system errors)

#2 2014-07-11 16:59:32

Re: pacman -Syu fails (countless No such file & exists in system errors)

#3 2014-07-11 18:14:56

Re: pacman -Syu fails (countless No such file & exists in system errors)

#4 2014-07-11 19:08:24

Re: pacman -Syu fails (countless No such file & exists in system errors)

#5 2014-07-11 19:12:32

Re: pacman -Syu fails (countless No such file & exists in system errors)

#6 2014-07-11 19:17:35

Re: pacman -Syu fails (countless No such file & exists in system errors)

#7 2014-07-11 19:23:24

Re: pacman -Syu fails (countless No such file & exists in system errors)

#8 2014-07-11 19:30:08

Re: pacman -Syu fails (countless No such file & exists in system errors)

#9 2014-07-11 20:27:42

Re: pacman -Syu fails (countless No such file & exists in system errors)

#10 2014-07-11 20:38:06

Re: pacman -Syu fails (countless No such file & exists in system errors)

#11 2014-07-11 20:49:39

Re: pacman -Syu fails (countless No such file & exists in system errors)

#12 2014-07-11 21:07:04

Re: pacman -Syu fails (countless No such file & exists in system errors)

#13 2014-07-11 21:27:45

Re: pacman -Syu fails (countless No such file & exists in system errors)

#14 2014-07-11 21:35:13

Re: pacman -Syu fails (countless No such file & exists in system errors)

#15 2014-07-12 12:48:21

Re: pacman -Syu fails (countless No such file & exists in system errors)

#16 2014-07-12 13:00:34

Re: pacman -Syu fails (countless No such file & exists in system errors)

#17 2014-07-12 14:36:01

Re: pacman -Syu fails (countless No such file & exists in system errors)

#18 2014-07-12 16:50:42

Re: pacman -Syu fails (countless No such file & exists in system errors)

Board footer