Search for installed but corrupted packages?

domanov · 2008-10-23 14:59:49

Hi all,

is there a way to automatically search through the entire system for corrupted packages (among the installed ones)? I mean: can we ask pacman to check if every file belonging to every installed package is actually there and tell something if not?

I run into some issues while upgrading (-Syu) and running out of disk space during the process; pacman interrupted. I've then *carefully* readen both pacman ouput and the pacman.log, and manually upgraded the "errored" packages after having freed enough disk space.
Still, I was left with *many* corrupted packages, resulting in random errors in several programs. I had no clue how to find the corrupted packages.... and find /var/lib/pacman/local/ -name files -size 0 didn't help a lot. That's why I came up with something like this poor man's check (very poor):

#!/bin/bash

## If run with arguments, those are treated as packages to check; 
## if no argument is given, check the entire database.
## Fails incorrectly for file names with spaces in them (e.g. in skype)

if [ $# -eq 0 ] ; then 
    LIST=$(pacman -Q | awk '{print $1}')
else
    LIST=$@
fi

LOG="checkpack-$(date +%Y-%m-%d-%H_%M).log"
echo "Output in file $LOG"

for PACK in $LIST; do 
    for FILE in $(pacman -Ql "$PACK" | awk '{print $2}') ; do
    if [ ! -e "$FILE" ] && [ ! -h "$FILE" ] ; then  
            # FILE is not a file nor a symb. link
        echo "$PACK $FILE" >> $LOG
    fi
    done
done

NUM=$(wc -l $LOG | awk '{print $1}')

if [ "$NUM" -ne 0 ]; then
    NPACKS=$(awk '{print $1}' $LOG | uniq | wc -l)
    echo "Missing $NUM files in $NPACKS packets"
    exit 10
fi

echo "Everything seems fine, bye."
exit 0

This allows to check for missing files, of course not for corrupted ones. For me it did the trick, by the way . Is there a way to check for corruption, or to do this elegantly in pacman?

domanov

Last edited by domanov (2008-10-23 15:28:49)

bender02 · 2008-10-23 17:48:57

There's no way of checking for corrupted files, since there's nothing to compare them to (nor their checksums), except for the contents of the packages themselves. What you can do though is
1) clean up your pacman cache (pacman -Sc) (so that you don't do unnecessary work in 2)
2) reinstall every package in the cache with pacman -U (the default location for the cache is /var/cache/pacman/pkg)

EDIT: typo

Last edited by bender02 (2008-10-23 17:49:29)

domanov · 2008-10-23 18:15:28

bender02 wrote:

There's no way of checking for corrupted files, since there's nothing to compare them to (nor their checksums), except for the contents of the packages themselves.

That's the point. I'm trying to think of a script, wrapped around pacman, which automatically makes the checksum of every installed file (yes, quite a big and boring thing) based on the /var/lib/pacman/local/$PACKAGE/files entries and stores them in some way in order to be able to eventually perform a check (after a file system failure, for example). Or at least, a general checksum for every package, if it's possible. I have to think of a way to automatically skip configuration files (skipping /etc would be quick and dirty), but I think it's quite simple to make something like that.

Could be anyone interested to make a pacman feature request for that? I mean: option to store files checksums right after the installation/upgrade of a package and option of checking the installed files against those checksums. I don't know if it was often discussed about that...

bender02 wrote:

1) clean up your pacman cache (pacman -Sc) (so that you don't do unnecessary work in 2)
2) reinstall every package in the cache with pacman -U (the default location for the cache is /var/cache/pacman/pkg)
EDIT: typo

I don't have local cache, cause I always run -Scc just after installations, to keep free disk space.

buddabrod · 2008-10-23 19:04:35

Isnt that something gentoo does for quite a very long time?

I think It'd be useful to create checksums of *every* single file pacman installes. fs corruption still does occur sometimes, so this would be great.

bender02 · 2008-10-23 19:34:22

buddabrod wrote:

Isnt that something gentoo does for quite a very long time?
I think It'd be useful to create checksums of *every* single file pacman installes. fs corruption still does occur sometimes, so this would be great.

I don't find it useful. What seems useful to me is to backup - either the whole thing, or at least /etc and the list of installed packages. If fs becomes corrupted, chances are that also files that don't come directly from packages (like config files) are corrupted as well. [Actually, by Murphy's law, these files have much higher chance of getting corrupted ] Hence backup.

Xyne · 2008-10-23 20:23:58

I was still in a coding mood when I saw this thread, so I wrote this:

#!/usr/bin/perl
use warnings;
use strict;

use IPC::Open2;

my ($pipe_in, $pipe_out);
my $pid = open2($pipe_out, $pipe_in, 'bash') or die "Unable to open bash: $!\n";

my $pkgs = "@ARGV";
my $file_list = `pacman -Ql $pkgs`;
my %pkg_to_file = ();
foreach my $line (split "\n",$file_list)
{
    my ($pkg,$path) = $line =~ m/(\S+)\s+(.+)/;
    if (-f $path)
    {
        if (exists($pkg_to_file{$pkg}))
        {
            push @{$pkg_to_file{$pkg}}, $path;
        }
        else
        {
            $pkg_to_file{$pkg} = [$path];
        }
    }
    else
    {
        next;
    }
}
foreach my $pkg (keys %pkg_to_file)
{
    my $files = join(' ', @{$pkg_to_file{$pkg}});
    my $md5sums = &exec_cmd("md5sum $files");
    foreach my $line (split "\n", $md5sums)
    {
        my ($md5sum,$path) = $line =~ m/(\S+)\s+(.+)/;
        print "$pkg\t$path\t$md5sum\n";
    }
}

close $pipe_in;
waitpid ($pid,0) if defined($pid);

sub exec_cmd
{
    my $end = 'END_OF_COMMAND_'.time;
    my $out = '';
    print $pipe_in "@_\necho '$end'\n";
    while (defined(my $line = <$pipe_out>))
    {
        if (substr($line,0,-1) eq $end)
        {
            last;
        }
        else
        {
            $out .= $line;
        }
    }
    return $out;
}

Pass that package names as arguments and it will print out a list with the following format:

<pkg_name>     <file_path>     <md5sum>

Ideas of what you can do with it:
create a verified md5sum log and use it for later reference (maybe invoke it with a pacman wrapper for all installations)
write a script that reuses the code or parses its output to compare installed files against the log or against a downloaded package tarball

zhuqin · 2008-10-23 22:51:24

There's one pkg called tupac may meet your need.

$ tupac -h
tupac: A cached pacman implementatioin. Version: 0.5.3.2

 Usage:
  tupac [word] [word] [word] ...     : Search for and install packages that match all [word]
  tupac -Ss [word] [word] [word] ... : Search for packages that match all [word]
  tupac -Qo [file] [file] [file] ... : Search for each [file] owner
  tupac --checkdir [directory]       : Check integrity of a directory.
  tupac --orphans [directory]        : Find files that are not part of any package
  tupac                              : Just update cache
  tupac [anything else]              : bypass to yaourt
  tupac --set-proxy [host:port|none] : set up a proxy

 Modifiers:
  --safe                             : Only search for safe packages
  --noaur                            : Don't search in AUR
  --noprompt                         : Don't prompt anything
  --color [darkbg|lightbg|nocolor]   : Choose color scheme
  --repos repo1,repo2,repo3,...      : Set active repositories
  --lang [xx_XX|machine]             : Set working language

Arch Linux

#1 2008-10-23 14:59:49

Search for installed but corrupted packages?

#2 2008-10-23 17:48:57

Re: Search for installed but corrupted packages?

#3 2008-10-23 18:15:28

Re: Search for installed but corrupted packages?

#4 2008-10-23 19:04:35

Re: Search for installed but corrupted packages?

#5 2008-10-23 19:34:22

Re: Search for installed but corrupted packages?

#6 2008-10-23 20:23:58

Re: Search for installed but corrupted packages?

#7 2008-10-23 22:51:24

Re: Search for installed but corrupted packages?

Board footer