paccheck - pacman package authenticity check

IgnorantGuru · 2011-02-21 15:18:41

I have written a script I can use to compare my pacman sync with multiple mirrors, to help reduce the risk of downloading tampered packages. I have made paccheck available for download.

paccheck downloads Arch Linux pacman sync database files from multiple mirrors and compares them against pacman's sync and package cache in an attempt to detect a compromised mirror.

paccheck helps to minimize the risk of inadvertently using a compromised mirror by comparing mirrors you choose against your pacman sync. It first runs sudo pacman to sync and download packages due for update on your system (but installs nothing). It then downloads database sync files from all the mirrors you select. It compares these files to your pacman sync files. If they don't match, it scans your package cache and compares the desc file of each package against the mirror. If they don't match, it reports this to you. (desc files contain the MD5 sum of the package, used by pacman for archive integrity checking, so tampering with a package on a mirror would also require changing its desc file.) In this way, paccheck is able to determine to some degree whether a problem is caused by an out-of-sync mirror or a compromised mirror.

Note that aside from triggering a pacman sync and package download, paccheck does not modify your system files or correct problems. It merely alerts you in plain english to what it finds. So it is imporant for you to read its report.

paccheck merely compares mirrors using your network connection. If your network connection or other system components are compromised, paccheck has no way of detecting this or other vulnerabilities. However, this polling method will somewhat improve your update security. If Arch adds signatures to the mirrors, I will consider updating paccheck to check the signatures, which will be a much more comprehensive solution.

Hopefully you will never see a report of a MISMATCH on a package, but if you do, do not proceed with pacman's update until you have determined which mirror is inaccurate.

paccheck --help

Compares Arch Linux pacman sync and package cache to multiple mirrors to help
detect compromised mirrors

Usage: paccheck

pacman Update Procedure:
   1) Run paccheck and examine report (do not proceed if MISMATCH occurs)
   2) Run sudo pacman -Su

Desired mirrors may be configured inside this script or 
saved to /etc/paccheck/mirrorlist

http://igurublog.wordpress.com/download … -paccheck/
http://aur.archlinux.org/packages.php?ID=46763

MOD-EDIT by ngoonee - While I appreciate you're trying to contribute, please avoid needless commentary disrespecting Arch Linux and its developers. I have removed some of the baiting-type comments.

Last edited by ngoonee (2011-02-22 06:42:27)

toad · 2011-02-21 17:16:49

Brilliant idea to get around that old caper of package signing. Certainly a nice way forward.

toad@archtop 515\16 ~ > paccheck
pacman sync...
>>> sudo pacman -Sy
:: Synchronising package databases...
 core is up to date
 extra                                                                    464.8K   43.6K/s 00:00:11 [----------------------------------------------------------] 100%
 community is up to date
 archlinuxfr                                                               24.5K   48.1K/s 00:00:01 [----------------------------------------------------------] 100%
 multilib is up to date
 arch-games is up to date
Download needed packages without installing...
>>> sudo pacman -w --noconfirm -Su
:: Starting full system upgrade...
 there is nothing to do

Timestamps:
/var/lib/pacman/core.db.tar.gz: 2011-02-20 22:41:33.000000000 +0100
/var/lib/pacman/extra.db.tar.gz: 2011-02-21 16:31:53.000000000 +0100
/var/lib/pacman/community.db.tar.gz: 2011-02-20 22:36:27.000000000 +0100

========== DOWNLOADING ============
2011-02-21 18:11:01 URL: ftp://ftp.archlinux.org/core/os/x86_64/core.db.tar.gz [27450] -> ".listing" [1]
2011-02-21 18:11:03 URL: ftp://ftp.archlinux.org/core/os/x86_64/core.db.tar.gz [37650] -> "core.db.tar.gz" [1]
2011-02-21 18:11:15 URL: ftp://ftp.archlinux.org/extra/os/x86_64/extra.db.tar.gz [392045] -> ".listing" [1]
2011-02-21 18:11:29 URL: ftp://ftp.archlinux.org/extra/os/x86_64/extra.db.tar.gz [475981] -> "extra.db.tar.gz" [1]
2011-02-21 18:11:39 URL: ftp://ftp.archlinux.org/community/os/x86_64/community.db.tar.gz [299376] -> ".listing" [1]
2011-02-21 18:11:52 URL: ftp://ftp.archlinux.org/community/os/x86_64/community.db.tar.gz [434826] -> "community.db.tar.gz" [1]
2011-02-21 18:11:54 URL:http://ftp.jaist.ac.jp/pub/Linux/ArchLinux/core/os/x86_64/core.db.tar.gz [37650/37650] -> "core.db.tar.gz" [1]
2011-02-21 18:12:06 URL:http://ftp.jaist.ac.jp/pub/Linux/ArchLinux/extra/os/x86_64/extra.db.tar.gz [475981/475981] -> "extra.db.tar.gz" [1]
2011-02-21 18:12:16 URL:http://ftp.jaist.ac.jp/pub/Linux/ArchLinux/community/os/x86_64/community.db.tar.gz [434826/434826] -> "community.db.tar.gz" [1]
2011-02-21 18:12:20 URL: ftp://locke.suu.edu/linux/dist/archlinux/core/os/x86_64/core.db.tar.gz [27450] -> ".listing" [1]
2011-02-21 18:12:22 URL: ftp://locke.suu.edu/linux/dist/archlinux/core/os/x86_64/core.db.tar.gz [37650] -> "core.db.tar.gz" [1]
2011-02-21 18:12:40 URL: ftp://locke.suu.edu/linux/dist/archlinux/extra/os/x86_64/extra.db.tar.gz [392045] -> ".listing" [1]
2011-02-21 18:13:00 URL: ftp://locke.suu.edu/linux/dist/archlinux/extra/os/x86_64/extra.db.tar.gz [475981] -> "extra.db.tar.gz" [1]
2011-02-21 18:13:16 URL: ftp://locke.suu.edu/linux/dist/archlinux/community/os/x86_64/community.db.tar.gz [299376] -> ".listing" [1]
2011-02-21 18:13:32 URL: ftp://locke.suu.edu/linux/dist/archlinux/community/os/x86_64/community.db.tar.gz [434826] -> "community.db.tar.gz" [1]
2011-02-21 18:13:35 URL:http://ftp.tku.edu.tw/Linux/ArchLinux/core/os/x86_64/core.db.tar.gz [37650/37650] -> "core.db.tar.gz" [1]
2011-02-21 18:13:47 URL:http://ftp.tku.edu.tw/Linux/ArchLinux/extra/os/x86_64/extra.db.tar.gz [475981/475981] -> "extra.db.tar.gz" [1]
2011-02-21 18:13:58 URL:http://ftp.tku.edu.tw/Linux/ArchLinux/community/os/x86_64/community.db.tar.gz [434826/434826] -> "community.db.tar.gz" [1]
2011-02-21 18:14:01 URL:http://mirrors.kernel.org/archlinux/core/os/x86_64/core.db.tar.gz [37650/37650] -> "core.db.tar.gz" [1]
2011-02-21 18:14:12 URL:http://mirrors.kernel.org/archlinux/extra/os/x86_64/extra.db.tar.gz [475981/475981] -> "extra.db.tar.gz" [1]
2011-02-21 18:14:21 URL:http://mirrors.kernel.org/archlinux/community/os/x86_64/community.db.tar.gz [434826/434826] -> "community.db.tar.gz" [1]

=========== ANALYZING =============
ftp.archlinux.org:
    core.db: timestamp mismatch   OK
    extra.db: timestamp mismatch   OK
    community.db: timestamp mismatch   OK
ftp.jaist.ac.jp:
    core.db: OK
    extra.db: OK
    community.db: OK
locke.suu.edu:
    core.db: timestamp mismatch   OK
    extra.db: timestamp mismatch   OK
    community.db: timestamp mismatch   OK
ftp.tku.edu.tw:
    core.db: OK
    extra.db: OK
    community.db: OK
mirrors.kernel.org:
    core.db: OK
    extra.db: OK
    community.db: OK

============ SUMMARY ==============

All OK.

Phew

IgnorantGuru · 2011-02-21 17:30:36

Thanks - I didn't get around to a usage example.

And here is what you might see if a mirror is out of sync:

=========== ANALYZING =============
mirrors.163.com:
    core.db: timestamp mismatch   CONTENT MISMATCH
    extra.db: timestamp mismatch   CONTENT MISMATCH
    community.db: timestamp mismatch   CONTENT MISMATCH
    Scanning packages...
        curl-7.21.4-2: MISSING
        glibc-2.13-4: MISSING
        kernel26-2.6.37.1-1: MISSING
        kernel26-headers-2.6.37.1-1: MISSING
        mpg123-1.13.2-1: MISSING
        wget-1.12-4: MISSING
ftp.archlinux.org:
    core.db: timestamp mismatch   OK
    extra.db: timestamp mismatch   CONTENT MISMATCH
    community.db: timestamp mismatch   OK
    Scanning packages...
ftp.tku.edu.tw:
    core.db: OK
    extra.db: timestamp mismatch   CONTENT MISMATCH
    community.db: OK
    Scanning packages...

============ SUMMARY ==============

THE FOLLOWING PACKAGES ARE MISSING FROM THE INDICATED MIRROR - if they
are listed as missing from all mirrors above, this indicates they could
not be tested for authenticity:

    curl-7.21.4-2  (mirrors.163.com)
    glibc-2.13-4  (mirrors.163.com)
    kernel26-2.6.37.1-1  (mirrors.163.com)
    kernel26-headers-2.6.37.1-1  (mirrors.163.com)
    mpg123-1.13.2-1  (mirrors.163.com)
    wget-1.12-4  (mirrors.163.com)

THERE WERE DB CONTENT MISMATCHES - This usually indicates some mirrors
were out of date with your pacman sync, but alone does not indicate
compromised mirrors.

And then in this example I also changed the MD5 sum for curl in pacman's sync to trigger a detection. Since the problem isn't on the mirror but in the sync, it will report it twice.

=========== ANALYZING =============
mirrors.163.com:
    core.db: timestamp mismatch   CONTENT MISMATCH
    extra.db: timestamp mismatch   CONTENT MISMATCH
    community.db: timestamp mismatch   CONTENT MISMATCH
    Scanning packages...
        curl-7.21.4-2: MISSING
        glibc-2.13-4: MISSING
        kernel26-2.6.37.1-1: MISSING
        kernel26-headers-2.6.37.1-1: MISSING
        mpg123-1.13.2-1: MISSING
        wget-1.12-4: MISSING
ftp.archlinux.org:
    core.db: timestamp mismatch   OK
    extra.db: timestamp mismatch   CONTENT MISMATCH
    community.db: timestamp mismatch   OK
    Scanning packages...
        curl-7.21.4-2: @@@@@@@@@@ MISMATCH @@@@@@@@@@@
ftp.tku.edu.tw:
    core.db: OK
    extra.db: timestamp mismatch   CONTENT MISMATCH
    community.db: OK
    Scanning packages...
        curl-7.21.4-2: @@@@@@@@@@ MISMATCH @@@@@@@@@@@

============ SUMMARY ==============

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
THE FOLLOWING PACKAGES HAVE MISMATCHES - THIS IS BAD - this may indicate
compromised mirrors (including your default pacman mirror):

    curl-7.21.4-2  (ftp.archlinux.org)
    curl-7.21.4-2  (ftp.tku.edu.tw)
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

THE FOLLOWING PACKAGES ARE MISSING FROM THE INDICATED MIRROR - if they
are listed as missing from all mirrors above, this indicates they could
not be tested for authenticity:

    curl-7.21.4-2  (mirrors.163.com)
    glibc-2.13-4  (mirrors.163.com)
    kernel26-2.6.37.1-1  (mirrors.163.com)
    kernel26-headers-2.6.37.1-1  (mirrors.163.com)
    mpg123-1.13.2-1  (mirrors.163.com)
    wget-1.12-4  (mirrors.163.com)

THERE WERE DB CONTENT MISMATCHES - This usually indicates some mirrors
were out of date with your pacman sync, but alone does not indicate
compromised mirrors.

That "@@ MISMATCH @@" is the one to look out for, because that isn't just an out-of-sync mirror, but represents differing MD5 sums somewhere.

Allan · 2011-02-22 00:24:48

Interesting approach. Could of points of interest:

1) I'm fairly sure there is already a pacman wrapper that downloads sync databases from one mirror and packages from another to automatically detect these differences.

2) Select your mirrors carefully. There is no point using a set of mirrors that all sync from the same Tier 1 mirror...

IgnorantGuru · 2011-02-22 12:28:25

Allan wrote:

2) Select your mirrors carefully. There is no point using a set of mirrors that all sync from the same Tier 1 mirror...

Thanks - I didn't know they were tiered. I wouldn't say there is "no point", as any polling will improve security, but it is certainly an advantage for the mirrors to be as diverse as possible. I don't see any information on which tier 1 mirror is used by a given tier 2 mirror. I only see this list of tier 1 mirrors, and this list of tier 1 mirrors in the US. Given this apparent lack of information, it would probably be best to use tier 1 mirrors in paccheck. And if anyone has more info I'd like to see - thanks.

Allan · 2011-02-22 12:49:34

I'm not sure that information is publicly available. There is this: http://www.archlinux.org/mirrors/ . Clearly the German mirrors all will sync from the German Tier 1 mirror, and the Australian ones from... etc.

keenerd · 2011-02-22 13:04:17

Nice! The only weakness I can see is that MD5 is easy to break. As far as I know, generating a collision is easier than hacking a mirror so it might be worthwhile to take this into consideration.

IgnorantGuru · 2011-02-22 14:06:43

keenerd wrote:

Nice! The only weakness I can see is that MD5 is easy to break. As far as I know, generating a collision is easier than hacking a mirror so it might be worthwhile to take this into consideration.

True MD5 is outdated - unfortunately Arch provides only MD5 sums. Another option would be to download the packages (or portion thereof) from multiple mirrors, which I'll consider adding an option for. But obviously that is less convenient in terms of download time and bandwidth.

I don't know which is easier - at least they are different skill sets. Last I knew, generating an MD5 collision on an arbitrary file was non-trivial, but I'm not up on the latest.

Last edited by IgnorantGuru (2011-02-22 14:14:30)

IgnorantGuru · 2011-02-22 17:06:06

http://www.archlinux.org/mirrors/ appears to provide the tier1 mirror used by tier2 mirrors - if you click on the mirror and look at the Upstream parameter.

Also, I did a bit of reading on MD5 vulnerabilities. Given the size of the packages I don't think there is a significant threat that someone will generate another package with the same hash. Generating two files with the same hash is somewhat trivial - generating a file with the same hash as an arbitrary file is not, AFAIK. Also, Red Hat's rpm files still use MD5.

And pacman's desc files also contain the package size. While pacman seems to ignore this during integrity check, I am going to see if paccheck could make use of it.

rich_o · 2011-02-22 18:59:47

Nice!

can you add different exit values so i can use paccheck && sudo pacman -Su?

rich_o

IgnorantGuru · 2011-02-22 21:49:01

paccheck 0.8.5 is available with these additions:

* added mirror tier and upstream info

* added warning if using mirrors fed from a single upstream mirror

* added test of package sizes in pacman's pkg cache

* added exit statuses:
3 Package MISMATCH, download failures, or other errors
2 Packages missing from some mirrors
1 Out of sync mirrors (DATABASE CONTENT MISMATCH) or other warnings
0 All OK

An example sample run (with lots of problems! Your runs shouldn't be this exciting):

=========== ANALYZING =============
mirror.cXsclub.uwaterloo.ca:  
    community.db: DOWNLOAD FAILED
    extra.db: DOWNLOAD FAILED
    core.db: DOWNLOAD FAILED
mirrors.163.com:  Tier 2 (China) Upstream: tku.edu.tw
    community.db: timestamp mismatch   CONTENT MISMATCH
    extra.db: timestamp mismatch   CONTENT MISMATCH
    core.db: timestamp mismatch   CONTENT MISMATCH
    Scanning packages...
        curl-7.21.4-2: @@@@@@@@@@ MISMATCH @@@@@@@@@@@
        redland-1.0.12-4: MISSING
        wget-1.12-4: MISSING
mirror.bjtu.edu.cn:  Tier 2 (China) Upstream: tku.edu.tw
    community.db: timestamp mismatch   CONTENT MISMATCH
    extra.db: OK
    core.db: timestamp mismatch   CONTENT MISMATCH
    Scanning packages...
        curl-7.21.4-2: @@@@@@@@@@ MISMATCH @@@@@@@@@@@
        wget-1.12-4: MISSING

Checking packages sizes...
    curl-7.21.4-2: @@@@@@@@ SIZE MISMATCH @@@@@@@@@

============ SUMMARY ==============

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
THE FOLLOWING PACKAGES HAVE MISMATCHES - THIS IS BAD - this may indicate
compromised mirrors (including your default pacman mirror):

    curl-7.21.4-2  (mirrors.163.com)
    curl-7.21.4-2  (mirror.bjtu.edu.cn)
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
THE FOLLOWING PACKAGES IN PACMAN'S PKG CACHE ARE THE WRONG SIZE - this
indicates they are corrupt, or they or the database has been tampered
with:

    curl-7.21.4-2  (451448 != 4514480)
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

THERE WERE DOWNLOAD FAILURES - This indicates unresponsive mirror(s).

THE FOLLOWING PACKAGES ARE MISSING FROM THE INDICATED MIRROR - if they
are listed as missing from all mirrors above, this indicates they could
not be tested for authenticity:

    redland-1.0.12-4  (mirrors.163.com)
    wget-1.12-4  (mirrors.163.com)
    wget-1.12-4  (mirror.bjtu.edu.cn)

THERE WERE DATABASE CONTENT MISMATCHES - This usually indicates some
mirrors were out of date with your pacman sync, but alone does not
indicate compromised mirrors.

WARNING: ALL MIRRORS APPEAR TO USE A SINGLE UPSTREAM MIRROR - For better
security, use several Tier 1 mirrors, or Tier 2 mirrors with different
upstream mirrors.  See http://www.archlinux.org/mirrors/

System update is NOT recommended until the above problems are fixed.

rich_o wrote:

can you add different exit values so i can use paccheck && sudo pacman -Su?

I added several exit statuses depending on the severity of the problem. If you want to ignore out-of-sync database and other warnings you can use:

# if exit status is less than 2, then proceed with update:
paccheck; if [ $? -lt 2 ]; then sudo pacman -Su; fi

Although really I recommend reading the report first if there are any problems, so your way is better.

http://igurublog.wordpress.com/download … -paccheck/
http://aur.archlinux.org/packages.php?ID=46763

Last edited by IgnorantGuru (2011-02-22 22:03:33)

Allan · 2011-02-22 23:15:20

keenerd wrote:

Nice! The only weakness I can see is that MD5 is easy to break. As far as I know, generating a collision is easier than hacking a mirror so it might be worthwhile to take this into consideration.

Generating a collision for a compressed tar.{g,x}z is a lot harder (has it been done?) as they effectively strip lots of NULL values from the ends of files which is the common way this is done.

IgnorantGuru · 2011-02-22 23:51:52

Allan wrote:

Generating a collision for a compressed tar.{g,x}z is a lot harder (has it been done?) as they effectively strip lots of NULL values from the ends of files which is the common way this is done.

It has been done, at least in theory. paccheck now checks the file size as well against %CSIZE% in desc, so that would be another parameter for the attacker to duplicate. How easy this is in reality I don't know.

toad · 2011-02-23 07:53:34

Nice stuff. Your changes work

toad@archtop 500\1 ~ > sudo reflector -l 5 -r -o /etc/pacman.d/mirrorlist
Password: 

toad@archtop 501\2 ~ > paccheck 
pacman sync...
>>> sudo pacman -Sy
Password: 
:: Synchronising package databases...
 core is up to date
 extra is up to date
 community                                                                425.1K   32.2K/s 00:00:13 [----------------------------------------------------------] 100%
 archlinuxfr is up to date
 multilib is up to date
 arch-games is up to date

Download needed packages without installing...
>>> sudo pacman -w --noconfirm -Su
:: Starting full system upgrade...
 there is nothing to do

Timestamps:
/var/lib/pacman/community.db.tar.gz: 2011-02-23 04:23:46.000000000 +0100
/var/lib/pacman/extra.db.tar.gz: 2011-02-22 21:20:35.000000000 +0100
/var/lib/pacman/arch-games.db.tar.gz: 2010-12-23 11:21:25.000000000 +0100
/var/lib/pacman/archlinuxfr.db.tar.gz: 2011-02-22 00:16:55.000000000 +0100
/var/lib/pacman/multilib.db.tar.gz: 2011-02-22 09:28:23.000000000 +0100
/var/lib/pacman/core.db.tar.gz: 2011-02-22 13:39:59.000000000 +0100

========== DOWNLOADING ============
Downloading info on mirror.aarnet.edu.au
2011-02-23 09:39:56 URL:http://mirror.aarnet.edu.au/pub/archlinux/community/os/x86_64/community.db.tar.gz [435268/435268] -> "community.db.tar.gz" [1]
2011-02-23 09:40:43 URL:http://mirror.aarnet.edu.au/pub/archlinux/extra/os/x86_64/extra.db.tar.gz [476466/476466] -> "extra.db.tar.gz" [1]
http://mirror.aarnet.edu.au/pub/archlinux/arch-games/os/x86_64/arch-games.db.tar.gz:
2011-02-23 09:40:44 ERROR 404: Not Found.
http://mirror.aarnet.edu.au/pub/archlinux/archlinuxfr/os/x86_64/archlinuxfr.db.tar.gz:
2011-02-23 09:40:45 ERROR 404: Not Found.
2011-02-23 09:40:47 URL:http://mirror.aarnet.edu.au/pub/archlinux/multilib/os/x86_64/multilib.db.tar.gz [24893/24893] -> "multilib.db.tar.gz" [1]
2011-02-23 09:40:50 URL:http://mirror.aarnet.edu.au/pub/archlinux/core/os/x86_64/core.db.tar.gz [37661/37661] -> "core.db.tar.gz" [1]
Downloading info on mirror.rit.edu
2011-02-23 09:41:04 URL:http://mirror.rit.edu/archlinux/community/os/x86_64/community.db.tar.gz [435268/435268] -> "community.db.tar.gz" [1]
2011-02-23 09:41:20 URL:http://mirror.rit.edu/archlinux/extra/os/x86_64/extra.db.tar.gz [476466/476466] -> "extra.db.tar.gz" [1]
http://mirror.rit.edu/archlinux/arch-games/os/x86_64/arch-games.db.tar.gz:
2011-02-23 09:41:21 ERROR 404: Not Found.
http://mirror.rit.edu/archlinux/archlinuxfr/os/x86_64/archlinuxfr.db.tar.gz:
2011-02-23 09:41:22 ERROR 404: Not Found.
2011-02-23 09:41:23 URL:http://mirror.rit.edu/archlinux/multilib/os/x86_64/multilib.db.tar.gz [24893/24893] -> "multilib.db.tar.gz" [1]
2011-02-23 09:41:25 URL:http://mirror.rit.edu/archlinux/core/os/x86_64/core.db.tar.gz [37661/37661] -> "core.db.tar.gz" [1]
Downloading info on ftp5.gwdg.de
2011-02-23 09:41:58 URL: ftp://ftp5.gwdg.de/pub/linux/archlinux/community/os/x86_64/community.db.tar.gz [299866] -> ".listing" [1]
2011-02-23 09:42:23 URL: ftp://ftp5.gwdg.de/pub/linux/archlinux/community/os/x86_64/community.db.tar.gz [435268] -> "community.db.tar.gz" [1]
2011-02-23 09:42:35 URL: ftp://ftp5.gwdg.de/pub/linux/archlinux/extra/os/x86_64/extra.db.tar.gz [393315] -> ".listing" [1]
2011-02-23 09:42:52 URL: ftp://ftp5.gwdg.de/pub/linux/archlinux/extra/os/x86_64/extra.db.tar.gz [476309] -> "extra.db.tar.gz" [1]
No such directory `pub/linux/archlinux/arch-games/os/x86_64'.

No such directory `pub/linux/archlinux/archlinuxfr/os/x86_64'.

2011-02-23 09:42:57 URL: ftp://ftp5.gwdg.de/pub/linux/archlinux/multilib/os/x86_64/multilib.db.tar.gz [23172] -> ".listing" [1]
2011-02-23 09:42:58 URL: ftp://ftp5.gwdg.de/pub/linux/archlinux/multilib/os/x86_64/multilib.db.tar.gz [24893] -> "multilib.db.tar.gz" [1]
2011-02-23 09:43:00 URL: ftp://ftp5.gwdg.de/pub/linux/archlinux/core/os/x86_64/core.db.tar.gz [27450] -> ".listing" [1]
2011-02-23 09:43:02 URL: ftp://ftp5.gwdg.de/pub/linux/archlinux/core/os/x86_64/core.db.tar.gz [37661] -> "core.db.tar.gz" [1]
Downloading info on ftp.tku.edu.tw
2011-02-23 09:43:26 URL:http://ftp.tku.edu.tw/Linux/ArchLinux/community/os/x86_64/community.db.tar.gz [435268/435268] -> "community.db.tar.gz" [1]
2011-02-23 09:44:29 URL:http://ftp.tku.edu.tw/Linux/ArchLinux/extra/os/x86_64/extra.db.tar.gz [476466/476466] -> "extra.db.tar.gz" [1]
http://ftp.tku.edu.tw/Linux/ArchLinux/arch-games/os/x86_64/arch-games.db.tar.gz:
2011-02-23 09:44:32 ERROR 404: Not Found.
http://ftp.tku.edu.tw/Linux/ArchLinux/archlinuxfr/os/x86_64/archlinuxfr.db.tar.gz:
2011-02-23 09:44:34 ERROR 404: Not Found.
2011-02-23 09:44:36 URL:http://ftp.tku.edu.tw/Linux/ArchLinux/multilib/os/x86_64/multilib.db.tar.gz [24893/24893] -> "multilib.db.tar.gz" [1]
2011-02-23 09:44:39 URL:http://ftp.tku.edu.tw/Linux/ArchLinux/core/os/x86_64/core.db.tar.gz [37661/37661] -> "core.db.tar.gz" [1]

=========== ANALYZING =============
mirror.aarnet.edu.au:  Tier 1 (Australia) 
    community.db: OK
    extra.db: timestamp mismatch   CONTENT MISMATCH
    arch-games.db: DOWNLOAD FAILED
    archlinuxfr.db: DOWNLOAD FAILED
    multilib.db: OK
    core.db: OK
    Scanning packages...
/usr/bin/paccheck: line 277: /tmp/paccheck.tmp/1-mirror.aarnet.edu.au/arch-games.db.tar.gz: No such file or directory
tar: This does not look like a tar archive
tar: Exiting with failure status due to previous errors
    EXTRACT ERROR
/usr/bin/paccheck: line 277: /tmp/paccheck.tmp/1-mirror.aarnet.edu.au/archlinuxfr.db.tar.gz: No such file or directory
tar: This does not look like a tar archive
tar: Exiting with failure status due to previous errors
    EXTRACT ERROR
        codecs-20100303-1: MISSING
        libao-1.0.0-5: MISSING
        package-query-0.6-1: MISSING
        pacman-color-3.4.3-1: MISSING
        yaourt-0.9.5.1-1: MISSING
mirror.rit.edu:  Tier 1 (United States) 
    community.db: OK
    extra.db: timestamp mismatch   CONTENT MISMATCH
    arch-games.db: DOWNLOAD FAILED
    archlinuxfr.db: DOWNLOAD FAILED
    multilib.db: OK
    core.db: OK
    Scanning packages...
/usr/bin/paccheck: line 277: /tmp/paccheck.tmp/2-mirror.rit.edu/arch-games.db.tar.gz: No such file or directory
tar: This does not look like a tar archive
tar: Exiting with failure status due to previous errors
    EXTRACT ERROR
/usr/bin/paccheck: line 277: /tmp/paccheck.tmp/2-mirror.rit.edu/archlinuxfr.db.tar.gz: No such file or directory
tar: This does not look like a tar archive
tar: Exiting with failure status due to previous errors
    EXTRACT ERROR
        codecs-20100303-1: MISSING
        libao-1.0.0-5: MISSING
        package-query-0.6-1: MISSING
        pacman-color-3.4.3-1: MISSING
        yaourt-0.9.5.1-1: MISSING
ftp5.gwdg.de:  Tier 1 (Germany) 
    community.db: timestamp mismatch   OK
    extra.db: timestamp mismatch   OK
    arch-games.db: DOWNLOAD FAILED
    archlinuxfr.db: DOWNLOAD FAILED
    multilib.db: timestamp mismatch   OK
    core.db: timestamp mismatch   OK
ftp.tku.edu.tw:  Tier 1 (Taiwan) 
    community.db: OK
    extra.db: timestamp mismatch   CONTENT MISMATCH
    arch-games.db: DOWNLOAD FAILED
    archlinuxfr.db: DOWNLOAD FAILED
    multilib.db: OK
    core.db: OK
    Scanning packages...
/usr/bin/paccheck: line 277: /tmp/paccheck.tmp/4-ftp.tku.edu.tw/arch-games.db.tar.gz: No such file or directory
tar: This does not look like a tar archive
tar: Exiting with failure status due to previous errors
    EXTRACT ERROR
/usr/bin/paccheck: line 277: /tmp/paccheck.tmp/4-ftp.tku.edu.tw/archlinuxfr.db.tar.gz: No such file or directory
tar: This does not look like a tar archive
tar: Exiting with failure status due to previous errors
    EXTRACT ERROR
        codecs-20100303-1: MISSING
        libao-1.0.0-5: MISSING
        package-query-0.6-1: MISSING
        pacman-color-3.4.3-1: MISSING
        yaourt-0.9.5.1-1: MISSING

Checking packages sizes...

============ SUMMARY ==============

THERE WERE ERRORS - This indicates one or more malfunctions.

THERE WERE DOWNLOAD FAILURES - This indicates unresponsive mirror(s).

THE FOLLOWING PACKAGES ARE MISSING FROM THE INDICATED MIRROR - if they
are listed as missing from all mirrors above, this indicates they could
not be tested for authenticity:

    codecs-20100303-1  (mirror.aarnet.edu.au)
    libao-1.0.0-5  (mirror.aarnet.edu.au)
    package-query-0.6-1  (mirror.aarnet.edu.au)
    pacman-color-3.4.3-1  (mirror.aarnet.edu.au)
    yaourt-0.9.5.1-1  (mirror.aarnet.edu.au)
    codecs-20100303-1  (mirror.rit.edu)
    libao-1.0.0-5  (mirror.rit.edu)
    package-query-0.6-1  (mirror.rit.edu)
    pacman-color-3.4.3-1  (mirror.rit.edu)
    yaourt-0.9.5.1-1  (mirror.rit.edu)
    codecs-20100303-1  (ftp.tku.edu.tw)
    libao-1.0.0-5  (ftp.tku.edu.tw)
    package-query-0.6-1  (ftp.tku.edu.tw)
    pacman-color-3.4.3-1  (ftp.tku.edu.tw)
    yaourt-0.9.5.1-1  (ftp.tku.edu.tw)

THERE WERE DATABASE CONTENT MISMATCHES - This usually indicates some
mirrors were out of date with your pacman sync, but alone does not
indicate compromised mirrors.

System update is NOT recommended until the above problems are fixed.

milomouse · 2011-02-23 15:08:24

Not a problem really, as these are the pacman defaults, but for those of us with a non-standard directory/hierarchy it might be prudent to check what pacman.conf says as far as where the database and cache are housed (DBPath and CacheDir). Otherwise, I just edited the script and changed those locations, no big deal, and it works nicely. Thanks for your work, as always. :}

Xyne · 2011-02-23 16:29:37

Allan wrote:

1) I'm fairly sure there is already a pacman wrapper that downloads sync databases from one mirror and packages from another to automatically detect these differences.
2) Select your mirrors carefully. There is no point using a set of mirrors that all sync from the same Tier 1 mirror...

Powerpill and Bauerbill do that. The idea is to restrict Pacman's mirrorlist to "trusted" mirrors, which are then used for the database. Packages are pulled in from numerous mirrors retrieved with reflector at runtime, but all checks are made against the checksums in the database from the "trusted" mirror(s).

Or course, you can bring up man-in-the-middle attacks etc. It's not "secure" in a strict sense, but as it stands, nothing is.

IgnorantGuru · 2011-02-23 18:53:25

Thanks for the feedback. paccheck 0.8.6 is available with the following changes and additions:

* added retries, connect timeout limits

* corrected extract error on download failed

* reuse http/ftp connection for faster downloads

* added --keep

* added --compare 'MIRROR' (see below)

* added --verbose

* added interrupt trap to remove tmp folder

* get DBPath and CacheDir from pacman.conf

paccheck version 0.8.6
Compares Arch Linux pacman sync and package cache to multiple mirrors to help
detect compromised mirrors

Usage: paccheck [OPTIONS]
OPTIONS:
--keep              Don't remove temporary files in /tmp/paccheck.tmp
--compare 'MIRROR'  Fully download and compare all non-expired packages in
                    pacman's pkg cache to MIRROR.  Can alternatively be
                    listed in /etc/paccheck/mirrorlist as "compare=MIRROR".
                    For best speed, remove unneeded (already installed)
                    packages from pacman's cache before using this option.
--verbose           show debugging output

pacman Update Procedure:
   1) Run paccheck and examine report
   2) If no package MISMATCH, then run sudo pacman -Su to update your system

Desired mirrors may be configured inside this script or 
saved to /etc/paccheck/mirrorlist

Exit Status:
    3  Package MISMATCH, download failures, or other errors
    2  Packages missing from some mirrors
    1  Out of sync mirrors (DATABASE CONTENT MISMATCH) or other warnings
    0  All OK

For those concerned about the risk of fraudulent packages constructed with MD5 collisions, paccheck now has a --compare 'MIRROR' option (or specify compare=MIRROR in /etc/paccheck/mirrorlist). All the non-expired packages in your cache will be fully downloaded from MIRROR and compared byte-for-byte. This is probably overkill, but it closes that hole for those concerned. For best performance, remove unneeded (already installed) packages from pacman's cache before using this option.

http://igurublog.wordpress.com/download … -paccheck/

toad · 2011-02-23 19:06:33

Xyne wrote:

Allan wrote:
1) I'm fairly sure there is already a pacman wrapper that downloads sync databases from one mirror and packages from another to automatically detect these differences.
2) Select your mirrors carefully. There is no point using a set of mirrors that all sync from the same Tier 1 mirror...
Powerpill and Bauerbill do that. The idea is to restrict Pacman's mirrorlist to "trusted" mirrors, which are then used for the database. Packages are pulled in from numerous mirrors retrieved with reflector at runtime, but all checks are made against the checksums in the database from the "trusted" mirror(s).
Or course, you can bring up man-in-the-middle attacks etc. It's not "secure" in a strict sense, but as it stands, nothing is.

What about reflector - wasn't that you as well?

toad · 2011-02-24 06:47:20

0.8.6. works as advertised - or do you want output, IG?

IgnorantGuru · 2011-02-24 11:54:52

toad wrote:

0.8.6. works as advertised - or do you want output, IG?

Thanks - as long as those extract errors on failed downloads cleared up for you, and it behaves reasonably for the repos that aren't on your mirrors, that's ok.

kgas · 2011-02-24 15:47:26

Thanks IG. The /etc/paccheck directory is not created by default. imho the mirror list can be generated from the current mirror list of pacman. I relayed on old mirrorlist and with that it gave download error (404) and final warning also.

IgnorantGuru · 2011-02-24 19:33:45

kgas wrote:

Thanks IG. The /etc/paccheck directory is not created by default.

True. If that's considered important I could add it to the PKGBUILD.

imho the mirror list can be generated from the current mirror list of pacman.

/etc/paccheck/mirrorlist accepts mirrors in this format:

ftp://ftp.archlinux.org/$repo/os/$arch
#or
http://mirror.rit.edu/archlinux/$repo/os/$arch
# or for a full compare:
compare=http://mirror.rit.edu/archlinux/$repo/os/$arch

The "$repo/os/$arch" part must be present just as shown or absent completely (in which case paccheck will add it).

I relayed on old mirrorlist and with that it gave download error (404) and final warning also.

That may be because older mirror lists specified the $arch value explicitly. I'll look at that behavior.

I will make the next version handle the "Server=" in front of them too, so they can be copied directly from pacman's mirrorlist.

Last edited by IgnorantGuru (2011-02-24 19:35:30)

berbae · 2011-02-24 22:54:33

If I understand the principle of your script, it looks for differences at a given time among a list of chosen mirrors.
It looks for possible modified packages from the data in their desc file.
But :
1) As the mirrors are not synchronized at the same time, many of the differences will just be for not yet updated packages.

2) If someone wants to inject some modified packages for evil reasons, he will try to put them in as many mirrors as he can have access to.
So how to know how many and which mirrors has been affected ? How to be sure that all the mirrors in the chosen list has not all be modified in the same way by the same person ?

3) will not the --compare option which downloads again all the packages generate a big traffic from mirrors ? And it is not sure that the mirror which is compared is not affected in the same way as the initial one.

For these reasons, I am not convinced that paccheck can give a suffisant assurance that everything is safe from the chosen mirrors, and the results in most cases will just be due to the difference in the synchronization time of the mirrors.

I cannot be convinced enough that using paccheck will give me more assurance of a safe upgrade than the present state of affairs.

But I agree with you on the signature of packages using gpg, and on the idea to do something urgently about this vulnerability in the upgrading in Arch Linux.

I am confident that the Arch dev and contributors will soon solve that problem, using the more secure gpg signing of the packages.

toad · 2011-02-24 23:30:31

1) if you run reflector beforehand this reduces that risk considerably
2) wot, signed packages
3) there is a fair amount of traffic going on, make of that what you will

I think this represents a novel approach and is the best we have at the mo (hey, 3 years no probs...)

Failing package signing I think that a combination of reflector and paccheck provides a decent alternative.

IgnorantGuru · 2011-02-25 06:04:44

berbae wrote:

If I understand the principle of your script, it looks for differences at a given time among a list of chosen mirrors.

Yes, that time being just before you install those packages.

As the mirrors are not synchronized at the same time, many of the differences will just be for not yet updated packages.

paccheck distinguishes between version changes and package tampering. Too many out of sync mirrors could cause paccheck to be unable to check some packages, but it will tell you that. With enough mirrors it shouldn't be a problem.

If someone wants to inject some modified packages for evil reasons, he will try to put them in as many mirrors as he can have access to.
So how to know how many and which mirrors has been affected ? How to be sure that all the mirrors in the chosen list has not all be modified in the same way by the same person ?

Security is not about making something impossible or being "sure", but about making something difficult and being reasonably sure. paccheck makes the problem substantially more difficult at the cost of time and bandwidth. Signatures for the packages would make the problem even more difficult and eliminate time and bandwidth as cost factors.

Put another way, through polling paccheck shifts the weight of security from a loose network of inconsistently configured and maintained mirrors to the primary arch server that feeds the tier 1 mirrors (from what I understand that server is not accessible to the public). That's not to say its invincible. On the other hand, package signing would shift the weight of security to the systems holding the packagers' keys and the systems they type the passphrases on, also not invincible.

Also, with people running paccheck and other similar programs, the likelihood of detecting compromised mirrors is greatly increased - effectively a monitoring system for the mirrors.

With paccheck I don't see it so much a question of whether it can improve security. It can handle as many mirrors as you give it, which should far exceed the scope of any likely multiple-mirror security breach. Instead it's a question of how much time and bandwidth are you willing to spend for what level of assurance? Personally I think giving it a few tier 1 mirrors is sufficient for reasonable security - certainly an improvement over no polling at all. But of course you must make your own security assessments for your purposes.

Arch Linux

#1 2011-02-21 15:18:41

paccheck - pacman package authenticity check

#2 2011-02-21 17:16:49

Re: paccheck - pacman package authenticity check

#3 2011-02-21 17:30:36

Re: paccheck - pacman package authenticity check

#4 2011-02-22 00:24:48

Re: paccheck - pacman package authenticity check

#5 2011-02-22 12:28:25

Re: paccheck - pacman package authenticity check

#6 2011-02-22 12:49:34

Re: paccheck - pacman package authenticity check

#7 2011-02-22 13:04:17

Re: paccheck - pacman package authenticity check

#8 2011-02-22 14:06:43

Re: paccheck - pacman package authenticity check

#9 2011-02-22 17:06:06

Re: paccheck - pacman package authenticity check

#10 2011-02-22 18:59:47

Re: paccheck - pacman package authenticity check

#11 2011-02-22 21:49:01

Re: paccheck - pacman package authenticity check

#12 2011-02-22 23:15:20

Re: paccheck - pacman package authenticity check

#13 2011-02-22 23:51:52

Re: paccheck - pacman package authenticity check

#14 2011-02-23 07:53:34

Re: paccheck - pacman package authenticity check

#15 2011-02-23 15:08:24

Re: paccheck - pacman package authenticity check

#16 2011-02-23 16:29:37

Re: paccheck - pacman package authenticity check

#17 2011-02-23 18:53:25

Re: paccheck - pacman package authenticity check

#18 2011-02-23 19:06:33

Re: paccheck - pacman package authenticity check

#19 2011-02-24 06:47:20

Re: paccheck - pacman package authenticity check

#20 2011-02-24 11:54:52

Re: paccheck - pacman package authenticity check

#21 2011-02-24 15:47:26

Re: paccheck - pacman package authenticity check

#22 2011-02-24 19:33:45

Re: paccheck - pacman package authenticity check

#23 2011-02-24 22:54:33

Re: paccheck - pacman package authenticity check

#24 2011-02-24 23:30:31

Re: paccheck - pacman package authenticity check

#25 2011-02-25 06:04:44

Re: paccheck - pacman package authenticity check

Board footer