You are not logged in.

#1 2024-03-29 19:13:28

ewaller
Administrator
From: Pasadena, CA
Registered: 2009-07-13
Posts: 19,784

xz Package Backdoor

Please see the Arch main page announcement and take appropriate action.

https://archlinux.org/news/the-xz-packa … ackdoored/


Nothing is too wonderful to be true, if it be consistent with the laws of nature -- Michael Faraday
Sometimes it is the people no one can imagine anything of who do the things no one can imagine. -- Alan Turing
---
How to Ask Questions the Smart Way

Offline

#2 2024-03-29 19:50:51

Luciddream
Member
From: Greece
Registered: 2014-12-08
Posts: 21

Re: xz Package Backdoor

Does anyone understand if we have been affected and how much?

Last edited by Luciddream (2024-03-29 19:51:02)

Offline

#3 2024-03-29 19:53:31

Scimmia
Fellow
Registered: 2012-09-01
Posts: 11,549

Re: xz Package Backdoor

For what is known *right now*, we aren't. The issue comes from what else it could have been doing that we aren't aware of yet.

Offline

#4 2024-03-29 20:16:18

avidseeker
Member
Registered: 2022-09-06
Posts: 45
Website

Re: xz Package Backdoor

What is the right course of action in such cases? should I immediately air gap my machine or nuke my Arch partition with a new Arch installation? If it's been backdoored, then is a full system upgrade really enough?

Last edited by avidseeker (2024-03-29 20:18:33)

Offline

#5 2024-03-29 20:21:05

ldarby
Member
From: UK
Registered: 2024-03-29
Posts: 14

Re: xz Package Backdoor

The advisory at https://security.archlinux.org/ASA-202403-1 and commit fix https://gitlab.archlinux.org/archlinux/ … f235629739 are very confusing.

It says:

> The problem has been fixed upstream in version 5.6.1.

which I believe is factually false, this contradicts every other advisory out there which says the issue is in both 5.6.0 and 5.6.1.

My understanding is that it's this commit to the arch package: https://gitlab.archlinux.org/archlinux/ … 9f637424ad that has "fixed" it, by changing the provenance of the build source to be from upstream git, instead of upstream's release tarballs, because the compromise is only in the tarballs.

Offline

#6 2024-03-29 20:54:16

Bumble
Member
Registered: 2016-12-04
Posts: 21

Re: xz Package Backdoor

i just set up a fresh install on a new machine with this, is updating the only thing i need to do?

Offline

#7 2024-03-29 20:58:40

eesa
Member
Registered: 2024-03-29
Posts: 1

Re: xz Package Backdoor

What is the latest uncompromised version?

Offline

#8 2024-03-29 21:06:34

seth
Member
Registered: 2012-09-03
Posts: 51,132

Re: xz Package Backdoor

avidseeker wrote:

What is the right course of action in such cases? should I immediately air gap my machine or nuke my Arch partition with a new Arch installation? If it's been backdoored, then is a full system upgrade really enough?

https://www.openwall.com/lists/oss-secu … 4/03/29/17
From what has been discovered so far this was a rather specific attack exploiting a downstream patch of sshd in debian and redhat.
I've not compared the binaries myself so I don't vouch for those findings but it's rather likely that your system has never been compromised tbw.

That being said, the known unknowns and the unkown unknowns and all that:
If you're running a hypercritical system and cannot affort *any* risk taking, you've to operate on the assumption that the system has been compromised and unless proof of difference cannot be trusted at all any longer.
Those proofs can take a while so if you've to act fast you need to reset the system and in theory you cannot trust any version of xz certainly including after https://github.com/tukaani-project/xz/c … 30ccab3ff1 (ie. after  5.2.10 in the repos) until it's clear what's going on there.
nb. that downgrading xz to 5.2.10 can cause major API/ABI incompatibilities w/ everthing compiled against newer versions and you can also not just use sources from github.com/tukaani-project

But then: if you're running a hypercritical system on a rolling release distro, wtf are you doing there? wink

Upstream "bug" (ie. people losing their shit - try to remember that you don't now what's going on and wild speculations are not only politically incorrect but can also lead you away from the truth):
https://github.com/tukaani-project/xz/issues/92

Offline

#9 2024-03-29 22:16:07

leonavis
Member
From: Bremen, Germany
Registered: 2021-06-27
Posts: 71

Re: xz Package Backdoor

But then: if you're running a hypercritical system on a rolling release distro, wtf are you doing there? wink

Well said big_smile

Offline

#10 2024-03-29 22:42:22

frostschutz
Member
Registered: 2013-11-15
Posts: 1,418

Re: xz Package Backdoor

Updated. Also ran mkinitcpio -P ...

Offline

#11 2024-03-29 22:46:13

seth
Member
Registered: 2012-09-03
Posts: 51,132

Re: xz Package Backdoor

A rather funny plot twist would be if the ifunc stuff (which imo should precauciously be completely disabled in builds now) would have lead to all the recent initramfs failures.

Offline

#12 2024-03-29 23:19:59

alvrogd
Member
Registered: 2024-03-29
Posts: 1

Re: xz Package Backdoor

seth wrote:

From what has been discovered so far this was a rather specific attack exploiting a downstream patch of sshd in debian and redhat.
I've not compared the binaries myself so I don't vouch for those findings but it's rather likely that your system has never been compromised tbw.

I've followed the conversation in the original report, and found some users comparing Arch's xz 5.6.1-1 vs. 5.6.1-2. By disassembling the liblzma library, it appears that the packages might have never been affected by the backdoor, due to the deb/rpm check in the script that decides whether to inject the vulnerability or not.

References:

https://www.openwall.com/lists/oss-secu … 4/03/29/17
https://www.openwall.com/lists/oss-secu … 4/03/29/20
https://www.openwall.com/lists/oss-secu … 4/03/29/22

Offline

#13 2024-03-30 01:47:57

vsczpv
Member
Registered: 2021-10-28
Posts: 3

Re: xz Package Backdoor

Just bumping here, the news page on https://archlinux.org/news/the-xz-packa … ackdoored/, which talks about the whole xz thing says this:

Arch does not directly link openssh to liblzma, and thus this attack vector is not possible. You can confirm this by issuing the following command:

    ldd "$(command -v sshd)"

However, out of an abundance of caution, we advise users to remove the malicious code from their system by upgrading either way. This is because other yet-to-be discovered methods to exploit the backdoor could exist.

In general you shouldn't tell people to run ldd on potentially compromised binaries, as it may invoke the dynamic linker and partially run the program your peeking at (and if the vuln is there, that'll get run too). There is a pretty big warning on ldd's manpage too.
(Granted, the issue at hand is on xz, not sshd, but still...)

Offline

#14 2024-03-30 06:24:01

ignuthat
Member
Registered: 2022-03-23
Posts: 18

Re: xz Package Backdoor

Some updates:

Github has taken the xz repository offline, and Jia Tan's account has been suspended:

[1] https://github.com/tukaani-project/xz
[2] https://github.com/JiaT75

Jia Tan's key was removed as a trusted key from Arch:

[3] https://gitlab.archlinux.org/archlinux/ … da2cb6c1ac

The xz PKGBUILD needs to be updated again, because it won't work anymore as it goes to the dead Github repository. Preferably replaced by an older version before Jia Tan took over; something in the 5.4.x series... as to how you get access to that? Not sure which mirror.

The malicious patch series submitted to the kernel has been abandoned:

[4] https://lkml.org/lkml/2024/3/29/1475

So far, it seems like openSUSE, Kali Linux, and Arch Linux are the only distributions to have shipped the known-backdoored xz package for any significant length of time. It was available in Arch between February 28 and March 28.

Edit: Here's the best timeline for the xz backdoor: https://boehs.org/node/everything-i-kno … z-backdoor

Last edited by ignuthat (2024-03-30 06:57:30)

Offline

#15 2024-03-30 08:02:31

OpusOne
Member
Registered: 2023-05-31
Posts: 82

Re: xz Package Backdoor

Will this case be investigated in any way? That's pretty disturbing.

Offline

#16 2024-03-30 08:37:11

ignuthat
Member
Registered: 2022-03-23
Posts: 18

Re: xz Package Backdoor

OpusOne wrote:

Will this case be investigated in any way? That's pretty disturbing.

I hope the Arch Security Team is continuing to look into this.

For now, I'm taking some extra precaution by downgrading to the 5.4.6 version of xz on Arch, as that's what other distributions like Nix are doing: https://github.com/NixOS/nixpkgs/pull/3 … 2027668527

Even 5.4.x is not guaranteed to be safe, but if parts of a malicious payload were quietly added in hundreds of seemingly innocuous commits over time, this could reduce the impact, so it's hopefully safer than remaining on the 5.6.1 codebase.

As a bystander, I can't do much more than remain calm and wait for the fallout.

Offline

#17 2024-03-30 09:59:14

Head_on_a_Stick
Member
From: London
Registered: 2014-02-20
Posts: 7,732
Website

Re: xz Package Backdoor

Should Arch now consider building packages from git directly rather than using tarballs? That would have prevented this problem entirely.

Offline

#18 2024-03-30 10:09:19

Allan
Pacman
From: Brisbane, AU
Registered: 2007-06-09
Posts: 11,387
Website

Re: xz Package Backdoor

alvrogd wrote:
seth wrote:

From what has been discovered so far this was a rather specific attack exploiting a downstream patch of sshd in debian and redhat.
I've not compared the binaries myself so I don't vouch for those findings but it's rather likely that your system has never been compromised tbw.

I've followed the conversation in the original report, and found some users comparing Arch's xz 5.6.1-1 vs. 5.6.1-2. By disassembling the liblzma library, it appears that the packages might have never been affected by the backdoor, due to the deb/rpm check in the script that decides whether to inject the vulnerability or not.

References:

https://www.openwall.com/lists/oss-secu … 4/03/29/17
https://www.openwall.com/lists/oss-secu … 4/03/29/20
https://www.openwall.com/lists/oss-secu … 4/03/29/22


This is the important bit.  There was not issue with Arch because the backdoor checked if xz was being built on an RPM or Deb based system before it was activated.  The rebuild is purely precautionary, and completely unneeded.

Offline

#19 2024-03-30 10:19:50

post
Member
Registered: 2015-02-15
Posts: 29

Re: xz Package Backdoor

Arch's repo also sees a somewhat remedying commit by Frederik on March 28; hours before Andres published their findings.

Offline

#20 2024-03-30 10:23:41

Fuxino
Member
From: Slovakia
Registered: 2014-09-26
Posts: 179

Re: xz Package Backdoor

If I understand correctly, this would only be a problem for a system with publicly accessible sshd? So it shouldn't affect most desktop users even if we did have the vulnerability (at least based on what is known right now about the backdoor). Good to upgrade (or downgrade!) anyway just as a precaution, though...

post wrote:

Arch's repo also sees a somewhat remedying commit by Frederik on March 28; hours before Andres published their findings.

I think distros were notified in advance before public disclosure.

Offline

#21 2024-03-30 10:47:50

ldarby
Member
From: UK
Registered: 2024-03-29
Posts: 14

Re: xz Package Backdoor

Fuxino wrote:

If I understand correctly, this would only be a problem for a system with publicly accessible sshd? So it shouldn't affect most desktop users even if we did have the vulnerability (at least based on what is known right now about the backdoor). Good to upgrade (or downgrade!) anyway just as a precaution, though...

sshd is just a known target of the attack, that's how this was found, we can't assume it's the only one. We will find out after experts (which I'm not) completey reverse engineer the payload, which will take some time.  E.g. it could target a web browser in some image display function (and I just checked that chromium does have /usr/lib/liblzma.so.5.6.1 loaded at run time, even though it's not shown by ldd), and trigger the payload by loading a special image (then you can attack large portions of the population by distributing the trigger image in an ad).

The payload that's activated only in rpm/deb builds is similarly just one known payload, so we should not assume it's the only one and that arch builds aren't affected, because it's not rpm/deb.

It seems unwise to me to continue running any code at all written by a known hostile actor, and one that already successfully pulled off one attack.

Offline

#22 2024-03-30 10:54:56

Fuxino
Member
From: Slovakia
Registered: 2014-09-26
Posts: 179

Re: xz Package Backdoor

ldarby wrote:

sshd is just a known target of the attack, that's how this was found, we can't assume it's the only one.

Yes, of course. Hence, I added "based on what is known right now" smile I'm also not an expert at all, just following along.

Last edited by Fuxino (2024-03-30 12:24:27)

Offline

#23 2024-03-30 11:34:29

dimich
Member
From: Kharkiv, Ukraine
Registered: 2009-11-03
Posts: 242

Re: xz Package Backdoor

github.com/tukaani-project/xz repository is deleted. It makes no sense to checkout sources in xz PKGBUILD from there.

Last edited by dimich (2024-03-30 11:43:42)

Offline

#24 2024-03-30 11:37:48

Awebb
Member
Registered: 2010-05-06
Posts: 6,286

Re: xz Package Backdoor

dimich wrote:

github.com/tukaani-project/xz repository is deleted. It makes no sence to checkout sources in xz PKGBUILD from there.

While the result is the same, it has not been deleted, but disabled by Github for TOS violations.

Offline

#25 2024-03-30 11:42:19

dimich
Member
From: Kharkiv, Ukraine
Registered: 2009-11-03
Posts: 242

Re: xz Package Backdoor

Sure, disabled. I misread github's message.

Offline

Board footer

Powered by FluxBB