[link-jack] Vendors are bad for security

phrakture · 2008-05-13 18:33:40

I read this today, and it really heightened the perception that Arch is doing The Right Thing(TM).

http://www.links.org/?p=327

Sjoden · 2008-05-13 20:04:38

I read about that earlier on /. It's kind of scary really. Maybe in the Beginners Guide, or Official Install Guide, we could have a "Read This" link, that points to the Philosophy of Arch, and examples like this. Then, "If you don't like this model, you won't like Arch"... Or something...

*edit*

Reading the comments on that article, it's kind of disappointing to see how many are, not defending but, aren't upset with Debian, or trying to just move along with out seeing that there is a big problem there.

Last edited by Sjoden (2008-05-13 20:10:18)

kensai · 2008-05-13 20:12:09

Everyday I'm convincing myself that vanilla is the way to go. Good to see that Aaron Griffin revisited how vanilla Arch was being and have decided to make it more vanila.

Last edited by kensai (2008-05-13 20:12:28)

PeteMo · 2008-05-13 23:39:53

To me, the key is not simply to stay vanilla. If you find something you think is wrong send it upstream for inclusion there instead of just patching your distro. That way, everybody benefits, and situations like this are avoided.

zodmaner · 2008-05-14 00:29:48

There are two important points from the article.
The first one is:

Firstly, vendors should not be fixing problems (or, really, anything) in open source packages by patching them locally - they should contribute their patches upstream to the package maintainers.

The second one is:

Secondly, if you are going to fix bugs, then you should install this maxim of mine firmly in your head: never fix a bug you don't understand.

Which, from my understanding, means: Made as little modification to the package as you can (preferably none) and report all bugs found upstream. If the package has a serious bug(s) and you made a fix for it, always send the patch upstream.

After reading the article, I'm glad that Arch Linux seems to have already been following these guideline, thanks to our philosophy and our fearless leader. However, as PeteMo said, there is nothing wrong with creating a patch to fix a problem, provided that we send the patch upstream.

Frem · 2008-05-14 01:56:54

As pointed out in the comments there, Debian *did* ask upstream a few times, and they were met with either silence or "seems OK to me".

dolby · 2008-05-14 02:07:10

What does *ask* upstream mean? You either *sumbit* upstream or you dont.

bender02 · 2008-05-14 02:23:23

Frem wrote:

As pointed out in the comments there, Debian *did* ask upstream a few times, and they were met with either silence or "seems OK to me".

From the comments, by the article writer:

* It seems that the Debian maintainer did, indeed, mention his plan on openssl-dev. Openssl-dev is a list for people developing OpenSSL based software, not a list for discussing the development of OpenSSL itself. I don't have the bandwidth to read it myself. If you want to communicate with the OpenSSL developers you need to use openssl-team@openssl.org. At no time, as people have suggested, was a patch offered to OpenSSL, and the discussion on openssl-dev was misleading.

So not quite the correct upstream...

cardinals_fan · 2008-05-14 04:26:53

Yikes. I'm an ex-Slackware user, and that really offends my Slack sensibilities . It's also true that FreeBSD has been guilty of this. Won't stop me from using (and enjoying) it, but it's a problem that needs addressing.

hussam · 2008-05-14 08:49:49

There are two points here:

1. "Vanilla" is now more secure than before. A few years ago, you HAD to backport fixes from cvs/svn/git risking regressions because upstream bug fix releases didn't arrive quickly. As Linux is more mainstream now, upstream bug fix releases arrive more quickly reducing the amount of fixes that have to be backported.
Still, backporting fixes written by upstream developers is recommended if we can't wait for the next upstream bug fix release or if the bug is a security issue.

2. Some distributions would create their own fixes and *not* send then upstream. This is *evil* and very risky. Most patches used in ArchLinux since I started using it in 2006 have been backports or written by upstream developers waiting to be merged. We never use 3rd party patches.
Also if an Archlinux developer does fix an upstream bug, he is most likely to submit it upstream.

Last edited by hussam (2008-05-14 08:50:17)

JGC · 2008-05-14 09:11:13

Sorry, but this OpenSSL issue is a bad example. The reason why this broke on Debian is because the OpenSSL PRNG is braindead anyways. What OpenSSL does is using uninitialized memory for random data in their PRNG. The contents of this memory don't have to be random: there's several C libraries around that use a constant string to identify uninitialized memory. On these platforms, OpenSSL is just as secure as the unfixed debian package.
The patch applied by Debian was discussed on the openssl mailinglists and nobody ever noticed this bug. To make things complete, the debian patch does the same thing as what is described in the FAQ about running openssl with valgrind. Compiling OpenSSL with -DPURIFY as advised in the OpenSSL FAQ results in the same bug.

ekerazha · 2008-05-14 10:50:26

never fix a bug you don't understand

Some Ubuntu devs should read this sentence several times.

Last edited by ekerazha (2008-05-14 10:52:41)

toofishes · 2008-05-14 11:59:49

JGC wrote:

Sorry, but this OpenSSL issue is a bad example. The reason why this broke on Debian is because the OpenSSL PRNG is braindead anyways. What OpenSSL does is using uninitialized memory for random data in their PRNG. The contents of this memory don't have to be random: there's several C libraries around that use a constant string to identify uninitialized memory. On these platforms, OpenSSL is just as secure as the unfixed debian package.
The patch applied by Debian was discussed on the openssl mailinglists and nobody ever noticed this bug. To make things complete, the debian patch does the same thing as what is described in the FAQ about running openssl with valgrind. Compiling OpenSSL with -DPURIFY as advised in the OpenSSL FAQ results in the same bug.

I thought the same thing before I read more about it.

What Debian did was comment out *two* different lines, one of which was uninitialized memory, the other of which was not. Why did they comment out two different lines? Because they looked the same and one of them did cause valgrind errors (the one surrounded by the #ifndef PURIFY block). However, it was commenting out the similar line in ssleay_rand_add() that caused the issue- they took the only true source of entropy (the passed in seed value) and completely discarded it. Oops!

So no, it wasn't just the uninitialized memory thing that caused the problem, that actually wasn't it at all. It was the overzealous commenting out of things that bit them here.

See the patch itself: http://svn.debian.org/viewsvn/pkg-opens … /md_rand.c . If you navigate to the actual revision of that file, you can see those updates take place in two different functions. The first update is the one that broke everything, the second one (the uninitialized memory fix) is not actually that big of deal.

Last edited by toofishes (2008-05-14 12:00:09)

Misery · 2008-05-15 12:29:52

Mail from Debian to OpenSSL: http://marc.info/?l=openssl-dev&m=114651085826293&w=2

New blog post: http://www.links.org/?p=328

dolby · 2008-05-15 12:43:37

Misery wrote:

Mail from Debian to OpenSSL: http://marc.info/?l=openssl-dev&m=114651085826293&w=2
New blog post: http://www.links.org/?p=328

Like explained in your second link the first link is useless.

The whole point is

I think that's about enough clarification. The question is: what should we do to avoid this happening again? Firstly, if package maintainers think they are fixing a bug, then they should try to get it fixed upstream, not fix it locally. Had that been done in this case, there is no doubt none of this would have happened.

Debian needs to know that they should ALWAYS submit their additions to the applications code upstream even if its a waste of time. They learned that the hard way.
Hope they learn their lesson from now on.

Misery · 2008-05-15 12:50:59

dolby wrote:

Like explained in your second link the first link is useless.

You don't need to click it. I like to have more info at once.

dolby · 2008-05-15 13:06:59

Misery wrote:

dolby wrote:
Like explained in your second link the first link is useless.
You don't need to click it. I like to have more info at once.

I meant, that it doesnt excuse the debian developers actions. not your posting it. but it was also a part of the original posters link

Misery · 2008-05-15 13:39:57

dolby wrote:

I meant, that it doesnt excuse the debian developers actions. not your posting it. but it was also a part of the original posters link

Ah, ok.... But it wasn't the same link. It's a new post of Ben Laurie in his blog.
Anyway..... it isn't important to talk about.

Maybe I should compile against gnutls instead of openssl.

B-Con · 2008-05-16 07:20:52

toofishes wrote:

What Debian did was comment out *two* different lines, one of which was uninitialized memory, the other of which was not. Why did they comment out two different lines? Because they looked the same and one of them did cause valgrind errors (the one surrounded by the #ifndef PURIFY block). However, it was commenting out the similar line in ssleay_rand_add() that caused the issue- they took the only true source of entropy (the passed in seed value) and completely discarded it. Oops!
So no, it wasn't just the uninitialized memory thing that caused the problem, that actually wasn't it at all. It was the overzealous commenting out of things that bit them here.
See the patch itself: http://svn.debian.org/viewsvn/pkg-opens … /md_rand.c . If you navigate to the actual revision of that file, you can see those updates take place in two different functions. The first update is the one that broke everything, the second one (the uninitialized memory fix) is not actually that big of deal.

I have to say, the mistake seems very, very naive. We're talking about OpenSSL here, not some small offbeat package that's the result of some college student's thesis project. Assuming you can glance over OpenSSL code and catch mistakes *that* obvious is arrogant. Just because one thinks that something should be different doesn't mean that they're right.

We're talking about a crypto library here. Commenting out initialization code without getting an explicit go-ahead from upstream is just not smart, especially for a "mistake" that sure seems like a curious mistake for the devs upstream to make. The fact that it's there is by default significant, which means that the cryptographer-wannabe at Debian should be trying to justify the upstream dev's decision to himself, not trying to out think them.

I'm not that familiar with Debian's dev hierarchy, but I would assume (hope...?) that, being who they are, that they would have devs who special in review security-oriented packages and who would have the know-how to not make such an obvious mistake.

There was no obligation to make the patch. No compile error. The library ran fine without it. The only problem was that the downstream non-specialized dev was second-guessing the specialized upstream dev and making changes without permission. Second-guessing is good, acting on wims is not. Commenting out the use of uninitialized memory wasn't harmful, but it wasn't helpful either. At worst, it added nothing -- who cares if its there?

Aye-yi-yi... I'm glad Arch does things the way it does.

Last edited by B-Con (2008-05-16 07:23:06)

peets · 2008-05-17 17:05:38

Good work Arch on doing The Right Thing, and being verbose and zealous about it so that more people can be enlightened. Plus, now I understand the latest xkcd!

Arch Linux

#1 2008-05-13 18:33:40

[link-jack] Vendors are bad for security

#2 2008-05-13 20:04:38

Re: [link-jack] Vendors are bad for security

#3 2008-05-13 20:12:09

Re: [link-jack] Vendors are bad for security

#4 2008-05-13 23:39:53

Re: [link-jack] Vendors are bad for security

#5 2008-05-14 00:29:48

Re: [link-jack] Vendors are bad for security

#6 2008-05-14 01:56:54

Re: [link-jack] Vendors are bad for security

#7 2008-05-14 02:07:10

Re: [link-jack] Vendors are bad for security

#8 2008-05-14 02:23:23

Re: [link-jack] Vendors are bad for security

#9 2008-05-14 04:26:53

Re: [link-jack] Vendors are bad for security

#10 2008-05-14 08:49:49

Re: [link-jack] Vendors are bad for security

#11 2008-05-14 09:11:13

Re: [link-jack] Vendors are bad for security

#12 2008-05-14 10:50:26

Re: [link-jack] Vendors are bad for security

#13 2008-05-14 11:59:49

Re: [link-jack] Vendors are bad for security

#14 2008-05-15 12:29:52

Re: [link-jack] Vendors are bad for security

#15 2008-05-15 12:43:37

Re: [link-jack] Vendors are bad for security

#16 2008-05-15 12:50:59

Re: [link-jack] Vendors are bad for security

#17 2008-05-15 13:06:59

Re: [link-jack] Vendors are bad for security

#18 2008-05-15 13:39:57

Re: [link-jack] Vendors are bad for security

#19 2008-05-16 07:20:52

Re: [link-jack] Vendors are bad for security

#20 2008-05-17 17:05:38

Re: [link-jack] Vendors are bad for security

Board footer