Arch Leadership

shining · 2007-10-05 21:05:15

codemac wrote:

Only if you can prove it would have equivalent or better speed. And at that point you'd probably be writing a lot of C extensions.. which to me makes the whole discussion kinda pointless Though if you wanna start the project off, go for it. Pacman is nifty in that the database is all happy and text files.

Is python that slow?
You can write stupid C code, and smart python code, C will still be faster?
I'm not saying pacman code is stupid, but in my opinion (and others), there are many places where it could use faster algorithms, and more efficient / adapted data structures.
So, I wonder if another language would really be that slower for the computation part.

But anyway, these parts are still small compared to the time needed to walk through these thousand files everytime..
A text database is very neat and useful indeed, but it could probably be optimized, maybe just by trying to reduce the number of files.

Also, as Nagy (an active guy on pacman-dev ML) noted previously, the runtime of pacman itself is often small compared to the time needed for ldconfig.
Like pacman, ldconfig probably has to walk through may files as well.
For example, I just did "pacman -S xterm --debug", it installed xterm in less than one second, but ldconfig took several seconds to complete.

Anyway, for having read pacman's code a bit, I think it would be great if the code complexity could be cut down by using an higher level language,
while trying to keep similar features and performance.
Though, having pacman in C limits the number / size of dependencies, so less likely to break, especially when considering that it allows pacman.static to be built.
This advantage was quickly mentioned on the ML, but it's a big one :
http://www.archlinux.org/pipermail/pacm … 09007.html

Cerebral · 2007-10-06 01:33:14

shining wrote:

Is python that slow?
You can write stupid C code, and smart python code, C will still be faster?
I'm not saying pacman code is stupid, but in my opinion (and others), there are many places where it could use faster algorithms, and more efficient / adapted data structures.
So, I wonder if another language would really be that slower for the computation part.

Well, that's not really an argument for switching language, that's an argument for making the algorithms and data structures better.

shining wrote:

Anyway, for having read pacman's code a bit, I think it would be great if the code complexity could be cut down by using an higher level language, while trying to keep similar features and performance.
Though, having pacman in C limits the number / size of dependencies, so less likely to break, especially when considering that it allows pacman.static to be built.

I recall phrakture talking about really really wanting to rewrite it in C++, and Dan was recently talking about a rewrite of libalpm as well. Who knows what's in the future for pacman?

I think an interpreted language might be a bad idea for it though - the advantages of a compiled language suit a package manager. If pacman is written in, say, python, and you run pacman -Syu and you get a new version of python that accidentally has something broken, suddenly pacman won't work anymore, no matter how hard you try.

If you grab a borked glibc, pacman.static should still work. That's a big bonus, as you mentioned.

veek · 2007-10-06 14:36:48

Sometimes I wondered about the creator of Arch, and all this time you were on the forum as apeiro, had no idea!

Thanks for a fantastic distro judd. Your vision made the community what it is today.

codemac · 2007-10-06 14:52:44

shining wrote:

Is python that slow?

Not usually, but compared to C, yeah.

shining wrote:

You can write stupid C code, and smart python code, C will still be faster?

You could also just write smart C code.

shining wrote:

I'm not saying pacman code is stupid, but in my opinion (and others), there are many places where it could use faster algorithms, and more efficient / adapted data structures.

There is no data structure that is more efficient than "int x". This is a silly argument in general as it doesn't matter what language you use to implement data structures.

shining wrote:

So, I wonder if another language would really be that slower for the computation part.

I wonder too... go implement it and we can test it out.

shining wrote:

But anyway, these parts are still small compared to the time needed to walk through these thousand files everytime..
A text database is very neat and useful indeed, but it could probably be optimized, maybe just by trying to reduce the number of files.

Use a loopback file and it's realllly fast. You could actually put this loopback file _into_ your ram, and just remember to write it back to your fs after pacman runs. Then it will be quite fast.

shining wrote:

Also, as Nagy (an active guy on pacman-dev ML) noted previously, the runtime of pacman itself is often small compared to the time needed for ldconfig.
Like pacman, ldconfig probably has to walk through may files as well.
For example, I just did "pacman -S xterm --debug", it installed xterm in less than one second, but ldconfig took several seconds to complete.

Great! So you want to go with an sql database then? Name something more KISS than the UNIX style files, and I could possibly get behind it.

shining wrote:

Anyway, for having read pacman's code a bit, I think it would be great if the code complexity could be cut down by using an higher level language,
while trying to keep similar features and performance.
Though, having pacman in C limits the number / size of dependencies, so less likely to break, especially when considering that it allows pacman.static to be built.
This advantage was quickly mentioned on the ML, but it's a big one :
http://www.archlinux.org/pipermail/pacm … 09007.html

So as you seem to be able to reason out in your own post, the real issue is that we don't have empirical evidence to support either conclusion. So go write it in python, and we'll compare. I personally don't think it will be faster, but I'll submit patches etc.. if it gets started off. I'm not going to criticize someone for trying something. I think it's the best part of OSS that you have a choice to do whatever you want.

So go do it.

arooaroo · 2007-10-06 16:39:34

The language speed debate is a bit pointless, at the moment at least due to some design deficiencies in pacman's database architecture. No one's debating that C ain't fast - of course it's quicker than Python, although for many tasks, is it discernibly quicker? I mean do you really notice the difference between 0.001s and 0.01s?

Pacman is like a Ferrari - the code is nice and tight and super-quick. However, the organisation of the package database is equivalent to driving said supercar through the streets of central London - it doesn't matter what the top-speed of your car is, you ain't going much faster than an average 15mph.

I always noticed the first run of pacman was pretty slow. Then I changed my file system from Reiser3 to Ext3 and then it was even slower, I mean, it took ages to do a pacman -Syu and wait for the download list. Whilst the database design is simple, and we like a bit of KISS here at Arch, it's not efficient and so at the moment, we could have pacman in any language and it wouldn't really matter because each port would still be idling away whilst the thousands of IO operations went on and on and on...

Now our Ferrari may be nice and slick, and there's definitely a certain amount of cachet just from possessing one, regardless of whether you're being overtaken by a cyclist. And that's the same with pacman - it makes us feel all warm and special that our PM is in super-fast C. But there's a massive cost: writing C is expensive; debugging C is expensive; experimenting in C is expensive.

The problem with KISS is that it's like the bible, you can cherry-pick when it suits you to support any argument. For me, writing in a high-level scripting language such as Python is KISS - the code is much clearer, easier to debug, the developers are productive - the code is ultimately more accessible and would probably see greater interaction from the community. It's easier to experiment with. For others, KISS means keeping pacman written in C and ensuring it's fast (yet, as I mentioned, it's still slow overall) and lightweight, and perhaps a big factor: no dependencies. Either way, don't bother me with your C is faster than <insert other language here> because choice of language is not the issue.

Now I think it's about time a mod split this thread.

Dusty · 2007-10-06 17:41:16

arooaroo wrote:

Now I think it's about time a mod split this thread.

I agree, but I can't. punbb default install doesn't support it.

If anyone wants to compare pacman to python pacman, take a look at http://home.gna.org/libpypac/ It might not be a fair comparison because this was Jens' reimplementation of pacman2 and its old, but it could be a start.

I'm only guessing here, but I bet if you profiled pacman, you'd find that the most amount of time is spent in file i/o and searching. I also suspect that file i/o is pretty much equally fast between python and c. Searching in Python is also very efficient. The only valid argument for not implementing in Python above is the python dependency one; it really is better to have a stand-alone package manager.

However, I'd be keenly interested to see a python version developed and maintained and may even contribute a little to the project. ;-) Something tells me it could be reimplemented in python more quickly by fewer people than it could be reimplemented in C or C++ (unless phrakture was doing the C++ version single handed, but its not fair to compare the likes of him to mere mortals).

Dusty

loserMcloser · 2007-10-07 16:39:00

Thank you Judd.

ekerazha · 2007-10-07 19:23:57

I vote for the current C Pacman (Python is slow, it has a stupid syntax and it requires python as dependency), but with a decent and flexible backend (sqlite) instead of the current "flat files" mess. However I'll survive also with the current, bad-designed backend.

Last edited by ekerazha (2007-10-07 19:51:00)

weseven · 2007-10-07 20:41:11

so long, and thanks for all the fish.
actually, thank you for creating the fish, Judd.

ihavenoname · 2007-10-10 01:19:47

Well, I think everyone has really said it pretty well, thanks for all your work. Looking back I think that a majority of my linux knowelage has been obtained here using arch linux, and browsing the forum and the wiki. It goes with out saying that you have not only played a role in creating a wonderful distro, but you have also created an excellent Linux learning tool, for work and play. Good Luck and happy sailing.

and now, for a new beginning. Aaron, everyone good luck to you.

As Robert Frost once said, "The woods are lovely, dark and deep. But I have promises to keep, and miles to go before I sleep." Let's get to it people. We have kernels to compile, and recompile because we realized we forgot to enable fbsplah.

drum · 2007-10-10 12:54:40

I've only been enjoying Arch for 6 months but I've got one big question...

Is there Arch after death?

Congratulations to all.

wantilles · 2007-10-10 14:35:34

Having passed 2.5 years from Gentoo and 0.5 years from Debian sid, the last six Arch months have provided by far the best Linux experience for me.

So congratulations to the new leader.

mclang · 2007-10-10 15:18:12

I as well want to express my gratitude to you Judd

Past couple of years has been very good with Arch. Not always easy, but always educational and inspiring.

And congrats phrakture!

cactus · 2007-10-12 03:12:05

Thanks again Juddster.

Welcome overlord Aaron:

nesrecar · 2007-10-22 10:21:49

Thanks for all your work Judd, you leave with successful work and fists in the air. ) Congratulations Aaron. As I told you in IRC, hope you come with the other Arch Devs still to Linuxtag. We have to throw some yofuhs (God, he'll kill me)

farhat · 2007-10-22 20:47:22

ekerazha wrote:

I vote for the current C Pacman (Python is slow, it has a stupid syntax and it requires python as dependency), but with a decent and flexible backend (sqlite) instead of the current "flat files" mess. However I'll survive also with the current, bad-designed backend.

stupid syntax?

desertViking · 2007-10-22 23:51:36

Judd, thanks for the great beginning. I really enjoy what you started, and, because of the wonderful community, what Arch Linux has become.

Phrakture, thanks for taking the leadership role. Always easy to kibitz from the back row, but it takes some hair to stand out in front. Looking forward to seeing where the ship sails. Should be a wonderful adventure.

dViking

Trac3R · 2007-10-23 16:00:44

Thanks Judd, you've started the best Linux Distro ever. I have no words that could discribe how much I appreciate for this. Nevertheless I hope we're still going to hear a lot from you.

VincentV · 2007-12-11 19:24:27

hey Judd,
Wow I need to hit these forums more often... Thanks man, Thanks Thanks Thanks! I absolutely love Arch. I work in a research department for a largish university & I get to play with lots of distros on boxes from laptops to desktops to clusters, and I by far prefer Arch. It's really nice to have most of my problems caused by my own stupidity rather than the folks putting out the distro. I was trying every Linux I could for a while until I ran across Arch about 2-2.5 years ago & it was love at first login, Arch & its ways just makes so much sense to me. Anyway, thank you so much for your ideas, your work, your love of Arch over the last few years. May you have great success and enjoyment in whatever is to come. And best of luck to you Phrakture!

Toad39 · 2020-08-04 20:30:45

Thank you, both of you, for all your hard work into ArchLinux.

2ManyDogs · 2020-08-04 22:20:48

Closing this ancient thread.

Arch Linux

#126 2007-10-05 21:05:15

Re: Arch Leadership

#127 2007-10-06 01:33:14

Re: Arch Leadership

#128 2007-10-06 14:36:48

Re: Arch Leadership

#129 2007-10-06 14:52:44

Re: Arch Leadership

#130 2007-10-06 16:39:34

Re: Arch Leadership

#131 2007-10-06 17:41:16

Re: Arch Leadership

#132 2007-10-07 16:39:00

Re: Arch Leadership

#133 2007-10-07 19:23:57

Re: Arch Leadership

#134 2007-10-07 20:41:11

Re: Arch Leadership

#135 2007-10-10 01:19:47

Re: Arch Leadership

#136 2007-10-10 12:54:40

Re: Arch Leadership

#137 2007-10-10 14:35:34

Re: Arch Leadership

#138 2007-10-10 15:18:12

Re: Arch Leadership

#139 2007-10-12 03:12:05

Re: Arch Leadership

#140 2007-10-22 10:21:49

Re: Arch Leadership

#141 2007-10-22 20:47:22

Re: Arch Leadership

#142 2007-10-22 23:51:36

Re: Arch Leadership

#143 2007-10-23 16:00:44

Re: Arch Leadership

#144 2007-12-11 19:24:27

Re: Arch Leadership

#145 2020-08-04 20:30:45

Re: Arch Leadership

#146 2020-08-04 22:20:48

Re: Arch Leadership

Board footer