Distributed building of packages

dp · 2004-02-03 15:12:49

building KDE takes about 13h on a usual computer (like mine) ... makeworld much longer :-(

what about a GRID to use over the whole arch-community to build the packages much faster than the usual way and help also the maintainers with resources? it would use only some resources of each of the users and we will have new versions of any pkg in hours

http://gridengine.sunsource.net/

for example is free

it can be also used by individual users to rebuild/configure e.g. a new kernel much faster

what do you think?

Xentac · 2004-02-03 15:39:40

There's a large security risk there. What if one of the users who's helping with the building inserts malicious code? Then everyone who uses the package is vulnerable.

How do we trust our users?

sarah31 · 2004-02-03 18:02:50

it is far easier just to have patience. do you absolutely need kde 3.2 to be functional in linux? abs is there for a reason. if you cannot wait then build it yourself.

dp · 2004-02-03 18:42:41

Xentac wrote:

There's a large security risk there. What if one of the users who's helping with the building inserts malicious code? Then everyone who uses the package is vulnerable.
How do we trust our users?

Q: What kind of security does Grid Engine use?

A: Grid Engine can be configured to accept only messages that are coming from special hosts and reserved port. This mechanism provides security comparable to that of rsh. For improved security installations an integration with Kerberos5 and DCE exists. This integration has not been used frequently and does not come with an out-of-the-box installation procedure. It still should be possible to get it working with a little effort if you are particularly concerned about security. In addition, a SSL based security framework prototype exists which represents an opportunity for an interesting and relatively small developement project toward a Grid Engine embedded security solution without a need for Kerberos or DCE.

a lot of financial, pharmaceutical and biotechnology companies use GRID's for supercomputing --- they are not trusting people who uses e.g. SETI@home or folding@home ... they more or less made sure that there can be no manipulation of data ... and if; the server is rejecting it --- very simple and working :-)

@ sarah31:
it's not about waiting to have kde compiled ... it's about the future of computing (check yourself, how much CPU do you use at the moment ... all under 40% is resource-unused)

Kin · 2004-02-03 20:15:16

Hi,

well we could try it at least...

Kin

sarah31 · 2004-02-03 22:24:05

dp wrote:

@ sarah31:
it's not about waiting to have kde compiled ... it's about the future of computing (check yourself, how much CPU do you use at the moment ... all under 40% is resource-unused)

my cpu is usually completely occupied with tasks. i contribute all unused cycles on all of my computers to a canadian protein folding project.

personally unless i know the credentials of those involved i would not trust one package built outside of the official team. besides those individuals would then be swamped with work from the community to check their work etc.

you would also have to ensure that each machine is set up in such a way as not to allow extraneous autodection .... and so forth.

dp · 2004-02-03 22:47:01

it's great that you offer your resources for protein-folding projects ... as a mol-bio student, i thank you in the name of all researchers on that subject :-)

thanx a lot for your arguments ... they give fundamental questions

what i found out is: main process is run on a main machine that sends then the computing-work to the GRID, that distributes it then to all clients ... no autodetection is done by the clients, they give only computing-power --- the main machine is doing the main thing; and outsourcing the "boring" jobs :-)

// i had a look at the technology of GRID's, but never ever i heard of anyone/anyproject that had tried to compile something with GRID-support --- but it could be very interesting matter; and as makepkg is really clean and well written, maybe it can be connected to a GRID more easily than something chaotically

theoretically you can build the whole xfree in about 5' (having about 100 clients @1GHz )

Arch Linux

#1 2004-02-03 15:12:49

Distributed building of packages

#2 2004-02-03 15:39:40

Re: Distributed building of packages

#3 2004-02-03 18:02:50

Re: Distributed building of packages

#4 2004-02-03 18:42:41

Re: Distributed building of packages

#5 2004-02-03 20:15:16

Re: Distributed building of packages

#6 2004-02-03 22:24:05

Re: Distributed building of packages

#7 2004-02-03 22:47:01

Re: Distributed building of packages

Board footer