Clustering

Ryujin · 2006-07-18 16:58:23

I am looking for information on clustering under Arch Linux, Does anyone know where I should start?

Kern · 2006-07-18 19:08:37

could you expand a bit on what you require? server, nodes, jobs etc.

ive checked a lot around clustering recently, and most seem to be their own distro specifically aimed at certain types of job.

one i found, iirc, that is a kernel patch for clustering, not an entire distro in its own right.
of course, now i need it, i cant find the link.
will get back later.

=====edit===
here'tis tho ive not tried this yet, its the way id prefer to go if running progs from inside Arch.

http://www.faqs.org/docs/Linux-HOWTO/op … TRIBUTIONS

4.8. Other distributions
Based on the explanations above you should be able to install openMosix on most other Linux platforms.

AndyRTR · 2006-07-18 20:24:23

http://wiki.archlinux.org/index.php/Distcc

someting like that?

Ryujin · 2006-07-21 17:17:03

Sorry, I guess I don't know what I want, moreso some information on the subject so I can get cooking.

ben017f · 2006-07-25 19:31:48

It's not clustering exactly, but if it's for web development, then have a look at zope/plone and particularly ZEO...

Zope Enterprise Objects is essentially a really neat way to scale zope. Basically you have a ZEO server running which is your central object database, and then zope clients connect to it to serve the content. Only object that have changed in the ZEO server or Zope instance actually have to sent therefore reducing load and network traffic.

I had a play with it a couple of days ago, and within 30mins I had 2 zope instances running the same plone site on different ports but with the same content... this could be insanely usefull if you want to change anything as you could have one running (behind apache say) as your production server, and the other one could be in debug mode.

Checkout The Zope Book and ZEO programming for some more details.

soloport · 2006-07-25 21:46:46

You can cluster servers by asking each to do a specific task that is a subset of a much larger task -- like compiling and linking a huge software project (think Windows or Linux) over several servers -- say, using distcc.

You can cluster services, such as Postgres, so that several servers all look like a single mega-server, handling database queries at the speed of light. (Also refered to as "load balancing".)

Beyond clustering, there's distributed computing. Every time a page loads from Amazon, for example, the images are loaded from who knows what or how many other servers -- distributed all over the world. When Amazon's web site designers make a change, it has to propagate all over the world. If the change is signifficant, the sum has to reach the servers at the same time -- otherwise, the graphics would not coinside with the HTML, JavaScript, etc.

V01D · 2006-07-29 23:01:34

I'm currently developing a system to be able to Kickstart archlinux just like redhat. I'm going to use this for the Cluster I'm administrating at work, an HPC Cluster.

I'm also creating PKGBUILDs for the following packages (altough they're a bit ad-hoc to the needs at work, so I'm not sending them to AUR): gridengine, ganglia, gaussian, amber, intel compiler and mkl libraries, siesta and hybrid.

For now you can check a wiki page I started to describe my intentions: http://wiki.archlinux.org/index.php/Kickstart.

chrismortimore · 2006-10-30 13:37:05

soloport wrote:

Beyond clustering, there's distributed computing. Every time a page loads from Amazon, for example, the images are loaded from who knows what or how many other servers -- distributed all over the world. When Amazon's web site designers make a change, it has to propagate all over the world. If the change is signifficant, the sum has to reach the servers at the same time -- otherwise, the graphics would not coinside with the HTML, JavaScript, etc.

I'm after this one if anyone has any information on it. I used to run openmosix (back in my 2.4 days), but my latest boxes need 2.6. I use it to do distributed solving of maths problems, and circuit simulation, and other such things.

V01D · 2006-10-30 16:18:04

chrismortimore wrote:

soloport wrote:
Beyond clustering, there's distributed computing. Every time a page loads from Amazon, for example, the images are loaded from who knows what or how many other servers -- distributed all over the world. When Amazon's web site designers make a change, it has to propagate all over the world. If the change is signifficant, the sum has to reach the servers at the same time -- otherwise, the graphics would not coinside with the HTML, JavaScript, etc.
I'm after this one if anyone has any information on it. I used to run openmosix (back in my 2.4 days), but my latest boxes need 2.6. I use it to do distributed solving of maths problems, and circuit simulation, and other such things.

Then you'll probably want to look for HA (High Availability) and Virtual Servers. The kernel has some features relating to this, but I think there are various software packages that provide this system.

rsantiba · 2010-06-30 15:57:41

I have 3 DELL PowerEdge servers by install a cluster. Does have anyone a suggestion? :-P

JackH79 · 2010-06-30 23:32:50

Maybe have a look at this: https://computing.llnl.gov/linux/index.html

They've got some good information and a large collection of software to download.

.:B:. · 2010-07-01 00:35:42

This thread is four years old. Please open one of your own.

http://wiki.archlinux.org/index.php/For … Bumping.27

Arch Linux

#1 2006-07-18 16:58:23

Clustering

#2 2006-07-18 19:08:37

Re: Clustering

#3 2006-07-18 20:24:23

Re: Clustering

#4 2006-07-21 17:17:03

Re: Clustering

#5 2006-07-25 19:31:48

Re: Clustering

#6 2006-07-25 21:46:46

Re: Clustering

#7 2006-07-29 23:01:34

Re: Clustering

#8 2006-10-30 13:37:05

Re: Clustering

#9 2006-10-30 16:18:04

Re: Clustering

#10 2010-06-30 15:57:41

Re: Clustering

#11 2010-06-30 23:32:50

Re: Clustering

#12 2010-07-01 00:35:42

Re: Clustering

Board footer