You are not logged in.

#1 2018-11-24 11:34:04

mizuchi
Member
Registered: 2017-03-09
Posts: 14

Message Passing Interface

Hi all!

I'd like to know, is there any reason for Arch community to prefer OpenMPI over other implementations such as MPICH? Why is OpenMPI in EXTRA, while MPICH is in AUR?
I think it's worth to give users a choice what implementation to use making Arch MPI-agnostic. Can it be done?

Anton K.

Offline

#2 2018-11-24 11:56:55

Allan
Pacman
From: Brisbane, AU
Registered: 2007-06-09
Posts: 11,365
Website

Re: Message Passing Interface

OpenMPI is in [extra] because a Developer wants it there.  MPICH is not in the repos because no Developer or Trusted Users wants it there...

Offline

#3 2018-11-24 12:23:23

mizuchi
Member
Registered: 2017-03-09
Posts: 14

Re: Message Passing Interface

Allan wrote:

OpenMPI is in [extra] because a Developer wants it there.

Can the developer explain his decision?

Offline

#4 2018-11-24 12:56:58

loqs
Member
Registered: 2014-03-06
Posts: 17,194

Re: Message Passing Interface

https://git.archlinux.org/svntogit/comm … f81cd68556 and stephane is retired.
Whatever the reason for OpenMPI's inclusion that does not address "MPICH is not in the repos because no Developer or Trusted Users wants it there..."

Offline

#5 2018-11-25 16:34:14

Archange
Package Maintainer (PM)
Registered: 2014-08-17
Posts: 40

Re: Message Passing Interface

I’ve opened a FR on the bug tracker (https://bugs.archlinux.org/task/60909) regarding the technical difficulties of providing both in the repo, so that we can gather information and knowledge about this in a dedicated place. I also believe we should let user choose, but that is not easy.

Offline

#6 2018-11-25 18:26:26

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 29,442
Website

Re: Message Passing Interface

Archange, are you considering maintaining the other alternative?  If so, great.  But if not (or really in either case) I think the terminology of "letting the user choose" is misleading.  It's not as if the present state of things is a deliberate constraint placed on the user's choice.  Rather we "let the developers/TUs choose."  They choose whether or not they want to maintain a package, and no one has chosen to maintain MPICH.


"UNIX is simple and coherent..." - Dennis Ritchie, "GNU's Not UNIX" -  Richard Stallman

Offline

#7 2018-11-25 19:03:53

Archange
Package Maintainer (PM)
Registered: 2014-08-17
Posts: 40

Re: Message Passing Interface

Yes, I’d like to provide both OpenMPI and MPICH in the repo. I’m just not sure which way to go.

Offline

#8 2018-11-25 19:12:11

eschwartz
Fellow
Registered: 2014-08-08
Posts: 4,097

Re: Message Passing Interface

The technical difficulties of providing both seem to be "either choose one and stick to it" or "recompile hundreds or thousands of packages to give 'user choice' in multiple variants of large parts of the entire distribution stack".

Recompiling two versions of every single package with any degree of indirect linkage against something which provides hacked-up sonames like libfoo-openmpi is so utterly insane that I'm not even sure why it was mentioned even as a joke.

I'm against any solution other than "pick one and stick to it", anyway.


Managing AUR repos The Right Way -- aurpublish (now a standalone tool)

Offline

#9 2018-11-25 20:00:15

loqs
Member
Registered: 2014-03-06
Posts: 17,194

Re: Message Passing Interface

If the request is limited to have packages compiled with one and make the other available that seems more manageable.

Offline

#10 2018-11-25 21:42:03

mizuchi
Member
Registered: 2017-03-09
Posts: 14

Re: Message Passing Interface

There are no hundreds or thousands packages to recompile; as it given on the page, openmpi is required by 10 (actually 9) packages and 9 more need it optionally.

Last edited by mizuchi (2018-11-25 22:26:25)

Offline

#11 2018-11-25 21:54:19

loqs
Member
Registered: 2014-03-06
Posts: 17,194

Re: Message Passing Interface

Are you referring to gap-4.8 not supporting mpi but listing openmpi as a dependency as the difference between 9 and 10?

Offline

#12 2018-11-25 22:21:25

Archange
Package Maintainer (PM)
Registered: 2014-08-17
Posts: 40

Re: Message Passing Interface

No, I think he meant python{,2}-mpi4py. And they are only 3 more that need it optionally, everything else is makedep amongst which 3 of the 6 are the optional ones, 2 are for split packages already listed before, and only one is apparently relevant (dmtcp). Note that because of the current transient dependency rule, some might not have been listed, but I’ve checked and that is not the case here.

Also “the page” was meant to be https://www.archlinux.org/packages/extr … 4/openmpi/.

Offline

#13 2018-11-25 22:28:51

mizuchi
Member
Registered: 2017-03-09
Posts: 14

Re: Message Passing Interface

Yes, you're quite right, I meant python{,2}-mpi4py. Fixed the link to "the page".

Offline

#14 2018-11-25 22:40:29

loqs
Member
Registered: 2014-03-06
Posts: 17,194

Re: Message Passing Interface

I missed gap-4.8 does support MPI anyway it is gap that no longer does.

Offline

#15 2018-11-25 22:40:59

eschwartz
Fellow
Registered: 2014-08-08
Posts: 4,097

Re: Message Passing Interface

mizuchi wrote:

There are no hundreds or thousands packages to recompile; as it given on the page, openmpi is required by 10 (actually 9) packages and 9 more need it optionally.

The "hundreds of packages to recompile" thing would be if we built two copies of each of those 10, and then built two copies of each package that requires one of those 10 (now 20), and so on and so forth.

It's exclusive to the suggestion that we allow people to co-install, for example, arpack-openmpi and arpack-mpich, and then also have octave, which requires arpack, in your choice of octave-openmpi and octave-mpich, and then limesuite which depends on octave, in your choice of limesuite-openmpi and limesuite-mpich, then repeat ad nauseam for urh and sdrangel which depend on limesuite...


Managing AUR repos The Right Way -- aurpublish (now a standalone tool)

Offline

#16 2018-11-25 22:42:43

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 29,442
Website

Re: Message Passing Interface

eschwartz wrote:

The "hundreds of packages to recompile" thing would be if we built two copies of each of those 10, and then built two copies of each package that requires one of those 10 (now 20), and so on and so forth.

It's exclusive to the suggestion that we allow people to co-install, for example, arpack-openmpi and arpack-mpich, and then also have octave, which requires arpack, in your choice of octave-openmpi and octave-mpich

Or we'd have arpack-openmpi and arpack-mpich which each provide arpack which is all that octave would depend on.


"UNIX is simple and coherent..." - Dennis Ritchie, "GNU's Not UNIX" -  Richard Stallman

Offline

#17 2018-11-25 22:47:26

Archange
Package Maintainer (PM)
Registered: 2014-08-17
Posts: 40

Re: Message Passing Interface

@eschwartz Right, but we shouldn’t go this path. My proposition is to have:

1. OpenMPI and MPICH themselves co-installable (you might want to check your code against both for instance).
2. Packages depending on MPI available in both flavours, that is e.g. arpack-openmpi and arpack-mpich, but having those conflicts each other (and provide arpack).

@Trilby What @eschwartz says is that you cannot have arpack-openmpi and arpack-mpich co-installable and other packages being able to depend on either.

Offline

#18 2018-11-25 22:49:43

eschwartz
Fellow
Registered: 2014-08-08
Posts: 4,097

Re: Message Passing Interface

Trilby wrote:

Or we'd have arpack-openmpi and arpack-mpich which each provide arpack which is all that octave would depend on.

Which one would it depend on, if we went for "user choice" by allowing you to install both at the same time?


Managing AUR repos The Right Way -- aurpublish (now a standalone tool)

Offline

#19 2018-11-25 23:04:02

mizuchi
Member
Registered: 2017-03-09
Posts: 14

Re: Message Passing Interface

It would depend on arpack which comes with 2 providers: arpack-mpich and arpack-openmpi. One cannot has both installed at the same time.

Last edited by mizuchi (2018-11-25 23:09:18)

Offline

#20 2018-11-25 23:11:23

Archange
Package Maintainer (PM)
Registered: 2014-08-17
Posts: 40

Re: Message Passing Interface

@mizuchi That is not what @eschwartz is discussing. He was especially talking about the case were we would make everything co-installable.

Offline

#21 2018-11-25 23:25:16

mizuchi
Member
Registered: 2017-03-09
Posts: 14

Re: Message Passing Interface

Sorry, I missed the point. Is it reasonable to have all packages depending on MPI co-installable? Sounds like tons of work.

Offline

#22 2018-11-25 23:29:25

Archange
Package Maintainer (PM)
Registered: 2014-08-17
Posts: 40

Re: Message Passing Interface

No, it is indeed not. It was mentioned as a possibility, and the first people to react were telling to no go the path of two conflicting stacks. But after those discussions it seems much more reasonable to go for conflicting stacks. I will package mpich and write a proposal in that direction asap (likely later this week).

Offline

#23 2018-11-25 23:40:27

Allan
Pacman
From: Brisbane, AU
Registered: 2007-06-09
Posts: 11,365
Website

Re: Message Passing Interface

Allan wrote:

OpenMPI is in [extra] because a Developer wants it there.  MPICH is not in the repos because no Developer or Trusted Users wants it there...

Given that situation has not changed at all...   This discussion is pointless.   

The way things actually happen in Arch, is for people to just do it themselves.  Compile a repo MPICH and all needed software variants to show what this requires.   Then a Dev or TU might become interested, or you would be in a position to apply to be a TU yourself.


Edit:  and while this sat being composed while I had coffee and meetings, a TU became interested....

Offline

#24 2018-11-25 23:42:24

Archange
Package Maintainer (PM)
Registered: 2014-08-17
Posts: 40

Re: Message Passing Interface

Allan wrote:
Allan wrote:

OpenMPI is in [extra] because a Developer wants it there.  MPICH is not in the repos because no Developer or Trusted Users wants it there...

Given that situation has not changed at all...   This discussion is pointless.   

The way things actually happen in Arch, is for people to just do it themselves.  Compile a repo MPICH and all needed software variants to show what this requires.   Then a Dev or TU might become interested, or you would be in a position to apply to be a TU yourself.

Because I don’t count as a TU wanting it in the repo?

Offline

#25 2018-11-26 00:01:55

Allan
Pacman
From: Brisbane, AU
Registered: 2007-06-09
Posts: 11,365
Website

Re: Message Passing Interface

Archange wrote:

Because I don’t count as a TU wanting it in the repo?

See my edit...   Hazards of leaving the compose window open too long.

Edit: and I missed it was you that opened the bug report, and not the OP.   Better finish coffee before posting...

Offline

Board footer

Powered by FluxBB