You are not logged in.

#1 2013-01-04 04:30:01

Xyne
Moderator/TU
Registered: 2008-08-03
Posts: 5,695
Website

Who is interested in creating an [arch-science] team?

I have been toying with this idea for a while.

There is a lot of academic and professional open source scientific software for Linux. There is already quite a lot of packages available on Arch, e.g.

There are several more packages in the repos.

There are some problems though:

  • many AUR packages are orphaned and need to be updated

  • many upstream projects are dead (this creates problems for dependencies and a low signal-to-noise ratio)

  • package quality varies and the ecosystem is fragmented (related packages are maintained independently, provides and conflicts arrays are not complete, etc.)

  • some projects are very difficult to package and not many individual users have the time do maintain them

There are nevertheless several users whose names reappear when looking through the scientific packages and whose packages are of good quality. What I would like to do is try to bring together new and existing packagers in a team to accomplish the following on Arch:

  • Collaborate on and share responsibility for packages: instead of having individual packagers who burn out and give up on problematic packages, we could work together and overcome problems collectively. This may lead to some bikeshed discussion but overall it should help to improve the quality of the packages (compared to some of the hack-n-slash kludgefests currently employed to package certain nasty projects).

  • Maintain categorized lists of currently working packages so users can quickly find what they need for given tasks. I would also like to maintain a "most wanted"/todo list for big projects that we have not yet been able to package.

  • Search for new/unknown projects that we can package and spread. It seems quite common for good projects to remain obscure outside of the university or company where they were born even when they are fully open. A team of scientifically-oriented users would be able to evaluate such projects and determine if they are worth packaging.

  • Maintain a scientific repo. Some popular packages will end up in the community repo but many scientific packages will be of limited utility to most users although invaluable to a few. Providing packages in our own repo will be convenient for users, especially for the packages that require a lot of time to compile

I haven't given the details much though yet. For collaboration we would need a mailing list and/or forum. I could provide the latter on xyne.archlinux.ca. I would like to use Git and probably Github for maintaining PKGBUILDs and related files. I could eventually host the pacman repo as well.

I don't know if it would be worth bothering with AUR packages. That issue came up in the arch-haskell group before and ultimately I think it is better to be completely independent instead of hoping that random users on the AUR will play nice. By "independent" I simply mean that we would provide a way to access the PKGBUILDs elsewhere. It would probably be as simple as creating an abs-like tool to pull them from Github or wherever we host them.

Finally, time is unfortunately an issue for me at the moment, so I am not expecting this to move forward very quickly if it starts moving at all, but even a slow start should be able to gather momentum.

So, who is interested in this?

Last edited by Xyne (2013-01-04 18:59:11)

Offline

#2 2013-01-04 05:18:26

mcmillan
Member
Registered: 2006-04-06
Posts: 694

Re: Who is interested in creating an [arch-science] team?

I'm interested, having some organization like this would be a good motivator for me to learn a few new things that I can contribute.

Time is a bit tight for me as well, but I'm settling in to a new job and should have more time in the near future.

Offline

#3 2013-01-04 05:29:09

tdy
Member
From: Sacremende
Registered: 2008-12-14
Posts: 438

Re: Who is interested in creating an [arch-science] team?

Xyne wrote:

Finally, time is unfortunately an issue for me at the moment, so I am not expecting this to move forward very quickly if it starts moving at all, but even a slow start should be able to gather momentum.

Yep, I'm thinking time will be the limiting factor for everyone.

Xyne wrote:

So, who is interested in this?

But +1 to the idea. You pretty much nailed all the points, so I don't have much to add. I'm also interested to hear what others think. From a personal standpoint, I don't know if I would gain too much from an [arch-science] per se. The scientific packages that I use heavily are already in the repos (octave, R, and to a lesser extent scipy), but people from other fields might benefit a lot (especially if the votes are even remotely accurate).

Offline

#4 2013-01-04 07:05:40

Giniu
Trusted User (TU)
From: Wrocław/Poland
Registered: 2007-04-02
Posts: 61
Website

Re: Who is interested in creating an [arch-science] team?

Xyne wrote:

Finally, time is unfortunately an issue for me at the moment

same here, but - that's why teaming up can give us better results than current scattered approach

Xyne wrote:

So, who is interested in this?

Count me in!


My GPG fingerprint: 7170 26A9 D477 9FC5 3940  7266 40F5 57B7 3149 6106

Offline

#5 2013-01-04 15:09:07

gborzi
Member
From: Sicilia
Registered: 2009-03-18
Posts: 102

Re: Who is interested in creating an [arch-science] team?

I'd be glad to partecipate in this project. IMHO, the packages should be made available on AUR. It would be nice to have an automatic upload of the source package to AUR from the prospective Arch scientific site/repo. Note that some packages that belong to the lib category are also in the scientific/engineering category. For example, atlas-lapack, openblas and opencascade.
I suggest that the members of this group declare their scientific field of interest, so that the packages are assigned accordingly. I work on Computational Electromagnetics, hence my interest in gmsh, openblas, spooles.


Giuseppe Borzi' - Registered Linux user #34028

Offline

#6 2013-01-04 20:20:04

Nepherte
Member
From: Belgium
Registered: 2008-09-09
Posts: 427
Website

Re: Who is interested in creating an [arch-science] team?

I am willing to contribute as well. I currently host my own repository for these purposes.

I maintain a couple of scientific-related packages such as a few compilers, support vector machine (SVM) implementations and natural language processing tools. I believe these are suitable candidates:

  • mercury-compiler

  • smarteiffel

  • svm_light

  • svm_rank

  • timbl, mbt, frog (computational linguistic tools)

Offline

#7 2013-01-04 20:37:33

IncredibleLaser
Member
From: Germany, NRW
Registered: 2008-07-16
Posts: 155

Re: Who is interested in creating an [arch-science] team?

I had a similiar idea 3 years ago when I was still studying. Unfortunately, I didn't have the time back then to do this and nowadays, I have no real use for it. I'd be glad to help though, just for learning to setup and maintain a repo, compile in a clean environment etc. - I'd like to try participating in my spare time.

Offline

#8 2013-01-04 21:33:54

kristof
Member
From: Sweden
Registered: 2012-10-03
Posts: 5

Re: Who is interested in creating an [arch-science] team?

I'm interested. A couple of things I'd like to see happen in the short term:

-A list of packages and/or applications that will be part of the "arch-science" collection.
-A list of users who have decided to volunteer
-A few users who have taken the initiative to be the leaders of this project. De-facto leader right now, I think, is Xyne.

This information needs to exist in a centralized location, so I think that should go on Xyne's forum.

For the purposes of this thread, though, I think Xyne should append and regularly update a list of volunteers. smile

Offline

#9 2013-01-04 22:00:47

mickele
Member
Registered: 2008-07-08
Posts: 12
Website

Re: Who is interested in creating an [arch-science] team?

I'm interested too.
Roadmap of krostof is good.
What about creating a repo on github?

Offline

#10 2013-01-04 22:50:12

Awebb
Member
Registered: 2010-05-06
Posts: 4,429

Re: Who is interested in creating an [arch-science] team?

I'll see if I can contribute related to linguistics. Maybe there is a free corpus for tigersearch.

Offline

#11 2013-01-05 09:01:56

Xyne
Moderator/TU
Registered: 2008-08-03
Posts: 5,695
Website

Re: Who is interested in creating an [arch-science] team?

Ok, I've put up a forum. Please continue discussion there if you are interested.

Let me know if you have any trouble registering.

Last edited by Xyne (2013-01-05 09:03:46)

Offline

#12 2013-01-05 23:18:47

karol
Archivist
Registered: 2009-05-06
Posts: 25,430

Re: Who is interested in creating an [arch-science] team?

Xyne wrote:

Let me know if you have any trouble registering.

I think registering is too easy: http://xyne.archlinux.ca/forum/viewtopic.php?id=8 (haven't registered yet myself).

Offline

#13 2013-01-06 08:07:10

Xyne
Moderator/TU
Registered: 2008-08-03
Posts: 5,695
Website

Re: Who is interested in creating an [arch-science] team?

karol wrote:

I think registering is too easy: http://xyne.archlinux.ca/forum/viewtopic.php?id=8 (haven't registered yet myself).

Yeah, I noticed that, even with the VSABR (very simple anti-bot registration) plugin. Presumably spammers have already incorporated the default questions in their scripts. I've replaced the default questions with my own now. Hopefully that will work.

Offline

#14 2013-01-07 06:29:42

fhr
Member
Registered: 2009-07-02
Posts: 15

Re: Who is interested in creating an [arch-science] team?

Hello everybody,

I've been using Linux for quite a few years, and I switched to Arch a few months ago. I've been looking for a way to contribute to the project, and Xyne's idea sounds great.

I'm a contributor to the MUMPS project, a linear algebra package. There's a package on AUR but it is orphaned, and I plan to update it. I also know my way around other related packages, like Scotch/PT-Scotch, Metis/ParMetis, Scalapack, BLAS, and other (parallel) linear algebra and scientific computing packages.

However, I have no experience on packaging for Arch or any other distro, but I'm willing to learn smile

PS: FYI, English is not my native language...

Offline

#15 2013-01-07 08:16:59

Xyne
Moderator/TU
Registered: 2008-08-03
Posts: 5,695
Website

Re: Who is interested in creating an [arch-science] team?

Hi fhr,

It sounds like would have a lot to offer the group. MUMPS is one of the packages that I consider problematic to package due to the need to register to access the source code (via email). If you can sort that out and help with some related packages, it would be great!

Packaging it relatively simple. A PKGBUILD is just a simple Bash file with some metadata and build instructions. Open up the PKGBUILD wiki article and use it to make sense of some existing PKGBUILDs. There are also several tools to facilitate PKGBUILD retrieval for existing packages. I'm understandably biased towards pbget. wink

I hope to see you on the other forum.

Offline

#16 2013-01-07 10:20:06

Allan
Developer
From: Brisbane, AU
Registered: 2007-06-09
Posts: 10,416
Website

Re: Who is interested in creating an [arch-science] team?

I'm not joining the other forum...  but I can supply a bunch of genetic software - all of which is probably not in the AUR now I think about it.

> ls -1 /var/abs/local/genetics/
crimap
haploview
mendel-bin
merlin
ms
pap
pedcheck
pedfiddler
pedstats
plink
plink-seq
qtdt
tabix
vcftools

Edit:  pedfiddler is not as bad as it sounds!

Offline

#17 2013-01-09 00:00:11

fhr
Member
Registered: 2009-07-02
Posts: 15

Re: Who is interested in creating an [arch-science] team?

I created a topic about linear algebra packages on Xyne's forum. I started listing what packages exist in the official repos and in AUR, and what could be done.

Concerning the access to MUMPS: some distributions, e.g., Ubuntu, provide binary packages, some toolkits like petsc can provide the package themselves (petsc-3.3-p5/config/PETSc/packages/MUMPS.py:    self.download  = ['http://ftp.mcs.anl.gov/pub/petsc/extern … -p3.tar.gz']), and the current PKGBUILD downloads directly the package from the MUMPS website... The MUMPS people appreciate to know who use the code and in what context, so that's why there's this form system, that's all.

Offline

#18 2013-01-17 14:27:29

tzok
Member
Registered: 2013-01-17
Posts: 1

Re: Who is interested in creating an [arch-science] team?

I am also interested in the [arch-science] project and will be glad to help. I have also registered to Xyne's forum right now to continue the discussion there

Offline

#19 2013-01-18 01:57:51

samsagax
Member
Registered: 2011-03-18
Posts: 33

Re: Who is interested in creating an [arch-science] team?

Count me in. I'm more of an "arch-engineering" than science. But I can give some love to packages like ElmerFEM, I compiled it myself under Arch and worked well... Unfortunately I lost he PKGBUILDs but I can rewrite them again (like most of the times tongue ).

There is room for making some noise about this? I would really like a lot of AUR packages related to science in the official repos.

Offline

#20 2013-01-18 05:36:52

solar
Member
Registered: 2011-03-01
Posts: 75

Re: Who is interested in creating an [arch-science] team?

Allan wrote:

I'm not joining the other forum...  but I can supply a bunch of genetic software - all of which is probably not in the AUR now I think about it.

Edit:  pedfiddler is not as bad as it sounds!

Allan.. all I can say is OMG!!! I certainly hope not smile

I had an online name eons ago; moon fiddler... and I was often referred to by my chaps, as something else due to the name hehe.

Anyway... out of curiosity here.. what thoughts on projects which are binary and/or commercial  then?

eg. Maple, Mathematica, Matlab and so on.

If one does set up such a targeted section, I wouldn't mind seeing sanitising such solutions which are intended for linux to being installed on arch. Even if it's not open source, many scientists would be happy to have it still easier obtained and configured through the 'science' repo as you put it with interaction for registering/licensing?

[science-oss]
bla bla

[science-prop]
bla bla

Needless to say.... the oss step is of course a more prudent and pressing issue (centralising maintenance/collaboration and so forth) but just thought I'd chuck in my tuppence.

Last edited by solar (2013-01-18 05:37:24)


I am hilariously insane. yup. you won't notice though.. I promise...I think.

Offline

#21 2013-01-18 06:33:55

progandy
Member
Registered: 2012-05-17
Posts: 2,151

Re: Who is interested in creating an [arch-science] team?

Anyway... out of curiosity here.. what thoughts on projects which are binary and/or commercial  then?

You could provide incubation packages which include a custom installer/uninstaller in the install-script. In this case I would propose an interactive setup script when necessary with automation support with a configfile in e.g. /etc/autoinstall/{PKGNAME}.cfg
Or just distribute PKGBUILD templates where the user has to fill in his license information.

Last edited by progandy (2013-01-18 06:35:57)

Online

#22 2013-01-19 06:21:52

Xyne
Moderator/TU
Registered: 2008-08-03
Posts: 5,695
Website

Re: Who is interested in creating an [arch-science] team?

samsagax wrote:

Count me in. I'm more of an "arch-engineering" than science. But I can give some love to packages like ElmerFEM, I compiled it myself under Arch and worked well... Unfortunately I lost he PKGBUILDs but I can rewrite them again (like most of the times tongue ).

elmerfem-svn is still in the AUR. I adopted it a couple of weeks ago (and still haven't touched it). It currently works but some files are installed under /usr/local despite the prefix option. The PKGBUILD needs an overhaul. If  you're up to the task I can orphan it for you. I don't have time to fiddle with it right now.

samsagax wrote:

There is room for making some noise about this? I would really like a lot of AUR packages related to science in the official repos.

This may well happen given that there are at least 2 TUs involved so far, but I think the initial goal should be to establish an independent binary repo before pushing everything into [community]. I have begun discussing some ideas of how to manage PKGBUILDs and repos on the other forum.



progandy wrote:

Anyway... out of curiosity here.. what thoughts on projects which are binary and/or commercial  then?

You could provide incubation packages which include a custom installer/uninstaller in the install-script. In this case I would propose an interactive setup script when necessary with automation support with a configfile in e.g. /etc/autoinstall/{PKGNAME}.cfg
Or just distribute PKGBUILD templates where the user has to fill in his license information.

Templates will probably be easier to manage. I have a script that might be useful for this. It can interpolate arbitrarily defined variables in any text file by prompting the user. I wrote it to generate wpa_supplicant files with passwords (there is a way to mark variables as passwords and the output is sent to a fifo to avoid writing to disk). That could be easily adapted to generate PKGBUILDs after prompting the user.

I might just package that script right now.

Offline

Board footer

Powered by FluxBB