Best way to install and use anaconda on Arch

newguy778 · 2019-10-07 08:04:51

My job requires me to use anaconda and i would like to install it. So far all the packages I have needed have either been official repo packages that I can just

sudo pacman -S packagename

or just easy/harmless things to install from AUR that I can just use yay.
I have heard that Anaconda messes with some sort of path when you install it/want to use it and generally can conflict with lots of things. What would be the best way to install it safely, the full process? In Anaconda itself you can create these things called encironments where when you install something it doesnt mess with anything else and it is very easy to delete that environment and everything in it in an instant. Is there an equivalent for this in Arch for installing anaconda itself? I know it is on AUR. Will simply running

yay -S anaconda

work and not mess with anything?
Edit: once I do install it, what is the sfae way to use it? I am asking these questions because from what I read on he internet anaconda is quite likely to break lots of things if you just simply execute the yay command i gave above
Edit 2: https://wiki.archlinux.org/index.php/Py … nvironment
I have been reading that page many times, but can't discern if this is the way to go and whether you can actually install anaconda itself in a venv or only Python packages

Last edited by newguy778 (2019-10-07 08:12:48)

schard · 2019-10-07 08:28:52

newguy778 wrote:

What would be the best way to install it safely, the full process?

Since the package is only available in the AUR, you either trust the developer to have correctly packaged it via the respective PKGBUILD or you don't.
In the latter case, you can download a snapshot from the PKGBUILD and modify it or create your own from scratch.
Then build it via makepkg and install it via pacman -U. That's about it.

Last edited by schard (2019-10-07 08:29:34)

d_fajardo · 2019-10-07 09:34:08

I have installed anaconda in the past via yay. Anaconda defaults it's installation to /opt including its own python environments. This can be a problem because /opt requires root permission to write to. As shard said, it might be better to create your own PKGBUILD and dictate where your environments are going to be.

WorMzy · 2019-10-07 10:40:40

Installing anaconda as a system package seems like bad idea to me. My recommendation is avoid anaconda if at all possible, but failing that, download the installer as an unprivileged user and install it to that user's home directory. Decline the option to add the conda activate call to your shell init files, and just source the activate script (`. $installdir/bin/activate`) manually whenever you need to run something in conda.

ngoonee · 2019-10-07 11:59:11

WorMzy wrote:

My recommendation is avoid anaconda if at all possible

Out of curiosity, why?

I've basically shifted all my python environments from venvs to conda environments, and really like doing it this way. I install anaconda as an unprivileged user and directly symlink only the conda executable to ~/.local/bin so I can activate/deactivate as necessary (within environments everything else becomes available).

Definitely would NOT recommend system install of anaconda, it's meant to be a package manager for your python stuff, so treat it that way. Similar to how I would not recommend using pacman for installing python virtualenv stuff.

WorMzy · 2019-10-07 13:39:25

Because to me, anaconda is just a way of giving users the ability to install the software they need on systems that they otherwise wouldn't be able to install software on (i.e. HPCs). I'd recommend writing a PKGBUILD (basing it on the conda recipe if necessary & available) and using native package management over a secondary package management system.

There are other cases where using conda (ideally Miniconda rather than Anaconda) makes sense, notably when you're working with a group and you just want to be able to set up and export your environment for others to import, or you need to have multiple versions of packages installed for processing different datasets.

When conda is used, my current advice is to 1) use a python3-based conda install (py2 installs don't seem to be tested upstream -- the latest repo and package format changes certainly caused a lot of trouble), 2) never touch base -- create envs to install the packages you need, 3) keep envs as minimal as possible -- the more packages you install into one, the longer dependency resolution will take (it's slow enough at the best of times...), and the more likely it is that it'll break during an update (and debugging a broken conda env is Not Fun™).

adigitoleo · 2020-08-15 06:42:25

Hi,

Sorry to piggyback on this thread but I'm in the same boat: need conda envs for science stuff (because MKL etc.) but don't want to compromise my Arch Experience. I wonder what OP chose to do or if anyone can help clear things up a bit.

I grabbed the shell script for miniconda and ran it in $HOME (didn't use the AUR packages because of the same concerns). I declined to run conda init and symlinked ~/miniconda3/bin/conda into my local script directory (on PATH), and a few things work (conda env list, etc.) but I can't activate envs.

Checking conda init --dry-run, it seems that conda basically prepends the whole of ~/miniconda3/bin to PATH (the rest is stuff to modify the shell prompt) and I guess that's the bit that causes problems? They have their own pip binary in there for example.

Is there a better way to do this?

ngoonee wrote:

I install anaconda as an unprivileged user and directly symlink only the conda executable to ~/.local/bin so I can activate/deactivate as necessary (within environments everything else becomes available)

Which binary did you link to?

d_fajardo · 2020-08-15 07:39:35

I installed anaconda once and didn't like it because I started having problems with file and directory permissions. I would avoid using it.
Now I just write my own PKGBUILD for the python modules I need. It's far simpler, cleaner since pacman keeps track of them, and it avoids using pip.
It's not at all complicated to write the PKGBUILD's. Just grabbed one of the python ones in AUR as example and just followed the logic of it. As long as the source files are available of course and I assume all if not most of what anaconda offers are available from source.

adigitoleo · 2020-08-15 12:13:57

Alright thanks for the heads up. Just took a peek at the AUR again and it seems that at least some of what I need has been packaged already, e.g. [1]. I was concerned about handling other dependencies like C/FORTRAN etc. which conda can handle, but PKGBUILDS probably work too... It's just a bit intimidating since I've ony been on arch for about a week

Some context, to be completely transparent about my concerns: this reddit thread and this post/article.

Specifically, conda provides the Intel Math Kernel Library (or binaries compiled with it?) and cuDNN (a deep neural network, haven't actually used this). I also get access to convenient channels/repos for "science" oriented (read: poorly packaged) software, which could apparently be hard to build on my own.

Before, I used to make new conda envs for each project (this is often recommended in python courses or by professors etc). It was probably redundant most of the time, but on the odd occasion it could be useful. Do you suggest virtualenv and friends for this? ( I suppose this is a whole other debate... )

Anyway, time to read up on AUR helpers for the Nth time

[1] https://aur.archlinux.org/packages/python-numpy-mkl/

loqs · 2020-08-15 12:17:49

https://www.archlinux.org/packages/comm … intel-mkl/
https://www.archlinux.org/packages/comm … _64/cudnn/

fistrosan · 2020-08-15 16:29:14

adigitoleo wrote:

Specifically, conda provides the Intel Math Kernel Library

If your main goal is to do science stuff I really see no point in using conda just to have IMKL.Why not directly download fftw, lapack, blas, fishpack, cfitsio, etc into /opt/, configure and compile there ? That is what I always do myself.

Edit: in fact if you don't mind having a some performance drop, you can directly install some of them (blas, lapack, fftw) with pacman as shared libraries.

Last edited by fistrosan (2020-08-15 16:36:55)

d_fajardo · 2020-08-15 18:00:46

fistrosan wrote:

in fact if you don't mind having a some performance drop, you can directly install some of them (blas, lapack, fftw) with pacman as shared libraries.

I'm not sure why you say there will be a performance drop and I'm also not sure also why you say pacman would treat them as shared libraries. If as you suggested to configure and compile these libraries manually, then there's not much difference if I write a PKGBUILD that does exactly the same.
What I believe pacman does is simply keep a good record of what is installed including their dependencies, to easily track newer packages, synchronise updates to avoid partial updates, and to cleanly delete them when you want to do so. How can there be a performance penalty?

fistrosan · 2020-08-15 19:44:43

Correct, there is no much difference between PKGBUILD/AUR and downloading and compiling yourself. Except that with PKGBUILD you will need to do more editing of the makefile.

But with pacman is a different story: pacman will install precompiled libraries and binaries. As such, the flags used in the compilation will be intended to work in a wide variety of cpus. If you download the package and compile it (yourself or via the AUR) you can optimize those flags and compiler options. For instance, if you compile lapack in an intel chip using icc or ifort (intel compilers) you will inmediately gain a 20 % performance as compared to gcc/gfortran (which is most likely how libraries in the arch repos were compiled). Similarly the compiler flags will be set to -02, but if you compile yourself you can go for -O3 or -fast-math and you will have even better performance.

But something tells me that performance is not your concern because it seems you want to program mostly in python (?) ... so maybe I am just talking nonsense as your primary goal might just be convenience and ease-of-use in which case of course python is the better choice.

d_fajardo · 2020-08-15 21:59:53

@fistrosan Thanks and yes you're partly right I like convenience and ease but I also think pacman is great. It's one reason I use Arch.
I would assume it is possible to write custom PKGBUILDs that would aim the optimizations but as you said it would require more editing.

adigitoleo · 2020-08-16 02:27:36

Thanks everyone for the detailed discussion. I'm convinced about managing this stuff through PKGBUILDs or from AUR instead of conda.

fitrosan wrote:

If you download the package and compile it [...] you can optimize those flags and compiler options.

This is the other thing that conda used to do for me (I think). I guess I would look for these in the package docs, right?

The intel-mkl PKGBUILD that loqs linked uses gcc-fortran, as you guessed. Maybe I will just use that for now, until I learn more about compilers etc. With pacman, it should be relatively painless to remove it later so I can recompile with an intel compiler/manual flags.

adigitoleo · 2020-08-31 15:27:19

Last post, since I've already derailed this thread too much...

Just in case anyone came here for MKL and Numpy stuff like me, if you want to do it "the Intel way" (note: beware messy licensing situation, see https://wiki.archlinux.org/index.php/Intel_C%2B%2B) then the instructions are at

https://software.intel.com/content/www/ … l-mkl.html

I've decided to stick to just using intel-mkl from the community repo (link in post by loqs 2020-08-15 22:17:49) and pyhon-numpy-mkl from AUR (see my earlier post). OK that's all sorry again for not starting my own thread earlier.

Arch Linux

#1 2019-10-07 08:04:51

Best way to install and use anaconda on Arch

#2 2019-10-07 08:28:52

Re: Best way to install and use anaconda on Arch

#3 2019-10-07 09:34:08

Re: Best way to install and use anaconda on Arch

#4 2019-10-07 10:40:40

Re: Best way to install and use anaconda on Arch

#5 2019-10-07 11:59:11

Re: Best way to install and use anaconda on Arch

#6 2019-10-07 13:39:25

Re: Best way to install and use anaconda on Arch

#7 2020-08-15 06:42:25

Re: Best way to install and use anaconda on Arch

#8 2020-08-15 07:39:35

Re: Best way to install and use anaconda on Arch

#9 2020-08-15 12:13:57

Re: Best way to install and use anaconda on Arch

#10 2020-08-15 12:17:49

Re: Best way to install and use anaconda on Arch

#11 2020-08-15 16:29:14

Re: Best way to install and use anaconda on Arch

#12 2020-08-15 18:00:46

Re: Best way to install and use anaconda on Arch

#13 2020-08-15 19:44:43

Re: Best way to install and use anaconda on Arch

#14 2020-08-15 21:59:53

Re: Best way to install and use anaconda on Arch

#15 2020-08-16 02:27:36

Re: Best way to install and use anaconda on Arch

#16 2020-08-31 15:27:19

Re: Best way to install and use anaconda on Arch

Board footer