Arch Linux

lilikoi

Member

Registered: 2022-02-01

Posts: 23

[solved-] Change Ollama models/ directory

I installed Ollama from the /extra, and it works fine.

However, my machine root partition has limited space.

Ollama installs models at /var/lib/ollama/.ollama/models, and the files in this directory are all owned by "user ollama".

I tried moving the models/ directory to /home as the root user, and created a symlink to it. The ollama server gives the following err journal:

  jan 31 22:01:11 archhost systemd[1]: ollama.service: Scheduled restart job, restart counter is at 41.
  jan 31 22:01:11 archhost systemd[1]: Started Ollama Service.
  jan 31 22:01:11 archhost ollama[1211268]: Error: mkdir /var/lib/ollama/.ollama/models: file exists
  jan 31 22:01:11 archhost systemd[1]: ollama.service: Main process exited, code=exited, status=1/FAILURE
  jan 31 22:01:11 archhost systemd[1]: ollama.service: Failed with result 'exit-code'.

I tried leaving the directory structure and creating symlinks of individual files under models/, but still this makes ollama server error out.

Symlinking models/ from another partition *does work* in my home Manjaro server. I noticed that on an older version of Manjaro, ollama models are located at  /usr/share/ollama/.ollama/models instead.

Any ideas of how we can have the models/ directory put in another partition and have ollama server work without permission errors?

Last edited by lilikoi (2024-02-04 22:24:39)

lilikoi

Member

Registered: 2022-02-01

Posts: 23

Re: [solved-] Change Ollama models/ directory

OK, ChatGPT solved it for me:

### Step 3: Create a Bind Mount Instead of a Symlink

Symlinks have limitations that might not satisfy some applications' checks for directories or files. A better approach could be using a bind mount, which makes a directory or a file available at another location in the file system hierarchy, effectively making it appear as though the files are located in both places.

To trial this, perform a bind mount:

sudo mount --bind /home/ollama_models /var/lib/ollama/.ollama/models

Note that directory /var/lib/ollama/.ollama/models must exist to serve as the mount point.

To make this persistent, must add this to fstab:

# /etc/fstab

# ollama models
/home/ollama_models     /var/lib/ollama/.ollama/models     none     defaults,bind     0 0

---

PS: Well, better yet, set environment with $OLLAMA_MODELS !   (or $HOME)
https://github.com/ollama/ollama/pull/897

PPS: I am still having troubles when setting $OLLAMA_MODELS, as it tries to create all the directory structure, and if it does not have permission to write even the top directory at $OLLAMA_MODELS, it fails. I reckon that is a bug.

So what worked for me was the first answer.

Last edited by lilikoi (2024-02-04 15:06:14)

renatocan

Member

Registered: 2012-10-31

Posts: 5

Re: [solved-] Change Ollama models/ directory

I have the same issue. When setting $OLLAMA_MODELS, as it tries to create all the directory structure and it fails.

I suppose it is better not to mark this post as "[solved]".

Thanks!

Renato.

lilikoi

Member

Registered: 2022-02-01

Posts: 23

Re: [solved-] Change Ollama models/ directory

I currently solved the issue with the ChatGPT response suggestion, and I reckon the err is a bug with Ollama. It should be reported to them.

As there are three solution suggestions in my previous answer, this post shall be kept as solved, because the workarounds work almost as they should.

In addition, if setting $OLLAMA_MODELS and the mkdir command for the path to the models throws error, and Ollama is being run by the `ollama' user, just add the `ollama' user to the same group as the root directory of $OLLAMA_MODELS required for writing. And set permissions to writing for the group as well (775) to the directories of $OLLAMA_MODELS path.

I am not really sure how to explain such a bug in the Ollama GitHub Issues, but I would generally say that that is not required to have writing permissions for all the of directory path in $OLLAMA_MODELS.

In fact, I would suggest, if the directory of $OLLAMA_MODELS exists and is writable, just use it! Otehrwise, try to create the needed directories recursively, if not then abort.

The Third suggestion would be to set `ollama''s $HOME. I did not set it for testing, but I fear that it would result in the same problem (but maybe the software checking code is different for such case....).

renatocan

Member

Registered: 2012-10-31

Posts: 5

Re: [solved-] Change Ollama models/ directory

Sure, I also solved the problem by using the bind mount you have suggested.

Anyway, I think the canonical way to solve it would be to set the env variable... About your suggestion of adding the ollama user to the same group of the root directory of $OLLAMA_MODELS, this was the first thing I tried, but it didn't work either... I created a "/home/ollama" directory owned by the "ollama" user and set $OLLAMA_MODELS to /home/ollama/models. Strangely, it kept trying to mkdir /home/ollama.

Thanks for the suggestions!

lesca

Member

Registered: 2024-05-24

Posts: 3

Re: [solved-] Change Ollama models/ directory

Hello, thanks for sharing.

I also solved the problem by turning the system wide unit to user unit.

Copy the default unit file to user's unit folder:
cp /usr/lib/systemd/system/ollama.service $HOME/.config/systemd/user/ollama.service

Modify it as below:

[Unit]
Description=Ollama Service
Wants=network-online.target
After=network.target network-online.target

[Service]
ExecStart=/usr/bin/ollama serve
#WorkingDirectory=/var/lib/ollama
#Environment="HOME=/var/lib/ollama"
Environment="GIN_MODE=release"
#User=ollama
#Group=ollama
Restart=on-failure
RestartSec=3
Type=simple
PrivateTmp=yes
ProtectSystem=full
#ProtectHome=yes
Environment="HSA_OVERRIDE_GFX_VERSION=11.0.2"

[Install]
#WantedBy=multi-user.target
WantedBy=default.target

I am using integrated graphic card on AMD platform. Adjust your HSA_OVERRIDE_GFX_VERSION variable accordingly.

Then, reload-deamon and enable, start it.

Cheers!

Last edited by lesca (2024-05-24 06:06:48)