Help In installing python application gpustack

Succulent of your garden · 2025-02-24 21:15:53

Hi

I'm want to install gpustack https://github.com/gpustack/gpustack , this applications let you use distributed inference which let you run AI models with more than one gpu, using the total amount of vram that all your gpus you have and setup the distribution of load of the neural network.

Sadly the script from gpustack doesn't consider arch linux :C, but if you try to install all the packages needed to do a successful installation the script fails. Also you can do the manual installation:

https://docs.gpustack.ai/latest/install … tallation/

I try this with python 3.13 but fails, I'm not sure if it is because of that or because I need to launch my programs outside a virtual environment. Nevertheless probably this application need to run not in a virtual environment in order to have access to ROCm and cuda. I need help and orientation so I can be able to use this program in my arch setup, i don't want to break my system because I'm installing python 3.12 applications when the system uses python 3.13 using the python 3.12 aur package. Also I don't want to use the docker container, at least is the only real solution for arch now :C

EDIT: Btw this package is not in the aur

Last edited by Succulent of your garden (2025-02-27 12:36:10)

mackin_cheese · 2025-02-24 21:21:56

what errors do you get when trying to install in the venv?

Succulent of your garden · 2025-02-24 21:56:49

this is the part when it breaks:

Using cached iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Using cached pycparser-2.22-py3-none-any.whl (117 kB)
Building wheels for collected packages: asyncpg
  Building wheel for asyncpg (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for asyncpg (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [218 lines of output]
      running bdist_wheel
      running build
      running build_py
      creating build/lib.linux-x86_64-cpython-313/asyncpg
      copying asyncpg/utils.py -> build/lib.linux-x86_64-cpython-313/asyncpg
      copying asyncpg/types.py -> build/lib.linux-x86_64-cpython-313/asyncpg

Then it just copy a lots of files, and then this happend: 

building 'asyncpg.pgproto.pgproto' extension
      creating build/temp.linux-x86_64-cpython-313/asyncpg/pgproto
      gcc -fno-strict-overflow -Wsign-compare -DNDEBUG -g -O3 -Wall -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=3 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -ffat-lto-objects -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=3 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=3 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -fPIC -I/home/myPC/projects/gpustack/include -I/usr/include/python3.13 -c asyncpg/pgproto/pgproto.c -o build/temp.linux-x86_64-cpython-313/asyncpg/pgproto/pgproto.o -O2 -fsigned-char -Wall -Wsign-compare -Wconversion
      asyncpg/pgproto/pgproto.c:864:1: warning: ‘Py_UNICODE’ is deprecated [-Wdeprecated-declarations]
        864 | static CYTHON_INLINE size_t __Pyx_Py_UNICODE_strlen(const Py_UNICODE *u) {
            | ^~~~~~
      asyncpg/pgproto/pgproto.c: In function ‘__Pyx_Py_UNICODE_strlen’:
      asyncpg/pgproto/pgproto.c:865:5: warning: ‘Py_UNICODE’ is deprecated [-Wdeprecated-declarations]
        865 |     const Py_UNICODE *u_end = u;
            |     ^~~~~
      asyncpg/pgproto/pgproto.c: In function ‘__Pyx_init_assertions_enabled’:
      asyncpg/pgproto/pgproto.c:1667:39: error: implicit declaration of function ‘_PyInterpreterState_GetConfig’; did you mean ‘PyInterpreterState_GetID’? [-Wimplicit-function-declaration]
       1667 |     __pyx_assertions_enabled_flag = ! _PyInterpreterState_GetConfig(__Pyx_PyThreadState_Current->interp)->optimization_level;
            |                                       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
            |                                       PyInterpreterState_GetID
      asyncpg/pgproto/pgproto.c:1667:105: error: invalid type argument of ‘->’ (have ‘int’)
       1667 |     __pyx_assertions_enabled_flag = ! _PyInterpreterState_GetConfig(__Pyx_PyThreadState_Current->interp)->optimization_level;
            |                                                                                                         ^~
      asyncpg/pgproto/pgproto.c: In function ‘__Pyx_PyUnicode_Join’:
      asyncpg/pgproto/pgproto.c:38932:13: error: implicit declaration of function ‘_PyUnicode_FastCopyCharacters’; did you mean ‘PyUnicode_CopyCharacters’? [-Wimplicit-function-declaration]
      38932 |             _PyUnicode_FastCopyCharacters(result_uval, char_pos, uval, 0, ulength);
            |             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
            |             PyUnicode_CopyCharacters
      In file included from /usr/include/python3.13/internal/pycore_code.h:12,
                       from /usr/include/python3.13/internal/pycore_frame.h:13,
                       from asyncpg/pgproto/pgproto.c:41520:
      /usr/include/python3.13/internal/pycore_backoff.h: In function ‘make_backoff_counter’:
      /usr/include/python3.13/internal/pycore_backoff.h:60:20: warning: conversion from ‘uint16_t’ {aka ‘short unsigned int’} to ‘short unsigned int:12’ may change value [-Wconversion]
         60 |     result.value = value;
            |                    ^~~~~
      /usr/include/python3.13/internal/pycore_backoff.h:61:22: warning: conversion from ‘uint16_t’ {aka ‘short unsigned int’} to ‘unsigned char:4’ may change value [-Wconversion]
         61 |     result.backoff = backoff;
            |                      ^~~~~~~
      /usr/include/python3.13/internal/pycore_backoff.h: In function ‘restart_backoff_counter’:
      /usr/include/python3.13/internal/pycore_backoff.h:78:66: warning: conversion from ‘int’ to ‘uint16_t’ {aka ‘short unsigned int’} may change value [-Wconversion]
         78 |         return make_backoff_counter((1 << (counter.backoff + 1)) - 1, counter.backoff + 1);
            |                                     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~
      asyncpg/pgproto/pgproto.c: In function ‘__Pyx_PyInt_As_int’:
      asyncpg/pgproto/pgproto.c:41809:27: error: too few arguments to function ‘_PyLong_AsByteArray’
      41809 |                 int ret = _PyLong_AsByteArray((PyLongObject *)v,
            |                           ^~~~~~~~~~~~~~~~~~~
      In file included from /usr/include/python3.13/longobject.h:107,
                       from /usr/include/python3.13/Python.h:81,
                       from asyncpg/pgproto/pgproto.c:33:
      /usr/include/python3.13/cpython/longobject.h:111:17: note: declared here
        111 | PyAPI_FUNC(int) _PyLong_AsByteArray(PyLongObject* v,
            |                 ^~~~~~~~~~~~~~~~~~~
      asyncpg/pgproto/pgproto.c: In function ‘__Pyx_PyInt_As_int64_t’:
      asyncpg/pgproto/pgproto.c:42043:27: error: too few arguments to function ‘_PyLong_AsByteArray’
      42043 |                 int ret = _PyLong_AsByteArray((PyLongObject *)v,
            |                           ^~~~~~~~~~~~~~~~~~~
      /usr/include/python3.13/cpython/longobject.h:111:17: note: declared here
        111 | PyAPI_FUNC(int) _PyLong_AsByteArray(PyLongObject* v,
            |                 ^~~~~~~~~~~~~~~~~~~
      asyncpg/pgproto/pgproto.c: In function ‘__Pyx_PyInt_As_int32_t’:
      asyncpg/pgproto/pgproto.c:42239:27: error: too few arguments to function ‘_PyLong_AsByteArray’
      42239 |                 int ret = _PyLong_AsByteArray((PyLongObject *)v,
            |                           ^~~~~~~~~~~~~~~~~~~
      /usr/include/python3.13/cpython/longobject.h:111:17: note: declared here
        111 | PyAPI_FUNC(int) _PyLong_AsByteArray(PyLongObject* v,
            |                 ^~~~~~~~~~~~~~~~~~~
      asyncpg/pgproto/pgproto.c: In function ‘__Pyx_PyInt_As_long’:
      asyncpg/pgproto/pgproto.c:42435:27: error: too few arguments to function ‘_PyLong_AsByteArray’
      42435 |                 int ret = _PyLong_AsByteArray((PyLongObject *)v,
            |                           ^~~~~~~~~~~~~~~~~~~
      /usr/include/python3.13/cpython/longobject.h:111:17: note: declared here
        111 | PyAPI_FUNC(int) _PyLong_AsByteArray(PyLongObject* v,
            |                 ^~~~~~~~~~~~~~~~~~~
      asyncpg/pgproto/pgproto.c: In function ‘__Pyx_PyInt_As_unsigned_char’:
      asyncpg/pgproto/pgproto.c:42745:27: error: too few arguments to function ‘_PyLong_AsByteArray’
      42745 |                 int ret = _PyLong_AsByteArray((PyLongObject *)v,
            |                           ^~~~~~~~~~~~~~~~~~~
      /usr/include/python3.13/cpython/longobject.h:111:17: note: declared here
        111 | PyAPI_FUNC(int) _PyLong_AsByteArray(PyLongObject* v,
            |                 ^~~~~~~~~~~~~~~~~~~
      asyncpg/pgproto/pgproto.c: In function ‘__Pyx_PyInt_As_int16_t’:
      asyncpg/pgproto/pgproto.c:43055:27: error: too few arguments to function ‘_PyLong_AsByteArray’
      43055 |                 int ret = _PyLong_AsByteArray((PyLongObject *)v,
            |                           ^~~~~~~~~~~~~~~~~~~
      /usr/include/python3.13/cpython/longobject.h:111:17: note: declared here
        111 | PyAPI_FUNC(int) _PyLong_AsByteArray(PyLongObject* v,
            |                 ^~~~~~~~~~~~~~~~~~~
      asyncpg/pgproto/pgproto.c: In function ‘__Pyx_PyInt_As_char’:
      asyncpg/pgproto/pgproto.c:43289:27: error: too few arguments to function ‘_PyLong_AsByteArray’
      43289 |                 int ret = _PyLong_AsByteArray((PyLongObject *)v,
            |                           ^~~~~~~~~~~~~~~~~~~
      /usr/include/python3.13/cpython/longobject.h:111:17: note: declared here
        111 | PyAPI_FUNC(int) _PyLong_AsByteArray(PyLongObject* v,
            |                 ^~~~~~~~~~~~~~~~~~~
      asyncpg/pgproto/pgproto.c: In function ‘__Pyx_PyInt_As_int8_t’:
      asyncpg/pgproto/pgproto.c:43485:27: error: too few arguments to function ‘_PyLong_AsByteArray’
      43485 |                 int ret = _PyLong_AsByteArray((PyLongObject *)v,
            |                           ^~~~~~~~~~~~~~~~~~~
      /usr/include/python3.13/cpython/longobject.h:111:17: note: declared here
        111 | PyAPI_FUNC(int) _PyLong_AsByteArray(PyLongObject* v,
            |                 ^~~~~~~~~~~~~~~~~~~
      asyncpg/pgproto/pgproto.c: In function ‘__Pyx_PyInt_As_uint32_t’:
      asyncpg/pgproto/pgproto.c:43719:27: error: too few arguments to function ‘_PyLong_AsByteArray’
      43719 |                 int ret = _PyLong_AsByteArray((PyLongObject *)v,
            |                           ^~~~~~~~~~~~~~~~~~~
      /usr/include/python3.13/cpython/longobject.h:111:17: note: declared here
        111 | PyAPI_FUNC(int) _PyLong_AsByteArray(PyLongObject* v,
            |                 ^~~~~~~~~~~~~~~~~~~
      asyncpg/pgproto/pgproto.c: In function ‘__Pyx_PyInt_As_uint64_t’:
      asyncpg/pgproto/pgproto.c:43915:27: error: too few arguments to function ‘_PyLong_AsByteArray’
      43915 |                 int ret = _PyLong_AsByteArray((PyLongObject *)v,
            |                           ^~~~~~~~~~~~~~~~~~~
      /usr/include/python3.13/cpython/longobject.h:111:17: note: declared here
        111 | PyAPI_FUNC(int) _PyLong_AsByteArray(PyLongObject* v,
            |                 ^~~~~~~~~~~~~~~~~~~
      asyncpg/pgproto/pgproto.c: At top level:
      asyncpg/pgproto/pgproto.c:23994:18: warning: ‘__pyx_f_7asyncpg_7pgproto_7pgproto_json_decode’ defined but not used [-Wunused-function]
      23994 | static PyObject *__pyx_f_7asyncpg_7pgproto_7pgproto_json_decode(struct __pyx_obj_7asyncpg_7pgproto_7pgproto_CodecContext *__pyx_v_settings, struct __pyx_t_7asyncpg_7pgproto_7pgproto_FRBuffer *__pyx_v_buf) {
            |                  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      asyncpg/pgproto/pgproto.c:23886:18: warning: ‘__pyx_f_7asyncpg_7pgproto_7pgproto_json_encode’ defined but not used [-Wunused-function]
      23886 | static PyObject *__pyx_f_7asyncpg_7pgproto_7pgproto_json_encode(struct __pyx_obj_7asyncpg_7pgproto_7pgproto_CodecContext *__pyx_v_settings, struct __pyx_obj_7asyncpg_7pgproto_7pgproto_WriteBuffer *__pyx_v_buf, PyObject *__pyx_v_obj) {
            |                  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      error: command '/usr/bin/gcc' failed with exit code 1
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for asyncpg
Failed to build asyncpg

And the final error of the log is this:

ERROR: ERROR: Failed to build installable wheels for some pyproject.toml based projects (asyncpg)

EDIT: Initial Command in python env:

pip install gpustack[all]

Last edited by Succulent of your garden (2025-02-27 12:28:41)

mackin_cheese · 2025-02-24 22:15:43

asyncpg is in the official repos, what happens if you install it and then try to build gpustack?

bulletmark · 2025-02-24 22:25:41

Whenever quoting output always include the initiating command.

Lone_Wolf · 2025-02-25 10:17:22

https://docs.gpustack.ai/latest/installation/installation-requirements/ wrote:

Python Requirements
GPUStack requires Python version 3.10 to 3.12.

Look into software allowing to use/switch between multiple python versions like pyenv or uv

Moderator Note
Moving to Aur board as gpustack is unsupported on archlinux

Succulent of your garden · 2025-02-27 13:56:25

Hi everyone:

I tried using pyenv and the installation works with using python 3.12. Nevertheless the installation doesn't detect any gpu, I assume it's because of the env created so I can't break my system. Now I'm trying the docker approach, but I tried to install gpustack using the command that said the official github repo:

curl -sfL https://get.gpustack.ai | sh -s -

But start asking me to use sudo privileges. Do you think it's a good idea to let this script https://get.gpustack.ai/install.sh to mess with sudo in arch ? Since I know that in a virt env works the installation, I'm guessing that using python 3.12 with pyenv could break my other python programs. Since the script support distros that are not arch based.

bulletmark · 2025-02-27 22:13:19

`uv` has supplanted `pyenv` and nearly all other python build/management tools nowadays so no need for `pyenv`. I suggest removing any python versions you installed with it and then `pyenv` itself. Then do the following:

$ mkdir ~/gpustack && cd ~/gpustack
$ uv venv -p 3.12
$ uv pip install gpustack[all]
$ .venv/bin/gpustack version

`uv` will install a private copy of python 3.12 (if it not found anywhere and done very quickly!) which is only for use with created venv's and will not affect anything else on your system.

Note the words on the error about the gpu. It is complaining about the `bitsandbytes` package. Search for that and find the problem and solution here.

There is no need for root for installation, and/or at runtime if you set the `--data-dir` option as described here. Also need to add a `--port 8000` option so it does not attempt to listen on port 80.

Succulent of your garden · 2025-02-28 11:29:54

bulletmark wrote:

`uv` has supplanted `pyenv` and nearly all other python build/management tools nowadays so no need for `pyenv`. I suggest removing any python versions you installed with it and then `pyenv` itself. Then do the following:
$ mkdir ~/gpustack && cd ~/gpustack
$ uv venv -p 3.12
$ uv pip install gpustack[all]
$ .venv/bin/gpustack version
`uv` will install a private copy of python 3.12 (if it not found anywhere and done very quickly!) which is only for use with created venv's and will not affect anything else on your system.
Note the words on the error about the gpu. It is complaining about the `bitsandbytes` package. Search for that and find the problem and solution here.
There is no need for root for installation, and/or at runtime if you set the `--data-dir` option as described here. Also need to add a `--port 8000` option so it does not attempt to listen on port 80.

Thanks for the info, But if I use the shell script installation, if I set the port and the data dir flags, it also needs sudo. I'm trying the docker approach.

Arch Linux

#1 2025-02-24 21:15:53

Help In installing python application gpustack

#2 2025-02-24 21:21:56

Re: Help In installing python application gpustack

#3 2025-02-24 21:56:49

Re: Help In installing python application gpustack

#4 2025-02-24 22:15:43

Re: Help In installing python application gpustack

#5 2025-02-24 22:25:41

Re: Help In installing python application gpustack

#6 2025-02-25 10:17:22

Re: Help In installing python application gpustack

#7 2025-02-27 13:56:25

Re: Help In installing python application gpustack

#8 2025-02-27 22:13:19

Re: Help In installing python application gpustack

#9 2025-02-28 11:29:54

Re: Help In installing python application gpustack

Board footer