Arch Linux

Der Chefkoch

Member

Registered: 2020-12-05

Posts: 100

Getting a powerful AI language model to run with NVIDIA GPU

I have started dabbling to get large language models to run locally on my machine, in particular Alpaca/Llama.

So far I gotten https://github.com/antimatter15/alpaca.cpp to run with the 7B Model, however including cuBLAS for my RTX 4080 did now work. I tried some time ago to get Dalai Llama to run, hower that did work and seems to be taken down since.

I wanted to know who got a more powerful Language Model to run on Arch, preferrably with support of a current gen NVIDIA 4000 series GPU?

Thx

Last edited by Der Chefkoch (2023-06-04 10:29:58)