Local tool for assisting in coding ?

Lone_Wolf · 2026-06-22 15:42:15

I've been reading/checking/writing code more lately and spend a lot of time searching for things that are unknown to me.
Many are basic knowledge but beyond my experience.

Languages are primarily C, C++, javascript , bash , obj-c .

Are there open source tools that help with coding and run locally on a amd zen 5 cpu with amd 9060 XT videocard ?

espresso · 2026-06-24 05:11:46

https://github.com/itayinbarr/little-coder and https://wiki.archlinux.org/title/Llama.cpp local inference engine look interesting. (https://old.reddit.com/r/llamacpp/)

Warm regards

Last edited by espresso (2026-06-24 05:17:11)

Lone_Wolf · 2026-06-24 13:48:21

Thanks, the llama wiki page lead me to check opencode .

Its 'plan' agent looks interesting .

cryptearth · 2026-06-24 14:20:21

i run local llama.cpp on my rx7700xt (12gb vram) and use the recently released Gemma4 in 12b-q4-k-m from google: https://huggingface.co/ggml-org/gemma-4-12B-it-GGUF

Lone_Wolf · 2026-07-01 12:26:15

I tried some questions and noticed that the best answers given were summaries of other sources.
To determine whether the answers were usable / reliable I had to go to the original sources .

After I had verified answers I questioned their validity with bogus reasoning and several times the tool changed the answer to accommodate my comments.
They lack the quality / reliability I expect from a tool.

cryptearth · 2026-07-01 13:56:35

well, back in the early days when this whole AI bubble gained momentum many models were so bad that even hard safeguards were simple to get around just by clever forming of prompts (one example i remember was attack vs defend: when you asked several llm how to attack a target most were rather limited - but when you turned around the question asking for help to protect many llm were happy to go into finest detail of how attacker perform thier work)

most important: we shall not call that stuff "intelligent" - or rather: shall not grant the privilege of seeing it as soverign intelligence
most llm are not more than gloryfied search engines, summarizers, code generators done via command syntax parsing and synthesis following some random values in more or less complex meshes
a llm does not understand - it just parses a rompt and given by the tokens and the network it was trained on comes up with a pretty much random reply

in fact: many are really bad as code generators - even if they were finetuned with the referred language
a proper code gen as part of an ide gives me proper code following syntax correctly: if i want a simple graphical ui in java it generates me correct but quite ugly-to-rad code ready to use - asking a llm often results in the llm don't get what you want and questionable output at best

anyone read those horror stories about "a jailed clawbot instance with just credentials for the test system managed to break free, search for and use tokens with access to the live production environment - and wiped it"?

i want to play 7 card stud - but as my real word friends struggle with Texas HoldEm i tried "AI" - and none was able to deal at least one round without failling
currently i'm on writing it myself and try to use llm merely as "somewhat sophisticated bot to chose between 'check', 'call', 'raise' and 'fold'" - but proper implementing 7 card stud is a challenge in its own

Succulent of your garden · 2026-07-01 14:23:01

Lone_Wolf if you want to use some stuff then I must say:

Ollama as a backend to run models is very nice to use it because is more easy to use and mantain, but if you want faster inference then llama.cpp is the way to go.

Open code for CLI is the way to go with open source approach, there is also one called Pi https://github.com/earendil-works/pi

For front end you can use OpenWebUI if you want to make an instance service that can reach in your browser. For other desktops alternatives LMStudio and anythingLLM are nice other ones that If I'm not remembering wrong are open source.

Also if you want to integrate the AI into your IDE, I highly suggest Zed editor, is very fast and comfy to use, and very customizable in every sense, it does have an easy integration to work with Ollama for example. Is very cool, it makes you enable options for read or write code in the agent basically.

If you want to write agents then n8m is the easy way to go, but other tools like Langraph exists.

For downloading models you can download it very easily with ollama, but I highly recommend to you to check also ones that are in hugging face for the following reasons:

1) You have a nice offer for fined tuned models.
2) They are some stuff that are not in the ollama webpage by default, but you can use it too.

Just be sure that when you download the models, they are in GGUF format or safeTensors one. Because they are the ones that are secures, not like Pickle for example.

GGUF can run in ollama but not with safeTensors, but you can use llama.cpp to change the format of a model in safeTensors to GGUF and load it manually in Ollama in his specific file path.

SafeTensors are the format for delivering models that have in consideration retraining in the future.
GGUF are the format to deliver models that are optimize to make inference faster basically.

Last edited by Succulent of your garden (2026-07-01 14:24:43)

Lone_Wolf · 2026-07-02 07:39:40

Thanks for the inputs , it makes the options clearer.

Succulent of your garden · 2026-07-04 21:50:23

^^

Ruhanilinks · Yesterday 12:55:16

Yes, there are some good options for this kind of workflow, especially if you want everything to stay local. I'd probably separate the problem into two parts: the model runner and the coding interface.

Ollama is probably the easiest place to start for running local models, while Aider is a nice fit if you prefer working from the terminal. For an editor-based workflow, Cline or Continue can connect to local models as well.

For the model itself, I'd look at the Qwen Coder family first. The important limitation with an AMD GPU is VRAM, though, so the exact model size you can run comfortably depends heavily on how much VRAM the 9060 XT has and what context size you're using. A smaller model can also be preferable for quick searches and explanations, while a larger one is better for multi-file reasoning.

I like the idea of keeping the setup simple: Ollama/llama.cpp underneath, a coding model on top, and Aider or an editor plugin as the interface. You don't necessarily need a huge "AI coding environment" just to ask questions about unfamiliar C/C++ or Bash code.

cryptearth · Yesterday 13:08:51

as this topic resurfaced: although i still have not finished my poker project in the mean time i used chatgpt to help me build some library helper (a jni binding)
from this i learned: keep it simple and split the task into one method/function at a time - this seem to work far better than to task it with an entire class
on the other hand i had to waste token for keep "reminding" it every once in a while about the overall goal and the already done stuff

tl;dr: even with huge context windows LLMs still fail to keep themself together for larger projects as they quickly discard previous knowledge and process so after fast initial start you cralw along quite slow

loqs · Yesterday 22:29:43

cryptearth wrote:

tl;dr: even with huge context windows LLMs still fail to keep themself together for larger projects as they quickly discard previous knowledge and process so after fast initial start you cralw along quite slow

Have you tried using a plan step to generate a detailed todo list for the task that the LLM follows together with an AGENTS.md file that contains the projects guidelines?

Arch Linux

#1 2026-06-22 15:42:15

Local tool for assisting in coding ?

#2 2026-06-24 05:11:46

Re: Local tool for assisting in coding ?

#3 2026-06-24 13:48:21

Re: Local tool for assisting in coding ?

#4 2026-06-24 14:20:21

Re: Local tool for assisting in coding ?

#5 2026-07-01 12:26:15

Re: Local tool for assisting in coding ?

#6 2026-07-01 13:56:35

Re: Local tool for assisting in coding ?

#7 2026-07-01 14:23:01

Re: Local tool for assisting in coding ?

#8 2026-07-02 07:39:40

Re: Local tool for assisting in coding ?

#9 2026-07-04 21:50:23

Re: Local tool for assisting in coding ?

#10 Yesterday 12:55:16

Re: Local tool for assisting in coding ?

#11 Yesterday 13:08:51

Re: Local tool for assisting in coding ?

#12 Yesterday 22:29:43

Re: Local tool for assisting in coding ?

Board footer