You are not logged in.

#1 2021-08-02 18:53:13

MountainX
Member
Registered: 2016-02-08
Posts: 371

What is the community's opinion of GitBSLR (make Git follow symlinks)?

My goal is to eliminate code duplication and better organize my bash scripts.

One idea for accomplishing this is utilizing symlinks (e.g., of files containing single function definitions) between separate projects. By default git doesn't treat symlinks the way I would need it do under this scenario. However, this tool is recommended (on Stackoverflow) for exactly this purpose:

Alcaro/GitBSLR: Make Git follow symlinks

I would like to know the Arch community's general opinion of both the specific tool, GitBSLR, and the general concept.

My goal is to have bash functions that are defined once, that live at one place in my filesystem, and that are utilized in multiple different git projects (such as Arch packages). Any time the original function is updated, the git projects that reference it will reflect those changes in their status.

I suspect there are various ways to achieve this. Is GitBSLR a good one? What might be better? (I'm not a developer, so it is very possible that I'm overlooking some obvious solutions that experienced developers are aware of and I'm not.)

Offline

#2 2021-08-02 19:38:44

Xyne
Administrator/PM
Registered: 2008-08-03
Posts: 6,963
Website

Re: What is the community's opinion of GitBSLR (make Git follow symlinks)?

How would that work in practice? If you have multiple Git repositories that depend on a common external file, then presumably changes to that external file will require updates to the code that depends on them and vice versa. So what happens if you update one repo and the external file and break backwards compatibility with the others? What about when you check out an old version of another repo with an old version of the external file? All the other repos break?

A Git submodule would probably be a better solution. Put the external file in it's own independent Git repo, then add that repo as a submodule to any project that depends on it. You can clone it locally using the file path. That way, different Git repos that depend on the external file can each check out their own version of it without interfering with each other.

If the data duplication from creating multiple submodules from the same module is an issue, maybe this will alleviate the issue (I haven't tested it): https://stackoverflow.com/questions/456 … t-with-git


My Arch Linux StuffForum EtiquetteCommunity Ethos - Arch is not for everyone

Offline

#3 2021-08-02 20:22:38

MountainX
Member
Registered: 2016-02-08
Posts: 371

Re: What is the community's opinion of GitBSLR (make Git follow symlinks)?

Xyne wrote:

How would that work in practice? If you have multiple Git repositories that depend on a common external file, then presumably changes to that external file will require updates to the code that depends on them and vice versa.

Yes. That's my goal.

Thanks for your questions. They are helping me think through this.

Xyne wrote:

So what happens if you update one repo and the external file and break backwards compatibility with the others?

I want to treat my projects more like a rolling release. (I hope that's an appropriate analogy.) When I update a function, I will expect to update / rebuild any projects that use it. For me, this seems like a feature rather than a bug.

Xyne wrote:

What about when you check out an old version of another repo with an old version of the external file? All the other repos break?

Obviously, I will have to avoid that. I'm not sure how yet. But I can say that checking out an old version of one of my projects is not something I usually do.

Xyne wrote:

A Git submodule would probably be a better solution.

As it says in this article Git subtree: the alternative to Git submodule | Atlassian Git Tutorial, I have noticed that "The Internet is full of articles on why you shouldn’t use Git submodules." I have never used it, the reading I have done so far indicates that it's not an ideal solution for me.

It's complex. As someone who doesn't work with git regularly, I need to keep it simple.

Also, I read that "Git submodule is better when you do not control the subprojects or more specifically wish to fix the subproject at a specific revision even as the subproject changes." That's not my use case. I control all the projects and I will usually be updating them together.

Xyne wrote:

That way, different Git repos that depend on the external file can each check out their own version of it without interfering with each other.

My main problem as of now is not keeping separate projects on their own version. It's the opposite. I want to keep everything updated to the latest code all the time (within reason).

Tell me if my analogy is wrong. It seems like your response is aligned with the way fixed release distros are managed. Obviously, there are good reasons for that approach, and it is a reason why many developers or sysadmins prefer CentOS, RedHat, etc. But I am happy with the rolling release approach of Arch, so why can't I align my personal code management to that kind of thinking?

I found this useful summary:

Alternatives to Git Submodules?

It mentions gitslave, git-submodules, git-subtree, Google repo, and mr.

But none of these seem to be as ideal for me as GitBSLR. Of course, that's an opinion formed without any experience with any of these tools and I came here looking for alternatives and reasons not to use GitBSLR. However, I can be sure that if any proposed solution is too complex, it won't be a good one for me. I'm not working with these tools every day (or even every week), so I need something that's simple.

Offline

#4 2021-08-03 00:46:52

Xyne
Administrator/PM
Registered: 2008-08-03
Posts: 6,963
Website

Re: What is the community's opinion of GitBSLR (make Git follow symlinks)?

If you use subtrees then you still need to deal with pushing and pulling changes between the subtree and the repo for the external file.  I haven't worked with subtrees but there seems to be considerable overlap in use-cases between them and submodules, with the main advantage being transparency to repo users and the main disadvantage being that the histories are managed together in the main repo (my impression after reading the link).

I have no opinion of the other suggested alternatives except that I would personally avoid adding external dependencies beyond a custom script or two just to manage my source code.

I don't think that either approach is analogous to "rolling" or "fixed" release. All packages that depend on an external dependency have to be upgraded together in either approach, but here we're talking about external files integrated in the code itself. If you force-link the version to all of your projects as you want, then you have to update/release everything together. While keeping things up-to-date is good practice, breaking things to force yourself to do so is not. What do you do the day that you need to update one project but don't have time to update another one that you also depend on? If you can't keep both working in that scenario, you're in trouble.

However, if this is just for your own scripts and you just want to re-use some code, why not just keep your external code separate and source it from your scripts? For example, create a directory with whatever organization that you want to hold your external code with their own git repos, etc, and then set up your environment with variables that point to it. Then in your scripts you can just use source "${PATH_TO_DIR_VAR}/subpath/my_functions.sh" or whatever. You then get the desired behavior of forced updates with no code duplication beyond the source line, and everything has it's own independent git repo. The only other thing you need is either to set the variable in your bash profile or bashrc, and/or create an environment file that you can source in your shell when running your code (or that you source automatically from your bash profile).


My Arch Linux StuffForum EtiquetteCommunity Ethos - Arch is not for everyone

Offline

#5 2021-08-03 13:56:43

MountainX
Member
Registered: 2016-02-08
Posts: 371

Re: What is the community's opinion of GitBSLR (make Git follow symlinks)?

Xyne wrote:

I don't think that either approach is analogous to "rolling" or "fixed" release.

OK. thanks. Your explanation makes a lot of sense.

Xyne wrote:

What do you do the day that you need to update one project but don't have time to update another one that you also depend on? If you can't keep both working in that scenario, you're in trouble.

Yes, that's a very real problem. You have changed my thinking about the best way to go forward.

Xyne wrote:

However, if this is just for your own scripts and you just want to re-use some code, why not just keep your external code separate and source it from your scripts?

This is a very tempting approach, but I hesitate to use it for several reasons. One is the argument you have made me aware of, quoted above. Another is that it is probably not very scalable or robust. (The scripts are just for my own use at present, but that could change.)

I will keep reading about the various options, but one that I am really liking now is git-subrepo:

AUR (en) - git-subrepo

GitHub: git-subrepo

This command is an improvement from git-submodule and git-subtree; two other git commands with similar goals, but various problems.

Offline

Board footer

Powered by FluxBB