Version controling system config files form another loaction with git

masterkorp · 2010-09-16 13:13:05

Hello i don't know if this is the correct section but i put this thread here since git is mainly programing related.

The problem is that i am trying to overcome the fact that git doesn't understand symbolic links that point to outside of the repository.
And that since some files and for some files ( system config files, ) and i can't reverse the symbolic linking ( having the real files in the in the repository and the symbolic link in their respective directory ) because some of those files are read even before the kernel loads.

So my idea of solution is having a 2 hooks sripts, one pre-commit hook and one post-commint.
The pre-commit hook is search in text file ( that contains a list of the files that is not possible to have in the repository and the symbolic links in their respective directories ) and if the file exists in that list they should compare ( by the last commit ) the file with the version in their respective directory and if it's different, stop the actual commit and tell me the file that are not equal.

The propose of this is to prevent a eventual file non-manual modification (e.g: the package manager ) to be overwritten.

The post commit is just only copy the file to their "real" location ( if exist on the list ).

I already tried etckeeper (link: http://tinyurl.com/35mfse7 ) but doesn't fit my needs.

I have also another question. If i push the commits form a server the pre-commit and post-commit hooks will run ?

So can anyone help me making the script ?
I know programming ( my job ), but if someone points me some useful documentation and tips.
And can i use git directly in the pre-commit script to compare the files, instead of coding it into the script or calling an external application ?

Thanks already,

Last edited by masterkorp (2010-09-16 13:15:19)

tavianator · 2010-09-16 15:01:11

Easier way: use hard links.

masterkorp · 2010-09-16 20:20:20

tavianator wrote:

Easier way: use hard links.

Please do more research about the subject before asking.
In here explains the difference or a hard link and a symbolic link.
Some programs when you edit a file will overwrite a file ( even when you add a single portion of it ) instead of appending the alteration to it, the file with two locations ( hard link ) is going to be two deferent files with the same name.

That's not a problem with vim ( my editor ), but git will break then in checkouts and much more and it's a problem with msysgit ( that's not a problem for me that i don't use ruidows, but is a minus ).

Don't take me bad for the tip, i am just trying to clarify you

Last edited by masterkorp (2010-09-16 23:03:26)

fukawi2 · 2010-09-16 22:58:01

You also can't cross file system boundaries with hard links, which may or may not affect the OP. And you can't hard link directories.

tavianator · 2010-09-17 03:56:05

masterkorp wrote:

tavianator wrote:
Easier way: use hard links.
Please do more research about the subject before asking.
In here explains the difference or a hard link and a symbolic link.
Some programs when you edit a file will overwrite a file ( even when you add a single portion of it ) instead of appending the alteration to it, the file with two locations ( hard link ) is going to be two deferent files with the same name.
That's not a problem with vim ( my editor ), but git will break then in checkouts and much more and it's a problem with msysgit ( that's not a problem for me that i don't use ruidows, but is a minus ).
Don't take me bad for the tip, i am just trying to clarify you

Not sure I understand your issue, so let me clarify my idea:

OP wants to put /etc/rc.conf, /etc/ssh/sshd_config, and a bunch of other stuff under git, but wants to keep the git tree separate from the actual files in the filesystem. To accomplish this without needing to write hooks and stuff, you can do this:

ln /etc/rc.conf /etc/scm/rc.conf
ln /etc/ssh/sshd_config /etc/scm/sshd_config
# ...

git init .
git add .
git commit
# etc.

Updates to the "official" copies of the files will automatically propagate to the linked copies in /etc/scm (which is in /etc so that it doesn't cross a filesystem boundary). As masterkorp pointed out, some of your files may be unlink()ed and re-written, and the updates to that file will stop propagating. So, save all your ln commands in a script, and re-run it before you commit (that could even be a pre-commit hook or something).

However, I just thought of what I think is a better way to accomplish this: make / (or /etc) a git repository. Just make a .gitignore file containing "*" so your status reports won't show every file on your filesystem as untracked, then git add -f the files you want to track. The only downside is that checkout out old revisions will actually delete any files that you've added since that revision, but checking out master again will bring them back.

mikesd · 2010-09-17 06:01:22

I store my user configs in a git repository and at the top of each config I have a comment that contains a header along with the actual location of the file. Also in the git repo is a simple BASH script that goes through each file, looks for the header and creates the required symbolic link pointing back to the config file. It is very rough but works ok. The BASH script is called cmgr and is included in my configs in my signature.

masterkorp · 2010-09-17 13:42:55

tavianator, your idea is cool but there is a problem:

tavianator wrote:

So, save all your ln commands in a script, and re-run it before you commit (that could even be a pre-commit hook or something).

The script to "restore" the hard links will cause error because a different file with the same name already exists ( assuming that the the link was been broken ).
- Just delete one file and them remake the link. - You may think, but since the since is broken there may be modifications on the files ( for sure, since the modifications is the reason for some programs break the links ), and how do i know whats as the "side" of the file that has been broken? The original or the one in the repository ?

To verify this i need a script to check the differences between those files, and the use of of links is pointless them, is better to have two separate files and always check the differences. and we return to my point of view ( which was thinking this way that i formulate my initial question ).

I recomend you a read about inode

tavianator wrote:

However, I just thought of what I think is a better way to accomplish this: make / (or /etc) a git repository. Just make a .gitignore file containing "*" so your status reports won't show every file on your filesystem as untracked, then git add -f the files you want to track. The only downside is that checkout out old revisions will actually delete any files that you've added since that revision, but checking out master again will bring them back.

Yes having the repository in the /etc directory is good you even can use etckeeper which i use to auto-commit some changes made by pacman and other applications. But i need to track more files (e.g my awesome wm config files ) and i initially created a folder those files in /etc and put them the real files and symbolic links in the location that they need to be. But it' feels strange, and is not a very correct way of doing it.
And no, i don't want to create 2 different repositories.

I hope i made you understand my point

masterkorp · 2010-09-17 13:52:01

mikesd wrote:

I store my user configs in a git repository and at the top of each config I have a comment that contains a header along with the actual location of the file. Also in the git repo is a simple BASH script that goes through each file, looks for the header and creates the required symbolic link pointing back to the config file. It is very rough but works ok. The BASH script is called cmgr and is included in my configs in my signature.

If you only want to track the files in /etc i recommend you etckeeper it auto commits modification made by pacman and other package managers like apt (originally designer for ) and yum.
it's really good you should give it a try.
For arch linux you can found here a testimonial.

I found it very usefull for track my web server because all my configurations are there in /etc and apt (the debian style ) have the habit of maintaining the system for you, although the arch linux usually don't mess with your configs files ( the arch-way xD ) i can eventually happen, this etckeeper can save your ass sometimes

tavianator · 2010-09-17 18:12:52

masterkorp wrote:

tavianator, your idea is cool but there is a problem:
tavianator wrote:
So, save all your ln commands in a script, and re-run it before you commit (that could even be a pre-commit hook or something).
The script to "restore" the hard links will cause error because a different file with the same name already exists ( assuming that the the link was been broken ).
- Just delete one file and them remake the link. - You may think, but since the since is broken there may be modifications on the files ( for sure, since the modifications is the reason for some programs break the links ), and how do i know whats as the "side" of the file that has been broken? The original or the one in the repository ?

Or just ln -f. I was assuming that you wouldn't ever edit the files in git directly, so the "real" files would take precedence.

I recomend you a read about inode

I know plenty about inodes, thanks...

Yes having the repository in the /etc directory is good you even can use etckeeper which i use to auto-commit some changes made by pacman and other applications. But i need to track more files (e.g my awesome wm config files ) and i initially created a folder those files in /etc and put them the real files and symbolic links in the location that they need to be. But it' feels strange, and is not a very correct way of doing it.
And no, i don't want to create 2 different repositories.
I hope i made you understand my point

So make / the git repo instead, and you can track stuff in /etc and /home/whomever.

As far as helping write the hooks that you were talking about in your OP, I'd be glad to except I have zero experience with git hooks. But I assume the logic would be just to bail out if there's changes to the local file that doesn't match the master, and otherwise overwrite it with the master.

masterkorp · 2010-09-17 18:49:40

tavianator wrote:

Or just ln -f. I was assuming that you wouldn't ever edit the files in git directly, so the "real" files would take precedence.

But this doesn't solve one problem, how do i know what file suffered modifications ? the "original" or the one in repository ? in forcing the link i will lose some modifications (that ones that broke the link )

tavianator wrote:

So make / the git repo instead, and you can track stuff in /etc and /home/whomever.
As far as helping write the hooks that you were talking about in your OP, I'd be glad to except I have zero experience with git hooks. But I assume the logic would be just to bail out if there's changes to the local file that doesn't match the master, and otherwise overwrite it with the master.

That would create a mess, With the quantity of git repositories that i have, i could mistake applying a command that is not the correct repo and will be applied in the the my config files repo.
And i like to have my stuff organized

djgera · 2010-09-17 18:59:43

You want to track /etc and /home/you. Do you try bind-mount?

mount --bind /etc /some/place/etc
mount --bind /home/you /some/place/you

Then your git repo will be located at /some/place/

Maybe this cover what you want. I hope if it helps

mikesd · 2010-09-18 01:18:24

masterkorp wrote:

If you only want to track the files in /etc i recommend you etckeeper it auto commits modification made by pacman and other package managers like apt (originally designer for ) and yum.
it's really good you should give it a try.
For arch linux you can found here a testimonial.
I found it very usefull for track my web server because all my configurations are there in /etc and apt (the debian style ) have the habit of maintaining the system for you, although the arch linux usually don't mess with your configs files ( the arch-way xD ) i can eventually happen, this etckeeper can save your ass sometimes

I haven't done anything about tracking /etc/, just my configs in my home directory. I'll have a look at etckeeper. I think I have seen it mentioned before but I never looked into it. Thanks.

tzervo · 2010-09-18 07:01:02

There is a technique in git called "detached worktree", first read about it in http://tensixtyone.com/perma/the-versio … -directory. I tried it today and it works, adapted to /etc. In a few words, you make a folder anywhere you want, say in somepath/etc.git.
Then you execute in that folder:

git init --bare
git config core.worktree /etc
export GIT_DIR=somepath/etc.git
export GIT_WORK_TREE=/etc

and you work with the worktree being /etc and the repo being elsewhere. The caveat is that you have to run the two export commands each time you work with that repo. The is a tool called mr tool (mr-git in AUR) that helps with managin git repos, if you also want to take a look at it. Hope that helps.

Last edited by tzervo (2010-09-18 07:02:22)

masterkorp · 2010-09-18 13:51:38

tzervo wrote:

There is a technique in git called "detached worktree", first read about it in http://tensixtyone.com/perma/the-versio … -directory. I tried it today and it works, adapted to /etc. In a few words, you make a folder anywhere you want, say in somepath/etc.git.
Then you execute in that folder:
git init --bare
git config core.worktree /etc
export GIT_DIR=somepath/etc.git
export GIT_WORK_TREE=/etc
and you work with the worktree being /etc and the repo being elsewhere. The caveat is that you have to run the two export commands each time you work with that repo. The is a tool called mr tool (mr-git in AUR) that helps with managin git repos, if you also want to take a look at it. Hope that helps.

thanks i will try for a few days and give you the feedback.

masterkorp · 2010-09-21 18:23:55

I am finding some trouble creating a repository for my home files, maybe is because the actual git repository is on the home directory it self it says an error when i add (eg. git add .basrc) a file that says the file doesn't exist in the path.
Any idea ?

tzervo · 2010-09-22 06:35:09

Try this, say if your git home repository is in
$your_git_home_repository_path:

GIT_DIR="$your_git_home_repository_path"
GIT_WORK_TREE="$HOME"
cd $HOME
git add .bashrc

Pointing to where your repo is with
GIT_DIR="$your_git_home_repository_path" essentially lets you perform
all git actions after you have cd'ed to $HOME, only the changes will be
made to your repo instead of going to a $HOME/.git.

masterkorp · 2010-09-22 10:07:08

tzervo wrote:

Try this, say if your git home repository is in
$your_git_home_repository_path:
GIT_DIR="$your_git_home_repository_path"
GIT_WORK_TREE="$HOME"
cd $HOME
git add .bashrc
Pointing to where your repo is with
GIT_DIR="$your_git_home_repository_path" essentially lets you perform
all git actions after you have cd'ed to $HOME, only the changes will be
made to your repo instead of going to a $HOME/.git.

It worked, but is strange because for the /etc repository (wich is located in $HOME/Documents/config_files/etc_files/etc.git) i need to be in the $GIT_DIR and execute the commands as i am in $GIT_WORK_TREE(/etc), for this i need to work alwyas in $GIT_WORK_TREE. Maybe because the $GIT_DIR (wich is located in $HOME/Documents/config_files/dot_files/dot.git) s a subdirectory of $GIT_WORK_TREE ($HOME) in this case ?

Arch Linux

#1 2010-09-16 13:13:05

Version controling system config files form another loaction with git

#2 2010-09-16 15:01:11

Re: Version controling system config files form another loaction with git

#3 2010-09-16 20:20:20

Re: Version controling system config files form another loaction with git

#4 2010-09-16 22:58:01

Re: Version controling system config files form another loaction with git

#5 2010-09-17 03:56:05

Re: Version controling system config files form another loaction with git

#6 2010-09-17 06:01:22

Re: Version controling system config files form another loaction with git

#7 2010-09-17 13:42:55

Re: Version controling system config files form another loaction with git

#8 2010-09-17 13:52:01

Re: Version controling system config files form another loaction with git

#9 2010-09-17 18:12:52

Re: Version controling system config files form another loaction with git

#10 2010-09-17 18:49:40

Re: Version controling system config files form another loaction with git

#11 2010-09-17 18:59:43

Re: Version controling system config files form another loaction with git

#12 2010-09-18 01:18:24

Re: Version controling system config files form another loaction with git

#13 2010-09-18 07:01:02

Re: Version controling system config files form another loaction with git

#14 2010-09-18 13:51:38

Re: Version controling system config files form another loaction with git

#15 2010-09-21 18:23:55

Re: Version controling system config files form another loaction with git

#16 2010-09-22 06:35:09

Re: Version controling system config files form another loaction with git

#17 2010-09-22 10:07:08

Re: Version controling system config files form another loaction with git

Board footer