You are not logged in.

#1 2015-12-28 14:04:51

Cube777
Member
Registered: 2015-09-15
Posts: 45
Website

dotgit - A comprehensive solution to managing your dotfiles

Hello fellow Archers,

Today I present to you dotgit - a solution to all your dotfile versioning/management needs. Why another dotfiles manager? It combines what I consider as strengths from the other dotfile managers out there like yadm, vcsh and homeshick while keeping it extremely simple to use and maintain, but still not taking any power away from the user. Some of the key features of dotgit:

* Keep multiple configs in a single repository (for instance different hosts) while keeping some of the files linked to each other (like a shared .bashrc) and others apart (different .vimrc)
* Supports symlinking and normal copying into your home folder
* Very easy to use syntax
* Since it is only one repository you can use all of your usual git magic
* Don't know how to use git? No problem - dotgit does it for you
* Simple file hierarchy - you will have no problem accessing your files without dotgit
* Just a bash script with no other dependencies (besides git and gnupg which you probably already have)
* Change the filelist, run one command and everything will be set up, no sweat
* Full support for directories
* Secure implementation of GnuPG AES encryption

For a more detailed comparison to other dotfiles managers and also a more in detail explanation of dotgit, please do visit the project Github repo.

Shameless Self-Promotion: If you like dotgit, please do star/vote the project on Github/AUR - It's great to see that other people like using it smile

Github repo
AUR Package
AUR Development package
Practical usage example (my own dotfiles github repo)

It would be great to hear what you guys think - have a good one

Last edited by Cube777 (2017-09-22 16:10:51)


dotgit - A comprehensive solution to managing your dotfiles

Offline

#2 2015-12-29 14:51:07

Cube777
Member
Registered: 2015-09-15
Posts: 45
Website

Re: dotgit - A comprehensive solution to managing your dotfiles

Bump! Again any feedback would be awesome smile If I should update the original post to read better or anything please do tell  - dying to know if this has any use for someone tongue


dotgit - A comprehensive solution to managing your dotfiles

Offline

#3 2015-12-29 22:10:42

sesese9
Member
From: California, United States
Registered: 2015-04-01
Posts: 75
Website

Re: dotgit - A comprehensive solution to managing your dotfiles

This looks like something that I've been looking for. I'll try it! I should have some feedback in a day or two.


Just trying to figure things out in this world

Offline

#4 2015-12-30 09:04:44

Cube777
Member
Registered: 2015-09-15
Posts: 45
Website

Re: dotgit - A comprehensive solution to managing your dotfiles

Thank you very much, I appreciate it smile


dotgit - A comprehensive solution to managing your dotfiles

Offline

#5 2016-03-05 13:25:12

seiichiro0185
Member
From: Leipzig/Germany
Registered: 2009-04-09
Posts: 226
Website

Re: dotgit - A comprehensive solution to managing your dotfiles

I use dotgit for some weeks now, and I have to say I really like it. It works without installing anything (except git, which is on my machines anyway), and does the job. I really like the hosts/categories feature btw wink

So far I didn't find any bugs or problems, it just works.

So thanks for creating dotgit!


My System: Dell XPS 13 | i7-7560U | 16GB RAM | 512GB SSD | FHD Screen | Arch Linux
My Workstation/Server: Supermicro X11SSZ-F | Xeon E3-1245 v6 | 64GB RAM | 1TB SSD Raid 1 + 6TB HDD ZFS Raid Z1 | Proxmox VE
My Stuff at Github: github
My Homepage: Seiichiros HP

Offline

#6 2016-03-09 22:40:22

aboe
Member
From: Netherlands
Registered: 2006-10-23
Posts: 19

Re: dotgit - A comprehensive solution to managing your dotfiles

Thanks for this shell script, I think it will help me backup important config files.

Offline

#7 2016-03-13 13:53:59

Cube777
Member
Registered: 2015-09-15
Posts: 45
Website

Re: dotgit - A comprehensive solution to managing your dotfiles

I am really glad there are people able to use this smile Thanks for the replies, I appreciate it!


dotgit - A comprehensive solution to managing your dotfiles

Offline

#8 2016-03-21 17:50:59

lino
Member
From: Wanne-Eickel
Registered: 2009-04-06
Posts: 16

Re: dotgit - A comprehensive solution to managing your dotfiles

Very cool idea, I'd very much like to use, but I dont get it running..

I've done "dotgit init" in my "~/.dotfiles" directory.
I've added some files to the filelist.
I've added my repo using "dotgit add-remote %repo_uri%".

When I run "dotgit update", I'm getting the help messages of the script.

Isnt that how i'ts supposed to work?
do I need to "init" in my $HOME?

Cheers!

edit: maybe it matters, I'm using zsh.

edit2: Nevermind, I tried using bash and it worked. Sort of. It closes if I dont choose to create a commit message, and it leaves out my ".config/interrobang/config" file for some reason...

Last edited by lino (2016-03-21 18:12:48)


»A keystroke is worth a thousand clicks«

Offline

#9 2016-03-21 20:28:33

Cube777
Member
Registered: 2015-09-15
Posts: 45
Website

Re: dotgit - A comprehensive solution to managing your dotfiles

Hi lino, thanks for trying out dotgit smile Sorry to hear there is problems, I am glad the initial one was worked out although I couldn't replicate it using the steps you provided (Using zsh worked fine for me, is your bash executable located under /bin?). After choosing not to generate a commit message it should close as this leaves the "git commit" part up to the user (it is after all only a git repo), so that is working normally. Would you mind posting your filelist? Maybe the problem with the interrobang config file lies there


dotgit - A comprehensive solution to managing your dotfiles

Offline

#10 2016-03-22 06:21:34

lino
Member
From: Wanne-Eickel
Registered: 2009-04-06
Posts: 16

Re: dotgit - A comprehensive solution to managing your dotfiles

Hi Cube777,
it was a quick test and I was tired, but I liked the idea very much!
Also, my zsh config is heavily customized, I'll try to sort out what the problem was. If you can run it through zshell I should be able to do so, too.
My filelist is easy, as said, it was a quick test:

.Xresources:desktop
.xinitrc:desktop
.config/nvim/init.vim:hh
.config/interrobang/config:desktop

But I believe it's rather some issue with my shell config.
I'll try one without loading my advanced config and let you know.
If there are problems with zsh in general, maybe I'll contribute something on github.

Thank you for your reply and have a nice day! smile


»A keystroke is worth a thousand clicks«

Offline

#11 2016-03-22 09:10:32

Cube777
Member
Registered: 2015-09-15
Posts: 45
Website

Re: dotgit - A comprehensive solution to managing your dotfiles

Hope all goes well, thanks for the reply - if there are problems with other shells in general (hopefully not, and there probably shouldn't be) I might rewrite this in C++ to make it shell-independent. Also, currently dotgit copies the files from the repo to your home dir, I don't know if you guys would prefer to use hardlinks? If there's interest I will release an update with the option


dotgit - A comprehensive solution to managing your dotfiles

Offline

#12 2016-03-23 05:57:57

mtrokic
Member
From: Montreal
Registered: 2009-10-24
Posts: 56
Website

Re: dotgit - A comprehensive solution to managing your dotfiles

I have a huge CAUTIONary tale I'd like to share. I just learned the hard way that if you run

dotgit update

without putting anything into the filelist, the script will start deleting dotfiles from the home directory and subdirectories automatically. I lost my .bashrc, my .ssh/known_hosts files, etc... like this. Luckily I had backups. In any case, this needs to be fixed as this is a serious flaw in the script, otherwise it works very nicely.

Offline

#13 2016-03-23 09:36:31

Cube777
Member
Registered: 2015-09-15
Posts: 45
Website

Re: dotgit - A comprehensive solution to managing your dotfiles

Hi mtrokic, I'm sorry to hear that happened, I'm glad you had backups. I'm trying to replicate this but I can't get it to happen - can you please post your original steps? Things I tried:

  • Running dotgit update in a subdirectory of the home folder with an completely empty filelist

  • Running dotgit init in the home folder (which you shouldn't do) and the dotgit update in the home folder (which you also shouldn't do)

I will put in an extra safeguard or two so long and release an update later today

EDIT: Update released, one of the safeguards now is to check for an empty filelist (although it never should've been a problem, still can't find code that could've caused this. To anyone with bash programming experience I would really appreciate it if you can help me find it).

Last edited by Cube777 (2016-03-23 11:00:33)


dotgit - A comprehensive solution to managing your dotfiles

Offline

#14 2016-03-23 17:14:28

mtrokic
Member
From: Montreal
Registered: 2009-10-24
Posts: 56
Website

Re: dotgit - A comprehensive solution to managing your dotfiles

Hi Cube777, thanks for the reply. I just checked my bash log and I'm ashamed to say, the error I reported is due to my carelessness. I don't know what happened, or what I was thinking, but I was under the impression I was already in my .dotfiles folder when I ran dotgit init, dotgit update. It turns out I was in the home folder. I'm really sorry to have raised to concern, but it turns out the error arises when you run dotgit update on an empty filelist when you're in the home folder.

In any case, thanks for the script. I appreciate your hard work and the quick response.

Offline

#15 2016-03-23 17:30:37

Cube777
Member
Registered: 2015-09-15
Posts: 45
Website

Re: dotgit - A comprehensive solution to managing your dotfiles

No problem smile Just glad it's worked out - the new safeguards should've been in place anyways so its a good thing this happened. Have a good one


dotgit - A comprehensive solution to managing your dotfiles

Offline

#16 2016-03-23 18:34:25

null
Member
Registered: 2009-05-06
Posts: 398

Re: dotgit - A comprehensive solution to managing your dotfiles

I just had a look for curiosity because I recently did a similar thing to distribute my config to all my computers.

One thing I don't understand though:
Why do you implement the restore as copy & overwrite? Because you save everything in a git repository you really don't need to have the safeguard "do you really want to overwrite this" if the old version also comes from git. There is a backup anyway and the person has to acknowledge the overwrite of an old git file.
I create links to my git repository so the only update I'd need is a `git pull`, if there are no new files, and totally no interaction. And also because I'd never remove anything else than links with this script the danger of data loss is non existent.
Are there any advantages of files here I didn't see?


Also a few shell things I've noticed:

* line 182:

if [ ! -z "$(diff "$i" "$HOME/$i")" ]

Could just be

if ! cmp -s "$i" "$HOME/$i"

(cmp is much faster because it stops after the first mismatch.
Here you could also save yourself a test execution and subshell)

* Often:

[ ! -z "$VAR" ]

instead of

[ -n "$VAR" ]

* Also stuff like:

if [ ! -z "$(git remote)" ]

could just be

if ! git remote >> /dev/null

(Saving the subshell and test. That's what they return 0 for ;)

Offline

#17 2016-03-23 19:45:50

Cube777
Member
Registered: 2015-09-15
Posts: 45
Website

Re: dotgit - A comprehensive solution to managing your dotfiles

Hi null, thanks for checking through the code I appreciate it smile I wrote this script before I really looked at how other people do this and then I wrote this whole thing only to realize I should've used links. The only real advantage to the copy approach was that the config files could live on in the absence of the repository - which is not much of an advantage. This is also why I asked earlier if someone would be interested in the linking option (post #11), but I believe an option is the wrong approach to this. I'm currently on vacation so I have the time and I'm really looking for a project, I'm going to do a rewrite to rather follow the linking approach. In this approach restore will only be used to set up the original links. Thanks for the advice, I'm still getting into bash and the advice is valuable, appreciate it smile


dotgit - A comprehensive solution to managing your dotfiles

Offline

#18 2016-03-23 19:55:39

Rasi
Member
From: Germany
Registered: 2007-08-14
Posts: 1,914
Website

Re: dotgit - A comprehensive solution to managing your dotfiles

great. looking forward to that


He hoped and prayed that there wasn't an afterlife. Then he realized there was a contradiction involved here and merely hoped that there wasn't an afterlife.

Douglas Adams

Offline

#19 2016-03-24 01:52:03

Cube777
Member
Registered: 2015-09-15
Posts: 45
Website

Re: dotgit - A comprehensive solution to managing your dotfiles

Ok update is out smile I wrote this quite late, so be weary for bugs but from my tests I couldn't find any. You should read the new help message as it describes update and restore's new features. Hope all goes well

EDIT: Follow these steps to set up new dogit repo from old repo:
- If you only use this on one host you need to delete all the files and folders inside the dotfiles folder inside your dotgit repo and then run dotgit init and then dotgit update
- If you use multiple hosts start with one host, follow the previous step and then push your changes to your online remote. On each of the following hosts pull the newest repository, then run dotgit hard-restore to restore the newest version of all your dotfiles to the host. After you have done this run dotgit update to set up symlinks for new files. After this run dotgit restore to set up symlinks for the rest of the files. Repeat these steps for each host/category

Last edited by Cube777 (2016-03-24 10:22:04)


dotgit - A comprehensive solution to managing your dotfiles

Offline

#20 2016-03-24 05:47:00

seiichiro0185
Member
From: Leipzig/Germany
Registered: 2009-04-09
Posts: 226
Website

Re: dotgit - A comprehensive solution to managing your dotfiles

I just did a quick test run with the new version, the linking stuff works fine. Two little things I spotted:

- Even if the target file is already a link to the dotgit repo, the script asks if it should overwrite it -> I think this is unneccessary and quite tedious to manually acknowledge every file thats already managed
- If you just hit Enter to acknowledge, I get an error:

File .bashrc already exists in home directory. Do you want to overwrite it? [N/y] 
/home/seiichiro/dotgit/dotgit: line 145: [: ==: unary operator expected
/home/seiichiro/dotgit/dotgit: line 145: [: ==: unary operator expected

As always thanks for your work smile


My System: Dell XPS 13 | i7-7560U | 16GB RAM | 512GB SSD | FHD Screen | Arch Linux
My Workstation/Server: Supermicro X11SSZ-F | Xeon E3-1245 v6 | 64GB RAM | 1TB SSD Raid 1 + 6TB HDD ZFS Raid Z1 | Proxmox VE
My Stuff at Github: github
My Homepage: Seiichiros HP

Offline

#21 2016-03-24 07:48:30

Cube777
Member
Registered: 2015-09-15
Posts: 45
Website

Re: dotgit - A comprehensive solution to managing your dotfiles

Thanks for the testing, this made me also find another small bug which is also now fixed. Update release

EDIT: Follow these steps to set up new dogit repo from old repo:
- If you only use this on one host you need to delete all the files and folders inside the dotfiles folder inside your dotgit repo and then run dotgit init and then dotgit update
- If you use multiple hosts start with one host, follow the previous step and then push your changes to your online remote. On each of the following hosts pull the newest repository, then run dotgit hard-restore to restore the newest version of all your dotfiles to the host. After you have done this run dotgit update to set up symlinks for new files. After this run dotgit restore to set up symlinks for the rest of the files. Repeat these steps for each host/category

Last edited by Cube777 (2016-03-24 10:22:23)


dotgit - A comprehensive solution to managing your dotfiles

Offline

#22 2016-03-24 11:37:21

null
Member
Registered: 2009-05-06
Posts: 398

Re: dotgit - A comprehensive solution to managing your dotfiles

Cube777 wrote:

Hi null, thanks for checking through the code I appreciate it :)

You're welcome :)

Another few things:
** 46:

if [ ! -e "$REPO/filelist" ]; then

is only a test for existence but now it would be fine if you had a folder called $REPO/filelist..

** 51:

if [ -z "$(grep -v "//" "$REPO/filelist")" ]; then

You really should look into the programs you use in your script. Most of them have an option to surpress any output, so for example here you could just do:

if grep -q -v "//" "$REPO/filelist"; then

** You implement the (Y/n) question three times (in lnconf, generate and addRemote). Make a funktion out of it that returns 0 or 1! Also I'd recommend the use of a case statement for this.

** Also it's good practice to actually return some number other than zero if anything goes wrong. Like these first four exists. That would make your script usable from other scripts very easely. Also if functions sets a sane return value you can just always do `if function arg1; then`


(Also from what I see so far there should be no big problem implementing this script in pure sh. If you looking for a project here you could try make it work in FreeBSDs /bin/sh to make it usable in FreeBSD installations without bash installed. But be carefull. /bin/sh in ArchLinux is just bash ;)

Last edited by null (2016-03-24 11:38:50)

Offline

#23 2016-03-24 16:44:45

Cube777
Member
Registered: 2015-09-15
Posts: 45
Website

Re: dotgit - A comprehensive solution to managing your dotfiles

Thanks for advice again null smile I made the changes you proposed - all the exits are now accompanied by exit codes. Thanks for the idea, I have never played around with BSD and the likes, this might be the perfect excuse smile


dotgit - A comprehensive solution to managing your dotfiles

Offline

#24 2016-03-29 15:37:44

Cube777
Member
Registered: 2015-09-15
Posts: 45
Website

Re: dotgit - A comprehensive solution to managing your dotfiles

New version released - I found a bug with the restore function where it would not create parent dirs for a symlink, hope this did not affect anyone. If you guys have any problems or feature requests please don't hesitate to contact me here or on the github repo.


dotgit - A comprehensive solution to managing your dotfiles

Offline

#25 2016-04-08 18:40:53

oliver
Member
Registered: 2007-12-12
Posts: 448

Re: dotgit - A comprehensive solution to managing your dotfiles

My git knowledge pretty much ends at 'git clone <URL>' and I was hoping dotgit could help me with dotfiles across 3 machines.

I think the setup went as expected:

[oliver@vb_a_fonddulac dotgit]$ dotgit init
Initializing dotgit repo...
Initialized empty Git repository in /home/oliver/downloads/dotgit/.git/
Initialized new dotgit repo in /home/oliver/downloads/dotgit
[oliver@vb_a_fonddulac dotgit]$ dotgit add-remote https://github.com/oliver-cfc/dotgit
Added new repository https://github.com/oliver-cfc/dotgit

I added one test file to the filelist and ran the update

[oliver@vb_a_fonddulac dotgit]$ dotgit update
Commencing filelist update...
-->Adding      common              .config/test/arch1  
Commencing cleanup...
Entering common...
[oliver@vb_a_fonddulac dotgit]$

All was good so far, my test config file is now a symlink

[oliver@vb_a_fonddulac dotgit]$ ls -l ~/.config/test/arch1 
lrwxrwxrwx 1 oliver oliver 64 Apr  8 14:19 /home/oliver/.config/test/arch1 -> /home/oliver/downloads/dotgit/dotfiles/common/.config/test/arch1

The last thing is to push to the github repo, right?

[oliver@vb_a_fonddulac dotgit]$ dotgit generate
[master (root-commit) f2147dc] Changes: added common/.config/test/arch1
 2 files changed, 2 insertions(+)
 create mode 100644 dotfiles/common/.config/test/arch1
 create mode 100644 filelist
Remote detected, do you want to push (sync) to it? [Y/n] 
fatal: The current branch master has no upstream branch.
To push the current branch and set the remote as upstream, use

    git push --set-upstream origin master

So I did what it said

[oliver@vb_a_fonddulac dotgit]$ git push --set-upstream origin master
Username for 'https://github.com': xoxxxxxxx@xxxxx.com
Password for 'https://xxxxxx@xxxxx.com@github.com': 
To https://github.com/oliver-cfc/dotgit
 ! [rejected]        master -> master (fetch first)
error: failed to push some refs to 'https://github.com/oliver-cfc/dotgit'
hint: Updates were rejected because the remote contains work that you do
hint: not have locally. This is usually caused by another repository pushing
hint: to the same ref. You may want to first integrate the remote changes
hint: (e.g., 'git pull ...') before pushing again.
hint: See the 'Note about fast-forwards' in 'git push --help' for details.

I'm assuming this is a git problem more than dotgit problem but I'm not sure if this is something thats expected because I've missed a step or what.

Any help would be great.  Thanks

Offline

Board footer

Powered by FluxBB