You are not logged in.
Hey there Archers, just to reiterate from the aur-general ml (cause I know not everyone reads it)
I decided to work on this little project over the weekend.
It is a complete git clone of the AUR from the source and it will be
updated regularly (at least once a day) though probably as frequent as
30minutes (please see `Side Notes' for explanation).
What this means is that history for the AUR will now be possible!
Also, for those of you wanting to have an abs like tree for all of the
aur, it is now possible.
I hope this opens up new opportunities, and new possibilities for everyone.
To get the complete git repo, run:
git clone git://pkgbuild.com/aur-mirror.git
The web interface is at:
http://pkgbuild.com/git/aur-mirror.git/
Special Thanks to Ioni for enabling the git server & Bluewind for
setting up the web interface.
Cheers!
-Thomas Dziedzic
Side Note:
Although it will be updated at least once a day, it will most likely
be updated more frequently (around 30 minute intervals with a script),
I just can't guarantee that for now.
Work is ongoing to automate the sync with the aur tree, but at this
moment, there is no secure way of doing this.
Automating the sync will guarantee the intervals between syncs, I am
aiming for around 30 minute to 3 hour intervals between syncs.
A final goal might be to get continuous updates. Enjoy!
Last edited by tomd123 (2012-01-31 22:04:43)
Offline
Just cloned it, pretty cool thing
Offline
Just cloned it, pretty cool thing
Yup, what I really love to see though is the git diffs between different commits.
(we finally have history!)
For instance, this is what you get when you run the cleanup script on the aur server: (warning, may take a while to load)
http://pkgbuild.com/git/aur.git/commit/ … 2061db2c34
Last edited by tomd123 (2011-02-06 21:06:21)
Offline
What! Amazing stuff! Even my useless kernel PKGBUILD is up there! This will definitely add new possibilities to Arch. Thank you.
Offline
Stickied.
ᶘ ᵒᴥᵒᶅ
Offline
Thanks for implementing a version control system for AUR. But this doesn't fulfill the purpose of using a VCS for AUR in the first place. Checking out a 450 MB to be able to maintain custom PKGBUILDs of a few packages is useless IMO. Since git does not allow per directory checkouts like svn does, it would have been better if each package was given its own repo instead of one common repo for the whole AUR tree. The same problem exists with svntogit repo for official repos PKGBUILDs in http://projects.archlinux.org/ . But a single git repo is ok if this is only for viewing purpose.
My new forum user/nick name is "the.ridikulus.rat" .
Offline
Thanks for implementing a version control system for AUR. But this doesn't fulfill the purpose of using a VCS for AUR in the first place. Checking out a 450 MB to be able to maintain custom PKGBUILDs of a few packages is useless IMO. Since git does not allow per directory checkouts like svn does, it would have been better if each package was given its own repo instead of one common repo for the whole AUR tree. The same problem exists with svntogit repo for official repos PKGBUILDs in http://projects.archlinux.org/ . But a single git repo is ok if this is only for viewing purpose.
Thanks for your concern, I will take a look at possible options later this evening,
#git gave me the following link http://www.kernel.org/pub/software/scm/ … e_checkout
If anything I can always convert to svn
Offline
git sparse checkout is not exactly what i mentioned. For sparse checkout to work , still the entire 60 MB clone of the repo has to be performed. There is currently no feature in git to do like
git clone git://pkgbuild.com/aur.git/grub2-efi-bzr
where it downloads only part relevant to grub2-efi-bzr instead of the whole 60 MB (or is it 400 MB) like in case of svn. Converting to svn will solve the problem but will not provide the advantages of a distributed VCS. I don't know whether bzr can provide best of both.
What i meant is something like this http://pkgs.fedoraproject.org/gitweb/ (each package has its own repo). Although i have to perform a clone for every package i want, this can be performed easily with a shell script by the user.
Also regarding to the discussion in the ML, storing tarballs in git is not feasible as git is not know to be useful to store binaries. Storing tarballs instead of the extracted test files like PKGBUILDs and the post install scripts will cause the git repo to take up lots of server space in the long run since the diff between tarballs are stored by git instead of diff between text files.
See http://stackoverflow.com/questions/5405 … s-with-git , http://stackoverflow.com/questions/9847 … git-limits , and http://stackoverflow.com/questions/2266 … nary-files . Thanks for considering my suggestions. I hope something similar is implemented for ABS so that svn usage can be removed altogether.
Last edited by skodabenz (2011-02-07 16:52:18)
My new forum user/nick name is "the.ridikulus.rat" .
Offline
git sparse checkout is not exactly what i mentioned. For sparse checkout to work , still the entire 60 MB clone of the repo has to be performed. There is currently no feature in git to do like
git clone git://pkgbuild.com/aur.git/grub2-efi-bzr
where it downloads only part relevant to grub2-efi-bzr instead of the whole 60 MB (or is it 400 MB) like in case of svn. Converting to svn will solve the problem but will not provide the advantages of a distributed VCS. I don't know whether bzr can provide best of both.
Well, as one guy put it in #git, it is only the initial 60MB download that is a small fee, but everything after that is free.
Currently I have no short term goals of implementing a per package repo, my task for now is getting per package commits.
After I deploy the per package system, I will look into the possibilities of adding support for per directory repos.
One thing that I haven't had a chance to explore that looks promising is, http://gitslave.sourceforge.net/
Also,
I would like to know what you guys prefer, the digested all in one commits, or would you rather prefer per package commits (stuff I'm working on right now)?
Last edited by tomd123 (2011-02-08 07:19:58)
Offline
Sorry, git repo is down for now.
Last edited by tomd123 (2011-02-08 20:01:32)
Offline
git repo back up
Offline
Great!
Offline
I have now set everything up in a cron job, so 30 minute intervals should now be guaranteed enjoy
Offline
Great Work
Archlinux x86_64 | post-engineering | last.fm
Offline
This is beautiful, many thanks
Offline
The aur git mirror hasn't been updated for 7 hours because of an error on the server which caused files to be lost. Luckily I pastebinned my update script a while back and managed to find it. I will try to get the aur git mirror updating soon enough.
Offline
aur mirror should be syncing regularly again!
Offline
whoa, great. it is a bit slow, but still !great!. This enhances my wrapper around abs a lot. nice nice nice :-)
Offline
I would like to know what you guys prefer, the digested all in one commits, or would you rather prefer per package commits (stuff I'm working on right now)?
per-package.
I wonder how much that would increase storage requirements, but I think probably not much.
< Daenyth> and he works prolifically
4 8 15 16 23 42
Offline
Since git does not allow per directory checkouts like svn does.
Actually it does since 1.7, they're called sparse checkouts; http://www.kernel.org/pub/software/scm/ … -1.7.0.txt. I think it will still grab the whole history, but you can also do a shallow checkout and grab only the first couple of commits (you likely won't need the history).
Offline
skodabenz wrote:Since git does not allow per directory checkouts like svn does.
Actually it does since 1.7, they're called sparse checkouts; http://www.kernel.org/pub/software/scm/ … -1.7.0.txt. I think it will still grab the whole history, but you can also do a shallow checkout and grab only the first couple of commits (you likely won't need the history).
This will still take lots of bandwidth space in terms of download. I am not bothered about the amount of space the git repo mya take in my system. I am instead bothered about amount of data downloaded (initially) by "git clone" to create a local copy of the repo.
My new forum user/nick name is "the.ridikulus.rat" .
Offline
released a quick abs-like tool which maintains a repo under /var/aur (as opposed to /var/abs for abs)
just run sudo aur and it should update/clone the aur git repo:
script at:
https://github.com/gostrc/aur
created a package for it:
aur-git
https://aur.archlinux.org/packages.php?ID=50660
Last edited by tomd123 (2011-07-13 16:17:57)
Offline
Very nice! Thanks
Offline
Hey guys, what are your opinions about making each package its own repository and connecting them as submodules under an umbrella project? This would allow someone to clone individual projects, make changes, and submit pull requests to the maintainer if he or she also happens to be using git. This end result would be something like the vim-scripts account on github.
I'm working on a simple script that separates each subdirectory in the aur repository into its own top level project with git filter-branch --subdirectory-filter. I created an 'aur' user on github and can mass import these into github with their deveoper api.
Last edited by moonboots (2011-11-07 23:22:53)
Offline