Repository management scripts

Andrwe · 2010-03-01 08:19:59

Hi,

I recently began to build my own repository with programs from AUR I'm using really often.
Therefore I've written some scripts to manage it easier.
So here I want to share these scripts and if someone of you has some scripts too he could post them here.
Also if you have some requests which would improve the scripts just ask.

The first script I've written is called makerepopkg.bsh and is designed to build a package in a [wiki=DeveloperWiki:Building_in_a_Clean_Chroot]clean chroot[/wiki] and copy the builded package to a repository location.

More information can be found on this site: http://andrwe.org/doku.php/scripting/bash/makerepopkg

My second script is createfiles.py which is a port of this bash script http://projects.archlinux.org/dbscripts … -filelists which is used to build .files.tar.gz for the official repositories which is needed by pkgtools/pkgfile to get a list of files provided by a package.
Because the official script has a lot of dependencies of the other files of the dbscripts-git and I didn't wanted to investigate much time into their configurations I've written this python script.

More information can be found here: http://andrwe.org/doku.php/scripting/python/createfiles

So these are my scripts which I use to manage my repository.
I would appreciate feedback.

Last edited by Andrwe (2011-06-22 20:29:08)

Andrwe · 2010-05-06 13:07:13

Hi again,

I've improved my scripts a bit.
makerepopkg is now version 0.7.5 and the main new feature is supporting the build of .xz packages.
For more information see the link I've already posted.

At the moment I'm working on xz-support for createfiles.py but this isn't easy IMO because there aren't many documentations about the lzma module for python.
But I'm sure I'll get it working.

With this post I also want to introduce a new script I've written.
I've named it upcheck and it is designed to check a list of repositories against AUR for newer packages as long as the version is numerical only if it is alphanumerical differences in version will be shown means if the AUR package version is older than repo it will also be shown.
E.g.:
repo: 123a-1
AUR: 122a-1
This package will be shown because the versions are alphanumerical and aren't equal.
Here you can find the script with further information:
http://andrwe.dyndns.org/doku.php/blog/ … sh/upcheck

Last edited by Andrwe (2010-05-13 18:31:24)

Andrwe · 2010-05-13 18:33:34

So, now I got the new version of createfiles online.
It now supports xz-packages and got some little speed up.

Just follow the link in first post.

daneel971 · 2010-05-14 20:46:29

I tried with two repos but I had an error. This is the config:

config = {
      "local": {
      "i686": "/warehouse/repos/archlinux/packages/i686",
      "x86_64": "/warehouse/repos/archlinux/packages/x86_64",
      }
      "local-any": {
      "any": "/warehouse/repos/archlinux/packages/any",
      }
}

And this is the error:

  File "./createfiles.py", line 34
    "local-any": {
              ^
SyntaxError: invalid syntax

I suppose I misunderstood something...

Last edited by daneel971 (2010-05-14 20:47:14)

Daenyth · 2010-05-14 23:51:00

Will this work with makechrootpkg from the devtools?

Andrwe · 2010-05-16 18:14:49

daneel971 wrote:

I tried with two repos but I had an error. This is the config:

config = {
      "local": {
      "i686": "/warehouse/repos/archlinux/packages/i686",
      "x86_64": "/warehouse/repos/archlinux/packages/x86_64",
      }
      "local-any": {
      "any": "/warehouse/repos/archlinux/packages/any",
      }
}

You forgot the comma after the local end-brace, should be like this:

config = {
      "local": {
      "i686": "/warehouse/repos/archlinux/packages/i686",
      "x86_64": "/warehouse/repos/archlinux/packages/x86_64",
      },
      "local-any": {
      "any": "/warehouse/repos/archlinux/packages/any",
      }
}

@Daenyth:
What do you mean?
If you're refering to makerepopkg then yes because it uses makechrootpkg from devtools.

Stythys · 2010-06-10 01:21:57

any support planned for -any packages? like instead of building an -any package in both i686/x86_64 chroots and adding it to both databases, just building it once, adding it to the 'any' folder, and putting a symlink in the i686/x86_64 folders

Andrwe · 2010-06-10 06:49:50

Stythys wrote:

any support planned for -any packages? like instead of building an -any package in both i686/x86_64 chroots and adding it to both databases, just building it once, adding it to the 'any' folder, and putting a symlink in the i686/x86_64 folders

Yes, I've already thougth about some ways to implement this and tested some but the results weren't as good as I wanted them to be.
This support will defenitely come with the next releases but at the moment I've less time because of exams and work.
I think I'll make it next month.

Stythys · 2010-06-10 06:53:24

Andrwe wrote:

..I think I'll make it next month.

awesomeness. you da man

jelly · 2010-06-10 08:24:34

Hi Andrwe, you might want to check our arch-games scripts out
We use a git repo, for storing the PKGBUILD and use packages from AUR, which we compare with our repo. If you have any improvements email me or fork our git repo
http://github.com/Stythys/arch-games/tr … scripts%2F

Andrwe · 2010-07-16 11:52:00

@jelly
nice scripts could learn a lot from them.

New versions of createfile.py and makerepopkg.sh are online.

makerepopkg now fully supports any-packages using either its own directory and being symlinked correctly into the other architectures or being build once and copied into the other architectures.
The behavior can be configured in the file.

createfile.py is just a bugfix release because now there is build a symlink for the repository database which is now skipped by createfile.py

dpreilan · 2010-07-23 22:24:12

How do I get this scripts? Andrwe, your site is asking me to login and I can't find out how to register.

Andrwe · 2010-07-25 20:21:03

Now the links are working.

Andrwe · 2011-06-22 20:29:43

Verion 1.0.0 of makerepopkg is now released.

Changelog:
rewrite of all code
dependency check
configuration changed to files in $XDG_CONFIG_HOME/<scriptname>
profile-support (create, delete, overwrite, list)
symlink support for any-packages
improved error-handling
improved debug-possibility
added quiet option (only error output)

You now have to create profiles first and then can build a package using multiple profiles.

Last edited by Andrwe (2011-06-22 20:31:45)

steabert · 2011-08-04 19:45:16

I was looking at your makerepopkg.sh script and noticed the following code:

if ! which wget sed grep bash >/dev/null 2>&1
then
    echo "One of the following Dependencies is missing: wget, sed, grep, bash. Exit"
    exit 1
fi

In order to tell which dependency is missing, you could easily write something like:

dependencies=('wget' 'sed' 'grep' 'bash')
for program in ${dependencies[@]}
do
  if ! which $program >/dev/null 2>&1
  then
    echo "the following dependency is missing: $program. Exit"
  fi
done

Just a minor remark that might make it more informative in case of problems.

Last edited by steabert (2011-08-04 19:52:27)

Andrwe · 2011-08-05 10:33:52

Thanks.

I hadn't thought about it but I'll add it.

Arch Linux

#1 2010-03-01 08:19:59

Repository management scripts

#2 2010-05-06 13:07:13

Re: Repository management scripts

#3 2010-05-13 18:33:34

Re: Repository management scripts

#4 2010-05-14 20:46:29

Re: Repository management scripts

#5 2010-05-14 23:51:00

Re: Repository management scripts

#6 2010-05-16 18:14:49

Re: Repository management scripts

#7 2010-06-10 01:21:57

Re: Repository management scripts

#8 2010-06-10 06:49:50

Re: Repository management scripts

#9 2010-06-10 06:53:24

Re: Repository management scripts

#10 2010-06-10 08:24:34

Re: Repository management scripts

#11 2010-07-16 11:52:00

Re: Repository management scripts

#12 2010-07-23 22:24:12

Re: Repository management scripts

#13 2010-07-25 20:21:03

Re: Repository management scripts

#14 2011-06-22 20:29:43

Re: Repository management scripts

#15 2011-08-04 19:45:16

Re: Repository management scripts

#16 2011-08-05 10:33:52

Re: Repository management scripts

Board footer