You are not logged in.

#1 2008-05-25 22:34:18

carlocci
Member
From: Padova - Italy
Registered: 2008-02-12
Posts: 368

repacman, regenerate an installed package

edit: the script will be available in pacman 3.2 when it's released (HOORAY!)

Meanwhile you can grab the most recent version of the script here:
http://projects.archlinux.org/?p=pacman … an;hb=HEAD




Hi,
I'm one of those guys who is always low on space:

Filesystem            Size  Used Avail Use% Mounted on
/dev/sda3              44G   43G  335M 100% /

I actually recovered those 300 mB by running pacman -Scc, but I forgot to safekeep some packages (especially truecrypt and the 2.6.24 kernel), so if this kernel upgrade goes wrong I can't downgrade.
Well, I couldn't downgrade because lately I've been trying to learn a little bit of bash scripting, so today I came up with this script which rebuilds an installed package using the metadata in the pacman db.

#!/bin/bash

#
# repacman 0.1
# Recreate a package from a running system
#

pac_db='/var/lib/pacman/local'
work_dir=$PWD
IFS=$'\n'

mkdir $work_dir/pkg
cd $work_dir/pkg || exit 1

echo 
echo Processing desc and generating .PKGINFO metadata
echo "# Generated by repacman 0.1"    >> .PKGINFO
echo "# $(date)"            >> .PKGINFO

for i in $(cat $pac_db/$1/desc) ; do

    if [[ "$i" =~ %*[A-Z]% ]] ; then
        current=$i
        continue
    fi
        case "$current" in
        %NAME%)
           echo "pkgname = $i"    >> .PKGINFO
           pkgname=$i
        ;;
        %VERSION%)
           echo "pkgver = $i"    >> .PKGINFO
           pkgver=$i
           ;;
        %DESC%)
           echo "pkgdesc = $i"    >> .PKGINFO
        ;;
        %URL%)
           echo "url = $i"    >> .PKGINFO
        ;;
        %LICENSE%)
           echo "license = $i"    >> .PKGINFO
        ;;
        %ARCH%)
           echo "arch = $i"    >> .PKGINFO
           pkgarch=$i
        ;;
        %BUILDDATE%)
           echo "builddate = $i" >> .PKGINFO
        ;;
        %PACKAGER%)
           echo "packager = $i"    >> .PKGINFO
        ;;
        %SIZE%)
           echo "size = $i"    >> .PKGINFO
        ;;
        %GROUPS%)
           echo "group = $i"    >> .PKGINFO
        ;;
    esac
done


echo Processing files and copying package files
for i in $(cat $pac_db/$1/files) ; do

    if [[ "$i" =~ %*[A-Z]% ]] ; then
        current=$i
        continue
    fi

    case $current in
        %FILES%)
           if [ -d /$i ]; then
            mkdir $i
           fi
           if [ -f /$i ]; then
            install -D /$i $i
           fi
        ;;

        %BACKUP%)
           echo "backup = $i"    >> .PKGINFO
        ;;
    esac
done


echo Adding install commands and changelogs, if any
if [ -f $pac_db/$1/install ] ; then
    cp $pac_db/$1/install $work_dir/pkg/.install
fi
if [ -f $pac_db/$1/changelog ] ; then
    cp $pac_db/$1/changelog $work_dir/pkg/.changelog
fi


echo Processing depends and adding the dependancies to the .PKGINFO
for i in $(cat $pac_db/$1/depends) ; do
    if [[ "$i" =~ %*[A-Z]% ]] ; then
        current=$i
        continue
    fi
    case "$current" in
        %DEPENDS%)
           echo "depend = $i"    >> .PKGINFO
        ;;
        %OPTDEPENDS%)
           echo "optdepend = $i" >> .PKGINFO
        ;;
        %CONFLICTS%)
           echo "conflict = $i"    >> .PKGINFO
        ;;
        %PROVIDES%)
           echo "provide = $i"    >> .PKGINFO
        ;;
    esac
done

echo 
echo Generating the package...
bsdtar -czf ../"$pkgname-$pkgver-$pkgarch.tar.gz" $(ls -A)

echo Removing temporary files...
cd $work_dir || exit 1
rm -r pkg/

This is one of my first scripts I write from scratch, so suggestions to improve my scripting style are very welcome.
I couldn't find %REPLACES% for the replace = in any of the package I have installed, but I think %CONFLICTS% is enough?

I think I will clear my cache everytime, and then, when I need to backup some pkgs before an upgrade, I will use the script (at least I won't have to keep the same files twice on my hard drive).


edit: understood what %REASON% meant

Last edited by carlocci (2008-06-23 15:16:55)

Offline

#2 2008-05-25 23:57:53

Allan
Pacman
From: Brisbane, AU
Registered: 2007-06-09
Posts: 11,365
Website

Re: repacman, regenerate an installed package

Ohh... nice.

Just so you know, there is an outdated version of this script (also called repacman) in the contrib directory of the pacman source.  Also, another version (possibly updated) here: http://foulmetal.free.fr/archlinux/repacman-0.98.tar.gz

My intention is to include this in the pacman-contrib package when it was updated/tested but I haven't had time to do this...  Now, I will probably borrow heavily from this script (unless you want to combine them and submit it to the pacman-dev mailing list) big_smile

Offline

#3 2008-05-26 14:57:42

carlocci
Member
From: Padova - Italy
Registered: 2008-02-12
Posts: 368

Re: repacman, regenerate an installed package

Allan wrote:

Ohh... nice.

Just so you know, there is an outdated version of this script (also called repacman) in the contrib directory of the pacman source.

Great minds think alike! smile

Also, another version (possibly updated) here: http://foulmetal.free.fr/archlinux/repacman-0.98.tar.gz

Thank you, that looks so much more complete and structured. I think I will fiddle with bash functions next: for once I think I could unify desc, files and depends under one function.

My intention is to include this in the pacman-contrib package when it was updated/tested but I haven't had time to do this...  Now, I will probably borrow heavily from this script (unless you want to combine them and submit it to the pacman-dev mailing list) big_smile

Well, it would be nice if pacman could autobackup packages: pacman -U installs, pacman -Rp removes and recreates the package

Last edited by carlocci (2008-05-26 14:57:54)

Offline

#4 2008-05-26 23:15:26

carlocci
Member
From: Padova - Italy
Registered: 2008-02-12
Posts: 368

Re: repacman, regenerate an installed package

Here we are, new version:
this one figures out takes the db path and pkg cache dir from pacman.conf, or default them
performs some checks, like asking to use the package you already have in the cache if any
I unified the part which generates .PKGINFO and used better variable names
Changed the name

#!/bin/bash
#
#   bacman: recreate a package from a running system
#   This script rebuilds an already installed package using metadata
#   stored into the pacman database and system files
#
#   (c) 2008 - locci <carlocci_at_gmail_dot_com>
#   
#   This program is free software; you can redistribute it and/or modify
#   it under the terms of the GNU General Public License as published by
#   the Free Software Foundation; either version 2 of the License, or
#   (at your option) any later version.
# 
#   This program is distributed in the hope that it will be useful,
#   but WITHOUT ANY WARRANTY; without even the implied warranty of
#   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
#   GNU General Public License for more details.
#  
#   You should have received a copy of the GNU General Public License
#   along with this program.  If not, see <http://www.gnu.org/licenses/>.
#

_bacman_ver="0.1.½"

#
# Setting env variables
#
IFS=$'\n'
eval $(awk '/DBPath|CacheDir/ {print $1$2$3}' /etc/pacman.conf)
pac_db=${DBPath:-'/var/lib/pacman/'}/local/
pac_cache=${CacheDir:-'/var/cache/pacman/pkg/'}
work_dir=$PWD
pkg_name="$1"
pkg_dir=$(echo $pac_db/$pkg_name-[0-9]*)

#
# Checks everything is in place
#
if [ $# -ne 1 ] ; then
    echo bacman $_bacman_ver
    echo "Usage: ./bacman <installed package name>"
    exit 1
fi

if [ ! -d $pac_db ] ; then
    echo There is no pacman database directory in $pac_db
    echo $pac_db is missing
    exit 1
fi
if [ ! -d $pac_cache ] ; then
    echo Warning, there is no pacman pkg cache directory
    echo $pac_cache is missing
    unset pac_cache
fi

if [ ! -d $pkg_dir ] ; then
    echo There is no package with this name, $pkg_name
    echo Maybe you are trying to pass the directory name, instead of the package name
    exit 1
fi

if [ -n $pac_cache ] && [ -f $pac_cache/${pkg_dir##*/}*.tar.gz ] ; then
    echo The package already exists in your cache
    read -n 1 -p 'Do you want to fetch it instead ? (y/N) '
    case $REPLY in
           y|Y)
           cp $pac_cache/${pkg_dir##*/}*.pkg.tar.gz .
           exit 0
        ;;
        *) ;;
    esac
fi

#
# Begin 
#
mkdir $work_dir/pkg
cd $work_dir/pkg || exit 1

#
# .PKGINFO stuff
#
echo 
echo Generating .PKGINFO metadata...
echo "# Generated by bacman $_bacman_ver"    > .PKGINFO
echo "# $(LC_ALL=C date)"            >> .PKGINFO
echo "#"                    >> .PKGINFO

for i in $(cat $pkg_dir/{desc,files,depends}) ; do

    if [[ "$i" =~ %[A-Z]*% ]] ; then
        current=$i
        continue
    fi
        case "$current" in

#desc
        %NAME%)
           echo "pkgname = $i"    >> .PKGINFO
           pkgname=$i
        ;;
        %VERSION%)
           echo "pkgver = $i"    >> .PKGINFO
           pkgver=$i
           ;;
        %DESC%)
           echo "pkgdesc = $i"    >> .PKGINFO
        ;;
        %URL%)
           echo "url = $i"    >> .PKGINFO
        ;;
        %LICENSE%)
           echo "license = $i"    >> .PKGINFO
        ;;
        %ARCH%)
           echo "arch = $i"    >> .PKGINFO
           arch=$i
        ;;
        %BUILDDATE%)
           echo "builddate = $i" >> .PKGINFO
        ;;
        %PACKAGER%)
           echo "packager = $i"    >> .PKGINFO
        ;;
        %SIZE%)
           echo "size = $i"    >> .PKGINFO
        ;;
        %GROUPS%)
           echo "group = $i"    >> .PKGINFO
        ;;

#files
        %BACKUP%)
           echo "backup = $i"   >> .PKGINFO
        ;;


#depends
                %DEPENDS%)
                   echo "depend = $i"   >> .PKGINFO
                ;;
                %OPTDEPENDS%)
                   echo "optdepend = $i" >> .PKGINFO
                ;;
                %CONFLICTS%)
                   echo "conflict = $i" >> .PKGINFO
                ;;
                %PROVIDES%)
                   echo "provide = $i"  >> .PKGINFO
                ;;

    esac
done

#
# Files stuff
#
echo Copying package files...
for i in $(cat $pkg_dir/files) ; do

    if [[ "$i" =~ %[A-Z]*% ]] ; then
        current=$i
        continue
    fi

    case $current in
        %FILES%)
           if [ -d /$i ]; then
            mkdir $i
           fi
           if [ -f /$i ]; then
            install -D /$i $i
           fi
        ;;
    esac
done


echo Adding install commands and changelogs, if any...
if [ -f $pkg_dir/install ] ; then
    cp $pkg_dir/install $work_dir/pkg/.install
fi
if  [ -f $pkg_dir/changelog ] ; then
    cp $pkg_dir/changelog $work_dir/pkg/.changelog
fi

#
# 
#
echo 
echo Generating the package...
bsdtar -czf ../"$pkgname-$pkgver-$arch.tar.gz" $(ls -A)

echo Removing temporary files...
cd $work_dir || exit 1
rm -r pkg/

exit 0

#
# End
#

Last edited by carlocci (2008-05-26 23:20:49)

Offline

#5 2008-05-27 06:44:37

shining
Pacman Developer
Registered: 2006-05-10
Posts: 2,043

Re: repacman, regenerate an installed package

I just made a very quick testing, it does not seem to work bad.
You made a typo though, it is provides, not provide.


pacman roulette : pacman -S $(pacman -Slq | LANG=C sort -R | head -n $((RANDOM % 10)))

Offline

#6 2008-05-27 07:09:14

Allan
Pacman
From: Brisbane, AU
Registered: 2007-06-09
Posts: 11,365
Website

Re: repacman, regenerate an installed package

I am going to add a couple of suggestions that you may or may not consider to be useful....

You could change the builddate to be the date the package is remade on.  Also, packager could take the value in /etc/makepkg.conf or default to something sensible.  The size might have changed with any changed configuration files that you are repacking (but the difference is probably too small to care about).

Offline

#7 2008-05-27 13:45:55

carlocci
Member
From: Padova - Italy
Registered: 2008-02-12
Posts: 368

Re: repacman, regenerate an installed package

shining wrote:

You made a typo though, it is provides, not provide.

Thank you, the esses were a little bit confusing smile

As suggested by Allan, I integrated some data from makepkg.conf (packager and PKGDEST), I might include the emptydirs flag and strip flag as soon as I figure out bash arrays. Even if the original idea was to create the exact same original package (I forgot about /etc smile ).
I added more checks and warnings.
I thought about using /tmp and mktemp for the temporary pkg directory name, but I tried to mimic the behaviour of makepkg: I removed the line deleting pkg/
I added a correct size calculator
I'm thinking about setting a variable for each field and then writing them to the PKGCONFIG.


#!/bin/bash
#
#   bacman: recreate a package from a running system
#   This script rebuilds an already installed package using metadata
#   stored into the pacman database and system files
#
#   (c) 2008 - locci <carlocci_at_gmail_dot_com>
#   
#   This program is free software; you can redistribute it and/or modify
#   it under the terms of the GNU General Public License as published by
#   the Free Software Foundation; either version 2 of the License, or
#   (at your option) any later version.
# 
#   This program is distributed in the hope that it will be useful,
#   but WITHOUT ANY WARRANTY; without even the implied warranty of
#   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
#   GNU General Public License for more details.
#  
#   You should have received a copy of the GNU General Public License
#   along with this program.  If not, see <http://www.gnu.org/licenses/>.
#

_bacman_ver="0.1.½"

#
# Setting env variables
#
IFS=$'\n'
eval $(awk '/DBPath|CacheDir/ {print $1$2$3}' /etc/pacman.conf)
pac_db="${DBPath:-/var/lib/pacman/}/local"
pac_cache="${CacheDir:-/var/cache/pacman/pkg/}"
work_dir="$PWD"
pkg_name="$1"
pkg_dir="$(echo $pac_db/$pkg_name-[0-9]*)"
eval $(awk '/PKGDEST|CARCH|PACKAGER/ {print $1$2$3}' /etc/makepkg.conf)
pkg_dest="${PKGDEST:-$work_dir}"
pkg_arch=${CARCH:-'noarch'}
pkg_pkger=${PACKAGER:-'Unknown Packager'}

#
# Checks everything is in place
#
if [ $# -ne 1 ] ; then
    echo bacman $_bacman_ver
    echo "Usage: ./bacman <installed package name>"
    exit 1
fi

if [ ! -d "$pac_db" ] ; then
    echo There is no pacman database directory in $pac_db
    echo $pac_db is missing
    exit 1
fi
if [ ! -d "$pac_cache" ] ; then
    echo Warning, there is no pacman pkg cache directory
    echo $pac_cache is missing
    unset pac_cache
fi

if [ ! -d "$pkg_dir" ] ; then
    echo There is no package with this name, $pkg_name
    echo Maybe you are trying to pass the directory name, instead of the package name
    exit 1
fi

if [ -n "$pac_cache" ] && [ -f "$pac_cache"/"${pkg_dir##*/}"*.tar.gz ] ; then
    echo The package already exists in your cache
    read -n 1 -p 'Do you want to fetch it instead ? (y/N) '
    case $REPLY in
           y|Y)
           cp "$pac_cache"/"${pkg_dir##*/}"*.pkg.tar.gz "$pkg_dest"
           exit 0
        ;;
        *) echo ""
        ;;
    esac
fi

#
# Begin 
#
echo Packet: ${pkg_dir##*/}
mkdir "$work_dir"/pkg || exit 1
cd "$work_dir"/pkg || exit 1

#
# File copying
#
echo Copying package files...
for i in $(cat "$pkg_dir"/files) ; do

    if [[ "$i" =~ %[A-Z]*% ]] ; then
        current=$i
        continue
    fi

    case $current in
        %FILES%)
           if [ -d "/$i" ]; then
            mkdir "$i"
           elif [ -f "/$i" ]; then
            install -D "/$i" "$i"
           else
            echo "/$i" is missing: this might result in a broken package
           fi
        ;;
    esac
    pkg_size=$(du -sb | awk '{print $1}')
done

echo Adding install commands and changelogs, if any...
if [ -f "$pkg_dir/install" ] ; then
    cp "$pkg_dir/install" "$work_dir/pkg/.install"
fi
if  [ -f $pkg_dir/changelog ] ; then
    cp "$pkg_dir/changelog" "$work_dir/pkg/.changelog"
fi

#
# .PKGINFO stuff
#
echo 
echo Generating .PKGINFO metadata...
echo "# Generated by bacman $_bacman_ver"    > .PKGINFO
echo "# $(LC_ALL=C date)"            >> .PKGINFO
echo "#"                    >> .PKGINFO

for i in $(cat "$pkg_dir"/{desc,files,depends}) ; do

    if [[ "$i" =~ %[A-Z]*% ]] ; then
        current=$i
        continue
    fi
        case "$current" in

        %NAME%)                    #desc
           echo "pkgname = $i"    >> .PKGINFO
           pkgname=$i
        ;;
        %VERSION%)
           echo "pkgver = $i"    >> .PKGINFO
           pkgver=$i
           ;;
        %DESC%)
           echo "pkgdesc = $i"    >> .PKGINFO
        ;;
        %URL%)
           echo "url = $i"    >> .PKGINFO
        ;;
        %LICENSE%)
           echo "license = $i"    >> .PKGINFO
        ;;
        %ARCH%)
           echo "arch = $i"    >> .PKGINFO
        ;;
        %BUILDDATE%)
           echo "builddate = $(date -u "+%s")"    >> .PKGINFO
        ;;
        %PACKAGER%)
           echo "packager = $pkg_pkger"        >> .PKGINFO
        ;;
        %SIZE%)
           echo "size = $pkg_size"        >> .PKGINFO
        ;;
        %GROUPS%)
           echo "group = $i"    >> .PKGINFO
        ;;

        %BACKUP%)                #files
           echo "backup = $i"   >> .PKGINFO
        ;;

                %DEPENDS%)                #depends
                   echo "depend = $i"   >> .PKGINFO
                ;;
                %OPTDEPENDS%)
                   echo "optdepend = $i" >> .PKGINFO
                ;;
                %CONFLICTS%)
                   echo "conflict = $i" >> .PKGINFO
                ;;
                %PROVIDES%)
                   echo "provides = $i"  >> .PKGINFO
                ;;

    esac
done

#
# 
#
echo 
echo Generating the package...
bsdtar -czf "$pkg_dest/$pkgname-$pkgver-$pkg_arch.tar.gz" $(ls -A) || echo Cannot write to $pkg_dest: maybe disk is full or you do not have write access 

#echo Removing temporary files...
#cd $work_dir || exit 1
#rm -r pkg/

exit 0

#
# End
#

edit: damned quotes, but now it supports spaces in the path!
edit2: minor bugfixes

Last edited by carlocci (2008-05-27 15:31:01)

Offline

#8 2008-05-27 14:17:05

shining
Pacman Developer
Registered: 2006-05-10
Posts: 2,043

Re: repacman, regenerate an installed package

carlocci wrote:
shining wrote:

You made a typo though, it is provides, not provide.

Thank you, the esses were a little bit confusing smile

As suggested by Allan, I integrated some data from makepkg.conf (packager and PKGDEST), I might include the emptydirs flag and strip flag as soon as I figure out bash arrays. Even if the original idea was to create the exact same original package (I forgot about /etc smile ).
I added more checks and warnings.
I thought about using /tmp and mktemp for the temporary pkg directory name, but I tried to mimic the behaviour of makepkg: I removed the line deleting pkg/
I added a correct size calculator
I'm thinking about setting a variable for each field and then writing them to the PKGCONFIG.

Nice job, I will try to test it again this evening.

Just a reminder for me : if we replace the repacman in pacman repo by this much better alternative,
and if we also fix the following problem : http://bugs.archlinux.org/task/10459#comment28645
then this script will need to be updated as well. But not for now.

Anyway, this script looks good as it is imo.


pacman roulette : pacman -S $(pacman -Slq | LANG=C sort -R | head -n $((RANDOM % 10)))

Offline

#9 2008-05-28 04:22:12

Allan
Pacman
From: Brisbane, AU
Registered: 2007-06-09
Posts: 11,365
Website

Re: repacman, regenerate an installed package

Gave this a quick spin and it looks great.  I will sit down and give this good look over in the next couple of days and then I would be very keen for this to replace repacman in the contrib section of pacman (and thus added to the pacman-contrib package).

Again, this is some really nice work!

Offline

#10 2008-05-28 15:48:47

shining
Pacman Developer
Registered: 2006-05-10
Posts: 2,043

Re: repacman, regenerate an installed package

I noticed a few other problems, but now I am stuck on a really weird issue.
First, the bug fixes :
* the hidden metainfo files are all uppercase : PKGINFO / INSTALL / CHANGELOG
* fix backup field by stripping the md5sum
now the addition I made :
* add REPLACES : this will be stored in local database with pacman 3.2
* add FORCE field : same as above
Now, this is the problematic one, as FORCE is the only empty field.
I first thought I needed all empty lines to handle this, so I changed the loop to a : cat foo | while read i
Anyway, I don' t think this is a bad change.
Now the crazy issue :
pkgname and pkgver are correctly set when desc is parsed, and are added to .PKGINFO
but at the end of the script, when the package is generated, they are always empty!

This issue has driven me crazy for one hour, so I give up now... Help! smile

patch :

--- repacman.orig    2008-05-28 14:53:30.000000000 +0200
+++ repacman    2008-05-28 17:39:42.000000000 +0200
@@ -25,7 +25,6 @@
 #
 # Setting env variables
 #
-IFS=$'\n'
 eval $(awk '/DBPath|CacheDir/ {print $1$2$3}' /etc/pacman.conf)
 pac_db="${DBPath:-/var/lib/pacman/}/local"
 pac_cache="${CacheDir:-/var/cache/pacman/pkg/}"
@@ -110,10 +109,10 @@
 
 echo Adding install commands and changelogs, if any...
 if [ -f "$pkg_dir/install" ] ; then
-    cp "$pkg_dir/install" "$work_dir/pkg/.install"
+    cp "$pkg_dir/install" "$work_dir/pkg/.INSTALL"
 fi
 if  [ -f $pkg_dir/changelog ] ; then
-    cp "$pkg_dir/changelog" "$work_dir/pkg/.changelog"
+    cp "$pkg_dir/changelog" "$work_dir/pkg/.CHANGELOG"
 fi
 
 #
@@ -125,7 +124,16 @@
 echo "# $(LC_ALL=C date)"            >> .PKGINFO
 echo "#"                    >> .PKGINFO
 
-for i in $(cat "$pkg_dir"/{desc,files,depends}) ; do
+cat "$pkg_dir"/{desc,files,depends} |
+while read i; do
+    if [[ -z "$i" ]]; then
+        continue;
+    fi
+
+    if [[ "$i" == "%FORCE%" ]] ; then
+        echo "force = true"    >> .PKGINFO
+        continue
+    fi
 
     if [[ "$i" =~ %[A-Z]*% ]] ; then
         current=$i
@@ -131,9 +139,11 @@
         current=$i
         continue
     fi
+
         case "$current" in
 
-        %NAME%)                    #desc
+        # desc
+        %NAME%)
            echo "pkgname = $i"    >> .PKGINFO
            pkgname=$i
         ;;
@@ -165,12 +175,18 @@
         %GROUPS%)
            echo "group = $i"    >> .PKGINFO
         ;;
+        %REPLACES%)
+        echo "replaces = $i"    >> .PKGINFO
+        ;;
 
-        %BACKUP%)                #files
-           echo "backup = $i"   >> .PKGINFO
+        # files
+        %BACKUP%)
+        # strip the md5sum after the tab
+        echo "backup = ${i%%    *}"   >> .PKGINFO
         ;;
 
-                %DEPENDS%)                #depends
+        # depends
+        %DEPENDS%)
                    echo "depend = $i"   >> .PKGINFO
                 ;;
                 %OPTDEPENDS%)

full script :

#!/bin/bash
#
#   bacman: recreate a package from a running system
#   This script rebuilds an already installed package using metadata
#   stored into the pacman database and system files
#
#   (c) 2008 - locci <carlocci_at_gmail_dot_com>
#   
#   This program is free software; you can redistribute it and/or modify
#   it under the terms of the GNU General Public License as published by
#   the Free Software Foundation; either version 2 of the License, or
#   (at your option) any later version.
# 
#   This program is distributed in the hope that it will be useful,
#   but WITHOUT ANY WARRANTY; without even the implied warranty of
#   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
#   GNU General Public License for more details.
#  
#   You should have received a copy of the GNU General Public License
#   along with this program.  If not, see <http://www.gnu.org/licenses/>.
#

_bacman_ver="0.1.½"

#
# Setting env variables
#
eval $(awk '/DBPath|CacheDir/ {print $1$2$3}' /etc/pacman.conf)
pac_db="${DBPath:-/var/lib/pacman/}/local"
pac_cache="${CacheDir:-/var/cache/pacman/pkg/}"
work_dir="$PWD"
pkg_name="$1"
pkg_dir="$(echo $pac_db/$pkg_name-[0-9]*)"
eval $(awk '/PKGDEST|CARCH|PACKAGER/ {print $1$2$3}' /etc/makepkg.conf)
pkg_dest="${PKGDEST:-$work_dir}"
pkg_arch=${CARCH:-'noarch'}
pkg_pkger=${PACKAGER:-'Unknown Packager'}

#
# Checks everything is in place
#
if [ $# -ne 1 ] ; then
    echo bacman $_bacman_ver
    echo "Usage: ./bacman <installed package name>"
    exit 1
fi

if [ ! -d "$pac_db" ] ; then
    echo There is no pacman database directory in $pac_db
    echo $pac_db is missing
    exit 1
fi
if [ ! -d "$pac_cache" ] ; then
    echo Warning, there is no pacman pkg cache directory
    echo $pac_cache is missing
    unset pac_cache
fi

if [ ! -d "$pkg_dir" ] ; then
    echo There is no package with this name, $pkg_name
    echo Maybe you are trying to pass the directory name, instead of the package name
    exit 1
fi

if [ -n "$pac_cache" ] && [ -f "$pac_cache"/"${pkg_dir##*/}"*.tar.gz ] ; then
    echo The package already exists in your cache
    read -n 1 -p 'Do you want to fetch it instead ? (y/N) '
    case $REPLY in
        y|Y)
        cp "$pac_cache"/"${pkg_dir##*/}"*.pkg.tar.gz "$pkg_dest"
        exit 0
        ;;
        *) echo ""
        ;;
    esac
fi

#
# Begin 
#
echo Packet: ${pkg_dir##*/}
mkdir "$work_dir"/pkg || exit 1
cd "$work_dir"/pkg || exit 1

#
# File copying
#
echo Copying package files...
for i in $(cat "$pkg_dir"/files) ; do

    if [[ "$i" =~ %[A-Z]*% ]] ; then
        current=$i
        continue
    fi

    case $current in
        %FILES%)
        if [ -d "/$i" ]; then
            mkdir "$i"
        elif [ -f "/$i" ]; then
            install -D "/$i" "$i"
        else
            echo "/$i" is missing: this might result in a broken package
        fi
        ;;
    esac
    pkg_size=$(du -sb | awk '{print $1}')
done

echo Adding install commands and changelogs, if any...
if [ -f "$pkg_dir/install" ] ; then
    cp "$pkg_dir/install" "$work_dir/pkg/.INSTALL"
fi
if  [ -f $pkg_dir/changelog ] ; then
    cp "$pkg_dir/changelog" "$work_dir/pkg/.CHANGELOG"
fi

#
# .PKGINFO stuff
#
echo 
echo Generating .PKGINFO metadata...
echo "# Generated by bacman $_bacman_ver"    > .PKGINFO
echo "# $(LC_ALL=C date)"            >> .PKGINFO
echo "#"                    >> .PKGINFO

cat "$pkg_dir"/{desc,files,depends} |
while read i; do
    if [[ -z "$i" ]]; then
        continue;
    fi

    if [[ "$i" == "%FORCE%" ]] ; then
        echo "force = true"    >> .PKGINFO
        continue
    fi

    if [[ "$i" =~ %[A-Z]*% ]] ; then
        current=$i
        continue
    fi

    case "$current" in

        # desc
        %NAME%)
        echo "pkgname = $i"    >> .PKGINFO
        pkgname=$i
        ;;
        %VERSION%)
        echo "pkgver = $i"    >> .PKGINFO
        pkgver=$i
        ;;
        %DESC%)
        echo "pkgdesc = $i"    >> .PKGINFO
        ;;
        %URL%)
        echo "url = $i"    >> .PKGINFO
        ;;
        %LICENSE%)
        echo "license = $i"    >> .PKGINFO
        ;;
        %ARCH%)
        echo "arch = $i"    >> .PKGINFO
        ;;
        %BUILDDATE%)
        echo "builddate = $(date -u "+%s")"    >> .PKGINFO
        ;;
        %PACKAGER%)
        echo "packager = $pkg_pkger"        >> .PKGINFO
        ;;
        %SIZE%)
        echo "size = $pkg_size"        >> .PKGINFO
        ;;
        %GROUPS%)
        echo "group = $i"    >> .PKGINFO
        ;;
        %REPLACES%)
        echo "replaces = $i"    >> .PKGINFO
        ;;

        # files
        %BACKUP%)
        # strip the md5sum after the tab
        echo "backup = ${i%%    *}"   >> .PKGINFO
        ;;

        # depends
        %DEPENDS%)
        echo "depend = $i"   >> .PKGINFO
        ;;
        %OPTDEPENDS%)
        echo "optdepend = $i" >> .PKGINFO
        ;;
        %CONFLICTS%)
        echo "conflict = $i" >> .PKGINFO
        ;;
        %PROVIDES%)
        echo "provides = $i"  >> .PKGINFO
        ;;

    esac
done

#
# 
#
echo 
echo Generating the package...
bsdtar -czf "$pkg_dest/$pkgname-$pkgver-$pkg_arch.tar.gz" $(ls -A) || echo Cannot write to $pkg_dest: maybe disk is full or you do not have write access 

#echo Removing temporary files...
#cd $work_dir || exit 1
#rm -r pkg/

exit 0

#
# End
#

pacman roulette : pacman -S $(pacman -Slq | LANG=C sort -R | head -n $((RANDOM % 10)))

Offline

#11 2008-05-28 18:27:17

carlocci
Member
From: Padova - Italy
Registered: 2008-02-12
Posts: 368

Re: repacman, regenerate an installed package

shining wrote:

Now the crazy issue :
pkgname and pkgver are correctly set when desc is parsed, and are added to .PKGINFO
but at the end of the script, when the package is generated, they are always empty!

This issue has driven me crazy for one hour, so I give up now... Help! smile

NICE JOB, YOU RUINED IT! big_smile
Thank you for your effort.

I think the problem is in the pipe you use in the main cycle which calls another shell and the variables created in the loop exist only in the subshell.
Actually I was dissatisfied with the IFS too. I think we can safely remove the two variables as we can take $name-$ver from the directory name in the database: I just wanted to be sure to get the right pkg and ver since I didn't know much about pacman db.



edit: here's the patch

--- bacman    2008-05-28 20:23:48.000000000 +0200
+++ bacman.new    2008-05-28 20:30:32.000000000 +0200
@@ -35,6 +35,7 @@
 pkg_dest="${PKGDEST:-$work_dir}"
 pkg_arch=${CARCH:-'noarch'}
 pkg_pkger=${PACKAGER:-'Unknown Packager'}
+pkg_namver="${pkg_dir##*/}"
 
 #
 # Checks everything is in place
@@ -62,12 +63,12 @@
     exit 1
 fi
 
-if [ -n "$pac_cache" ] && [ -f "$pac_cache"/"${pkg_dir##*/}"*.tar.gz ] ; then
+if [ -n "$pac_cache" ] && [ -f "$pac_cache"/"$pkg_namver"*.tar.gz ] ; then
     echo The package already exists in your cache
     read -n 1 -p 'Do you want to fetch it instead ? (y/N) '
     case $REPLY in
         y|Y)
-        cp "$pac_cache"/"${pkg_dir##*/}"*.pkg.tar.gz "$pkg_dest"
+        cp "$pac_cache"/"${pkg_namver##*/}"*.pkg.tar.gz "$pkg_dest"
         exit 0
         ;;
         *) echo ""
@@ -145,11 +146,9 @@
         # desc
         %NAME%)
         echo "pkgname = $i"    >> .PKGINFO
-        pkgname=$i
         ;;
         %VERSION%)
         echo "pkgver = $i"    >> .PKGINFO
-        pkgver=$i
         ;;
         %DESC%)
         echo "pkgdesc = $i"    >> .PKGINFO
@@ -207,7 +206,7 @@
 #
 echo 
 echo Generating the package...
-bsdtar -czf "$pkg_dest/$pkgname-$pkgver-$pkg_arch.tar.gz" $(ls -A) || echo Cannot write to $pkg_dest: maybe disk is full or you do not have write access 
+bsdtar -czf "$pkg_dest/$pkg_namver-$pkg_arch.tar.gz" $(ls -A) || echo Cannot write to $pkg_dest: maybe disk is full or you do not have write access 
 
 #echo Removing temporary files...
 #cd $work_dir || exit 1

I wondered if there was a particular reason you placed %FORCE% outside of the case, I was planning of moving it into the case.
Thank you again for adding the unusual options!

Last edited by carlocci (2008-05-28 18:43:22)

Offline

#12 2008-05-29 07:59:18

shining
Pacman Developer
Registered: 2006-05-10
Posts: 2,043

Re: repacman, regenerate an installed package

carlocci wrote:
shining wrote:

Now the crazy issue :
pkgname and pkgver are correctly set when desc is parsed, and are added to .PKGINFO
but at the end of the script, when the package is generated, they are always empty!

This issue has driven me crazy for one hour, so I give up now... Help! smile

NICE JOB, YOU RUINED IT! big_smile
Thank you for your effort.

I think the problem is in the pipe you use in the main cycle which calls another shell and the variables created in the loop exist only in the subshell.

That is weird, I was totally not aware of that, but it would explain everything.

Actually I was dissatisfied with the IFS too. I think we can safely remove the two variables as we can take $name-$ver from the directory name in the database: I just wanted to be sure to get the right pkg and ver since I didn't know much about pacman db.

edit: here's the patch

Great, thanks! that should do the job.

I wondered if there was a particular reason you placed %FORCE% outside of the case, I was planning of moving it into the case.
Thank you again for adding the unusual options!

Hmm, I thought I needed to handle the empty fields separately, but looking at the final code, I always start by skipping empty lines so it seems like it does not have to be handled separately. Good point.


pacman roulette : pacman -S $(pacman -Slq | LANG=C sort -R | head -n $((RANDOM % 10)))

Offline

#13 2008-05-29 08:49:14

shining
Pacman Developer
Registered: 2006-05-10
Posts: 2,043

Re: repacman, regenerate an installed package

Ok it looks perfectly functional now. I am just sending the whole file again so that it is easier for others to pick up last version.
Now the goal is to make it appear there :
http://projects.archlinux.org/?p=pacman … ;f=contrib
so that we can remove the broken re-pacman script. At least this was my only motivation for fixing bacman, I was never really interested by the feature provided smile

Anyway I will let carlocci and Allan work together on that now smile

#!/bin/bash
#
#   bacman: recreate a package from a running system
#   This script rebuilds an already installed package using metadata
#   stored into the pacman database and system files
#
#   (c) 2008 - locci <carlocci_at_gmail_dot_com>
#   
#   This program is free software; you can redistribute it and/or modify
#   it under the terms of the GNU General Public License as published by
#   the Free Software Foundation; either version 2 of the License, or
#   (at your option) any later version.
# 
#   This program is distributed in the hope that it will be useful,
#   but WITHOUT ANY WARRANTY; without even the implied warranty of
#   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
#   GNU General Public License for more details.
#  
#   You should have received a copy of the GNU General Public License
#   along with this program.  If not, see <http://www.gnu.org/licenses/>.
#

_bacman_ver="0.1.1"

#
# Setting env variables
#
eval $(awk '/DBPath|CacheDir/ {print $1$2$3}' /etc/pacman.conf)
pac_db="${DBPath:-/var/lib/pacman/}/local"
pac_cache="${CacheDir:-/var/cache/pacman/pkg/}"
work_dir="$PWD"
pkg_name="$1"
pkg_dir="$(echo $pac_db/$pkg_name-[0-9]*)"
eval $(awk '/PKGDEST|CARCH|PACKAGER/ {print $1$2$3}' /etc/makepkg.conf)
pkg_dest="${PKGDEST:-$work_dir}"
pkg_arch=${CARCH:-'noarch'}
pkg_pkger=${PACKAGER:-'Unknown Packager'}
pkg_namver="${pkg_dir##*/}"

#
# Checks everything is in place
#
if [ $# -ne 1 ] ; then
    echo bacman $_bacman_ver
    echo "Usage: ./bacman <installed package name>"
    exit 1
fi

if [ ! -d "$pac_db" ] ; then
    echo There is no pacman database directory in $pac_db
    echo $pac_db is missing
    exit 1
fi
if [ ! -d "$pac_cache" ] ; then
    echo Warning, there is no pacman pkg cache directory
    echo $pac_cache is missing
    unset pac_cache
fi

if [ ! -d "$pkg_dir" ] ; then
    echo There is no package with this name, $pkg_name
    echo Maybe you are trying to pass the directory name, instead of the package name
    exit 1
fi

if [ -n "$pac_cache" ] && [ -f "$pac_cache"/"$pkg_namver"*.tar.gz ] ; then
    echo The package already exists in your cache
    read -n 1 -p 'Do you want to fetch it instead ? (y/N) '
    case $REPLY in
        y|Y)
        cp "$pac_cache"/"${pkg_namver##*/}"*.pkg.tar.gz "$pkg_dest"
        exit 0
        ;;
        *) echo ""
        ;;
    esac
fi

#
# Begin 
#
echo Package: ${pkg_dir##*/}
mkdir "$work_dir"/pkg || exit 1
cd "$work_dir"/pkg || exit 1

#
# File copying
#
echo Copying package files...
for i in $(cat "$pkg_dir"/files) ; do

    if [[ "$i" =~ %[A-Z]*% ]] ; then
        current=$i
        continue
    fi

    case $current in
        %FILES%)
        if [ -d "/$i" ]; then
            mkdir "$i"
        elif [ -f "/$i" ]; then
            install -D "/$i" "$i"
        else
            echo "/$i" is missing: this might result in a broken package
        fi
        ;;
    esac
    pkg_size=$(du -sb | awk '{print $1}')
done

echo Adding install commands and changelogs, if any...
if [ -f "$pkg_dir/install" ] ; then
    cp "$pkg_dir/install" "$work_dir/pkg/.INSTALL"
fi
if  [ -f $pkg_dir/changelog ] ; then
    cp "$pkg_dir/changelog" "$work_dir/pkg/.CHANGELOG"
fi

#
# .PKGINFO stuff
#
echo 
echo Generating .PKGINFO metadata...
echo "# Generated by bacman $_bacman_ver"    > .PKGINFO
echo "# $(LC_ALL=C date)"            >> .PKGINFO
echo "#"                    >> .PKGINFO

cat "$pkg_dir"/{desc,files,depends} |
while read i; do
    if [[ -z "$i" ]]; then
        continue;
    fi

    if [[ "$i" == "%FORCE%" ]] ; then
        echo "force = true"    >> .PKGINFO
        continue
    fi

    if [[ "$i" =~ %[A-Z]*% ]] ; then
        current=$i
        continue
    fi

    case "$current" in

        # desc
        %NAME%)
        echo "pkgname = $i"    >> .PKGINFO
        ;;
        %VERSION%)
        echo "pkgver = $i"    >> .PKGINFO
        ;;
        %DESC%)
        echo "pkgdesc = $i"    >> .PKGINFO
        ;;
        %URL%)
        echo "url = $i"    >> .PKGINFO
        ;;
        %LICENSE%)
        echo "license = $i"    >> .PKGINFO
        ;;
        %ARCH%)
        echo "arch = $i"    >> .PKGINFO
        ;;
        %BUILDDATE%)
        echo "builddate = $(date -u "+%s")"    >> .PKGINFO
        ;;
        %PACKAGER%)
        echo "packager = $pkg_pkger"        >> .PKGINFO
        ;;
        %SIZE%)
        echo "size = $pkg_size"        >> .PKGINFO
        ;;
        %GROUPS%)
        echo "group = $i"    >> .PKGINFO
        ;;
        %REPLACES%)
        echo "replaces = $i"    >> .PKGINFO
        ;;

        # files
        %BACKUP%)
        # strip the md5sum after the tab
        echo "backup = ${i%%    *}"   >> .PKGINFO
        ;;

        # depends
        %DEPENDS%)
        echo "depend = $i"   >> .PKGINFO
        ;;
        %OPTDEPENDS%)
        echo "optdepend = $i" >> .PKGINFO
        ;;
        %CONFLICTS%)
        echo "conflict = $i" >> .PKGINFO
        ;;
        %PROVIDES%)
        echo "provides = $i"  >> .PKGINFO
        ;;

    esac
done

#
# Generate the package 
#
echo 
echo Generating the package...
bsdtar -czf "$pkg_dest/$pkg_namver-$pkg_arch.tar.gz" $(ls -A) || echo Cannot write to $pkg_dest: maybe disk is full or you do not have write access 

exit 0

# vim: set ts=2 sw=2 noet:

pacman roulette : pacman -S $(pacman -Slq | LANG=C sort -R | head -n $((RANDOM % 10)))

Offline

#14 2008-05-29 10:51:32

Allan
Pacman
From: Brisbane, AU
Registered: 2007-06-09
Posts: 11,365
Website

Re: repacman, regenerate an installed package

shining wrote:

Anyway I will let carlocci and Allan work together on that now smile

So I'm going to have to do something!  Here I thought I could get away with just saying I was going to do stuff...

Anyway, I see nothing more to add to the core functionality.  Just a couple of tidy ups I think are necessary before it can make it into the official pacman contrib directory.  I will post my changes in a few hours.

Offline

#15 2008-05-29 11:36:56

shining
Pacman Developer
Registered: 2006-05-10
Posts: 2,043

Re: repacman, regenerate an installed package

Allan wrote:
shining wrote:

Anyway I will let carlocci and Allan work together on that now smile

So I'm going to have to do something!  Here I thought I could get away with just saying I was going to do stuff...

Ahah, nice try indeed smile

Anyway, I see nothing more to add to the core functionality.  Just a couple of tidy ups I think are necessary before it can make it into the official pacman contrib directory.  I will post my changes in a few hours.

Thanks, that is perfect.


pacman roulette : pacman -S $(pacman -Slq | LANG=C sort -R | head -n $((RANDOM % 10)))

Offline

#16 2008-05-29 12:14:30

Allan
Pacman
From: Brisbane, AU
Registered: 2007-06-09
Posts: 11,365
Website

Re: repacman, regenerate an installed package

Ok, here is some changes I would like to make...

1)Catch some commonly used flags at the top of the script.  Also, change usage of "bacman" and the version variable throughout. I.e.,

readonly progname="bacman"
readonly progver="0.1.1"

if [ "$1" = "--help" -o "$1" = "-h" ] ; then
    echo "Usage: $progname <installed package name>"
    echo "Example: $progname kernel26"
    exit 0
fi

if [ "$1" = "--version" -o "$1" = "-v" ]; then
    echo "$progname version $version"
    echo "Copyright (C) 2008 locci"
    exit 0
fi

This will make the script more in line with the others in the pacman contrib directory

2) You do not need to remove whitespace when reading makepkg.conf.  It is read by bash so reading the variables should be fine without this.  It screws up my PACKAGER entry otherwise.

3) We should not do arch detection from makepkg.conf.  It should come form the %ARCH% entry in the local database.

4) We can tidy up the error messages so they are a bit clearer/cleaner. 

5) I am not so sure about using the $PWD/pkg directory.  If one already exists there will be trouble.  Given it is left in the directory after usage, that is likely to happen (e.g. bacman kernel26, bacman ipw3945).  I think it should create a temporary directory for creating the package and remove it at the end.

6) Fix up the error message during the actual package generation including exiting with a error condition.

I will post a version with the changes made just as soon as Family Guy finishes on tv...

Offline

#17 2008-05-29 12:26:09

shining
Pacman Developer
Registered: 2006-05-10
Posts: 2,043

Re: repacman, regenerate an installed package

Allan wrote:

2) You do not need to remove whitespace when reading makepkg.conf.  It is read by bash so reading the variables should be fine without this.  It screws up my PACKAGER entry otherwise.

How are whitespace removed?

5) I am not so sure about using the $PWD/pkg directory.  If one already exists there will be trouble.  Given it is left in the directory after usage, that is likely to happen (e.g. bacman kernel26, bacman ipw3945).  I think it should create a temporary directory for creating the package and remove it at the end.

Oops, have a look at the older versions, there was a block commented out for removing pkg/


All the other points look right.


pacman roulette : pacman -S $(pacman -Slq | LANG=C sort -R | head -n $((RANDOM % 10)))

Offline

#18 2008-05-29 12:41:56

Allan
Pacman
From: Brisbane, AU
Registered: 2007-06-09
Posts: 11,365
Website

Re: repacman, regenerate an installed package

shining wrote:
Allan wrote:

2) You do not need to remove whitespace when reading makepkg.conf.  It is read by bash so reading the variables should be fine without this.  It screws up my PACKAGER entry otherwise.

How are whitespace removed?

Currently is it like:

eval $(awk '/PKGDEST|CARCH|PACKAGER/ {print $1$2$3}' /etc/makepkg.conf)

This makes my PACKAGER get extracted like: PACKAGER="AllanMcRae<mcrae_allan@hotmail.com>".  Getting rid of the the "{print $1$2$3}" fixes this


shining wrote:
Allan wrote:

5) I am not so sure about using the $PWD/pkg directory.  If one already exists there will be trouble.  Given it is left in the directory after usage, that is likely to happen (e.g. bacman kernel26, bacman ipw3945).  I think it should create a temporary directory for creating the package and remove it at the end.

Oops, have a look at the older versions, there was a block commented out for removing pkg/

From you comment, I'm not sure whether you are for or against that change...  I think the script at least needs to create a unique working directory to avoid problems.  Also, there is very little loss in removing pkg/ because you can just extract your newly created package.

Offline

#19 2008-05-29 12:57:00

shining
Pacman Developer
Registered: 2006-05-10
Posts: 2,043

Re: repacman, regenerate an installed package

Allan wrote:

This makes my PACKAGER get extracted like: PACKAGER="AllanMcRae<mcrae_allan@hotmail.com>".  Getting rid of the the "{print $1$2$3}" fixes this

Ah indeed.

From you comment, I'm not sure whether you are for or against that change...  I think the script at least needs to create a unique working directory to avoid problems.  Also, there is very little loss in removing pkg/ because you can just extract your newly created package.

That is because I don't really have an opinion.
I used the pkg dir during debugging but I could indeed have used the package instead, so no problem.
About the location, I was not sure either how to handle this.

carlocci wrote:

I thought about using /tmp and mktemp for the temporary pkg directory name, but I tried to mimic the behaviour of makepkg: I removed the line deleting pkg/

So after thinking a bit about this, I am not sure we need to follow makepkg behavior here. makepkg needs a package per directory, with a pkgbuild inside, so it makes sense to put the pkg/ and src/ dir in there.
But bacman can be run anywhere, so it is probably a good idea to use /tmp and mktemp and remove the temp dir afterward.


pacman roulette : pacman -S $(pacman -Slq | LANG=C sort -R | head -n $((RANDOM % 10)))

Offline

#20 2008-05-29 14:00:40

Allan
Pacman
From: Brisbane, AU
Registered: 2007-06-09
Posts: 11,365
Website

Re: repacman, regenerate an installed package

OK, here is the updated version.

#!/bin/bash
#
#   bacman: recreate a package from a running system
#   This script rebuilds an already installed package using metadata
#   stored into the pacman database and system files
#
#   (c) 2008 - locci <carlocci_at_gmail_dot_com>
#   
#   This program is free software; you can redistribute it and/or modify
#   it under the terms of the GNU General Public License as published by
#   the Free Software Foundation; either version 2 of the License, or
#   (at your option) any later version.
# 
#   This program is distributed in the hope that it will be useful,
#   but WITHOUT ANY WARRANTY; without even the implied warranty of
#   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
#   GNU General Public License for more details.
#  
#   You should have received a copy of the GNU General Public License
#   along with this program.  If not, see <http://www.gnu.org/licenses/>.
#

readonly progname="bacman"
readonly progver="0.1.2"

if [ "$1" = "--help" -o "$1" = "-h" ] ; then
    echo "Usage:   $progname <installed package name>"
    echo "Example: $progname kernel26"
    exit 0
fi

if [ "$1" = "--version" -o "$1" = "-v" ]; then
    echo "$progname version $version"
    echo "Copyright (C) 2008 locci"
    exit 0
fi


#
# Setting environmental variables
#
eval $(awk '/DBPath|CacheDir/ {print $1$2$3}' /etc/pacman.conf)
pac_db="${DBPath:-/var/lib/pacman/}/local"
pac_cache="${CacheDir:-/var/cache/pacman/pkg/}"
pkg_name="$1"
pkg_dir="$(echo $pac_db/$pkg_name-[0-9]*)"
eval $(awk '/PKGDEST|PACKAGER/' /etc/makepkg.conf)
pkg_dest="${PKGDEST:-$PWD}"
pkg_pkger=${PACKAGER:-'Unknown Packager'}
pkg_arch="unknown"
pkg_namver="${pkg_dir##*/}"

#
# Checks everything is in place
#
if [ $# -ne 1 ] ; then
    echo "Usage:   $progname <installed package name>"
    echo "Example: $progname kernel26"
    exit 1
fi

if [ ! -d "$pac_db" ] ; then
    echo Error: pacman database directory ${pac_db} not found
    exit 1
fi

if [ ! -d "$pac_cache" ] ; then
    echo Warning: pacman pkg cache directory ${pac_cache} not found
    unset pac_cache
fi

if [ ! -d "$pkg_dir" ] ; then
    echo Error: package ${pkg_name} not found in pacman database
    exit 1
fi

if [ -n "$pac_cache" ] && [ -f "$pac_cache"/"$pkg_namver"*.tar.gz ] ; then
    echo The package ${pkg_name} exists in your pacman cache
    read -n 1 -p 'Do you want to fetch it instead? (y/N) '
    case $REPLY in
        y|Y)
        cp "$pac_cache"/"${pkg_namver##*/}"*.pkg.tar.gz "$pkg_dest"
        exit 0
        ;;
        *) echo ""
        ;;
    esac
fi

#
# Begin 
#
echo Package: ${pkg_dir##*/}
work_dir=$(mktemp -d -p /tmp)
cd $work_dir || exit 1

#
# File copying
#
echo Copying package files...
for i in $(cat "$pkg_dir"/files) ; do

    if [[ "$i" =~ %[A-Z]*% ]] ; then
        current=$i
        continue
    fi

    case $current in
        %FILES%)
        if [ -d "/$i" ]; then
            mkdir "$i"
        elif [ -f "/$i" ]; then
            install -D "/$i" "$i"
        else
            echo "/$i" is missing: this might result in a broken package
        fi
        ;;
    esac
    pkg_size=$(du -sb | awk '{print $1}')
done

echo Adding install commands and changelogs, if any...
if [ -f "$pkg_dir/install" ] ; then
    cp "$pkg_dir/install" "$work_dir/.INSTALL"
fi
if  [ -f $pkg_dir/changelog ] ; then
    cp "$pkg_dir/changelog" "$work_dir/.CHANGELOG"
fi

#
# .PKGINFO stuff
#
echo 
echo Generating .PKGINFO metadata...
echo "# Generated by $progname $progver"    > .PKGINFO
echo "# $(LC_ALL=C date)"            >> .PKGINFO
echo "#"                    >> .PKGINFO

cat "$pkg_dir"/{desc,files,depends} |
while read i; do
    if [[ -z "$i" ]]; then
        continue;
    fi

    if [[ "$i" == "%FORCE%" ]] ; then
        echo "force = true"    >> .PKGINFO
        continue
    fi

    if [[ "$i" =~ %[A-Z]*% ]] ; then
        current=$i
        continue
    fi

    case "$current" in

        # desc
        %NAME%)
        echo "pkgname = $i"    >> .PKGINFO
        ;;
        %VERSION%)
        echo "pkgver = $i"    >> .PKGINFO
        ;;
        %DESC%)
        echo "pkgdesc = $i"    >> .PKGINFO
        ;;
        %URL%)
        echo "url = $i"    >> .PKGINFO
        ;;
        %LICENSE%)
        echo "license = $i"    >> .PKGINFO
        ;;
        %ARCH%)
        echo "arch = $i"    >> .PKGINFO
        pkg_arch=$i
        ;;
        %BUILDDATE%)
        echo "builddate = $(date -u "+%s")"    >> .PKGINFO
        ;;
        %PACKAGER%)
        echo "packager = $pkg_pkger"        >> .PKGINFO
        ;;
        %SIZE%)
        echo "size = $pkg_size"        >> .PKGINFO
        ;;
        %GROUPS%)
        echo "group = $i"    >> .PKGINFO
        ;;
        %REPLACES%)
        echo "replaces = $i"    >> .PKGINFO
        ;;

        # files
        %BACKUP%)
        # strip the md5sum after the tab
        echo "backup = ${i%%    *}"   >> .PKGINFO
        ;;

        # depends
        %DEPENDS%)
        echo "depend = $i"   >> .PKGINFO
        ;;
        %OPTDEPENDS%)
        echo "optdepend = $i" >> .PKGINFO
        ;;
        %CONFLICTS%)
        echo "conflict = $i" >> .PKGINFO
        ;;
        %PROVIDES%)
        echo "provides = $i"  >> .PKGINFO
        ;;

    esac
done

#
# Generate the package 
#
echo 
echo Generating the package...

ret=0
bsdtar -czf "$pkg_dest/$pkg_namver-$pkg_arch.tar.gz" $(ls -A) || ret=$?
if [ $ret -ne 0 ]; then
    echo Error: Unable to write package to $pkg_dest
    echo Maybe the disk is full or you do not have write access
    exit 1
fi

#rm -rf $work_dir

exit 0

# vim: set ts=2 sw=2 noet:

There are still problems.....

It seem the reading of info from makepkg.conf does not work.  In the .PKGINFO file I always get "Unknown Packager".  I haven't actually checked any of the other fields that are read in from files yet as I have these set as default values.

There are problems running this as a user when packages do not have read permissions for a user.  E.g. sudo.

Also, the setting of the pkg_arch within the case statement does not work.



Edit: Reading in the details from the files works when done one field at a time.
Edit2: I think the pkg_arch problem is reminiscent of a problem above....

Last edited by Allan (2008-05-29 14:23:46)

Offline

#21 2008-05-29 14:24:50

shining
Pacman Developer
Registered: 2006-05-10
Posts: 2,043

Re: repacman, regenerate an installed package

Allan wrote:

There are still problems.....

It seem the reading of info from makepkg.conf does not work.  In the .PKGINFO file I always get "Unknown Packager".  I haven't actually checked any of the other fields that are read in from files yet as I have these set as default values.

Edit: Reading in the details from the files works when done one field at a time.

I just figured it out : when you have some commented out fields, it will hide the others.
Either you take them one by one, or you don't grep the commented one (starting with #), or my favorite:
why don' t we simply source the whole makepkg.conf ?

There are problems running this as a user when packages do not have read permissions for a user.  E.g. sudo.

I don' t see what we can do about this. install errors out with permission denied, which is explicit enough.
The user can manually run it again as root.

Also, the setting of the pkg_arch within the case statement does not work.

Same problem as with pkgname and pkgver, see above posts from me and carlocci.
That issue is problematic tongue


pacman roulette : pacman -S $(pacman -Slq | LANG=C sort -R | head -n $((RANDOM % 10)))

Offline

#22 2008-05-29 14:39:59

Allan
Pacman
From: Brisbane, AU
Registered: 2007-06-09
Posts: 11,365
Website

Re: repacman, regenerate an installed package

shining wrote:

or my favorite: why don' t we simply source the whole makepkg.conf ?

We could do.  I have implemented the read one in at a time strategy below.  It is getting late at night here so if someone wants to look into this... 
Anyway, I don't see any further changes that need made.

Edit: some of that was stupid...


#!/bin/bash
#
#   bacman: recreate a package from a running system
#   This script rebuilds an already installed package using metadata
#   stored into the pacman database and system files
#
#   (c) 2008 - locci <carlocci_at_gmail_dot_com>
#   
#   This program is free software; you can redistribute it and/or modify
#   it under the terms of the GNU General Public License as published by
#   the Free Software Foundation; either version 2 of the License, or
#   (at your option) any later version.
# 
#   This program is distributed in the hope that it will be useful,
#   but WITHOUT ANY WARRANTY; without even the implied warranty of
#   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
#   GNU General Public License for more details.
#  
#   You should have received a copy of the GNU General Public License
#   along with this program.  If not, see <http://www.gnu.org/licenses/>.
#

readonly progname="bacman"
readonly progver="0.1.2"

if [ "$1" = "--help" -o "$1" = "-h" ] ; then
    echo "Usage:   $progname <installed package name>"
    echo "Example: $progname kernel26"
    exit 0
fi

if [ "$1" = "--version" -o "$1" = "-v" ]; then
    echo "$progname version $version"
    echo "Copyright (C) 2008 locci"
    exit 0
fi


#
# Setting environmental variables
#
eval $(awk '/DBPath/ {print $1$2$3}' /etc/pacman.conf)
pac_db="${DBPath:-/var/lib/pacman/}/local"
eval $(awk '/CacheDir/ {print $1$2$3}' /etc/pacman.conf)
pac_cache="${CacheDir:-/var/cache/pacman/pkg/}"
pkg_name="$1"
pkg_dir="$(echo $pac_db/$pkg_name-[0-9]*)"
eval $(awk '/CARCH/' /etc/makepkg.conf)
pkg_arch=${CARCH:-'unknown'}
eval $(awk '/PKGDEST/' /etc/makepkg.conf)
pkg_dest="${PKGDEST:-$PWD}"
eval $(awk '/PACKAGER/' /etc/makepkg.conf)
pkg_pkger=${PACKAGER:-'Unknown Packager'}
pkg_namver="${pkg_dir##*/}"

#
# Checks everything is in place
#
if [ $# -ne 1 ] ; then
    echo "Usage:   $progname <installed package name>"
    echo "Example: $progname kernel26"
    exit 1
fi

if [ ! -d "$pac_db" ] ; then
    echo Error: pacman database directory ${pac_db} not found
    exit 1
fi

if [ ! -d "$pac_cache" ] ; then
    echo Warning: pacman pkg cache directory ${pac_cache} not found
    unset pac_cache
fi

if [ ! -d "$pkg_dir" ] ; then
    echo Error: package ${pkg_name} not found in pacman database
    exit 1
fi

if [ -n "$pac_cache" ] && [ -f "$pac_cache"/"$pkg_namver"*.tar.gz ] ; then
    echo The package ${pkg_name} exists in your pacman cache
    read -n 1 -p 'Do you want to fetch it instead? (y/N) '
    case $REPLY in
        y|Y)
        cp "$pac_cache"/"${pkg_namver##*/}"*.pkg.tar.gz "$pkg_dest"
        exit 0
        ;;
        *) echo ""
        ;;
    esac
fi

#
# Begin 
#
echo Package: ${pkg_dir##*/}
work_dir=$(mktemp -d -p /tmp)
cd $work_dir || exit 1

#
# File copying
#
echo Copying package files...
for i in $(cat "$pkg_dir"/files) ; do

    if [[ "$i" =~ %[A-Z]*% ]] ; then
        current=$i
        continue
    fi

    case $current in
        %FILES%)
        if [ -d "/$i" ]; then
            mkdir "$i"
        elif [ -f "/$i" ]; then
            install -D "/$i" "$i"
        else
            echo "/$i" is missing: this might result in a broken package
        fi
        ;;
    esac
    pkg_size=$(du -sb | awk '{print $1}')
done

echo Adding install commands and changelogs, if any...
if [ -f "$pkg_dir/install" ] ; then
    cp "$pkg_dir/install" "$work_dir/.INSTALL"
fi
if  [ -f $pkg_dir/changelog ] ; then
    cp "$pkg_dir/changelog" "$work_dir/.CHANGELOG"
fi

#
# .PKGINFO stuff
#
echo 
echo Generating .PKGINFO metadata...
echo "# Generated by $progname $progver"    > .PKGINFO
echo "# $(LC_ALL=C date)"            >> .PKGINFO
echo "#"                    >> .PKGINFO

cat "$pkg_dir"/{desc,files,depends} |
while read i; do
    if [[ -z "$i" ]]; then
        continue;
    fi

    if [[ "$i" == "%FORCE%" ]] ; then
        echo "force = true"    >> .PKGINFO
        continue
    fi

    if [[ "$i" =~ %[A-Z]*% ]] ; then
        current=$i
        continue
    fi

    case "$current" in

        # desc
        %NAME%)
        echo "pkgname = $i"    >> .PKGINFO
        ;;
        %VERSION%)
        echo "pkgver = $i"    >> .PKGINFO
        ;;
        %DESC%)
        echo "pkgdesc = $i"    >> .PKGINFO
        ;;
        %URL%)
        echo "url = $i"    >> .PKGINFO
        ;;
        %LICENSE%)
        echo "license = $i"    >> .PKGINFO
        ;;
        %ARCH%)
        echo "arch = $i"    >> .PKGINFO
        ;;
        %BUILDDATE%)
        echo "builddate = $(date -u "+%s")"    >> .PKGINFO
        ;;
        %PACKAGER%)
        echo "packager = $pkg_pkger"        >> .PKGINFO
        ;;
        %SIZE%)
        echo "size = $pkg_size"        >> .PKGINFO
        ;;
        %GROUPS%)
        echo "group = $i"    >> .PKGINFO
        ;;
        %REPLACES%)
        echo "replaces = $i"    >> .PKGINFO
        ;;

        # files
        %BACKUP%)
        # strip the md5sum after the tab
        echo "backup = ${i%%    *}"   >> .PKGINFO
        ;;

        # depends
        %DEPENDS%)
        echo "depend = $i"   >> .PKGINFO
        ;;
        %OPTDEPENDS%)
        echo "optdepend = $i" >> .PKGINFO
        ;;
        %CONFLICTS%)
        echo "conflict = $i" >> .PKGINFO
        ;;
        %PROVIDES%)
        echo "provides = $i"  >> .PKGINFO
        ;;

    esac
done

#
# Generate the package 
#
echo 
echo Generating the package...

ret=0
bsdtar -czf "$pkg_dest/$pkg_namver-$pkg_arch.tar.gz" $(ls -A) || ret=$?
if [ $ret -ne 0 ]; then
    echo Error: Unable to write package to $pkg_dest
    echo Maybe the disk is full or you do not have write access
    exit 1
fi

#rm -rf $work_dir

exit 0

# vim: set ts=2 sw=2 noet:

Last edited by Allan (2008-05-29 14:45:09)

Offline

#23 2008-05-29 14:50:43

shining
Pacman Developer
Registered: 2006-05-10
Posts: 2,043

Re: repacman, regenerate an installed package

Allan wrote:
shining wrote:

or my favorite: why don' t we simply source the whole makepkg.conf ?

We could do.  I have implemented the read one in at a time strategy below.  It is getting late at night here so if someone wants to look into this... 
Anyway, I don't see any further changes that need made.

Forget about sourcing, it is fine that way.
So the script is ready now, great smile


pacman roulette : pacman -S $(pacman -Slq | LANG=C sort -R | head -n $((RANDOM % 10)))

Offline

#24 2008-05-29 15:06:22

Allan
Pacman
From: Brisbane, AU
Registered: 2007-06-09
Posts: 11,365
Website

Re: repacman, regenerate an installed package

I will make the patch which add this into the contrib directory and the description to the README file in the morning.  I guess we remove repacman at the same time.

@carlocci: This is unless you wants to submit it yourself.  Either way, you bask in the glory that is contributing to the pacman code base!

Offline

#25 2008-05-29 15:50:00

carlocci
Member
From: Padova - Italy
Registered: 2008-02-12
Posts: 368

Re: repacman, regenerate an installed package

I addressed some bugs:
for example the file processing still had for i in $pac_db/blabla/files which doesn't work if the path to the files has spaces inside (without setting IFS=\n)

I added a usage function in order to remove a redundancy, I fixed some variables names, messed a bit with the main loop.
There are still some issues, for example pacman's database path and PKGDEST path can't have spaces inside.
I'm learning sed to fix this

#!/bin/bash
#
#   bacman: recreate a package from a running system
#   This script rebuilds an already installed package using metadata
#   stored into the pacman database and system files
#
#   (c) 2008 - locci <carlocci_at_gmail_dot_com>
#   
#   This program is free software; you can redistribute it and/or modify
#   it under the terms of the GNU General Public License as published by
#   the Free Software Foundation; either version 2 of the License, or
#   (at your option) any later version.
# 
#   This program is distributed in the hope that it will be useful,
#   but WITHOUT ANY WARRANTY; without even the implied warranty of
#   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
#   GNU General Public License for more details.
#  
#   You should have received a copy of the GNU General Public License
#   along with this program.  If not, see <http://www.gnu.org/licenses/>.
#

readonly progname="bacman"
readonly progver="0.1.2"

#
# User Friendliness
#
function usage(){
    echo "This program recreates a package using pacman's db and system files"
    echo "Usage:   $progname <installed package name>"
    echo "Example: $progname kernel26"
}

if [ $# -ne 1 ] ; then
    usage
    exit 1
fi

if [ "$1" = "--help" -o "$1" = "-h" ] ; then
    usage
    exit 0
fi

if [ "$1" = "--version" -o "$1" = "-v" ]; then
    echo "$progname version $version"
    echo "Copyright (C) 2008 locci"
    exit 0
fi

#
# Setting environmental variables
#
eval $(sed -nre 's:[[:space:]]*(DBPath|CacheDir)[[:space:]]*=[[:space:]]*([[:print:]]*):\1="\2":p' /etc/pacman.conf)
pac_db="${DBPath:-/var/lib/pacman/}/local"
pac_cache="${CacheDir:-/var/cache/pacman/pkg/}"
pkg_name="$1"
pkg_dir="$(echo $pac_db/$pkg_name-[0-9]*)"
eval $(sed -nre 's:[[:space:]]*(CARCH|PKGDEST|PACKAGER)[[:space:]]*=[[:space:]]*([[:print:]]*):\1=\2:p' /etc/makepkg.conf)
pkg_arch=${CARCH:-'unknown'}
pkg_dest="${PKGDEST:-$PWD}"
pkg_pkger=${PACKAGER:-'Unknown Packager'}
pkg_namver="${pkg_dir##*/}"

#
# Checks everything is in place
#
if [ ! -d "$pac_db" ] ; then
    echo Error: pacman database directory ${pac_db} not found
    exit 1
fi

if [ ! -d "$pac_cache" ] ; then
    echo Warning: pacman pkg cache directory ${pac_cache} not found
    unset pac_cache
fi

if [ ! -d "$pkg_dir" ] ; then
    echo Error: package ${pkg_name} not found in pacman database
    exit 1
fi

if [ -n "$pac_cache" ] && [ -f "$pac_cache"/"$pkg_namver"*.tar.gz ] ; then
    echo The package ${pkg_name} exists in your pacman cache
    read -n 1 -p 'Do you want to fetch it instead? (y/N) '
    case $REPLY in
        y|Y)
        cp "$pac_cache"/"$pkg_namver"*.pkg.tar.gz "$pkg_dest"
        exit 0
        ;;
        *) echo ""
        ;;
    esac
fi

#
# Begin 
#
echo Package: ${pkg_namver}
work_dir=$(mktemp -d -p /tmp)
cd $work_dir || exit 1

#
# File copying
#
echo Copying package files...
while read i; do
    if [ -z "$i" ] ; then
        continue
    fi

    if [[ "$i" =~ %[A-Z]*% ]] ; then
        current=$i
        continue
    fi

    case $current in
        %FILES%)
        if [ -d "/$i" ]; then
            mkdir "$i"
        elif [ -f "/$i" ]; then
            install -D "/$i" "$i"
        else
            echo "/$i" is missing: this might result in a broken package
        fi
        ;;
    esac
    pkg_size=$(du -sb | awk '{print $1}')
done

echo Adding install commands and changelogs, if any...
if [ -f "$pkg_dir/install" ] ; then
    cp "$pkg_dir/install" "$work_dir/.INSTALL"
fi
if  [ -f $pkg_dir/changelog ] ; then
    cp "$pkg_dir/changelog" "$work_dir/.CHANGELOG"
fi

#
# .PKGINFO stuff
#
echo 
echo Generating .PKGINFO metadata...
echo "# Generated by $progname $progver"    > .PKGINFO
echo "# $(LC_ALL=C date)"            >> .PKGINFO
echo "#"                    >> .PKGINFO

cat "$pkg_dir"/{desc,files,depends} |
while read i; do
    if [[ -z "$i" ]]; then
        continue;
    fi

    if [[ "$i" =~ %[A-Z]*% ]] ; then
        current=$i
        continue
    fi

    case "$current" in

        # desc
        %NAME%)
        echo "pkgname = $i"    >> .PKGINFO
        ;;
        %VERSION%)
        echo "pkgver = $i"    >> .PKGINFO
        ;;
        %DESC%)
        echo "pkgdesc = $i"    >> .PKGINFO
        ;;
        %URL%)
        echo "url = $i"    >> .PKGINFO
        ;;
        %LICENSE%)
        echo "license = $i"    >> .PKGINFO
        ;;
        %ARCH%)
        echo "arch = $i"    >> .PKGINFO
        ;;
        %BUILDDATE%)
        echo "builddate = $(date -u "+%s")"    >> .PKGINFO
        ;;
        %PACKAGER%)
        echo "packager = $pkg_pkger"        >> .PKGINFO
        ;;
        %SIZE%)
        echo "size = $pkg_size"        >> .PKGINFO
        ;;
        %GROUPS%)
        echo "group = $i"    >> .PKGINFO
        ;;
        %REPLACES%)
        echo "replaces = $i"    >> .PKGINFO
        ;;

        # files
        %BACKUP%)
        # strip the md5sum after the tab
        echo "backup = ${i%%    *}"   >> .PKGINFO
        ;;

        # depends
        %DEPENDS%)
        echo "depend = $i"   >> .PKGINFO
        ;;
        %OPTDEPENDS%)
        echo "optdepend = $i" >> .PKGINFO
        ;;
        %CONFLICTS%)
        echo "conflict = $i" >> .PKGINFO
        ;;
        %PROVIDES%)
        echo "provides = $i"  >> .PKGINFO
        ;;

                %FORCE%)
                echo "force = true" >> .PKGINFO

    esac
done

#
# Generate the package 
#
echo 
echo Generating the package...

ret=0
bsdtar -czf "$pkg_dest/$pkg_namver-$pkg_arch.tar.gz" $(ls -A) || ret=$?
if [ $ret -ne 0 ]; then
    echo Error: Unable to write package to $pkg_dest
    echo Maybe the disk is full or you do not have write access
    exit 1
fi

rm -rf $work_dir

exit 0

# vim: set ts=2 sw=2 noet:

Allan, thank you: I finally understood what "# vim: set ts=2 sw=2 noet:" means

As for the arch detection from makepkg.conf Allan brought about, I thought there could be more than 1 arch declared in the desc file which could have been a problem.
Thanks for your effort.

@carlocci: This is unless you wants to submit it yourself.  Either way, you bask in the glory that is contributing to the pacman code base!

That's awesome!
I'll submit the script in a couple of hours.


edit: sed learnt: REGEXP+sed = brain cancer

Last edited by carlocci (2008-05-29 17:48:28)

Offline

Board footer

Powered by FluxBB