An audacious plan for Package management...

StanIsTheMan · 2009-10-01 02:23:53

I have completed a CLFS build, and have really enjoyed configuring to my hearts content. I have added Roy Marples OpenRC 0.5.0, and have added the zen-sources kernel among other things. This has been a great learning experience & enjoyable.

Now I am tasked with getting a proper package management system. I have seen people use dpkg & rpm's and paco. Since I will not be running a X environment paco is out of the question, I need a good tool. So I installed Arch's pacman.static, and added the mirrorlists, and pacman.conf file. I now can successfully download packages with pacman. But.... I have a problem with its dependency check. I need to trick pacman into thinking I have the core, utils, headers and build utils installed. Note this is a barebones CLFS install at the moment, meaning only the toolchain, coreutils and devices are installed. I do have dhcp working and is active.

So, I was thinking about using ARCH's ABS to download the PKGBUILDS, and manually install the compiled packages. This gives me the freedom to edit the PKGBUILD to suit my standard FHS layout and any needed dependences. Before I can get pacman.static working, I need to know where pacman stores its db, so I can edit it manually to override any unwanted package downloads. Something like Gentoo's world file.

An example of pacman.static output

localhost / # pacman.static -S abs 
resolving dependencies...
looking for inter-conflicts...

Targets (10): kernel-headers-2.6.30.5-1  tzdata-2009m-1  glibc-2.10.1-4  ncurses-5.7-2.1  
              readline-6.0.004-1  bash-4.0.033-1  attr-2.4.43-1  acl-2.2.47-2  rsync-3.0.6-1  abs-2.3.3-1  

Total Download Size:    16.81 MB
Total Installed Size:   59.43 MB

Proceed with installation? [Y/n]

FYI: I am an New ARCH user & learning as I go... And I thought a CLFS build would help with me understanding the Linux core and toolchain a bit more

Allan · 2009-10-01 02:38:36

To manually get the abs tree...

rsync -rv rsync.archlinux.org::abs/{i686,any}/ ./

StanIsTheMan · 2009-10-01 02:50:32

Allan wrote:

To manually get the abs tree...
rsync -rv rsync.archlinux.org::abs/{i686,any}/ ./

Thanks, for the quick reply... I assume I can change the Arch type to x86_64?

Allan · 2009-10-01 02:52:35

StanIsTheMan wrote:

Thanks, for the quick reply... I assume I can change the Arch type to x86_64?

Yes

Themaister · 2009-10-01 10:32:26

I made a little script that did this once. You just do dummypkg -S <package> <version>. And it will install a dummy file that will make pacman think that the package is installed. The package is a completely valid package, so when there is an update, you can install those, or ignore all of them if you want a set-in-stone core. The script relies on sudo, so if you don't have it, or want to use it, just change it to your prefs.

#!/bin/bash

###########
#####
##       Dummypkg v0.1
#####
###########

## Script to install dummy packages in pacman

## Use at own risk!

function install(){
    if [ -d ~/.dummybuild-tmp/$PACKAGE ]; then
       rm -r ~/.dummybuild-tmp/$PACKAGE 
    fi
    
    mkdir -p ~/.dummybuild-tmp/$PACKAGE
    echo "pkgname=$PACKAGE" >> ~/.dummybuild-tmp/$PACKAGE/PKGBUILD
    echo "pkgver=$PKGVER" >> ~/.dummybuild-tmp/$PACKAGE/PKGBUILD
    echo "pkgrel=1" >> ~/.dummybuild-tmp/$PACKAGE/PKGBUILD
    echo "pkgdesc=\"Dummy package created by dummypkg\"" >> ~/.dummybuild-tmp/$PACKAGE/PKGBUILD
    echo "arch=('i686' 'x86_64')" >> ~/.dummybuild-tmp/$PACKAGE/PKGBUILD
    echo "url=""" >> ~/.dummybuild-tmp/$PACKAGE/PKGBUILD
    echo "license=\"GPL\"" >> ~/.dummybuild-tmp/$PACKAGE/PKGBUILD
    echo "depends=()" >> ~/.dummybuild-tmp/$PACKAGE/PKGBUILD
    echo "source=()" >> ~/.dummybuild-tmp/$PACKAGE/PKGBUILD
    echo "build(){" >> ~/.dummybuild-tmp/$PACKAGE/PKGBUILD
    echo "mkdir -p ~/.dummybuild-tmp/$PACKAGE/pkg/etc/dummy" >> ~/.dummybuild-tmp/$PACKAGE/PKGBUILD
    echo "echo >> \"Installed by dummypkg\" >> ~/.dummybuild-tmp/$PACKAGE/pkg/etc/dummy/$PACKAGE-$PKGVER" >> ~/.dummybuild-tmp/$PACKAGE/PKGBUILD
    echo "}" >> ~/.dummybuild-tmp/$PACKAGE/PKGBUILD
    
    cd ~/.dummybuild-tmp/$PACKAGE/
    makepkg || exit 1
    sudo pacman -U *pkg.tar.gz
    cd ~
    rm -r ~/.dummybuild-tmp/
    exit 0
}

function remove(){
   if [ -e /etc/dummy/$RPACKAGE* ]; then
     sudo pacman -R $RPACKAGE
   else
     echo "This package is not installed by dummypkg ... Exiting"
     exit 1
   fi
   exit 0
}

function list(){
    for file in `ls /etc/dummy -1` 
    do
       
          echo ">>  $file"
       
    done
    exit 0
}

function usage(){
    echo "usage: info [[--install] [--list] [--help] [--remove] [--version]]"
    exit 0
}


INSTALL=0
REMOVE=0
LIST=0



if [ "$1" = "" ]; then
    usage ; exit 1
fi 
while [ "$1" != "" ]; do
    case $1 in
        -S | --install )    
            INSTALL=1 
            PACKAGE="$2"
            PKGVER="$3"
            if [ "$2" = "" ]; then
                echo "--install requires a package name"
                exit 1
            fi
            if [ "$3" = "" ]; then
                echo "--install requires a package version"
                exit 1
            fi
            shift
            shift
            ;;
        -R | --remove )    
                REMOVE=1 
                RPACKAGE="$2"
                if [ "$2" = "" ]; then
                    echo "--remove requires a package name"
                    exit 1
                fi
                shift 
                ;;
        -h | --help  )    usage; exit 0 ;;
        -l | --list   )    LIST=1 ;;
        * )               usage ; exit 1 
    esac
    shift
done



if [ $INSTALL = 1 ]; then
  install
fi
if [ $REMOVE = 1 ]; then
   remove
fi
if [ $LIST = 1 ]; then
   list
fi

StanIsTheMan · 2009-10-01 17:29:30

Themaister wrote:

I made a little script that did this once. You just do dummypkg -S <package> <version>. And it will install a dummy file that will make pacman think that the package is installed. The package is a completely valid package, so when there is an update, you can install those, or ignore all of them if you want a set-in-stone core. The script relies on sudo, so if you don't have it, or want to use it, just change it to your prefs.

Ahh... very nice ... This is something I can work with. I was looking for a way to trick pacman into thinking that a package was installed ...

When there are any updates I'll just mask them in pacman.conf, or install a dummy package version.

Thanks

toxygen · 2009-10-02 00:44:48

StanIsTheMan - do you have a webpage or are you willing to create a thread describing your experiences with CLFS, and then adding pacman to transform the installation into arch? I have been thinking about doing something similar and would appreciate any stories of the process you went through.

Gen2ly · 2009-10-02 00:59:57

Just a warning Stan, last time I checked baselayout, Roy had set the permissions on /etc/shadow to 666. You might want to check and see if that got fixed.

StanIsTheMan · 2009-10-02 01:03:11

toxygen wrote:

StanIsTheMan - do you have a webpage or are you willing to create a thread describing your experiences with CLFS, and then adding pacman to transform the installation into arch? I have been thinking about doing something similar and would appreciate any stories of the process you went through.

Actually, I am in the process of documenting my experiences, and hacks. And, within the next week I'll have it published on my Wordpress. I'll publish the CLFS pure64 and CLFS multilib setups. This is all pending the fact that I can get sufficient package management. As you know there lacks any good documentation on package management on a CLFS builds.

My site is down (pending updates, and cataloging of my documentation), but if you subscribe to the thread I'll post a link by the end of next week. If all else fails, I'll start a new forum thread.

Right now, I am testing the multilib setup. It seems that the pure64 CLFS is not very compatible with other current package managers, (Has something to do with FHS layout of different distro's). So I'll be focusing more on the multilib (seems more practical).

StanIsTheMan · 2009-10-02 01:05:38

Gen2ly wrote:

Just a warning Stan, last time I checked baselayout, Roy had set the permissions on /etc/shadow to 666. You might want to check and see if that got fixed.

Thx for the tip .. I'll check it... I'll just add it to my patch set...

-r-------- 1 root root 59 Sep 29 23:12 shadow

Last edited by StanIsTheMan (2009-10-02 01:09:23)

mikesd · 2009-10-02 23:09:21

This is an interesting idea. Please post/blog how this system works out.

StanIsTheMan · 2009-10-03 00:10:39

mikesd wrote:

This is an interesting idea. Please post/blog how this system works out.

Thanks for the interest... I actually started with the idea when I was helping lloydsmart on the http://www.linuxquestions.org/questions … ta-757909/ forums. The thread is is just a brief start of how I got started with this idea by helping lloydsmart. On post #21, I give a brief overview of my status of this project. Which will follow up with a blog post, and will also be posting on this forum, the steps I used to get ABS working on CLFS (still pending).

mikesd · 2009-10-03 03:55:33

Cool. The idea of bringing up a Linux system from source is cool and while there is BLFS, some form of package manager would be useful.

StanIsTheMan · 2009-10-03 04:03:33

mikesd wrote:

Cool. The idea of bringing up a Linux system from source is cool and while there is BLFS, some form of package manager would be useful.

It seems that I am having trouble with the new dependency of pacman "libfetch" ... Its source from NetBSD... Unfortunately, I have very limited experience in making files from NetBSD ... I have been looking over ARCH's PKGBUILD here http://repos.archlinux.org/viewvc.cgi/l … iew=markup, and it refers to a simple make command as expected, though it keeps spitting out this error:

Makefile:13: *** missing separator.  Stop.

I know its probably a simple fix, but I dont know how to get around it yet... Any idea's ..?

Here's where I stand on getting pacman working

pacman: error while loading shared libraries: libfetch.so: cannot open shared object file: No such file or directory

Last edited by StanIsTheMan (2009-10-03 04:12:01)

StanIsTheMan · 2009-10-03 04:25:48

Pacman is officially up and running ... I had to MAKE things work... No lets see what I can do with "Themaister's" handy script....

I think I could have this up and going soon ... oh ya ...!

StanIsTheMan · 2009-10-03 05:13:47

Pacman & ABS are officially up and running on my CLFS box... With much credit that goes to "Themaister", and a, "oh so handy script..." I can confirm their are no conflicts as of yet ... I'll post all of what a did over the weekend to get this working ... Really "Themeister" was the key to getting this working correctly... He helped me jump a big hurdle. Thanks...

Last edited by StanIsTheMan (2009-10-03 05:14:29)

StanIsTheMan · 2009-10-03 05:36:27

Issue's so far that I have came across is Perl... I am working setting proper links & changing any global variable paths.

Aprz · 2009-10-03 05:57:04

Instead of tricking pacman into thinking you have certain things, why can't you just do

pacman -Sd <package>

I would think that would work.

StanIsTheMan · 2009-10-03 07:28:08

Aprz wrote:

Instead of tricking pacman into thinking you have certain things, why can't you just do
pacman -Sd <package>
I would think that would work.

LOL... I must say I overlooked that option in the man pages ... I'll give it a try

StanIsTheMan · 2009-10-03 10:18:19

Aprz wrote:

Instead of tricking pacman into thinking you have certain things, why can't you just do
pacman -Sd <package>
I would think that would work.

Yes this will work in some scenario's, except when I actually will need a dependency from a particular package.

Last edited by StanIsTheMan (2009-10-03 10:18:38)

StanIsTheMan · 2009-10-03 12:35:29

I thought I would post a screenshot of my CLFS with TWM w/ the nvidia drivers.

I haven't had much luck with Hal, and getting gpm working was a handful, and It took sometime to get mouse functionality in TWM... I have to write a hal iinit from scratch to get it working.... Since ARCH's init's are not compatible with CLFS.

http://s911.photobucket.com/albums/ac31 … 1024_5.png

Last edited by StanIsTheMan (2009-10-03 12:36:00)

toxygen · 2009-10-07 06:41:55

A couple of questions - I take it you were familiar with linux before tacklign CLFS, what was your distro before?
and have you noticed any performance difference from before to running CLFS?

I remember going through LFS when it was semi-new, and i was using mandrake at the time (before it was mandriva) and needless to say LFS (with some optimizations) was much more stable and had better performance. I'm wondering how things have come along since then (been over 10 years! jebus i feel old)

toxygen · 2009-10-07 06:46:50

StanIsTheMan wrote:

I thought I would post a screenshot of my CLFS with TWM w/ the nvidia drivers.
I haven't had much luck with Hal, and getting gpm working was a handful, and It took sometime to get mouse functionality in TWM... I have to write a hal iinit from scratch to get it working.... Since ARCH's init's are not compatible with CLFS.

That's not good. Have you read the boot process wiki? that might have some thing you could try out.

StanIsTheMan wrote:

http://s911.photobucket.com/albums/ac31 … 1024_5.png

That's very nice. I wish I had the time like i did way back when to devote to LFS and setting everything up. Maybe little by little eh?

StanIsTheMan · 2009-10-07 07:49:06

toxygen wrote:

A couple of questions - I take it you were familiar with linux before tacklign CLFS, what was your distro before?
and have you noticed any performance difference from before to running CLFS?
I remember going through LFS when it was semi-new, and i was using mandrake at the time (before it was mandriva) and needless to say LFS (with some optimizations) was much more stable and had better performance. I'm wondering how things have come along since then (been over 10 years! jebus i feel old)

First off ... love the avatar ...:lol: ...

Ya, I started using Linux when I was working at Novell "I was a technical sales rep (you know the type that talk's the talk, but forgot how to walk" I started with Mandrake, and Redhat around 2000 (and yes where old ... lol..)... Then became frustrated with the lack of user control, and being so frustrated. I reverted back to using M$.

So last year I tried tackling LFS (I was ambitious), and well with my lack of any knowledge of the toolchain & alot of WTF's ... I said the hell with it. (mostly my lack of understanding the coreutils.) So, installed Debian, Mandriva, Ubuntu, Fedora ... etc ... Hated all of them. So again I became ambitious ... And went for Gentoo, at first a very love hate relationship. Hated the install mistakes .. alot of "DOPE'S...!" and redo's ... Eventually started to see how everything was put together ..(Google was my friend and still is ...lol) After learning the "Gentoo way" I found "Funtoo" & been with it ever since. My bash programming skills are novice ... but, I'm catching on quickly.

I really like the Arch way... I like how they have a minimal approach (Which is a minimalist dream)... And I like the chroot approach to an install. So.. Arch officially became my secondary distro of choice. Its flexibility with its package manager, the ABS and AUR are a bonus... Though I am still much addicted to the Gentoo way.

I started the CLFS recently, because over time I have a better understanding of the coreutils & toolchain, and the way software compiles (Gentoo, and making my own ebuilds helped) And it didn't seem to over ambitious, as was it for me at first (the first LFS install)... Oh man that was harsh .. Now it seemed like a breeze... (w/ CLFS) I was actually understanding what I was typing.

Since I had been so addicted to the way of Funtoo, and its boot scripts, I had to implement OpenRC into CLFS... And to my much surprise, I was able to implement it .... Man that was cool. I am not using Roy's Baselayout as the previous poster mentioned , only the RC.

Performance wise ... Well FAST .. very fast (Using the zen-sources kernel). After I had built TWM and posted screenshots... I decided to revert back to my backed up clone... I made several backups along the way ( so at anytime I made a mistake or broke the toolchain I could revert back) ... what a relief... I did this especially before I started using Arch's binaries... I wasn't sure if this would cause problems with my toolchain. So I backed up right before I loaded pacman. And I was right...! Through testing, I had made some mistakes with improper symlinks etc... So my general idea now is only to use PKGBUILD's through ABS... This way I can adjust any dependencies, dir .. or remove, add any symlinks... Though the dummypkg script above is useful... I just need to look at it closer. After testing it, it seemed fickle... and it wasn't working for certain packages...

I am actually about 75% done with my write up... I am actually finishing a script that automates the pacman install into CLFS, by building all the pacman packages needed with reverse dependencies removed from the PKGBUILD's... Its handy for a newbie's and the lazy's ... he he ... Though the script will not install into CLFS... only copy them over into the CLFS partition. Reason for this, is explained in my How To ... Just needed a clean environment to install w/ pacman.static. And it seemed cleaner this way ... Though you can easily hack the script to install in a chroot.

So I'm debating on just posting on the forums... then later work with the blog explaining some of my Hal, locale, and OpenRC hacks, and .bashrc tricks. My attention was diverted to only getting pacman, ABS working on CLFS only. All the other hacks, like getting a working X environment, Drivers .. etc. will have to be in my blog. I just dont know much about CSS, XML. And that would take at least another week to get a good page.

As for "the boot process" I am using something completely different then the Arch way. I'm using the OpenRC model, and IMHO its better... (Debatable I'm sure), but just a preference. So my hal init needs to be compatible with OpenRC. Really shouldn't be hard... just a few functions w/ start|stop|restart .

Thanks for the interest and feedback.... I have been working on my script.... and my write up is about finished. So I'm thinking this weekend I'll open a new forum topic ... then see if I get some good feedback .. revisions ... etc... I love the open source community.

Last edited by StanIsTheMan (2009-10-07 09:21:38)

Arch Linux

#1 2009-10-01 02:23:53

An audacious plan for Package management...

#2 2009-10-01 02:38:36

Re: An audacious plan for Package management...

#3 2009-10-01 02:50:32

Re: An audacious plan for Package management...

#4 2009-10-01 02:52:35

Re: An audacious plan for Package management...

#5 2009-10-01 10:32:26

Re: An audacious plan for Package management...

#6 2009-10-01 17:29:30

Re: An audacious plan for Package management...

#7 2009-10-02 00:44:48

Re: An audacious plan for Package management...

#8 2009-10-02 00:59:57

Re: An audacious plan for Package management...

#9 2009-10-02 01:03:11

Re: An audacious plan for Package management...

#10 2009-10-02 01:05:38

Re: An audacious plan for Package management...

#11 2009-10-02 23:09:21

Re: An audacious plan for Package management...

#12 2009-10-03 00:10:39

Re: An audacious plan for Package management...

#13 2009-10-03 03:55:33

Re: An audacious plan for Package management...

#14 2009-10-03 04:03:33

Re: An audacious plan for Package management...

#15 2009-10-03 04:25:48

Re: An audacious plan for Package management...

#16 2009-10-03 05:13:47

Re: An audacious plan for Package management...

#17 2009-10-03 05:36:27

Re: An audacious plan for Package management...

#18 2009-10-03 05:57:04

Re: An audacious plan for Package management...

#19 2009-10-03 07:28:08

Re: An audacious plan for Package management...

#20 2009-10-03 10:18:19

Re: An audacious plan for Package management...

#21 2009-10-03 12:35:29

Re: An audacious plan for Package management...

#22 2009-10-07 06:41:55

Re: An audacious plan for Package management...

#23 2009-10-07 06:46:50

Re: An audacious plan for Package management...

#24 2009-10-07 07:49:06

Re: An audacious plan for Package management...

Board footer