[SOLVED]Recursively use makepkg with python...or bash!

rhune · 2009-08-20 21:33:51

I am basically trying to build a base Archlinux system from scratch on my eee pc. I researched Pacbuild a little bit but decided to try my own had at this.

After being completely unsuccessful at trying to write bash scripts with loops and variables and whatnot, I took a look at python. I basically just searched a bunch of python tutorials on google and came up with this. So..this being my first script and all I was wondering if this would be the right way to do this. It basically works except with no error checking and if I want it to stop i have to press ctrl-c until the loop ends...but other that it works fine.

#! /usr/bin/env python

#import bash commands
import os
import sys

#open pkglist file
pkglist = open('/home/rhune/list', 'r')
pkgname = pkglist.readline() #read pkgname in file 

while len(pkgname) != 0:    
    pkgname = pkgname.rstrip('\n') #delete newline    
    os.chdir('/home/rhune/eeepkgs')
    os.chdir(pkgname) #cd /home/rhune/pkgname
    os.system('makepkg') #run makepkg in working dir
    os.chdir('/home/rhune/eeepkgs') #cd ../
    pkgname = pkglist.readline() #read pkgname in file


pkglist.close() #close file

The "list" file is a list of folders in the directory im pointing to.

I also plan on having it accept an argument of what folder to run in and using the tempfile to create the directory list, and with some sort of error checking. But thats a whole nother project.

Any comments or contributions appreciated!

edit: is there a way to check if makepkg successfully compiles the package? i.e. does it return a 1 or 0 or some sort of value? Or would i have to parse the actual output of it?

Last edited by rhune (2009-08-28 04:48:05)

rhune · 2009-08-26 05:24:40

Well I figured out a simple bash script (with the help of the internet) to do it without using a text file containing the folder names.

#!/bin/bash

base=$1
cd $base
ls -l
for folder in $(find * -type d)
do
    cd $folder
    makepkg -s    
    cd ../ 
done

Now again, I need to check for to see if makepkg fails and then make a list of all the pkgs that failed to build. Thanks in advance and ill keep searching on the internet. Keep getting hung up on this error checking stuff!

Also working on another python script to grab PKGBUILDs with pbget (or perhaps searching through the local ABS directory) along with their dependencies. Should make it fairly easy to maintain a custom repo with all pkgs built from source. So if anyone would like ill post the code once its working.

Thanks in advance!

Last edited by rhune (2009-08-26 05:32:33)

wankel · 2009-08-26 05:27:21

Im very interested once its figured out, good luck!

klixon · 2009-08-26 06:31:04

rhune wrote:

Now again, I need to check for to see if makepkg fails and then make a list of all the pkgs that failed to build. Thanks in advance and ill keep searching on the internet. Keep getting hung up on this error checking stuff!

check if $? is 0, like this:

    if [ $? -gt 0 ]; then
        echo $folder failed to build
        echo $folder >> $base/failed
    fi

EDIT: You can probably even get away with:

if makepkg -s; then
    echo building $folder succeeded!
else
    echo building $folder failed!
    echo $folder >> $base/failed
fi

Last edited by klixon (2009-08-26 06:34:08)

rhune · 2009-08-28 04:47:46

Thanks much! This works, just gotta pick a better place to store the output.

#!/bin/bash

cd $1
for folder in $(find * -type d)
do
    cd $folder
    makepkg -s
    if [ $? -gt 0 ]; then
        echo $folder >> '/home/rhune/failed'
    else
        echo $folder >> '/home/rhune/passed'
    fi
    cd ../
done

Also figured out this in python:

!/usr/bin/env python

import os
import subprocess

base = "/home/rhune/test/pkgs"
failed=[]
dir = os.listdir(base)
dir.sort()
os.chdir(base)
print dir
for x in range(len(dir)):
    os.chdir(dir[x])
    retcode = subprocess.call(['makepkg', '-s'])
    if retcode==1:
        failed.append(dir[x])
    os.chdir(base)

print failed

The bash one seems a bit simpler though.

Also, this:

#! /usr/bin/env python

import os
import sys

deplist=[]
depname=""
count=0 #count apostrophes

#search for depends= line in pkgbuild
with open('/home/rhune/pkgs/yaourt/PKGBUILD', 'r') as pkg:
    for deps in pkg:
        if "depends" in deps: break

deps=deps.strip('depends=')

#insert depnames into deplist
for x in deps:
    if x=="'": count+=1 #counts apostrophes
    if x.isalpha(): depname+=x
    if count==2:
        deplist.append(depname)
        depname="" #reset depname
        count=0 #reset count    

#pbget for each dependency in list
for x in range(len(deplist)):
    os.system('pbget --arch=i686 '+deplist[x])

Reads a PKGBUILD file and pbgets dependencies. Need to change sys to subprocess, didnt realize it was outdated. Still needs a little work, but its practically done. Also made a sed script to go through each directory and change the PKGBUILD to use gcc-4.5 for the atom optimization.

#!/bin/bash

cd $1
ls -l
for folder in $(find * -type d)
do
    cd $folder
    sed -s 's/\.\/configure/CC=gcc-4.5 \.\/configure/' PKGBUILD > tmpfile ; mv tmpfile PKGBUILD    
    cd ../
done

These being the first time ever writing any sort of scripts, i think im getting the hang of it. Thanks to a little (and i mean little) bit of c++ knowledge.

So im basically ready to compile the whole system, hopefully itll all work!

Thanks

edit: also, maybe a stupid question, but why does $0 -gt 0 work? does the script return a 1 or 0 after each iteration or is it from makepkg?

Last edited by rhune (2009-08-28 04:53:52)

steve___ · 2009-08-28 14:15:06

The '0' is the exit code from the command.

It says "if the test of the exit code is true..." -- where '[' means 'test', '$?' is the exit code and '0' is true ('1' would be false).

And you mean "$?" not "$0". FWIW those are what Bash calls 'Special Parameters'. If you want to know more about Bash I suggest

http://mywiki.wooledge.org/BashGuide

Arch Linux

#1 2009-08-20 21:33:51

[SOLVED]Recursively use makepkg with python...or bash!

#2 2009-08-26 05:24:40

Re: [SOLVED]Recursively use makepkg with python...or bash!

#3 2009-08-26 05:27:21

Re: [SOLVED]Recursively use makepkg with python...or bash!

#4 2009-08-26 06:31:04

Re: [SOLVED]Recursively use makepkg with python...or bash!

#5 2009-08-28 04:47:46

Re: [SOLVED]Recursively use makepkg with python...or bash!

#6 2009-08-28 14:15:06

Re: [SOLVED]Recursively use makepkg with python...or bash!

Board footer