auto checking for new package ! really usefull

geekitus · 2005-05-07 14:54:20

hi archer's

first .. i want to say that i'am very beginner with bash and also with english .. so sorry if my script is not perfect

I don't know if anyone already speak about this for archlinux but frugalware use a text web navigator to find the latest release of a package. I juste write a litle script for this idea.

I have some PKGBUILD in my local repo , and i really don't like to check every week if a new release is out. But it will be very usefull like for AUR to send mail automatiquely to the maintener of a PKGBUILD if it need to be updated.

So i had a new line in my PKGBUILD like this :

exemple for tuxracer :

MAJ='elinks -dump "http://tuxracer.sourceforge.net/download.html" | grep tuxracer-.*.tar.gz | sed -n "s/.*tuxracer-(.*).tar.gz.*/1/; 1 p"'

the $MAJ value is :

[root@geekitus geekitus]# lynx -dump "http://tuxracer.sourceforge.net/download.html" | grep tuxracer-.*.tar.gz | sed -n "s/.*tuxracer-(.*).tar.gz.*/1/; 1 p"
0.61

Ok, so $MAJ is the latest release number of the package ...

exemples for some others PKGBUILD :
for t1utils :

[/[root@geekitus geekitus]# lynx -dump http://www.lcdf.org/type | grep "t1utils-.*.tar.gz" | tail -n 1 | sed -e "s/.*-(.*).tar.gz.*/1/"
1.32

for ppracer :

[geekitus@geekitus Mail]$ lynx -dump "http://projects.planetpenguin.de/racer/downloads.php" | grep "ppracer-.*.tar.bz2" | tail -n 1 | sed "s/.*ppracer-(.*).tar.bz2.*/1/"
0.3.1

Pour postgresql :

[geekitus@geekitus Mail]$ wget --passive-ftp -O - -q ftp://ftp.postgresql.org/pub/source/|grep '.[0-9]/</a> $'|sed -n 's|.*v([0-9.]*)/</a> $|1|;$ p'
8.0.3

After i just create a very litle script (with my bad bash level) wich read the $pkgver= and $MAJ and wich compares their value.

So this script could be launched with crond in the night or i could be send a mail at the package maintener for exemple.

Results of the script : (the log file)
exemple with the mode "log all" :
(i had some colors , i like the color ! )

***********************************************************************************
check on :dimanche mai 08, 2005 at 05:27:44 CEST
***********************************************************************************
the log file exist OK
check of amarok :
---------------------
pkgver in the PKGBUILD : 1.2.3
latest release : 1.2.3
pkgver of the amarok PKGBUILD is updated : 1.2.3

check of amarok2 :
---------------------
pkgver in the PKGBUILD : 1.2.4
latest release : 1.2.3
pkgver of the amarok2 PKGBUILD is newest : 1.2.4 <-> 1.2.3

check of dillo :
---------------------
pkgver in the PKGBUILD : 0.8.3
latest release : 0.8.4
the dillo PKGBUILD need to be updated : 0.8.3 -> 0.8.4

The script :

the link :
http://perso.wanadoo.fr/cornjump/linux/ … -script.sh

preview :

#!/bin/bash

#define color  : 
xC1='e[0;37m' # white
xC2='e[1;32m' # green
xC3='e[1;34m' # blue
xC4='e[1;31m' # red
NC='e[0m'     # end of color  
#-------------------------------

# vérify the user :
#--------------------------------
if [ $UID != 0 ] ; then
        echo -e "${xC4} You must be root to exec this script $NC"
        exit 0
fi


# where are PKGBUILD and the file log ? :
#-------------------------------------
# directory where are the PKGBUILDs : 
rep_PKGBUILD=/home/geekitus/script-MAJ-PKGBUILD   #exemple /var/abs/local

# log file :
log=/var/log/check_PKGBUILD_update.log     

# verbose :  0: log only the packages wich need to be updated
#            1: log all       
verbose=1

#  add the date of the check in the log : 
#----------------------------------------
echo -e "***********************************************************************************"   >> $log
echo -e "check on :`date +%A` `date +%b` `date +%d,` `date +%Y` at `date +%H:%M:%S` `date +%Z`" >> $log
echo -e "***********************************************************************************"   >> $log

#test and/or create the log file :
#---------------------------------
# first : test if the log file exist :
if [ -f $log ]; then
    echo -en "the log file exist n"                                                  >>$log
    echo -e "                                                          ${xC2}OK$NC"    >>$log
    # if it don't exist, i create it :
else
    echo -en "${xC4}create the log file :$NC n"                                      >>$log
    touch $log
    # after i test if it was created : 
    if [ -f $log ]; then
        echo -e "                                                     ${xC2}OK$NC"    >>$log
    # if not : there is an error :
    else "${xC4} error creating the log file$NC n"                                   >>$log
    fi         
fi


# check the pkgver and the latest package release
#------------------------------------------------

cd $rep_PKGBUILD                                # go in the PKGBUILD directory
for PKGBUILD in `find * | grep PKGBUILD`;       # find all PKGBUILD in the $rep_PKGBUILD=
  do
  
      eval `grep pkgname= $PKGBUILD`;             # find the pkgname
      eval `grep pkgver= $PKGBUILD`;              # find the pkgver
      eval `grep MAJ $PKGBUILD`;                  # find the latest release
      # if verbose = 1, we log all messages  :   
    if (( $verbose == 1 )); then  
        echo "check of $pkgname :  "               >> $log
        echo "---------------------"               >> $log  
        echo "pkgver in the PKGBUILD   : $pkgver"  >> $log 
        echo "latest release           : $MAJ"     >> $log
          
# if the PKGBUILD release of the package is newest than the latest release :        
           if (( $((10#${pkgver//[^0-9]})) >  $((10#${MAJ//[^0-9]})) )); then
              echo -e "${xC3}pkgver of the $pkgname PKGBUILD is newest : $pkgver <-> $MAJ $NC n" >>$log

# if the PKGBUILD is updated :            
          elif  (( $((10#${pkgver//[^0-9]})) ==  $((10#${MAJ//[^0-9]})) )); then
              echo -e "${xC2}pkgver of the $pkgname PKGBUILD is updated :   $pkgver $NC n"       >>$log

# if the PKGBUILD need to be updated :           
          elif (( $((10#${pkgver//[^0-9]})) <  $((10#${MAJ//[^0-9]})) )); then
              echo -e "${xC4}the $pkgname PKGBUILD need to be updated :  $pkgver -> $MAJ $NC n"  >>$log

# if other : there is an error :                 
          else 
              echo -e "${xC4}error calculating... $NC n"                                          >>$log
         fi       
    fi

# if we want to log only the package wich need to be updated :        
    # if verbose = 0 , we log only PKGBUILD we need to be updated
    if (( $verbose == '0' )); then
         if (( $((10#${pkgver//[^0-9]})) <  $((10#${MAJ//[^0-9]})) )); then
              echo -e "${xC4}the $pkgname PKGBUILD need to be updated :  $pkgver -> $MAJ $NC n"  >>$log  
         fi         
     fi
unset MAJ  

done

FUBAR · 2005-05-07 17:42:44

Man, I can't wait to try it out! Doh, where's my PuTTY so I can give it a try right away!

Very nice effort.

FUBAR · 2005-05-07 17:49:30

Well, it seems to work: there was a result log in /var/log and all my PKGBUILDs seem up to date. I haven't verified so far tho.

I also have a feature request: after running the script it would be nice to get a little output, something like "go check /var/log/check_PKGBUILD_update.log" .

Maybe it would be cool to use the output layout pacman uses, to give the script the Arch-feel. But that's of course not important right now.

FUBAR · 2005-05-07 17:55:20

I've got some bad news: there seems to be a newer version for KRename (3.0.4) but your script reported my version (3.0.3) to be the newest. In your defense, version 3.0.3 is still shown on the website.

[edit]
Other package incorrectly reported to be up to date: Leafnode.
[/edit]

geekitus · 2005-05-07 18:30:53

oh, sorry , i missing to explain something :

the line MAJ= can change for each packages .

Exemple for your package :

 lynx -dump "http://www.krename.net/Stable.6.0.html" |grep "krename-.*.tar.bz2" | tail -n 1 | sed -e "s/.*krename-(.*).tar.bz2.*/1/"
3.0.4

Now with the good line it will report the good released

the MAJ line are very often the same or practically the same. Just need a litle "trainning" i think

leeghoofd · 2005-05-07 18:41:34

I think it's a very origninal idee!

One question out of curiosity, why do you use

lynx -dump | ...

isn't this the same as

wget <file> && cat <file> | ...

?

Cheers,
David

neotuli · 2005-05-07 19:10:12

Hate to break it to you, but our buddies over at http://frugalware.org/ came up with this. Frugalware is an Arch-based distro, and I noticed several months ago that they had this [pretty much exact] mechanism in use.

geekitus · 2005-05-07 19:44:08

neotuli wrote:

Hate to break it to you, but our buddies over at http://frugalware.org/ came up with this. Frugalware is an Arch-based distro, and I noticed several months ago that they had this [pretty much exact] mechanism in use.

Arf, i write my first post too quickly ,
My first post for this script was written on the archlinuxfr forum : http://forums.archlinuxfr.org/viewtopic.php?p=1587#1587

(I just research a system to help maintening my PKGBUILD on archlinuxfr.)
On this post, i wrote clearly that i found this idea in frugalware and i just wrote a script to use this idea. I write english with a lot of more difficulty than french and it's true that i should have say ( like on the french forum) that frugalware use this système and i only write a script to use this.

Really sorry for this missing, :oops: (but i didn't thing bad )

I correct this now

neotuli · 2005-05-07 19:54:04

Don't feel bad, I didn't read very carefully, and I didn't mean to accuse, just mention that I've seen it somewhere before, heh.

geekitus · 2005-05-08 10:17:34

neotuli wrote:

Don't feel bad, I didn't read very carefully, and I didn't mean to accuse, just mention that I've seen it somewhere before, heh.

no problems

#*******************************************************

An other solution (nore easy to use but not perfect) will be to get the download internet page of a package, calculate the md5sum, and if the md5sum is different , then, mail the maintener .

So the PKGBUILD will contain the url of the download page and a md5sum of this page. The script will get the download page and calculate the md5sum, if it's diffents then it could send a mail to the package maintener.

In some case this is not perfect, but the download page of a package don't change very often.

What do you think about this ?

dtw · 2005-05-08 10:34:43

wouldn't it be better to pass the pkgname and wildcards to MAJ with a command line option maybe

geekitus · 2005-05-08 10:48:09

dibblethewrecker wrote:

wouldn't it be better to pass the pkgname and wildcards to MAJ with a command line option maybe

oh yes, but i'm not very good in bash (i write my first bash script four weeks ago ) .. so if anyone want to continue/modify/ or add some new functions in the script i will be very happy because i think that it's a really good idea and usefull to use it (like for mailling mainteners automatically in AUR).

just one question : how do the mainteners of archlinux (for current and extra ) to know when a new release of a package is out for the 2500 packages !! ? (i think that all morning they don't search for all packages if a new release is out )

dtw · 2005-05-08 10:53:46

we have to tell them - it's a good system - except when a packge is flagged out of date for about 4 releases then you have to submit a bug report

FUBAR · 2005-05-08 12:05:52

geekitus wrote:

oh, sorry , i missing to explain something :
the line MAJ= can change for each packages .
Exemple for your package :
 lynx -dump "http://www.krename.net/Stable.6.0.html" |grep "krename-.*.tar.bz2" | tail -n 1 | sed -e "s/.*krename-(.*).tar.bz2.*/1/"
3.0.4
Now with the good line it will report the good released
the MAJ line are very often the same or practically the same. Just need a litle "trainning" i think

So, worst case scenario, I'd have to change the MAJ line for each package?

I'll look into it better later: I'm at home on a Windows laptop right now. :cry:

Arch Linux

#1 2005-05-07 14:54:20

auto checking for new package ! really usefull

#2 2005-05-07 17:42:44

Re: auto checking for new package ! really usefull

#3 2005-05-07 17:49:30

Re: auto checking for new package ! really usefull

#4 2005-05-07 17:55:20

Re: auto checking for new package ! really usefull

#5 2005-05-07 18:30:53

Re: auto checking for new package ! really usefull

#6 2005-05-07 18:41:34

Re: auto checking for new package ! really usefull

#7 2005-05-07 19:10:12

Re: auto checking for new package ! really usefull

#8 2005-05-07 19:44:08

Re: auto checking for new package ! really usefull

#9 2005-05-07 19:54:04

Re: auto checking for new package ! really usefull

#10 2005-05-08 10:17:34

Re: auto checking for new package ! really usefull

#11 2005-05-08 10:34:43

Re: auto checking for new package ! really usefull

#12 2005-05-08 10:48:09

Re: auto checking for new package ! really usefull

#13 2005-05-08 10:53:46

Re: auto checking for new package ! really usefull

#14 2005-05-08 12:05:52

Re: auto checking for new package ! really usefull

Board footer