PRIMAL

cactus · 2005-05-06 05:16:03

PRIMAL - Personal Repository Information Manager for ArchLinux

It is an easy way for people to notify you that a package has become stale, and for people to request an update.

For more information, a screenshot, and the location of the code..see HERE

phrakture is trying to use it. Maybe someone else might find it useful.
8)

cactus · 2005-05-09 22:36:15

updated. Added functionality to work against a remote repository.
I also changed the way the package list is obtained. It used to do a dir listing on the pkg directory, now it just snags the db.tar.gz file and with a little creative piping, gets the list of packages.

ps.

wget -q -O - http://path/to/repo.db.tar.gz | tar -tvz | grep -e "^d" | awk '{print $6}'

phrakture · 2005-05-10 14:39:24

cactus wrote:

ps.

wget -q -O - http://path/to/repo.db.tar.gz | tar -tvz | grep -e "^d" | awk '{print $6}'

nice!

cactus · 2005-05-11 00:44:27

updated primal.
more goodies for phrakture.

· split configs into a seperate file (ruby mixin)
· removed wget dependency (currently only requires tar, grep, and awk. in the near future, it will only require tar).
· can use ftp or http or local path for repo.db.tar.gz (ftp is untested, but should work)

cactus · 2005-05-12 00:59:43

more updates
· Removed requirement for grep and awk (thanks phrakture)

Also, phrakture uncovered an issue with ruby 1.6. Apparently uri open is not available in 1.6. I will probably go with using net/* to get the files directly, instead of the wrapper class.

phrakture · 2005-05-12 02:49:16

damn scrubs installing ruby 1.6 on my server! lol

cactus · 2005-05-12 03:01:46

tell em to upgrade, or there will be pimp slaps for everyone!

phrakture · 2005-05-12 03:32:22

cactus wrote:

Also, phrakture uncovered an issue with ruby 1.6. Apparently uri open is not available in 1.6. I will probably go with using net/* to get the files directly, instead of the wrapper class.

it's basically replicating what uri open does... however, I saw somewhere that uri open is a bad idea, because it overwrites the main "open" call with something exploitable (according to what I read, php did something like this and it was a big mess...)

anyway, something like this doesn't seem too bad:

uri=URI.parse(REPO_URI)
case uri.scheme
  when 'file'
    #use IO.open
  when 'ftp'
    #use Net::FTP
  when 'http'
    #user Net::HTTP
  else
    print "not a uri"
end

then just constrain the REPO to be a valid uri (file:// for local) and viola

cactus · 2005-05-12 05:59:50

sounds like a good plan phrak..
I was using uri-open for simplicity sake. Since the input is being parsed from a prespecified file, and not from user input, the possability of it being abused is negligable. If I was parsing user input and trying to open a file based on that input, then indeed, I would not be using it..
Heh...I will probably not be using it anyway.

*goes off to code*

cactus · 2005-05-13 09:22:51

Due to the fact that ftp transactions can take some time, and web queries in general can be slow to unknown remote sites, I am thinking of breaking primal into two components. One component that updates the repository information backend, handles cleaning up stale settings when packages are updated, etc.
The other portion will just be flagging things stale, and sending the notices (if configured to send notices).

The backend update can be run by hand, or in a cron. At least, that is what I am envisioning. This should speed up the frontend, and reduce the cpu time the script uses up, as well as make it more responsive..

Ideas? Comments?

phrakture · 2005-05-13 15:23:55

sounds like a good idea... cron jobs are always fun

it also makes more sense this way, as personal repos don't change all that often (i'd say mine changes about once a week)... so a daily cron job would handle it perfectly (and one could always force the update too)

One suggestion, if this is going to be done in a background process, it may make sense then to, after the compare, do a quick check of the flagged versions vs the newly grabbed versions, and unflag them:
that is, [if(new_version > old_version) dbclear package_name;]

that way, almost no db management would have to take place, as the cron job would take care of it... (or the manual update)

cactus · 2005-05-13 18:55:35

that is the plan phrak. the current script does this, and that will be migrating to the other script.

cactus · 2005-05-14 02:54:44

pretty much finished the new backend script. It is svn if anyone wants to look at the code..

the frontend (index.rb) shoudn't take too long. The backend came out pretty nice in the end...

EDIT: Frontend code is done now too. It was quicker than I expected. Not a whole lot changed from before, except quite a bit of simplification due to moving functionality to the backend.

The script seems faster too, by a little bit. yay!
Also got http working with net/http, so it should be phrakture compatible. ftp also works. Here is a screenshot I tested using the arch current repo and ftp. IMAGE

EDIT2: Release a version 1.0 tarball. More info

wain · 2007-06-26 21:24:44

Please where can I find a tarball ? Primal looks very cool (better than APRV) :-)

cactus · 2007-06-26 21:39:42

hmm.
I will look around for the code.
It should be laying around somewhere on one of my servers...

cactus · 2007-06-28 22:08:32

wain. I couldn't find the code in my backups.
I was long ago and far away.
Looks like it was lost in a server explosion I had a while back...

I did find somebody running it here: http://www.jiwe.org/primal/
I think that is dibble's site for aeges gis stuff. You might see if he has a source tarball for it still.

tardo · 2007-06-29 04:45:48

there's nothing wget -r can't achieve

jb · 2007-06-29 12:39:58

...but wouldn't that just grab the html aftermath of the scripts?

tardo · 2007-06-29 19:26:14

apparently not. grabbed all the required php files for.

wain · 2007-06-29 20:14:57

php files are here: ftp://ftp.sunsite.dk/projects/aegis/www/htdocs/primal/
but it's impossible to download ".configs.php" file (550 Prohibited file name).

tardo · 2007-06-29 20:21:03

http://nagi-fanboi.net/arch/primal.tar.gz

I haven't tested it... so I'm not sure if I grabbed everything right.

cactus · 2007-06-29 20:32:36

ahh cool.
you don't need the .configs.php file itself. you can extrapolate the config values from the code...

Just do a `grep -Ri "config->get" *` from the top level of the resultant source dir.
The config file was a .ini format file, that had a header at the top that wouldn't allow you to view the contents of it via a browser.

Or..you can wait a few hours, and I can post an extrapolated config based on the source code...
shouldn't be too hard.

cactus · 2007-06-29 21:08:13

This will probably work.

;<?php if (eregi('.configs.php', $PHP_SELF)) { die("You can't access this file directly..."); } ?>

[TEMPLATE]
image = "path/to/image"
pkgbuild_url = "url/path/to/pkgbuilds"
packages_url = "url/path/to/binaries"

[DB]
username = "user"
password = "pass"
host = "localhost"
dbname = "test"
table_prefix = "primal_"

[NOTIFY]
email =

[GENERAL]
homepage = 
viewcvs = 
repo_db_url =

wain · 2007-06-29 21:49:58

cactus wrote:

This will probably work.

Thanks

Arch Linux

#1 2005-05-06 05:16:03

PRIMAL

#2 2005-05-09 22:36:15

Re: PRIMAL

#3 2005-05-10 14:39:24

Re: PRIMAL

#4 2005-05-11 00:44:27

Re: PRIMAL

#5 2005-05-12 00:59:43

Re: PRIMAL

#6 2005-05-12 02:49:16

Re: PRIMAL

#7 2005-05-12 03:01:46

Re: PRIMAL

#8 2005-05-12 03:32:22

Re: PRIMAL

#9 2005-05-12 05:59:50

Re: PRIMAL

#10 2005-05-13 09:22:51

Re: PRIMAL

#11 2005-05-13 15:23:55

Re: PRIMAL

#12 2005-05-13 18:55:35

Re: PRIMAL

#13 2005-05-14 02:54:44

Re: PRIMAL

#14 2007-06-26 21:24:44

Re: PRIMAL

#15 2007-06-26 21:39:42

Re: PRIMAL

#16 2007-06-28 22:08:32

Re: PRIMAL

#17 2007-06-29 04:45:48

Re: PRIMAL

#18 2007-06-29 12:39:58

Re: PRIMAL

#19 2007-06-29 19:26:14

Re: PRIMAL

#20 2007-06-29 20:14:57

Re: PRIMAL

#21 2007-06-29 20:21:03

Re: PRIMAL

#22 2007-06-29 20:32:36

Re: PRIMAL

#23 2007-06-29 21:08:13

Re: PRIMAL

#24 2007-06-29 21:49:58

Re: PRIMAL

Board footer