Pacman with db support?

cactus · 2005-03-22 19:00:40

:oops:
musta been tired. my apologies for being an asshat...

xerverius · 2005-03-22 19:47:59

Maybe it's usefull to take a look at the packagemanager of trustix, named SWUP it works with rpm files and uses 1 big package list and a separate file for each package (with discription etc.).

ftp://ftp.trustix.org/pub/Trustix/curre … stix/rdfs/

I don't think it's the solution to this problem, but you can even take a look. I (also) don't know if it's a slow system, because it's a server distro. But it works quite the same as pacman.

xerxes2 · 2005-03-23 00:08:27

maybe it's better to have a "/var" partition,

miqorz · 2005-03-26 14:04:09

I have almost a full blown GNOME install and at one point a all of kde installed.

A ton of media packages and -ALOT- of hardrive space being taken up very fast.

And I STILL haven't experienced these slowdowns with pacman.

This is the same first install I did on 2004-12-30.. Heh.

c0ldevil · 2005-03-28 18:13:23

How can you compare portage to pacman saying that portage is slower? Portage has thousands more packages available. Moreover, portage is a superior package manager, IMHO. It's true that Arch may not need all the features and capabilities that portage offers, but still you can't compare the two saying that pacman is faster than portage.

My 2 cents...

miqorz · 2005-03-28 18:18:44

Hah! Oh boy we got ourselves a gentoo fanboy here.

Portage is slow for one very big reason. it's written in python.

The ammount of ebuilds on the system has little to do with how fucking slow the piece of crap is.

And another thing that pisses me off about Gentoo is they have the nerve to start a gui installer project without reverse dependency code? Don't try to tell me "its coming soon".

We've been hearing that song and dance well over a year..

Pacman is *much* better than portage.
So are most package management systems.

phrakture · 2005-03-28 18:26:01

c0ldevil wrote:

How can you compare portage to pacman saying that portage is slower? Portage has thousands more packages available.

and this matters *how* when it comes to execution speed? we're talking about downloading and installing packages... it's not like pacman or portage have to sequentially search through all packages to do something...

c0ldevil wrote:

Moreover, portage is a superior package manager, IMHO.

any facts? anything else besides the statement? no? I can do that too: pigs have sixteen feet and shoot out of my ass, IMHO

c0ldevil wrote:

It's true that Arch may not need all the features and capabilities that portage offers

lemme guess "use flags", eh?

c0ldevil wrote:

but still you can't compare the two saying that pacman is faster than portage.

sure I can... pacman is faster than portage

i3839 · 2005-03-28 18:43:49

Both Pacman and Portage use the dumb way of syncing the whole database each time. Only thing Pacman does better is to compress it and download it in its whole instead of running rsync for 10+ minutes. In the long run both aren't very scalable, though increased network speeds may save Pacman.

Pajaro · 2005-03-28 19:31:16

pacman could use a mixed db system: download the tar.gz files that contain the filesystem based db, then read it and create a single file db that will get refreshed anytime you sync again the db. This way you could have an option to not create the single file db and keep using the filesystem based db.

It has an easy implementation, will speed up pacman and will end with this allways repeating topics in the forum.

pacman is slow. In my system it spends about 1 minute to load the db when /var/lib/pacman is not cached. I experimented with this: a single file db with no optimization needed only 1-2 seconds to load.

rasat · 2005-03-28 19:41:11

Pajaro wrote:

pacman could use a mixed db system: download the tar.gz files that contain the filesystem based db, then read it and create a single file db....

I am doing an experiment:
http://bbs.archlinux.org/viewtopic.php?t=11193

Moo-Crumpus · 2005-03-29 07:56:21

Pajaro wrote:

In my system it spends about 1 minute to load the db when /var/lib/pacman is not cached. I experimented with this: a single file db with no optimization needed only 1-2 seconds to load.

I have no problem at all running pacman. Maybe because I make use of several scsi discs in a - ermh - somehow useful way:

/dev/sda2 none swap defaults 0 0
/dev/sdb2 none swap defaults 0 0
/dev/sdc2 none swap defaults 0 0
/dev/sdd2 none swap defaults 0 0
/dev/sde2 none swap defaults 0 0

/dev/sda1 / jfs defaults 0 1
/dev/sdb1 /home jfs defaults 0 2
/dev/sdc1 /var jfs defaults 0 2
/dev/sdd1 /opt jfs defaults 0 2
/dev/sde1 /usr jfs defaults 0 2

Having my pacman finished reading its local files takes some seconds. Some minutes? Puh - and if it was this way, I would'nt care that much. I don't see linux updating as a sport.

phrakture · 2005-03-29 15:38:15

I only have a bit of a hang right before an installation returns to the prompt - I'm not sure what's going on there, but it takes maybe 3-5 seconds

berkus · 2005-03-29 17:09:59

If you guys ever consider using a db, take a look at either
a) Metakit since its rather compact itself, produces compact db and is very fast for exactly the thing we need - indexing based on package name hash (or any other hash), or
b) cdb (http://cr.yp.to/cdb.html) since its way simple and can be updated asyncronously with running pacman by separate process (e.g. by running make-pacman-cdb from cron or something).

sqlite adds complexity of an sql parser which is absolutely not needed imo

phrakture · 2005-03-29 17:14:05

berkus wrote:

sqlite adds complexity of an sql parser which is absolutely not needed imo

that's actually not a bad point. there's much more efficient ways to structurally store data... and it's not like we need advanced indexing, foreign key references, and data transformation....

good call berkus, I'll check out some of the stuff you posted

cactus · 2005-03-29 18:47:42

heh. good succinct post berkus. I have been trying to elucidate a similar argument, but I guess people stopped listening to me.

Good to see someone "pick up the ball" and run with it.

phrakture · 2005-03-29 18:51:10

cdb looks better than metakit... and it seems to have all sorts of language bindings... their tutorial sucks, but it still looks like a very simple interface

berkus · 2005-03-29 19:34:40

phrakture wrote:

cdb looks better than metakit... and it seems to have all sorts of language bindings... their tutorial sucks, but it still looks like a very simple interface

yes, cdb is the simplest db in the world i guess... well if not then its in the first 3 simplest for sure.

metakit is a bit pervert because its c++-wannabe with no native bash bindings (but python and tcl only), and it supports more advanced concepts which may not be required by pacman (otoh ability to add new fields or restructure the db on the run without losing it and transactions are always good to keep in mind).

i3839 · 2005-03-29 20:12:20

What about tiny db (tdb)? Or is that too simple? (static lib is 22Kb)

phrakture · 2005-03-29 20:42:09

you got a link? I keep finding either shareware, or russian sites

mercy · 2005-03-29 20:50:44

database.. possible... but is it rly necessary?

drop humanreadability, drop the structure...
the data should be binary and arranged in a binary tree and that can reside in one ~some megs~big file...
easy/fast IO.. easy to compute... what it NEEDS to be.. nothing else

if you wanna *read* something
change a pgk
create a pgk
compile something
whatever
.... there is ABS wich is intendet for that... isn't it?

pacman's "files" dont need to be human readable
ABS "files" (pgk-builds) have to be anyway...

and you NEVER need to touch a DB.. its simply not nesessary

also ... im afread a DB-approach will not speedup things... all you do is that you move the IO from pacman into the DB... and what does the DB? also reading/writing the HD...
so i'd say... due to there is NO NEED for the functions of a DB where it rly IS fast ... just find a better way of file-IO (see beginning of the post)

in worst case you add an resources hungry, time consuming layer with loads of complexity never needed to a realy handy tool

my 2 cents
hope they help
live long and pacman

cactus · 2005-03-29 20:59:11

binary? your kidding right?
Unix == text
text is compressable, transmittable, and doesn't change from version to version. a bit flipped here and there does not ruin the whole thing either.

A db might be a workable solution, or consolodation of the files into a more meaningful/easily used structure..or even a database of some kind...but custom rolled binary formats are inherently evil...

i3839 · 2005-03-29 21:17:18

phrakture wrote:

you got a link? I keep finding either shareware, or russian sites

$ pacman -Qi tdb
Name           : tdb
Version        : 1.0.6-2
Groups         : None
Packager       : dorphell <dorphell@archlinux.org>
URL            : http://sourceforge.net/projects/tdb/
License        : 
Architecture   : 
Size           : 77490
Build Date     : Thu Jan  1 00:55:12 2004 UTC
Install Date   : Wed Aug 25 13:20:00 2004 UTC
Install Script : No
Reason:        : explicitly installed
Provides       : None
Depends On     : glibc 
Required By    : None
Conflicts With : None
Description    : TDB is a Trivial Database

Nothing wrong with custom binary formats, as long as they are used as a cache to avoid using the too slow text database often.

cactus · 2005-03-29 21:23:34

http://www.faqs.org/docs/artu/ch05s01.html
caching is not what I understood him to be meaning. pacman runs very well from cache. It is the first run after boot that people are having a problem with..and of course I was speaking in a broad generalization. There are a few well written binary formats. It is easier to go awry with a pure binary format, but it can be done well...

mercy · 2005-03-29 21:28:23

nope im not kidding

unix == text? i thougt its an OS.. not a parser (that was kidding 8) )

... and personally speaking.. i never lurked arround there.. just in the ABS tree if i was looking for something...
so i say.. we got abs and there is no need for the same info twice

with a DB you have the same troubles ... or even worse...

add an hash to the binary-file -> flipped bits while transmission solved
add parity -> you can correct lot of faults
add backwardcompatibility -> never messup

to fiddle out a good structure for that binary would be much easier/faster to implement and maintain in the future than any db-approach

mercy · 2005-03-29 21:29:32

i didnt talk about caching but about storing the data

Arch Linux

#51 2005-03-22 19:00:40

Re: Pacman with db support?

#52 2005-03-22 19:47:59

Re: Pacman with db support?

#53 2005-03-23 00:08:27

Re: Pacman with db support?

#54 2005-03-26 14:04:09

Re: Pacman with db support?

#55 2005-03-28 18:13:23

Re: Pacman with db support?

#56 2005-03-28 18:18:44

Re: Pacman with db support?

#57 2005-03-28 18:26:01

Re: Pacman with db support?

#58 2005-03-28 18:43:49

Re: Pacman with db support?

#59 2005-03-28 19:31:16

Re: Pacman with db support?

#60 2005-03-28 19:41:11

Re: Pacman with db support?

#61 2005-03-29 07:56:21

Re: Pacman with db support?

#62 2005-03-29 15:38:15

Re: Pacman with db support?

#63 2005-03-29 17:09:59

Re: Pacman with db support?

#64 2005-03-29 17:14:05

Re: Pacman with db support?

#65 2005-03-29 18:47:42

Re: Pacman with db support?

#66 2005-03-29 18:51:10

Re: Pacman with db support?

#67 2005-03-29 19:34:40

Re: Pacman with db support?

#68 2005-03-29 20:12:20

Re: Pacman with db support?

#69 2005-03-29 20:42:09

Re: Pacman with db support?

#70 2005-03-29 20:50:44

Re: Pacman with db support?

#71 2005-03-29 20:59:11

Re: Pacman with db support?

#72 2005-03-29 21:17:18

Re: Pacman with db support?

#73 2005-03-29 21:23:34

Re: Pacman with db support?

#74 2005-03-29 21:28:23

Re: Pacman with db support?

#75 2005-03-29 21:29:32

Re: Pacman with db support?

Board footer