How to pacman -Sy by hand, or against only a single repo? [SOLVED]

Profjim · 2009-07-31 15:37:54

I've written a wrapper for makepkg that builds my packages and adds them to my local custom repo. If I give the wrapper a -i flag, I want to then install them from the repo. I suppose I could just use pacman -U, but I prefer to use yaourt -S local_repo/package_name=desired_version. To do this, though, I have to synch my pacman lib db against the newly updated local_repo. Currently I'm just doing a pacman -Sy to do that. Downside is that hits all the other repos too. Often I find myself building and rebuilding and reinstalling a package multiple times as I tweak the PKGBUILD. Repeatedly synching against all the repos is bad for me (few extra seconds every cycle) and bad for the repos.

What I need is a way to synch against only my local_repo. I can't figure out how to do that using any pacman cmd-line switch. The format of the repo file looks very simple, so I tried doing this manually:

sudo tar -xzf /path/to/local_repo/local_repo.db.tar.gz -C /var/lib/pacman/sync/local_repo
stat --printf "%X" /path/to/local_repo/local_repo.db.tar.gz | sudo tee /var/lib/pacman/sync/local_repo/.lastupdate > /dev/null

But pacman still seems not to notice the changes. E.g. if I do pacman -Si local_repo/new_package, I get a report of the previous package version, not the newly added one.

Is there any way to synch just my local repo, either using pacman or by hand?

Last edited by Profjim (2009-07-31 21:14:34)

djszapi · 2009-07-31 18:31:31

Hello Profjim!

At the company, without network connection, I just use tar -xvpzf $repo.db.tar.gz to sync my system.

Xyne · 2009-07-31 18:40:32

You can use rebase. Not only can it sync individual repos, it will do it incrementally (i.e. new files get added, deleted files get removed, everything else stays the same).

I've had it running in a cronjob for months now and have never had a problem with it

djszapi · 2009-07-31 18:51:39

Really, It's in AUR from Xyne hehe

Profjim · 2009-07-31 19:05:37

Thanks for the responses. @Xyne: yeah I had seen rebase. You have a nice collection of packages going. But I figured this was a one- or two-liner and it seemed easier to me (in the long-run) to figure out what it is than to merge my customized build system with yours. I glanced at the source for rebase but haven't yet had time to find what I'm looking for (I think the relevant part is in the perl libs, no? My ability to read perl is quite brittle)

@djszapi: so you're just able to untar into /var/lib/pacman/synch/local_repo and pacman immediately sees the new stuff? Funny, I don't think that's working for me... It's seeming to take a pacman -Sy before it shows up (which polls all the repos).

Xyne · 2009-07-31 19:24:36

Yeah, the backend is mostly in perl-xyne-arch. It uses the same code as powerpill and some other apps to parse the pacman configuration file etc.

I admit that the rest of its functionality is overkill for your purpose though. The relevant part of the code is just using tar extract files from the database archive and add them to the database directory (using pipes though... I avoid writing to the disk when I can).

It should be enough to just overwrite the old files with the new ones from the archive, but if you're doing this constantly then it seems like unnecessary wear-n-tear on the disk to me. It probably won't matter in the grand scheme of things, but I would avoid it if I could. Anyway, I just thought it might be handy to have a ready-made app with some other features. All my lib stuff takes up less than 500K.

Last edited by Xyne (2009-07-31 19:49:41)

Profjim · 2009-07-31 19:29:01

So there's nothing else that needs to be done to mark the repo as updated? You just untar on top of the directory (or untar the new files) and leave the .lastupdate file alone?

Xyne · 2009-07-31 19:53:28

All you should need to do is extract the database archive into the sync directory (I think that removes all existing files in the related database, but if not, just delete them). You don't have to worry about ruining the sync database either. If you bork something, just remove everything in the sync dir and run "pacman -Sy". As long as you don't touch the local database, it should all be fine.

Profjim · 2009-07-31 20:58:37

You're right, that works. As happens too often, I was making things more difficult than they needed to be...
EDIT: it works sometimes, and sometimes doesn't. I still can't figure out what determines the difference...

If you untar on top of a directory, any existing directories (at least at the top level) that don't exist in the archive will be left there, not removed. But this is fine for my purposes. I can wait until the next full pacman sync to delete them.

And the problem of I-dont-wanna-untar-the-old-stuff-too can be solved by doing:

    sudo tar -xzvf "$PKGDEST/$REPONAME.db.tar.gz" --after-date="@$(cat /var/lib/pacman/sync/$REPONAME/.lastupdate)"  -C /var/lib/pacman/sync/$REPONAME

That is, we use the sync dir's existing .lastupdate file to give us a date (in Unix epoch format, which tar needs to see as "@12234556"), and we tell tar just to extract the files modified after that date. I can tell from the -v output that this is working properly.

But it's still a mystery why sometimes this works and sometimes doesn't...

# ...add new package version to local_repo, then...
sudo repo-add path/to/local_repo.db.tar.gz path/to/new/package
# untar local_repo.db.tar.gz on top of /var/lib/pacman/sync/local_repo, as above, then...
sudo pacman -S local_repo/new_package=new_version

Last edited by Profjim (2009-07-31 21:00:56)

Profjim · 2009-07-31 21:14:12

OK, what's happening is if the old version's metadata is left in the /var/lib/pacman/sync/local_repo dir, then pacman won't see the new version. So all one needs to do is insert a rm -rf on those old directories, then untar the new database file on top of this directory (it's enough just to untar the fresh material from the database file, as described above), then pacman -S local_repo/new_package=new_version will work.

Marking SOLVED.

Arch Linux

#1 2009-07-31 15:37:54

How to pacman -Sy by hand, or against only a single repo? [SOLVED]

#2 2009-07-31 18:31:31

Re: How to pacman -Sy by hand, or against only a single repo? [SOLVED]

#3 2009-07-31 18:40:32

Re: How to pacman -Sy by hand, or against only a single repo? [SOLVED]

#4 2009-07-31 18:51:39

Re: How to pacman -Sy by hand, or against only a single repo? [SOLVED]

#5 2009-07-31 19:05:37

Re: How to pacman -Sy by hand, or against only a single repo? [SOLVED]

#6 2009-07-31 19:24:36

Re: How to pacman -Sy by hand, or against only a single repo? [SOLVED]

#7 2009-07-31 19:29:01

Re: How to pacman -Sy by hand, or against only a single repo? [SOLVED]

#8 2009-07-31 19:53:28

Re: How to pacman -Sy by hand, or against only a single repo? [SOLVED]

#9 2009-07-31 20:58:37

Re: How to pacman -Sy by hand, or against only a single repo? [SOLVED]

#10 2009-07-31 21:14:12

Re: How to pacman -Sy by hand, or against only a single repo? [SOLVED]

Board footer