Project ARM :: Arch Rollback Machine

kumyco · 2008-08-18 12:54:42

After a long period of inactivity I've started maintaining the ARM repos again.
The wiki http://wiki.archlinux.org/index.php/Dow … er_Version should detail the relevant info.
Note: the search is currently disabled, though the repositories are fully functional.

Last edited by kumyco (2009-11-01 15:09:52)

timetrap · 2008-08-18 12:59:44

Will your repo include security patches?

This would be really useful (and time consuming) if implemented. We could have a stable Arch, say one year between kernel upgrades.

Just a thought.

kumyco · 2008-08-18 13:35:27

at the moment all that's been implemented are the scripts
well one script and the arch rsync script in the wiki
together this allows a static mirror of the arch (iblibio) mirror
so it's what was there on that day

it would probably be a lot of work to maintain a security patched repo
that's not so much a problem except i don't currently have the facilities
i don't think so anyway...
by that i mean
i am not able to backport/rebuild packages as the number is too large
and i don't have access to my pc, don't know when ill be back on it
for now i only have a laptop
-----
i suppose i could implement a system where different releases of the same pkg as one
so 'pacman 3.2.0-1' would be treated the same as 'pacman 3.2.0-3'
in therory i don't see why this wouldn't work
and it's doable
i'll try and add that to the draft of ARM
it would need some testing though
.......

p.s good idea you've raised

rooloo · 2008-08-18 14:07:28

why not just have a repository that only deletes packages that are say 6 months old as of current date and that package has had at least 5 other releases in that 6 months.

This makes sure of two things, you don't get extrememly old and usless packages and you will always keep at least the 5 latest packages released, even if that was 5 years ago.

That is the only criteria I would look for in a rollback repository.

I do believe you can use pacman as such

pacman -S rollback<or_name_of_repository>/<name_of_rollback_pkg>

kumyco · 2008-08-18 14:41:23

the mirror purge is currently set at approx 1 year post epoch.
in implementing the security patched system (if that's what you're reffering to)
the packages aren't actually delete
the new -pkgrel is linked from a newer sync
the old database (.db.tar.gz) is moved for archival purposes (regeneration of the mirror's databases, reversion, etc)
and a new .db.tar.gz is generated to reflect the current state of the repo
....
after a year or so, it could be after each iso release (depends of how large the collection of pkgs get),, the purge happens,
the purging stage involves deleting redundant pkgs
so if the security oriented repo is deemed usuable, the old releases are removed

as long as space is not a problem, i aim to keep all pkgs in sync for 5 years
or whatever the gpl requires archlinux to do (yes i may soon start syncing the source codes as well)
.......

yes you can install one pkg via pacman as you suggested, i forgot to mention that
but the idea of the mirror to to rollback the whole system
and doing it via the command line would become tedious
.......................

peets · 2008-08-18 15:21:35

Thank you for doing this.

timetrap · 2008-08-18 15:23:58

Ah.

Thank you. It's nice to know that this will be available.

shining · 2008-08-18 16:04:35

How would it work, for making this possible : [<arch_repo>::<year><month><day>] ?
Is it dynamic or is there a real directory for every day, and maybe symlinks inside? or what?

/me feels stupid

PS : I totally support this, I just reverted back to 2.6.25 from 2.6.26 today for a virtualbox guest because of big instability / crash.
I was lucky the chemnitz mirror had it :
http://wiki.archlinux.org/index.php/Downgrade_packages

Last edited by shining (2008-08-18 16:06:33)

kumyco · 2008-08-18 18:00:19

it's a mix of a dynamic and static system, by that i mean
the [<arch_repo>::<year><month><day>], i'll just call it repo structure
is realised by symlinks, the repo is sync via rsync
and then a 10 line? script moves some files around
....
i'm not too sure yet how the security patched pkgs will be done,
i wanted to do it this wasy as well, but am not sure about the effects of having os many symlinks
if i don't go that way
then it will be provided by the dynamic system
probably implemented in php as i don't wanna mess with settign up python(it would also mean if i change server there might be problems)
the arms(script) as i'll call it, will simply lookup the pkg/filename in a database and redirect
so pacman will request arm.kh.nu/arms/pacman-3.2.0-1.pkg.tar.gz and it will redirect you to /core/os/i686/pac~ for example
this will be the basis for allowing to sync back by simply changing the mirror
so you set the mirror to /arms/history/2008-8-18 for example and it redirects pacman, or your browser to /core,extr~/os~
......
i should point out that the files are kept in their respective repos
so they other scripts or whatever can interact with them
.....
long story short
arm.kh.nu/core::2008-8-18 is a symlink to arm.nrk.cc/core
and all the pkgs are where they'd otherwise be (a huge mess, i know)
------------------------------------------------------------------------------------------------------------------------

first feature of `arms`::
list the pkgs that were there on a certain day
currently if u visit the site, u simply get a directory list
--------

p.s thanks for all your support and idea

Last edited by kumyco (2009-05-22 19:41:17)

shining · 2008-08-18 20:59:34

arm.nrk.cc/core::2008-8-18 is a symlink to arm.nrk.cc/core

Oh but wait, you don't need that actually, do you?

You just need a arm.nrk.cc/core/core::2008-8-18.db.tar.gz
And having all core packages (old and new) in arm.nrk.cc/core
That should work.

Then you just do this :

[core:2008-8-18]
Server = http://arm.nrk.cc/core/

Or :

[core:2008-8-19]
Server = http://arm.nrk.cc/core/

kumyco · 2008-08-18 22:27:04

yeh you're right
that was my original idea until i went to put it in pacman mirrors
i realised id need to add one for each repo i synced back to
so i thought 1830 symlinks aren't that bad
--------
implementing the redirect functionality which would take care of this is simple enough
i'm just not too sure the effects of having at least two request just to get one file, hmm.
it's actually partly done already , just need to put in the usual security checks, etc. might get it done by tomorrow
---
actually it's good you brought that up, i just realised the mirror entry in the original post is wrong
-----

it should be

Server = http://arm.kh.nu/$repo/os/i686

substituting i686 forx86_64 where applicable of course
-------
slaps self in the face with foot
....
edit: corrected the number of symlinks

Last edited by kumyco (2009-05-22 19:41:45)

kumyco · 2008-08-19 12:40:05

ARM::Arms

the first stage of the arms script goes live

http://arm.kh.nu/arms/[repo]/[arch] will list the pkgs synced in that snapshot
currently the links are all broken

http://arm.kh.nu/arms/(full repo path) will redirect to the correct url of requested file
i.e both core{,::2008-8-18}/os/core.db.tar.gz redirects to the correct core::2008-8-18.db.tar.gz

this means that we can now sync simpl by changing the mirror

Server = http://arm.kh.nu/arms/$repo::2008-8-19/os/[i686,x86_64]

enjoy
the server can be synced by adding
----
i've only test the db sync and a few pkg downloads so i dont know if it flakes out

Last edited by kumyco (2009-05-22 19:42:17)

kumyco · 2009-05-22 09:37:44

moving to new domain http://arm.kh.nu(prolly not propagated the internet yet (dns)), everything should be as it was @ arm.nrk.cc -- for now you can test it out with arm.nrk.cc
new features:
http://arm.kh.nu/search allows searching through all the archived packages
http://arm.kh.nu/arms now support downloading with just the pakgname-pkgver-pkgrel as opposed to needing the full repo path

the dir-listing inside the repos is currently broken, will fix soon.

litemotiv · 2009-05-22 10:38:13

this is really useful, thanks, i've already used it a couple of times to backtrace regressions. keep up the good work!

kumyco · 2009-05-22 16:07:02

wow that was embarrassing, rooted it to wrong user account lol
i think it's all fixed now, thanks. and thanks, Karol

brisbin33 · 2009-05-22 18:36:19

just to let you know i made a quick bash script to downgrade a single package using your archive. user says for instance `downgrade xorg-server`, then the script spits back the results from your search page (i.e. all old versions of xorg that you've got available); after selecting one, that package is downloaded and installed.

the script can be found here.

thanks for doing this, although i don't use the Rollback Machine per say, the site's been a godsend for me in the past for simply finding old versions of packages (hence the script).

i'll probably refine it later, this is my first pass.

thanks again,
pat

tomk · 2009-05-22 18:49:17

URL updated on wiki's Downgrade page.

kumyco · 2009-05-22 19:38:15

btw, d means set defaults (core, extra, community), it takes effect in the event no repos are selected AND d != 0
added list feature, http://arm.kh.nu/search/list/ it's the same as /search excet it prints no html,
packages are listed 1 per line
it accept a list specifice option: split_url , if it's not 0 it breaks the url up into the important bits

normal:    /$repo/os/$arch/$pkgname-$pkgver-$pkgrel{$pkgarch}.pkg.tar.$ext
split_url: $repo|$arch|$pkgname|$pkgver-$pkgrel|$pkgarch.pkg.tar.$ext;

fyi, $pkgarch is one of -i686, -x86_64 or blank(to support some [community] packages not having an arch in their names)
$ext, is currently only gz, but the commpression may be changed or new ones added in the future

error, errors are printed in a very minimal form,

ERR!DB -- couldn't load db
ERR!RESULT -- no results found
ERR!QUERY -- something wrong with query(q) most likely zero length, valid chars are alphanumeric, _, - .. ^ and $ don't count
ERR!REQUEST -- there was no request

i though to just redirect to the normal search page in the event there was no request, but i think it makes scripting harder

tomk wrote:

URL updated on wiki's Downgrade page.

thanks forgot about that

---
directory listing fixed

Last edited by kumyco (2009-05-23 10:35:51)

derkosak · 2009-05-29 23:15:35

Sorry, but... how does this work? Does this enable some sort of automated downgrade?

kumyco · 2009-05-30 00:58:39

in a sense, yes. what you have are snapshots of the iblibio mirror going back since October 1st last year(i think a few days are missing from late last year). at the moment pacman cannot resolve some dependencies(e.g libgnomeui), needed by newer packages, so in that case you'd have to do it manually, i want to write a script or something to help automate this but it's not as simple as it appears (force re-install all currently installed packages) as it's quite dangerous.
i said snapshots, if you go to arm.kh.nu you will see a directory listing of repos in the naming scheme repo::year-month-day, point pacman to one of these using the instructions above(early posts) and it will sync to them, if there are no dependency errors, then a pacman -Syu will downgrade all packages currently installed as they appeared on iblibio on the chosen date, it's just a normal pacman update from a pacman mirror that just happens to be in the past, if you don't use a date in the repo name you get the last snaphot, from midnight western? time usa.

the main use has been to downgrade certain packages, you can search http://arm.kh.nu/search , if you know the package name and version then http://arm.kh.nu/arms/pacman-3.2.2-1-i686 the pkg.tar.gz extension is optional, there is also a script above that helps with this from the command-line.

DonVla · 2009-07-30 18:49:41

hi kumyco,

is http://arm.kh.nu down?

kumyco · 2009-07-30 18:57:08

Nope, working fine here - just extremely slowly.
The search page works ok, though. http://arm.kh.nu/search/

kumyco · 2009-11-01 15:13:05

Anouncing ARM2, featuring a new repository layout arm.kh.nu/YYYY/MM/DD/$repo/os/$CARCH
see the wiki http://wiki.archlinux.org/index.php/Dow … er_Version for instructions on how to use it.

Last edited by kumyco (2009-11-01 15:14:34)

grey · 2009-11-02 09:01:16

To get automatic access to packages from a given date you need to use

[extra::2009-10-30]
Server = http://arm.kh.nu/old/extra/os/x86_64

Differences to previous: 1. add /old to the path, 2. using the variable $repo in the Server line doesn't work

Last edited by grey (2009-11-02 09:02:15)

DonVla · 2009-11-04 12:44:34

Hallo kumyco,

Does the search interface work anymore?
Eg, when searching for "kernel26" (http://arm.kh.nu/search/index.php?a=64& … &testing=1) I get only:

core
kernel26-firmware Array ( [2.6.31] => Array ( [1] => 1003440 ) )
kernel26-lts Array ( [2.6.27.38] => Array ( [1] => 28353051 ) )
kernel26 Array ( [2.6.31.5] => Array ( [1] => 36630994 ) )
...

Is this a bug?

Vlad

Arch Linux

#1 2008-08-18 12:54:42

Project ARM :: Arch Rollback Machine

#2 2008-08-18 12:59:44

Re: Project ARM :: Arch Rollback Machine

#3 2008-08-18 13:35:27

Re: Project ARM :: Arch Rollback Machine

#4 2008-08-18 14:07:28

Re: Project ARM :: Arch Rollback Machine

#5 2008-08-18 14:41:23

Re: Project ARM :: Arch Rollback Machine

#6 2008-08-18 15:21:35

Re: Project ARM :: Arch Rollback Machine

#7 2008-08-18 15:23:58

Re: Project ARM :: Arch Rollback Machine

#8 2008-08-18 16:04:35

Re: Project ARM :: Arch Rollback Machine

#9 2008-08-18 18:00:19

Re: Project ARM :: Arch Rollback Machine

#10 2008-08-18 20:59:34

Re: Project ARM :: Arch Rollback Machine

#11 2008-08-18 22:27:04

Re: Project ARM :: Arch Rollback Machine

#12 2008-08-19 12:40:05

Re: Project ARM :: Arch Rollback Machine

#13 2009-05-22 09:37:44

Re: Project ARM :: Arch Rollback Machine

#14 2009-05-22 10:38:13

Re: Project ARM :: Arch Rollback Machine

#15 2009-05-22 16:07:02

Re: Project ARM :: Arch Rollback Machine

#16 2009-05-22 18:36:19

Re: Project ARM :: Arch Rollback Machine

#17 2009-05-22 18:49:17

Re: Project ARM :: Arch Rollback Machine

#18 2009-05-22 19:38:15

Re: Project ARM :: Arch Rollback Machine

#19 2009-05-29 23:15:35

Re: Project ARM :: Arch Rollback Machine

#20 2009-05-30 00:58:39

Re: Project ARM :: Arch Rollback Machine

#21 2009-07-30 18:49:41

Re: Project ARM :: Arch Rollback Machine

#22 2009-07-30 18:57:08

Re: Project ARM :: Arch Rollback Machine

#23 2009-11-01 15:13:05

Re: Project ARM :: Arch Rollback Machine

#24 2009-11-02 09:01:16

Re: Project ARM :: Arch Rollback Machine

#25 2009-11-04 12:44:34

Re: Project ARM :: Arch Rollback Machine

Board footer