You are not logged in.

#1 2013-03-25 04:31:05

Korrode
Member
From: Australia
Registered: 2009-11-02
Posts: 110

How many other people run off a repo snapshot rather than rolling?

I've been running Arch for many years now, for more than 2 years i haven't been running it in a rolling-release fashion.
Every 8-12 months, usually at a time when I perceive the state of Arch's repo to be comparatively 'stable'*, I make a local copy of the entire Arch repo and I run my system off that.

* i.e. I don't do this 2 days after some huge change to core software (eg. when I saw systemd as default on the horizon, I made a snapshot of the repo, because I didn't want a snapshot just after the implementation of a whole new init system. Also at that time version 3.4 of the Kernel was still in [core] and GKH had just announced he would maintain 3.4 as LTS for some time).

Doing this means I can fix any problems I find (whether it be by adjusting a PKGBUILD and/or adding patches and recompiling the package or running an older or newer version of the piece of software, compiled against my current libraries, that seems to fix the issue and not introduce any new ones, etc.) and then be free to install new userland software from my snapshot, or AUR for that matter (sometimes requiring PKGBUILD adjustments), without hitting dependency requirements that involved updating software integral to the core operation of my system, and thus without the risk of breakage to said core softwares.

As would be obvious to many, I'm talking about a type of 'Arch stable repo', of which i'm aware this isn't the first time such a thing has been discussed. However, I'm not proposing to the current Arch devs and package maintainers to create such a thing, one because I know this has been proposed and not acted upon before, and two because as I've just described; I have no need for current Arch staffers to maintain such a repo, I fairly easily do this myself already.
Note that I don't do anything like tracking security bulletins or go looking for bugs to solve that aren't affecting me (à la ArchServer project), that would be way too much work and does not impact my purpose. This is what I run on my personal desktop which does not take any incoming connections from the internet. My server, which handles such tasks, runs Debian Stable.

The question I ask here is how many other people do this or something like it?
I ask first and foremost out of pure curiosity, and secondarily to gauge interest for a potential community project to make such a repo publicly available.


xfce | compiz | gmrun | urxvt | chromium | geany | aqualung | vlc | geeqie

Offline

#2 2013-03-25 04:51:25

WonderWoofy
Member
From: Los Gatos, CA
Registered: 2012-05-19
Posts: 8,412

Re: How many other people run off a repo snapshot rather than rolling?

You know that there exists a thing called the Arch Rollback machine, which is pretty much exactly what you are doing, but it is far mroe frequent.

As far as being able to install stuff on a non-updated system, you might be able to get away with things here and there, but you are asking for way more trouble than it is really worth IMO.

Offline

#3 2013-03-25 05:08:30

Korrode
Member
From: Australia
Registered: 2009-11-02
Posts: 110

Re: How many other people run off a repo snapshot rather than rolling?

WonderWoofy wrote:

You know that there exists a thing called the Arch Rollback machine, which is pretty much exactly what you are doing, but it is far mroe frequent.

True, though as I said, I do bugfix stuff. EDIT: and installing across the 1Gbit connection between my workstation and my server is doubtless faster than from the Rollback machine's webserver(s).

EDIT2: Thinking more about it, a snapshot within the Rollback machine could be used as a base for a 'static' repo, with another repo containing bugfixed packages for that dated snapshot existing and being listed higher in pacman.conf.... but in regards to a potential public project, it'd be leeching the rollback machine's webservers bandwidth for the bulk of the project.

WonderWoofy wrote:

As far as being able to install stuff on a non-updated system, you might be able to get away with things here and there, but you are asking for way more trouble than it is really worth IMO.

Not really sure if you're saying this in context of the Rollback machine, but certainly installing software from my snapshot of the entire Arch repo poses no more risk of trouble than someone who's pacman -Syu'ing daily installing from the official Arch repo. As for AUR; I rarely encounter problems, and even when I do usually they're easily fixed. (eg. Recently I was compiling/packaging newer version of Wine for usage with my from-older-snapshot install, the most I had to do was comment out the glu dependency.)

Last edited by Korrode (2013-03-25 05:19:24)


xfce | compiz | gmrun | urxvt | chromium | geany | aqualung | vlc | geeqie

Offline

#4 2013-03-25 05:14:52

opt1mus
Member
From: UK
Registered: 2011-12-31
Posts: 197
Website

Re: How many other people run off a repo snapshot rather than rolling?

I'm in agreement with WonderWoofy.

I find it unnecessary due to so few issues - despite having enabled [testing] over a year ago. My 'winning streak' may be due to a bare/crude install. Perhaps others are having a very different experience with Arch than I am, however from where I'm sat all is peachy.

To supplement W'W's mention of the Arch Rollback Machine, this may be of interest to you;

ARM search engine

Offline

#5 2013-03-25 08:01:45

Awebb
Member
Registered: 2010-05-06
Posts: 4,331

Re: How many other people run off a repo snapshot rather than rolling?

So, basically you update Arch only once or twice a year and fix problems with ABS, but with causing way more traffic on both ends?

I would not use any networking component at all, that does not get regular security update love from upstream. Stuff like browsers and java are too hot for me to leave them unpatched for a year.

Offline

#6 2013-03-25 10:21:33

bohoomil
Member
Registered: 2010-09-04
Posts: 2,376
Website

Re: How many other people run off a repo snapshot rather than rolling?

Korrode wrote:

I've been running Arch for many years now, for more than 2 years i haven't been running it in a rolling-release fashion.
Every 8-12 months, usually at a time when I perceive the state of Arch's repo to be comparatively 'stable'*, I make a local copy of the entire Arch repo and I run my system off that.

Actually, changing the default update schedule is not what makes a distro stable... Which, I believe, you are aware of. The stable paradigm involves, among others, a different package maintenance routine, including security patches being applied on a regular basis. Thus a stable distribution (or a hardened one) is usually updated quite often and those regular updates are, to a certain degree at least, the warrant of its stability.

This is of course entirely up to you how you are going to maintain your system, but wouldn't it be easier to choose a distribution that shares some functionality of Arch while offering a safer upgrade path? Frugalware comes to my mind in the first place...


:: Registered Linux User No. 223384

:: github
:: infinality-bundle+fonts: good looking fonts made easy

Offline

#7 2013-03-25 12:15:25

blackout23
Member
Registered: 2011-11-16
Posts: 780

Re: How many other people run off a repo snapshot rather than rolling?

I just keep on rolling. Why shouldn't I the repos are very stable all the time.

Offline

#8 2013-03-25 15:36:24

Korrode
Member
From: Australia
Registered: 2009-11-02
Posts: 110

Re: How many other people run off a repo snapshot rather than rolling?

Awebb wrote:

So, basically you update Arch only once or twice a year and fix problems with ABS, but with causing way more traffic on both ends?

Well,
1. Depends on if pulling the whole repo 1-2 times a year is more traffic than daily pacman -Syu'ing. It'd really depend on the amount of and size of installed packages.
2. My ISP mirrors Arch and I rsync off their servers. I'm not causing any traffic that anyone cares about.

Awebb wrote:

I would not use any networking component at all, that does not get regular security update love from upstream. Stuff like browsers and java are too hot for me to leave them unpatched for a year.

I don't use Java online and my browser security settings are manually hardened some.
Nonetheless I take your point.


bohoomil wrote:

Actually, changing the default update schedule is not what makes a distro stable... Which, I believe, you are aware of. The stable paradigm involves, among others, a different package maintenance routine, including security patches being applied on a regular basis. Thus a stable distribution (or a hardened one) is usually updated quite often and those regular updates are, to a certain degree at least, the warrant of its stability.

Indeed, and I would never refer to my setup as "hardened", I like the term "static". Unchanging. Any components installed and working today remain working tomorrow.

bohoomil wrote:

This is of course entirely up to you how you are going to maintain your system, but wouldn't it be easier to choose a distribution that shares some functionality of Arch while offering a safer upgrade path? Frugalware comes to my mind in the first place...

I did try Frugalware, ran it for some months.
At first I thought this was cool, but quickly decided I didn't like that many of their PKGBUILDs contain little other than the running of an automated script; hindered my ability to customise, or at least made the process lengthier. I also found it obstructive. It'd be some massive script designed for auto-updating software from, say, Sourceforge, making figuring out what exactly was going on with the piece of software in question unnecessarily difficult and time consuming.
I also found that, generally, the amount of modification required to AUR PKGBUILD's to get them working was more than with my current setup. Sometimes a lot more (especially considering that much more often than not I need make no modifications).
Lastly; from what I could tell, Frugalware don't really do any kind of 'hardening' or tracking of security patches or anything. They basically do what I do, snapshot their repo at a certain time point and only patch operational bugs, even then they didn't patch at least one proven, known, quite annoying yet easily fixed bug with a package (it was something XFCE related) for the duration of one of their release. Only 6 months later at the next release was it fixed.

Long story short; with the exception of the rolling model, I love Arch, everything about it, I don't want to and have no reason to run anything else.

edit:typos

Last edited by Korrode (2013-03-25 15:39:22)


xfce | compiz | gmrun | urxvt | chromium | geany | aqualung | vlc | geeqie

Offline

#9 2013-03-25 15:40:08

Awebb
Member
Registered: 2010-05-06
Posts: 4,331

Re: How many other people run off a repo snapshot rather than rolling?

An ISP that allows you to rsync stuff from their servers? Besides that, you're right. Depending on the updates, I can create anything between 300MB and 1GB a week worth of updates. I have never tried to download the entire repository. How large is it?

Offline

#10 2013-03-25 15:48:47

Korrode
Member
From: Australia
Registered: 2009-11-02
Posts: 110

Re: How many other people run off a repo snapshot rather than rolling?

Awebb wrote:

An ISP that allows you to rsync stuff from their servers?

http://www.internode.on.net/
http://mirror.internode.on.net/pub/archlinux/

http://mirror.internode.on.net/ wrote:

The Internode File Mirror is available via HTTP, FTP and RSYNC.

---

Awebb wrote:

Besides that, you're right. Depending on the updates, I can create anything between 300MB and 1GB a week worth of updates. I have never tried to download the entire repository. How large is it?

My current snapshot is from August 2012. It includes [core], [community] and [extra] (not [multilib] or [testing] or anything else) for both i686 and x86_64. It is 43.3GB.


xfce | compiz | gmrun | urxvt | chromium | geany | aqualung | vlc | geeqie

Offline

#11 2013-03-25 16:12:15

blasse
Member
From: Poland
Registered: 2008-04-24
Posts: 303

Re: How many other people run off a repo snapshot rather than rolling?

Korrode wrote:
Awebb wrote:

Besides that, you're right. Depending on the updates, I can create anything between 300MB and 1GB a week worth of updates. I have never tried to download the entire repository. How large is it?

My current snapshot is from August 2012. It includes [core], [community] and [extra] (not [multilib] or [testing] or anything else) for both i686 and x86_64. It is 43.3GB.

I'm updating my two arch based laptops once a week and using pacserve to save bandwidth. Quick calculation tells me, that I need about 50gb yearly for both. So I don't see the point... Also, if you rsync repo from your ISP what's stopping you from setting your mirrorlist to this mirror only? Effect would be the same, but you would be up-to-date wink


Proud ex-maintainer of firefox-pgo

Offline

#12 2013-03-25 16:45:51

Korrode
Member
From: Australia
Registered: 2009-11-02
Posts: 110

Re: How many other people run off a repo snapshot rather than rolling?

blasse wrote:

Also, if you rsync repo from your ISP what's stopping you from setting your mirrorlist to this mirror only?

Nothing. That's what I used to do when I still rolled.

blasse wrote:

I'm updating my two arch based laptops once a week and using pacserve to save bandwidth. Quick calculation tells me, that I need about 50gb yearly for both. So I don't see the point...

See below for the point;

blasse wrote:

Effect would be the same, but you would be up-to-date wink

The entire point is to not take on the risks included with "being up-to-date". The point is to not be pacman -Syu'ing regularly, with software integral to basic system operation being regularly updated and thus risking breakage.
There's also other considerations for me, like:

  • System configuration: I know the method of which my system is configured will not change. Once I learn how to configure a component of my system, that method of configuration is the one I will be using and all I need to know until I decide to update the snapshot, something that I'd do when I have at minimum a couple of free days. (Consider my mentioning earlier about taking a snapshot before the jump to systemd as default init system. That is a good example.)

  • Software features and functionality: If upstream change or remove features in such a way I am not happy with, I won't suddenly get it dropped on me. Again, when I have the time to investigate the current state of things prior to intention of a snapshot update (or post update), I can. A good example of this is changes to the XFCE panel layout options between 4.8 and 4.10. My last snapshot updated too me to 4.10 and I was annoyed to find it was not longer possible to configure my lower panel exactly how I had it in 4.8. I spent some time investigating options. In the end I went with a slightly modified setup that vanilla 4.10 supports, but I looked into other options like staying with 4.8 or trying to patch the panel's code to obtain the old setup. This is only one example but the time taken adds up, at least this way when the time will be spent is on my schedule, not when it gets dropped on me by pacman -Syu'ing because I can't be bothered to read every single change to every single piece of software that's going to be updated; that would take even more time.

Kinda feels like i'm 'defending' my setup with all this, which i realise no one is necessarily 'attacking', really all i mean to do is point out that although I realise there are plenty of benefits to rolling, it should be recognised that there are also tangible benefits to a 'repo snapshot' type system that to myself are, and no doubt would be to some other people, quite worthwhile.


xfce | compiz | gmrun | urxvt | chromium | geany | aqualung | vlc | geeqie

Offline

#13 2013-03-25 19:36:10

Awebb
Member
Registered: 2010-05-06
Posts: 4,331

Re: How many other people run off a repo snapshot rather than rolling?

Korrode wrote:

Kinda feels like i'm 'defending' my setup with all this, which i realise no one is necessarily 'attacking', really all i mean to do is point out that although I realise there are plenty of benefits to rolling, it should be recognised that there are also tangible benefits to a 'repo snapshot' type system that to myself are, and no doubt would be to some other people, quite worthwhile.

Yeah, no need for defence in any way. We are just poking little holes in your setup, to see if it starts leaking. You are being very helpful by calmly answering all those questions. It is nice to see, that Arch can be maintained in such a fashion, I always said, that irregular updates on Arch are not as problematic, as some young padawan always try to impose on new users.

Offline

#14 2013-03-26 06:13:53

ngoonee
Forum Fellow
From: Between Thailand and Singapore
Registered: 2009-03-17
Posts: 6,833

Re: How many other people run off a repo snapshot rather than rolling?

Awebb wrote:
Korrode wrote:

Kinda feels like i'm 'defending' my setup with all this, which i realise no one is necessarily 'attacking', really all i mean to do is point out that although I realise there are plenty of benefits to rolling, it should be recognised that there are also tangible benefits to a 'repo snapshot' type system that to myself are, and no doubt would be to some other people, quite worthwhile.

Yeah, no need for defence in any way. We are just poking little holes in your setup, to see if it starts leaking. You are being very helpful by calmly answering all those questions. It is nice to see, that Arch can be maintained in such a fashion, I always said, that irregular updates on Arch are not as problematic, as some young padawan always try to impose on new users.

They ARE problematic with new users who don't know what they're doing. Just look at the process Korrode has described (couple of free days needed for updating).

Its very possible, but the caveat which inexperienced users don't normally think of is that 'possible' doesn't necessarily meaning 'possible for user X'.


Allan-Volunteer on the (topic being discussed) mailn lists. You never get the people who matters attention on the forums.
jasonwryan-Installing Arch is a measure of your literacy. Maintaining Arch is a measure of your diligence. Contributing to Arch is a measure of your competence.
Griemak-Bleeding edge, not bleeding flat. Edge denotes falls will occur from time to time. Bring your own parachute.

Online

#15 2013-03-26 06:57:08

elkoraco
Member
Registered: 2013-02-18
Posts: 121

Re: How many other people run off a repo snapshot rather than rolling?

Seems like a lot of work. Updating any Linux distro is a pretty easy thing to do in my book, and the occasional breakage no biggie to fix.

Offline

#16 2013-03-26 12:55:43

blasse
Member
From: Poland
Registered: 2008-04-24
Posts: 303

Re: How many other people run off a repo snapshot rather than rolling?

I get you're point. We have linux-lts in repo, I think your local repo snapshot with pkgs patched for security may be described as [lts] repo wink

Last edited by blasse (2013-03-26 12:56:06)


Proud ex-maintainer of firefox-pgo

Offline

#17 2013-03-26 20:23:48

Awebb
Member
Registered: 2010-05-06
Posts: 4,331

Re: How many other people run off a repo snapshot rather than rolling?

ngoonee wrote:

problematic …  new users who don't know what they're doing

Yes, new users. They always have issues the old hares don't have. Speaking of which, I just updated my desktop, which had not see an internet connection since October. It was quite unspectacular. Everything post-systemd is more or less boring :-D

Offline

#18 2013-03-26 23:58:22

Primoz
Member
From: Ljubljana-Slovena-EU
Registered: 2009-03-04
Posts: 661

Re: How many other people run off a repo snapshot rather than rolling?

There are distros which have this as their only or main way of updating, you know...
I'm just saying that kind of destroys what Arch does for me...


Arch x86_64 ATI AMD APU KDE frameworks 5
---------------------------------
Whatever I do, I always end up with something horribly mis-configured.

Offline

#19 2013-03-27 02:32:35

ngoonee
Forum Fellow
From: Between Thailand and Singapore
Registered: 2009-03-17
Posts: 6,833

Re: How many other people run off a repo snapshot rather than rolling?

Primoz wrote:

There are distros which have this as their only or main way of updating, you know...
I'm just saying that kind of destroys what Arch does for me...

As long as he's not demanding other people maintain his repo, its up to him how he wants to update. He knows what he's doing and doesn't seem to need assistance (not that he'd get much if he had a problem updating a year-old system, I'd think).

Some people use Arch because "hey, we got software X THREE DAYS AFTER RELEASE", but not everyone.


Allan-Volunteer on the (topic being discussed) mailn lists. You never get the people who matters attention on the forums.
jasonwryan-Installing Arch is a measure of your literacy. Maintaining Arch is a measure of your diligence. Contributing to Arch is a measure of your competence.
Griemak-Bleeding edge, not bleeding flat. Edge denotes falls will occur from time to time. Bring your own parachute.

Online

#20 2013-03-27 08:56:04

Awebb
Member
Registered: 2010-05-06
Posts: 4,331

Re: How many other people run off a repo snapshot rather than rolling?

He'd get some assistance from the Arch-is-stable-as-rock faction :-D

Offline

#21 2013-03-29 16:42:56

jeffmikels
Member
Registered: 2007-04-19
Posts: 31

Re: How many other people run off a repo snapshot rather than rolling?

I have often thought of doing this very same thing. I'm running Arch as a home server, and though the rolling release model is awesome for getting the latest and greatest software and getting the latest security updates, it is troublesome for these following issues:

  • Using old hardware that depends on proprietary drivers not available for the latest kernel (ahem... nVidia).

  • Using server software that expects specific versions of other server software (mythtv, php, perl, python, mysql) to do it's job.

I used to follow the ArchServer project, but I fear it has too little traction and I think there is a better way to do things within the Arch ecosystem.

Python is a perfect example of how Arch can cross the line between rolling release and stability. With Python, there are simply two versions of the codebase that are fully supported. python2 and python each get rolling updates, but maintain version compatibility.

If I had the authority to recommend something to Arch maintainers, I would suggest they implement this same thing across the board with one of the following three options:

  • in pacman.conf enable an option like HoldPkgVersion to allow packages like

    mythtv 1:0.26.0-8

    to be upgraded to

    mythtv 1:0.26.0-9

    but not to

    mythtv 1:0.27.0-1
  • in the major repositories always support all currently supported versions of software. For example, just like python, there should be a package called

    php

    to provide the latest php (5.4.13 currently), but there should also be a package called

    php53

    to provide the latest updates to the php 5.3 branch.

  • transfer old versions of software to the AUR when new versions arise.

Now, I'm not advocating for any change to the rolling release model. If I do

pacman -Sy php

I should get the latest version of php available unless I have specifically told pacman to hold onto a specific version or something like that. I don't want Arch to become Debian. However, I also want to see Arch be more widely usable in server situations where using older php versions and mysql versions may be needed but still require security patches.

Perhaps we could get the ArchServer guys to come back to Arch by simply implementing one of my suggestions.

What do you all think? Should I start a new thread about this?


...using Arch as a home server since 2006.

Offline

#22 2013-03-29 17:28:52

cookies
Member
Registered: 2013-01-17
Posts: 253

Re: How many other people run off a repo snapshot rather than rolling?

jeffmikels wrote:
  • in the major repositories always support all currently supported versions of software. For example, just like python, there should be a package called

    php

    to provide the latest php (5.4.13 currently), but there should also be a package called

    php53

    to provide the latest updates to the php 5.3 branch.

  • transfer old versions of software to the AUR when new versions arise.

That's more or less how it works now, both php 5.2 and php 5.3 can be found in the AUR.

Offline

#23 2013-03-30 06:37:00

jeffmikels
Member
Registered: 2007-04-19
Posts: 31

Re: How many other people run off a repo snapshot rather than rolling?

I expected that to be the case, even though I didn't search for it. I was trying to make a point that having pacman manage these old versions doesn't seem like it would be that difficult. The AUR is wonderful for what it is, but I haven't discovered an easy way to do an "update" for packages in AUR without using something like yaourt. I'd much prefer to simply tell pacman to hold onto an old version. As things currently stand, there is no way to get security updates with pacman while holding back version upgrades.

I'm wondering if this isn't something that could be implemented with a small change to pacman... perhaps I should dig into the code some.


...using Arch as a home server since 2006.

Offline

#24 2013-03-30 11:03:09

Awebb
Member
Registered: 2010-05-06
Posts: 4,331

Re: How many other people run off a repo snapshot rather than rolling?

Yes, you should really have a look at the code. You assume, that it will not be a big deal, but I'm afraid it might be, because you would have to come up with a concept, that is a) robust and b) does not break the package creation process with makepkg and c) does not break compatibility with the current pacman. Then you will have to convince the devs, that they should change their attitudes concerning partial updates and multiple package versions. I do not want to discourage you at this point, but without a solid concept and a great patch set, you will be turned down before you even started. Also make sure to find someone who is working with pacman a lot to tutor you through your process, because it happened more than once that somebody spent a lot of hours working on something, just to be sent away by the devs, because they didn't like it.

Offline

#25 2013-03-31 02:49:14

jeffmikels
Member
Registered: 2007-04-19
Posts: 31

Re: How many other people run off a repo snapshot rather than rolling?

Actually, I have no intention of doing any of that unless there is some interest in it, and based on the number of posts on this thread, it seems there is very little interest in it. I may end up starting a new thread just to see if anyone else would be interested in joining me in this project, but as I said, if there isn't a groundswell of support, there's no way I would even approach the devs with it. They have enough on their plate.

So, back to my original question... Is this a worthwhile thing to pursue? Is anyone else out there in Arch-land interested in a pacman that understands different software versions?

Last edited by jeffmikels (2013-03-31 02:49:45)


...using Arch as a home server since 2006.

Offline

Board footer

Powered by FluxBB