You are not logged in.

#1 2008-12-22 03:20:08

Allan
Developer
From: Brisbane, AU
Registered: 2007-06-09
Posts: 10,409
Website

Pacman update and mirrorlist

With the upgrade to pacman-3.2.1-2, we have split the mirrorlist into the pacman-mirrorlist package. This will allow us to push more frequent updates to the mirrorlist, instead of only updating it with pacman releases.  As part of this upgrade, your current /etc/pacman.d/mirrorlist file will be saved as mirrorlist.pacorig and the default mirrorlist put in its place.  Remember to merge these files in order to keep using your mirror(s) of choice.

Offline

#2 2008-12-22 03:21:45

rson451
Member
From: Annapolis, MD USA
Registered: 2007-04-15
Posts: 1,233
Website

Re: Pacman update and mirrorlist

Thanks for the heads up.  I'm assuming this will go on the front page as well?


archlinux - please read this and this — twice — then ask questions.
--
http://rsontech.net | http://github.com/rson

Offline

#3 2008-12-22 03:24:26

.:B:.
Forum Fellow
Registered: 2006-11-26
Posts: 5,819

Re: Pacman update and mirrorlist

Nice smile. A few less MB to pull in every time tongue.


Got Leenucks? :: Arch: Power in simplicity :: Get Counted! Registered Linux User #392717 :: Blog thingy

Offline

#4 2008-12-22 03:29:19

Allan
Developer
From: Brisbane, AU
Registered: 2007-06-09
Posts: 10,409
Website

Re: Pacman update and mirrorlist

rson451 wrote:

Thanks for the heads up.  I'm assuming this will go on the front page as well?

Yes.  I have posted it but it takes time for it to actually appear.

Offline

#5 2008-12-22 03:38:02

Xyne
Moderator/TU
Registered: 2008-08-03
Posts: 5,688
Website

Re: Pacman update and mirrorlist

[possible noob question]Is there a way to merge the files automatically or will this require a manual edit with each update?[/possible noob question]

Offline

#6 2008-12-22 04:02:42

Allan
Developer
From: Brisbane, AU
Registered: 2007-06-09
Posts: 10,409
Website

Re: Pacman update and mirrorlist

Xyne wrote:

[possible noob question]Is there a way to merge the files automatically or will this require a manual edit with each update?[/possible noob question]

Just to clarify, it is only this update which will move the mirrorlist file to mirrorlist.pacorig and install the default as mirrorlist.  From now on, new mirrorlists will be installed as mirrorlist.pacnew as usual.  This is being caused by the initial splitting of the mirrorlist away from the pacman package.

If you are happy with your current mirrorlist you can use this poor man's automatic merging (not a recommendation!):

mv /etc/pacman.d/mirrorlist.paorig /etc/pacman.d/mirrorlist

Of course, you would miss out on any new mirrors or mirror removals....

Offline

#7 2008-12-22 04:48:43

sterling
Member
Registered: 2008-11-10
Posts: 5

Re: Pacman update and mirrorlist

Why .pacorig instead of the standard .pacsave?

Offline

#8 2008-12-22 04:59:07

Allan
Developer
From: Brisbane, AU
Registered: 2007-06-09
Posts: 10,409
Website

Re: Pacman update and mirrorlist

sterling wrote:

Why .pacorig instead of the standard .pacsave?

The sort answer, because we are moving that file between packages.

The long answer.... well, time for a lesson in pacman internals!  Pacman notes that is needs to update the pacman and pacman-mirrorlist packages.  It does conflict checking which is smart enough to know that the mirrorlist file is being moved from pacman to pacman-mirror list.  When it goes to install pacman-mirrorlist, the old pacman package still is installed and thus the old mirrorlist file is still on the system.  Thus we have a file that pacman does not think is a conflict but is on the system.  Normally pacman would just overwrite it but given it is in the backup array, it saves a copy to a .pacorig file.  At least, that is my understanding of the code...  I'm sure toofishes or shining can point out where I went wrong.

Offline

#9 2008-12-22 05:13:19

sterling
Member
Registered: 2008-11-10
Posts: 5

Re: Pacman update and mirrorlist

That doesn't differentiate between .pacsave and .pacorig for me unfortunately. The reason I'm concerned with this is I've recently expanded the .pac* article on the wiki and if .pacorig is something distinct then that should probably be covered as well:

http://wiki.archlinux.org/index.php/Pac … save_Files

Offline

#10 2008-12-22 05:21:18

Allan
Developer
From: Brisbane, AU
Registered: 2007-06-09
Posts: 10,409
Website

Re: Pacman update and mirrorlist

This is one of the very few times, or possibly the only time, most people will encounter .pacorig files which is why I posted the news item.  And the difference is mainly due to conflict handling as I tried to explain above.  I would not be too concerned that it is missing from the wiki page, but if you do included it, make it very short.

Offline

#11 2008-12-22 05:47:07

Xyne
Moderator/TU
Registered: 2008-08-03
Posts: 5,688
Website

Re: Pacman update and mirrorlist

If I remember (and still think it would be useful), I'll write a merge script to grab the original mirrorlist, append new mirrors to it, and warn about mirrors that have been removed (and maybe remove them with confirmation maybe). It would probably work for locale.gen and other similar lists too.

Offline

#12 2008-12-22 10:09:02

sterling
Member
Registered: 2008-11-10
Posts: 5

Re: Pacman update and mirrorlist

Allan wrote:
sterling wrote:

Why .pacorig instead of the standard .pacsave?

The sort answer, because we are moving that file between packages.

The long answer.... well, time for a lesson in pacman internals!  Pacman notes that is needs to update the pacman and pacman-mirrorlist packages.  It does conflict checking which is smart enough to know that the mirrorlist file is being moved from pacman to pacman-mirror list.  When it goes to install pacman-mirrorlist, the old pacman package still is installed and thus the old mirrorlist file is still on the system.  Thus we have a file that pacman does not think is a conflict but is on the system.  Normally pacman would just overwrite it but given it is in the backup array, it saves a copy to a .pacorig file.  At least, that is my understanding of the code...  I'm sure toofishes or shining can point out where I went wrong.

After reconsidering the subtlety of what's going on there, I think it would be better to omit any mention of .pacorig from the wiki and just leave it up to the News (and ML/forum announcements) to tell users what specific steps to take when a .pacorig file pops up. wink

Offline

#13 2008-12-22 11:55:40

rson451
Member
From: Annapolis, MD USA
Registered: 2007-04-15
Posts: 1,233
Website

Re: Pacman update and mirrorlist

sterling wrote:

After reconsidering the subtlety of what's going on there, I think it would be better to omit any mention of .pacorig from the wiki and just leave it up to the News (and ML/forum announcements) to tell users what specific steps to take when a .pacorig file pops up. wink

I think that's the right decision.  In the two years I've been around this is the first time I've seen this happen (that I can remember).  Unique situations usually get a news post anyway, the wiki should be for general information.


archlinux - please read this and this — twice — then ask questions.
--
http://rsontech.net | http://github.com/rson

Offline

#14 2008-12-23 06:02:57

sterling
Member
Registered: 2008-11-10
Posts: 5

Re: Pacman update and mirrorlist

Oops, I spoke too soon. phrakture added a stub section for .pacorig to the wiki article and someone else came along and filled it out. So I've tacked on a bit about always going to the News for instructions on dealing with .pacorig. Please proofread and correct any inaccuracies.

Offline

#15 2008-12-23 17:52:46

JonathanArcher
Member
From: San Francisco
Registered: 2008-11-12
Posts: 108

Re: Pacman update and mirrorlist

So, if I understand it correctly, after I checked the new mirrolist file for the mirrors I can savely delete the mirrorlist.pacorig because it will not be needed anymore?

Offline

#16 2008-12-23 18:06:55

R00KIE
Forum Moderator
From: Between a computer and a chair
Registered: 2008-09-14
Posts: 3,218

Re: Pacman update and mirrorlist

The news did the trick for me, manually commented all mirrors that I didn't want and left the only listed Portuguese mirror and added two more and as the last one the arch mirror. It looks like this

# My Servers
Server = ftp://cesium.di.uminho.pt/pub/archlinux/$repo/os/i686
Server = ftp://darkstar.ist.utl.pt/pub/archlinux/$repo/os/i686
Server = ftp://ftp.nux.ipb.pt/pub/dists/archlinux/$repo/os/i686
Server = ftp://ftp.archlinux.org/$repo/os/i686

R00KIE
Tm90aGluZyB0byBzZWUgaGVyZSwgbW92ZSBhbG9uZy4K

Offline

#17 2008-12-23 20:27:27

Xyne
Moderator/TU
Registered: 2008-08-03
Posts: 5,688
Website

Re: Pacman update and mirrorlist

This is somewhat less generic than what I had in mind because I realized that the order of the mirrors need to be preserved. Anyway, the script does the following:

read in uncommented mirrors from mirrorlist.pacorig
check them against the new mirrorlist
for each one that's not in the new list, ask if the user would like to keep it
create a new file with the previously selected mirrors appended to the top followed by the complete new mirrorlist with the server lines commented out

usage:

merge_pacorig_mirrors.pl /path/to/output_file

Check the output file then move it to /etc/pacman.d/mirrorlist if you're happy with it. The script will not overwrite your mirrorlists (unless you specify them as the output file, but I do not recommend that).

#!/usr/bin/perl
use strict;
use warnings;

sub read_file
{
    my $file = shift;
    my $text;
    open(my $fh,'<',$file) or die "unable to open $file: $!\n";
    {
        local $/;
        $text = <$fh>;
    }
    close $fh;
    return $text;
}
sub save_file
{
    my ($file,$text) = @_;
    open(my $fh,'>',$file) or die "unable to open $file: $!\n";
    print $fh $text;
    close $fh;
}

sub confirm
{
    my ($question,$default_answer) = @_;
    print "$question ";
    if (defined($default_answer) and $default_answer eq 'y')
    {
        print '[Y/n] ';
    }
    elsif (defined($default_answer) and $default_answer eq 'n')
    {
        print '[y/N] ';
    }
    else
    {
        print '[y/n] ';
    }
    my $answer = lc(<STDIN>);
    chomp $answer;
    $answer =  $default_answer if (defined($default_answer) and $answer eq '');
    while ($answer ne 'y' and $answer ne 'n')
    {
        print "Please enter 'y' or 'n': ";
        $answer = lc(<STDIN>);
        chomp $answer;
    }
    return ($answer eq 'y') ? 1 : 0;
}


my $output = shift @ARGV;
die "Please specify an output file.\n" if (not defined($output));

my $new = &read_file('/etc/pacman.d/mirrorlist');
my $orig = &read_file('/etc/pacman.d/mirrorlist.pacorig');

my @new_servers = map {/^Server\s*=\s*(.+)\s*$/} grep {/^[#\s]*Server/} split "\n",$new;
my @orig_servers = map {/^Server\s*=\s*(.+)\s*$/} grep {/^Server/} split "\n",$orig;
my %servers = ();

foreach my $server(@new_servers,@orig_servers)
{
    $servers{$server}++;
}

my $mirrorlist = '';
foreach my $server (@orig_servers)
{
    if ($servers{$server}>1)
    {
        $mirrorlist .= "Server = $server\n";
    }
    else
    {
        print "The following server is no longer in the mirror list:\n\t$server\n";
        if (&confirm("Would you like to remove it?", "y"))
        {
            next;
        }
        else
        {
            $mirrorlist .= "Server = $server\n";
        }
    }
}

$mirrorlist .= "\n\n";
$new =~ s/^Server/#Server/gm;
$mirrorlist .= $new;

&save_file($output,$mirrorlist);

Last edited by Xyne (2008-12-23 21:07:20)

Offline

#18 2008-12-23 20:41:08

EVRAMP
Member
From: Czech Republic
Registered: 2008-10-03
Posts: 173
Website

Re: Pacman update and mirrorlist

Perhaps all mirrors should be commented next time. It's easier to uncomment one, than comment all of them except one smile

Offline

#19 2008-12-24 00:21:26

unehed
Member
Registered: 2008-10-23
Posts: 10

Re: Pacman update and mirrorlist

Hey guys! I think I've had too much christmas beer cause I just did the mv mirrorlist.pacorig mirrorlist without checking the new file. Could anyone please post the new file here on the forums so I can use that one instead?

Merry Christmas, Archers!

Offline

#20 2008-12-24 00:41:27

Allan
Developer
From: Brisbane, AU
Registered: 2007-06-09
Posts: 10,409
Website

Re: Pacman update and mirrorlist

Just do a "pacman -S pacman-mirrorlist" and you will get the new version again (as a .pacnew)

Offline

#21 2008-12-27 06:45:52

jarav
Member
Registered: 2008-09-04
Posts: 63

Re: Pacman update and mirrorlist

Thanks for the script, Xyne.

Offline

#22 2008-12-27 12:20:34

fede
Member
Registered: 2007-06-29
Posts: 119

Re: Pacman update and mirrorlist

Thanks for the script!

Offline

Board footer

Powered by FluxBB