Pacman update and mirrorlist

Allan · 2008-12-22 03:20:08

With the upgrade to pacman-3.2.1-2, we have split the mirrorlist into the pacman-mirrorlist package. This will allow us to push more frequent updates to the mirrorlist, instead of only updating it with pacman releases. As part of this upgrade, your current /etc/pacman.d/mirrorlist file will be saved as mirrorlist.pacorig and the default mirrorlist put in its place. Remember to merge these files in order to keep using your mirror(s) of choice.

rson451 · 2008-12-22 03:21:45

Thanks for the heads up. I'm assuming this will go on the front page as well?

.:B:. · 2008-12-22 03:24:26

Nice . A few less MB to pull in every time .

Allan · 2008-12-22 03:29:19

rson451 wrote:

Thanks for the heads up. I'm assuming this will go on the front page as well?

Yes. I have posted it but it takes time for it to actually appear.

Xyne · 2008-12-22 03:38:02

[possible noob question]Is there a way to merge the files automatically or will this require a manual edit with each update?[/possible noob question]

Allan · 2008-12-22 04:02:42

Xyne wrote:

[possible noob question]Is there a way to merge the files automatically or will this require a manual edit with each update?[/possible noob question]

Just to clarify, it is only this update which will move the mirrorlist file to mirrorlist.pacorig and install the default as mirrorlist. From now on, new mirrorlists will be installed as mirrorlist.pacnew as usual. This is being caused by the initial splitting of the mirrorlist away from the pacman package.

If you are happy with your current mirrorlist you can use this poor man's automatic merging (not a recommendation!):

mv /etc/pacman.d/mirrorlist.paorig /etc/pacman.d/mirrorlist

Of course, you would miss out on any new mirrors or mirror removals....

sterling · 2008-12-22 04:48:43

Why .pacorig instead of the standard .pacsave?

Allan · 2008-12-22 04:59:07

sterling wrote:

Why .pacorig instead of the standard .pacsave?

The sort answer, because we are moving that file between packages.

The long answer.... well, time for a lesson in pacman internals! Pacman notes that is needs to update the pacman and pacman-mirrorlist packages. It does conflict checking which is smart enough to know that the mirrorlist file is being moved from pacman to pacman-mirror list. When it goes to install pacman-mirrorlist, the old pacman package still is installed and thus the old mirrorlist file is still on the system. Thus we have a file that pacman does not think is a conflict but is on the system. Normally pacman would just overwrite it but given it is in the backup array, it saves a copy to a .pacorig file. At least, that is my understanding of the code... I'm sure toofishes or shining can point out where I went wrong.

sterling · 2008-12-22 05:13:19

That doesn't differentiate between .pacsave and .pacorig for me unfortunately. The reason I'm concerned with this is I've recently expanded the .pac* article on the wiki and if .pacorig is something distinct then that should probably be covered as well:

http://wiki.archlinux.org/index.php/Pac … save_Files

Allan · 2008-12-22 05:21:18

This is one of the very few times, or possibly the only time, most people will encounter .pacorig files which is why I posted the news item. And the difference is mainly due to conflict handling as I tried to explain above. I would not be too concerned that it is missing from the wiki page, but if you do included it, make it very short.

Xyne · 2008-12-22 05:47:07

If I remember (and still think it would be useful), I'll write a merge script to grab the original mirrorlist, append new mirrors to it, and warn about mirrors that have been removed (and maybe remove them with confirmation maybe). It would probably work for locale.gen and other similar lists too.

sterling · 2008-12-22 10:09:02

Allan wrote:

sterling wrote:
Why .pacorig instead of the standard .pacsave?
The sort answer, because we are moving that file between packages.
The long answer.... well, time for a lesson in pacman internals! Pacman notes that is needs to update the pacman and pacman-mirrorlist packages. It does conflict checking which is smart enough to know that the mirrorlist file is being moved from pacman to pacman-mirror list. When it goes to install pacman-mirrorlist, the old pacman package still is installed and thus the old mirrorlist file is still on the system. Thus we have a file that pacman does not think is a conflict but is on the system. Normally pacman would just overwrite it but given it is in the backup array, it saves a copy to a .pacorig file. At least, that is my understanding of the code... I'm sure toofishes or shining can point out where I went wrong.

After reconsidering the subtlety of what's going on there, I think it would be better to omit any mention of .pacorig from the wiki and just leave it up to the News (and ML/forum announcements) to tell users what specific steps to take when a .pacorig file pops up.

rson451 · 2008-12-22 11:55:40

sterling wrote:

After reconsidering the subtlety of what's going on there, I think it would be better to omit any mention of .pacorig from the wiki and just leave it up to the News (and ML/forum announcements) to tell users what specific steps to take when a .pacorig file pops up.

I think that's the right decision. In the two years I've been around this is the first time I've seen this happen (that I can remember). Unique situations usually get a news post anyway, the wiki should be for general information.

sterling · 2008-12-23 06:02:57

Oops, I spoke too soon. phrakture added a stub section for .pacorig to the wiki article and someone else came along and filled it out. So I've tacked on a bit about always going to the News for instructions on dealing with .pacorig. Please proofread and correct any inaccuracies.

JonathanArcher · 2008-12-23 17:52:46

So, if I understand it correctly, after I checked the new mirrolist file for the mirrors I can savely delete the mirrorlist.pacorig because it will not be needed anymore?

R00KIE · 2008-12-23 18:06:55

The news did the trick for me, manually commented all mirrors that I didn't want and left the only listed Portuguese mirror and added two more and as the last one the arch mirror. It looks like this

# My Servers
Server = ftp://cesium.di.uminho.pt/pub/archlinux/$repo/os/i686
Server = ftp://darkstar.ist.utl.pt/pub/archlinux/$repo/os/i686
Server = ftp://ftp.nux.ipb.pt/pub/dists/archlinux/$repo/os/i686
Server = ftp://ftp.archlinux.org/$repo/os/i686

Xyne · 2008-12-23 20:27:27

This is somewhat less generic than what I had in mind because I realized that the order of the mirrors need to be preserved. Anyway, the script does the following:

read in uncommented mirrors from mirrorlist.pacorig
check them against the new mirrorlist
for each one that's not in the new list, ask if the user would like to keep it
create a new file with the previously selected mirrors appended to the top followed by the complete new mirrorlist with the server lines commented out

usage:

merge_pacorig_mirrors.pl /path/to/output_file

Check the output file then move it to /etc/pacman.d/mirrorlist if you're happy with it. The script will not overwrite your mirrorlists (unless you specify them as the output file, but I do not recommend that).

#!/usr/bin/perl
use strict;
use warnings;

sub read_file
{
    my $file = shift;
    my $text;
    open(my $fh,'<',$file) or die "unable to open $file: $!\n";
    {
        local $/;
        $text = <$fh>;
    }
    close $fh;
    return $text;
}
sub save_file
{
    my ($file,$text) = @_;
    open(my $fh,'>',$file) or die "unable to open $file: $!\n";
    print $fh $text;
    close $fh;
}

sub confirm
{
    my ($question,$default_answer) = @_;
    print "$question ";
    if (defined($default_answer) and $default_answer eq 'y')
    {
        print '[Y/n] ';
    }
    elsif (defined($default_answer) and $default_answer eq 'n')
    {
        print '[y/N] ';
    }
    else
    {
        print '[y/n] ';
    }
    my $answer = lc(<STDIN>);
    chomp $answer;
    $answer =  $default_answer if (defined($default_answer) and $answer eq '');
    while ($answer ne 'y' and $answer ne 'n')
    {
        print "Please enter 'y' or 'n': ";
        $answer = lc(<STDIN>);
        chomp $answer;
    }
    return ($answer eq 'y') ? 1 : 0;
}


my $output = shift @ARGV;
die "Please specify an output file.\n" if (not defined($output));

my $new = &read_file('/etc/pacman.d/mirrorlist');
my $orig = &read_file('/etc/pacman.d/mirrorlist.pacorig');

my @new_servers = map {/^Server\s*=\s*(.+)\s*$/} grep {/^[#\s]*Server/} split "\n",$new;
my @orig_servers = map {/^Server\s*=\s*(.+)\s*$/} grep {/^Server/} split "\n",$orig;
my %servers = ();

foreach my $server(@new_servers,@orig_servers)
{
    $servers{$server}++;
}

my $mirrorlist = '';
foreach my $server (@orig_servers)
{
    if ($servers{$server}>1)
    {
        $mirrorlist .= "Server = $server\n";
    }
    else
    {
        print "The following server is no longer in the mirror list:\n\t$server\n";
        if (&confirm("Would you like to remove it?", "y"))
        {
            next;
        }
        else
        {
            $mirrorlist .= "Server = $server\n";
        }
    }
}

$mirrorlist .= "\n\n";
$new =~ s/^Server/#Server/gm;
$mirrorlist .= $new;

&save_file($output,$mirrorlist);

Last edited by Xyne (2008-12-23 21:07:20)

EVRAMP · 2008-12-23 20:41:08

Perhaps all mirrors should be commented next time. It's easier to uncomment one, than comment all of them except one

unehed · 2008-12-24 00:21:26

Hey guys! I think I've had too much christmas beer cause I just did the mv mirrorlist.pacorig mirrorlist without checking the new file. Could anyone please post the new file here on the forums so I can use that one instead?

Merry Christmas, Archers!

Allan · 2008-12-24 00:41:27

Just do a "pacman -S pacman-mirrorlist" and you will get the new version again (as a .pacnew)

jarav · 2008-12-27 06:45:52

Thanks for the script, Xyne.

fede · 2008-12-27 12:20:34

Thanks for the script!

Arch Linux

#1 2008-12-22 03:20:08

Pacman update and mirrorlist

#2 2008-12-22 03:21:45

Re: Pacman update and mirrorlist

#3 2008-12-22 03:24:26

Re: Pacman update and mirrorlist

#4 2008-12-22 03:29:19

Re: Pacman update and mirrorlist

#5 2008-12-22 03:38:02

Re: Pacman update and mirrorlist

#6 2008-12-22 04:02:42

Re: Pacman update and mirrorlist

#7 2008-12-22 04:48:43

Re: Pacman update and mirrorlist

#8 2008-12-22 04:59:07

Re: Pacman update and mirrorlist

#9 2008-12-22 05:13:19

Re: Pacman update and mirrorlist

#10 2008-12-22 05:21:18

Re: Pacman update and mirrorlist

#11 2008-12-22 05:47:07

Re: Pacman update and mirrorlist

#12 2008-12-22 10:09:02

Re: Pacman update and mirrorlist

#13 2008-12-22 11:55:40

Re: Pacman update and mirrorlist

#14 2008-12-23 06:02:57

Re: Pacman update and mirrorlist

#15 2008-12-23 17:52:46

Re: Pacman update and mirrorlist

#16 2008-12-23 18:06:55

Re: Pacman update and mirrorlist

#17 2008-12-23 20:27:27

Re: Pacman update and mirrorlist

#18 2008-12-23 20:41:08

Re: Pacman update and mirrorlist

#19 2008-12-24 00:21:26

Re: Pacman update and mirrorlist

#20 2008-12-24 00:41:27

Re: Pacman update and mirrorlist

#21 2008-12-27 06:45:52

Re: Pacman update and mirrorlist

#22 2008-12-27 12:20:34

Re: Pacman update and mirrorlist

Board footer