Arch Linux iso and img metalinks

Xyne · 2009-04-24 22:00:42

I've finally created a script to automatically generate metalinks for the current iso and img downloads using the most recent official mirrorlist
The generated metalinks are currently listed here: http://xyne.archlinux.ca/arch

I might move the links page around on the site but the metalink directory should not change. If any of the site admins would like to include these on the downloads page, send me an email so we can work out the optimal way of doing it (e.g. I could create a custom page that you could scrape and show you how to update the metalinks via the web interface when mirrorlists/downloads change).

For anyone not familiar with metalinks, here's an example of how to use one with aria2c:

aria2c --follow-metalink=mem -j45 -C45 http://xyne.archlinux.ca/metalinks/archlinux-2009.02-ftp-x86_64.iso.metalink

aria2c should be capable of downloading from the metalink and the torrent simultaneously but I gave up trying to get it to work after a couple of minutes.

The Firefox add-on "DownThemAll!" can also handle metalinks.

archlinux-*-all.metalink contains all of the files. Don't use this link unless you either:

a) really need all of the files
b) know how to select files from a metalink

Last edited by Xyne (2009-04-24 22:05:25)

MoonSwan · 2009-04-24 23:50:32

I'm dense (you knew that already but I love to state the obvious) . Whats the advantage of this over a regular torrent exactly?

Last edited by MoonSwan (2009-04-24 23:51:08)

smartboyathome · 2009-04-25 00:33:50

MoonSwan wrote:

I'm dense (you knew that already but I love to state the obvious) . Whats the advantage of this over a regular torrent exactly?

It allows you to download the ISO from different mirrors (instead of different peers). This means it balances the load out more and can speed up the download in the process.

Xyne · 2009-04-25 01:07:54

smartboyathome wrote:

It allows you to download the ISO from different mirrors (instead of different peers). This means it balances the load out more and can speed up the download in the process.

Also, because the metalink points to mirrors instead of peers, the highest attainable download rate is relatively stable (the mirrors are normally always online while peers vary).

That written, there's no real advantage over torrents per se (both do the same thing... segmented downloading from different hosts simultaneously). The main use would be for people who can't use the torrent (firewall restrictions, oppressive ISP that throttles p2p traffic, etc) and who would normally have to use a single mirror.

The ideal would be if aria2c and other download managers could pull from both the metalink and the torrent at the same time (even though the metalink will probably max out most users' connections).

Dieter@be · 2009-04-25 06:19:41

our torrents are webseed enabled. this means (if your torrent client supports it) it will pull from torrent peers *and* webmirrors.

Xyne · 2009-04-25 06:30:00

ah

well, nvm then

I need to test it but I remember the torrent being slower when I downloaded an iso a few weeks ago... it could just be my settings though.

Last edited by Xyne (2009-04-25 06:30:22)

twanj · 2009-06-23 00:20:10

Xyne wrote:

aria2c should be capable of downloading from the metalink and the torrent simultaneously but I gave up trying to get it to work after a couple of minutes.
The Firefox add-on "DownThemAll!" can also handle metalinks.

Xyne, thanks for adding these!

aria2 should work well w/ mirrors + torrents. if it doesn't, please report it on the aria2 sourceforge bugtracker.

speaking of DTA, anyone that knows javascript willing to add chunk checksumming support? that's the one thing missing, we want it to be able to automatically repair bad downloads like other metalink clients.

Xyne · 2009-07-04 00:46:16

@Dieter@be
The post about metalinks on pacman-dev reminded me about this thread. The advantage of the metalinks over torrents, even webseeded torrents, is that you don't need a torrent client to download them. You can use DownThemAll (and maybe other add-ons) to download them directly in Firefox, for example.

twanj · 2009-07-04 04:44:35

Xyne wrote:

@Dieter@be
The post about metalinks on pacman-dev reminded me about this thread. The advantage of the metalinks over torrents, even webseeded torrents, is that you don't need a torrent client to download them. You can use DownThemAll (and maybe other add-ons) to download them directly in Firefox, for example.

exactly, most download apps (FTP clients, non-torrent P2P clients, most download managers, and browsers, with the exception of Opera) don't support torrents directly. and there are many situations where P2P uploading is banned or frowned upon, like corporate/university networks, places with slow uplink, misconfigured hardware, etc...

torrents are awesome, but they're not a perfect fit in every situation. metalinks give you alternates, failover options, P2P-like features - so if there's any way for a download to complete, it will. there's the slightly added complexity & effort of getting em in place, but once that's automated it's usually pretty helpful.

here's a good description of the situation:

http://www.techradar.com/news/computing … ads-611025

Dieter@be · 2009-07-04 10:13:35

Xyne wrote:

@Dieter@be
The post about metalinks on pacman-dev reminded me about this thread. The advantage of the metalinks over torrents, even webseeded torrents, is that you don't need a torrent client to download them. You can use DownThemAll (and maybe other add-ons) to download them directly in Firefox, for example.

I find this a rather awkward comparison. Of course you need torrent support if you want to download torrents (be it in your browser on with an external program). How are metalinks different? I don't think I'll be able to use metalinks without installing a tool that supports them.

Xyne · 2009-07-04 11:32:55

Some people already have what they need to download torrents and some already have what they need to download metalinks. I think DownThemAll and Flashgot are far more common in Firefox than any of the torrent add-ons but I could be wrong.

Anyway, they are just an additional way to download the latest releases. Some people will find them useful while others won't. I'm not advocating that they should be the preferred download method. I've posted them online for those who wish to use them and they only complement the official download methods.

Just because some people won't use them, it doesn't mean they're useless.

twanj · 2009-07-04 17:19:31

Dieter@be wrote:

Xyne wrote:
@Dieter@be
The post about metalinks on pacman-dev reminded me about this thread. The advantage of the metalinks over torrents, even webseeded torrents, is that you don't need a torrent client to download them. You can use DownThemAll (and maybe other add-ons) to download them directly in Firefox, for example.
I find this a rather awkward comparison. Of course you need torrent support if you want to download torrents (be it in your browser on with an external program). How are metalinks different? I don't think I'll be able to use metalinks without installing a tool that supports them.

of course. like I was saying tho (this is more forward looking), will basic tools like wget eventually have metalink support? yes. will they ever support torrents? no...

are features like error correction, failover, etc good things we want more widespread? sure.

Arch Linux

#1 2009-04-24 22:00:42

Arch Linux iso and img metalinks

#2 2009-04-24 23:50:32

Re: Arch Linux iso and img metalinks

#3 2009-04-25 00:33:50

Re: Arch Linux iso and img metalinks

#4 2009-04-25 01:07:54

Re: Arch Linux iso and img metalinks

#5 2009-04-25 06:19:41

Re: Arch Linux iso and img metalinks

#6 2009-04-25 06:30:00

Re: Arch Linux iso and img metalinks

#7 2009-06-23 00:20:10

Re: Arch Linux iso and img metalinks

#8 2009-07-04 00:46:16

Re: Arch Linux iso and img metalinks

#9 2009-07-04 04:44:35

Re: Arch Linux iso and img metalinks

#10 2009-07-04 10:13:35

Re: Arch Linux iso and img metalinks

#11 2009-07-04 11:32:55

Re: Arch Linux iso and img metalinks

#12 2009-07-04 17:19:31

Re: Arch Linux iso and img metalinks

Board footer