source() for sourceforge ... a kind of feature-request ...

dp · 2003-08-30 23:05:30

while building some packages and rebuilding existing ones and trying to make new ones i realized something that can be enhanced in this process:

e.g. you want to build kguitar as a package:

you can find the project page at sourceforge
http://sourceforge.net/projects/kguitar/

and the download-url is ... hmm sourceforge has a lot of mirrors, but not all have all things always available ... this is why sometimes a package in incoming doesnt want to build imediately (you must change the sourceforge-mirror in PKGBUILD)

ok, let's go by hand to sourceforge and click the file you want ... you get this url:

http://prdownloads.sourceforge.net/kgui … 2?download

and exactly this is the url for the source() in the PKGBUILD in general --- you must only change instead of "prdownloads" you must put a mirror-prefix like "switch.dl" (and remove the "download" variable from the url, that's clear)

MY IDEA:

what about having a variable for all packages from sourceforge that generates a prefix of a mirror for sourceforge, so that you can say simply in PKGBUILD

source(http://$sfmirror/$pkgname/...)

and if the download fails with this mirror, makepkg can have a fallback to change $sfmirror to the next mirror

the mirrors i found out till now are:

twtelecom.dl.sourceforge.net
cesnet.dl.sourceforge.net
heanet.dl.sourceforge.net
keihanna.dl.sourceforge.net
unc.dl.sourceforge.net
aleron.dl.sourceforge.net
umn.dl.sourceforge.net
belnet.dl.sourceforge.net
switch.dl.sourceforge.net

-> this would help building packages from sourceforge

sarah31 · 2003-08-31 05:32:16

i use umn.dl.sourceforge almost exclusively i have never had issues with it at all unc.dl.sourceforge no longer seems to work for me at all. ymmv.

all that being siad that may not be a bad idea though it seems to be a bit of a kludge to the system.

andy · 2003-08-31 12:15:03

I checked a few things. First, I checked where FreeBSD grabs their sourceforge packages. Because they do something similar as you suggested. They use the following sites :

http://us.dl.sourceforge.net/
http://eu.dl.sourceforge.net/
ftp://ftp.kddlabs.co.jp/sourceforge/
ftp://ftp.chg.ru/pub/sourceforge/

Now, I checked some name lookups. The site us.dl.sourceforge.net actually resolves to unc umn and another one in the US via DNS load balancing. Similarly for the site eu.dl.sourceforge.net. The site in Japan is also listed on sourceforge itself - at least for the package I checked. I don't know why the other US and European sites don't show up in the load balancing DNS look up. And if there are other similar schemes for the rest of the world.

However,

prdownloads.sourceforge.net
downloads.sourceforge.net

seem to always resolve to always the same two hosts that are in the same subnet as the rest of sourceforge. So, going there should always get you the latest and up to date package, but access speed might be slow. In one package I did I used http://downloads.sourceforge.net/$pkgname

mgushee · 2004-12-09 00:21:33

Another approach is to use a HOST variable like this:

HOST=umn.dl   # default value
source=(http://$HOST.sourceforge.net/foo/$pkgname-$pkgver.tar.gz)

Then if the download fails, you can override it on the command line:

HOST=unc.dl  makepkg

Not as automatic as one might like, but it should work well.

dp · 2004-12-09 00:34:58

mgushee wrote:

Another approach is to use a HOST variable like this:
HOST=umn.dl   # default value
source=(http://$HOST.sourceforge.net/foo/$pkgname-$pkgver.tar.gz)
Then if the download fails, you can override it on the command line:
HOST=unc.dl  makepkg
Not as automatic as one might like, but it should work well.

the "dl" should be in the source(), because it can then work also without HOST sometimes

sarah31 · 2004-12-09 01:03:44

good lord i told jason over half a year ago how to have one universal entry in a PKGBUILD for a sourceforge source()

mgushee · 2004-12-09 01:35:36

Re: sarah31's post:

You mean umn.dl.sourceforge.net?

· Just because it has always worked in the past, you can't assume it will always work in the future. The University of Minnesota could decide, tomorrow, for their own political reasons, to cut off funding for Linux archives. Or they could move the stuff to a different host without warning anyone. I've seen it happen many times.
· For me, umn does not always work. Maybe I download weird software ...

If you meant something else, could you please explain (or if your solution is documented somewhere, feel free to just point to the URL)?

dp · 2004-12-09 02:28:20

umn.dl.sourceforge.net does not work for me for about 50% the time (no idea why ... i get timeouts and waiting for more than 2 minutes) ... belnet works best and switch is the fastest (because around the corner) but switch does not mirror everything (no idea why), and for some sources i get 404's *bah*

so i still believe, that a centralized handling and multi-server-download (at the same time ... so that it is faster, as more open connections are used) seems cool, but as this subject is now open for more than a year and has no solution, it does not hurry at all ;-)

sarah31 · 2004-12-09 02:32:40

mgushee wrote:

Re: sarah31's post:
You mean umn.dl.sourceforge.net?
· Just because it has always worked in the past, you can't assume it will always work in the future. The University of Minnesota could decide, tomorrow, for their own political reasons, to cut off funding for Linux archives. Or they could move the stuff to a different host without warning anyone. I've seen it happen many times.

uh no look at the date of that post. at that time i was still maintaining packages for arch. four months later i was not using arch at all. somewhere between january this year and now i told Xentac/Jason how it was handled with that other distro i used after arch. i suppose he mever bothered to pass it on. .....

· For me, umn does not always work. Maybe I download weird software ...
If you meant something else, could you please explain (or if your solution is documented somewhere, feel free to just point to the URL)?

so why should i bother now? if you search my posts from earlier this year you will know what distro i used before coming back to arch. ..... but i will give you a hint .... your "gift" is in the "CLC".

mgushee · 2004-12-10 21:32:49

sarah31 wrote:

so why should i bother now? if you search my posts from earlier this year you will know what distro i used before coming back to arch. ..... but i will give you a hint .... your "gift" is in the "CLC".

You may be a Zen master, but I'm not your disciple

Dunno what CLC stands for, and it probably won't kill me to never find out. But I did find out the answer to the SourceForge URL problem ... and posted it to the Wiki, where it should have been a long time ago:

http://wiki2.archlinux.org/index.php/SourceForge%20URLs

(also linked from the "Packages & makepkg" page)

So there.

sarah31 · 2004-12-10 23:40:44

if you had done a search you would have found out that i had used crux for a spell.

CLC is a crux repo management system.

and in the CLC there is a build for gift which would have shown you the syntax that you discovered anyway.

welcome to my ignore list.

Dusty · 2004-12-11 04:33:47

sarah31 wrote:

welcome to my ignore list.

Oh goodie, somebody to keep me company. 8)

Xentac · 2004-12-12 06:12:16

sarah31 wrote:

uh no look at the date of that post. at that time i was still maintaining packages for arch. four months later i was not using arch at all. somewhere between january this year and now i told Xentac/Jason how it was handled with that other distro i used after arch. i suppose he mever bothered to pass it on. .....

I did bother to pass it on. I'm pretty sure that every PKGBUILD in current and extra uses dl.sourceforge.net now.

Hell, I passed it on so much that Ben wrote a namcap rule to tell you when you were using a specific mirror instead of the right one!

Don't go blaming this on me.

mgushee · 2004-12-12 21:22:18

No, not all of them. I grepped my /var/abs tree for 'sourceforge' the other day. Probably 2/3 of the hits used dl.sourceforge.net. Among the rest, I saw a lot of 'telia' and 'belnet', with a few packages using <project_name>.sourceforge.net ...

Not trying to finger-point here. It may be that everybody's doing it right now, but there are just a lot of older packages that haven't been fixed yet.

dp · 2004-12-13 12:24:03

mgushee wrote:

...with a few packages using <project_name>.sourceforge.net ...

yea, that's the URL of the pkg that is this way

Arch Linux

#1 2003-08-30 23:05:30

source() for sourceforge ... a kind of feature-request ...

#2 2003-08-31 05:32:16

Re: source() for sourceforge ... a kind of feature-request ...

#3 2003-08-31 12:15:03

Re: source() for sourceforge ... a kind of feature-request ...

#4 2004-12-09 00:21:33

Re: source() for sourceforge ... a kind of feature-request ...

#5 2004-12-09 00:34:58

Re: source() for sourceforge ... a kind of feature-request ...

#6 2004-12-09 01:03:44

Re: source() for sourceforge ... a kind of feature-request ...

#7 2004-12-09 01:35:36

Re: source() for sourceforge ... a kind of feature-request ...

#8 2004-12-09 02:28:20

Re: source() for sourceforge ... a kind of feature-request ...

#9 2004-12-09 02:32:40

Re: source() for sourceforge ... a kind of feature-request ...

#10 2004-12-10 21:32:49

Re: source() for sourceforge ... a kind of feature-request ...

#11 2004-12-10 23:40:44

Re: source() for sourceforge ... a kind of feature-request ...

#12 2004-12-11 04:33:47

Re: source() for sourceforge ... a kind of feature-request ...

#13 2004-12-12 06:12:16

Re: source() for sourceforge ... a kind of feature-request ...

#14 2004-12-12 21:22:18

Re: source() for sourceforge ... a kind of feature-request ...

#15 2004-12-13 12:24:03

Re: source() for sourceforge ... a kind of feature-request ...

Board footer