[SOLVED] Correct way of forcing a manual file download

Malvineous · 2014-10-15 04:53:54

Hi all,

I have just switched to mkaurball for one of the AUR packages I maintain and I'm getting a warning message. I am wondering whether I should report this as a bug in mkaurball, or whether I am going about things the wrong way.

Basically one of the files in the package can't be downloaded automatically as you have to agree to a specific licence, so I can't include the URL in the PKGBUILD as you can only download it through a web browser thanks to a convoluted login procedure.

However the only "best practice" I can find ends up including the file as if it's one of the support files that belong with the PKGBUILD itself, which means when you create the AUR tarball you end up including the ~100MB file that's supposed to be manually downloaded.

I worked around this (PKGBUILD here) by using the $SOURCEONLY variable and only adding the hashes and filenames of the manually downloaded files if a binary package was being built, and omitting them if a source-only package was being built.

This works, but now mkaurball throws up an error because it doesn't define the (I think undocumented) $SOURCEONLY variable.

So before I report this as a bug (or at least a limitation) in mkaurball, can anyone let me know whether there is a better way of requiring a manual download of a file, such that it won't get bundled up in the AUR tarball?

Last edited by Malvineous (2014-10-15 13:10:57)

jasonwryan · 2014-10-15 05:05:09

Isn't this the same sort of thing as MS fonts? https://aur.archlinux.org/packages/tt/t … 8/PKGBUILD

Malvineous · 2014-10-15 12:05:11

Wow that looks pretty scary! Although looking more closely, it appears to put all the files in the source=() array, which means that sadly, should you try to run "makepkg -S" on it, you'll get all your manually downloaded files bundled up in the tarball which means you then can't distribute the tarball.

So unfortunately no, this method is the same as the "best practice" one in my original post, which means it won't work with AUR.

If you want to check yourself, download the AUR tarball, extract it, then try to recreate it:

$ makepkg -S --skipinteg
==> Making package: ttf-ms-win8 6.3.9600.17238-1 (Wed Oct 15 22:01:43 AEST 2014)
==> WARNING: Skipping all source file integrity checks.
==> Entering fakeroot environment...
==> Creating source package...
  -> Adding PKGBUILD...
  -> Adding arial.ttf...
  -> Adding arialbd.ttf...
  -> Adding ariali.ttf...
  -> Adding arialbi.ttf...

You can see that it includes the .ttf files into the AUR tarball, which you don't want because you aren't allowed to distribute those files. I'm not sure how they generated the tarball for AUR for this package, but I'm guessing they removed the .ttf files manually and then recreated the .tar.gz.

Malvineous · 2014-10-15 12:30:30

And with that I think I worked out how to do it.

It looks like you invent a fake URL scheme, such as "manual://filename". Making an AUR tarball will then exclude this file from the .tar.gz because it looks like a URL. Making a package normally with the file present will just use the local copy because it doesn't need to be downloaded, so that works fine.

Then you just have to add a handler to actually do something should makepkg attempt to download this fake URL, such as:

DLAGENTS+=('manual::/usr/bin/echo You must download these files manually; exit 1;')

or in my case, I changed this to print detailed instructions every time and just fail the download:

msg "Warning: This software cannot be downloaded automatically."
plain "You will need to obtain the software by doing blah blah etc."

DLAGENTS+=('manual::/usr/bin/false')

This works perfectly!

daniel_shub · 2014-10-30 13:16:51

I use file://filename instead of manual://filename and makepkg and mkaurball seem to handle it correctly. When I tried to define my own download agent, I could never really figure out where to put: DLAGENTS+=('manual::/usr/bin/false'). The only place I got it to work was by adding it to /etc/makepkg.conf. Where do you put the DLAGENTS line?

Last edited by daniel_shub (2014-10-30 13:17:10)

Malvineous · 2014-10-30 16:27:57

I'm not sure that it really matters as long as it's outside the function declarations. I just put it before I included any manual:// URLs because it just seemed nice that way.

I hadn't thought of using the file:// scheme. I think the manual:// scheme seems to be used for situations where instead of calling /usr/bin/false, you can call "echo" and print instructions that way, so they'll only show up if the file hasn't been downloaded yet. I'm guessing you can't do that with file:// although it would make no difference in my script/example because the message gets printed every time anyway.

I would rather only print the message when the file doesn't exist, but I can't see a way of calling makepkg's "msg" function from within a DLAGENT handler.

Arch Linux

#1 2014-10-15 04:53:54

[SOLVED] Correct way of forcing a manual file download

#2 2014-10-15 05:05:09

Re: [SOLVED] Correct way of forcing a manual file download

#3 2014-10-15 12:05:11

Re: [SOLVED] Correct way of forcing a manual file download

#4 2014-10-15 12:30:30

Re: [SOLVED] Correct way of forcing a manual file download

#5 2014-10-30 13:16:51

Re: [SOLVED] Correct way of forcing a manual file download

#6 2014-10-30 16:27:57

Re: [SOLVED] Correct way of forcing a manual file download

Board footer