You are not logged in.

#1 2016-02-03 15:28:55

Gingko
Member
Registered: 2015-07-04
Posts: 17

Creating package from local files

Hello.

I am trying to create a package made from a collection of local files.

I mean, all files that I want to put in my package are static files (there is also at least one symbolic link that I want to install as a symbolic link), they exists somewhere else in the same filesystem, they are not URLs, they are not in GIT, HG or SVN, they are not in a tarball, they are not compressed, they are just files that I want to put in the package as they are.

My problem is that I don't know what I have to put into the source variable of my PKGBUILD file :

  • If I put the full path of the given files, makepkg claims that it cannot find these files (looking like if it was stripping the directory part and trying to read the file from the local directory).

  • If I put a symbolic link to the file, I get the same result after dereferencing. Actually this seems to work only for tarball files.

  • If I put nothing, leaving the 'sources' variable empty (and expecting getting the files through the package function), I get the following error: "==> ERROR: Integrity checks (md5) differ in size from the source array."

  • I tried also to put the name of a local symlink to an entire directory without better result.

In fact, the actual use of the source variable is very obscure to me.

Maybe I could put my files into a tarball before building my package, but this looks like a useless step to me.

What should I put inside this source variable in order to get the files that I want to use?

Regards,

Gingko

Offline

#2 2016-02-03 15:41:07

ayekat
Member
Registered: 2011-01-17
Posts: 1,589

Re: Creating package from local files

What are you trying to do?

PKGBUILD files expect files to either be located in a remote location (HTTP, git, ...) or in the current directory, shipped together with the PKGBUILD in a tarball.
If you want to put those file into the package, you have to put them at least in the same directory as the PKGBUILD.

Gingko wrote:

"==> ERROR: Integrity checks (md5) differ in size from the source array."

This means that your `md5sums` array holds a different number of entries than your `source` array.
In theory you should be able to copy the files in in the `package()` function - but beware that this is not portable, and you should not distribute that PKGBUILD.


pkgshackscfgblag

Offline

#3 2016-02-03 16:00:59

Gingko
Member
Registered: 2015-07-04
Posts: 17

Re: Creating package from local files

ayekat wrote:

What are you trying to do?

PKGBUILD files expect files to either be located in a remote location (HTTP, git, ...) or in the current directory, shipped together with the PKGBUILD in a tarball.
If you want to put those file into the package, you have to put them at least in the same directory as the PKGBUILD.

Why can't I use symlinks instead?

ayekat wrote:
Gingko wrote:

"==> ERROR: Integrity checks (md5) differ in size from the source array."

This means that your `md5sums` array holds a different number of entries than your `source` array.
In theory you should be able to copy the files in in the `package()` function - but beware that this is not portable, and you should not distribute that PKGBUILD.

I don't use md5sums so I put 'SKIP' in this variable. But this doesn't seem to be accepted for an empty list.

But actually, as you point it to me, it seems that if I leave md5sums empty also, the error disappears.

Gingko

Offline

#4 2016-02-03 16:22:47

ayekat
Member
Registered: 2011-01-17
Posts: 1,589

Re: Creating package from local files

Gingko wrote:

Why can't I use symlinks instead?

You should be able to. I just successfully created a package by putting a symlink pointing outside into the local directory, adding it to the source array (and SKIP to the md5sums array), installing it in the package() function, and running `makepkg`.

I'm running pacman<=5.0, though, so I can't tell you if that behaviour has changed (but I doubt it). Could you paste your PKGBUILD, please?


pkgshackscfgblag

Offline

#5 2016-02-04 13:49:38

Gingko
Member
Registered: 2015-07-04
Posts: 17

Re: Creating package from local files

ayekat wrote:

Could you paste your PKGBUILD, please?

I can, but right now it is just a tiny skeleton:

# Maintainer: Myself <myself@thisisnotreallymyemail.com>
pkgname=mywebapp_demo
pkgver=1.0
pkgrel=1
epoch=
pkgdesc="Demonstrations files for my application"
arch=('x86_64')
url="http://www.thisisnotreallymywebsite.com/"
license=('custom')
groups=()
depends=(
	mywebapp
	)
makedepends=()
checkdepends=()
optdepends=()
provides=()
conflicts=()
replaces=()
backup=()
options=()
install=$pkgname.install
changelog=
source=(
	demo
	)
noextract=()
md5sums=(SKIP)
validpgpkeys=()

package() {
	mkdir -p $pkgdir/srv/http
	rsync -rlpt --delete $srcdir/demo $pkgdir/srv/http/demo/
	return 0
}

Actually I have somewhere a directory named demo, containing the following files:

  • demo/

    • demo1.html

    • demo1.swf

    • index.html       (=> which is actually a relative symlink to demo1.html)

After installation, I want to have it as /srv/http/demo.

So my first try was to create a symlink (named demo) to that directory in the 'PKGBUILD' directory, expecting makepkg to copy it as src/demo, after what I would have used the function package for placing it into the proper subdirectory under pkg.
But if I do that, I get the following message: "==> ERROR: demo was not found in the build directory and is not a URL."

I tried also to make symlinks to all three files in the 'PKGBUILD' directory, and list all of them in the source variable. If I do that, I just get a copy of my three symlinks into the src directory, so I still need to process them individually in the package function. Especially knowing that it is not easy to differenciate between a symlink to a file that should be included as a file and a symlink to a symlink that should be included as a symlink. smile

Likely I will end by building all my package with the package function, taking the files directly from where they are, rather than taking the files from the src directory. But if it appears better to do that, I am wondering why the source variable and the src directory do even exist....

Regards,

Gingko

Offline

#6 2016-02-04 14:51:52

ayekat
Member
Registered: 2011-01-17
Posts: 1,589

Re: Creating package from local files

Gingko wrote:

So my first try was to create a symlink (named demo) to that directory in the 'PKGBUILD' directory, expecting makepkg to copy it as src/demo, after what I would have used the function package for placing it into the proper subdirectory under pkg.
But if I do that, I get the following message: "==> ERROR: demo was not found in the build directory and is not a URL."

The source array expects regular files, not directories. So if the symlink resolves to a directory (or you just put a directory there directly), makepkg will complain - although admittedly it could give a more informative error message.

I tried also to make symlinks to all three files in the 'PKGBUILD' directory, and list all of them in the source variable. If I do that, I just get a copy of my three symlinks into the src directory, so I still need to process them individually in the package function.

Yes, because the source array specifies sources, which don't necessarily get also installed (actually they hardly ever do). That's why you need the second step, after the source is compiled to the final files, to install those. Of course this won't apply in your situation, so it might seem useless, but if you think of a normal package, it makes sense.

Especially knowing that it is not easy to differenciate between a symlink to a file that should be included as a file and a symlink to a symlink that should be included as a symlink. smile

I see - although... you can directly copy the files in the package() function, so there is no need to symlink them, rendering this a non-issue. smile

INTERMISSION
You seem to be working on a website that you want to distribute to multiple machines - that's all I know so far. Why do you want to use the Arch Linux packaging system for that? How do you intend to distribute the package across your machines? If you host it on that one, "packager" machine, then couldn't you simply rsync it to/from the other machines? Or use git? Latter would also allow you to use a remote storage location.

-- edit --   the last few questions may seem a little ignorant, but you haven't quite stated your situation and why you chose to use the Arch Linux packaging system over other (distribution-agnostic) methods that are perhaps more adapted for distributing data (not programs/libraries).

Last edited by ayekat (2016-02-04 14:55:36)


pkgshackscfgblag

Offline

#7 2016-02-08 09:06:28

Gingko
Member
Registered: 2015-07-04
Posts: 17

Re: Creating package from local files

Hello, and sorry for the delay… I'm not working on this project full time.

ayekat wrote:
Gingko wrote:

So my first try was to create a symlink (named demo) to that directory in the 'PKGBUILD' directory, expecting makepkg to copy it as src/demo, after what I would have used the function package for placing it into the proper subdirectory under pkg.
But if I do that, I get the following message: "==> ERROR: demo was not found in the build directory and is not a URL."

The source array expects regular files, not directories. So if the symlink resolves to a directory (or you just put a directory there directly), makepkg will complain - although admittedly it could give a more informative error message.

I see.

Effectively it could give a more informative error message.
It could also accept giving a directory as source, maybe this could make sense in some situations.
If ever it happens you want to include that as a feature request… smile

ayekat wrote:

INTERMISSION
You seem to be working on a website that you want to distribute to multiple machines - that's all I know so far. Why do you want to use the Arch Linux packaging system for that? How do you intend to distribute the package across your machines? If you host it on that one, "packager" machine, then couldn't you simply rsync it to/from the other machines? Or use git? Latter would also allow you to use a remote storage location.

-- edit --   the last few questions may seem a little ignorant, but you haven't quite stated your situation and why you chose to use the Arch Linux packaging system over other (distribution-agnostic) methods that are perhaps more adapted for distributing data (not programs/libraries).

Actually I am working on embedded display devices, which can display web pages (normally external, but local contents can also eventually be displayed) full screen, on one or several display panels located in some public or private spaces.

These devices can be administrated by TCP/IP, either by SSH or by webpages, normally across some Intranet network.

So the website is mostly the internal administration interface, and a xulrunner application is used for displaying web pages to users on the display panels.

These devices are also expected to be periodically auto-upgradable, meaning new releases can be made available on the web by us, and the devices can upgrade themselves, on user request and possibly later automatically, by downloading new contents from our servers accessible from the web.

So the devices have users, but these users are not expected to get inside the system (although if they really want to…).

This is effectively working now, but the installation and the upgrades are made mostly by shell scripts, and I want to make this more robust by switching to a better use of the pacman system, and thus I am trying to create packages for installing what was previously installed using bash scripts.

For that purpose, I know that I will have problems with existing files when switching from the old to the new system, as pacman doesn't seem to accept overwriting existing files, but I think that I have a solution to that: I will create another, fake and empty packet, consisting of only its install script, and included as a dependency of the main one, for the only purpose of cleaning the previous installation before upgrading.

I choose the ArchLinux system for the following reasons:
When I was asked to create the embedded device, I had very little time for doing that (sometime employers tends to think that as soon as I don't write the code myself, everything should be expected to be made almost instantly smile ).
About Linux I was used mostly with Debian distributions, but these distributions are mostly made for creating either workstations or servers, thus installing many things that I don't need by default, and not installing many things that I need.
On the other hands, ArchLinux begins by installing a very minimal system, and if I want more, I have to manually add what I want. This is much more what I expected.
Particularly, I wanted to have a X display system, but I very NOT wanted to have a graphical Desktop!!!!
Also, ArchLinux includes xulrunner among its packages, this is not necessarily the case of all other distros.

For the sake of completeness, I would like to say also that I don't create my packages on an ArchLinux system:
I order to get some more automatic package creations, I want to do that on an external web server, not owned by us and without root access, but I can login to it using SSH and make many things that way. This server seems to be (also) Debian based, with a quite old version (6.0.10).
I recreated there all pacman utilities, along with many of their dependencies, by compiling them from sources with some tweakings at the"configure" step, and this seems to work.

I also expect stopping using ArchLinux packages repositories directly, and instead copying in our own repository all packages that we effectively use (along with their upgrades), in order to get more control over upgrading the underlying system, by testing each new releases before giving them to our users, and not running in the risk that a bug in a package distributed by ArchLinux could break some of these devices beyond our control.

Regards,

Gingko

Last edited by Gingko (2016-02-08 09:15:38)

Offline

Board footer

Powered by FluxBB