PKGBUILD review: zpaqfranz

fcorbelli · 2023-09-14 14:48:25

I'm writing a PKGBUILD for a little program of mine, zpaqfranz
I'm not at all well versed in Arch packages: I've been lurking around coming up with this

# Maintainer: Franco Corbelli <franco@francocorbelli.com>
# Upstream  : the same :-)

pkgname=zpaqfranz
pkgver=58.10
pkgrel=1
pkgdesc='Swiss army knife for the serious backup and disaster recovery manager'
arch=('x86_64' 'i686')
depends=('gcc-libs' 'glibc')
url='https://github.com/fcorbelli/zpaqfranz'
license=(MIT)
# yes, a dirty trick 
source=("https://github.com/fcorbelli/zpaqfranz-stuff/blob/main/NONWINDOWS/ARCH/zpaqfranz_$pkgve
r.zip?raw=true")
sha1sums=('53A25D56C6359E1477ECBC557A17681FE15C7675')

build() {
cd "${srcdir}/"
  c++  -O3 -Dunix $pkgname.cpp -o $pkgname -Wl,-z,relro,-z,now -pthread -s
}

package() {
  cd "${srcdir}/"
  install -Dm 755 $pkgname "$pkgdir/usr/bin/$pkgname"
  install -D LICENSE -t "${pkgdir}/usr/share/licenses/${pkgname}"
}

Is it OK for Arch standards?

Things I would/could have done differently

1) Use two different uncompressed files (.cpp and LICENSE) instead of a single zip. It would have approximately tripled the data to transfer (~3MB vs ~1MB), but downloading 100% text
2) Enable compilation on non-Intel platforms (zpaqfranz works on almost everything, including PowerPC), but I don't think it's interesting for Arch
3) Remove the division into versions, download the latest source from github and compile it, skipping the hash check. This way you would always have the latest version of the program. Could it make sense?
Translation. zpaqfranz does need a Makefile nor patches. Can be compiled straight in one step. The normal update/upgrade is
... download the latest .cpp
... compile
... write in /bin
4) I like -static very, very much. Is it banned from Arch's packages?
5) The "dirtiest of all"
A PKGBUILD with... the source code inside (!), so you don't have to download it from the internet. Obviously the PKGBUILD becomes large, around 3MB. Is this "something" tolerated?

Thanks

Lone_Wolf · 2023-09-15 09:21:53

A2 :
Users running AArch64 (arm) do use AUR and might be interested. I am not sure but Asahi Linux (targets apple M1/M2 ) may also be able to use aur packages.

A4:
discouraged, but not banned. If there are .a files in the build you need to use options=('static') to ensure they're not removed.

A5 :
No.

A1 & A3 :
using separate files tends to be a bad idea, but you could switch to a git based source, see https://wiki.archlinux.org/title/VCS_package_guidelines
If it uses a fixed commit you don't need to add -git to the package name.

the functions in PKGBUILD start in $srcdir , so there's no need to cd there (doesn't harm though).

Last edited by Lone_Wolf (2023-09-15 09:22:45)

fcorbelli · 2023-09-15 11:36:23

Lone_Wolf wrote:

A2 :
Users running AArch64 (arm) do use AUR and might be interested. I am not sure but Asahi Linux (targets apple M1/M2 ) may also be able to use aur packages.

I found this https://aur.archlinux.org/cgit/aur.git/ … 7-zip-full

I need something like that
(aka: if the system is NOT "normal" Intel, add -DNOJIT)
For a big endian CPU a -DBIG too (does arch like PowerPC?)

Just a snippet

(...)
build() {
    local -A platforms=(['x86_64']='x64' ['i686']='x86' 
    (
        set -a
        PLATFORM="${platforms["${CARCH}"]}"
        IS_X64=$([ "${CARCH}" = 'x86_64' ] && echo '1' || echo '')
        IS_X86=$([ "${CARCH}" = 'i686' ] && echo '1' || echo '')
        set +a

if !(IS_X64 || IS_X86)
 c++ -DNOJIT blablabla
else
 c++ blablabla


    )
}

How can be done?

A1 & A3 :
using separate files tends to be a bad idea, but you could switch to a git based source, see https://wiki.archlinux.org/title/VCS_package_guidelines

I was referring to downloading two text files (from github), a .cpp and a LICENSE, instead of a binary file (.zip or tar.gz or whatever)
I'll leave the zip, it's more convenient

6) [A3] even more dirty
Bypass each version of the program, downloading the latest .cpp from github, and compiling it (+ the LICENSE file, that's why two files), skipping hash check.
Translation
Does it make sense to keep different versions of my program (58.1, 58.2, 58.3... 58.10...) or just take the latest one each time?
Normally it's important to maintain different versions, but for such a small program?

Thank you!

Last edited by fcorbelli (2023-09-15 11:38:41)

Lone_Wolf · 2023-09-17 11:14:27

fcorbelli wrote:

6) [A3] even more dirty smile
Bypass each version of the program, downloading the latest .cpp from github, and compiling it (+ the LICENSE file, that's why two files), skipping hash check.
Translation
Does it make sense to keep different versions of my program (58.1, 58.2, 58.3... 58.10...) or just take the latest one each time?
Normally it's important to maintain different versions, but for such a small program?

That is a reasonable description of a Version Control System package aka VCS-package aka *-git package.
/usr/share/pacman/PKGBUILD-vcs.proto shows how such a package is structured and function as atemplate to get started.
https://wiki.archlinux.org/title/VCS_package_guidelines has more details.

It's not the size that determines if releases are needed, but the nature of the changes.
Many developers seem to like semver aka semantic versioning. In case you're not aware of it, https://www.baeldung.com/cs/semantic-versioning should be helpful.

fcorbelli wrote:

(aka: if the system is NOT "normal" Intel, add -DNOJIT)
For a big endian CPU a -DBIG too (does arch like PowerPC?)

There is an ArchPOWER distro that targets Little Endian powerpc & riscv64 . It's github repo shows recent activity, no idea if it can use aur packages.

fcorbelli wrote:

I found this https://aur.archlinux.org/cgit/aur.git/ … 7-zip-full
need something like that
Just a snippet
How can be done?

The code in that package looks complicated and I don't know how it works (didn't look at upstream sourcecode) .
'set -a' / 'set +a' suggests the environment vars used in that block are exported. An educated guess is that the sourecode contains lines to handle certain envvars.
(if that is correct it seems much simpler to put them on the commandline of the compile command instead of exporting them.)

Setting those envvars looks very much like it doesn't need a platform array, and the multiple echo commands obscure what it does imo . A case statement that directly sets values looks like a much better fit.

example (untested)

case $CARCH in
"x86_64")
        IS_X64=1;;
"i686")
        IS_X86=1;;
"Aarch64")
        IS_ARM64=1
"powerpcbe")
        IS_POWERPC=1
        IS_BIG_ENDIAN=1
        ;;
*)
        IS_UNSUPPORTED=1
        ;;
esac

fcorbelli · 2023-09-17 11:56:14

Lone_Wolf wrote:

That is a reasonable description of a Version Control System package aka VCS-package aka *-git package (...)
Many developers seem to like semver aka semantic versioning. In case you're not aware of it, https://www.baeldung.com/cs/semantic-versioning should be helpful.

Certainly there are cases in which having different versions of the same software is important

Sometimes less so, it matters to have the latest version, because getting an older one is not particularly useful
Aka: a "rolling release" of zpaqfranz

For example, in the case of 7z (I mention it because in the same category) having distinct versions 21.04 or 21.06 is not so relevant
Normally I keep around fifty (!) old versions on sourceforge, then I gradually prune them, like here https://sourceforge.net/projects/zpaqfranz/files/ if someone really wanted to choose a specific version

Recap: does Arch need a explicit version for a package, or can be something like "zpaqfranz-latest"?
This would seem to fit with Arch's general philosophy: get the latest version, compile it, and good luck

Something like that

# Maintainer: Franco Corbelli <franco@francocorbelli.com>
# Upstream  : the same :-)

pkgname=zpaqfranz
pkgver=latest
pkgrel=1
pkgdesc='Swiss army knife for the serious backup and disaster recovery manager'
arch=('x86_64' 'i686')
depends=('gcc-libs' 'glibc')
url='https://github.com/fcorbelli/zpaqfranz'
license=(MIT)
# yes, a dirty trick 
source=("$pkgname.cpp::https://github.com/fcorbelli/zpaqfranz/blob/main/zpaqfranz.cpp?raw=true"
             "LICENSE::https://github.com/fcorbelli/zpaqfranz/blob/main/LICENSE?raw=true")
sha1sums=('SKIP' 'SKIP')

build() {
# as a "rolling package" we get the version straight from the source
  pkgver=$(grep "#define ZPAQ_VERSION" zpaqfranz.cpp |awk '{ print $3}' | tr -d \": )
  echo Compiling $pkgname build $pkgver
  c++   -Dunix $pkgname.cpp -o $pkgname -Wl,-z,relro,-z,now -pthread -s
}

package() {
  install -Dm 755 $pkgname "$pkgdir/usr/bin/$pkgname"
  install -D LICENSE -t "${pkgdir}/usr/share/licenses/${pkgname}"
}

fcorbelli wrote:

(aka: if the system is NOT "normal" Intel, add -DNOJIT)
For a big endian CPU a -DBIG too (does arch like PowerPC?)
...

For non-Intel seems a bit complex, for an Arch newbie just like me.
I think I'll try to start with the Intel-only version and then, if necessary, I'll improve it

Thanks for your help, you are clarifying specific aspects for me (as mentioned, each platform and each distribution has customs and practices that are often different and sometimes incompatible)

Last edited by fcorbelli (2023-09-17 12:39:16)

Lone_Wolf · 2023-09-17 12:41:51

pkgver has to follow certain ruless and is used to distinguish between versions.
Also static files need a checksum to ensure users get the same file/contnet as the maintainer.

Just change it to a *-git package. *-git pacakges dynamically update the version at buildtime so no need to change it manually.

here's a diff against your PKGBUILD. It does build, have not done any other testing.

--- PKGBUILD.zpaqfranz  2023-09-17 14:25:56.815769899 +0200
+++ PKGBUILD    2023-09-17 14:36:33.624172716 +0200
@@ -1,24 +1,30 @@
 # Maintainer: Franco Corbelli <franco@francocorbelli.com>
 # Upstream  : the same :-)
 
-pkgname=zpaqfranz
-pkgver=latest
+pkgname=zpaqfranz-git
+_pkgname=zpaqfranz
+pkgver=1
 pkgrel=1
 pkgdesc='Swiss army knife for the serious backup and disaster recovery manager'
 arch=('x86_64' 'i686')
 depends=('gcc-libs' 'glibc')
 url='https://github.com/fcorbelli/zpaqfranz'
 license=(MIT)
-# yes, a dirty trick 
-source=("$pkgname.cpp::https://github.com/fcorbelli/zpaqfranz/blob/main/zpaqfranz.cpp?raw=true"
-"LICENSE::https://github.com/fcorbelli/zpaqfranz/blob/main/LICENSE?raw=true")
-sha1sums=('SKIP' 'SKIP')
+source=("$_pkgname::git+https://github.com/fcorbelli/zpaqfranz.git")
+sha1sums=('SKIP')
 
-build() {
-  c++  -O3 -Dunix $pkgname.cpp -o $pkgname -Wl,-z,relro,-z,now -pthread -s
+pkgver() {
+  cd $_pkgname
+  printf "r%s.%s" "$(git rev-list --count HEAD)" "$(git rev-parse --short=7 HEAD)"
+}
+
+ build() {
+ cd $_pkgname
+  c++  -O3 -Dunix $_pkgname.cpp -o $_pkgname -Wl,-z,relro,-z,now -pthread -s
 }
 
 package() {
-  install -Dm 755 $pkgname "$pkgdir/usr/bin/$pkgname"
+  cd $_pkgname
+  install -Dm 755 $_pkgname "$pkgdir/usr/bin/$_pkgname"
   install -D LICENSE -t "${pkgdir}/usr/share/licenses/${pkgname}"
 }

Last edited by Lone_Wolf (2023-09-17 12:43:32)

fcorbelli · 2023-09-17 12:43:40

Maybe we have "crossed"

  pkgver=$(grep "#define ZPAQ_VERSION" zpaqfranz.cpp |awk '{ print $3}' | tr -d \": )
  echo Compiling $pkgname build $pkgver

I'll try your diff
Thanks

fcorbelli · 2023-09-17 14:12:30

After a bit of digging I have made two
First: git clone
Not really sure, transfer a lot (conflicts just for develop)

# Maintainer: Franco Corbelli <franco@francocorbelli.com>
# Upstream  : the same :-)
pkgname=zpaqfranz-git
pkgver=git
pkgrel=1
pkgdesc="Swiss army knife for the serious backup and disaster recovery manager"
arch=('i686' 'x86_64')
url="https://github.com/fcorbelli/zpaqfranz"
license=('MIT')
depends=('gcc-libs' 'glibc')
makedepends=('git')
conflicts=('zpaqfranz')
source=("git+https://github.com/fcorbelli/zpaqfranz.git")
sha256sums=('SKIP')

pkgver() {

  cd "zpaqfranz"
# we do not want version from releases. sometimes the last release is behind very latest cpp
#  _tag=$(git tag -l --sort -v:refname | sed '/rc[0-9]*/d' | head -n1)
#  _hash=$(git rev-parse --short HEAD)

 _tag=$(grep "#define ZPAQ_VERSION" zpaqfranz.cpp |awk '{ print $3}' | tr -d \": )
 _hash=$(md5sum zpaqfranz.cpp|awk '{ print substr($1,1,6)}')
 printf "%s.g%s" "$_tag" "$_hash" | sed 's/^v//'
}

build() {
  cd "zpaqfranz"
  c++  -O3 -Dunix zpaqfranz.cpp -o zpaqfranz -Wl,-z,relro,-z,now -pthread -s
}

package() {
  cd "zpaqfranz"
  install -Dm 755 zpaqfranz "$pkgdir/usr/bin/zpaqfranz"
  install -D LICENSE -t "${pkgdir}/usr/share/licenses/${pkgname}"
}

Second: direct download of the .cpp

# Maintainer: Franco Corbelli <franco@francocorbelli.com>
# Upstream  : the same :-)

pkgname=zpaqfranz
pkgver=latest
pkgrel=1
pkgdesc='Swiss army knife for the serious backup and disaster recovery manager'
arch=('x86_64' 'i686')
depends=('gcc-libs' 'glibc')
url='https://github.com/fcorbelli/zpaqfranz'
license=(MIT)
# yes, a dirty trick 
source=("$pkgname.cpp::https://github.com/fcorbelli/zpaqfranz/blob/main/zpaqfranz.cpp?raw=true"
             "LICENSE::https://github.com/fcorbelli/zpaqfranz/blob/main/LICENSE?raw=true")
sha1sums=('SKIP' 'SKIP')

pkgver() {
# as a "rolling package" we get the version straight from the source
 _tag=$(grep "#define ZPAQ_VERSION" zpaqfranz.cpp |awk '{ print $3}' | tr -d \": )
 _hash=$(md5sum zpaqfranz.cpp|awk '{ print substr($1,1,6)}')
 printf "%s.g%s" "$_tag" "$_hash" | sed 's/^v//'
}

build() {
  c++   -Dunix $pkgname.cpp -o $pkgname -Wl,-z,relro,-z,now -pthread -s
}

package() {
  install -Dm 755 $pkgname "$pkgdir/usr/bin/$pkgname"
  install -D LICENSE -t "${pkgdir}/usr/share/licenses/${pkgname}"
}

Are they eligible for a submission?

Last edited by fcorbelli (2023-09-17 14:13:00)

Scimmia · 2023-09-17 14:15:15

Either way, this is a -git PKGBUILD, and you haven't fixed the pkgname.

Downloading the unversioned file like that won't work, since the file already exists, it'll never update when run again.

Why are you not using Arch's CFLAGS/CPPFLAGS/LDFLAGS, etc?

Using the md5 hash of the .cpp file in the pkgver is pretty pointless.

Last edited by Scimmia (2023-09-17 14:24:35)

fcorbelli · 2023-09-17 14:37:09

Scimmia wrote:

Either way, this is a -git PKGBUILD, and you haven't fixed the pkgname.
Downloading the unversioned file like that won't work, since the file already exists, it'll never update when run again.

Very good point, so going to version 1

Why are you not using Arch's CFLAGS/CPPFLAGS/LDFLAGS, etc?

Because... I do not know how to (yet)

Should become

# Maintainer: Franco Corbelli <franco@francocorbelli.com>
# Upstream  : the same :-)
pkgname=zpaqfranz-git
pkgver=git
pkgrel=1
pkgdesc="Swiss army knife for the serious backup and disaster recovery manager"
arch=('i686' 'x86_64')
url="https://github.com/fcorbelli/zpaqfranz"
license=('MIT')
depends=('gcc-libs' 'glibc')
makedepends=('git')
conflicts=('zpaqfranz')
source=("git+https://github.com/fcorbelli/zpaqfranz.git")
sha256sums=('SKIP')

pkgver() {
  cd "zpaqfranz"
 _tag=$(grep "#define ZPAQ_VERSION" zpaqfranz.cpp |awk '{ print $3}' | tr -d \": )
 _hash=$(md5sum zpaqfranz.cpp|awk '{ print substr($1,1,6)}')
 printf "%s.g%s" "$_tag" "$_hash" | sed 's/^v//'
}

build() {
  cd "zpaqfranz"
  c++  $CXXFLAGS $LDFLAGS -Dunix zpaqfranz.cpp -o zpaqfranz -pthread 
}

package() {
  cd "zpaqfranz"
  install -Dm 755 zpaqfranz "$pkgdir/usr/bin/zpaqfranz"
  install -D LICENSE -t "${pkgdir}/usr/share/licenses/${pkgname}"
}

Is it OK?

fcorbelli · 2023-09-17 15:45:47

Scimmia wrote:

Using the md5 hash of the .cpp file in the pkgver is pretty pointless.

I need it to quickly recognize different (or the same) versions. It is rare but can happen (that I leave the same version number but changing something, typically to accommodate very strict packagers like Fedora or Debian)

Personally, as mentioned, I would eliminate any kind of versioning altogether, but it seems to be preferred on Arch
I would do zpaqfranz - fullstop

Maybe a $CPPFLAGS too should be added

loqs · 2023-09-17 16:57:43

fcorbelli wrote:

I need it to quickly recognize different (or the same) versions. It is rare but can happen (that I leave the same version number but changing something, typically to accommodate very strict packagers like Fedora or Debian)

Why not use one of the suggested pkgver() functions from VCS_package_guidelines#Git. Which use git-describe.1 and cover commits without pkgver updates or new tag updates. I think the only case they do not handle is if you change the git tag with force push.

fcorbelli wrote:

Personally, as mentioned, I would eliminate any kind of versioning altogether, but it seems to be preferred on Arch

The pkgver is how pacman determines if a package in a repository should be updated or not.

fcorbelli · 2023-09-17 17:10:48

loqs wrote:

fcorbelli wrote:
I need it to quickly recognize different (or the same) versions. It is rare but can happen (that I leave the same version number but changing something, typically to accommodate very strict packagers like Fedora or Debian)
Why not use one of the suggested pkgver() functions from VCS_package_guidelines#Git. Which use git-describe.1 and cover commits without pkgver updates or new tag updates. I think the only case they do not handle is if you change the git tag with force push.
fcorbelli wrote:
Personally, as mentioned, I would eliminate any kind of versioning altogether, but it seems to be preferred on Arch
The pkgver is how pacman determines if a package in a repository should be updated or not.

I'll use the rev-parse, stripping tr and updating makedepends

# Maintainer: Franco Corbelli <franco@francocorbelli.com>
# Upstream  : the same :-)
pkgname=zpaqfranz-git
pkgver=git
pkgrel=1
pkgdesc="Swiss army knife for the serious backup and disaster recovery manager"
arch=('i686' 'x86_64')
url="https://github.com/fcorbelli/zpaqfranz"
license=('MIT')
depends=('gcc-libs' 'glibc' )
makedepends=('git' 'grep' 'awk' 'sed')
conflicts=('zpaqfranz')
source=("git+https://github.com/fcorbelli/zpaqfranz.git")
sha256sums=('SKIP')

pkgver() {
  cd "zpaqfranz"
 _tag=$(grep "#define ZPAQ_VERSION" zpaqfranz.cpp |awk '{gsub(/"/, "", $3); print $3}')
 _hash=$(git rev-parse --short HEAD)
 printf "%s.g%s" "$_tag" "$_hash" | sed 's/^v//'
}

build() {
  cd "zpaqfranz"
  c++  $CXXFLAGS $CPPFLAGS $LDFLAGS -Dunix zpaqfranz.cpp -o zpaqfranz -pthread 
}

package() {
  cd "zpaqfranz"
  install -Dm 755 zpaqfranz "$pkgdir/usr/bin/zpaqfranz"
  install -D LICENSE -t "${pkgdir}/usr/share/licenses/${pkgname}"
}

Thanks for the suggestion

I could not find a way to delete a file before downloading the source
Strange
Patience

Scimmia · 2023-09-17 22:46:27

fcorbelli wrote:

Scimmia wrote:
Using the md5 hash of the .cpp file in the pkgver is pretty pointless.
I need it to quickly recognize different (or the same) versions. It is rare but can happen...

In that case, using that or git rev-parse are worse than pointless, they're actively harmful. That means your pkgver will randomly go up or down. You MUST have something before it that will go up with each commit.

fcorbelli · 2023-09-18 03:30:41

What's the problem if pkgver change?
In fact md5 hash do exactly what I want: a new pgkver for a different source code, enforcing an update at every run

My "first choiche" for pkgver is just md5(source code), no semantic versioning

Different source, different pkgvers, forced update

Scimmia · 2023-09-18 04:01:19

The pkgver cannot go backwards.

fcorbelli · 2023-09-18 04:39:58

Ahhh.. OK
I missed this point
Then I'll take care "manually"

Is it possible to enforce overwrite from source, o execute a rm before source?

Translation
Afaik (no git) work this way

Some kind of curl of the file(s) WITHOUT overwrite: hard coded different name is mandatory
Run optional functions in optional .install script
Run others function in PKGBUILD

I'd like to either
Download with overwrite with the same name

Or
Download with runtime name (ex current timestamp to get different name each time, like a GID)

Or

rm thesource
(Do as usual)

?
Thanks

Lone_Wolf · 2023-09-18 08:11:43

man makepkg wrote:

-C, --cleanbuild
Remove the $srcdir before building the package.

In case you don't want to rely on users building with that flag :

Use

source=("zpaqfranz.$pkgver::git+https://github.com/fcorbelli/zpaqfranz.git")

so makepkg will create a new folder for every new version.
(don't forget to change the cd commands in the functions)

To avoid multiple downloads of unchanged files :

man makepkg.conf wrote:

SRCDEST="/path/to/directory"
If this value is not set, downloaded source files will only be stored in the current directory. Many people like to keep all source files in a central location for easy cleanup, so this path can be set here.

--

fcorbelli wrote:

Run optional functions in optional .install script

on archlinux .install scripts are always run when pacman installs/updates a package.
If you want to set up things when a user runs zpaqfranz, use a wrapper script to start zpaqfranz.
https://aur.archlinux.org/cgit/aur.git/ … e?h=oolite is an example of such a wrapper script

fcorbelli · 2023-09-18 08:20:38

WOW!

source=("zpaqfranz.$pkgver::git+https://github.com/fcorbelli/zpaqfranz.git")
so makepkg will create a new folder for every new version.

pkgver must be setted... how?
I tried to put dates, i.e. a timestamp (like 20210920) but I couldn't
i.e. pkgver="pippo" will create every time zpaqfranz.pippo (or at least I think so)
But I'll try, just to learn something new

To avoid multiple downloads of unchanged files :

... I want the opposite.
Download every time

on archlinux .install scripts are always run when pacman installs/updates a package.

The problem in "when-when"
I want to delete a file (say, zpaqfranz.cpp) BEFORE downloading (of course lacking an "overwrite flag")
In this case, EVERY TIME, the PKGBUILD will download the "latest" zpaqfranz.cpp.
The the "other things" will continue as usual.

Thanks very much!

Lone_Wolf · 2023-09-18 08:43:15

pkgver must be setted... how?
I tried to put dates, i.e. a timestamp (like 20210920) but I couldn't
i.e. pkgver="pippo" will create every time zpaqfranz.pippo (or at least I think so)
But I'll try, just to learn something new

pkgver always has a value. for release versions this is (supposed to be) static and needs to be set before it's used anywhere.
For packages with a pkgver() function makepkg changes all references to pkgver dynamically whenever pkgver() gets a new value.

Short answer : let makepkg + pkgver() take care of that.

The problem in "when-when"
I want to delete a file (say, zpaqfranz.cpp) BEFORE downloading (of course lacking an "overwrite flag")
In this case, EVERY TIME, the PKGBUILD will download the "latest" zpaqfranz.cpp.

for a *-git pacakge that's easy : the sourcecode will always be at the revision that is cloned.
No need to overwrite anything.

Last edited by Lone_Wolf (2023-09-18 08:43:59)

fcorbelli · 2023-09-18 09:30:52

Lone_Wolf wrote:

pkgver always has a value. for release versions this is (supposed to be) static and needs to be set before it's used anywhere.
For packages with a pkgver() function makepkg changes all references to pkgver dynamically whenever pkgver() gets a new value.

I'll try after lunch :-)

for a *-git pacakge that's easy : the sourcecode will always be at the revision that is cloned.
No need to overwrite anything.

Yes, indeed.

But I'm trying NOT to use the -git method, precisely because it downloads the entire repository, taking a lot of data unnecessarily (in reality it's quite small, for Fedora I already had to remove a lot of "things", but it's still about 10 times the necessary amount)
Afterall it is a deduplication and compression software, reducing to the minimum possible is the "keyword" :-)

I will almost certainly opt for -git, in the end, anyway.
But it's fun to try the roads less traveled, learning new techniques

fcorbelli · 2023-09-18 11:31:32

OK, I declare myself defeated.
I can't change pkgver BEFORE the download is done, and then change its value
Only AFTER

So I'm going back to git
I will carefully maintain the version in the source, removing any need for hashes, direct or indirect on git

# Maintainer: Franco Corbelli <franco@francocorbelli.com>
pkgname=zpaqfranz-git
pkgver=git
pkgrel=1
pkgdesc="Swiss army knife for the serious backup and disaster recovery manager"
arch=('i686' 'x86_64')
url="https://github.com/fcorbelli/zpaqfranz"
license=('MIT')
depends=('gcc-libs' 'glibc')
makedepends=('git' 'grep' 'awk')
conflicts=('zpaqfranz')
source=("git+https://github.com/fcorbelli/zpaqfranz.git")
sha256sums=('SKIP')
pkgver() 
{
  cd "zpaqfranz"
  _tag=$(grep "#define ZPAQ_VERSION" zpaqfranz.cpp |awk '{gsub(/"/, "", $3); print $3}')
  printf "%s" "$_tag"
}
build() 
{
  cd "zpaqfranz"
  c++ $CXXFLAGS $CPPFLAGS $LDFLAGS -Dunix zpaqfranz.cpp -o zpaqfranz -pthread 
}
package() {
  cd "zpaqfranz"
  install -Dm 755 zpaqfranz "$pkgdir/usr/bin/zpaqfranz"
  install -D LICENSE -t "${pkgdir}/usr/share/licenses/${pkgname}"
}

That seems decent.
Any further observations?
If not, I'll try uploading to AUR

Last edited by fcorbelli (2023-09-18 11:33:52)

Lone_Wolf · 2023-09-19 09:53:57

I still think you should add atleast the commit hash in the pkgver so there's a direct link between the pkgver and upstream code version, but it will be your package.

The readme.md file indicates zpafranz is a fork of zpaq 7.15 , I do think you should mention that in pkgdesc .

Other then that there are 2 warnings from gcc during build. While such warnings are often compiler specific these have to do with potential buffer overflows.
Do you want me to file a github issue for them ?

fcorbelli · 2023-09-19 10:21:46

Lone_Wolf wrote:

I still think you should add atleast the commit hash in the pkgver so there's a direct link between the pkgver and upstream code version, but it will be your package.

In fact there is (taken from the source code).
I would have even put part of the source's md5 in there, to avoid collisions, but another user advised against it

Since I DO NOT use git to upload data to github, I will be careful... to be careful.
I use my own VCS, which is called... zpaqfranz

Short version: the pkgver and the upstream will always be aligned, just because the pkgver is taken from upstream via grep

The readme.md file indicates zpafranz is a fork of zpaq 7.15 , I do think you should mention that in pkgdesc .

I agree

Other then that there are 2 warnings from gcc during build. While such warnings are often compiler specific these have to do with potential buffer overflows.
Do you want me to file a github issue for them ?

These are problems with the gcc compiler, which has various bugs (aka: A LOT)
I decided to leave it alone, it's a waste of time to keep up with it.

Like here, already discussed with BLAKE3's team,
https://github.com/BLAKE3-team/BLAKE3/issues/189

here, GCC 10.3.0 on Ubuntu 21
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=101558

and others

One in particular, latest version gcc for Fedora 32bit (!), required me to reverse engineer (!!) the gcc assembly optimizer (obviously only confined to that specific bug), with the help of the Fedora zpaqfranz's maintainer
This one https://github.com/fcorbelli/zpaqfranz/issues/71

If you delete the call to fixgcc(), the program will segfault

/// this is a weird "fix" for gcc bug https://github.com/fcorbelli/zpaqfranz/issues/71
/// please don't ask anything, took a couple of hours

void fixgcc(const char* fmt, ...)
{
   va_list args;
   va_start(args, fmt);
   va_end(args);
}

void HighwayHashReset(const uint64_t key[4], HighwayHashState* state) {
  state->mul0[0] = 0xdbe6d5d5fe4cce2full;
  state->mul0[1] = 0xa4093822299f31d0ull;
  state->mul0[2] = 0x13198a2e03707344ull;
  state->mul0[3] = 0x243f6a8885a308d3ull;
  state->mul1[0] = 0x3bd39e10cb0ef593ull;
  state->mul1[1] = 0xc0acf169b5f18a8cull;
  state->mul1[2] = 0xbe5466cf34e90c6cull;
  state->mul1[3] = 0x452821e638d01377ull;
  state->v0[0] = state->mul0[0] ^ key[0];
  state->v0[1] = state->mul0[1] ^ key[1];
  state->v0[2] = state->mul0[2] ^ key[2];
  state->v0[3] = state->mul0[3] ^ key[3];
  state->v1[0] = state->mul1[0] ^ ((key[0] >> 32) | (key[0] << 32));
  state->v1[1] = state->mul1[1] ^ ((key[1] >> 32) | (key[1] << 32));
  state->v1[2] = state->mul1[2] ^ ((key[2] >> 32) | (key[2] << 32));
	fixgcc("ciao");
  state->v1[3] = state->mul1[3] ^ ((key[3] >> 32) | (key[3] << 32));
}

Yes, my very own fixgcc "breaks down" the ...broken logic
https://bugzilla.redhat.com/show_bug.cgi?id=2238781

Short version: I am not very upset by warnings from compilers who "think" they are "smart"
They aren't

fcorbelli · 2023-09-19 12:06:19

Can I mention

Lone_Wolf https://bbs.archlinux.org/profile.php?id=4504
Scimmia https://bbs.archlinux.org/profile.php?id=63385
Loqs https://bbs.archlinux.org/profile.php?id=80022

as contributors (in PKGBUILD) and in the greetings (in the source)

# Contributor: Lone_Wolf (bbs.archlinux.org)
# Contributor: Scimmia (bbs.archlinux.org)
# Contributor: Loqs (bbs.archlinux.org)

29 Thanks to Lone_Wolf (bbs.archlinux.org)              for reviewing PKGBUILD
30 Thanks to Scimmia   (bbs.archlinux.org)              for reviewing PKGBUILD
31 Thanks to Loqs      (bbs.archlinux.org)              for reviewing PKGBUILD

PS I am double-checking the last warning, but everything seems fine

53678: 00000000000 buffer8bit  1.000.000
53676: 00000000000 chunksize   1.000.000
53676: 00000000000 chunksize-8 999.992
53676: 00000000000 j is 999.992
53676: 00000000000 j is 999.993
53676: 00000000000 j is 999.994
53676: 00000000000 j is 999.995
53676: 00000000000 j is 999.996
53676: 00000000000 j is 999.997
53676: 00000000000 j is 999.998
53676: 00000000000 j is 999.999
Iteration 1/9 chunksize      333.333
53678: 00000000000 buffer8bit  333.333
53676: 00000000000 chunksize   333.333
53676: 00000000000 chunksize-8 333.325
53676: 00000000000 j is 333.325
53676: 00000000000 j is 333.326
53676: 00000000000 j is 333.327
53676: 00000000000 j is 333.328
53676: 00000000000 j is 333.329
53676: 00000000000 j is 333.330
53676: 00000000000 j is 333.331
53676: 00000000000 j is 333.332
Iteration 2/9 chunksize      111.111
53678: 00000000000 buffer8bit  111.111
53676: 00000000000 chunksize   111.111
53676: 00000000000 chunksize-8 111.103
53676: 00000000000 j is 111.103
53676: 00000000000 j is 111.104
53676: 00000000000 j is 111.105
53676: 00000000000 j is 111.106
53676: 00000000000 j is 111.107
53676: 00000000000 j is 111.108
53676: 00000000000 j is 111.109
53676: 00000000000 j is 111.110
Iteration 3/9 chunksize       37.037
53678: 00000000000 buffer8bit  37.037
53676: 00000000000 chunksize   37.037
53676: 00000000000 chunksize-8 37.029
53676: 00000000000 j is 37.029
53676: 00000000000 j is 37.030
53676: 00000000000 j is 37.031
53676: 00000000000 j is 37.032
53676: 00000000000 j is 37.033
53676: 00000000000 j is 37.034
53676: 00000000000 j is 37.035
53676: 00000000000 j is 37.036
Iteration 4/9 chunksize       12.345
53678: 00000000000 buffer8bit  12.345
53676: 00000000000 chunksize   12.345
53676: 00000000000 chunksize-8 12.337
53676: 00000000000 j is 12.337
53676: 00000000000 j is 12.338
53676: 00000000000 j is 12.339
53676: 00000000000 j is 12.340
53676: 00000000000 j is 12.341
53676: 00000000000 j is 12.342
53676: 00000000000 j is 12.343
53676: 00000000000 j is 12.344
Iteration 5/9 chunksize        4.115
53678: 00000000000 buffer8bit  4.115
53676: 00000000000 chunksize   4.115
53676: 00000000000 chunksize-8 4.107
53676: 00000000000 j is 4.107
53676: 00000000000 j is 4.108
53676: 00000000000 j is 4.109
53676: 00000000000 j is 4.110
53676: 00000000000 j is 4.111
53676: 00000000000 j is 4.112
53676: 00000000000 j is 4.113
53676: 00000000000 j is 4.114
Iteration 6/9 chunksize        1.371
53678: 00000000000 buffer8bit  1.371
53676: 00000000000 chunksize   1.371
53676: 00000000000 chunksize-8 1.363
53676: 00000000000 j is 1.363
53676: 00000000000 j is 1.364
53676: 00000000000 j is 1.365
53676: 00000000000 j is 1.366
53676: 00000000000 j is 1.367
53676: 00000000000 j is 1.368
53676: 00000000000 j is 1.369
53676: 00000000000 j is 1.370
Iteration 7/9 chunksize          457
53678: 00000000000 buffer8bit  457
53676: 00000000000 chunksize   457
53676: 00000000000 chunksize-8 449
53676: 00000000000 j is 449
53676: 00000000000 j is 450
53676: 00000000000 j is 451
53676: 00000000000 j is 452
53676: 00000000000 j is 453
53676: 00000000000 j is 454
53676: 00000000000 j is 455
53676: 00000000000 j is 456
Iteration 8/9 chunksize          152
53678: 00000000000 buffer8bit  152
53676: 00000000000 chunksize   152
53676: 00000000000 chunksize-8 144
53676: 00000000000 j is 144
53676: 00000000000 j is 145
53676: 00000000000 j is 146
53676: 00000000000 j is 147
53676: 00000000000 j is 148
53676: 00000000000 j is 149
53676: 00000000000 j is 150
53676: 00000000000 j is 151
Iteration 9/9 chunksize           50
53678: 00000000000 buffer8bit  50
53676: 00000000000 chunksize   50
53676: 00000000000 chunksize-8 42
53676: 00000000000 j is 42
53676: 00000000000 j is 43
53676: 00000000000 j is 44
53676: 00000000000 j is 45
53676: 00000000000 j is 46
53676: 00000000000 j is 47
53676: 00000000000 j is 48
53676: 00000000000 j is 49

Is it possible to grab the "full" build string?

Arch Linux

#1 2023-09-14 14:48:25

PKGBUILD review: zpaqfranz

#2 2023-09-15 09:21:53

Re: PKGBUILD review: zpaqfranz

#3 2023-09-15 11:36:23

Re: PKGBUILD review: zpaqfranz

#4 2023-09-17 11:14:27

Re: PKGBUILD review: zpaqfranz

#5 2023-09-17 11:56:14

Re: PKGBUILD review: zpaqfranz

#6 2023-09-17 12:41:51

Re: PKGBUILD review: zpaqfranz

#7 2023-09-17 12:43:40

Re: PKGBUILD review: zpaqfranz

#8 2023-09-17 14:12:30

Re: PKGBUILD review: zpaqfranz

#9 2023-09-17 14:15:15

Re: PKGBUILD review: zpaqfranz

#10 2023-09-17 14:37:09

Re: PKGBUILD review: zpaqfranz

#11 2023-09-17 15:45:47

Re: PKGBUILD review: zpaqfranz

#12 2023-09-17 16:57:43

Re: PKGBUILD review: zpaqfranz

#13 2023-09-17 17:10:48

Re: PKGBUILD review: zpaqfranz

#14 2023-09-17 22:46:27

Re: PKGBUILD review: zpaqfranz

#15 2023-09-18 03:30:41

Re: PKGBUILD review: zpaqfranz

#16 2023-09-18 04:01:19

Re: PKGBUILD review: zpaqfranz

#17 2023-09-18 04:39:58

Re: PKGBUILD review: zpaqfranz

#18 2023-09-18 08:11:43

Re: PKGBUILD review: zpaqfranz

#19 2023-09-18 08:20:38

Re: PKGBUILD review: zpaqfranz

#20 2023-09-18 08:43:15

Re: PKGBUILD review: zpaqfranz

#21 2023-09-18 09:30:52

Re: PKGBUILD review: zpaqfranz

#22 2023-09-18 11:31:32

Re: PKGBUILD review: zpaqfranz

#23 2023-09-19 09:53:57

Re: PKGBUILD review: zpaqfranz

#24 2023-09-19 10:21:46

Re: PKGBUILD review: zpaqfranz

#25 2023-09-19 12:06:19

Re: PKGBUILD review: zpaqfranz

Board footer