You are not logged in.

#1 2021-02-22 15:38:35

dry
Member
Registered: 2020-12-23
Posts: 6

PKGBUILD RFC: making different versions of wxWidgets coexist

Hi! I'm currently developing an application using wxWidgets, a cross-platform GUI library. While developing I ran into the issue that the latest "stable" 3.0 version that the official wxgtk packages ship does not have XDG File Layout support and I found out that the 3.1 version of the API does. I can't install wxgtk-git from the AUR either because it conflicts with wxgtk and I can't uninstall wxgtk either because I use Audacity which depends on it. So I set out to try and resolve these conflicts to have both 3.0 and 3.1 installed, and while I'm at it find out whether it would be okay to push them as official packages, that's why the Request For Comments.

Now, I know that 3.1 isn't on the official packages because it's technically not a "stable" version, however I think it has a similar situation to that of the libreoffice-still (7.0) and libreoffice-fresh (7.1) packages, altough in this case it's a library instead of a program.

Some considerations:

- Version 3.0 of the API was released back in 2013 (source, third paragraph), during which a lot of useful features have been implemented in the library, one of which is XDG Layout support.

As the result of all this work, we are close to making 3.1.5 which should be the last release before 3.2.0 which will become the new stable version, after 3.0.0 released back in 2013. It will have too many enhancements and improvements to list in this blog post without turning it into a book

- The wxWidgets website actually recommends using 3.1.4 in production (source, second paragraph).

Please notice that while 3.1.4 is officially a “development” version because it is not fully compatible with the “stable” 3.0.x, the list of backwards incompatible changes is very short, so you shouldn’t have any problems updating to this version from 3.0.x in practice, and you’re encouraged to use this release, including in production.

- How the wxWidgets Version Numbering Scheme works (source):

The Version Numbering Scheme

wxWidgets version numbers can have up to four components, with trailing zeros sometimes omitted:

major.minor.release.sub-release

A stable release of wxWidgets will have an even number for minor, e.g. 2.6.0. Stable, in this context, means that the API is not changing. In truth, some changes are permitted, but only those that are backward compatible. For example, you can expect later 2.6.x releases, such as 2.6.1 and 2.6.2 to be backward compatible with their predecessor.

When it becomes necessary to make changes which are not wholly backward compatible, the stable branch is forked, creating a new development branch of wxWidgets. This development branch will have an odd number for minor, for example 2.7.x. Releases from this branch are known as development snapshots.

The stable branch and the development branch will then be developed in parallel for some time. When it is no longer useful to continue developing the stable branch, the development branch is renamed and becomes a new stable branch, for example: 2.8.0. And the process begins again. This is how the tension between keeping the interface stable, and allowing the library to evolve is managed.

You can expect the versions with the same major and even minor version number to be compatible, but between minor versions there will be incompatibilities. Compatibility is not broken gratuitously however, so many applications will require no changes or only small changes to work with the new version.


So I think that just like with the libreoffice packages we could have both "stable/even" packages for established applications and "development/odd" packages for new applications and applications that need new features.

There are two conflicts that need to be resolved for the packages to coexist and I have worked with upstream to fix them (mailing list thread).

The first one had to do with adding release suffixes to gettext catalogs, which is necessary for runtime, which has been resolved with this commit that has already been merged with upstream.

The second one has to do with some development utilities, about which the maintainer told me that they want to keep the filenames as is if possible. After some research I concluded that the 3.1 version of these utilities are backwards compatible with the 3.0 version, these are not needed for applications to run, just for development. So my idea is that the simplest solution would be to have a third flavor agnostic package that ships these utilites, but one of them will depend on the development package base library. Both stable and development packages would need to pull this one just for the utils. Let me detail a little more, these are the files that would be shipped with the utils package:

- /usr/share/aclocal/wxwin.m4 - A m4 file
- /usr/share/bakefiles/presets/*.bkl - some Bakefile scripts
- /usr/bin/wxrc - This is a program to zip xrc resource files (source), it will depend on the 3.1 libwx_base* libraries, even if you just want the stable packages you would still need to compile the development packages
- /usr/bin/wx-config - is a symlink to a script that is generated according to build configurations /usr/lib/wx/config/gtkx-unicode-3.y, where x is the gtk version and y the wxWidgets minor version, which is a script that returns the right compiler and linker flags given the desired configuration, these scripts are pretty much identical to each other except that they define a default version to use but they can detect other of their own and you can specify which one version you want to use through any of them, even versions newer that themselves I think, what I'm trying to say is that if such a script was generated with 3.0 you can still call a 3.1 script through it, in the end it's about deciding which of these scripts to symlink, I would choose the stable gtk3 3.0 one, and pass --version=3.1 to get the right flags as long as 3.1 is a development version

Anyway, I think that the wxrc solution isn't too good, since it implies compiling wxWidgets 4 times, for each gtk version and wx minor version. That is compiling about a million lines of C++ 4 times. Each one of them takes about 5 minutes using all of my 12 i7-9750H threads. I don't know if it would be possible to ship these as compiled binaries nor what are the requirements to ship them as such, let me know if you do.

Here's the repo where I keep the PKGBUILDs that I wrote, these are based on the current official PKGBUILD.
https://github.com/andriybyelikov/archl … s-packages

I've commented out the dependencies as I was testing the PKGBUILDs on a VM with a fresh Arch install, and finding out which packages I needed to compile. I determined these using the deps.sh script which runs find-libdeps on each create package and prints the dependencies as packages. I think these should do, but I don't know if this is the proper way to determine dependencies, nor if they are makedepends or optdepends. The wiki said not to rely on transitive dependencies so I only rely on transitive dependencies that I'm aware of inside the PKGBUILD, you might see that I omitted some dependencies that are already pulled by gtk-common.

Sorry for the kinda long post and thank you for your help and comments.

Offline

#2 2021-03-09 11:49:12

Awebb
Member
Registered: 2010-05-06
Posts: 5,808

Re: PKGBUILD RFC: making different versions of wxWidgets coexist

Without having read everything, I have one question: Can you compile Audacity against wxgtk-git? That's what the ABS is for. Perhaps audacity-git works for you. Since you'll have some compiling to do anyway, this might be the simpler option.

EDIT: The package seems to be called "wxgtk3-dev" and it provides wxgtk3. The package page lists 33 packages that depend on wxgtk3.

Last edited by Awebb (2021-03-09 11:54:22)

Offline

#3 2021-03-10 21:26:00

dry
Member
Registered: 2020-12-23
Posts: 6

Re: PKGBUILD RFC: making different versions of wxWidgets coexist

I believe Audacity ships its own fork of wxWidgets. But my objective was to make the stable and development versions of the packages coexist without conflicts so that (maybe) one day I can ship my app which uses the newer features.

I actually ended up solving the problem of packaging the wxWidgets utilities. Basically just symlink everything and use a post_install/post_remove install script to reconfigure the symlinks based on the versions that are installed or which ones are left after uninstalling.

Here are the PKGBUILDS and install script (which is the same for both PKGBUILDs):

# Maintainer: Andriy Byelikov <andriybyelikov@gmail.com>

pkgbase=wxgtk
pkgname=(wxgtk-common wxgtk2 wxgtk3 wx-utils)
_wx_release=3.0
pkgver=$_wx_release.5.1
pkgrel=1
arch=('x86_64')
url="https://wxwidgets.org"
license=('custom:wxWindows')
makedepends=(libpng libjpeg-turbo libtiff gtk2 libgl glu gst-plugins-base gtk3)
options=('!emptydirs')
source=("https://github.com/wxWidgets/wxWidgets/releases/download/v$pkgver/wxWidgets-$pkgver.tar.bz2"
        "make-abicheck-non-fatal.patch")
sha256sums=('440f6e73cf5afb2cbf9af10cec8da6cdd3d3998d527598a53db87099524ac807'
            '46a1bb97d69163547da13d5e23a4c73e68de27ee601da5d2fb5bc5c417931453')

prepare() {
  cd wxWidgets-$pkgver
  patch -Np1 -i ../make-abicheck-non-fatal.patch
}

_build() {
  mkdir -p gtk$1-build gtk$1-install
  cd gtk$1-build
  ../configure --prefix=/usr --with-opengl --enable-graphics_ctx \
    --enable-mediactrl --with-regex=builtin --disable-precomp-headers \
    --with-libpng=sys --with-libxpm=sys --with-libjpeg=sys --with-libtiff=sys \
    --with-gtk=$@
  make DESTDIR="$srcdir"/wxWidgets-$pkgver/gtk$1-install install
}

build() {
  cd wxWidgets-$pkgver
  make -C locale allmo
  _build 2
  cd ..
  _build 3 --enable-webview
}

_package() {
  cd wxWidgets-$pkgver/gtk$1-install
  shift
  cp -r --preserve=mode --parents $@ "$pkgdir"
  cd ../docs
  install -Dm644 licence.txt "$pkgdir"/usr/share/licenses/$pkgname/LICENSE
}

package_wxgtk-common() {
  pkgdesc="wxWidgets base libraries, headers and gettext catalogs"
  depends=(expat gcc-libs zlib)

  _package 2 ./usr/{include/,lib/libwx_base*,share/locale}
}

package_wxgtk2() {
  pkgdesc="wxGTK 2 libraries, config script and setup.h"
  depends=(cairo gdk-pixbuf2 glib2 gst-plugins-base-libs gstreamer gtk2
    libglvnd libjpeg-turbo libnotify libpng libsm libtiff libx11 libxxf86vm
    pango wxgtk-common)

  _package 2 ./usr/lib/{libwx_gtk*,wx}
}

package_wxgtk3() {
  pkgdesc="wxGTK 3 libraries, config script and setup.h"
  depends=(cairo gdk-pixbuf2 glib2 gst-plugins-base-libs gstreamer gtk3
    libglvnd libjpeg-turbo libnotify libpng libsm libtiff libx11 libxxf86vm
    pango wxgtk-common)
  optdepends=('webkit2gtk: for webview support')

  _package 3 ./usr/lib/{libwx_gtk*,wx}
}

package_wx-utils() {
  pkgdesc="wxWidgets utilities"
  depends=(wxgtk-common)
  install=wxgtk.install

  _package 2 ./usr/{bin/wxrc-$_wx_release,share/{aclocal,bakefile}}
  for file in "$pkgdir"/usr/share/{aclocal,bakefile/presets}/*
    do mv $file "$(echo $file | sed 's/\./-'$_wx_release'./')"; done
}
# Maintainer: Andriy Byelikov <andriybyelikov@gmail.com>

pkgbase=wxgtk-dev
pkgname=(wxgtk-common-dev wxgtk2-dev wxgtk3-dev wx-utils-dev)
_wx_release=3.1
pkgver=$_wx_release.5
pkgrel=1
arch=('x86_64')
url="https://wxwidgets.org"
license=('custom:wxWindows')
makedepends=(git libpng libjpeg-turbo libtiff gtk2 libgl glu gst-plugins-base gtk3)
options=('!emptydirs')
source=("git+https://github.com/wxWidgets/wxWidgets.git"
        "make-abicheck-non-fatal.patch")
sha256sums=('SKIP'
            '46a1bb97d69163547da13d5e23a4c73e68de27ee601da5d2fb5bc5c417931453')

prepare() {
  cd wxWidgets
  git submodule update --init 3rdparty/catch
  patch -Np1 -i ../make-abicheck-non-fatal.patch
}

_build() {
  mkdir -p gtk$1-build gtk$1-install
  cd gtk$1-build
  ../configure --prefix=/usr --with-opengl --enable-graphics_ctx \
    --enable-mediactrl --with-regex=builtin --disable-precomp-headers \
    --with-libpng=sys --with-libxpm=sys --with-libjpeg=sys --with-libtiff=sys \
    --with-gtk=$@
  make DESTDIR="$srcdir"/wxWidgets/gtk$1-install install
}

build() {
  cd wxWidgets
  make -C locale allmo
  _build 2
  cd ..
  _build 3 --enable-webview
}

_package() {
  cd wxWidgets/gtk$1-install
  shift
  cp -r --preserve=mode --parents $@ "$pkgdir"
  cd ../docs
  install -Dm644 licence.txt "$pkgdir"/usr/share/licenses/$pkgname/LICENSE
}

package_wxgtk-common-dev() {
  pkgdesc="wxWidgets base libraries, headers and gettext catalogs"
  depends=(expat gcc-libs zlib)

  _package 2 ./usr/{include/,lib/libwx_base*,share/locale}
}

package_wxgtk2-dev() {
  pkgdesc="wxGTK 2 libraries, config script and setup.h"
  depends=(cairo gdk-pixbuf2 glib2 gst-plugins-base-libs gstreamer gtk2
    libglvnd libjpeg-turbo libnotify libpng libsm libtiff libx11 libxxf86vm
    pango wxgtk-common-dev)

  _package 2 ./usr/lib/{libwx_gtk*,wx}
}

package_wxgtk3-dev() {
  pkgdesc="wxGTK 3 libraries, config script and setup.h"
  depends=(cairo gdk-pixbuf2 glib2 gst-plugins-base-libs gstreamer gtk3
    libglvnd libjpeg-turbo libnotify libpng libsm libtiff libx11 libxxf86vm
    pango wxgtk-common-dev)
  optdepends=('webkit2gtk: for webview support')

  _package 3 ./usr/lib/{libwx_gtk*,wx}
}

package_wx-utils-dev() {
  pkgdesc="wxWidgets utilities"
  depends=(wxgtk-common-dev)
  install=wxgtk.install

  _package 2 ./usr/{bin/wxrc-$_wx_release,share/{aclocal,bakefile}}
  for file in "$pkgdir"/usr/share/{aclocal,bakefile/presets}/*
    do mv $file "$(echo $file | sed 's/\./-'$_wx_release'./')"; done
}
_update_symlinks() {
    stable="$(ls /usr/bin | grep wxrc- | sed 's/wxrc-\(.*\)/\1/' | sort | head -n 1)"
    dev="$(ls /usr/bin | grep wxrc- | sed 's/wxrc-\(.*\)/\1/' | sort | tail -n 1)"
    gtk="$(ls /usr/lib/wx/config | grep $stable | sed 's/.*\(gtk.\).*/\1/' | sort | tail -n 1)"
    if [[ -z "$stable" && -z "$dev" ]]; then
        # if there is no version left installed then delete all symlinks
        rm -f /usr/bin/wx-config
        rm -f /usr/bin/wxrc
        rm -f /usr/share/aclocal/wxwin.m4
        rm -f /usr/share/bakefile/presets/wx_presets.py
        rm -f /usr/share/bakefile/presets/wx_xrc.bkl
        rm -f /usr/share/bakefile/presets/wx_win32.bkl
        rm -f /usr/share/bakefile/presets/wx_unix.bkl
        rm -f /usr/share/bakefile/presets/wx.bkl
    else
        # symlink stable wx-config, favoring gtk3 over gtk2
        ln -sf /usr/lib/wx/config/$gtk-unicode-$stable /usr/bin/wx-config
        # symlink dev wxrc, wxwin.m4, and bakefile presets
        ln -sf /usr/bin/wxrc-$dev /usr/bin/wxrc
        ln -sf /usr/share/aclocal/wxwin-$dev.m4 /usr/share/aclocal/wxwin.m4
        ln -sf /usr/share/bakefile/presets/wx_presets-$dev.py /usr/share/bakefile/presets/wx_presets.py
        ln -sf /usr/share/bakefile/presets/wx_xrc-$dev.bkl /usr/share/bakefile/presets/wx_xrc.bkl
        ln -sf /usr/share/bakefile/presets/wx_win32-$dev.bkl /usr/share/bakefile/presets/wx_win32.bkl
        ln -sf /usr/share/bakefile/presets/wx_unix-$dev.bkl /usr/share/bakefile/presets/wx_unix.bkl
        ln -sf /usr/share/bakefile/presets/wx-$dev.bkl /usr/share/bakefile/presets/wx.bkl
    fi
}

post_install() {
    _update_symlinks
}

post_remove() {
    _update_symlinks
}

I actually don't plan to use the git version on the -dev packages, but later after the official 3.1.5 release is out I will replace it.

Offline

#4 2021-03-11 11:18:20

Lone_Wolf
Member
From: Netherlands, Europe
Registered: 2005-10-04
Posts: 9,024

Re: PKGBUILD RFC: making different versions of wxWidgets coexist

$ pacman -F /usr/bin/wx-config
usr/bin/wx-config is owned by extra/wxgtk2 3.0.5.1-2
$
$ pacman -F /usr/share/bakefile/presets/wx_presets.py
usr/share/bakefile/presets/wx_presets.py is owned by extra/wxgtk-common 3.0.5.1-2
$

Looks like your install script changes files tracked with pacman without using pacman .
That will cause problems.

Having multiple versions of java installed pose  a similar problem and the helper script archlinux-java is our method to deal with that.
Maybe that method also works for your usecase.
Check https://wiki.archlinux.org/index.php/Ja … etween_JVM and https://archlinux.org/packages/extra/an … me-common/

Last edited by Lone_Wolf (2021-03-11 11:18:56)


Disliking systemd intensely, but not satisfied with alternatives so focusing on taming systemd.
Did you use the guided installer ? If yes, I can't help you.

(A works at time B)  && (time C > time B ) ≠  (A works at time C)

Offline

#5 2021-03-11 12:51:45

dry
Member
Registered: 2020-12-23
Posts: 6

Re: PKGBUILD RFC: making different versions of wxWidgets coexist

I think you're looking at the official wxgtk packages, not my version, I didn't set pkgrel to 2 in my package. Also I don't install the wx-config and wxrc symlinks in the package, but using the install script.

Actually now that I run the same command I get the same thing because it is a remote database of files in the official repos. Technically in my version the symlinks shouldn't be owned by the packages...? Unless you mean that it could create problems if my version were to replace the current official one that users have installed on their systems.

Offline

#6 2021-03-15 18:05:14

Lone_Wolf
Member
From: Netherlands, Europe
Registered: 2005-10-04
Posts: 9,024

Re: PKGBUILD RFC: making different versions of wxWidgets coexist

Having files on your system that come with packages but aren't managed by pacman is  abad idea.

Working around the issue by moving files / symlinks out of the packages and into .install is possible, but makes maintenance harder.
f.e You would have to make sure changes in the .install script will propagate to all packages .

Splitting the files/symlinks to a separate package (wxgtk-runtime-common ?)  and using update_symlinks as a base for a helper script to switch between versions seems the best/cleanest method to me.
(It's also the method used by java-runtime-common / archlinux-java ) .


Disliking systemd intensely, but not satisfied with alternatives so focusing on taming systemd.
Did you use the guided installer ? If yes, I can't help you.

(A works at time B)  && (time C > time B ) ≠  (A works at time C)

Offline

Board footer

Powered by FluxBB