KDE Plasma: plasmashell using 8GB after update

Potomac · 2019-02-10 19:41:17

I think there a problem in your PKGBUILD, it creates a package named "qt5-base-git", but when I want to install this package pacman refuses because he found files owned by qt5-base (conflict files),

the problem can be here in the PKGBUILD :

  provides=(package_qt5-base)
  conflicts+=(package_qt5-base)

maybe it should be :

  provides=(qt5-base)
  conflicts+=(qt5-base)

or a simple solution is to keep the original name of the package (qt5-base) in your modified PKGBUILD

Last edited by Potomac (2019-02-10 19:50:40)

loqs · 2019-02-10 19:57:23

You are right with the correction to the PKGBUILD does the the amended PKGBUILD then install?

makepkg --repackage -fi

Potomac · 2019-02-10 21:44:08

I tried the second solution : keep the original name of the package,

there is another problem after typing these commands :

makepkg -ors
cd src/qt5-base/
git checkout v5.12.0
cd ../..
makepkg -ersi # build v5.12.0 locally make sure it works
cd src/qt5-base/
git bisect start
git bisect good
git checkout v5.12.1
cd ../..
makepkg -ersi # build v5.12.1 locally make sure it fails
cd src/qt5-base/
git bisect bad

after the "git bisect bad" I have an error:
git warns about a "src/qt5-base/config.tests/openssl/openssl.pro" file not registered,
and wants me to delete or rename this file,

so I rename this file, then I retype "git bisect bad", then git checkouts to the next bisect point,

and I type these commands

cd ../..
makepkg -ersi # test git bisection point

then I notice that makepkg seems to build the same git revision than the previous makepkg command,

maybe PKGBUILD file needs to be modified after each new git bisection point ?

Last edited by Potomac (2019-02-10 21:49:23)

Potomac · 2019-02-10 21:50:43

the build fails with a lot of error messages like

 "/bin/ld: qcolormap_x11.cpp:(.text+0x1ca9): undefined reference to `XFree'

we need to find a better procedure for git bisecting qt5-base

Last edited by Potomac (2019-02-10 21:50:51)

loqs · 2019-02-10 21:51:07

It should start off with the pkgver from the previous build and be updated by the pkgver function to 5.12.0.beta4.r249.gc0dd445564 did makepkg not produce a package with that version in the directory?
Edit:
What commit failed to build for you c0dd44556454ce0fa451f83decdad3931a6deaec ?

Last edited by loqs (2019-02-10 21:54:04)

Potomac · 2019-02-10 22:01:56

I lost the makepkg log in the console, so I can not answer to your question,

what I am sure is that at the beginning of the outpout of makepkg I read something like "build package revision XXX" where XXX is the revision number of the previous qt5-base package created by the previous makepkg command,

then I notice that the PKGBUILD file is modified automatically for the line "pkgver", currently it is :

pkgver=5.12.0.beta4.r249.gc0dd445564

I think something is wrong with your method, the first attempt of makepkg command works (when building v5.12.0), an archlinux package is produced,

but after when I make "git checkout v5.12.1" then the next makepkg command in your method will led to a build failure, with messages like "undefined reference to `XFree'" in the console

loqs · 2019-02-10 22:08:44

The following should produce the string used for the updated pkgver

cd src/qt5-base/
git describe --long --tags | sed 's/^v//;s/\([^-]*-g\)/r\1/;s/-/./g'

`makepkg -ersi` should reproduce the build failure output

Before making suggestions such as adding make clean before make but I would like to try and understand the issue more but you could try adding make clean before make.

Potomac · 2019-02-10 22:16:54

yes, adding "make clean" before "make" (or ./configure ?) may solve the problems,

I notice that when typing "git checkout v5.12.1" or "git bisect bad" then git will keep the files generated by a previous build (makepkg), which may cause problems for the next compilation ?

Last edited by Potomac (2019-02-10 22:17:49)

loqs · 2019-02-10 22:36:59

In general git only changes files it tracks. You can in many cases bisect and only rebuild code that has changed without issue.

git clean -dxf # will clean up all files not tracked by git and without using the standard ignore rules it should give you a clean src/qt5-base directory apart from the changes made in the prepare function.

Makepkg#Parallel_compilation will parallelize the build reduce build times.
Edit:
removed -n dry run option.

Last edited by loqs (2019-02-10 22:45:22)

Potomac · 2019-02-10 23:16:08

yes I use the 4 cores of my cpu intel quadcore Q9650 with this option in makepkg.conf :

MAKEFLAGS="-j4"

I use also ccache :

BUILDENV=(!distcc color ccache check !sign)

Using a compilation cache

The use of ccache can improve build times by caching the results of compilations for successive use.

Potomac · 2019-02-11 04:35:25

I did a git bisect and I manage to find the faulty commit, it's the commit :

[9d90c0edac91b35ec96646fd3e6cdd339639ca79] QImage: merge the size calculations with proper (non-UB) checks

https://github.com/qt/qtbase/commit/9d9 … 339639ca79

If I revert this commit then the bug disapears, all is ok, I created a patch which reverts the faulty commit :
https://bugreports.qt.io/secure/attachm … ecks.patch

I rebuilt qt5-base 5.12.1 with this patch and now all is ok, here is the modified PKGBUILD which contains my patch :

# Maintainer: Felix Yan <felixonmars@archlinux.org>
# Contributor: Andrea Scarpino <andrea@archlinux.org>

pkgbase=qt5-base
pkgname=(qt5-base qt5-xcb-private-headers)
_qtver=5.12.1
pkgver=${_qtver/-/}
pkgrel=3
arch=('x86_64')
url='https://www.qt.io'
license=('GPL3' 'LGPL3' 'FDL' 'custom')
pkgdesc='A cross-platform application and UI framework'
depends=('libjpeg-turbo' 'xcb-util-keysyms' 'xcb-util-renderutil' 'libgl' 'fontconfig' 'xdg-utils'
         'shared-mime-info' 'xcb-util-wm' 'libxrender' 'libxi' 'sqlite' 'xcb-util-image' 'mesa'
         'tslib' 'libinput' 'libxkbcommon-x11' 'libproxy' 'libcups' 'double-conversion')
makedepends=('libfbclient' 'mariadb-libs' 'sqlite' 'unixodbc' 'postgresql-libs' 'alsa-lib' 'gst-plugins-base-libs'
             'gtk3' 'libpulse' 'cups' 'freetds' 'vulkan-headers')
optdepends=('qt5-svg: to use SVG icon themes'
            'qt5-translations: for some native UI translations'
            'postgresql-libs: PostgreSQL driver'
            'mariadb-libs: MariaDB driver'
            'unixodbc: ODBC driver'
            'libfbclient: Firebird/iBase driver'
            'freetds: MS SQL driver'
            'gtk3: GTK platform plugin')
conflicts=('qtchooser')
groups=('qt' 'qt5')
_pkgfqn="${pkgbase/5-/}-everywhere-src-${_qtver}"
source=("https://download.qt.io/official_releases/qt/${pkgver%.*}/${_qtver}/submodules/${_pkgfqn}.tar.xz"
        qt-delayed-highlight.patch::https://code.qt.io/cgit/qt/qtbase.git/patch/?id=f8f0f3ee
        qtbug-69310.patch::"https://code.qt.io/cgit/qt/qtbase.git/patch/?id=fcba9fa8"
        qt5-base-nouveau-freeze.patch::"http://code.qt.io/cgit/qt/qtbase.git/patch?id=97600d2c"
	"revert_size_calculations_with_proper_non-UB_checks.patch")
sha256sums=('533078ce02678988576e224cb3844979907950cf83e0fda3364bc1d5701c9049'
            '0080b1e208090c729018db1ac224e5940d06a38b9c28c918a892b01f84a82cf9'
            '2186ce7b8d6310ebc005da9b00dcc88388df3e8b83bff37230fa1bb9253c4970'
            'e0a7cdda2ebc403e2b3a62b3d104d2dd3f8b4e79d92395cf3c6419ebe6d8b19c'
            'SKIP')

prepare() {
  cd ${_pkgfqn}

  # Build qmake using Arch {C,LD}FLAGS
  # This also sets default {C,CXX,LD}FLAGS for projects built using qmake
  sed -i -e "s|^\(QMAKE_CFLAGS_RELEASE.*\)|\1 ${CFLAGS}|" \
    mkspecs/common/gcc-base.conf
  sed -i -e "s|^\(QMAKE_LFLAGS_RELEASE.*\)|\1 ${LDFLAGS}|" \
    mkspecs/common/g++-unix.conf

  patch -p1 -i ../qt-delayed-highlight.patch # https://codereview.qt-project.org/#/c/249881/
  patch -p1 -i ../qtbug-69310.patch # https://bugreports.qt.io/browse/QTBUG-69310
  patch -Rp1 -i ../qt5-base-nouveau-freeze.patch # Revert commit that causes freezes on nouveau https://bugreports.qt.io/browse/QTBUG-73691
  patch -p1 -i ../revert_size_calculations_with_proper_non-UB_checks.patch # https://bugreports.qt.io/browse/QTBUG-73691
}

build() {
  cd ${_pkgfqn}

  ./configure -confirm-license -opensource -v \
    -prefix /usr \
    -docdir /usr/share/doc/qt \
    -headerdir /usr/include/qt \
    -archdatadir /usr/lib/qt \
    -datadir /usr/share/qt \
    -sysconfdir /etc/xdg \
    -examplesdir /usr/share/doc/qt/examples \
    -plugin-sql-{psql,mysql,sqlite,odbc,ibase} \
    -system-sqlite \
    -openssl-linked \
    -nomake examples \
    -no-rpath \
    -optimized-qmake \
    -dbus-linked \
    -system-harfbuzz \
    -journald \
    -no-use-gold-linker \
    -reduce-relocations
  make
}

package_qt5-base() {
  pkgdesc='A cross-platform application and UI framework'

  cd ${_pkgfqn}
  make INSTALL_ROOT="${pkgdir}" install

  install -Dm644 LICENSE* -t "$pkgdir"/usr/share/licenses/$pkgbase

  # Drop QMAKE_PRL_BUILD_DIR because reference the build dir
  find "${pkgdir}/usr/lib" -type f -name '*.prl' \
    -exec sed -i -e '/^QMAKE_PRL_BUILD_DIR/d' {} \;

  # Fix wrong qmake path in pri file
  sed -i "s|${srcdir}/${_pkgfqn}|/usr|" \
    "${pkgdir}"/usr/lib/qt/mkspecs/modules/qt_lib_bootstrap_private.pri

  # Symlinks for backwards compatibility
  for b in "${pkgdir}"/usr/bin/*; do
    ln -s $(basename $b) "${pkgdir}"/usr/bin/$(basename $b)-qt5
  done
}

package_qt5-xcb-private-headers() {
  pkgdesc='Private headers for Qt5 Xcb'

  depends=("qt5-base=$pkgver")
  optdepends=()
  groups=()
  conflicts=()

  cd ${_pkgfqn}
  install -d -m755 "$pkgdir"/usr/include/qtxcb-private
  cp -r src/plugins/platforms/xcb/*.h "$pkgdir"/usr/include/qtxcb-private/
}

put the patch in the same directory of PKGBUILD file, then type "makepkg -c", it will create a package "qt5-base-5.12.1-3-x86_64.pkg.tar.xz", you can also download the PKGBUILD with the patch :
https://bugs.archlinux.org/task/61638?getfile=17204

Last edited by Potomac (2019-02-11 04:41:47)

kokoko3k · 2019-02-11 13:17:54

Thank you so much for your efforts!

edit: (all of you )

Last edited by kokoko3k (2019-02-11 13:18:24)

Storm · 2019-02-12 11:14:18

Thank you everyone who worked on solving this! I'm sorry I couldn't be of much help but my knowledge is limited, and the affected PC is my work machine so I cannot afford to mess around with it too much, it has to be in working order so I just kept it reverted to a known good state.

How can I tell when is either the patch added to Arch's package or the upstream bug fixed so that it's safe to just update my system the regular way without building patched Qt myself?

Potomac · 2019-02-12 15:29:19

for the moment I have just a workaround by reverting the faulty commit (see my patch for rebuilding qt5-base archlinux package https://bugs.archlinux.org/task/61638?getfile=17204),

a Qt5 developper is trying to find the definitive fix (Thiago Macieira) :
https://bugreports.qt.io/browse/QTBUG-73691

so you have 2 solutions :

- rebuild the qt5-base archlinux package with my patch each time a new version is released

- or block qt5-updates in the pacman.conf file

# Pacman won't upgrade packages listed in IgnorePkg and members of IgnoreGroup
IgnorePkg   = qt5-*
#IgnoreGroup =

but the second solution will not work longer, because partial-updates are not supported in archlinux, so I hope that Qt5 developpers will find soon the solution,

you can also switch to another desktop theme which is not affected by the bug, breeze theme or a variant of the oxygen theme, like "oxygen future", very similar to oxygen theme :
https://store.kde.org/p/1213571/

Last edited by Potomac (2019-02-12 15:36:01)

RickDeckard · 2019-02-12 19:56:05

After something of a forced hiatus, I've done my own research and found that apparently the Glassified theme also uses some of the same elements of Oxygen that are messing up here. So I'd recommend to anyone that's running it to also switch away from it. You don't actively have to be running Oxygen for this to happen.

kokoko3k · 2019-02-12 19:58:52

are we talking about oxygen decoration or widget style?

RickDeckard · 2019-02-12 21:03:40

kokoko3k wrote:

are we talking about oxygen decoration or widget style?

The decoration accessible from System Settings > Workspace Theme > Appearance/Desktop Theme. The widget style, as per "watch -n 1 free -mh", appears to be unaffected.

Potomac · 2019-02-13 14:58:46

an archlinux user has made a test with valgrind, and shows huge memory leak of plasmashell :
https://bugs.archlinux.org/task/61741

note the reference to QImage Qt5 class, which is involved in the faulty commit found in the git bisect :

Peak memory usage trace:
#-----------
snapshot=55
#-----------
time=10514349267
mem_heap_B=8785823950
mem_heap_extra_B=9383130
mem_stacks_B=0
heap_tree=detailed
n2: 8785823950 (heap allocation functions) malloc/new/new[], --alloc-fns, etc.
n1: 8623439016 0x6520097: QImageData::create(QSize const&, QImage::Format) (in /usr/lib/libQt5Gui.so.5.12.1)
n2: 8623439016 0x65200EB: QImage::QImage(QSize const&, QImage::Format) (in /usr/lib/libQt5Gui.so.5.12.1)
n2: 8592471880 0x6520124: QImage::QImage(int, int, QImage::Format) (in /usr/lib/libQt5Gui.so.5.12.1)
n1: 8591856548 0x65633CC: QRasterPlatformPixmap::resize(int, int) (in /usr/lib/libQt5Gui.so.5.12.1)
n1: 8591856548 0x6562837: QPlatformPixmap::create(int, int, QPlatformPixmap::PixelType) (in /usr/lib/libQt5Gui.so.5.12.1)
n2: 8591856548 0x6559906: QPixmap::doInit(int, int, int) (in /usr/lib/libQt5Gui.so.5.12.1)
n2: 8591070780 0x4D4B198: ??? (in /usr/lib/libKF5Plasma.so.5.55.0)
n2: 8590365468 0x4D4C1F2: Plasma::Svg::image(QSize const&, QString const&) (in /usr/lib/libKF5Plasma.so.5.55.0)
n1: 8590311172 0x13FEDB3E: ??? (in /usr/lib/qt/qml/org/kde/plasma/core/libcorebindingsplugin.so)
n1: 8590311172 0x4FB1BCE: QQuickWindowPrivate::updateDirtyNode(QQuickItem*) (in /usr/lib/libQt5Quick.so.5.12.1)
n1: 8590311172 0x4FB2042: QQuickWindowPrivate::updateDirtyNodes() (in /usr/lib/libQt5Quick.so.5.12.1)
n2: 8590311172 0x4FB3575: QQuickWindowPrivate::syncSceneGraph() (in /usr/lib/libQt5Quick.so.5.12.1)
n1: 8589949160 0x4F58E77: ??? (in /usr/lib/libQt5Quick.so.5.12.1)
n1: 8589949160 0x4F5A24B: ??? (in /usr/lib/libQt5Quick.so.5.12.1)
n1: 8589949160 0x4F5DB56: ??? (in /usr/lib/libQt5Quick.so.5.12.1)
n1: 8589949160 0x6A3496A: ??? (in /usr/lib/libQt5Core.so.5.12.1)
n1: 8589949160 0x7CD6A9B: start_thread (in /usr/lib/libpthread-2.28.so)
n0: 8589949160 0x7130B21: clone (in /usr/lib/libc-2.28.so)
n0: 362012 in 1 place, below massif's threshold (1.00%)
n0: 54296 in 1 place, below massif's threshold (1.00%)
n0: 705312 in 2 places, all below massif's threshold (1.00%)
n0: 785768 in 4 places, all below massif's threshold (1.00%)
n0: 615332 in 6 places, all below massif's threshold (1.00%)
n0: 30967136 in 3 places, all below massif's threshold (1.00%)
n0: 162384934 in 3666 places, all below massif's threshold (1.00%)

it would be interesting to write a simple program which triggers immediately the memory leak, by using QImage Qt5 class, in the same way that oxygen theme managed to trigger the memory leak, it can help Qt5 developper to understand better the bug and help to fix it

Last edited by Potomac (2019-02-13 14:59:30)

Storm · 2019-02-14 10:21:27

RickDeckard wrote:

After something of a forced hiatus, I've done my own research and found that apparently the Glassified theme also uses some of the same elements of Oxygen that are messing up here.

I have the same bug but I do NOT use any parts of Oxygen. However, I use "Simple Glass" for panel. Maybe that's why I have it then?

(I can't test now, my machine is reverted to earlier state because it's my work machine and I need to work now)

Last edited by Storm (2019-02-14 10:21:56)

RickDeckard · 2019-02-14 17:59:33

Storm wrote:

I have the same bug but I do NOT use any parts of Oxygen. However, I use "Simple Glass" for panel. Maybe that's why I have it then?
(I can't test now, my machine is reverted to earlier state because it's my work machine and I need to work now)

I believe so. Maybe after work try "systemsettings5" while logged in from another window manager and revert your panel to Breeze?

Kicer · 2019-02-16 08:54:04

Is it safe to upgrade system now? Was proper package reverted or patch reverting broken commit applied to arch packages?

SanskritFritz · 2019-02-16 11:01:19

Yes, the fix was applied to tha package, it is safe to upgrade (having said that, note, it is never "safe" ).

UPDATE: sorry I was wrong, see next comment.

Last edited by SanskritFritz (2019-02-16 19:12:10)

loqs · 2019-02-16 12:35:41

SanskritFritz wrote:

Yes, the fix was applied to tha package, it is safe to upgrade (having said that, note, it is never "safe" ).

https://bugreports.qt.io/browse/QTBUG-73691 is not https://bugreports.qt.io/browse/QTBUG-73715 fixed by qt5-base 5.12.1-2
Edit:
Fixed second bug url

Last edited by loqs (2019-02-16 12:37:53)

Potomac · 2019-02-17 00:05:07

in fact there is no official patch approved by Qt5 developers,

the archlinux maintainer (Felix Yan) has decided for qt5-base 5.12.-2 to revert a commit for the "nouveau bug" (https://bugreports.qt.io/browse/QTBUG-73715)

patch -Rp1 -i ../qt5-base-nouveau-freeze.patch # Revert commit that causes freezes on nouveau https://bugreports.qt.io/browse/QTBUG-73691

I think for the "oxygen bug" (https://bugreports.qt.io/browse/QTBUG-73691) the archlinux maintainer should apply the same logic, by reverting the faulty commit already identified by a git bisect, it will cause no problems for those who don't have the bug,

for the moment we have to rebuild ourself the qt5-base package, I have posted the PKGBUILD with the patch :
https://bugs.archlinux.org/task/61638?getfile=17204

Kicer · 2019-02-17 08:15:26

I don't know how are your systems behaving guys, but on mine RAM usage eventually drops to normal levels. Yet I still fill like it was freezing a little bit from time to time.

Arch Linux

#26 2019-02-10 19:41:17

Re: KDE Plasma: plasmashell using 8GB after update

#27 2019-02-10 19:57:23

Re: KDE Plasma: plasmashell using 8GB after update

#28 2019-02-10 21:44:08

Re: KDE Plasma: plasmashell using 8GB after update

#29 2019-02-10 21:50:43

Re: KDE Plasma: plasmashell using 8GB after update

#30 2019-02-10 21:51:07

Re: KDE Plasma: plasmashell using 8GB after update

#31 2019-02-10 22:01:56

Re: KDE Plasma: plasmashell using 8GB after update

#32 2019-02-10 22:08:44

Re: KDE Plasma: plasmashell using 8GB after update

#33 2019-02-10 22:16:54

Re: KDE Plasma: plasmashell using 8GB after update

#34 2019-02-10 22:36:59

Re: KDE Plasma: plasmashell using 8GB after update

#35 2019-02-10 23:16:08

Re: KDE Plasma: plasmashell using 8GB after update

#36 2019-02-11 04:35:25

Re: KDE Plasma: plasmashell using 8GB after update

#37 2019-02-11 13:17:54

Re: KDE Plasma: plasmashell using 8GB after update

#38 2019-02-12 11:14:18

Re: KDE Plasma: plasmashell using 8GB after update

#39 2019-02-12 15:29:19

Re: KDE Plasma: plasmashell using 8GB after update

#40 2019-02-12 19:56:05

Re: KDE Plasma: plasmashell using 8GB after update

#41 2019-02-12 19:58:52

Re: KDE Plasma: plasmashell using 8GB after update

#42 2019-02-12 21:03:40

Re: KDE Plasma: plasmashell using 8GB after update

#43 2019-02-13 14:58:46

Re: KDE Plasma: plasmashell using 8GB after update

#44 2019-02-14 10:21:27

Re: KDE Plasma: plasmashell using 8GB after update

#45 2019-02-14 17:59:33

Re: KDE Plasma: plasmashell using 8GB after update

#46 2019-02-16 08:54:04

Re: KDE Plasma: plasmashell using 8GB after update

#47 2019-02-16 11:01:19

Re: KDE Plasma: plasmashell using 8GB after update

#48 2019-02-16 12:35:41

Re: KDE Plasma: plasmashell using 8GB after update

#49 2019-02-17 00:05:07

Re: KDE Plasma: plasmashell using 8GB after update

#50 2019-02-17 08:15:26

Re: KDE Plasma: plasmashell using 8GB after update

Board footer