Java applications rendering issues with vmwgfx on X11

Tuetuopay · 2025-11-11 23:18:14

Hi,

A few months ago, the xf86-video-vmware X11 driver was retired, in favor of modesetting. Ever since, Java-based applications using Swing are basically unusable. They weren't great with the vmware-specific driver, but somewhat workable with a combination of jvm flags as documented on the Java Troubleshooting wiki page. Long story short, the symptoms are somewhat random text rendering, ranging from "half of it missing" to "i'm lucky to have a single letter rendered". When interacting (clicking or hovering) interactive UI elements, sometimes they can be brought back into rendering.

This is all in a VMware workstation VM that's nearing 10 years old, and is my daily work VM. It's running i3 thus X11. Note: i'm still running X11 for i3, as sway segfaults in less than a minute in this VM. Making sway works is an acceptable solution to this issue, but I don't know how either.

Here are a few examples. The reference screenshots come from a Debian 13 VM running Gnome on Wayland, on the same host with the same VM settings.

Quartus Example 1: left is freshly opened, middle is after hovering the combo box, right is what it should look like.

Quartus Example 2 : left is freshly opened, middle is after clicking and hovering, right is what it should look like.

Ghidra Example : left is the main UI straight after open (some text is rendered, some is not, sometimes only a single color is rendered), right is when interacting with the menus on the top (notice the text that disappeared south of the open menus)

I've tried every combination of JVM flags mentioned in the Java Troubleshooting page to no avail.

The X11 config is so plain it does not exist: Xorg detects the display, loads modesetting properly, as well as detecting the vmwgfx DRI driver.

So yeah, shot in the dark, if anyone has an idea I'm all ears. Thanks!

Lone_Wolf · 2025-11-12 11:45:34

xf86-video-vmware is present in the aur.
Have you tried running that to verify if the switch from ddx to modesetting is indeed the culprit ?

archaea · 2025-11-12 19:25:44

As a test, any change if you run these applications with LIBGL_ALWAYS_SOFTWARE=1 environment variable set?

seth · 2025-11-12 19:39:33

xf86-video-vmware is present in the aur.

And it will break the server, https://bbs.archlinux.org/viewtopic.php … 9#p2260789
Don't install that.

Tuetuopay · 2025-11-13 19:40:22

Lone_Wolf wrote:

xf86-video-vmware is present in the aur.
Have you tried running that to verify if the switch from ddx to modesetting is indeed the culprit ?

As mentioned by @seth, it's broken, and I was actually involved in dropping it I even have a branch somewhere with fixes for it (like a ~2s startup delay for any application). Anyways, I tried, for science, both the regular and -git packages, and they both fail the same way they did when it got dropped: a segfault somewhere in the code, related to upstream mesa dropping a basically required library. You can read this gitlab thread to learn more about this driver. It's in AUR as known broken and left for someone to pick up.

archaea wrote:

As a test, any change if you run these applications with LIBGL_ALWAYS_SOFTWARE=1 environment variable set?

Huh my shell history remembered that I did try that in the past, when trying to get Ghidra working properly. I don't recall it being in the env var soup that worked under the VMware driver. Just tried it naked and with the soup on both Ghidra (native java 25) and Quartus (bundled Java 8), and the results are the same. Thanks for the suggestion, I forgot about it.

Note: the soup i'm talking about is sombinations of
- -Dawt.useSystemAAFontSettings=on
- -Dswing.aatext=true
- -Dswing.defaultlaf=com.sun.java.swing.plaf.gtk.GTKLookAndFeel
- -Dswing.crossplatformlaf=com.sun.java.swing.plaf.gtk.GTKLookAndFeel
- -Dsun.java2d.opengl=false -Dawt.useSystemAAFontSettings=lcd

EDIT: fixed formatting. my bbcode days are far behind.

Last edited by Tuetuopay (2025-11-13 19:41:09)

seth · 2025-11-13 20:40:23

Just tried it naked and with the soup on both Ghidra (native java 25) and Quartus (bundled Java 8), and the results are the same.

Same "working" or same "not working"?
If it has no impact, does it if you https://wiki.archlinux.org/title/Java#B … erformance ?
Have you tried to disable the 3D acceleration in the VM settings?

Tuetuopay · 2025-11-13 21:46:39

Same "not working" unfortunately.

As for turning off 3D acceleration, it does indeed work! But at a significant cost: anything using opengl is dog slow, which includes my usual terminal (kitty) that consumes 20% CPU, and has noticeable input lag. Using a non-gpu terminal like terminator does not have the issue, so not a dealbreaker. What is though are applications that KiCAD that run in seconds per frame, not frames per second, territory. Heck, even moving the mouse around eats 10% of the CPU. All of this was expected, however not to that extent.

So yeah, it's weird that 2D acceleration-related settings in the JVM don't work. Perhaps there could be some magic Xorg settings to disable all form of acceleration for some applications/windows/etc given it's fine in the Debian VM with Java apps running under Xwayland? I don't know enough about it.

Thanks a lot. At least I have a workaround (sluggish, or keep a debian VM for the occasional quartus megawizard or ghidra). Or dig into why sway segfaults in my face if I dare use the computer. Or ditch this setup entirely and run arch natively.

seth · 2025-11-13 22:33:23

export JDK_JAVA_OPTIONS='-Dsun.java2d.opengl=true'
export LIBGL_ALWAYS_SOFTWARE=1
quartus

exporting LIBGL_ALWAYS_SOFTWARE has no impact in this situation?

Tuetuopay · 2025-11-13 22:59:33

Sadly none. I tried both =1 as you mentioned and =true as mentioned by the mesa docs, no dice. Still no text

It's quite puzzling considering what the variable does and what the issue appears to be.

Last edited by Tuetuopay (2025-11-13 23:09:17)

seth · 2025-11-14 08:29:37

Have you tried different JDK versions (assuming the clients support such)?
https://wiki.archlinux.org/title/Java#L … va_version

Tuetuopay · 2025-11-16 15:28:20

Quartus is bundled with its own Java 8 runtime, and I don't really want to start digging into how to override it (esp. considering the breaking stuff between 8 and 11), so I'll assume I cannot change it.

As for Ghidra, it requires a rather recent version of Java so I just tested it against Java 21 and 25, still the same rendering issues.

Arch Linux

#1 2025-11-11 23:18:14

Java applications rendering issues with vmwgfx on X11

#2 2025-11-12 11:45:34

Re: Java applications rendering issues with vmwgfx on X11

#3 2025-11-12 19:25:44

Re: Java applications rendering issues with vmwgfx on X11

#4 2025-11-12 19:39:33

Re: Java applications rendering issues with vmwgfx on X11

#5 2025-11-13 19:40:22

Re: Java applications rendering issues with vmwgfx on X11

#6 2025-11-13 20:40:23

Re: Java applications rendering issues with vmwgfx on X11

#7 2025-11-13 21:46:39

Re: Java applications rendering issues with vmwgfx on X11

#8 2025-11-13 22:33:23

Re: Java applications rendering issues with vmwgfx on X11

#9 2025-11-13 22:59:33

Re: Java applications rendering issues with vmwgfx on X11

#10 2025-11-14 08:29:37

Re: Java applications rendering issues with vmwgfx on X11

#11 2025-11-16 15:28:20

Re: Java applications rendering issues with vmwgfx on X11

Board footer