Possible regression in glibc breaking old version of java

alex.forencich · 2017-05-29 15:52:50

I have a rather large piece of proprietary software that I need to run (specifically, Xilinx ISE 14.7) that contains an old version of java (1.5.0_11). Previously, this has worked just fine. However, sometime relatively recently, something broke:

$ /opt/Xilinx/14.7/ISE_DS/ISE/java/lin64/jre/bin/java -version
Error occurred during initialization of VM
java/lang/NoClassDefFoundError: java/lang/Object

If I mount an old arch ISO, bind mount in /opt, chroot in, and run the same file from there, it works:

# /opt/Xilinx/14.7/ISE_DS/ISE/java/lin64/jre/bin/java -version
java version "1.5.0_11"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_11-b03)
Java HotSpot(TM) 64-Bit Server VM (build 1.5.0_11-b03, mixed mode)

I ran the binary with strace and found some nonsensical calls to open:

stat("/opt/Xilinx/14.7/ISE_DS/ISE/java/lin64/jre/lib/rt.jar", {st_mode=S_IFREG|0644, st_size=35757927, ...}) = 0
open("\270\302\3A", O_RDONLY)           = -1 ENOENT (No such file or directory)
stat("/opt/Xilinx/14.7/ISE_DS/ISE/java/lin64/jre/lib/i18n.jar", 0x7fff6a1fd320) = -1 ENOENT (No such file or directory)
stat("/opt/Xilinx/14.7/ISE_DS/ISE/java/lin64/jre/lib/sunrsasign.jar", 0x7fff6a1fd320) = -1 ENOENT (No such file or directory)
stat("/opt/Xilinx/14.7/ISE_DS/ISE/java/lin64/jre/lib/jsse.jar", {st_mode=S_IFREG|0644, st_size=487141, ...}) = 0
open("\270\302\3A", O_RDONLY)           = -1 ENOENT (No such file or directory)
stat("/opt/Xilinx/14.7/ISE_DS/ISE/java/lin64/jre/lib/jce.jar", {st_mode=S_IFREG|0644, st_size=82642, ...}) = 0
open("\270\302\3A", O_RDONLY)           = -1 ENOENT (No such file or directory)
stat("/opt/Xilinx/14.7/ISE_DS/ISE/java/lin64/jre/lib/charsets.jar", {st_mode=S_IFREG|0644, st_size=8723578, ...}) = 0
open("\270\302\3A", O_RDONLY)           = -1 ENOENT (No such file or directory)
stat("/opt/Xilinx/14.7/ISE_DS/ISE/java/lin64/jre/classes", 0x7fff6a1fd320) = -1 ENOENT (No such file or directory)

On a machine running RHEL and I presume on the old version of Arch, there are a number of lstat calls between each stat and each open, working up the directory tree, then the open accesses the same file as the call to stat, like so:

[pid 25132] stat("/opt/Xilinx/14.7/ISE_DS/ISE/java/lin64/jre/lib/rt.jar", {st_mode=S_IFREG|0644, st_size=35757927, ...}) = 0
[pid 25132] lstat("/opt", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
[pid 25132] lstat("/opt/Xilinx", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
[pid 25132] lstat("/opt/Xilinx/14.7", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
[pid 25132] lstat("/opt/Xilinx/14.7/ISE_DS", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
[pid 25132] lstat("/opt/Xilinx/14.7/ISE_DS/ISE", {st_mode=S_IFDIR|0775, st_size=4096, ...}) = 0
[pid 25132] lstat("/opt/Xilinx/14.7/ISE_DS/ISE/java", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
[pid 25132] lstat("/opt/Xilinx/14.7/ISE_DS/ISE/java/lin64", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
[pid 25132] lstat("/opt/Xilinx/14.7/ISE_DS/ISE/java/lin64/jre", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
[pid 25132] lstat("/opt/Xilinx/14.7/ISE_DS/ISE/java/lin64/jre/lib", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
[pid 25132] lstat("/opt/Xilinx/14.7/ISE_DS/ISE/java/lin64/jre/lib/rt.jar", {st_mode=S_IFREG|0644, st_size=35757927, ...}) = 0
[pid 25132] open("/opt/Xilinx/14.7/ISE_DS/ISE/java/lin64/jre/lib/rt.jar", O_RDONLY) = 5
[pid 25132] fstat(5, {st_mode=S_IFREG|0644, st_size=35757927, ...}) = 0
[pid 25132] lseek(5, 0, SEEK_END)       = 35757927
[pid 25132] mmap(NULL, 35757927, PROT_READ, MAP_SHARED, 5, 0) = 0x7f7a13457000
[pid 25132] close(5)                    = 0
[pid 25132] mmap(NULL, 430080, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f7a133ee000
[pid 25132] stat("/opt/Xilinx/14.7/ISE_DS/ISE/java/lin64/jre/lib/i18n.jar", 0x7ffd3d9986d0) = -1 ENOENT (No such file or directory)
...

I believe this is a glibc issue as it's not a kernel issue (the test with the mounted iso is running under the same kernel - I did not boot the iso) and because glibc is the only thing that java accesses outside of /opt:

open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
open("/usr/lib/libpthread.so.0", O_RDONLY|O_CLOEXEC) = 3
open("/usr/lib/libdl.so.2", O_RDONLY|O_CLOEXEC) = 3
open("/usr/lib/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
...
open("/usr/lib/libm.so.6", O_RDONLY|O_CLOEXEC) = 3
...
open("/usr/lib/libnsl.so.1", O_RDONLY|O_CLOEXEC) = 3
...
open("/usr/lib/libnss_files.so.2", O_RDONLY|O_CLOEXEC) = 3

I attempted various combinations of LD_PRELOAD and LD_LIBRARY_PATH with a slightly older version of glibc, but this only resulted in segfaults.

Does anyone have any ideas about either 1. somehow running this thing with an old version of glibc or 2. isolating specifically what broke so it can be fixed?

R00KIE · 2017-05-29 17:32:57

This might not be what you want to hear but I've learned the same way you have that trying to run old proprietary software with arch is a losing proposition.

What I've done to run such programs is have a full install of centos 5 that I enter/chroot into with systemd-nspawn. I then run a vncserver inside and use it to run all the proprietary programs. This ensures two things, the programs should work as intended as they claimed support for rhel 5 and they will keep working because there is no need to update the centos files so it will not break due to updates. It can still break if systemd-nspawn has bugs, which it had for me a couple of times before.

alex.forencich · 2017-05-29 18:17:33

Is there a tutorial for how to set that up? I have never used containers before. I have used VMs, but I'm highly opposed to installing this thing in a VM because the software requires 20 GB of disk space and at least 8 GB of RAM.

Also, is it possible to bind-mount /home and /opt into a container and have applications open up on the local desktop directly instead of over VNC?

R00KIE · 2017-05-29 20:09:04

I did come to my setup bit by bit, if there is a tutorial I'm not aware of one. I don't know if what I've described is a container it's just something I've cobbled up to solve my problem.

It is not a virtual machine, I don't "boot" it with systemd-nspawn, I just use systemd-nspawn as a chroot on steroids. I do bind mount some directories inside the container (lets call it that) so it's possible. As for having the applications open directly on the desktop it is theoretically possible but that includes extra setup work and if I recall correctly I believe someone has tried before and things didn't work. However I don't remember if it was showing the host's programs inside the container or if it was showing the container's apps on the host (most probably the later).

If you decide you want to try this, the general idea is:
- Install a supported distro to an external disk
- Make sure to install everything needed to make the programs you want run
- Use that install's root as the container/chroot root
- Fine tune scripts to run/launch the container and also scripts to run when starting the container, this can include scripts to launch vnc for example. You might also need to use either scripts or edit the shell's rc file to setup the environment if you manage to get programs to display on the host.
- ? (I'm probably forgetting something here)
- Profit.

ribalda · 2018-01-02 21:01:58

I had the same issue with Debian and Xilinx 14.7. After losing a day of my life (thanks again Xilinx) I figured out that:

cp /tmp/x86_64-linux-gnu/libm-2.24.so /opt/Xilinx/14.7/ISE_DS/ISE/lib/lin64/libm.so.6

Fixes the issue for me (libm-2.24 from debian's libc6_2.24-11+deb9u1_amd64.deb)

I hope that you can do something similar on your system. On the future I am going to put all the proprietary s***t in a container.

PS: I am running a 2 hours bitstream build as I write this message, if it fails i will let you know, so far it has past the point where it used to crash

alex.forencich · 2018-01-02 21:05:26

Interesting idea. I'll give that a shot at some point. However, I have been running in to issues with the licensing subsystem latching on to the wrong network card, so I'm working on a container based setup that only has 1 network card to hopefully solve both of these problems once and for all.

R00KIE · 2018-01-02 21:32:24

This is an old thread[1] and the previous two posts don't add any useful information. One seems to apply to debian, this are the support forums for Arch Linux not debian[2].

Closing.

[1] https://wiki.archlinux.org/index.php/Co … bumping.22
[2] https://wiki.archlinux.org/index.php/Co … .2Aonly.2A

Arch Linux

#1 2017-05-29 15:52:50

Possible regression in glibc breaking old version of java

#2 2017-05-29 17:32:57

Re: Possible regression in glibc breaking old version of java

#3 2017-05-29 18:17:33

Re: Possible regression in glibc breaking old version of java

#4 2017-05-29 20:09:04

Re: Possible regression in glibc breaking old version of java

#5 2018-01-02 21:01:58

Re: Possible regression in glibc breaking old version of java

#6 2018-01-02 21:05:26

Re: Possible regression in glibc breaking old version of java

#7 2018-01-02 21:32:24

Re: Possible regression in glibc breaking old version of java

Board footer