NX variant libraries RPM ld linking problem
All dynamically linked binaries on a linux system use ld.so to find the libraries they need at run time. The paths that are searched to find a given library are controlled by the ld.so cache, which is generated by ldconfig. ldconfig is controlled by /etc/ld.so.conf and /etc/ld.so.conf.d/*
The ld.so.conf.d directory contains files that list directories that should be searched for libraries. Those directories are searched first; if a given library is in none of them, ls.so falls back to the default system library locations, like /usr/lib and /usr/lib64.
The problem occurs when a variant of a widely used system library - in this case libX11.so - is in an alternate location, and the alternate location is specified in a file in ld.so.conf.d. Then all binaries end up using the variant lib. This is OK, until the variant lib has a problem.
The specific case we discovered has to do with NX. NX is a system for improving the performance of X windows when run across a high-latency network, eg a WAN. NX has its own variants of for example the standard X library, libX11.so.
The NX RPM that we are using (nx-3.5.0-5.el6.nux.x86_64) includes its own version of libX11.so and some other libraries:
[root@perlw2 etc]# rpm -q -l nx | grep libX11 /usr/lib64/nx/libX11.so.6 /usr/lib64/nx/libX11.so.6.2
The nx binaries link to these explicitly. So far, no problem.
However, the RPM nx-libs introduces a file under ld.so.conf.d:
[root@perlw2 conf]# rpm -q -l nx-libs-3.5.0.22-1.el6 | grep ld.so.conf.d /etc/ld.so.conf.d/nx-libs-x86_64.confwhich causes /usr/lib64/nx to be included in the library search path before the default /usr/lib64; thus, all binaries now link to the variant libX11.so.6.
The nx-libs RPM is not actually necessary for our system, it ended up getting installed because an NX RPM with a higher version number from another repository was available and required nx-libs as a dependency.
Linking to a different version of libX11.so is still not a problem, until version skew reached a point where other libraries wanted a symbol _XGetRequest that was defined by one variant libX11 but not the other.