prestodb / presto-hadoop-apache2

Shaded version of Apache Hadoop 2.x for Presto

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Statically link (or package along) Glibc to libhadoop.so and libsnappy.so

sopel39 opened this issue · comments

Hi,

Current Presto master requires GlibC to be at least 2.12. However SLES11 and probably some other commercial distros still have GlibC 2.11 or lower. Would you mind if I compile GlibC statically into libhadoop and libsnappy so that no GlibC problems occur?

I'm not sure what the implications of that are, especially given that the JVM links to glibc. It appears that the JVM requires version GLIBC_2.2.5. I'm guessing you are reporting this because the previous version was built against CentOS 5 but the latest one used CentOS 6.

I did a quick search but didn't see an obvious way to force linking to an older version of glibc, especially with the Hadoop CMake stuff. Probably the easiest way is to rebuild the libraries on a CentOS 5 machine.

I think it should work because in SLES11 older Presto (< 130) works correctly even though GlibC is lower than 2.2.5.
Therefore will you accept pull request with both libhadoop and libsnappy statically linked with Glibc?

The previous one was built on CentOS 5 and worked, so we'll rebuild the libraries with that. We definitely do not want to statically link.

Great, thanks!

I found another issue here: the OpenSSL library on CentOS 5 (and likely SLES 11) are too old to be supported by Hadoop's native library. This seems to be fine since it is only used for one crypto codec and there is a fallback JCE version.