Giters

lambgeo / docker-lambda

AWS Lambda friendly GDAL Docker images and AWS Lambda layer

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

libsqlite3 linking broken on 3.8 images

jasongi-ac opened this issue · comments

commented

Having a similar issue to this one: #56

Steps for a simple reproduction:

docker run --rm -it ghcr.io/lambgeo/lambda-gdal:3.8-python3.11
cd /var/task
pip install django -t /var/task
python -c 'from django.contrib.gis import gdal; print("success")'

workaround is to add LD_PRELOAD in your lambda env vars like so:

LD_PRELOAD=/var/task/lib/libsqlite3.so

You can test this fix by doing the following simple repro:

docker run --rm -it ghcr.io/lambgeo/lambda-gdal:3.8-python3.11
cd /var/task
pip install django -t /var/task
LD_PRELOAD=/opt/lib/libsqlite3.so python -c 'from django.contrib.gis import gdal; print("success")'

I suspect the culprit is one of these libs outside of /opt/lib/:

bash-4.2# find /  -name "libsql*"
/opt/lib/libsqlite3.so.0.8.6
/opt/lib/libsqlite3.la
/opt/lib/libsqlite3.so.0
/opt/lib/libsqlite3.so
/usr/lib64/libsqlite3.so.0.8.6
/usr/lib64/libsqlite3.so.0
/var/lang/lib/libsqlite3.so.0.8.6
/var/lang/lib/libsqlite3.so.0
/var/lang/lib/libsqlite3.so

I think it can be fixed by specifying either CMAKE_LIBRARY_PATH or SQLITE3_LIBRARY

commented

The culprit is /var/lang/lib/libsqlite3.so.0

bash-4.2# ldd /opt/lib/libgdal.so | grep sqlite
        libsqlite3.so.0 => /var/lang/lib/libsqlite3.so.0 (0x00007f33c6e5f000)

edit:
Actually I don't think that's the case, all this cmake stuff really goes over my head.
Appears to be dynamically being set at runtime. You can fix it by setting LD_PRELOAD or even LD_LIBRARY_PATH works. Would be better if this worked without that though, since the only viable option in lambda is LD_PRELOAD. Isn't this the reasoning behind doing the rpath stuff, so this doesn't happen?

commented

after a couple of false starts adding

RUN patchelf --force-rpath --set-rpath '$ORIGIN' $PREFIX/lib/libgdal.so

to the docker build seems to do the trick.
PR here: #74

commented

confirmed

 docker run --platform=linux/amd64 --entrypoint bash --rm ghcr.io/lambgeo/lambda-gdal:3.8 ldd /opt/lib/libgdal.so | grep "sql"
        libsqlite3.so.0 => /lib64/libsqlite3.so.0 (0x0000004006e28000)
commented

@jasongi really appreciate your effort here, I have myself spends days on similar issues 😭

Sadly, as noted in your PR I think there is a bigger issue (see proj lib). I'm not sure I have enough free time at the moment to work on this but I'll try