lambci:python-3.7 can't open dll (.so) lib

[Tran Xuan Tu]

Hi there,

I am trying to create a layer which includes dremio-odbc driver and pyodbc to let my Python lambda function connect with Dremio database.

I was able to create a layer with everything include with structure like this:

|-- ODBCDataSources
|-- bin
|-- dremio-odbc
|-- include
|-- lib
|-- odbc.ini
|-- odbcinst.ini
|-- python
`-- share
opt/python
|-- pyodbc-4.0.30.dist-info
`-- pyodbc.cpython-37m-x86_64-linux-gnu.so

Unfortunately whenever I try to run my lambda function as follow:

docker run \
    -it \
    --rm \
    --entrypoint bash \
    -v "$PWD"/var/task:/var/task \
    -v "$PWD"/opt:/opt \
    lambci/lambda:python3.7

With lambda function code as:

import sys
import os
import pyodbc

print('Successfully import pyodbc')

host = '<HOST>'
port = 31010
uid = 'dev'
driver = 'Dremio ODBC Driver 64-bit'
pwd = '<PASSWORD>'

def lambda_handler(event, context):
    print("ok")
    os.system("ls -l /opt/")
    # conn_str = 'DRIVER={ODBC Driver 17 for SQL Server};SERVER=' + server + ';UID=' + username + ';PWD=' + password
    conn_str = f"Driver={driver};ConnectionType=Direct;HOST={host};PORT={port};AuthenticationType=Plain;UID={uid};PWD={pwd}"
    print(conn_str)
    db = pyodbc.connect(conn_str, autocommit=True)
    print(db.execute('show databases').fetchall())

Here is the error log:

Successfully import pyodbc
START RequestId: e49ba986-812c-189c-19f0-85a2e1804e9c Version: $LATEST
ok
-rwxrwxrwx 1 sbx_user1051 990 57222867 Sep 16 00:44 /opt/dremio-odbc/lib64/libdrillodbc_sb64.so


Driver=Dremio ODBC Driver 64-bit;ConnectionType=Direct;HOST=52.58.148.113;PORT=31010;AuthenticationType=Plain;UID=dev;PWD=@123Test
[ERROR] Error: ('01000', "[01000] [unixODBC][Driver Manager]Can't open lib '/opt/dremio-odbc/lib64/libdrillodbc_sb64.so' : file not found (0) (SQLDriverConnect)")
Traceback (most recent call last):
  File "/var/task/lambda_function.py", line 20, in lambda_handler
    db = pyodbc.connect(conn_str, autocommit=True)
END RequestId: e49ba986-812c-189c-19f0-85a2e1804e9c
REPORT RequestId: e49ba986-812c-189c-19f0-85a2e1804e9c  Init Duration: 346.13 ms        Duration: 108.57 ms     Billed Duration: 109 ms Memory Size: 1536 MB    Max Memory Used: 24 MB  

{"errorType":"Error","errorMessage":"('01000', \"[01000] [unixODBC][Driver Manager]Can't open lib '/opt/dremio-odbc/lib64/libdrillodbc_sb64.so' : file not found (0) (SQLDriverConnect)\")","stackTrace":["  File \"/var/task/lambda_function.py\", line 20, in lambda_handler\n    db = pyodbc.connect(conn_str, autocommit=True)\n"]}

Given that /opt/dremio-odbc/lib64/libdrillodbc_sb64.so does exist, the issue looks like lambda doesn't have right permission to open the lib. I even tried to chmod 777 for entire files in /opt but no luck.

Hope if someone faced this issue before and could give me some hint!

Regards,
Tran