Docker file not pulling in all needed libraries for proper lambda execution

Question

Docker file not pulling in all needed libraries for proper lambda execution

gmirsky opened this issue 4 years ago · comments

In order to get this to work I had to modify the Dockerfile.

I was running into the issue where the lambda function was not finding the proper run time libraries with the current code utilized by the make command.

I changed this from:

WORKDIR /tmp
RUN yumdownloader -x \*i686 --archlist=x86_64 clamav clamav-lib clamav-update json-c pcre2
RUN rpm2cpio clamav-0*.rpm | cpio -idmv
RUN rpm2cpio clamav-lib*.rpm | cpio -idmv
RUN rpm2cpio clamav-update*.rpm | cpio -idmv
RUN rpm2cpio json-c*.rpm | cpio -idmv
RUN rpm2cpio pcre*.rpm | cpio -idmv

To this:

WORKDIR /tmp
RUN yumdownloader -x \*i686 --archlist=x86_64 clamav clamav-lib clamav-update json-c pcre2 libprelude gnutls libtasn1 lib64nettle nettle
RUN rpm2cpio clamav-0*.rpm | cpio -idmv
RUN rpm2cpio clamav-lib*.rpm | cpio -idmv
RUN rpm2cpio clamav-update*.rpm | cpio -idmv
RUN rpm2cpio json-c*.rpm | cpio -idmv
RUN rpm2cpio pcre*.rpm | cpio -idmv
RUN rpm2cpio gnutls* | cpio -idmv
RUN rpm2cpio nettle* | cpio -idmv
RUN rpm2cpio lib* | cpio -idmv
RUN rpm2cpio *.rpm | cpio -idmv
RUN rpm2cpio libtasn1* | cpio -idmv

I'm not sure if this is the most elegant solution to the issue but it works.

Jacob · Answer 1 · Wed Jun 03 2020 04:29:16 GMT+0800 (China Standard Time)

It worked like a charm. Thanks a lot!

Jamie Armstrong · Answer 2 · Thu Jun 04 2020 15:31:12 GMT+0800 (China Standard Time)

This fixed the issue for me too, thanks @gmirsky.

iwt-nduesing · Answer 3 · Fri Jun 05 2020 19:49:09 GMT+0800 (China Standard Time)

I could solve it with the following which is a bit more precise in the paths:

WORKDIR /tmp
RUN yumdownloader -x \*i686 --archlist=x86_64 clamav clamav-lib clamav-update json-c pcre2 libprelude gnutls libtasn1 lib64nettle nettle
RUN rpm2cpio clamav-0*.rpm | cpio -idmv
RUN rpm2cpio clamav-lib*.rpm | cpio -idmv
RUN rpm2cpio clamav-update*.rpm | cpio -idmv
RUN rpm2cpio json-c*.rpm | cpio -idmv
RUN rpm2cpio pcre*.rpm | cpio -idmv
RUN rpm2cpio gnutls*.rpm | cpio -idmv
RUN rpm2cpio nettle*.rpm | cpio -idmv
RUN rpm2cpio libprelude*.rpm | cpio -idmv
RUN rpm2cpio libtasn1*.rpm | cpio -idmv

morufevari · Answer 4 · Wed Jul 01 2020 00:34:10 GMT+0800 (China Standard Time)

God bless you man. I've been pulling my hair for a fix to this issue all day until i stumbled on this.

Gregory Mirsky · Answer 5 · Wed Jul 01 2020 01:03:41 GMT+0800 (China Standard Time)

Glad I could help. I've been there, done that and have the scars to prove it.

rowanbarrie · Answer 6 · Thu Sep 17 2020 07:44:08 GMT+0800 (China Standard Time)

The above comments almost resolved the issue for me, although I also needed to download and convert the libtool-ltdl library:

WORKDIR /tmp
RUN yumdownloader -x \*i686 --archlist=x86_64 clamav clamav-lib clamav-update json-c pcre2 libprelude gnutls libtasn1 lib64nettle nettle libtool-ltdl
RUN rpm2cpio clamav-0*.rpm | cpio -idmv
RUN rpm2cpio clamav-lib*.rpm | cpio -idmv
RUN rpm2cpio clamav-update*.rpm | cpio -idmv
RUN rpm2cpio json-c*.rpm | cpio -idmv
RUN rpm2cpio pcre*.rpm | cpio -idmv
RUN rpm2cpio gnutls*.rpm | cpio -idmv
RUN rpm2cpio nettle*.rpm | cpio -idmv
RUN rpm2cpio libprelude*.rpm | cpio -idmv
RUN rpm2cpio libtasn1*.rpm | cpio -idmv
RUN rpm2cpio libtool-ltdl*.rpm | cpio -idmv

Satya Ramani · Answer 7 · Wed Nov 18 2020 15:11:37 GMT+0800 (China Standard Time)

Hello,
I am seeing this error after following all above steps, any help?

2020-11-18T02:07:18.192-05:00 | LibClamAV Error: cli_loaddbdir(): No supported database files found in /tmp/clamav_defs
| 2020-11-18T02:07:18.192-05:00CopyERROR: Can't open file or directory | ERROR: Can't open file or directory
| 2020-11-18T02:07:18.192-05:00 | Unexpected exit code from clamscan: 2.
| 2020-11-18T02:07:18.201-05:00 | [ERROR] Exception: Unexpected exit code from clamscan: 2.
| 2020-11-18T02:07:18.201-05:00 | Traceback (most recent call last): File "/var/task/scan.py", line 236, in lambda_handler scan_result, scan_signature = clamav.scan_file(file_path) File "/var/task/clamav.py", line 210, in scan_file raise Exception(msg)
| 2020-11-18T02:07:18.202-05:00 | END RequestId: 1e4b7dc6-bc7a-45a2-b723-e3908ef7feb2

Gopal Kankanhalli · Answer 8 · Wed Nov 18 2020 18:56:36 GMT+0800 (China Standard Time)

Same Error as above. All help appreciated

Starting clamscan of /tmp/clamscanbucket/Harmonics1.pdf.
clamscan output:
LibClamAV Error: cli_loaddbdir(): No supported database files found in /tmp/clamav_defs
ERROR: Can't open file or directory

----------- SCAN SUMMARY -----------
Known viruses: 0
Engine version: 0.102.4
Scanned directories: 0
Scanned files: 0
Infected files: 0
Data scanned: 0.00 MB
Data read: 0.00 MB (ratio 0.00:1)
Time: 0.057 sec (0 m 0 s)

Unexpected exit code from clamscan: 2.

[ERROR] Exception: Unexpected exit code from clamscan: 2.

Traceback (most recent call last):
  File "/var/task/scan.py", line 236, in lambda_handler
    scan_result, scan_signature = clamav.scan_file(file_path)
  File "/var/task/clamav.py", line 210, in scan_file
    raise Exception(msg)

Satya Ramani · Answer 9 · Fri Nov 20 2020 01:22:41 GMT+0800 (China Standard Time)

Same Error as above. All help appreciated

Starting clamscan of /tmp/clamscanbucket/Harmonics1.pdf.
clamscan output:
LibClamAV Error: cli_loaddbdir(): No supported database files found in /tmp/clamav_defs
ERROR: Can't open file or directory

----------- SCAN SUMMARY -----------
Known viruses: 0
Engine version: 0.102.4
Scanned directories: 0
Scanned files: 0
Infected files: 0
Data scanned: 0.00 MB
Data read: 0.00 MB (ratio 0.00:1)
Time: 0.057 sec (0 m 0 s)

Unexpected exit code from clamscan: 2.

[ERROR] Exception: Unexpected exit code from clamscan: 2.

Traceback (most recent call last):
  File "/var/task/scan.py", line 236, in lambda_handler
    scan_result, scan_signature = clamav.scan_file(file_path)
  File "/var/task/clamav.py", line 210, in scan_file
    raise Exception(msg)

I got rid of that error by replacing docker code with @iwt-nduesing suggested code above. Try and see if it works for you.

Satya Ramani · Answer 10 · Fri Nov 20 2020 01:24:23 GMT+0800 (China Standard Time)

Now I have now issue... its trying to scan but nothing happens
2020-11-18T02:27:58.033-05:00 Starting clamscan of /tmp/av-scan-bucket/example_3.json.
2020-11-18T02:32:57.171-05:00 END RequestId: 1ba83ac6-bd77-4d35-881e-d2489bfc7c87
2020-11-18T02:32:57.171-05:00 Copy
REPORT RequestId: 1ba83ac6-bd77-4d35-881e-d2489bfc7c87 Duration: 300748.32 ms Billed Duration: 300000 ms Memory Size: 1024 MB Max Memory Used: 1025 MB
REPORT RequestId: 1ba83ac6-bd77-4d35-881e-d2489bfc7c87 Duration: 300748.32 ms Billed Duration: 300000 ms Memory Size: 1024 MB Max Memory Used: 1025 MB
2020-11-18T02:32:57.171-05:00 2020-11-18T07:32:57.170Z 1ba83ac6-bd77-4d35-881e-d2489bfc7c87 Task timed out after 300.75 seconds
2020-11-18T02:34:44.374-05:00 START RequestId: 1ba83ac6-bd77-4d35-881e-d2489bfc7c87 Version: $LATEST
2020-11-18T02:34:44.585-05:00 Script starting at 2020/11/18 07:34:44 UTC
2020-11-18T02:34:45.321-05:00 Copy
Not downloading main.cvd because local md5 matches s3.
Not downloading main.cvd because local md5 matches s3.
2020-11-18T02:34:45.904-05:00 Not downloading daily.cvd because local md5 matches s3.
2020-11-18T02:34:45.995-05:00 Not downloading bytecode.cvd because local md5 matches s3.
2020-11-18T02:34:45.996-05:00 Starting clamscan of /tmp/av-scan-bucket/example_3.json.
2020-11-18T02:39:44.407-05:00 END RequestId: 1ba83ac6-bd77-4d35-881e-d2489bfc7c87

alages97 · Answer 11 · Mon Jan 25 2021 20:07:27 GMT+0800 (China Standard Time)

Thank you for this fix! The developers should add this into the original repository

Andrey Bronin · Answer 12 · Tue Apr 20 2021 04:41:57 GMT+0800 (China Standard Time)

I got the same issues (definitions update lambda fails). Fixed by updating Dockerfile. Thanks @gmirsky!

Abhi · Answer 13 · Fri Apr 30 2021 00:57:23 GMT+0800 (China Standard Time)

Thanks for your help @gmirsky!!