Docker build fails on MacOS Catalina 10.15.7

[Blake Luther]

Building the docker image fails when trying to compile hyperscan:

$ docker build --rm=true --tag=deepfenceio/secretscanning:latest -f Dockerfile .

[snip]

#6 341.4 [ 54%] Building CXX object CMakeFiles/hs_compile.dir/src/nfagraph/ng_literal_decorated.cpp.o
#6 341.6 [ 54%] Building C object CMakeFiles/hs_exec_shared_corei7.dir/src/nfa/truffle.c.o
#6 343.6 [ 54%] Building C object CMakeFiles/hs_exec_shared_corei7.dir/src/som/som_runtime.c.o
#6 343.9 [ 54%] Building C object CMakeFiles/hs_exec_shared_corei7.dir/src/som/som_stream.c.o
#6 344.3 [ 54%] Building CXX object CMakeFiles/hs_compile.dir/src/nfagraph/ng_mcclellan.cpp.o
#6 345.9 [ 54%] Building C object CMakeFiles/hs_exec_shared_core2.dir/src/nfa/sheng.c.o
#6 346.6 [ 54%] Building CXX object CMakeFiles/hs_compile.dir/src/nfagraph/ng_limex.cpp.o
#6 347.0 [ 54%] Building C object CMakeFiles/hs_exec_shared_corei7.dir/src/rose/block.c.o
#6 347.5 [ 55%] Building CXX object CMakeFiles/hs_compile.dir/src/nfagraph/ng_limex_accel.cpp.o
#6 348.8 [ 55%] Building CXX object CMakeFiles/hs_compile.dir/src/nfagraph/ng_misc_opt.cpp.o
#6 350.8 [ 55%] Building C object CMakeFiles/hs_exec_shared_core2.dir/src/nfa/shufti.c.o
#6 350.8 [ 55%] Building C object CMakeFiles/hs_exec_shared_corei7.dir/src/rose/catchup.c.o
#6 353.6 [ 55%] Building C object CMakeFiles/hs_exec_shared_core2.dir/src/nfa/tamarama.c.o
#6 389.1 c++: internal compiler error: Killed (program cc1plus)
#6 389.1 Please submit a full bug report,
#6 389.1 with preprocessed source if appropriate.
#6 389.1 See <file:///usr/share/doc/gcc-7/README.Bugs> for instructions.
#6 389.1 CMakeFiles/hs_compile.dir/build.make:1536: recipe for target 'CMakeFiles/hs_compile.dir/src/nfagraph/ng_mcclellan.cpp.o' failed
#6 389.1 make[2]: *** [CMakeFiles/hs_compile.dir/src/nfagraph/ng_mcclellan.cpp.o] Error 4
#6 389.1 make[2]: *** Waiting for unfinished jobs....
#6 389.2 [ 56%] Building C object CMakeFiles/hs_exec_shared_corei7.dir/src/rose/init.c.o
#6 392.1 [ 56%] Building C object CMakeFiles/hs_exec_shared_core2.dir/src/nfa/truffle.c.o
#6 392.3 [ 57%] Building C object CMakeFiles/hs_exec_shared_core2.dir/src/som/som_runtime.c.o
#6 392.3 [ 57%] Building C object CMakeFiles/hs_exec_shared_core2.dir/src/som/som_stream.c.o
#6 392.3 [ 57%] Building C object CMakeFiles/hs_exec_shared_corei7.dir/src/rose/stream.c.o
#6 393.0 [ 57%] Building C object CMakeFiles/hs_exec_shared_corei7.dir/src/rose/match.c.o
#6 394.3 CMakeFiles/Makefile2:512: recipe for target 'CMakeFiles/hs_compile.dir/all' failed
#6 394.3 make[1]: *** [CMakeFiles/hs_compile.dir/all] Error 2
#6 394.3 [ 57%] Building C object CMakeFiles/hs_exec_shared_core2.dir/src/rose/block.c.o
#6 394.8 [ 57%] Building C object CMakeFiles/hs_exec_shared_core2.dir/src/rose/catchup.c.o
#6 395.1 [ 57%] Building C object CMakeFiles/hs_exec_shared_core2.dir/src/rose/init.c.o
#6 398.7 [ 57%] Building C object CMakeFiles/hs_exec_shared_corei7.dir/src/rose/program_runtime.c.o
#6 398.9 [ 57%] Building C object CMakeFiles/hs_exec_shared_corei7.dir/src/util/multibit.c.o
#6 399.9 [ 57%] Building C object CMakeFiles/hs_exec_shared_core2.dir/src/rose/stream.c.o
#6 400.1 [ 58%] Building C object CMakeFiles/hs_exec_shared_core2.dir/src/rose/match.c.o
#6 400.2 [ 58%] Building C object CMakeFiles/hs_exec_shared_core2.dir/src/rose/program_runtime.c.o
#6 403.2 [ 58%] Building C object CMakeFiles/hs_exec_shared_core2.dir/src/util/multibit.c.o
#6 404.2 [ 58%] Building C object CMakeFiles/hs_exec_shared_core2.dir/src/util/simd_utils.c.o
#6 406.8 [ 58%] Building C object CMakeFiles/hs_exec_shared_core2.dir/src/util/state_compress.c.o
#6 408.1 [ 58%] Building C object CMakeFiles/hs_exec_shared_core2.dir/src/database.c.o
#6 408.1 [ 58%] Building C object CMakeFiles/hs_exec_shared_corei7.dir/src/util/simd_utils.c.o
#6 408.4 [ 59%] Building C object CMakeFiles/hs_exec_shared_corei7.dir/src/util/state_compress.c.o
#6 411.4 [ 59%] Building C object CMakeFiles/hs_exec_shared_corei7.dir/src/database.c.o
#6 421.1 [ 59%] Built target hs_exec_shared_corei7
#6 421.3 [ 59%] Built target hs_exec_shared_core2
#6 421.3 make: *** [all] Error 2
#6 421.3 Makefile:129: recipe for target 'all' failed
------
executor failed running [/bin/sh -c mkdir -p /usr/local/include/ &&     cd /usr/local/include/ &&     git clone https://github.com/intel/hyperscan.git &&     mkdir /usr/local/include/hs &&     cd /usr/local/include/hs &&     export MAKEFLAGS=-j$(nproc) &&     cmake -DBUILD_STATIC_AND_SHARED=1 /usr/local/include/hyperscan &&     echo "/usr/local/lib" | tee --append /etc/ld.so.conf.d/usrlocal.conf &&     cd /usr/local/include/hs && make && make install]: exit code: 2

I'm relatively new to docker, so if you can provide the steps to grab any log files you might be interested in, I'm happy to do so. Pulling the docker image from docker hub appears to work fine, and is a valid workaround.

[Shyam Krishnaswamy]

Hello;

Thank you for bringing this issue to our notice.

At this point in time, Linux systems are the support build environment. We will work to add additional build environments in the future.

Having said that, patches are most welcome !

Thanks
Shyam

[Dr. Swarup Kumar Sahoo]

Looks like your system is running out of memory during hyperscan build. Can you try with "MAKEFLAGS=-j2" option in Dockerfile to see if you are able to build it without any errors?

[Blake Luther]

Looks like your system is running out of memory during hyperscan build. Can you try with "MAKEFLAGS=-j2" option in Dockerfile to see if you are able to build it without any errors?

Yep, this fixed it for me. Successful compile on MacOS. Thanks!