YARA module

[Julien Vehent]

YARA is a client tool for scanning file system for IOCs. Go-Yara implements binding to the yara C library that we could ship as a module via cgo.

On the client side, we should load YARA files into MIG actions, possibly compressed, the same way we load scribe files.

note: this is not an easy one.

[Julien Vehent]

@hillu: do you know if the Yara library can be statically compiled into a Go binary, or if the only way to ship it is the dynamically link to libyara?

[Hilko Bengen]

* Julien Vehent [:ulfr]:

@hillu: do you know if the Yara library can be statically compiled into a Go binary, or if the only way to ship it is the dynamically link to libyara?

It's possible to link statically. Use something like go build -x --ldflags '-extldflags "-static"' However, on Linux the resulting binary would still be dynamically linked against GNU libc which may not be what you want for MiG. Using libmusl instead is a solid option, though. It is also possible to cross-build a static libyara for Windows on Linux (and presumably MacOS) hosts using MinGW. Cheers, -Hilko

[Aaron Meihm]

I've got a working implementation of this for Linux in #335 which uses go-yara and looks good.

Doing the same for the OSX builds seems rather troublesome.

Out of the box with the changes, just trying to compile it for OSX:

GOOS=darwin GOARCH=amd64 GO15VENDOREXPERIMENT=1 go build  -tags netgo -o bin/darwin/amd64/mig-agent-20170329-0.e3b4219.dev"" -ldflags "-X mig.ninja/mig.Version=20170329-0.e3b4219.dev  -extldflags '-static '" mig.ninja/mig/mig-agent
# mig.ninja/mig/vendor/github.com/hillu/go-yara
vendor/github.com/hillu/go-yara/compiler.go:15:10: fatal error: 'yara.h' file not found
#include <yara.h>
         ^
1 error generated.
make: *** [mig-agent] Error 2

Add a few flags so we can find things

GOOS=darwin GOARCH=amd64 GO15VENDOREXPERIMENT=1 go build  -tags netgo -o bin/darwin/amd64/mig-agent-20170329-0.e3b4219.dev"" -ldflags "-X mig.ninja/mig.Version=20170329-0.e3b4219.dev  -extldflags '-static '" mig.ninja/mig/mig-agent
# mig.ninja/mig/mig-agent
/usr/local/go/pkg/tool/darwin_amd64/link: running clang failed: exit status 1
ld: library not found for -lcrt0.o
clang: error: linker command failed with exit code 1 (use -v to see invocation)

make: *** [mig-agent] Error 2

This leads me to:

http://stackoverflow.com/questions/5259249/creating-static-mac-os-x-c-build and other assorted info

The executable can be dynamic but we want the yara library in there so we don't need to ship it, unclear at this stage what is required, will look into it more and see if it is possible.

[Hilko Bengen]

I have had success building binaries using go-yara for OSX, with everything but the libc linked statically. I just made sure that I configured yara with --disable-shared --disable-magic --disable-cuckoo --without-crypto and then passed --ldflags '-w -s' to go build.

This was on MacOSX 10.7 (I think) and using GCC, though.

[Aaron Meihm]

@hillu indeed, this does work thank you. I was originally trying to use the brew yara package and have go build prefer the .a file, what I ended up doing is just compiling it as described manually and given just the presence of the static archive for libyara, the compiler makes use of that.

[Aaron Meihm]

Merged in #335