- C++14 Compiler (LLVM/Clang and GCC are tested)
- CMake 3.4.3+
- LLVM 7.X or LLVM 8.X
- Curl
- Doxygen (for docs and install)
The majority of the CoreGen library and tool dependencies are contained
within the source tree. The only outside library dependencies are met
with the libstdc++ (from GCC or LLVM) and the LLVM libraries (for
StoneCutter support). As a result, building CoreGen is generally
considered to be fairly simple. The current tested platforms
are listed below, each with their respective build options.
Generally speaking, CoreGen is built using the following steps:
- Clone the CoreGen repository. There are currently no git dependencies
- Create a
builddirectory within the CoreGen source tree (and change to that directory) - Execute cmake to generate the target-specific makefiles
- Execute the build
Building CoreGen with Clang/LLVM can be done by setting the CC and CXX environment variables.
- Override default compiler
export CC=clang
export CXX=clang++
We currently support building and installing on LTS versions of Ubuntu. While it is certainly possible to build/install on intermediate release and/or development versions, we do not currently test these scenarios.
- Install the necessary apt repositories
wget https://apt.llvm.org/llvm.sh
chmod +x llvm.sh
sudo ./llvm.sh 8
- Install the system dependencies:
sudo apt-get install -y libllvm-8-ocaml-dev libllvm8 llvm-8 llvm-8-dev llvm-8-doc llvm-8-examples llvm-8-runtime clang-8 clang-tools-8 clang-8-doc libclang-common-8-dev libclang-8-dev libclang1-8 clang-format-8 python-clang-8 zlib1g zlib1g-dev bison libcurl4-gnutls-dev zlib1g-dev
- Clone the CoreGen repository
git clone https://github.com/opensocsysarch/CoreGen.git
- Setup your build tree
cd CoreGen
mkdir build
cd build
- Execute CMake to generate the makefiles
cmake -DLLVM_DIR=/usr/lib/llvm-8/cmake ../
- Execute the build
make
- Install the necessary apt repositories
wget https://apt.llvm.org/llvm.sh
chmod +x llvm.sh
sudo ./llvm.sh 8
- Install the system dependencies:
sudo apt-get install -y clang-8 lldb-8 lld-8 libllvm8 llvm-8 llvm-8-dev llvm-8-doc llvm-8-examples llvm-8-runtime python-dev libcurl4 libcurl4-gnutls-dev zlib1g-dev
- Clone the CoreGen repository
git clone https://github.com/opensocsysarch/CoreGen.git
- Setup your build tree
cd CoreGen
mkdir build
cd build
- Execute CMake to generate the makefiles
cmake -DLLVM_DIR=/usr/lib/llvm-8/cmake ../
- Execute the build
make
- Install the necessary apt repositories
wget https://apt.llvm.org/llvm.sh
chmod +x llvm.sh
sudo ./llvm.sh 8
- Install the system dependencies:
sudo apt-get install -y clang-8 lldb-8 lld-8 libllvm8 llvm-8 llvm-8-dev llvm-8-doc llvm-8-examples llvm-8-runtime python-dev libcurl4 libcurl4-gnutls-dev
- Clone the CoreGen repository
git clone https://github.com/opensocsysarch/CoreGen.git
- Setup your build tree
cd CoreGen
mkdir build
cd build
- Execute CMake to generate the makefiles
cmake -DLLVM_DIR=/usr/lib/llvm-8/cmake ../
- Execute the build
make
Centos7 (CentOS 7.4, 1704) has somewhat of an older toolchain. As a result, the build requires some rather unique system packages. Utilize the following steps on vanilla CentOS 7.4 systems.
- Install the system dependencies:
sudo yum -y install cmake centos-release-scl texlive texlive-*.noarch llvm-static llvm-toolset-7-llvm-devel llvm-toolset-7-build llvm-toolset-7-cmake llvm-toolset-7-lldb-devel llvm-toolset-7-libomp-devel llvm-toolset-7-clang-devel llvm-toolset-7-llvm-static zlib-devel devtoolset-8 python-devel bison libcurl-devel
- [Optional] For those wishing to build CentOS7 RPMS, you also need the following packages.
sudo yum install rpm-build rpmdevtools
- Enable the SCL LLVM environment (this will create a fresh bash shell with the correct paths)
scl enable llvm-toolset-7 devtoolset-8 bash - Clone the CoreGen repository
git clone https://github.com/opensocsysarch/CoreGen.git
- Setup your build tree
cd CoreGen
mkdir build
cd build
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/rh/llvm-toolset-7.0/root/lib64/
- Execute CMake to generate the makefiles
cmake -DLLVM_DIR=/opt/rh/llvm-toolset-7.0/root/usr/lib64/cmake/llvm ../
- Execute the build
make
Centos8 has a native toolchain that is much more adept at building modern C++ applications. Utilize the following steps on vanilla CentOS 8 systems.
- Install the system dependencies:
sudo yum -y install cmake gcc-toolset-9 texlive texlive-*.noarch bison libcurl-devel zlib-devel llvm llvm-toolset llvm-libs llvm-devel llvm-static graphviz ncurses-*
- [Optional] For those wishing to build Centos8 RPMS, you also need the following packages.
sudo yum install rpm-build rpmdevtools
- Clone the CoreGen repository
git clone https://github.com/opensocsysarch/CoreGen.git
- Setup your build tree
cd CoreGen
mkdir build
cd build
- Execute CMake to generate the makefiles
cmake -DLLVM_DIR=/usr/lib64/cmake/llvm ../
- Execute the build
make
The Darwin (OSX) build is currently tested on 10.13.6. However, these
build steps are also known to work on older versions of Darwin providing
that the necessary packages are installed. For this tutorial, we use
the HomeBrew package manager in order to fulfill these dependencies.
- Install the system dependencies (assuming HomeBrew is installed):
brew install --cask mactex
brew install cmake
brew install llvm@8
brew install pkg-config
- Clone the CoreGen repository
git clone https://github.com/opensocsysarch/CoreGen.git
- Setup your build tree
cd CoreGen
mkdir build
cd build
- Execute CMake to generate the makefiles
cmake -DLLVM_DIR=/usr/local/opt/llvm\@8/lib/cmake/llvm ../
- Execute the build
make
Once the CoreGen infrastructure has been installed, it is advisable that you initialize the CoreGen archive.
The CoreGen archive provides LLVM compiler templates and libraries that will be made available to all
CoreGen-derived projects without repeatedly downloading large packages. The contents of the
archive is determined by a master YAML file. The master
YAML file is located in the installation directory at ~/INSTALL_PREFIX/archive/master.yaml. Initializing
the archive will initiate downloads from a number of public source code repositories, including LLVM. Note
that anyone initializing the archive must ensure that sufficient disk space is available and they
have the necessary permissions to populate the archive directory. All the archive contents will be downloaded
to ~/INSTALL_PREFIX/archive/. Initializing the archive can be performed using the CGCLI tool as follows:
cgcli --init /path/to/installation/archive/master.yaml
- -DCMAKE_BUILD_TYPE=Release : Builds a release build with optimization (Default)
- -DCMAKE_BUILD_TYPE=Debug : Builds with debugging options (-O0 -g, etc)
- -DCMAKE_INSTALL_PREFIX=/path/to/install : Forces alternate install locations
- -DLLVM_DIR=/path/to/llvm/cmake/scripts : Sets the path for the LLVM cmake scripts. This must be set for the build to be successful. This is generally difficult to derive automatically.
- -DBUILD_DOCUMENTATION=ON : Enables Doxygenized documentaition.
make docwill build the documentation (requires Doxygen to be installed)
- -DYAML_PATH=/path/to/alternate/yaml-cpp : Set an alternate yaml-cpp library path
- -DCOREGEN_BUILD_RPM=ON : Enable RPM builds (CentOS)
- -DCOREGEN_BUILD_DEB=ON : Enable DEB builds (Ubuntu)
- -DCOREGEN_BUILD_DMG=ON : Enable DMG builds (Darwin)
- -DCOREGEN_BUILD_TGZ=ON : Enable TGZ builds (generic)
- -DCOREGEN_MEMORY_SANITY=ON : Enables Clang's memory sanitizer testing. You MUST be using Clang as your target C++ compiler!!
- -DBUILD_ALL_TESTING=ON : Enable ALL unit tests for CoreGen
- -DBUILD_COREGEN_CGCLI_TESTING=ON : Enable CGCLI test harness
- -DBUILD_COREGEN_CGCLI_SPECDOC_TESTING=ON : Enable CGCLI SpecDoc pass test harness
- -DBUILD_COREGEN_DAG_TESTING=ON : Enable IR DAG test harness
- -DBUILD_COREGEN_DAG_DOT_TESTING=ON : Enable DAG's DOT file test harness
- -DBUILD_COREGEN_DAG_FULLPASS_TESTING=ON : Enable DAG's full pass test harness
- -DBUILD_COREGEN_DAG_INDIVIDUALPASS_TESTING=ON : Enable DAG's individual pass test harness
- -DBUILD_COREGEN_DAG_INDIVIDUALPASS_KNOWNPASS_TESTING=ON : Enable DAG's individual pass test harness for known correct tests
- -DBUILD_COREGEN_DAG_INDIVIDUALPASS_KNOWNPASS_STATSPASS_TESTING=ON : Enable DAG's individual pass test harness for the StatsPass
- -DBUILD_COREGEN_DAG_INDIVIDUALPASS_KNOWNPASS_MULTSOCPASS_TESTING=ON : Enable DAG's individual pass test harness for the MultSoCPass
- -DBUILD_COREGEN_DAG_INDIVIDUALPASS_KNOWNPASS_DANGLINGREGIONPASS_TESTING=ON : Enable DAG's individual pass test harness for the DanglingRegionPass
- -DBUILD_COREGEN_DAG_INDIVIDUALPASS_KNOWNPASS_DANGLINGNODEPASS_TESTING=ON : Enable DAG's individual pass test harness for the DanglingNodePass
- -DBUILD_COREGEN_DAG_INDIVIDUALPASS_KNOWNPASS_ENCODINGCOLLISIONPASS_TESTING=ON : Enable DAG's individual pass test harness for the EncodingCollisionPass
- -DBUILD_COREGEN_DAG_INDIVIDUALPASS_KNOWNPASS_REGIDXPASS_TESTING=ON : Enable DAG's individual pass test harness for the RegIdxPass
- -DBUILD_COREGEN_DAG_INDIVIDUALPASS_KNOWNPASS_CACHELEVELPASS_TESTING=ON : Enable DAG's individual pass test harness for the CacheLevelPass
- -DBUILD_COREGEN_DAG_INDIVIDUALPASS_KNOWNPASS_REGSAFETYPASS_TESTING=ON : Enable DAG's individual pass test harness for the RegSafetyPass
- -DBUILD_COREGEN_DAG_INDIVIDUALPASS_KNOWNPASS_SPADSAFETYPASS_TESTING=ON : Enable DAG's individual pass test harness for the SpadSafetyPass
- -DBUILD_COREGEN_DAG_INDIVIDUALPASS_KNOWNPASS_ICACHECHECKERPASS_TESTING=ON : Enable DAG's individual pass test harness for the RegSafetyPass
- -DBUILD_COREGEN_DAG_INDIVIDUALPASS_KNOWNPASS_L1SHAREDPASS_TESTING=ON : Enable DAG's individual pass test harness for the RegSafetyPass
- -DBUILD_COREGEN_DAG_INDIVIDUALPASS_KNOWNPASS_REGFIELDPASS_TESTING=ON : Enable DAG's individual pass test harness for the RegSafetyPass
- -DBUILD_COREGEN_DAG_INDIVIDUALPASS_KNOWNPASS_CORESAFETYPASS_TESTING=ON : Enable DAG's individual pass test harness for the RegSafetyPass
- -DBUILD_COREGEN_DAG_INDIVIDUALPASS_KNOWNPASS_COMMSAFETYPASS_TESTING=ON : Enable DAG's individual pass test harness for the RegSafetyPass
- -DBUILD_COREGEN_DAG_INDIVIDUALPASS_KNOWNPASS_MANDATORYFIELDPASS_TESTING=ON : Enable DAG's individual pass test harness for the RegSafetyPass
- -DBUILD_COREGEN_DAG_INDIVIDUALPASS_KNOWNPASS_ENCODINGGAPPASS_TESTING=ON : Enable DAG's individual pass test harness for the RegSafetyPass
- -DBUILD_COREGEN_DAG_INDIVIDUALPASS_KNOWNPASS_PINSTSAFETYPASS_TESTING=ON : Enable DAG's individual pass test harness for the RegSafetyPass
- -DBUILD_COREGEN_DAG_INDIVIDUALPASS_KNOWNFAIL_TESTING=ON : Enable DAG's individual pass test harness for known fail tests
- -DBUILD_COREGEN_DAG_INDIVIDUALPASS_KNOWNFAIL_ASP_TESTING=ON : Enable DAG's individual ASP pass test harness for known fail tests
- -DBUILD_COREGEN_DAG_DELETENODE_TESTING=ON : Enable DAG's node deletion test harness
- -DBUILD_COREGEN_PLUGIN_TESTING=ON : Enable plugin test harness
- -DBUILD_COREGEN_YAML_TESTING=ON : Enable yaml test harness
- -DBUILD_COREGEN_YAML_READER_TESTING=ON : Enable yaml reader test harness
- -DBUILD_COREGEN_YAML_READER_MISSING_DATA_TESTING=ON : Enable yaml reader test harness for yaml with known missing data
- -DBUILD_COREGEN_YAML_READER_INDENT_FAIL_TESTING=ON : Enable yaml reader test harness for yaml with known issues in indentations
- -DBUILD_COREGEN_YAML_READER_CONV_FAIL_TESTING=ON : Enable yaml reader test harness for yaml with known issues in type conversions
- -DBUILD_COREGEN_YAML_WRITER_TESTING=ON : Enable yaml writer test harness
- -DBUILD_COREGEN_STONECUTTER_TESTING=ON : Enables the StoneCutter test harness
- -DBUILD_COREGEN_STONECUTTER_CLI_TESTING=ON : Enables the StoneCutter CLI test harness
- -DBUILD_COREGEN_STONECUTTER_PIPELINER_TESTING=ON : Enables the StoneCutter pipeliner test harness
- -DBUILD_COREGEN_STONECUTTER_PARSER_TESTING=ON : Enables the StoneCutter parser test harness
- -DBUILD_COREGEN_STONECUTTER_PARSER_KNOWNFAIL_TESTING=ON : Enables the StoneCutter parser known fail test harness
- -DBUILD_COREGEN_STONECUTTER_PARSER_KNOWNPASS_MULTIFILE_TESTING=ON : Enables the StoneCutter parser multi-file test harness
- -DBUILD_COREGEN_STONECUTTER_SIGMAP_TESTING=ON : Enables the StoneCutter SigMap pass testing
- -DBUILD_COREGEN_STONECUTTER_CODEGEN_TESTING=ON : Enables the StoneCutter codegen test harness
- -DBUILD_COREGEN_STONECUTTER_CODEGEN_KNOWNPASS_TESTING=ON : Enables the StoneCutter codegen knownpass test harness
- -DBUILD_COREGEN_STONECUTTER_CODEGEN_KNOWNFAIL_TESTING=ON : Enables the StoneCutter codegen knownfail test harness
- -DBUILD_COREGEN_STONECUTTER_SCSIGV_TESTING=ON : Enables SCSigV sigmap test harness
- -DBUILD_COREGEN_STONECUTTER_SCSIGV_KNOWNPASS_TESTING=ON : Enables SCSigV sigmap knowpass test harness
- -DBUILD_COREGEN_STONECUTTER_SCSIGV_KNOWNFAIL_TESTING=ON : Enables SCSigV sigmap knowfail test harness
- -DBUILD_COREGEN_CODEGEN_TESTING=ON : Enables the CodeGen test harness
- -DBUILD_COREGEN_CODEGEN_STONECUTTERIMPL_TESTING=ON : Enables the CodeGen StoneCutter implementation test harness
- -DBUILD_COREGEN_ARCHIVE_TESTING=ON : Enables teh archive library test harness
- -DBUILD_COREGEN_POAR_TESTING=ON : Enables all the POAR tests
- -DBUILD_COREGEN_POAR_CLI_TESTING=ON : Enable the POAR CLI tests
- -DBUILD_COREGEN_POAR_TEXT_TESTING=ON : Enable the POAR text output tests
- -DBUILD_COREGEN_POAR_YAML_TESTING=ON : Enable the POAR YAML output tests
- -DBUILD_COREGEN_POAR_XML_TESTING=ON : Enable the POAR XML output tests
- -DBUILD_COREGEN_POAR_LATEX_TESTING=ON : Enable the POAR LaTeX output tests
- -DBUILD_COREGEN_POAR_SIGMAP_TESTING=ON : Enable the POAR sigmap output tests
We welcome outside contributions from corporate, acaddemic and individual developers. However, there are a number of fundamental ground rules that you must adhere to in order to participate. These rules are outlined as follows:
- All code must adhere to the existing C++ coding style. While we are somewhat flexible in basic style, you will adhere to what is currently in place. This includes camel case C++ methods and inline comments. Uncommented, complicated algorithmic constructs will be rejected.
- We support compilaton and adherence to C++11 methods. We do not currently accept C++14 and beyond.
- All new methods and variables contained within public, private and protected class methods must be commented using the existing Doxygen-style formatting. All new classes must also include Doxygen blocks in the new header files. Any pull requests that lack these features will be rejected.
- All changes to functionality and/or the API infrastructure must be accompanied by complementary tests
- All external pull requests must target the devel branch. No external pull requests will be accepted to the master branch.
- All external pull requests must contain sufficient documentation in the pull request comments in order to be accepted.
- All external pull requests that update the CoreGen IR specification or the StoneCutter language specification (et al. syntax) must contain complementary pull requests for the CoreGen IR spec or the StoneCutter language spec.
- All pull requests will be reviewed by the core development staff. Any necessary changes will be marked
in the respective pull request comments. All pull requests will be tested against in the Tactical
Computing Laboratories development infrastructure. This includes tests against all supported platforms.
Any failures in the test infrastructure will be noted in the pull request comments.
All tests must be formatted using standard CMake directives. See the existing CMakeLists for examples on how these tests are to be formatted. Any updates to the test infrastructure (new test directory structure) must include complementary, unique directives as outlined in the Test Options section above. Any new directives must be documented in this Readme file within the respective pull request.
CoreGen is licensed under an Apache-style license - see the LICENSE file for details
- CoreGen makes use of the yaml-cpp (https://github.com/jbeder/yaml-cpp) library for its internal IR parsing. yaml-cpp carries an MIT license.
- CoreGen's StoneCutter infrastructure makes use of the LLVM compiler (http://www.llvm.org). LLVM carries a UIUC, BSD-style license.
- John Leidel - Chief Scientist - Tactical Computing Labs
- David Donofrio - Chief Hardware Architect - Tactical Computing Labs
- Ryan Kabrick - Research Engineer - Tactical Computing Labs
- Frank Conlon - Research Engineer - Tactical Computing Labs
- None at this time