Dogen is the reference implementation of the MASD Project code generator. MASD - Model Assisted Software Development - is a new methodology for the development of software systems based on MDE (Model Driven Engineering). The theoretical framework for MASD is under development as part of my PhD thesis, and is not yet ready for public review. However, the development of Dogen remains as it always was, an open source project available to all.
In lay terms, Dogen's objective is to automate the development process as much as possible: the user creates models in one of the supported frontend formats and respecting a set of predefined restrictions; Dogen then uses this input to generate a source code representation of the model. At present the main frontend is Dia, but we also support JSON. Depending on use cases and user demand, other frontends may follow.
The generated code contains most of the functionality required from a typical C++ domain object such as serialisation, hashing, streaming and so on. Whilst we intend for the application to be useful out of the box for most use cases, our end goal is to enable users to extend Dogen, adapting it to the vagaries of their specific needs. Note that Dogen has experimental C# support, but it is not in feature parity with C++ support.
Dogen provides a reference implementation for each supported language, available as a separate git repo:
- C++ Reference Implementation. Models available here.
- C# Reference Implementation. Models available here.
These are designed to give you a feel for how the generated code will look like, and they attempt to exercise all of Dogen's functionality.
Binary packages are available for each release on BinTray, covering Debian Linux, OSX and Windows - 64-bit only. Packages are generated for each commit. Development is done from git master, so latest should always be greatest, but you may want to use packages from the previous sprint rather than what is currently under development (see badges above).
Notes
- The OSX installer provides you with a DMG. Once you open it in Finder,
it mounts under
/Volumes/, e.g.:
/Volumes/dogen-${VERSION}-Darwin-x86_64/dogen.app/Contents/Resources/bin
Where ${VERSION} is your Dogen version such as 1.0.23.
In order to build Dogen you will need a C++ toolchain. On Linux and OSX, you'll need a recent compiler with C++ 17 support, such as GCC 8 or Clang 7 - and Ninja or GNU Make. On Windows you'll need Visual Studio 2017. Older compilers may work, but we try to always use the most recent releases in MASD. So, if you can, please try using to those.
Dogen has the following additional dependencies, across all operative systems:
| Name | Type | Version | Description |
|---|---|---|---|
| CMake | Mandatory | 3.12 or later. | Required to generate the build files.Earlier versions may also work. |
| Boost | Mandatory | 1.68 or later. | Earlier versions may also work, but patches may be needed. Very Important: We link statically against Boost at present, so be sure to build and install the static libraries. |
| LibXml2 | Mandatory | 2.9.4 | Earlier versions may work but haven't been tested. |
| DTL | Mandatory | 1.19 | Earlier versions may work but haven't been tested. |
Though Dogen should build fine with package manager supplied libraries - or even with hand-built dependencies - the easiest way to setup a development environment on all supported platforms is by using vcpkg. We have a vcpkg fork with a MASD Branch that is setup correctly to build both Dogen and the C++ Reference Implementation and is used/validated by our CI. If at all possible, please use this instead of the mainline vcpkg because it contains a few changes that cannot be easily mainlined (C++ 17 on all dependencies, ODB 2.5, etc).
Either way, you can compile vcpkg as per vcpkg documentation, and then install packages by running:
./vcpkg install libxml2 boost-system boost-serialization boost-date-time boost-log boost-filesystem boost-program-options boost-test boost-scope-exit boost-graph boost-uuid boost-di boost-spirit dtl
Notes
- The default vcpkg triplet on Windows is 32-bit
dynamic whereas we
build with
--triplet x64-windows-static(e.g. add this to the line above if you are on Windows). If you are experiencing weird and wonderful errors building Dogen, check your triplet. - If you are on OSX, you probably should rely on the system's LibXML2
(e.g. remove it from the vcpkg list above) or else you may see some
interesting linking
errors related to
iconv. - Remember that the recommended compiler for vcpkg on OSX is Homebrew's GCC. If
you do decide to use XCode Clang or LLVM Clang, be ready to handle some pain. In addition, beware that for some reason boost does
not
default
to C++ 14. You'll need to add
cxxstd=14. Our vcpkg repo has fixes for this. - Setting up ODB is not quite as simple as we would like. If you are using a
stable version of ODB such as 2.4 you can just install the compiler and libraries
from your distribution packages or download the compiler and libraries
form the ODB downloads page
or even install it from vcpkg (i.e., add
libodb libodb-boost libodb-pgsqlto the vcpkg line above). However, as ODB 2.4 has limited support for C++ 17, you will likely end up needing the beta version 2.5. This is not as trivial. If you definitely want to do this, follow the instructions in the ODB documentation and then ensure you place the installed files in the path (e.g. setupPATH,CMAKE_INCLUDE_PATHandCMAKE_LIBRARY_PATHaccordingly).
Once you have all dependencies set up, you can then clone the repository and create the build directory:
git clone https://github.com/MASD-Project/dogen.git
cd dogen/build
mkdir output
cd output
On Linux and OSX, you can build using GNU Make as follows:
cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_TOOLCHAIN_FILE=${PATH_TO_VCPKG_DIR}/vcpkg/scripts/buildsystems/vcpkg.cmake ../..
make -j${CORES}
Where PATH_TO_VCPKG_DIR is the directory in which you've
downloaded and built vcpkg and CORES is the number of cores
available on your machine. Alternatively, you can use Ninja:
cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_TOOLCHAIN_FILE=${PATH_TO_VCPKG_DIR}/vcpkg/scripts/buildsystems/vcpkg.cmake ../.. -G Ninja
ninja -j${CORES}
On Windows, the incantation is slightly different:
cmake -DCMAKE_TOOLCHAIN_FILE=${PATH_TO_VCPKG_DIR}/vcpkg/scripts/buildsystems/vcpkg.cmake -DVCPKG_TARGET_TRIPLET=x64-windows-static ../.. -DCMAKE_BUILD_TYPE=Release -G "Visual Studio 15 2017 Win64"
cmake --build . --config Release --target ALL_BUILD
Notes
- If you are not using vcpkg, you can omit
-DCMAKE_TOOLCHAIN_FILEand-DVCPKG_TARGET_TRIPLET. However if the dependencies are not on the standard paths, you must then setCMAKE_INCLUDE_PATHandCMAKE_LIBRARY_PATHaccordingly, e.g.:
CMAKE_INCLUDE_PATH=/my/boost/include/path CMAKE_LIBRARY_PATH=/my/boost/lib/path cmake ../..
- If you are using vcpkg in Windows, make sure you are targeting
the right triplet. As explained above. We tend to use
x64-windows-static. If you have installed all the libraries (or you are using our export) but somehow CMake just doesn't seem to be able to find them, the likely cause is the wrong vcpkg triplet. - Note that we are targeting 64-bit builds (e.g.
"Visual Studio 15 2017 Win64"). If you want to do a 32-bit build, you will need to have a vcpkg export built for 32-bits. Note also that if you forget to set the generator, it will default to 32-bits and you'll experience weird and wonderful linker errors.
If you'd like to run the project tests, execute the target
run_all_tests or its abbreviation rat. This is sufficient
if you just need to gain some confidence on the code generator as we
perform tests against Dogen's own models. However, if you are trying
to submit a PR that changes the behaviour of the code generator, you
must also run the tests against the C++ and C# reference
implementation models. For this you need to download the zips or clone
the repositories locally:
Then, point the corresponding environment variables to the respective projects directory, e.g.:
export MASD_CPP_REF_IMPL_PROJECTS_DIRECTORY=${DIR}/cpp_ref_impl/projects
export MASD_CSHARP_REF_IMPL_PROJECTS_DIRECTORY=${DIR}/csharp_ref_impl/Src
Where DIR is set to the local directory where you placed the
repositories. You should then ensure Dogen correctly recognises the
additional models (SOME_PATH being your local directory):
...
-- CMake Version: 3.13.4
-- MASD_DOGEN_PROJECTS_DIRECTORY=SOME_PATH/dogen/projects
-- MASD_CPP_REF_IMPL_PROJECTS_DIRECTORY=SOME_PATH/cpp_ref_impl/integration/projects
-- MASD_CSHARP_REF_IMPL_PROJECTS_DIRECTORY=SOME_PATH/csharp_ref_impl/master/Src
...
Executing the run_all_tests target will now also include the
reference implementations. Make sure your changes do not break these models.
If you'd like to run Dogen to regenerate all of its models, you can do
so by using the target generate_all_dia or its abbreviation
gad. This is useful, for example, to update the models to the
latest version of Dogen, or just to see how Dogen works. By default
Dogen uses the current built version, but you can also use
another. For this you may need to tell CMake of its location by
setting CMAKE_PROGRAM_PATH:
CMAKE_PROGRAM_PATH=/path/to/dogen/binary cmake ../..
After regeneration, you can then use git diff to inspect the
differences produced by regeneration, if any. The build directory
contains all of the logs, under the directory log.
A number of videos are now available discussing Dogen, gathered under "themed" playlists:
- MASD - Dogen Demos: end of sprint demos.
- MASD - Dogen Coding: Relational Model for Tracing: Narrates the design and implementation of the relational model for tracing.
Over time, a number of assorted blog posts have been published on Dogen, mainly covering its development process:
- Nerd Food: The Refactoring Quagmire: How we solved the "refactoring ad infinitum" problem in Dogen development.
- Nerd Food: Northwind, or Using Dogen with ODB: Four part series on using Dogen with ODB. Part I, Part II, Part III, Part IV.
- Nerd Food: The Strange Case of the Undefined References: Problems faced using Conan with Dogen.
- Nerd Food: On Product Backlog: The importance of the Product Backlog in Dogen development.
- Nerd Food: Dogen: The Package Management Saga: Dogen's attempt at finding a package management for C++.
- Nerd Food: Pull Request Driven Development: Not directly related to Dogen as we don't get many PRs, but may be of interest; reflections on the importance of a Pull-Request Driven Development approach.
- Nerd Food: Dogen: Lessons in Incremental Coding: How the Dogen development process works, via very small, incremental changes.
The release notes are also worth perusing as contain quite a lot of material about Dogen's internals.
If you have encountered a failure building Dogen or running its unit tests, please submit a bug report. It must include:
- the error messages;
- the versions of all dependencies (compiler, Boost etc);
- the operative system name and version;
If you find a bug whilst using Dogen with one of your models, please also include:
- a small model reproducing the problem, called the MWE (Minimum Working Example).
- the compressed log. By default, dogen creates a directory structure
for all of its byproducts, in the folder it was executed from. It
has the form
dogen.byproducts/cli.generate.MODEL, whereMODELis the file name and extension of the generated model. The log file is stored in this directory, with a name such ascli.generate.MODEL.log.
Bugs can be submitted using github Issues.