Emacs Development Environment (EDE) wrapper for Compilation Database projects

EDE-compdb is a library that enables the Emacs Development Environment (EDE), which is part of CEDET, to be used with a compilation database, as provided by build tools such as CMake or Ninja. This enables CEDET to be automatically configured for use to support parsing, navigation, completion, and so on. This is especially useful for C and C++ projects which are otherwise quite tricky to configure for use with CEDET and other libraries.

Features:

Reads compilation database on-demand and provides include paths and preprocessor symbols to the Semantic parser
For GCC-compatible compilers, reads the internal include paths (no need to use semantic-gcc-setup)
Most EDE commands supported, such as “C-c . c” to build the current target (source file)
Autoloads based on the presence of compile_commands.json file, with no additional configuration typically required
Supports multiple build configurations (eg Debug, Release) and corresponding build directories
Supports Flymake, allowing source files to be checked on-the-fly with no additional setup required
Easy integration with auto-complete-c-headers, for include file completion
(For Ninja projects only) Automatically generates the compilation database as required without needing to generate a compile_commands.json file
(For Ninja projects only) Loads and caches available phony build targets (eg “test”) for convenient builds

License: GPLv2

Quickstart

The easiest method to install EDE-compdb is via the MELPA package repository. No further configuration is needed with this method.

Alternatively you can download/clone the ede-compdb repo to a suitable directory and add it to your emacs load-path. You should then it to your .emacs file after loading CEDET, using:

(require 'ede-compdb)

In order to use EDE-compdb you need to ensure your project has a compilation database. If you are using CMake you can do something like the following:

$ cd .../yourproj
$ cmake -DCMAKE_EXPORT_COMPILE_COMMANDS=TRUE .

This will generate build files in the source directory, as well as a compile_commands.json file, which is the compilation database.

Note that if you are using the Ninja build tool, you don’t need to generate the compile_commands.json file, because ninja can generate the compilation database as-needed.

Now open any of your source files in emacs, and you should see all the includes correctly parsed. Be aware that due to a limitation in the current CEDET, if you use “Summarize includes current buffer” you will NOT see the project include path for the current source file.

Build Directories

The quickstart example above assumes that you are building your binaries into the same directory. By default, EDE-compdb projects are loaded when a compile_commands.json file is found in the current directory, or in a parent directory. If you build your projects in the same directory as your source, no additional configuration is required.

However, EDE-compdb can be configured to use a separate build directory, or even multiple build directories. This is desirable when developing for multiple configurations (eg a debug build for feature development, and a release build for performance/integration testing). Furthermore, the build directories can be either “in-source” or “out-of-source”, meaning they can be subdirectories of your project source tree, or not.

In this (admittedly complex) example, we have a possible four different types of build for each project. Each build type is assigned a separate directory, relative to the project root. At load time, we examine the project to see which, if any, of the build directories is present. This directory is selected as the build directory, and additionally we set the :configuration-default to the corresponding value.

(defvar my-cmake-build-directories
  '(("None" . "build")
    ("Debug" . "build.dbg")
    ("Release" . "build.rel")
    ("RelWithDebInfo" . "build.r+d")))

(defun my-load-cmake-project (dir)
  "Creates a project for the given directory sourced at dir"
  (let ((default-directory dir)
        (config-and-dir (car (cl-member-if (lambda (c)
                                             (file-readable-p
                                              (expand-file-name "compile_commands.json" (cdr c))))
                                           my-cmake-build-directories))))
    (unless config-and-dir
      (error "Couldn't determine build directory for project at %s" dir))
    (ede-add-project-to-global-list
     (ede-compdb-project 
      (file-name-nondirectory (directory-file-name dir))
      :file (expand-file-name "CMakeLists.txt" dir)
      :compdb-file (expand-file-name "compile_commands.json" (cdr config-and-dir))
      :configuration-default (car config-and-dir)
      :configuration-directories (mapcar #'cdr my-cmake-build-directories)
      :configurations (mapcar #'car my-cmake-build-directories)
      :build-command "cmake --build ."
      ))))

(defun vc-project-root (dir)
  (require 'vc)
  (let* ((default-directory dir)
         (backend (vc-deduce-backend)))
    (and backend (vc-call-backend backend 'root default-directory))))

(ede-add-project-autoload
 (ede-project-autoload "CMake"
                       :file 'ede-compdb
                       :proj-file "CMakeLists.txt"
                       :proj-root 'vc-project-root
                       :load-type 'my-load-cmake-project
                       :class-sym 'ede-compdb-project))

The :build-command attribute sets the command to be used to build the entire project. This is available using the existing EDE shortcuts (eg “C-c . C” by default). This command is run from the current configuration directory.

Header files

One of the limitations of using the compilation database is that it only contains the compilation commands for source files. However files such as header files are not generally compiled independently, hence are not generally inserted into the compilation database. This means that although header files are still detected as part of the project, are not assigned the include paths and preprocessor symbols which are available to source files.

EDE-compdb works around this limitation using some heuristics to locate a compilation database entry for each buffer file. This is the process that is followed when a new file is opened within an existing EDE-compdb project.

If the current buffer file is in the compilation database, that is used.
If there is an “other” file associated with the current buffer which is also in the compilation database, that is used. The definition of an “other” file is almost exactly the same as that used by the the built-in emacs function ff-get-other-file. By default, ff-get-other-file will search the current directory for an equivalent .cpp file, so if the current buffer is visiting an .hpp file and the equivalent .cpp file is in the compilation database, that is used. Other directories can be searched, and indeed custom functions can be provided to search for arbitary files.
Otherwise the compilation database is searched, and the entry which has the longest common prefix with the current buffer file is used. So for example if you are visiting include/config.hpp, and there is an entry for main.cpp, this will be used in preference to src/config.cpp.

This technique ensures that every header file should be matched to a compilation database entry. To see the compilation database entry for a given header file, just compile it! (Use ede-compile-target, which is bound to C-c . c by default).

Flymake Support

The ede-compdb-flymake-init function is suitable for use with flymake-mode, which enables on-the-fly compilation checking of the current buffer. To configure it, simply add the following to your emacs init file:

(require 'flymake)
(setq flymake-allowed-file-name-masks
      (cons '("\\.[ch]pp$" ede-compdb-flymake-init)
            flymake-allowed-file-name-masks))

(add-hook 'find-file-hook 'flymake-find-file-hook)

This will enable the use of flymake for all .cpp and .hpp files. Header files are supported, as long as a matching source file can be located, as described above.

auto-complete-c-headers

The auto-complete-c-headers package provides auto-completion for C and C++ header files. To do this successfully, it needs to know the current include directories. EDE-compdb can be configured to provide this information, as in the following example:

(add-hook 'ede-minor-mode-hook (lambda ()
    (setq achead:get-include-directories-function 'ede-object-system-include-path)))

Ninja Build Tool

The Ninja build tool is a great alternative to GNU Make for building large-scale projects. EDE-compdb supports some additional features when Ninja is used.

Firstly, the compilation database is generated on-demand using ninja -t compdb, rather than a separate compile_commands.json file.

Also, EDE-compdb supports automatically loading the list of top-level phony projects, like “all” and “test”. These are often useful during development, and EDE-compdb makes these available for use via a ede-compdb-build-target command, also available from the “Build Other Target…” menu item. This is useful for building phony targets you may have set up such as “all” or “test”. These phony targets are queried using ninja -t targets and cached per-project.

Compilation Database

A compilation database provides a way for tools to get access to the compilation commands that are to be executed for a given source file. The following is an example of a compilation database entry:

{
    "directory": "/home/user/llvm/build",
    "command": "/usr/bin/clang++ -Irelative -DSOMEDEF=\"With spaces, quotes and \\-es.\" -c -o file.o file.cc",
    "file": "file.cc"
},

This information is very useful for tools like CEDET, as it enables the tool to unambiguously determine the include paths and preprocessor definitions for C and C++ source files. This information is otherwise quite difficult to determine automatically, and most current tools typically require it to be provided redundantly (eg once in the build tool input file and again in an EDE project).

When CEDET is able to use the information in a compilation database, it significantly simplifies the configuration and setup of a typical C/C++ project, and possibly helps with other languages/projects. Furthermore it helps to improve the accuracy of the parser and provide many other benefits besides.

So how is the compilation database generated? Several methods are possible:

For CMake-based projects using the GNU Make build tool, there is the CMAKE_EXPORT_COMPILE_COMMANDS option (described above) which tells CMake to write out a compile_commands.json file along with the generated Makefiles in the build directory. This file contains the entire compilation database for the project.
For projects using the Ninja build tool, the compilation database can be generated on-demand using the -t compdb command.
The Build EAR (Bear) tool can generate a compilation database from any build system by sniffing the compiler commands as they are executed.

Use of the compilation database is becoming more and more common, particularly for those projects using the clang toolset.

Development

There is an ert test suite which uses a sample CMake project, with a temporary directory as a build directory. CMake, Ninja and a C++ compiler are required to run these tests successfully.

Current limitations/TODOs/Wishlist:

As stated above, if you use “Summarize includes current buffer” you will NOT see the system include path for the buffer. The reason is that the include path is set on the target, and not on the project. However, the summarize function only prints out the system include path for the project, and not the target. You can of course use (ede-system-include-path ede-object) to check the include path instead.
EDE-compdb only does very basic parsing of the GCC (or compatible) command line options, and doesn’t support any of the more esoteric GCC-specfic ones such as “-imacros”, “-idirafter”, “-iprefix”, etc.
Currently uses the json module for loading the compilation database. This can get slow for large projects.
Full Ninja target heirarchy parsing. Basically we can use the ninja -t targets tool to query the target heirarchy. We would need to insert the source targets into the heirarchy at the right locations.
Support Debug/Run target. This doesn’t really make sense for an individual source file, but we should be able to prompt for or guess (as per the previous point) the appropriate executable.
Automated setup of build directory. Given a compdb generator (eg cmake) we should be able to automate the setup of a new build directory. Ideally this would work for a new source tree.

tsuri / ede-compdb