makem.sh is a script that helps to build, lint, and test Emacs Lisp packages. It aims to make linting and testing as simple as possible without requiring per-package configuration.
It works similarly to a Makefile in that “rules” are called to perform actions such as byte-compiling, linting, testing, etc.
Source and test files are discovered automatically from the project’s Git repo, and package dependencies within them are parsed automatically.
Output is simple: by default, there is no output unless errors occur. With increasing verbosity levels, more detail gives positive feedback. Output is colored by default to make reading easy.
The script can run Emacs with the developer’s local Emacs configuration, or with a clean, “sandbox” configuration that can be optionally removed afterward. This is especially helpful when upstream dependencies may have released new versions that differ from those installed in the developer’s personal configuration.
Some example output. The first shows running the test rule with verbosity level 1, which shows which tests are run but omits each test’s output unless it fails:
Increasing the verbosity shows green output for passing tests:
The lint-compile rule treats byte-compiler warnings as errors:
The all rule runs all rules and treats warnings as errors:
Of course, with increased verbosity, it also shows which rules did not signal errors:
The included test.yml GitHub Actions file can be used to easily set up CI, giving output like:
Copy makem.sh into your package’s root directory. Optionally, also copy Makefile, to make calling the script easier.
makem.sh script can be called directly or through a Makefile. Use makem.sh --help to list available rules. The Emacs library makem.el provides a Transient dispatcher (like Magit) to easily run the script from within Emacs with selected options.
- makem.sh script
- Transient menu (makem.el)
- Makefile
- GitHub Action
- GitHub Linguist statistics
- git pre-push hook
The script may be called directly to specify additional options.
makem.sh [OPTIONS] RULES...
Linter- and test-specific rules will error when their linters or tests
are not found. With -vv, rules that run multiple rules will show a
message for unavailable linters or tests.
Rules:
all Run all lints and tests.
compile Byte-compile source files.
lint Run all linters, ignoring unavailable ones.
lint-checkdoc Run checkdoc.
lint-compile Byte-compile source files with warnings as errors.
lint-declare Run check-declare.
lint-elsa Run Elsa (not included in "lint" rule).
lint-indent Lint indentation.
lint-package Run package-lint.
lint-regexps Run relint.
test, tests Run all tests, ignoring missing test types.
test-buttercup Run Buttercup tests.
test-ert Run ERT tests.
test-ert-interactive Run ERT tests interactively.
batch Run Emacs in batch mode, loading project source and test files
automatically, with remaining args (after "--") passed to Emacs.
interactive Run Emacs interactively, loading project source and test files
automatically, with remaining args (after "--") passed to Emacs.
Options:
-d, --debug Print debug info.
-h, --help I need somebody!
-v, --verbose Increase verbosity, up to -vvv.
--no-color Disable color output.
--debug-load-path Print load-path from inside Emacs.
-E, --emacs PATH Run Emacs at PATH.
-e, --exclude FILE Exclude FILE from linting and testing.
-f, --file FILE Check FILE in addition to discovered files.
-c, --compile-batch Batch-compile files (instead of separately; quicker, but
may hide problems).
-C, --no-compile Don't compile files automatically.
Sandbox options:
-s[DIR], --sandbox[=DIR] Run Emacs with an empty config in a sandbox DIR.
If DIR does not exist, make it. If DIR is not
specified, use a temporary sandbox directory and
delete it afterward, implying --install-deps and
--install-linters.
--install-deps Automatically install package dependencies.
--install-linters Automatically install linters.
-i, --install PACKAGE Install PACKAGE before running rules.
An Emacs version-specific subdirectory is automatically made inside
the sandbox, allowing testing with multiple Emacs versions. When
specifying a sandbox directory, use options --install-deps and
--install-linters on first-run and omit them afterward to save time.
Source files are automatically discovered from git, or may be
specified with options. Package dependencies are discovered from
"Package-Requires" headers in source files, from -pkg.el files, and
from a Cask file.
The Elisp file makem.el provides a Transient dispatcher (this file should be installed into your Emacs configuration rather than into a project’s directory). Use M-x makem RET to show it.
A default Makefile is provided which calls the makem.sh script. Call it with the name of a rule and an optional verbosity level, like:
# Run all rules.
$ make all
# Run all lints.
$ make lint
# Run all tests.
$ make test
# Run ERT tests with verbosity level 1.
$ make v=v test-ert
# Run Buttercup tests with verbosity level 2.
$ make v=vv test-buttercup
# Run tests with emacs-sandbox.sh in a temporary sandbox.
# Implies install-deps=t.
$ make sandbox=t test
# Initialize a permanent sandbox directory, DIR (the developer might
# choose to recreate it manually when necessary, leaving it in place
# to save time otherwise). Then run all linters and tests.
$ make sandbox=DIR install-deps=t install-linters=t
$ make sandbox=DIR allUsing Steve Purcell’s setup-emacs Action, it’s easy to set up CI on GitHub for an Emacs package.
- Put
makem.shin your package’s repo and make it executable. - Add test.yml (from the
makem.shrepo) to your package’s repo at.github/workflows/test.yml. It should work without modification for most Emacs packages.
Having makem.sh in your repository will affect GitHub’s language stats provided by Linguist, which might cause it to be classified as a Shell project rather than an Emacs Lisp one. The Linguist documentation explains how to avoid this. Probably the most appropriate way is to use a .gitattributes file to classify makem.sh as vendored, like:
makem.sh linguist-vendored
It’s often helpful to run tests automatically before pushing with git. Here’s an example of using makem.sh in a pre-push hook:
#!/bin/sh
# * Commit parameters
# Unused now, but good for future reference. See man 5 githooks.
remote="$1"
url="$2"
read local_ref local_sha remote_ref remote_sha
# * Run tests
# Not using sandbox and auto-install, because "git push" shouldn't
# cause remote code to be downloaded and executed (i.e. what would
# happen by installing packages). It can be done manually when
# needed. However, in a CI system running in a container, where
# testing in a clean config against the latest available dependency
# versions is desired, one could use:
# make sandbox=t install-deps=t test
make testAdded
lint-elintrule (not enabled by default inlintrule due to Elint’s output not seeming very useful).makem.ellibrary with Transient dispatcher.
Fixed
- Set
package-user-dir(needed for Emacs 28 compatibility). - Rule
lint-indentfor Emacs 28. - Install Ispell in CI for
checkdoclinting.
Changed
- Display all byte-compile warnings when linting, not just the first.
Fixed
- Always set
load-prefer-newertot(rather than only when initializing packages). - When running
interactive, automatically byte-compile source files unless--no-compileis used, and load filenames sans extension so Emacs will prefer to load byte-compiled files.
Fixed
- Show all
checkdocwarnings, not just the first one.
Added
- Verbosity level 3 (i.e.
-vvv), currently only used in per-file byte-compilation output.
Fixed
- Redundant byte-compilation error message.
Added
- Option
-c/--compile-batchcompiles files as a batch, in a single Emacs process (faster, but may hide problems).
Changed
- Compile files separately rather than as a batch. (Slower, but doesn’t hide problems due to compilation order.)
Fixed
- Use
-aargument togrepin case an Elisp file contains control characters (rare, but sometimes necessary).
Added
- Emacs 27.1 to
test.yml.
Updated
test.yml: Use new GitHub environment variable syntax. (See notice, documentation.)
First tagged release.
There are several similar tools, each of which is slightly different.
Notes:
- In these comparisons,
makem.sh’s Makefile is not included, because it only provides an alternative,make-style calling convention; it provides no functionality. - These notes were compiled by reading these projects’ documentation and source code, but the author is not an expert on these tools. Corrections are welcome.
Cask is a classic Emacs package project management tool. It’s powerful and well-documented. It’s much more sophisticated than makem.sh.
- Cask requires configuration and initialization for each project before use.
makem.shis designed to work without initialization or configuration. - Cask maintains a project-local Emacs configuration for building and testing.
makem.shprovides similar, optional sandboxing to install dependencies separately from the developer’s Emacs configuration. - Cask is intended to be installed by using
curlto download a script which is piped to Python. This is a dangerous, insecure anti-pattern, compounded by the size of the code.makem.shis intended to be copied into place by the package developer, and its code is easy to inspect. - Cask is intended to be installed locally on each developer’s machine.
makem.shis intended to be dropped in to a package’s repo, requiring no local installation. - Cask’s documentation is extensive and well-presented on its Web site.
makem.shcan be used by reading a standard--helpusage guide. - Cask is over 3,000 lines of Emacs Lisp and Python code.
makem.shis about 600 lines of very simple code in one file.
Eldev is a powerful, flexible tool. It has many features and can be extended and configured for each project. It’s designed to be much more sophisticated than makem.sh.
- Eldev requires some initialization and configuration for each project before use.
makem.shis designed to work without initialization or configuration. - Eldev installs dependencies in an Emacs version-specific directory in the package repo, which allows testing with multiple Emacs versions.
makem.shcan either use dependencies that exist in the developer’s local Emacs configuration, or it can use built-in sandboxing to install dependencies separately; it does not support separate, Emacs version-specific sandboxes. - Eldev is intended to be installed by using
curlto download a script which is piped to a shell. This is a dangerous, insecure anti-pattern, compounded by the size of the code.makem.shis intended to be copied into place by the package developer, and its code is easy to inspect. - Eldev is intended to be installed locally on each developer’s machine.
makem.shis intended to be dropped in to a package’s repo, requiring no local installation. - Eldev’s documentation is comprehensive and well-written, and it’s about 8,000 words.
makem.shcan be used by reading a standard--helpusage guide. - Eldev runs from within Emacs, within the same process as the operations being run (such as testing).
makem.shruns outside of Emacs, and each operation is run in a separate Emacs process. - Eldev is over 4,000 lines of dense code across 8 source files.
makem.shis about 600 lines of very simple code in one file.
emake is intended for continuous integration testing. It is powerful and well-documented, and provides some more specific flexibility than makem.sh.
emakerequires that a variety of project-specific, Emacs-specific variables be configured before use.makem.shis designed to work without initialization or configuration.- It appears that
emakemay be run locally rather than only on remote systems like Travis CI or GitHub Actions, but that extensive configuration and initialization is required.makem.shis designed to be equally simple to use for both local developer systems and remote CI testing. emakeprovides some tools for building specific Emacs versions when running on CI systems.makem.shitself uses only the locally installed version of Emacs; for CI use, a GitHub Actions configuration is provided that uses other tools to install specific Emacs versions.emakeis intended to be installed by usingcurlto download a script which is piped to a shell, and it appears to make further use of downloading remote shell scripts at runtime, at least for initialization. This is a dangerous, insecure anti-pattern.makem.shis intended to be copied into place by the package developer, and its code is easy to inspect. No remote code is downloaded, other than installing Emacs package dependencies when requested.emake’s documentation is comprehensive and well-written, and it’s about 2,000 words.makem.shcan be used by reading a standard--helpusage guide.emakeis a 700-line Emacs Lisp file, with an optional 100-line Makefile that provides some default configuration.makem.shis about 600 lines of Bash code in one file.
Of these alternatives, makel is most like makem.sh. It’s simple and requires little configuration.
makelrequires configuring several variables before use.makem.shis designed to work without initialization or configuration.makelcan install package dependencies which are manually specified, and it appears to download them into the local package repo directory.makem.shonly installs dependencies into a sandbox directory, which, by default, is a temporary directory that is automatically removed.makelcan be used on remote CI systems, but no specific integration tools are provided.makem.shprovides a GitHub Actions file that can be used as-is.makelis intended to be used by copying two Make files into the project repo directory. It recommends allowing one of them to download the other automatically from the Internet when not present.makem.shis intended to be copied into place by the package developer. No remote code is downloaded, other than installing Emacs package dependencies when requested.makelprovides no built-in documentation, but it is very simple to use.makem.shcan be used by reading a standard--helpusage guide.makelis about 150 lines of Make code in one file.makem.shis about 600 lines of Bash code in one file.
Inspired by Damien Cassou’s excellent makel project.
Bug reports, feature requests, suggestions — oh my!
GPLv3