bruceluk / magiskboot_build

a simple portable CMake-based build system for magiskboot

magiskboot_build

a simple portable CMake-based build system for magiskboot

Requirements

magiskboot itself depends on the following libraries:

1. LZMA
2. lz4
3. bzip2
4. zlib

for build-time dependencies:

1. pkg-config
2. Clang
3. LLD (optional)
4. Rust
5. Cargo
6. CMake
7. Libc++ (optional, see this part)

please make sure you have installed the above softwares before building

here are some examples for popular operating systems/distributions:

Linux

Ubuntu 22.04 (jammy)

Make sure to read this part before you start.

sudo apt update
sudo apt upgrade  # upgrade all existing packages (optional)
# replace clang-15, libc++-15-dev, libc++abi-15-dev with clang, libc++-dev and libc++abi-dev
#  if your Ubuntu is too old, do the same for lld if you want to use it
sudo apt install build-essential lzma-dev liblzma-dev liblz4-dev libbz2-dev \
                 zlib1g-dev pkg-config clang-15 libc++-15-dev libc++abi-15-dev cmake \
                 ninja-build rustc cargo  # optional: lld-15
# the following cmds are only for Ubuntu jammy:
mkdir ~/.bin
ln -s `which clang-15` ~/.bin/clang
ln -s `which clang++-15` ~/.bin/clang++

# optional:
# ln -s `which lld-15` ~/.bin/lld

# finally 
export PATH=~/.bin:$PATH

Alpine Linux (edge)

sudo apk update
sudo apk upgrade  # upgrade all existing packages (recommended)
sudo apk add build-base linux-headers xz-dev lz4-dev bzip2-dev \
             zlib-dev pkgconf clang libc++-dev cmake \
             samurai rust cargo  # optional: lld

archlinux

sudo pacman -S --needed base-devel xz lz4 bzip2 zlib pkgconf \
                        clang libc++ cmake ninja rust  # optional: lld

See also

xiaoxindada/magiskboot_ndk_on_linux: minimal build system for magiskboot with ondk on Linux

macOS

macOS Big Sur (or higher verison)

install Homebrew first

brew update
brew upgrade  # upgrade all existing packages (optional)
brew install xz lz4 bzip2 zlib pkg-config cmake ninja rust

Android

Termux

Note

Termux build is not actively tested by CI, but it might work without problem.

If you find the build is broken, please file a bug report.

apt update
apt upgrade  # upgrade all existing packages (optional)
apt install build-essential liblzma liblz4 libbz2 zlib pkg-config \
            clang lld rust cmake ninja

When configuring, pass -DCMAKE_INSTALL_PREFIX=$PREFIX to CMake.

You can also directly use the libmagiskboot.so extracted from the Magisk APK, it's just a static ELF program.

Windows

Windows (MinGW)

Note

A minor amount of POSIX functions in src/winsup/*_compat.c are currently stubbed and no-op (e.g. chmod, chown, mknod), but it shouldn't cause too much trouble for magiskboot to work.

However, if you know a better way to do this, please feel free to open a Pull Request to change it :)

Install MSYS2 first, and change the setting of mintty.exe to grant it with administrator privileges (needed for using native symlinks in some conditions).

don't forget to set this environtment variable to allow symlinks to work properly: export MSYS=winsymlinks:native (required for the build I guess)

pacman -Syu  # upgrade all existing packages (optional, you may need to do this for multiple times)
pacman -S --needed base-devel pactoys
pacboy -S --needed {xz,lz4,bzip2,zlib,pkgconf,clang,lld,cmake,libc++,ninja,rust}:p

There is also an old MinGW port, it works great:

svoboda18/magiskboot: a dirty Windows port with custom GNU Make based build system

Cygwin (test)

Note

Cygwin build is not actively tested by CI, but it might work without problem.

If you find the build is broken, please file a bug report.

An experimental build is available here.

To build for Cygwin, you need to compile a Rust toolchain from source, for more info: Cygwin Rust porting

You will also need to compile the LLVM & Clang from source with Cygwin patches, you can pick them from: Ookiineko's unofficial cygpkgs

Currently Cygwin Rust has no host tools support, so cross compiling is needed, make sure to read the instructions for Cross compiling.

The cross compiler is available on Fedora Linux, provided by the Fedora Cygwin.

Install these packages: cygwin, cygport and cygwin-{filesystem,binutils,gcc,pkg-config}.

Build and install the dependencies from source to sysroot using cygport, or download and extract the prebuilt package from a Cygwin mirror. Another way is to extract those files from an actual Cygwin installation.

When configuring, use cygwin64-cmake instead of cmake, but don't use it for cmake --build and other CMake commands.

To use the patched Clang with Fedora Cygwin, set both CMAKE_C_COMPILER_TARGET and CMAKE_CXX_COMPILER_TARGET to x86_64-unknown-windows-cygnus and CMAKE_SYSROOT to /usr/x86_64-pc-cygwin/sys-root when configuring.

Another issue is cygwin64-cmake overrides C and C++ compilers to GCC which is not supported by Magisk, and somehow ignoring the CC and CXX variables set by us. To workaround this, set CMAKE_C_COMPILER and CMAKE_CXX_COMPILER manually with the path to the previous Cygwin cross Clang and Clang++.

Cygwin's Libc++ is buggy, if you can't pass linking with Libc++, see this part to apply patches for building without Libc++. (You will also need to apply patches to compile with old Rust toolchain.)

Finally, you should be able to pass the build and get it working now :D

Feel free to ask questions about Cygwin support in Issues.

Web Assembly

Emscripten (WIP)

Note

Currently this port only compiles, prints a usage, and does nothing.

Helps are welcome QwQ

Currently, cross-compiling from Arch Linux is tested, please read the Cross compiling instructions.

First, install Emscripten and basic build dependencies using pacman.

sudo pacman -S --needed base-devel pkgconf emscripten cmake ninja rust

Use vcpkg to install the depended libraries (vcpkg itself can be installed from AUR), the triplet is called wasm32-emscripten.

If you install vcpkg from AUR, the installation path is /opt/vcpkg.

When configuring, use emcmake instead of cmake (but don't use it for cmake --build and other CMake commands) , and use /usr/lib/emscripten/cmake/Modules/Platform/Emscripten.cmake as the toolchain file for vcpkg.

the cross Rust target is wasm32-unknown-emscripten and you will need to enable Rust STD build (install the rust-src package via pacman).

finally, copy the output magiskboot.{js,html,wasm} to a web server and open the html page, usage will be printed in the browser's developer console.

Download

For prebuilt binaries, go check GitHub Releases for selected CI builds.

Build & Install

First get and extract the latest source tarball (called magiskboot_<COMMIT_ID>_<VERCODE>-src.tar.xz) from Github Releases.

Or clone this repository using Git with this option: --recurse-submodules. (If you already have a cloned repository without using that option, run: git submodule update --init --recursive instead)

CC=clang CXX=clang++ cmake -G Ninja -B build -DCMAKE_BUILD_TYPE=Release  # configure
cmake --build build -j $(nproc)  # build
./build/magiskboot  # running
# install to system (may need sudo, to specify different install dir, set the `DESTDIR' environment variable)
cmake --install build

If you prefer a statically linked binary (optional), pass -DPREFER_STATIC_LINKING=ON to CMake while configuring, make sure your distribution provided you the static version of the depended libraries, otherwise you'll run into configure errors and you have to change your distribution or try to compile the static version of those libraries yourself and install them to your system for the static build.

Cross compiling

First get a cross-compiler with Libc++ and Clang by either installing from source or downloading it from somewhere (usually your distribution's package manager).

To cross-compile, you may need a CMake toolchain file describing your target specs, the location of the cross compiler toolchain and strategy for CMake to seek for the depended libraries.

You can install the depended libraries for your cross target by using vcpkg, for example:

vcpkg install bzip2:arm64-linux

arm64-linux is the triplet of your cross target, pass this value to CMake using the VCPKG_TARGET_TRIPLET variable during configuration.

Set variable RUSTC_TARGET to the Rust target you wanted to cross compile for, e.g. aarch64-unknown-linux-gnu.

To use your toolchain file with vcpkg, first pass -DCMAKE_TOOLCHAIN_FILE=/path/to/your/vcpkg/install/path/scripts/buildsystems/vcpkg.cmake to CMake, and set the variable VCPKG_CHAINLOAD_TOOLCHAIN_FILE to the path of your actual toolchain file.

If the cross Rust target is not installed, you can still compile if you build the Rust standard library (STD) from source, to do this, pass -DRUST_BUILD_STD=ON during configuration (Note this will require the Rust source code to be installed on your system).

LTO

If you want to perform LTO at the final link time, pass -DUSE_LTO_LINKER_PLUGIN=ON to CMake during configuring.

And you will need to install LLD.

Note: you may need to make sure your LLVM and LLD are sharing the same LLVM version with Rust.

Testing

Note

Features not tested by CI doesn't necessarily mean it can't work.

A very basic shell script is provided under the scripts directory, currently the following functions are automatically tested with Github CI:

• unpack* (currently not all header versions and variants are tested)
• repack* (same as above)
• verify
• sign
• extract
• hexpatch

• cpio* (currently not widely tested on all common ramdisks)

  • cpio exists
  • cpio ls
  • cpio rm
  • cpio mkdir
  • cpio ln
  • cpio mv
  • cpio add
  • cpio extract
  • cpio test
  • cpio patch
  • cpio backup
  • cpio restore

• dtb

  • dtb print
  • dtb test
  • dtb patch

• split
• sha1
• cleanup
• compress
• decompress

Generating source tarball

cmake -G Ninja -B build -DNO_TARGETS_OR_DEPS=ON  # configure
cmake --build build -t package_source  # make a source package

Make sure you passed -DNO_TARGETS_OR_DEPS=ON to CMake while configuring, which will allow creating source tarball without installing the build dependencies.

you should be able to find your source package under the build folder

FAQ

Something isn't working

Please use debug builds and paste your information (like crash or error logs) in a new Issue.

Help, my build has failed

A recent version of Rust is needed since the upstream is actually using a custom nightly Rust compiler,

if your distribution's Rust compiler is too old (e.g. Debian/Ubuntu based), manually applying some extra patches is probably needed,

you can try to pick some known ones from here, or report it by filing an Issue if that doesn't help.

Or you can try installing Rust via rustup instead.

If you need to build with Libstdc++ instead of Libc++ (not recommended), pass -DWITHOUT_LIBCXX=ON to CMake during configuring, also apply this patch.

Is this thing using the latest Magisk source?

Check the version status badges at the top of README.

This project is very similiar to android-tools which just maintains a set of patches on top of a specific upstream Magisk commit and require manual adaption for compiling with newer source.

Although I may update the version once in a while, Pull requests are welcome.

Eh, so my platform is not supported by your sh*t

This project aims to be portable, so it should be possible to port it to new platforms with some efforts, as long as your platform meets the the above requirements.

Not? Check out the Cygwin platform for example, maybe you can try to port those dependencies yourself.

Or, try to make a port without the missing dependencies (and you may need to rewrite some stuffs, this may not always be easy, for example the upstream Rust codebase is likely to grow). However, these are not covered by this project, just some friendly suggestions.

Development

Clean builds

To quickly discard the current build directory and dirty vendor/ submodule changes, please run make clean.

Tweaking patches

To temporarily disable vendor/ projects patching, re-configure with -DPATCH_VENDOR_PROJECTS=OFF (useful if you are patching them manually). To enable it again, use -DPATCH_VENDOR_PROJECTS=ON (Note this will clean up changes in vendor/ modules and re-apply all the patches).

Rust stuffs

If you modify something in the Rust part, you will have to perform rm build/libmagiskboot-rs.a manually before rebuilding. (TODO: let CMake detect source changes)

Debug builds

Pass -DCMAKE_BUILD_TYPE=Debug to CMake instead of Release during configuring to make some error log more verbose.

Updating sources

For vendor/ submodules with their name starting with android_, most patches are imported from android-tools, and don't always require updating.

When syncing upstream vendor/{android_libbase,Magisk} changes, here is a few things to check:

• build.py changes
• {,{base,boot}/}Android.mk updates
• if external projects were added, deleted, or moved, update the list of excluded directories at the bottom of CMakeLists.txt accordingly
• switch to winsup stub for MinGW for if any new file or spot reference to any of the following standard libc identifiers:
  • struct DIR
  • struct dirent
  • mkdir
  • {open,read,close,rewind,seek,tell}dir
  • creat
  • open
  • fopen
  • fstat

Special thanks to

• android-tools developers for many code and inspiration of this repository
• Magisk developers for the magiskboot utility
• all other used projects' developers (mentioned in the Requirements section)