Giters

Esysc / tart

macOS and Linux VMs on Apple Silicon to use in CI and other automations

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Tart – open source virtualization for your automation needs

Tart is a virtualization toolset to build, run and manage macOS and Linux virtual machines on Apple Silicon. Built by CI engineers for your automation needs. Here are some highlights of Tart:

  • Tart uses Apple's own Virtualization.Framework for near-native performance.
  • Push/Pull virtual machines from any OCI-compatible container registry.
  • Use Tart Packer Plugin to automate VM creation.
  • Built-in CI integration.

Tart is already adopted by several automation services:

Usage

Try running a Tart VM on your Apple Silicon device running macOS Monterey or later (will download a 25 GB image):

brew install cirruslabs/cli/tart
tart clone ghcr.io/cirruslabs/macos-ventura-base:latest ventura-base
tart run ventura-base

CI Integration

Tart already powers several CI services mentioned above including our own Cirrus CI which offers unlimited concurrency with per-second billing. For services that haven't leveraged Tart yet, we offer fully managed runners via a monthly subscription. Cirrus Runners is the fastest way to get your current CI workflows to benefit from Apple Silicon hardware. No need to manage infrastructure or migrate to another CI provider. Please read down below about currently supported services.

Managed runners for your CI-as-a-service

At the moment Cirrus Runners only supports GitHub Actions, but we are actively working on adding more options. Please email us if you are interested in a particular one.

GitHub Actions

Configuring Cirrus Runners for GitHub Actions is as simple as installing Cirrus Runners App. After successful installation and subscription configuration, use any of Ventura images managed by us in runs-on:

name: Test Suite
jobs:
  test:
    runs-on: ghcr.io/cirruslabs/macos-ventura-xcode:latest

When workflows are executing you'll see Cirrus on-demand runners on your organization's settings page at https://github.com/organizations/<ORGANIZATION>/settings/actions/runners.

Self-hosted CI

Tart itself is only responsible for managing virtual machines, but we've built Tart support into a tool called Cirrus CLI also developed by Cirrus Labs. Cirrus CLI is a command line tool with one configuration format to execute common CI steps (run a script, cache a folder, etc.) locally or in any CI system. We built Cirrus CLI to solve "But it works on my machine!" problem.

Here is an example of a .cirrus.yml configuration file which will start a Tart VM, will copy over working directory and will run scripts and other instructions inside the virtual machine:

task:
  name: hello
  macos_instance:
    # can be a remote or a local virtual machine
    image: ghcr.io/cirruslabs/macos-monterey-base:latest
  hello_script:
    - echo "Hello from within a Tart VM!"
    - echo "Here is my CPU info:"
    - sysctl -n machdep.cpu.brand_string
    - sleep 15

Put the above .cirrus.yml file in the root of your repository and run it with the following command:

brew install cirruslabs/cli/cirrus
cirrus run

Cirrus CI already leverages Tart to power its macOS cloud infrastructure. The .cirrus.yml config from above will just work in Cirrus CI and your tasks will be executed inside Tart VMs in our cloud.

Note: Cirrus CI only allows images managed and regularly updated by us.

Retrieving artifacts from within Tart VMs

In many cases there is a need to retrieve particular files or a folder from within a Tart virtual machine. For example, the below .cirrus.yml configuration defines a single task that builds a tart binary and exposes it via artifacts instruction:

task:
  name: Build
  macos_instance:
    image: ghcr.io/cirruslabs/macos-monterey-xcode:latest
  build_script: swift build --product tart
  binary_artifacts:
    path: .build/debug/tart

Running Cirrus CLI with --artifacts-dir will write defined artifacts to the provided local directory on the host:

cirrus run --artifacts-dir artifacts

Note that all retrieved artifacts will be prefixed with the associated task name and artifacts instruction name. For the example above, tart binary will be saved to $PWD/artifacts/Build/binary/.build/debug/tart.

Virtual Machine Management

Creating from scratch

Tart supports macOS and Linux virtual machines. All commands like run and pull work the same way regarding of the underlying OS a particular VM image has. The only difference is how such VM images are created. Please check sections below for macOS and Linux instructions.

Creating a macOS VM image from scratch

Tart can create VMs from *.ipsw files. You can download a specific *.ipsw file here or you can use latest instead of a path to *.ipsw to download the latest available version:

tart create --from-ipsw=latest monterey-vanilla
tart run monterey-vanilla

After the initial booting of the VM you'll need to manually go through the macOS installation process. As a convention we recommend creating an admin user with an admin password. After the regular installation please do some additional modifications in the VM:

  1. Enable Auto-Login. Users & Groups -> Login Options -> Automatic login -> admin.
  2. Allow SSH. Sharing -> Remote Login
  3. Disable Lock Screen. Preferences -> Lock Screen -> disable "Require Password" after 5.
  4. Disable Screen Saver.
  5. Run sudo visudo in Terminal, find %admin ALL=(ALL) ALL add admin ALL=(ALL) NOPASSWD: ALL to allow sudo without a password.

Creating a Linux VM image from scratch

# Create a bare VM
tart create --linux ubuntu

# Install Ubuntu
tart run --disk focal-desktop-arm64.iso ubuntu

# Run VM
tart run ubuntu

After the initial setup please make sure your VM can be SSH-ed into by running the following commands inside your VM:

sudo apt update
sudo apt install -y openssh-server
sudo ufw allow ssh

Configuring a VM

By default, a tart VM uses 2 CPUs and 4 GB of memory with a 1024x768 display. This can be changed with tart set command. Please refer to tart set --help for additional details.

Building with Packer

Please refer to Tart Packer Plugin repository for setup instructions. Here is an example of a template to build monterey-base local image based of a remote image:

packer {
  required_plugins {
    tart = {
      version = ">= 0.5.3"
      source  = "github.com/cirruslabs/tart"
    }
  }
}

source "tart-cli" "tart" {
  vm_base_name = "ghcr.io/cirruslabs/macos-ventura-base:latest"
  vm_name      = "my-custom-ventura"
  cpu_count    = 4
  memory_gb    = 8
  disk_size_gb = 70
  ssh_password = "admin"
  ssh_timeout  = "120s"
  ssh_username = "admin"
}

build {
  sources = ["source.tart-cli.tart"]

  provisioner "shell" {
    inline = ["echo 'Disabling spotlight indexing...'", "sudo mdutil -a -i off"]
  }

  # more provisioners
}

Here is a repository with Packer templates used to build all the images managed by us.

Working with a Remote OCI Container Registry

For example, let's say you want to push/pull images to a registry hosted at https://acme.io/.

Registry Authorization

First, you need to log in and save credential for acme.io host via tart login command:

tart login acme.io

Credentials are securely stored in Keychain.

Pushing a Local Image

Once credentials are saved for acme.io, run the following command to push a local images remotely with two tags:

tart push my-local-vm-name acme.io/remoteorg/name:latest acme.io/remoteorg/name:v1.0.0

Pulling a Remote Image

You can either pull an image:

tart pull acme.io/remoteorg/name:latest

...or instantiate a VM from a remote image:

tart clone acme.io/remoteorg/name:latest my-local-vm-name

This invocation calls the tart pull implicitly (if the image is not being present) before doing the actual cloning.

FAQ

How Tart is different from Anka

Under the hood Tart is using the same technology as Anka 3.0 so there should be no real difference in performance or features supported. If there is some feature missing please don't hesitate to create a feature request.

Instead of Anka Registry, Tart can work with any OCI-compatible container registry.

Tart doesn't yet have an analogue of Anka Controller for managing long living VMs. Please take a look at CI integration section for an option to run ephemeral VMs for your needs.

Why Tart is free and open sourced?

Apple did all the heavy lifting with their Virtualization.Framework and it just felt right to develop Tart in the open. Please consider becoming a sponsor if you find Tart saving a substantial amount of money on licensing and engineering hours for your company.

How to change VM's disk size?

You can choose disk size upon creation of a virtual machine:

tart create --from-ipsw=latest --disk-size=25 monterey-vanilla

For an existing VM please use Packer Plugin which can increase disk size for new virtual machines. Here is an example of how to change disk size in a Packer template.

VM location on disk

Tart stores all it's files in ~/.tart/ directory. Local images that you can run are stored in ~/.tart/vms/. Remote images are pulled into ~/.tart/vms/cache/OCIs/.

Nested virtualization support?

Tart is limited by functionality of Apple's Virtualization.Framework. At the moment Virtualization.Framework doesn't support nested virtualization.

Changing the default NAT subnet

To change the default network to 192.168.77.1:

sudo defaults write /Library/Preferences/SystemConfiguration/com.apple.vmnet.plist Shared_Net_Address -string 192.168.77.1

Note that even through a network would normally be specified as 192.168.77.0, the vmnet framework seems to treat this as a starting address too and refuses to pick up such network-like values.

The default subnet mask 255.255.255.0 should suffice for most use-cases, however, you can also change it to 255.255.0.0, for example:

sudo defaults write /Library/Preferences/SystemConfiguration/com.apple.vmnet.plist Shared_Net_Mask -string 255.255.0.0

About

macOS and Linux VMs on Apple Silicon to use in CI and other automations

License:GNU Affero General Public License v3.0

Languages

Language:Swift 99.6%Language:Shell 0.2%Language:ANTLR 0.1%Language:HCL 0.1%Language:Makefile 0.0%