frdeso / vlttng

a tool to create LTTng virtual environments

vlttng

vlttng is a tool which creates a virtual environment (a sandbox) to run specific versions of the LTTng packages. The Babeltrace, Userspace RCU, LTTng analyses, and Trace Compass projects are also supported, as well as some of the project dependencies.

Install vlttng

To install vlttng on the system:

• Use pip3:

  sudo pip3 install --upgrade vlttng

To install vlttng in your home directory:

• Use pip3:

  pip3 install --user --upgrade vlttng

  The commands are installed in ~/.local/bin.

Two new commands are available: vlttng and vlttng-quick.

Quickstart

The easiest way to use vlttng is to use the vlttng-quick command. This command interactively asks you a few questions to create a basic vlttng command line that you can use later or immediately after.

How does it work?

The vlttng command does the following:

1. It reads one or more profiles that you give on the command line to know which packages to fetch and build.

2. It fetches and extracts the requested packages. Git, with a specific branch/tag/commit, as well as HTTP/FTP tarball sources are supported. The Git clone URL can point to a local Git repository using the file:// protocol.

3. It builds one package at a time, setting some environment variables and configure options so that the dependencies of the packages are contained within the virtual environment.

4. It creates an activate script which you can source from your Bash/Zsh prompt to “enter” the virtual environment. This script sets a few environment variables, like PATH, LD_LIBRARY_PATH, and PYTHONPATH, to achieve this. By default, it also prepends the name of the virtual environment directory to your shell prompt for you to know which virtual environment is active.

   When you source the activate script, if the LTTng-modules project is part of the effective profile, vlttng removes the currently loaded LTTng kernel modules and sets the MODPROBE_OPTIONS environment variable so that the LTTng session daemon loads the virtual environment modules.

Example:

vlttng -p lttng-stable-2.8 -p babeltrace-master -p babeltrace-python \
       -p lttng-tools-no-lttng-relayd -p urcu-stable-0.9 virt

Here, five profiles are used to create a virtual environment in the virt directory. You can source the activate script to enter the virtual environment:

. ./virt/activate

Your prompt should start with [virt] after this (the name of the virtual environment directory).

You can “exit” the virtual environment with the vlttng-deactivate command. Your prompt should return to its previous value.

Write and use profiles

A vlttng profile is a layer of configuration. You can use multiple profiles to create an effective profile.

The following default profiles are shipped with the project:

all-master.yml
babeltrace-master.yml
babeltrace-python.yml
babeltrace-stable-1.2.yml
babeltrace-stable-1.3.yml
babeltrace-stable-1.4.yml
elfutils-0.166.yml
glib-2.22.5.yml
glib-2.23.6.yml
glib-2.24.2.yml
glib-2.25.17.yml
glib-2.26.1.yml
glib-2.27.93.yml
glib-2.28.8.yml
glib-2.29.92.yml
glib-2.30.3.yml
glib-2.31.22.yml
glib-2.32.4.yml
glib-2.33.14.yml
glib-2.34.3.yml
glib-2.35.9.yml
glib-2.36.4.yml
glib-2.37.93.yml
glib-2.38.2.yml
glib-2.39.92.yml
glib-2.40.2.yml
glib-2.41.5.yml
glib-2.42.2.yml
glib-2.43.92.yml
glib-2.44.1.yml
glib-2.45.8.yml
glib-2.46.2.yml
glib-2.47.92.yml
glib-2.48.1.yml
glib-2.48.2.yml
glib-2.49.7.yml
glib-2.50.1.yml
glib-master.yml
libxml2-2.8.0.yml
libxml2-2.9.0.yml
libxml2-2.9.1.yml
libxml2-2.9.2.yml
libxml2-2.9.3.yml
libxml2-2.9.4.yml
libxml2-master.yml
lttng-analyses-0.3.0.yml
lttng-analyses-0.4.0.yml
lttng-analyses-0.4.1.yml
lttng-analyses-0.4.2.yml
lttng-analyses-0.4.3.yml
lttng-analyses-0.5.0.yml
lttng-analyses-0.5.1.yml
lttng-analyses-0.5.2.yml
lttng-analyses-0.5.3.yml
lttng-analyses-0.5.4.yml
lttng-analyses-master.yml
lttng-master.yml
lttng-modules-master.yml
lttng-modules-stable-2.6.yml
lttng-modules-stable-2.7.yml
lttng-modules-stable-2.8.yml
lttng-stable-2.6.yml
lttng-stable-2.7.yml
lttng-stable-2.8.yml
lttng-tools-master.yml
lttng-tools-no-lttng-consumerd.yml
lttng-tools-no-lttng-crash.yml
lttng-tools-no-lttng-relayd.yml
lttng-tools-no-lttng-sessiond.yml
lttng-tools-no-lttng.yml
lttng-tools-no-man-pages.yml
lttng-tools-no-python.yml
lttng-tools-python.yml
lttng-tools-stable-2.6.yml
lttng-tools-stable-2.7.yml
lttng-tools-stable-2.8.yml
lttng-ust-jul-agent.yml
lttng-ust-log4j-agent.yml
lttng-ust-master.yml
lttng-ust-no-man-pages.yml
lttng-ust-python-agent.yml
lttng-ust-stable-2.6.yml
lttng-ust-stable-2.7.yml
lttng-ust-stable-2.8.yml
tracecompass-1.1.0.yml
tracecompass-1.2.0.yml
tracecompass-1.2.1.yml
tracecompass-2.0.0.yml
tracecompass-master.yml
urcu-master.yml
urcu-stable-0.7.yml
urcu-stable-0.8.yml
urcu-stable-0.9.yml

You can get this list using vlttng --list-default-profiles.

Profiles are written in YAML. Here is an example:

build-env:
  CFLAGS: -O0 -g3
virt-env:
  ENABLE_FEATURE: '1'
  SOME_PATH: /path/to/omg
projects:
  lttng-tools:
    source: 'git://git.lttng.org/lttng-tools.git'
    checkout: stable-2.7
    build-env:
      CC: clang
      CFLAGS: ''
  lttng-ust:
    source: 'http://lttng.org/files/lttng-ust/lttng-ust-2.7.2.tar.bz2'
    configure: --enable-python-agent
  lttng-modules:
    source: 'git://git.lttng.org/lttng-modules.git'
    checkout: stable-2.7
  urcu:
    source: 'git://git.liburcu.org/userspace-rcu.git'

A few things to note here:

• The root build-env property defines the base build environment variables. They are set when building the projects. vlttng also passes exported shell variables to the executed programs, so you can do:

  CC=clang CFLAGS='-O0 -g3' vlttng ...

• The root virt-env property defines the virtual environment variables, which are set when activating the virtual environment. Exported shell variables, when invoking vlttng, are not set when activating the resulting virtual environment.

• The available project names, as of this version, are:

  • babeltrace

  • elfutils

  • glib

  • libxml2

  • lttng-analyses

  • lttng-modules

  • lttng-tools

  • lttng-ust

  • tracecompass

  • urcu

• The build-env property of a specific project defines environment variables to be used only during the build stage of this project. A project-specific build-time environment variable overrides a base build-time environment variable sharing its name.

• When the source property contains a Git URL, or when checkout property is set, the checkout property indicates which branch, tag, or commit to check out. When it’s not specified, vlttng checks out the master branch.

• The configure property specifies the options to pass to the configure script of a given project. vlttng handles some options itself, like --prefix and --without-lttng-ust, to create a working virtual environment.

The profile above can be saved to a file, for example my-profile.yml, and you can create a virtual environment out of it:

vlttng -p my-profile.yml virt

When you give multiple profiles to vlttng, the first profile is “patched” with the second, which is then patched with the third, and so on. Nonexistent properties are created; existing ones are replaced recursively. The configure properties are joined. For example, let’s add the following profile (call it more.yaml) to the example above:

build-env:
  CFLAGS: -O0
  SOMEVAR: ok
projects:
  lttng-tools:
    source: 'https://github.com/lttng/lttng-tools.git'
  lttng-ust:
    configure: --enable-java-agent-jul

With this command:

vlttng -p my-profile.yml -p more.yaml virt

the effective profile is:

build-env:
  CFLAGS: -O0
  SOMEVAR: ok
projects:
  lttng-tools:
    source: 'https://github.com/lttng/lttng-tools.git'
    checkout: stable-2.7
    build-env:
      CC: clang
      CFLAGS: ''
  lttng-ust:
    source: 'http://lttng.org/files/lttng-ust/lttng-ust-2.7.2.tar.bz2'
    configure: --enable-python-agent --enable-java-agent-jul
  lttng-modules:
    source: 'git://git.lttng.org/lttng-modules.git'
    checkout: stable-2.7
  urcu:
    source: 'git://git.liburcu.org/userspace-rcu.git'

Override a profile property

You can replace, append to, and remove effective profile properties (after vlttng has merged all the profiles given with the --profile option as an effective profile) with the --override (-o) option.

The three override operations are:

Replace a property

PATH=REPLACEMENT

Append to a property

PATH+=APPEND

Remove a property

!PATH

PATH is the path to the property, from the root of the profile, using a dot-separated list of keys to find recursively.

Example:

-o projects.lttng-tools.configure+=--disable-bin-lttng-relayd \
-o '!projects.lttng-ust.checkout' \
-o build-env.CC=clang

In replace and append modes, vlttng creates the property if it does not exist. This allows you to create projects on the command line:

-o projects.lttng-tools.source=https://github.com/lttng/lttng-tools.git \
-o projects.lttng-tools.checkout=v2.7.1 \
-o projects.lttng-tools.configure='--disable-bin-lttng --disable-man-pages'

Note that the overrides are applied in command line order.

Ignore a project

You can ignore specific projects that exist in the effective profile using the --ignore-project (-i) option:

vlttng -p lttng-stable-2.7 -p urcu-master -i lttng-ust virt

This is the equivalent of removing the project’s property with an override:

vlttng -p lttng-stable-2.7 -p urcu-master -o '!projects.lttng-ust' virt

Make the output verbose

By default, vlttng hides the standard output and error of the commands it runs. In this mode, vlttng prints all the commands to run and the exported environment variables along with comments, so that you can “replay” the entire output as is to create the same virtual environment (except for the activate script which would not be generated).

You can use the --verbose (-v) option to also print the standard output and error of all the executed commands, and the effective profile used to create the virtual environment.

Define the number of make jobs

vlttng passes its --jobs (-j) option as is to make.

activate script options

When you source the activate script, you can use the following environment variables to alter its behaviour:

VLTTNG_NO_RMMOD

Set to 1 to disable the unloading of the currently loaded LTTng kernel modules.

VLTTNG_NO_PROMPT

Set to 1 to keep your current shell prompt after the activation.

Use sudo

If you use sudo when the virtual environment is activated, make sure to use its --preserve-env (-E) option, so that the virtual environment is preserved when it executes the command.

For example, to start a root session daemon which loads the LTTng kernel modules installed in the virtual environment:

sudo --preserve-env lttng-sessiond --daemonize

Trace a Java application

When the LTTng-UST project is built with a Java agent, the activation of the virtual environment sets the VLTTNG_CLASSPATH environment variable to a Java class path to use when you compile and run Java applications.

Example:

javac -cp $VLTTNG_CLASSPATH MyClass.java
java -cp $VLTTNG_CLASSPATH:. MyClass

Trace a Python application

If the LTTng-UST Python agent is built and installed in the virtual environment, there’s nothing special to do to trace a Python application: the PYTHONPATH environment variable contains the path to the LTTng-UST Python agent package in the virtual environment. You can import the lttngust package as usual.

Update a project with a Git source

When a project’s source uses Git, vlttng generates the following scripts in the virtual environment’s root directory (name is the project name):

conf-name.bash

Runs the configuration step of the project.

build-name.bash

Runs the build step of the project.

install-name.bash

Runs the install step of the project.

update-name.bash

Fetches the project’s configured Git remote, checks out the latest version of the configured branch, and runs conf-name.bash, build-name.bash, and install-name.bash.

Important

Use those scripts with caution. For a stable branch, they should work most of the time. For the master branch, some required implicit configuration and build command lines might be missing from the scripts.