ingydotnet / LisRoot

Calling into CERN's Root data analysis framework from Lisp

LisRoot

Abstract of the according ArXiv paper "A functional scripting interface to an object oriented C++ library"

The object oriented programming paradigm is widely used in science and engineering. Many open and commercial libraries are written in C++ and increasingly provide bindings to Python, which is much easier to learn, but still partly encourages the use of object oriented programming. However, scientific ideas are much more directly and meaningfully expressed in the purely functional programming paradigm. Here, we take a best practice example, CERNs Python binding for its ROOT library, designed to handle the enormous amounts of data generated by the worlds largest particle accelerator, and translate a simple segment of its tutorial into Clojure, a functional language from the Lisp family. The code examples demonstrate how a purely functional language straightforwardly expresses scientific ideas. Subsequently, we develop a compiled Lisp-C++ interoperation layer to access the ROOT library exclusively via functional code. To preserve the expressivity of the Lisp code, the type hints necessary for C++ code generation are stored in a separate file. The interop system presented here is a generic framework that, when provided with a suitable file of type hints, facilitates access to methods of arbitrary C++ libraries and platforms like real-time microcontrollers.

TL;DR Description

A set of Macros for Lisp interop with Root, CERN's C++ data analysis framework. The project uses Ferret, the Clojure-syntax to C++ compiler.

Python

CERN provides splendid Python interop for its C++ framework. Here is the most basic Python tutorial

import ROOT

class Linear:
    def __call__(self, arr, par):
        return par[0] + arr[0]*par[1]

# create a linear function with offset 5, and pitch 2
l = Linear()
f = ROOT.TF1('pyf2', l, -1., 1., 2)
f.SetParameters(5., 2.)

# plot the function
c = ROOT.TCanvas()
f.Draw()

Clojure

Translation to Ferret (see translation.clj)

(native-header "ROOT.h")
(require '[cxx :as ROO])

(defn Linear []
  (fn [[x] [d k]]
    (+ d (* x k))))

(def l (Linear))

(def c (ROO/T new TCanvas))

(def f ((ROO/T new TF1) "pyf2" l -1. 1. 2))
((ROO/T SetParameters TF1) f 5. 2.)
((ROO/T Draw TF1) f)

Since (ROO/T Draw TF1) is a macro call that expands into a lambda-function that does C++ calls, after adding the obvious bindings, the last three lines can be combined to one expression

(doto (newTF1 "pyf2" l -1. 1. 2)
  (SetParameters 5. 2.)
  Draw)

YAMLScript

I think the translation from Lisp syntax to YAMLScript looks pretty neat.

defn Linear():
  fn([x] [d k]): d + (k * x)

l =: Linear()

newTF1 =: ROO/T(new TF1)
SetParameters =: ROO/T(SetParameters TF1)
Draw =: ROO/T(Draw TF1)

doto:
  newTF1 'pyf2': l -1. 1. 2
  SetParameters: 5. 2.
  Draw:

Why YAMLScript

• YAMLScript is a possible future of scientific computing
• While Python is statement based, YAMLScript is expression based
• Expressions are like mathematical formulas, known to science since ages
• Nevertheless, YAMLScript looks similar to popular Python
• Mathematica(TM) language very successfully shows the way of expressions
• It is time to base whole scientific computing on expressions
• Python is good, but will be eventually replaced by something else
• C++ is fast and will still be there when Python is gone
• To mix C++ with Python, knowledge beyond the average scientist's is needed
• YAMLScript compiles to C++ and thus can readily be mixed with C++ when speed is more important than succinct notation
• In principle C++ is, through Cling, also an interpreted language, however as a major roadblock Cling is as of 2024 not feature complete

Handling Runtime Polymorphism in C++ Interop

Adding a Malli-style Schema,

(ROO/Ts [:TF1 :Draw :your-hint]
        [:string]
        [[:style ::one-letter]])

we can define a fallback enabled "multimethod" function that checks arguments at runtime and accordingly dispatches to different C++ calls,

(defn fallbackDraw [f params]
  (when (:mismatch ((ROO/T Draw TF1 :your-hint) f params))
    ((ROO/T Draw TF1) f)))

so we can call

(fallbackDraw f {:style "P"})

as well as

(fallbackDraw f {:style "unknown"})

Prerequisites

Install Root https://root.cern.ch

Install Java https://openjdk.org

Download ferret.jar from https://github.com/nakkaya/ferret

Optional: if you like to run the respective examples, also install Python or YAMLScript

Try it out

Run all scripts with

./runall.sh

Try it easier with Docker and make

If you have Docker and GNU make installed you use these commands (without installing anything else):

• make docker-runall

  Run the ./runall.sh script

• make docker-generate

  Generate all the test PDF files.

• OPENER=<some-opener-viewer-command> make open

  Open all the PDF files in some viewer. Tested with the Chromium browser using OPENER=chromium make open.

• make docker-shell

  Start a Bash interactive shell in Docker inside this repo. Environment is Ubuntu Linux 23.10 with all the repo deps pre-installed. Do anything including running make commands from inside. Bash history is saved between shell sessions.

• make clean or make docker-clean

  Delete all the generate files. Use docker-clean if some files end up being owned by root.

Types are valid Malli specs

To generate C++ code for accessing ROOT, type information about ROOT classes and methods is necessary. This information is stored as a Malli structure in the file malli_types.edn

Although Malli does not exist for Ferret, nevertheless, the conformity of any EDN structure contained in a file can be checked using standard Clojure:

clojure -Sdeps '{:deps {metosin/malli {:mvn/version "0.11.0"}}}' -M  mallitypes.clj

You need to install https://clojure.org to run this command.