non / onnx-scala

An ONNX (Open Neural Network eXchange) API and Code Generator for Typeful, Numerically Generic, Functional Deep Learning in Scala

ONNX-Scala

This project currently provides:

A) a complete, versioned, numerically generic, type-safe / typeful API to ONNX(Open Neural Network eXchange, an open format to represent deep learning and classical machine learning models), derived from the Protobuf definitions and the operator schemas (defined in C++) via the JavaCPP Preset for ONNX.

This API is expressed via traits, with version-named methods. For example, Abs, the absolute value operator (defined in operator sets 1 and 6):

trait Abs extends Operator {

    def Abs1[@sp T: Numeric: ClassTag](name: String,
                                       consumed_inputs: Option[(Array[Int])] =
                                         None,
                                       X: Option[Tensor[T]])(
        implicit evT: (UNil TypeOr Float16 TypeOr Float TypeOr Double TypeOr UByte TypeOr UShort TypeOr UInt TypeOr ULong TypeOr Byte TypeOr Short TypeOr Int TypeOr Long TypeOr Float16 TypeOr Float TypeOr Double)#check[
          T]): (Tensor[T])

    def Abs6[@sp T: Numeric: ClassTag](name: String, X: Option[Tensor[T]])(
        implicit evT: (UNil TypeOr Float16 TypeOr Float TypeOr Double TypeOr UByte TypeOr UShort TypeOr UInt TypeOr ULong TypeOr Byte TypeOr Short TypeOr Int TypeOr Long TypeOr Float16 TypeOr Float TypeOr Double)#check[
          T]): (Tensor[T])

}

B) a program generator, capable of translating ONNX model Protobuf (.onnx) files into Scala programs written in terms of this API. For example, an "absolute value network":

trait AbsNet {
  val dataSource: DataSource
  val abs: Abs
  def program[
      T: (UNil TypeOr Float16 TypeOr Float TypeOr Double)#check: Numeric: ClassTag]
    : List[Tensor[T]] =
    for {
      nodedata <- List(dataSource.inputData[T])
      nodeabs <- List(
        Abs.Abs6[T](
          "abs",
          X = Some(nodedata)
        ))
    } yield (nodeabs)
}

ONNX-Scala will also be home to:

C) at least one, eventually many backend implementations of ONNX accessible from the JVM.

Currently there is a rough sketch of an nGraph backend, which is capable of running a single ReLU node network, based on the new nGraph JavaCPP Preset.

All together, these should enable model inspection and modification, extra compile-time assurances, mixing/matching of backend operator implementations and integration into JVM-based production systems, for a start.

Getting Started

You'll need sbt.

sbt publishLocal

or

sbt +publishLocal

to build against all of Scala 2.11, 2.12, 2.13 Milestone, and Dotty/3.0, where possible.

Then you can add this to your project's build.sbt

libraryDependencies += "org.emergentorder.onnx" %% "onnx-scala" % "1.4.1-0.1.0-SNAPSHOT"

or

libraryDependencies += "org.emergentorder.onnx" %% "onnx-scala-free" % "1.4.1-0.1.0-SNAPSHOT"

and build away with the traits provided. Backend implementation (and other) PRs welcome!

Program Generator

To generate an ONNX-Scala program from an ONNX Protobuf file (often *.onnx):

Depending on the size of the ONNX model, you may need to

export SBT_OPTS="-XX:+CMSClassUnloadingEnabled -Xmx28G -Xss8M -XX:MaxMetaspaceSize=1024M"

either each time in the terminal, or in your ~/.bashrc file or equivalent, or you will encounter errors.

Now, get squeezenet.onnx here (rename model.onnx from inside the tar.gz). This model and more can be found in the ONNX Model Zoo.

Then:

sbt "project programGeneratorJVM" "run squeezenet.onnx"

The resulting generated program appears as programGenerator/src/main/scala/generatedprograms/Squeezenet.scala and you can run sbt compile to confirm that the generated code compiles.

Project Details

Automatic differentiation to enable training is under consideration (ONNX does not provide facilities for training).

Balancing the interests of minimal imposition of dependencies with stack-safe, purely functional programming, ONNX-Scala comes in two flavors: Vanilla and Freestyle-infused.

ONNX-Scala is cross-built against Scala JVM (for Scala 2.11, 2.12 and 2.13.0-M5) , Scala.js / JavaScript (for Scala 2.11 and 2.12) and Scala Native (for Scala 2.11).

To take advantage of union types to express type constraints, a Dotty (Scala 3) build is available. The Dotty build does not support Scala.js or Scala Native.

Due to Freestyle's dependency on the EOLed scalameta paradise compiler plugin, the free variant is not available for Scala 2.13 or Dotty.

Currently at ONNX 1.4.1.

Built With

Core

• Spire - Typelevel project enabling generic numeric programming (For support for unsigned ints, complex numbers, the Numeric type class and type specialization to avoid boxing overhead)

• Singleton-ops - Operations for primitive and String singleton types (For compile-time dimension checking)

Optional - Dotty Variant

• Dotty - A next-generation compiler that will become Scala 3 (For native union types, used here to express ONNX type constraints)

Optional - Free Variant

• Cats-effect - standard IO type together with Sync, Async and Effect type classes (Or your effect type here)

• Freestyle - pure functional framework for Free and Tagless Final apps & libs (For stack safety and parallelism without sacrificing composition)

Program Generator

• JavaCPP Preset for ONNX 1.4.1 - The missing bridge between Java and native C++ libraries (For access to Protobuf definitions and operator schemas)

• Scalameta - Library to read, analyze, transform and generate Scala programs (For a runtime parse pass of generated programs)

Inspiration

• Neurocat - From neural networks to the Category of composable supervised learning algorithms in Scala with compile-time matrix checking based on singleton-types

• Nexus - Experimental typesafe tensors & deep learning in Scala

• Lantern - Machine learning framework prototype in Scala. The design of Lantern is built on two important and well-studied programming language concepts, delimited continuations (for automatic differentiation) and multi-stage programming (staging for short).

• Backprop - Heterogeneous automatic differentiation ("backpropagation") in Haskell

• Grenade - Grenade is a composable, dependently typed, practical, and fast recurrent neural network library for concise and precise specifications of complex networks in Haskell.

• DeepLearning.scala - A simple library for creating complex neural networks

• Deeplearning4j / Scalnet / ND4S - ScalNet is a wrapper around Deeplearning4J emulating a Keras like API for deep learning.