sekwiatkowski / komputation

Komputation is a neural network framework for the Java Virtual Machine written in Kotlin and CUDA C.

Komputation

Komputation is a neural network framework for the Java Virtual Machine written in Kotlin and CUDA C.

Maven

Komputation is available through Maven Central:

<dependency>
    <groupId>com.komputation</groupId>
    <artifactId>komputation</artifactId>
    <version>0.12.5</version>
</dependency>

Layers

• Entry points:

  • Input
  • Lookup

• Standard feed-forward networks:

  • Weighting
  • Bias
  • Projection
  • Dense

• Convolutional neural networks (CNNs):

  • Convolution
  • Max-pooling

• Recurrent neural networks:

  • Recurrent layer
  • Bidirectional recurrent layer

• Dropout

• Activation functions:

  • Identity
  • Rectified Linear Units (ReLUs)
  • Sigmoid
  • Softmax
  • Tanh

• Other layers:

  • Stack
  • Exponentiation
  • Normalization

CPU demos

• Boolean functions:

  • AND
  • NOT
  • XOR

• Total:

  • Fixed length
  • Variable length

• Running total:

  • Left-to-right:
    • Fixed length
    • Variable length
  • Right-to-left:
    • Fixed length
    • Variable length
  • Bidirectional:
    • Fixed length
    • Variable length

• Increment:

  • One layer
  • Two layers

• Word embedding toy problem:

  • Feed-forward network
  • CNN with one filter width
  • CNN with two filter widths

• Sequence labeling toy problem

• Computer vision toy problem

• MNIST:

  • Minimal
  • Dropout

• TREC:

  • One filter width
  • Two filter widths

GPU/CUDA demos

• Boolean functions:

  • AND
  • Negation
  • XOR

• Word embedding toy problem:

  • Feed-forward network
  • CNN with one filter width
  • CNN with two filter widths

• Total:

  • Fixed length

• Increment:

  • One layer
  • Two layers

• MNIST:

  • Minimal
  • Dropout

• TREC:

  • One filter width
  • Two filter widths

Sample code

The following code instantiates a GPU-accelerated convolutional neural network for sentence classification:

    val sentenceClassifier = cudaNetwork(
        batchSize,
        lookup(embeddings, maximumDocumentLength, embeddingDimension, optimization),
        convolution(numberFilters, filterWidth, filterHeight, initialization, optimization),
        relu(),
        dropout(random, keepProbability),
        dense(numberCategories, Activation.Softmax, initialization, optimization)
    )

See the TREC demo for more details.

Initialization

• Provided
• Constant
• Gaussian
• He
• Identity
• Uniform
• Zero

Loss functions

• Cross-entropy loss
• Logistic loss
• Squared loss

Optimization

• Stochastic Gradient Descent
• Historical:
  • Momentum
  • Nesterov's Accelerated Gradient
• Adaptive:
  • Adagrad
  • Adadelta
  • RMSProp
  • Adam