theolepage / tensorflood

Tiny automatic differentiation (autodiff) engine for NumPy tensors implemented in Python.

TensorFlood

Tiny automatic differentiation (autodiff) engine implemented in Python.

The following resources were used while creating this project:

• Build Your Own Automatic Differentiation Program by Jonathan Kernes
• Yaae by Ferdinand Mom

Example

The notebook examples/classification.ipynb demonstrates how to use the engine and how it can be used to train a small neural network.

Usage

from tensorflood.engine import Graph, Variable, Constant

with Graph() as g:
    x = Variable(2, name='x')
    y = Variable(10, name='y')
    c = Constant(7, name='c')
    
    z = (x * y) + c
    # z.data = 27
    
    z.backward()
    # x.grad = 10 (dz/dx)
    # y.grad = 2  (dz/dy)
    # c.grad = 1  (dz/dc)

To-Do

• NN: Adam optimizer, more layers
• Merge backward_ and backward
• Refactor hasattr(node, 'input_nodes')
• Unit tests with pytorch