A simple C++11 deep learning framework, easy to use

Introduce

LittleConv is a simple deep learning framework based on pure c++11, and does not need anything else. It is suitable for deep learning on limited computational resource on macos and linux.And it is cpu only.

Advantage

Modern C++
MultiThread
Simple Interface
Easy To Install

Dependecy

g++4.8 or later(with c++11)
cmake tools
linux or macos

Compile and Build

It uses cmake to build the project, run shell command as follows:

mkdir -p build
cd build
rm -rf *
cmake ..
make -j8

The output is a shared library(.so file) in the lib directory.

How To Use

There are some special examples in the example directory such as lenet-5, cifar10-quick. LittleConv has given interfaces to load mnist and cifar10-bin libraries.

How To Build And Run Examples

go to the example directory and run the script(Users need to modify the link_directoies in the CMakeLists.txt)

cd example
sh build_example.sh
cd bin
./lenet --input_dir=xxx --mode=train

How To Define Model

The most important for deep learning framework is defining models.
LittleConv has an easy way to define models, just like below:

nn::Model get_model() {
    auto block = nn::Squential::CreateSquential();
    block->Add(nn::Conv::CreateConv(1, 20, 5));
    block->Add(nn::Relu::CreateRelu());
    block->Add(nn::MaxPool::CreateMaxPool(2)); //12 * 12

    block->Add(nn::Conv::CreateConv(20, 50, 5));
    block->Add(nn::Relu::CreateRelu());
    block->Add(nn::MaxPool::CreateMaxPool(2)); //4 * 4

    block->Add(nn::Linear::CreateLinear(4 * 4 * 50, 500));
    block->Add(nn::Relu::CreateRelu());
    block->Add(nn::Linear::CreateLinear(500, 10));
    block->Add(nn::Softmax::CreateSoftmax());
    return block;
}

How To Train Models

It is also easy to train the model

    // the parameters
    int batch_size = 256;
    int epoch_num = 10;
    float lr = 1e-4;
    int display = 150;
    // get model
    auto model = get_model();

    model->set_node_name("Lenet-5");

    // define the loss
    auto cross_entropy_loss = nn::CrossEntropyLoss::CreateCrossEntropyLoss();

    // define the optimizer
    auto optimizer = optim::Adam::CreateAdam(model->RegisterWeights(), lr);

    // load data
    io::MnistDataset mnist(input_dir, io::MnistDataset::TRAIN);
    io::MnistDataset mnist_test(input_dir, io::MnistDataset::TEST);
    mnist.Load();
    mnist_test.Load();

    // define a trainer to train the classification task
    auto trainer = utils::ClassifyTrainer::CreateClassfyTrainer(model, cross_entropy_loss, optimizer, mnist,
        batch_size, epoch_num, display, &mnist_test);

    // train
    trainer->Train();
    // save model
    io::Saver::Save("lenet-5.liconmodel", model);

Users only need to define the loss, optimizer and the dataset(mnist or cifar10, other dataset need users to write the dataset reading interface themselves).
Then pass them to the trainer and call the Train interface.
Actually, the process is very simple. Model has the interface Forward and Backward, users can DIY the train and test process themselves.
And the all codes are in the licon namespace.
Please see the example of lenet-5 or cifar10_quick.

Supported networks

layer-types and activation functions

fully-connected
dropout
convolution
average pooling
max pooling
softmax
tanh
sigmoid
relu(leaky relu)
ELU
SELU
Softplus
batch-normalization

loss functions

cross-entropy

optimization algorithms

SGD(with momentum and nesterov)
Adam
RMSprop

optimization adjust method

LambdaLR
StepLR

layer/operation node container

Squential
Stack
EltWiseSum
ChannelConcat

Todo List

normalization layers
split and concat layers
more loss functions
more optimization algorithms

xiaoxiaoluo1 / LittleConv