BeeDNN is a deep learning library.
The git repository is https://github.com/edeforas/BeeDNN
The API is clear and simple, the goal is that every C++ developer can contribute, test, and use Deep Neural Network. BeeDNN can run on small devices. It is even possible to learn directly on the device.
Please see at: https://github.com/edeforas/BeeDNN/issues for contributing ideas. No dependencies needed, every algorithm rewritten in C++ from scratch. To increase speed, ones can choose the Eigen library (http://eigen.tuxfamily.org), instead of the internal matrix library.
Initializers:
- GlorotUniform, GlorotNormal
- HeUniform, HeNormal
- LecunUniform, LecunNormal
- Zeros, Ones
Layers:
- Dense (with bias), Dot (without bias)
- GlobalGain, GlobalBias, GlobalAffine, Gain, Bias, Affine
- Softmax, Softmin
- PRelu, RRelu, PELU, TERELU, CRelu
- Gated activations: GLU, ReGLU, Bilinear, SwiGLU, GEGLU, GTU, SeGLU
- Layers and activations are decoupled and can be in any order
Time series:
- TimeDistributedBias
- TimeDistributedDot
- TimeDistributedDense
- WIP
2D layers:
- Convolution2D, ChannelBias
- MaxPool2D, GlobalMaxPool2D,
- AveragePooling2D, GlobalAveragePooling2D
- ZeroPadding2D
Activations (in alphabetical order):
- Absolute, Asinh, Atan
- Bent, BinaryStep, Bipolar, BipolarSigmoid, Bump
- ComplementaryLogLog, CELU
- dSiLU
- ELiSH, Elliot, ELU, Exponential, E2RU, E3RU, Eswish
- FTS, FTS+
- Gauss, GELU
- HardELU, HardSigmoid, HardShrink, HardTanh, HardSwish, Hann
- ISRELU
- Linear, LeakyRelu, LeakyRelu256, LecunTanh, LiSHT, Logit, LogSigmoid
- Mish
- NLRelu
- O2RU, O3RU
- SmoothSoftPlus
- SmoothTanh
- SmoothSigmoid
- Relu, Relu6
- Selu, SQNL, SQ-RBF, SoftPlus, Sin, SinC, Sigmoid, SiLU, Swish, SoftShrink, SoftSign, SoftSteps, SineReLU
- Tanh, TanhExp, TanhShrink, ThresholdedRelu,TwiceLeakyRelu6
Loss functions:
- MeanSquareError, MeanAbsoluteError, MeanCubicError
- L2, L1, L3
- LogCosh
- Huber, PseudoHuber
- SparseCategoricalCrossEntropy, CategoricalCrossEntropy, BinaryCrossEntropy
Overfitting:
- Layers: Dropout, GaussianNoise, GaussianDropout, UniformNoise
- Regularizer: GradientClip, GradientNormClip, GradientClipTanh, L1, L2, L1L2
Data augmentation
- RandomFlip
Learning and optimization:
- Classification or regression, test and/or learn
- SGD, Momentum, MomentumNg, Nesterov
- Adam, AdamW, Nadam
- Adagrad
- Adamax
- Amsgrad
- RMSprop
- RPROP-, iRPROP-
- MetaOptimizer (V1)
- Class balancing if needed
- Keep best model vs epochs
- LearnMore mode
- Reboost mode
KMeans:
- can use any loss
- batchmode learning
I/O:
- MNIST reader
- CIFAR10 reader
- csv file reader
- model, weights and training parameters are saved in a simple .json file
Precomputing:
- StandardScaler, MinMaxScaler
Commented samples:
- Simple XOR classification, with and w/o softmax
- Simple sinus regression
- MNIST with dense net
- MNIST using time serie (a time frame is an image row)
- MNIST with poolmax2D
- MNIST all convolutional
- MNIST and Meta Optimizer: select best activation
- CIFAR10 conv2D using poolmax2D
- MNIST with kmeans and custom loss
Build with vs2019 or CMake. To compile, run the samples, etc, please read the HOWTO.md file