zhuhaipeng-byte / NN_Test

This project contains some neural network code

NN_Test

This project contains some neural network code(Note: Clone this repository to E:/GitCode/ in windows):

• neural network code(C++)
  • Perceptron
  • BP(Back Propagation)
  • CNN(Convolutional Neural Networks)
  • Linear Regression(gradient descent、least squares)
  • Naive Bayes Classifier(sex classification)
  • Logistic Regression(gradient descent, Batch/Mini Batch)
  • KNN(K-Nearest Neighbor, classification)
  • PCA(Principal Components Analysis)
  • single hidden layer(two categories)
  • K-Means
  • Decision Tree(CART)
• mathematical formula's implement with C++
  • Linear Algebra
    • transpose
    • determinant
    • adjoint matrix
    • inverse matrix
    • norm(vector、matrix)
    • eigenvalues/eigenvectors(real symmetric matrix)
    • SVD(Singular Value Decomposition)
    • pseudoinverse
    • trace
  • Probability
    • mean、variance、standard deviation
    • covariance matrix
  • Activation Function
    • logistic sigmoid function
    • softplus function
    • ReLU(rectified linear unit)
    • Leaky ReLUs(leaky rectified linear units)
    • ELUs(Exponential linear units)
    • softmax function
    • tanh(Hyperbolic tangent)
  • Loss Function
    • MSE(Mean Square Error)
    • Cross-Entropy
  • Optimization
    • BGD(Batch Gradient Descent)
    • SGD(Stochastic Gradient Descent)
    • MBGD(Mini-batch Gradient Descent)
    • SGD with Momentum
    • AdaGrad(Adaptive Gradient)
    • RMSProp(Root Mean Square Propagation)
    • Adadelta(an adaptive learning rate method)
    • Adam(Adaptive Moment Estimation)
    • AdaMax(a variant of Adam based on the infinity norm)
    • NAG(Nesterov Accelerated Gradient)
    • Nadam(Nesterov-accelerated Adaptive Moment Estimation)
  • Layers
    • Dropout
    • LRN(Local Response Normalization)
    • BN(Batch Normalization)
  • common algorithm
    • Brute Force(string match)
• open source library's usage(C++)
  • tiny-cnn GitHub
  • tiny-dnn GitHub
  • libsvm GitHub
  • OpenCV 3.3 GitHub
    • Logistic Regression
    • Support Vector Machines(SVM)
    • KNN(K-Nearest Neighbor)
    • Decision Tree
    • PCA(Principal Components Analysis)
    • k-means
• Python code(support windows and linux platform)
  • KNN(K-Nearest Neighbor)
  • Decision Tree(CART)
  • PCA(Principal Components Analysis)
  • k-means
• Database convert to Images or Images conver to Database
  • MNIST
  • CIFAR10/CIFAR100
  • ORL Faces Database

The project support platform:

• windows7/10 64 bits: It can be directly build with VS2017 in windows7/10 64bits.
• Linux:
  • DatasetToImage support cmake build(file position: prj/linux_cmake_DatasetToImage)
  • Math support cmake build(file position: prj/linux_cmake_Math)
  • tiny-cnn support cmake build(file position: prj/linux_cmake_tiny-cnn)
  • tiny-dnn support cmake build(file position: prj/linux_cmake_tiny-dnn)
  • NN support cmake build(file position: prj/linux_cmake_NN)

Windows VS Screenshot:

Blog: fengbingchun