100 Day ML Challenge to learn and implement ML/DL concepts ranging from the basics to more advanced state of the art models. Finished machine learning concepts from Andrew NG's course by Standford University on Coursera.
- Started "Machine learning- A Probabilistic Perspective" by Kevin Murphy.
- Introduction to machine learning- learnt about: Matrix completion, Image Inpainting, collaborative filtering, No free lunch theorem and Market Basket analysis.
- 4 Distance measures widely used in machine learning.
- Started by solving a problem related to the previous day which involved KNN and MNIST Dataset.
- Learnt about different types of regression namely Linear Regression, Logistic Regression, Ridge Regression, Lasso Regression and Polynomial Regression.
- Each one with their Equations and Graphs
- Understood the intuition behind SVMs.
- Implemented a simple classification model using scikit-learn SVM for the Bank-retirement dataset available on Kaggle.
- Understood the intuition behind Naive Bayes Classifier.
- Implemented a simple Naive Bayes Classification model using scratch for Iris dataset available on Kaggle.
- learnt the importance of hyperparameter tuning in machine learning algorithms.
- saw different parameters for SVMs, Naive Bayes and KNNs.
- Learnt concepts Bias, Variance, Cross Validation, Confusion Matrix.
- Implemented GridSearchCV and selected the best hyperparamter for Support Vector machine on a dataset.
- saw the stanford cheatsheet for basics of machine learning.
- Went through the documentation and implemented topics.
- Pandas cheatsheet
- Matplotlib Cheatsheet
- Seaborn Cheatsheet
- Learnt basics of kernal functions.
- Also started watching Stanford's CS299 lecture on kernels
- Learnt kernels for SVMs-polynomial and Radial Kernal(Radial bias function).
- Learnt kernels and filters for convolution.
- Referred to chapter 14 in Machine learning-A Probabilistic perspective by Kevin Murphy.
- Learnt Basics of Decision Trees through numerous examples.
- learnt how to calculate gini index and other parameters.
- Watched Video by StatQuest on Classification and regression Decision Trees.
- Learnt the intuition behind random forest and adaboost.
- learnt concepts related to proximity matrix and distance matrix.
- How to combine different learning algorithms and average their results.
- Advantages and disadvantages of decision trees.
- Implemented and visualised a decision tree using scikit learn and seaborn.
- Learnt how to prune regresstion trees using the residual sum of squares and tree score.
- Learnt about the ID3 algorithm, Entropy and Information gain.
- Started Stanford's CS299 lecture on Introduction to Neural Networks.
- Learned about: Equational form of neurons and models.
- Terminology
- Advantages of neural networks.
- Softmax
- Watched StatQuest video on very idea of Backpropagation.
- Learnt concepts of Chain rule in backpropagation and optimizing three parameters in a Neural Network simultaneously.
- Again analysed the concepts of Chain rule in backpropagation and optimizing three parameters in a Neural Network simultaneously.
- Learnt in depth the concepts of weights, bias, Gradient Descent and optimsation of parameters.
- Introduction to K nearest neighbours algorithm.
- Applications and Advantages and disadvantages of KNNs.
- Learnt about hyperparameter tuning of K.
- Watched StatQuest video on KNN.
- Learnt concepts like Euclidean Distance and how exactly the KNN Algorithm works.
- Revised certain concepts
- Learnt about various activation functions and which one to use for different models.
- Learnt about various optimizers.
- overfitting, underfitting and how to solve them.
- different cases with variance and bias and how to reach the appropriate training model.
- Learnt about data augmentation and how to implement it on deep learning models.
- Built a project on KNN that implements and uses Machine Learning in trading.
- Reference for the project.
Some of the things I learned today:
- What are convolutional neural networks?
- What is the function of the CNN kernel?
- Continued to read up on ConvNet.
- Learned about the max pooling layer.
Some of the things I learned today:
- What are recurrent neural networks?
- What makes RNNs more powerful than other architectures?
- Learned about the different RNNs architectures.
- Explored the different applications of RNNs.
- Learnt basics of LSTM and GRU. learnt the intuition behind them
Learnt the following today:
- basics of LSTM.
- The Core Idea Behind LSTMs(Long Short Trem Memory)
- Step-by-Step LSTM Walk Through
- Variants on Long Short Term Memory
- Learnt about update gate and reset gate.
- basic understanding of the hidden state.
- how to execute GRUs using keras.
- read about working of gates and GRUs.
- Worked on sequence prediction problem using RNN.
- Reference for the project
- In machine learning, the vanishing gradient problem is encountered when training artificial neural networks with gradient-based learning methods and backpropagation.
- Learnt the disadvantages of it.
- Learnt how to fix the vanishing gradient problem using ReLU.
Learnt basics of :
- NLP and its applications in various domains.
- Stop Words
- Sentence and Word Tokenization.
- Text Stemming
- Text Lemmatization
- Regex
- Bag-of-Words
Learn the basics of various types of vectorizers:
- Count Vectorizer
- Tf-Idf
- Word2Vec
- Worked on Sentiment Analysis using NLP.
- Sentiment analysis refers to the application of natural language processing, computational linguistics, and text analysis to identify and classify subjective opinions in source documents.
- Reference for the project.
In statistics, exploratory data analysis is an approach of analyzing data sets to summarize their main characteristics, often using statistical graphics and other data visualization methods
- Learnt about different proprocessing techniques on a dataset using pandas and numpy.
- Learn what plots are significant to different types of data.
- Learnt how to identify outliers.
- Learnt about various techniques to create models for predicting certain kind of outputs.
- How to make inferences and write a detailed report.
- Learnt about Batch Gradient Desent.
- Learnt Stochastic Gradient Desent.
- Learnt major differences between Batch Gradient Desent and Stochastic Gradient Desent.
- Other References.
- intuition behind K-means clustering
- Interesting visualization of K-means clustering
- Hyperparameter tuning in K-means clustering algorithm
- Learnt some of the applications of K-means clustering.
- Learnt the Math behind K-means clustering.
- Using Python (Pandas, NumPy) to gather and assess the data and scikit-learn to train a K-Means model to detect if a banknote is genuine or forged.
- Reference for the project
- Watched DeepLearningAl video on Gradient Descent with momemtum.
- Learnt the function of each parameter (especially the role of β in both (β) and (1 - β)) that is extremely useful in gradient descent.
- Read an article on momemtum optimiser.
Learned about:
- What Hopfield networks are.
- Learnt about the Hebbian postulate: "Neurons that fire together wire together, neurons out of sync, fail to link."
- Learnt about the energy function of the Hopfield Nets.
- How it is similar to our brain.
- Advantages of Hopfield Nets.
- How to use Hopfield networks.
- How Hopfield networks improve on the RNN model.
- Learnt what Boltzmann machines are.
- Learnt types of Boltzmann Machines and working of them.
- Learnt:
- Restricted Boltzmann Machines (RBMs)
- Deep Belief Networks (DBNs)
- Deep Boltzmann Machines (DBMs)
Learned about:
- What Autoencoder networks are.
- How an Autoecoder functions.
- The components that make up an Autoencoder.
- Applications of Autoencoders.
- Watched a video on Deep Belief Networks(DBN).
- DBN method is also known as Layer-wise, unsupervised, greedy pre-training.
- Learnt that in this type of net each layer ends up learning the full input structure.
- Learnt that DBN is an effective solution to the vanishing gradient problem.
- A restricted Boltzmann machine is a generative stochastic artificial neural network that can learn a probability distribution over its set of inputs.
- Reference
- Learn their importance in deep belief networks.
- Practical Application of DBN's on MNIST Dataset.
- Reference to the project
- Taking time off to study for 3rd sem internals and final exams. Will be back soon !!
- Did a short Literature survey:
- written by Sushil S. Ranavare and Rajani S Kamath
- Artificial Intelligence based Chatbot for Placement Activity at College Using DialogFlow
- NLP module of DialogFlow translates students’ queries into structured data in order to understand institute’s service
Learnt
- Basic working of Sequence To Sequence (Seq2Seq) model.
- Encoder-Decoder Architecture.
- Reference
- This technique involves taking a subset of data from a given array or string, expanding or shrinking that subset to satisfy certain conditions, hence the sliding effect.
- It is also used in object detection and many other machine learning algorithms.
- Learnt the basics of the algorithm and it's applications.
- Reference
- Understood the intuition behind Attention Models.
- Reference
- Understood the intuition behind the algorithm.
- Applications of the YOLO algorithm.
- Reference
Revised :
- Encoder-Decoder Model.
- Attention Model.
Learnt :
- How does Attention work in Encoder Decoder RNN(Recurrent Neural Networks)
- Worked examples of Attention.
- Reference
- Learnt various applications.
- AI in the news.
- Recent developments in the field.
Learnt :
- Basics of ResNet Architecture.
- Working of ResNet.
- Did a short Literature survey:
- written by Mahsa Shafaei, Niloofar Safi Samghabadi, Sudipta Kar and Thamar Solorio
- Age Suitability Rating: Predicting the MPAA Rating Based on Movie Dialogues
- Movies help us learn and inspire societal change. But they can also contain objectionable content that negatively affects viewers’ behavior, especially children. In this paper, our goal is to predict the suitability of movie content for children and young adults based on scripts. The criterion that we use to measure suitability is the MPAA rating that is specifically designed for this purpose. We create a corpus for movie MPAA ratings and propose an RNN-based architecture with attention that jointly models the genre and the emotions in the script to predict the MPAA rating. We achieve 81% weighted F1-score for the classification model that outperforms the traditional machine learning method by 7%.
- Understood ResNet in Keras.
- Built a ResNet50 Model.
- Reference
Learnt :
- Basics of MobileNet Architecture.
- Image classification with MobileNet.
- Learnt both the ResNet and MobileNet models again.
- Implemented both ResNet and MobileNet models on CIFAR-10 dataset.
- Analysed the efficiency of both the models on CIFAR-10 dataset.
- Reference
- Build an Image classification model using the pretrained model.
- used Keras for the implementation.
Learnt:
- VGG(Visual Geometry Group)16 Architecture.
- Performance of VGG models.
- Reference
- VGG16 Neural Network Visualization
- started off with matrix theory.
- learnt about linear combination.
- vector spaces.
- Learnt about Matrices and Matrix Arithmetic.
- Implemented them using numpy and pandas.
- Learnt Matrix Types in Linear Algebra for machine learning.
- Went through few Matrix Operations for machine learning.
- learnt about gaussian elimination.
- rank of a matrix.
- Built a Simple Photo Classifier using VGG16.
- Reference
- In machine learning, the vanishing gradient problem is encountered when training artificial neural networks with gradient-based learning methods and backpropagation.
- learnt :
-
- What is vanishing gradient problem
- Read an article titled "The 10 Neural Network Architectures Machine Learning Researchers Need To Learn" - which summarised all the models we learnt so far.
- Reference
- Adam is a replacement optimization algorithm for stochastic gradient descent for training deep learning models.
- Adam combines the best properties of the AdaGrad and RMSProp algorithms to provide an optimization algorithm that can handle sparse gradients on noisy problems.
- Intuition behind Adam.
- learnt why Adam is used over other optimizers.
- Learnt basics of NAG(Nesterov Accelerated Gradient) Descent.
- Reference
- Cross-Entropy Loss Function. Also called logarithmic loss, log loss or logistic loss.
- Cross Entropy is definitely a good loss function for Classification Problems, because it minimizes the distance between two probability distributions - predicted and actual.
- Reference
Learnt:
- The Cost Function of Cross-Entropy.
- Multi-class Classification Cost Functions.
- Binary Cross-Entropy Cost Function, Sparse Categorical Cross-Entropy, Categorical Cross-Entropy.
- Reference
- Adaptive Gradient Algorithm (Adagrad) is an algorithm for gradient-based optimization.
- It performs smaller updates As a result, it is well-suited when dealing with sparse data (NLP or image recognition) Each parameter has its own learning rate that improves performance on problems with sparse gradients.
- Reference
- Adadelta is an extension of Adagrad that attempts to solve its radically diminishing learning rates.
- Adadelta is a machine learning optimization algorithm that was created by Matthew D. Zeiler with the goal of addressing two drawbacks of the Adagrad method.
- Reference
- It is an improvement to the Adagrad optimizer.
- This aims to reduce the aggressiveness of the learning rate by taking an exponential average of the gradients instead of the cumulative sum of squared gradients.
- Reference
- Read an article on "Various Optimization Algorithms For Training Neural Network" which summarized all the optimizers we learnt so far.
- Reference
- Understood how artificial neurons and perceptrons were inspired by the biological processes and tried to draw the relationship between both.
- Reference
- Cognitive science is the interdisciplinary, scientific study of the mind and its processes with input from linguistics, psychology, neuroscience, philosophy, computer science/artificial intelligence, and anthropology.
- Read about Donald Hoffman's Case against reality
- Read an article about cognitive sciences and machine learning.
Problem Statement:
- Provide a sophisticated and useful model using model using different NLP or ML techniques.
- Find out the most useful drugs for each condition.
- Find out some of the hidden trends or patterns that could help the company to make precise data driven decisions.
Problem Statement:
- to identify the view or emotion behind a situation. It basically means to analyze and find the emotion or intent behind a piece of text or speech or any mode of communication.
- Reference
Problem Statement:
- Stock market investments generate great profits and is a fundamental stimulus for most of the market researchers in this area.Technical analysis focuses on analyzing the direction of prices to predict future prices.
- It's mostly based on analysing text and sentiments, where we have a bunch of numeric past information which has to be mapped to a number which we expect to be the future price which suits to be a perfect problem to be solved using Deep learning.
- Technology: Python (along with pandas, matplotlib, sklearn libraries).
- Applied SVM, K-Nearest Neighbors, Logistic Regression, Naïve Bayes and Random Forest algorithms to the Wisconsin Breast Cancer dataset from the UCI ML Repository (Kaggle)
- To predict whether the breast cancer tumor is malignant or benign.
- Compared the performance results of all the algorithms based on the accuracy and ROC values.
- Dealing with Image data sets.
- Performing Data Processing and Augmentation as and when required.
- Creating and training a Convolutional Neural Network using ResNet Architecture.
- Predicting the missing number given the previous number sequence using LSTM on Keras.
- ToyLSTM.py is a toy LSTM model to learn simple number sequences consisting of four consecutive numbers. The goal is to predict the last number given the first three consecutive numbers.
- Data Visualization and Exploratory Data Analysis on Zomato dataset to get a fair idea about the factors affecting the aggregate rating of each restaurant, establishment of different types of restaurant at different places in Bengaluru.
- Use of Pandas ,Seaborn, Matplotlib ,Plotly ,Folium and Wordcloud.
- Depth analyses with Bar graphs, Scatter Plots, Displots, Pie charts, Wordcloud.
- You can learn how to distinguish fake news from a real one. You can use supervised learning to implement a model like this.
- Data Analysis and Visualiztion to draw inference on certain factors like Maximum Win by Runs, Highest Wins by Wickets, Maximum Toss Winners, Total Matches vs Wins for Teams, Runs Across the Seasons, Comparison between 2 Teams on the Basis of their Wins and many more.
- Implement Convolution Neural Network(CNN) Classifier for Classifying dog and cat images.
- The Total number of images available for training is 25,000 and final testing is done on seperate 10,000 images.
-
Unsupervised learning approach to find the sentences similarity and rank them accordingly using Cosine Similarity.
-
Cosine similarity is a measure of similarity between two non-zero vectors(here sentences are represented as vectors) that measures the cosine of the angle between them. Angle will be 0 if sentences are similar.
-
Code flow:
Input article → split into sentences → remove stop words → build a similarity matrix → generate rank based on matrix → pick top N sentences for summary.
Day-83 [11/02/2022] Research Paper 1 ==> Large scale video classification using convolution networks:
- Used to classify 1 million Youtube videos belonging to 487 classes.
- Video is treated as a range of short fixed size clips and passed through a convolution.
- Paper link
Day-84 [12/02/2022] Research Paper 2 ==> Using Support Vector Machines to Classify Student Attentiveness for the Development of Personalized Learning Systems:
- This article describes a system that uses a commercial RGB-D camera to monitor, count, and record student gestures, postures, facial expressions, and verbalizations in order to produce data for determining student attentiveness.
- Machine learning algorithms are then used to cluster, label, and classify the data for the purpose of classifying subsequent students as attentive or inattentive.
Day-85 [14/02/2022] Research Paper 3 ==> Classification Technique of Interviewer-Bot Result using Naïve Bayes and Phrase Reinforcement Algorithms:
- The response indicating that the Naïve Bayes method and Reinforcement Phrase algorithm can be used to classify the inter-viewer-bot interview result. In addition, interviewer-bot system has the intelligent ability so it could contribute positively for students practicing interviews in English.
- The accuracy level is about 86.93%.
- Link
- The paper investigates how to detect the horizon line in a set of images using a machine learning approach.
- The performance of the SVM, J48, and Naïve Bayes classifiers, used for the problem, has been compared.
- Accuracy of 90-99% in identifying horizon was achieved on image data set of 20 images.
Day-87 [16/02/2022] Research Paper 5 ==> Predicting Frequently Asked Questions (FAQs) on the COVID-19 Chatbot using the DIET Classifier:
- chatbot answers to questions about COVID-19 with the RASA framework and uses the DIET Classifier pipeline for 300 training data
- DIET Classifier model can predict each word or sentence from the chatbot with different confidence values for all of the intents used.
- The Paper discusses on a study aimed to develop an approach that is able to identify the vehicles through aerial images by using the YOLO (You Only Look Once) algorithm, feeding it with a trained convolutional neural network structure.
- As a result of the study, an application that can detect vehicles was developed, increased the performance rate of the YOLO algorithm by 3.2%.
- The project involves creating a chatbot to be used in healthcare treatment. Helps in diagnosing various diseases and providing necessary details about the patient's disease or illness. It also enhances the user to chat and know about their medical status and issues.
- Implements chatbot using NLTK, sentence tokenizing and stemming. Basically breaks sentences into words, and finds similar words in database using cosine similarity
- link
Day-90 [25/02/2022] Research Paper 8 ==> Identifying Traffic Congestion Pattern using K-means Clustering Technique:
- The aim of this paper is to propose a traffic congestion pattern recognition system. It acquires real time road traffic information and categorizes the different road segments based on traffic density and average speed of vehicles.
- The traffic parameters are captured by in-road stationary sensors deployed in road segments. The proposed system uses k means clustering algorithm to categorize the different road segments.
Day-91 [28/02/2022] Research Paper 9 ==> Traffic Graph Convolutional Recurrent Neural Network: A Deep Learning Framework for Network-Scale Traffic Learning and Forecasting:
- Traffic forecasting, Spatial-temporal, Graph convolution, LSTM, Recurrent neural network
- link
Day-92 [04/03/2022] Research Paper 10 ==> Diagnosis of Crime Rate against Women using k-fold Cross Validation through Machine Learning Algorithms.
- The main aim of this work is to develop prediction model that can be used to predict crime rate accurately.
- This paper introduces six different types of Machine learning algorithms such as KNN and decision trees, Naïve Bayes, Linear Regression CART (Classification and Regression Tree) and SVM using similar characteristics on crime data. Those algorithms are tested for accuracy.
- The main objective of this research is to evaluate the efficacy and application of the machine learning algorithms in data analytics.
- Started watching stanford cs224w videos for graph theory.
- Overview of graphs, why we need graphs and applications of graphs in ml.
- Read about operations on graphs using Node, edges and graph centric operations
- various methods and techniques related to random walk and its optimisation.
- Learnt about page rank and personalised page rank.
Day-95 [13/03/2022] Semi-Supervised Learning, Correct & Smooth, Relational and Iterative Classification, Deep Graph Encoders:
- Learnt about 2 approaches to collective classification i.e Relational and Iterative Classification.
- Learnt how to leverage correlation in graphs to make prediction on nodes.
- Started with Deep Graph Encoders.
Day-96 [14/03/2022] Basics of neural networks, Idea for Deep Learning for Graphs, GNN vs CNN, GNN Augmentation and training
Learnt:
- Loss, Optimization, Gradient, SGD, non-linearity, MLP, Multiple layers of embedding transformation, At every layer use the embedding at previous layer as the input, Aggregation of neighbors and self-embeddings.
- CNN and Transformer can be viewed as a special GNN.
- Transformation + Aggregation, Classic GNN layers: GCN, GraphSAGE, GAT.
- The over-smoothing problem, Feature augmentation, Structure augmentation etc.
- Learnt about knowledge graphs and relations between them.
- Learnt about Subgraphs, how to query subgraphs and frequency of subgraphs.
- Learnt different types of graph models.
- Kronecker graph model
- small world model
- How to generate graph models.
Day-99 [17/03/2022] Adversarial robustness of GCN, Adversarial direct attack, Random attack, Indirect attack:
- GCN is not robust to adversarial attacks but it is somewhat robust to indirect attacks and random noise.
- Deep learning models are often not robust.
- Clustering of GNNs
- Hyperbolic Graph embeddings
- Pre training Graph neural networks