Mohammed Sabry's repositories
BERT_with_Residual_vs_Highway
Comparing between residual stream and highway stream in transformers(BERT) .
baby_mamba
An implementation of Mamba to develop an understanding of its functioning.
Universal_Decoder
This is the Transformers Decoder version, based on the "Universal Transformers (Encoder-Decoder)" paper, and trained on natural data rather than algorithmic tasks.
FP_vs_1Bit_InductionHeadCircuit
Comparing the full-precision Query, Keys, and Values matrices with their 1-bit counterparts in a two-layer, attention-only transformer trained on a synthetic copying task.
16S-rDNA-dada2-pipeline
dada2 Nextflow pipeline
Billing-System
Educational project
CollectionOf-ML-Aritcles
Articles
Sudan_Temperature1901-2016_
Visualizing the change of Sudan temperature from 1901 to 2016
ECNMT
Emergent Communication Pretraining for Few-Shot Machine Translation
EigenWords
Playing with SVD and Word Vectors
FCnet_fromScratch_onTitanic_dataset
learning project
function_vectors
Function Vectors in Large Language Models
ML_DL-Tutorials
Supplementary Machine Learning and Deep Learning tutorial sessions with code to AMMI program 2019-2020 cohort.
pytextrank
Python implementation of TextRank algorithms ("textgraphs") for phrase extraction