This repo holds my notes from when I learned information theory in summer 2023.
- Chapter1: Probability
- Chapter2: Measure of Information
- Chapter3: Entropy Rate
- Chapter4: Markov Chains
- Chapter5: Source Coding: Lossless Compression
- Chapter6: Channel Coding
Reference:
https://www.probabilitycourse.com/ https://mbernste.github.io/posts/self_info/#:~:text=Information
https://en.wikipedia.org/wiki/Entropy_(information_theory)
https://www.studypage.in/physics/model-communication-system
https://statproofbook.github.io/P/logsum-ineq
https://statproofbook.github.io/P/kl-conv
https://statproofbook.github.io/P/ent-conc.html
https://en.wikipedia.org/wiki/Fano's_inequality
https://www.cs.cmu.edu/~aarti/Class/10704_Fall16/lec9.pdf
https://www.cs.purdue.edu/homes/spa/courses/msri04/data_comp_part1.pdf
https://mortada.net/simple-proof-for-krafts-inequality.html
https://www.techtarget.com/whatis/definition/Hamming-code?Offer=abt_pubpro_AI-Insider