JayAhn0104 / Reading-List

A Curated List of Learning Materials on Machine Learning

Reading List

Text books

Machine Learning

• Lectures on Machine Learning by Hyeong In Choi, Seoul National University, Department of Mathematics

Deep Learning

• Dive into Deep Learning by Zhang, Aston, et al.
• Deep learning by Goodfellow, Ian, Yoshua Bengio, and Aaron Courville

Recommender System

• Recommender-Systems: The Textbook by Charu C. Aggarwal

Outlier Detection

• Outlier Analysis by Charu C. Aggarwal

Papers

Recommender System

• Survey
  • Bobadilla, J., Ortega, F., Hernando, A., & Gutiérrez, A. (2013). Recommender systems survey). Knowledge-based systems, 46, 109-132.
• Collaborative Filtering
  • Matrix Factorization
    • Survey
      • Koren, Y., Bell, R., & Volinsky, C. (2009). Matrix factorization techniques for recommender systems. Computer, 42(8), 30-37.
    • Lecture Note
      • CS168: The Modern Algorithmic Toolbox Lecture #9: The Singular Value Decomposition (SVD) and Low-Rank Matrix Approximations
      • CME 323: Distributed Algorithms and Optimization, Spring 2015 Matrix Completion via Alternating Least Square(ALS)
    • Methods
      • Alternative Least Square (ALS)
        • Bell, R. M., & Koren, Y. (2007, October). Scalable collaborative filtering with jointly derived neighborhood interpolation weights. In Seventh IEEE international conference on data mining (ICDM 2007) (pp. 43-52). IEEE.
      • Non-negative Matrix Factorization
        • Gillis, N. (2017). Introduction to nonnegative matrix factorization. arXiv preprint arXiv:1703.00663.
      • Probabilistic Matrix Factorization
        • Mnih, A., & Salakhutdinov, R. R. (2007). Probabilistic matrix factorization. Advances in neural information processing systems, 20.
      • Temporal Dynamics
        • Koren, Y. (2009, June). Collaborative filtering with temporal dynamics. In Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining (pp. 447-456).
      • Neural Collaborative Filtering
        • He, X., Liao, L., Zhang, H., Nie, L., Hu, X., & Chua, T. S. (2017, April). Neural collaborative filtering. In Proceedings of the 26th international conference on world wide web (pp. 173-182).
      • Factorization Machines (FM)
        • Rendle, S. (2010, December). Factorization machines. In 2010 IEEE International conference on data mining (pp. 995-1000). IEEE.
      • Deep Factorization Machines (DeepFM)
        • Guo, H., Tang, R., Ye, Y., Li, Z., & He, X. (2017). DeepFM: a factorization-machine based neural network for CTR prediction. arXiv preprint arXiv:1703.04247.
• Evaluation
  • Ge, M., Delgado-Battenfeld, C., & Jannach, D. (2010, September). Beyond accuracy: evaluating recommender systems by coverage and serendipity. In Proceedings of the fourth ACM conference on Recommender systems (pp. 257-260).
  • Kotkov, D., Wang, S., & Veijalainen, J. (2016). A survey of serendipity in recommender systems. Knowledge-Based Systems, 111, 180-192.

Educational Data Mining

• Survey
  • Baker, R. S., & Inventado, P. S. (2014). Educational data mining and learning analytics. In Learning analytics (pp. 61-75). Springer, New York, NY.
• Knowledge Tracing (answer correctness prediction)
  • Survey
    • Pelánek, R. (2017). Bayesian knowledge tracing, logistic models, and beyond: an overview of learner modeling techniques. User Modeling and User-Adapted Interaction, 27(3), 313-350.
    • Gervet, T., Koedinger, K., Schneider, J., & Mitchell, T. (2020). When is deep learning the best approach to knowledge tracing?. Journal of Educational Data Mining, 12(3), 31-54.
  • Bayesian Knowledge Tracing (BKT)
    • Lecture Notes
      • CS229: Hiden Markov Models Fundamentals by Daniel Ramage
      • Hiden Markov Models by Fran ̧cois Caron
    • Methods
      • Yudelson, M. V., Koedinger, K. R., & Gordon, G. J. (2013, July). Individualized bayesian knowledge tracing models. In International conference on artificial intelligence in education (pp. 171-180). Springer, Berlin, Heidelberg.
    • Review
      • van De Sande, B. (2013). Properties of the Bayesian Knowledge Tracing Model. Journal of Educational Data Mining, 5(2), 1-10.
    • Implementation
      • pyBKT
        • Badrinath, A., Wang, F., & Pardos, Z. (2021). pyBKT: an accessible python library of Bayesian knowledge tracing models. arXiv preprint arXiv:2105.00385.
  • Deep Knowledge Tracing (DKT)
    • Methods
      • DKT
        • Piech, C., Bassen, J., Huang, J., Ganguli, S., Sahami, M., Guibas, L. J., & Sohl-Dickstein, J. (2015). Deep knowledge tracing. Advances in neural information processing systems, 28.
      • DKT+
        • Yeung, C. K., & Yeung, D. Y. (2018, June). Addressing two problems in deep knowledge tracing via prediction-consistent regularization. In Proceedings of the Fifth Annual ACM Conference on Learning at Scale (pp. 1-10).
      • PEBG
        • Liu, Y., Yang, Y., Chen, X., Shen, J., Zhang, H., & Yu, Y. (2020). Improving knowledge tracing via pre-training question embeddings. arXiv preprint arXiv:2012.05031.
      • DKVMN
        • Zhang, J., Shi, X., King, I., & Yeung, D. Y. (2017, April). Dynamic key-value memory networks for knowledge tracing. In Proceedings of the 26th international conference on World Wide Web (pp. 765-774).
      • Deep-IRT
        • Yeung, C. K. (2019). Deep-IRT: Make deep learning based knowledge tracing explainable using item response theory. arXiv preprint arXiv:1904.11738. Chicago
      • SAKT
        • Pandey, S., & Karypis, G. (2019). A self-attentive model for knowledge tracing. arXiv preprint arXiv:1907.06837.
    • Review
      • Khajah, M., Lindsey, R. V., & Mozer, M. C. (2016). How deep is knowledge tracing?. arXiv preprint arXiv:1604.02416.
      • Xiong, X., Zhao, S., Van Inwegen, E. G., & Beck, J. E. (2016). Going deeper with deep knowledge tracing. International Educational Data Mining Society.
  • Logistic Knowledge Tracing (LKT)
    • Methods
      • LFA (Learning Factor Analysis)
        • Cen, H., Koedinger, K., & Junker, B. (2006, June). Learning factors analysis–a general method for cognitive model evaluation and improvement. In International conference on intelligent tutoring systems (pp. 164-175). Springer, Berlin, Heidelberg.
      • PFA (Performance Factor Analysis)
        • Pavlik Jr, P. I., Cen, H., & Koedinger, K. R. (2009). Performance Factors Analysis--A New Alternative to Knowledge Tracing. Online Submission.
      • R-PFA
        • Galyardt, A., & Goldin, I. (2015). Move Your Lamp Post: Recent Data Reflects Learner Knowledge Better than Older Data. Journal of Educational Data Mining, 7(2), 83-108.
      • LKT (Logistic Knowledge Tracing)
        • Pavlik, P. I., Eglington, L. G., & Harrell-Williams, L. M. (2021). Logistic Knowledge Tracing: A Constrained Framework for Learner Modeling. IEEE Transactions on Learning Technologies, 14(5), 624-639.
      • KTM (Knowledge Tracing Machine)
        • Vie, J. J., & Kashima, H. (2019, July). Knowledge tracing machines: Factorization machines for knowledge tracing. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 33, No. 01, pp. 750-757).
      • DAS3H
        • Choffin, B., Popineau, F., Bourda, Y., & Vie, J. J. (2019). DAS3H: modeling student learning and forgetting for optimally scheduling distributed practice of skills. arXiv preprint arXiv:1905.06873.
    • Review
      • Gong, Y., Beck, J. E., & Heffernan, N. T. (2011). How to construct more accurate student models: Comparing and optimizing knowledge tracing and performance factor analysis. International Journal of Artificial Intelligence in Education, 21(1-2), 27-46.
      • Rosé, C. P., McLaughlin, E. A., Liu, R., & Koedinger, K. R. (2019). Explanatory learner models: Why machine learning (alone) is not the answer. British Journal of Educational Technology, 50(6), 2943-2958.
      • Wu, M., Davis, R. L., Domingue, B. W., Piech, C., & Goodman, N. (2020). Variational item response theory: Fast, accurate, and expressive. arXiv preprint arXiv:2002.00276.
• Exercise Recommendation
  • Survey
    • Nabizadeh, A. H., Leal, J. P., Rafsanjani, H. N., & Shah, R. R. (2020). Learning path personalization and recommendation methods: A survey of the state-of-the-art. Expert Systems with Applications, 159, 113596.
  • Methods
    • Wu, Z., Li, M., Tang, Y., & Liang, Q. (2020). Exercise recommendation based on knowledge concept prediction. Knowledge-Based Systems, 210, 106481.
    • Ai, F., Chen, Y., Guo, Y., Zhao, Y., Wang, Z., Fu, G., & Wang, G. (2019). Concept-Aware Deep Knowledge Tracing and Exercise Recommendation in an Online Learning System. International Educational Data Mining Society.
    • Cai, D., Zhang, Y., & Dai, B. (2019, December). Learning path recommendation based on knowledge tracing model and reinforcement learning. In 2019 IEEE 5th International Conference on Computer and Communications (ICCC) (pp. 1881-1885). IEEE.
  • Lecture Notes
    • Combinatorial Optimization
      • Simulated Annealing Overview by Zak Varty

Outlier Detection

• Survey
  • Chandola, V., Banerjee, A., & Kumar, V. (2009). Anomaly detection: A survey. ACM computing surveys (CSUR), 41(3), 1-58.
• Unsupervised Learning Methods
  • Methods
    • LOF
      • Breunig, M. M., Kriegel, H. P., Ng, R. T., & Sander, J. (2000, May). LOF: identifying density-based local outliers. In Proceedings of the 2000 ACM SIGMOD international conference on Management of data (pp. 93-104).
    • Isolation Forest
      • Liu, F. T., Ting, K. M., & Zhou, Z. H. (2008, December). Isolation forest. In 2008 eighth ieee international conference on data mining (pp. 413-422). IEEE.
  • Use of Outlier Scores
    • Gao, J., & Tan, P. N. (2006, December). Converting output scores from outlier detection algorithms into probability estimates. In Sixth International Conference on Data Mining (ICDM'06) (pp. 212-221). IEEE.
    • Schubert, E., Wojdanowski, R., Zimek, A., & Kriegel, H. P. (2012, April). On evaluation of outlier rankings and outlier scores. In Proceedings of the 2012 SIAM International Conference on Data Mining (pp. 1047-1058). Society for Industrial and Applied Mathematics.
    • Kriegel, H. P., Kroger, P., Schubert, E., & Zimek, A. (2011, April). Interpreting and unifying outlier scores. In Proceedings of the 2011 SIAM International Conference on Data Mining (pp. 13-24). Society for Industrial and Applied Mathematics.
  • For High-Dimensional Data
    • Survey
      • Zimek, A., Schubert, E., & Kriegel, H. P. (2012). A survey on unsupervised outlier detection in high‐dimensional numerical data. Statistical Analysis and Data Mining: The ASA Data Science Journal, 5(5), 363-387.
      • Campos, G. O., Zimek, A., Sander, J., Campello, R. J., Micenková, B., Schubert, E., ... & Houle, M. E. (2016). On the evaluation of unsupervised outlier detection: measures, datasets, and an empirical study. Data mining and knowledge discovery, 30(4), 891-927. Chicago
      • Xu, X., Liu, H., Li, L., & Yao, M. (2018). A comparison of outlier detection techniques for high-dimensional data. International Journal of Computational Intelligence Systems, 11(1), 652.
    • Concentration Effect (Motivation for HD Outlier detection)
      • Beyer, K., Goldstein, J., Ramakrishnan, R., & Shaft, U. (1999, January). When is “nearest neighbor” meaningful?. In International conference on database theory (pp. 217-235). Springer, Berlin, Heidelberg.
      • Durrant, R. J., & Kabán, A. (2009). When is ‘nearest neighbour’meaningful: A converse theorem and implications. Journal of Complexity, 25(4), 385-397.

etc

Hyper-parameter Tuning

• Survey
  • Yu, T., & Zhu, H. (2020). Hyper-parameter optimization: A review of algorithms and applications. arXiv preprint arXiv:2003.05689.

Model Combination

• Motivation
  • Can multi-model combination really enhance the prediction skill of probabilistic ensemble forecasts?
• Review
  • Model Combination in multiclass classification
• Methods
  • Stacked Generalization
    • Issues in Stacked Generalization

Ordinal Classification

• Methods
  • Frank, E., & Hall, M. (2001, September). A simple approach to ordinal classification. In European conference on machine learning (pp. 145-156). Springer, Berlin, Heidelberg.

Websites

• Distill

Soft Readings (in Korean)

• Kakao AI Report
• 세상에서 가장 쉬운 베이즈 통계학
• 세상에서 가장 쉬운 통계학 입문
• 헬로 데이터 과학