- When: 3pm - 3:50pm, Monday and Wednesday
- Where: Steward 208 (or N305 if needed)
This is a graduate level elective course aiming at providing the interface between astronomical data analysis problems and modern statistics methods. Modern astronomy and astrophysics is undergoing a revolution with dramatic increases in both the volume and complexity of astronomical data. The last decade saw the emergence of many terabyte-level sky surveys across the electromagnetic spectrum; the next decade, data volumes will enter the petabyte regime, with an ever strong time domain component. These new data sets represent quantum leaps in our abilities for new astronomical discoveries, but also present significant challenges to standard analysis tools normally employed in astronomy.
The goal of this course is to bridge the gap between modern large data surveys and the data analysis tools that have been provided in normal graduate courses. The course will start with a brief review of the modern statistics framework relevant to large scale data analysis, including probabilities and statistical distribution, classical and Bayesian statistical inferences. Then it will cover the main topics of the course: data mining and machine learning, including density estimation, clustering analysis, dimensionality reduction, regression and model fitting, classification and time series analysis. Another key component of the course is to introduce commonly used data mining and machine learning tools, in the context of Python-based packages, which will be used in solving data problems throughout the course.
- 0115 - Introduction
- 0122 - Tools
- 0127 - Statistics refresher
- 0129 - No class
- 0203 - Palmer: maximum likelihood estimate (4.2 - 4.5)
- 0205 - Zeljko Ivezic: LSST overview (guest lecture)
- 0210 - Chen: non-parametric modeling (4.8 - 4.9)
- 0212 - RS and Lo: Bayesian parameter estimation (5.3, 5.6)
- 0217 - Tang: Bayesian model selection (5.4, 5.7)
- 0219 - Peter Behroozi: Big Data and the Universe Machine (guest lecture)
- 0224 - Xu: MCMC (5.8)
- 0226 - Pearce and Rodozenski: PCA (7.1 - 7.3)
- 0302 - CK Chan: Big Data Challenges in EHT (guest lecture)
- 0304 - No class
- 0316 - Stephanie Juneau and Robert Nikutta: Science Platforms and Data Lab (guest lecture)
- 0318 - Stephanie Juneau and Robert Nikutta: Science Platforms and Data Lab (guest lecture)
- 0323 - Scott: Dimensionality: Manifold learning and ICA (7.5, 7.6)
- 0325 - Woodrum: Regression: linear models (8.1 - 8.5)
- 0330 - White: Regression: nonlinear models (8.7 - 8.10)
- 0401 - Tom Matheson: ANTARES (guest lecture)
- 0406 - Chamberlain: Classification: Generative (9.3)
- 0408 - Fan and Hayati: Classification: SVM (9.5, 9.6)
- 0413 - Liang: Classification: trees and forest (9.7)
- 0415 - Purdy: Deep learning and neural networks (9.8)
- 0420 - Jones: time series: basic models (10.1, 10.2)
- 0422 - Wolfe: time series: periodic (10.3)
- 0427 - Zhang: time series: localized and stochastic (10.4, 10.5)
- 0429 - Ann Zabludoff: Frontier Science with LSST (guest lecture)
- 0504 - project reports
- 0506 - project reports
The course will be given in a combination of instructor lectures, student-led seminars and guest lectures. After the first section of introductory material, students will lead discussion and demonstration of most of the topics, and guest lectures will introduce important current and future key big data projects in astronomy. The class will conclude with final projects on using data mining and machine learning tools of your own research data.