Instructor: Dr. Brian Spiering brian.spiering@galvanize.com Office Hours: Wednesday 11a-12n & By Appointment
Assistant Faculty: Edward Banner edward.banner@galvanize.com
Office Hours: By Appointment
Class Location: 44 Tehama St, 309 311 classroom, San Francisco, CA
Lecture Times/Days: 11a-1p & 3-5p, Monday & Tuesday
Lab Times/Days: 1p-2p & 5-6p, Monday & Tuesday
Communication: #gu4_sf_dl
This course provides a broad overview of Deep Learning, aka artificial neural networks. We will cover the mathematics and programming required to understand the fundamentals of Deep Learning and design contemporary architectures. Students will apply these techniques to real-world problems, for example image classification.
- Apply Deep Learning to solve real world problems
- Build a 3-layer artificial neural network from scratch
- Explain and implement backpropagation algorithm
- Build the following architectures:
- Convolutional Neural Networks (CNN)
- Recurrent Neural Network (RNN) and Long Short Term Memory (LSTM)
- Generative Adversarial Nets (GANs)
- Reinforcement Learning (RL)
- All other kinds of machine learning (We are only covering Deep Learning)
- All other Deep Learning frameworks (We are only covering Keras and TensorFlow)
- Theano, Caffe, CNTK, DSSTNE, PaddlePaddle, …
- OpenAI Gym/Universe & DeepMind Lab
- High Performance Computing (HPC)
- Distributed systems
- ALL other hardware implementations (We are only covering CPU and GPUs)
- ASIC
- Mobile
- Deep Learning research (We are only covering applied Deep Learning)
- Memory Networks
- Neural Turing Machines
- Energy–Based Models
Successful completion of:
- DSCI-6001: Mathematics for Data Scientists
- DSCI-6002: Data Exploration, Feature Engineering, and Statistics for Data Scientists
- DSCI-6003: Machine Learning
- Fundamentals of Deep Learning
- Deep Learning
This course is an "active" learning environment. You'll learn through doing. The focus will be on explaining concepts in your words and applying concepts through programming.
Before class you will complete preparation materials (e.g., watch videos and read book chapters / blogs). All preparation materials should be covered prior to the start of each class session. They are always required unless explicitly labeled as optional. These materials will be the source of factual knowledge. You are expected to be familiar with the basic concepts and technical jargon before the start of class.
In-class time is precious - We'll reserve it for discussion, presenting complex material, answering questions, and working on exercises.
Typical class structure:
- On Your Own (OYO) activity
- RAT (Readiness Assessment Test)
- Lecture
- Lab
OYO activity is a creative activity to help you to integrate and apply the the preparation materials.
The Readiness Assessment Tests (RATs) are intended to test your understanding of the materials presented thus far in the course. This includes material from previous classes. There are 3 parts: individual, small-group, and class.
- Each student will answer all the questions on the RAT individually.
- Then the class will split into teams of 3-4. Each team will answer the same questions again, the goal is to reach consensus. This is an opportunity for peer-to-peer instruction which is often more effective than lectures!
- Finally, the answers to the questions will be gone over by the class, hopefully resolving any misunderstandings before proceeding.
The RATs are meant to assess the first three levels of Bloom's Taxonomy: knowledge, comprehension, and analysis. The lab work is meant to develop the latter three levels: analysis, synthesis, and evaluation. Students will separate into two (or threes) for "pair programming". These exercises may involve a series of short questions, single day projects, or multi-day projects.
| Item | Weight |
|---|---|
| Mastery Tracker | 10% |
| Participation | 20% |
| Lab | 30% |
| Final Project | 40% |
The expected grade is B+. Getting an A- or above requires completion of the Mastery Tracker, high level participation, and a stellar final project.
Mastery Tracking is a tool to provide feedback student learning. Standards are the core-competencies of MSDS graduates - the knowledge, skills, and habits every student should possess by time they graduate. Standards are measurable, student-focused outcomes that state what students are expected to be able to do by the end of the course. Students who are below ‘mastery’ on a standard are expected to continue practicing said standard (with the instructor's guidance) until they reach mastery. What matters is that students eventually learn the material, not how many attempts it takes to get there. The Instructor and Data Scientist in Residence are available to offer feedback and help guide everyone on their mastery journey.
Mastery Tracking uses a 4-point scale. Every student is expected to achieve 3 or above (Mastery) across all Standards by the end of the course. 1s and 2s indicate areas where students need further practice and/or interventions to reach mastery.
4 pt Scale:
0 = Has not been covered
1 = Falling far below mastery - Meeting none of the success criteria or has egregious errors
2 = Approaching mastery - Meeting some of the success criteria
3 = Mastery - Meeting all of the success criteria
4 = Exceeding mastery - Truly exceeding expectations and demonstrating proficiency at a higher level of rigor
We will be using Galvanize's Learning Management System (LMS) which can be found at learn.galvanize.com.
You must also show up prepared. Each person is important to the dynamic of the class, and therefore students are required to participate in class activities. Expect to be "cold called". I call on students at random, not to put you on the spot but, to keep you engaged in the material at all times.
Attendance is mandatory. It is the responsibility of each student to attend all classes. If you have to miss class, due to any circumstances, please notify Brian by Slack ASAP. Supporting documents (e.g., doctor’s notes) should accompany absences due to sickness. Each excused absences beyond 2 or each unexcused absences will result in lowering your overall course grade by ⅓ of a letter grade (A->A-, A->B+, …). It is at the instructor’s discretion to deny any absences or to allow students to make-up course work resulting from any absences.
Details will be covered in a future class (mostly likely tomorrow).
- Neural Network Fundamentals
- Welcome
- Fundamentals
- Backpropagation I
- Backpropagation II
- Neural Network Training
- Keras
- Training Neural Networks I
- Images and Convolution
- Convolutional Neural Networks (CNNs)
- Image Classification
- Image Classification I
- Optimization I
- Image Classification II
- Review / Project Check-in
- Sequence Learning (aka, Natural Language Processing)
- Recurrent Neural Networks (RNNs)
- Long Short Term Memory (LSTM) Networks
- Sequence-to-Sequence Learning
- Natural Language Processing (NLP)
- Reinforcement Learning (RL)
- TensorFlow I
- Reinforcement Learning (RL) I
- Reinforcement Learning (RL) II
- Training Neural Networks II
- Generative Adversarial Nets (GANs)
- Optimization II
- Generative Adversarial Nets (GANs) I
- Generative Adversarial Nets (GANs) II
- Review / Project Check-in
- Unsupervised learning
- TensorFlow II
- Unsupervised Learning I
- Attention-Based Models
- Unsupervised Learning II
- Final Project
- Review
- Final Project Worksession
- Presentations
- Presentations