Welcome to CS 159!

The goal of the class is to bring students up to speed in two topics in modern machine learning research through a series of lectures. Students will then go on to conduct a mini research project at the end of the class. The two topics are:

  1. Predictive control & model-based reinforcement learning;
  2. Neural network theory: learning & generalisation.

Caltech listing: CS/CNS/EE/IDS 159 (3-0-6) TTh 2:30-4:00.
Office hours: MW 3:00-4:00.

Instructors

Teaching assistants

Structure

  • Weeks 1–3: lectures on reinforcement learning & control;
  • Weeks 4–6: lectures on neural network theory;
  • Weeks 7–10: research project and special guest lectures.

Discord

Office hours will be held on the class Discord server. A signup link will be sent to students during the first week of class. Students are encouraged to use the Discord freely at other times to discuss homeworks, projects and any other class-related matters. If you do not have access to the Discord by the end of the first week of class, email an instructor.

It goes without saying, but please remember to treat other students with respect and don’t say anything you wouldn’t in a Caltech classroom.

Grading

The final grade will be based 20% on homeworks and 80% on the final project.

Homeworks

Students should complete homeworks in groups of two to three—submitting one solution document per group. All students in a group should understand all parts of the completed homework. If you don’t have a group, use the #matchmaking channel on Discord to find one.

Completed homeworks should be submitted on Gradescope. If you do not have access to the Gradescope by the end of the first week of class, email an instructor.

Homework schedule

Homeworks will be set on Thursday and due the Thursday after.

# Date set Date due Resources
1 4/01 4/08 hw1.zip
2 4/08 4/15 hw2.zip
3 4/22 4/29 hw3.zip
4 4/29 5/06 hw4.pdf

Late policy

A group has a total of 48 late hours for the term. This means that if the first homework is submitted 47 hours late and the second homework is submitted two hours late, the group would incur no penalty on the first homework but score zero on the second.

Final project

The final project will be conducted during the last four weeks of class. Students should work in teams of two to three. Students may either pick one of the special topics to build their project on, or alternatively they may attempt to mix the two topics. The poster session is on 6/03 and project reports are due 6/04.

Lecture schedule

# Date Subject Resources
0 3/30 Introduction pdf / vid
    Topic 1—RL & Control  
1 3/30 Discrete MDPs pdf / vid
2 4/01 Optimal Control pdf / vid
3 4/06 Model Predictive Control pdf / vid
4 4/08 Learning MPC pdf / vid / supp
5 4/13 Model Learning in MPC pdf / vid
6 4/15 Planning Under Uncertainty and Project Ideas pdf / vid
    Topic 2—Neural Network Theory  
7 4/20 Neural Architecture Design pdf / vid
8 4/22 Network Function Spaces pdf / vid / ipynb
9 4/27 Network Optimisation pdf / vid
10 4/29 Statistical Learning Theory pdf / vid
11 5/04 PAC-Bayesian Theory pdf / vid
12 5/06 Project Ideas pdf / vid
    Final Project & Guest Lectures  
13 5/11 Joe Marino pdf / vid
14 5/13 Yasaman Bahri pdf / vid
15 5/18 —  
16 5/20 Guanya Shi pdf / vid
17 5/25 Roberto Calandra pdf / vid
18 5/27 Guillermo Valle-Pérez & Ard Louis pdf / vid
19 6/01 SueYeon Chung pdf / vid
20 6/03 Poster session  

Reading material

Additional references are listed on the control and learning subpages.

Acknowledgements