Informations

DS-GA 1014: Optimization and Computational Linear Algebra for Data Science
NYU Center for Data Science, Fall 2021
Instructor: Marylou Gabriémg6658
Lectures: Thu. 4:55pm-6:35pm, 12 Waverly Place, room G08.
Marylou's Office Hours: Mondays 3-4pm (Zoom link in Brightspace)
Graders: Maitri Shah mns9841, Melody Yin yy1932, Di He dh3171, Jack Zhu jackzhu
Graders Office Hour: Tuesdays 2-3 pm (Zoom, link in Brightspace)
Section leaders
Zahra Kadkhodaie zk388 (Sec. 2) Tues 4-4:50pm - 60FA_150
Office Hour: Tuesday 5-6 pm - 60 FA, room 660
Colin Wan colin.wan (Sec. 3) Tues 8-8:50 am 60FA_150
Office Hour: Tuesday 9-10 am - 60 FA, room 660
Ying Wang yw3076 (Sec. 4) Tues 9-9:50am - GCASL_279
Office Hour: Tuesday 10-11 am - 60 FA, room 660

Links

  • NYU brightspace: links to Zoom meetings and codes for Piazza and Gradescope.
  • Piazza: online forum used for asking/answering questions, make announcements...
  • Gradescope: this is where you have to upload your homeworks.

Contents

This course covers the basics of optimization and computational linear algebra used in Data Science. About 66% of the lectures will be about linear algebra and ~33% about convex optimization. The first 5 lectures will cover basic linear algebra. Then we will study applications: Markov chains and PageRank, PCA and dimensionality reduction, spectral clustering, linear regression. Lastly, we will go over convex functions, optimality conditions and gradient descent. See the syllabus pdf.
Important: This course is "proof-based", meaning that we will prove theorems and that you will have to prove things in homeworks/exams.
This course was previously taught by Afonso Bandeira and then by Léo Miolane. The material of this page was inspired by their Fall 2016, Fall 2018 and Fall 2019/2020 lectures.

Questions and feedback

Feel free to ask me any questions you may have, in class, during office hours or by email.

Feedback: If you have any feedback on the class (it's going too fast, too slow...) please let me know (in person or through email) or submit an anonymous comment to this Google form.

Grading

There will be weekly homeworks, a midterm and a final exam. Checkout the syllabus for more details. You will find exams from past years in the Archive section.
Grade = 40% Homework + 25% Midterm + 35% Final. The exams are open book/notes.

Books (optional, some references are also given in the notes)

We will not follow any book. If you are looking for practice exercises, I would recommend to look at the course's archive. However, if you are looking for further development about linear algebra and optimization, you can refer to the following books:
  • Strang: Introduction to Linear Algebra (there are very good lecture videos available on YouTube)
  • Boyd & Vandenberghe: Introduction to applied linear algebra (available online here)
  • Nocedal & Wright: Numerical Optimization (should be available online via NYU here)
  • Boyd & Vandenberghe: Convex Optimization (available online here)
There are also great videos from Stephen Boyd teaching Linear Algebra for linear dynamical systems.

Weekly Material

Slides for the lectures will be posted here.
  1. Vector spaces Logistics, Session 01, Lab 01
  2. Linear transformations Session 02, Lab 02
  3. Rank Session 03, Lab 03
  4. Norm & dot product Session 04, Illegal mistakes, Lab 04
  5. Matrices & orthogonality Session 05, Lab 05
  6. Eigenvalues, eigenvectors & Markov chains Session 06,Lab 06
  1. Spectral Theorem, PCA, Singular values decomposition Session 07, Lab 07
  2. Graphs and linear algebra (& SVD) Session 08, Lab 08
  3. Convex functionsSession 09, Lab 09
  4. Regression Session 10, notebook, Lab 10
  5. Optimality conditions Session 11, Lab 11,
  6. Gradient descent Session 12, Lab 12, Lab 13
  7. TBA

Homeworks

Homeworks are typically weekly and will be posted here.
You are encouraged to write your homeworks using Latex, the most popular way to produce nice and clean scientific documents. That is not difficult! If you are new to Latex I would recommend you:
  1. Sign up on Overleaf, an online Latex editor.
  2. Create a new document and paste inside the following template.

Notes (by Léo Miolane)

Warning: These notes are not meant to be proper lecture notes! They only gather the main concepts and results from the lecture, without any additional explanation, motivation, examples, figures... they may be preliminary versions.
  1. Vector spaces
  2. Linear transformations
  3. Rank
  4. Norm & dot product
  5. Matrices & orthogonality
  6. Eigenvalues, eigenvectors & Markov chains
  1. Singular value decomposition
  2. Graphs and linear algebra
  3. Convex functions
  4. Regression
  5. Optimality conditions
  6. Gradient descent
You can download the source files from the GitHub Repository, or a .zip containing all the notes.

Archive from past years

2018-2019

2019-2020