EE364b - Convex Optimization II

Instructor: Mert Pilanci, pilanci@stanford.edu

EE364b is the same as CME364b and was originally developed by Stephen Boyd

Announcements

  • Homework 8 is released and due June 4. In this homework, you will explore relaxations of cardinality constraints, and implement sequential convex programming for non-convex optimization.

  • Homework 7 is released and due May 28. This homework is about interior point and truncated Newton methods.

  • Homework 6 is released and due May 21. This homework is about the conjugate gradient method and preconditioning using randomized Hadamard transforms.

  • Homework 5 is released and due May 14. In this homework, you will experiment with proximal methods including the proximal gradient method, its variants with acceleration, line-search and adaptive restarts, Alternating Direction Method of Multipliers and Dykstra’s alternating projections. You will also derive the resolvent operators of certain neural network activations.

  • Homework 4 is released and due May 7. This homework is about primal and dual decomposition, distributed ridge regression and cutting plane methods.

  • Homework 3 is released and due April 30. This homework will involve implementing constrained and primal-dual subgradient methods, mirror descent and experimenting with stochastic optimization in portfolio design.

  • Homework 2 is released and due April 23. In this homework, you will implement the subgradient method and its primal-dual and heavy-ball variants, experiment with alternating projections for signal reconstruction, examine the feasibility of line-search for nonsmooth functions and analyze the Clarke subdifferential of ReLU neural network objectives.

  • Homework 1 is released and due April 16. This homework is about subgradients, subdifferentials, parallel subgradient computation using Dask, how autodifferentiation might fail and how well ChatGPT knows about the rules of differentiation.

  • The first lecture will be on Monday April 3, 1:30pm-3:00pm at Shriram Center BioChemE 104

  • The lectures will be recorded and be available to enrolled students

  • Welcome to EE364b, Spring quarter 2022-2023!

Course description

Continuation of 364A. Subgradient, cutting-plane, and ellipsoid methods. Decentralized convex optimization via primal and dual decomposition. Monotone operators and proximal methods; alternating direction method of multipliers. Exploiting problem structure in implementation. Convex relaxations of hard problems. Global optimization via branch and bound. Robust and stochastic optimization. Convex formulations of neural networks and Monte Carlo sampling. Applications in areas such as control, circuit design, signal processing, machine learning and communications. This class will culminate in a final project.

Prerequisites:

EE364a - Convex Optimization I