MS&E314 Optimization Algorithms in Market Design, Data Science and Machine Learning Winter 2023-2024

| Announcements (Check it!) | General Information | Course Outline | Handouts | Assignments |

• Class Times and Locations
  • Lecture Room: Building 160, Rm 315
    
  • Lecture Time: MoWe 11:30AM - 12:50PM
    
  • Tutorial/Problem Session (HW Session, Midterm Exam, Project Session/Presentation, Make-Up Lectures) Time: Fr 11:30AM - 12:50PM
    

• Prerequisites : Stanford University Math 113, 115 (or equivalent).

• Text: David Luenberger and Yinyu Ye, Linear and Nonlinear Programming, 5th Edition. Also click for Various Matlab Solvers, such as hsdLPsolver.m .

Course Staff

• Professor: Yinyu Ye
  • Office Hour Place: Huang 308
  • Office hours: by appointment or Wednesdays 1:00pm - 2:00pm (Check Canvas for any possible change)
  • Email: yinyu-ye@stanford.edu
  • Phone:
• Course Assistant: ?
  • Office: ?
  • Email: ?
  • Phone: ?
• Staff Support: ?
  • Office: ?
  • Email: ?
  • Phone: ?

  
  

  Grading and Exams

  • Homework Exercises: 5 homework assignments will not be graded but discussed in Problem Sessions.
  • Computational Project: A team and quarter-long project (up to 3 people) -- 100%

  
  

  About Optimization

  MS&E314 emphasizes on computational algorithms for (convex or nonconvex) Optimization/game, including classical duality and fix-point theories, KKT conditions, and recent progresses in Linear and Nonlinear Optimization/Game---one of the central mathematical decision models in Market Design, Data Science, Machine Learning, Reinforcement Learning, Business Analytics, and Operations Management . MS&E314 can be viewed as a continuation course of MS&E211 and MS&E310. Although there are some minor overlaps, MS&E314 covers more advanced application models, in-depth optimization algorithm analyses, and state of the art numerical solvers which were not able to be covered by MS&E211 and MS&E310. MS&E314 also extensively covers Nonlinear and Nonconvex Optimization/Game problems, such as zero-sum schochatic games, complementing to MS&E310 (Linear Optimization) and other "Convex Optimization" courses.

  The field of optimization is concerned with the study of maximization and minimization of mathematical functions. Very often the arguments of (i.e., variables in) these functions are subject to side conditions or constraints. By virtue of its great utility in such diverse areas as applied science, engineering, economics, finance, medicine, data analysis, machine learning and statistics, optimization holds an important place in both the practical world and the scientific world. Indeed, as far back as the Eighteenth Century, the famous Swiss mathematician and physicist Leonhard Euler (1707-1783) proclaimed that ... nothing at all takes place in the Universe in which some rule of maximum or minimum does not appear. The subject is so pervasive that we even find some optimization terms in our everyday language.

  Optimization often goes by the name Mathematical Programming (MP). The latter name tends to be used in conjunction with finite-dimensional optimization problems, which in fact are what we shall be studying here. The word "Programming" should not be confused with computer programming which in fact it antedates. As originally used, the term refers to the timing and magnitude of actions to be carried out so as to achieve a goal in the best possible way. Application highlights of this year are: Convex and Nonconvex Auction Market, Online Pricing and Resource Allocation, Markov Decision Process and Reinforcement Learning, Data Classification via Wasserstein Barycenter, Sensor-Network Localization, Neural-Learning Verification, Distributionally Robust Decisioning and Learning, Economic/Game Equilibrium, Financial Techniques and Risk Management, Sparse and Low Rank Regression, Conic Linear Optimization,...,
  
  Theory Reviews -- Alternative-Theorems, Optimality Conditions, Conic and Lagrangian Duality, Complexity Analyses;
  
  Algorithms/Methods -- Steepest Descent, Multiplicative-Descent, Accelerated Descent, Block-Coordinate Descent, Stochastic Gradient Descent, Primal-Dual aternating gradient, Newton Descent, Path-Following Descent, Descent-First and Feasible-Second, Interior-Point Methods, Lagrangian Relaxations, ADMM methods, Optimization with random samplings and column generation, Trust-Region and Dimension-Reduced Trust-Region, Homogenized Second-Order, and other fast/heuristic Algorithms for NONCONVEX optimization with certain provable guarantee, which you would learn during the process of the course.

  
  

  Course Requirements

  What background is needed for MS&E314? This is a graduate-level Core course in the ICME and MS&E Department.  No prior optimization background is required (although it should do no harm to have some).  In this sense, it is an introductory course, but it is not intended to be an elementary course. Students who have taken courses such as MS&E 211 (or MS&E 310) will see some repetition of material. This is unavoidable, but MS&E 314 are intended to be more theoretical than MS&E 211.

  Students in this course will be expected to possess a firm background in the following mathematical subjects: multivariate differential calculus; fundamental concepts of analysis; linear algebra and matrix theory. Familiarity with computers and computer programming are also be useful, since various algorithm implementation projects can be substituted for the final exam. Above all, it is essential to have a tolerance for mathematical discourse plus an ability to follow - and devise one's own - mathematical proofs. .

  Question, Comments and Suggestions: If you have any comments or suggestions on lectures, recitation sessi$sessions or any other issures, you can send us messages via Piazza.