MS&E338 Aligning SuperintelligenceWithin a couple of decades, or less, it is plausible that humans will create an AI that is much smarter than humans in practically all domains of human activity. We refer to such an AI as a superintelligence. The alignment problem is how to make sure that such a superintelligence acts according to its human operator's intent. This course is intended for a technical audience interested in thinking about this problem. But why would an AI not act according to its human operator’s intent? And if the AI were to misbehave, wouldn't the operator just modify it or shut it down? Furthermore, even if we accept that the AI will not always behave as intended, why should this be considered a major source of risk, let alone, a catastrophic risk? Why are some people saying that these risks should be a global priority on par with pandemics and nuclear war while others are saying that these concerns are overhyped? In this course, we will discuss:
Please note: Enrolled students will be invited to a Google doc that includes more detailed course information, including schedule and assignments. If you are auditing the course, please reach out and we will invite you. PrerequisitesTo have the background to participate in this, each student is recommended to have taken
Students will be required to write opinion, review, or research papers on aspects of the alignment problem.
What this course is not about
Logistics
Course Notes
Some of the articles we will draw from:
|