Computer vision can be a powerful tool for HCI applications. Camera-based input has been used effectively on mobile devices, in tangible user interfaces, and in a variety of ubiquitous computing applications. The computer vision literature contains a wide variety of sensing and processing techniques that are applicable to HCI, but in general these techniques have not been widely adopted by HCI practitioners.

“Designing Applications that See” introduces students to computer vision and image processing from an interaction design standpoint. In readings, lectures, discussions, and project work, the course focuses on applying computer vision techniques in the design of complex interactive systems. In addition to covering the history and basic principles of computer vision, this course will examine some ways in which it has been applied to interaction design, and the unique set of challenges and opportunities that this presents.

Rather than focus on the gory mathematical details of computer vision algorithms, we consider how these algorithms can be used effectively in real systems. Students will be shown how to apply existing computer vision tools, techniques, and libraries, without needing any in-depth experience in mathematics or programming.

Applying computer vision to HCI has different requirements and goals than the traditional applications of computer vision: results must be available quickly, but the detail of information required is often much more limited (e.g. high-level classification of events rather than detailed 3D reconstruction of a scene). In addition, since a human is in the loop, feedback can be used to allow iteration and adaptation of software and user behavior.

The primary objectives of CS377S are:

1. To survey the history of computer vision and summarize its main concepts, goals, and techniques.
2. To provide an overview of past HCI research as it relates to computer vision, including the design tools and applications that have been developed.
3. To teach students how to apply computer vision techniques to interface design in a real-world context.

Course Website: http://cs377s.stanford.edu/

Course Newsgroup: su.class.cs377s

Please use the newsgroup for discussion and general questions about assignments.

Course Staff Mailing List: cs377s-win0708-staff at lists dot stanford dot edu

Lectures are held on Tuesdays and Thursdays from 11:00AM-12:30PM in Wallenberg 120.

The “Nuts & Bolts” workshop sessions are interactive tutorials. If you have a laptop computer, please bring it to these sessions.

This is a preliminary version of the course calendar. Due dates and lecture content are subject to change.

Date	Topic	Slides	Coursework Due	Readings
Jan 8	Course Overview; Introduction to Computer Vision			Piccardi, Levin
Jan 10	Human Vision and Perception		Course Sign-up	Sacks85, Poggio, Sacks94
Jan 15	Image Processing: Features, Filters, Color, Shape			Russ
Jan 17	Nuts & Bolts I: Matlab, Camera Capture			Matlab
Jan 22	Motion and Tracking: Optical Flow, Temporal Filtering		Assignment #1	Moeslund
Jan 24	Nuts & Bolts II: Processing, JMyron			Nimoy
Jan 29	Object Detection, Machine Learning			Liter, Lowe, Viola
Jan 31	Nuts & Bolts III: OpenCV			Agam
Feb 5	Computer Vision in HCI: Applications and Examples		Assignment #2	Freeman, Maes, Smith
Feb 7	Nuts & Bolts IV: Eyepatch		Interim Course Evaluation	Klemmer00, Wilson04
Feb 12	Computer Vision in HCI: Design Tools and Design Challenges			Klemmer04, Fails02
Feb 14	Guest Lecture: Johnny Chung Lee on Interaction Techniques Using the Wii Remote, Project Brainstorming		Team Registration	Olsen, Mankoff
Feb 19	Project Proposal Presentations
Feb 21	Individual Team Meetings		Project Proposal
Feb 26	Guest Lecture: David Stavens on Stanley, Junior, and the DARPA Challenges		Assignment #3
Feb 28	Individual Team Meetings		Project Checkpoint #1
Mar 4	Guest Lecture: Alex Holub on Computer Vision Research at Ooyala
Mar 6	Individual Team Meetings		Project Checkpoint #2
Mar 11	Guest Lecture: Manu Kumar on Eye Tracking and Gaze-enhanced User Interfaces
Mar 13	Individual Team Meetings		Project Checkpoint #3
Mar 17	Final Project Presentations 7:00PM-10:00PM in Wallenberg 124		Final Report and Team Evaluation

CS377S has three types of coursework requirements.

Readings: Readings are assigned to correspond to each lecture topic. Students should complete the readings before lecture and be prepared to participate in class discussion of the readings.

Individual Assignments: Three individual “problem sets” are assigned during the first half of the quarter. Each of these assignments has a written portion and an implementation portion. The written portions involve reflection, problem solving, and idea generation based on the readings. The implementation portions are mini-projects that allow students to become familiar with various computer vision tools and techniques.

Project Assignments: During the second half of the quarter, students will form project teams and build a working prototype of a computer vision-based interactive system. This five-week design project includes four milestones: an initial written project proposal, two intermediate checkpoints, and a final presentation and written report.

For more details on these coursework requirements, refer to the individual assignment descriptions on the assignments page.

This page last updated March 11, 2008.