CS 224U / LING 188 / LING 288 Natural Language Understanding Autumn 2007

ANNOUNCEMENTS
COURSE INFORMATION
Staff	Instructor: Dan Jurafsky, jurafsky @ stanford.edu Office Hours: TBD Instructor: Chris Manning,    manning @ cs.stanford.edu Office Hours: TBD 460-102 TA: Nate Chambers, cs224u-ta @ cs.stanford.edu Office Hours: Gates 228, Wed. 10-11:30am (and by appointment) All of us:    cs224u-aut0708-staff @ lists.stanford.edu
Time	TuTh 3:15-4:30pm
Location	ART 4 (click for map)
Catalog Description	Machine understanding of human language. Computational semantics (determination of sense, event structure, thematic role, time, aspect, synonymy/meronymy, causation, compositional semantics, treatment of scopal operators), and computational pragmatics and discourse (coherence relations, anaphora resolution, information packaging, generation). Theoretical issues, online resources, and relevance to applications including question answering, summarization, and textual inference. Prerequisites: one of LINGUIST 180, CS 224N,S; and knowledge of logic (LINGUIST 130A or B, CS157, or PHIL159)
Requirements	Read the papers before each class Do the 6 data homeworks. Each one of these will require a small amount of work to learn about a natural language understanding data source. You will be asked to answer small questions and make note of interesting points you notice about the data. Be creative! You can talk to friends about the data, but the actual data homeworks must be your own work. Do the 1 programming homework Write a literature review paper: a short 6-page single-spaced spaper summarizing and synthesizing 5 papers on the area of your final project. Do a final project: a research project due at the end of the quarter. Both the final project and the review paper may be done in groups. Paper formatting and etc details here Your grade is determined based on: Class participation: 10% Data HW assignments: 15% Programming HW assignments: 15% Literature review: 20% Final Project: 40% HW assignments are graded on a -/v/+ basis, where '-' is insufficient and '+' is exceptional. In accord with the usual practice at Stanford, the work of students registered for the undergrad version of the course (LING188) will be graded on an easier basis.
Errata	09/25: Homeworks require access to the linguistic data on AFS. Instructions to get access. 09/29: More specific homework policy is posted. 10/04: WSD Programming Assignment is posted. 10/15: Programming Submission Instructions are posted. 10/20: Lit Review Updated with a few more specifics. 10/24: All Results from the WSD project! 12/4: Final Project Paper Formatting Instructions are posted.

	SCHEDULE
Wk	Date	HW Due	Who	Topic and Readings
1	Sep 25		Dan	History Terry Winograd. 1972. Understanding Natural Language. Academic Press, New York. Sections 1-2. Schank and Abelson. 1977. Scripts, Plans, Goals, and Understanding: An Inquiry into Human Knowledge Structures. Hillsdale, NJ: L. Erlbaum, . Chapters 1-3. OPTIONAL ADVANCED READING: Robert F. Simmons. 1970. Natural language question-answering systems: 1969 Communications of the ACM 13:1, 15-30.
PART I: Lexical Semantics
1	Sep 27	Data HW 1	Dan Lec 2 (ppt) Lec 2 (6up pdf)	Lexical Relations: WordNet, Synonymy, Hyponymy Jurafsky and Martin 2nd edition Chapter 19, Lexical Semantics, page 1-10 Jurafsky and Martin 2nd edition Chapter 20, Computational Lexical Semantics, pages 16-31 Lin, Dekang. 1998. Automatic Retrieval and Clustering of Similar Words. COLING-ACL98, Montreal, Canada, August, 1998. Hearst, M. 1998. Automated Discovery of WordNet Relations. In WordNet: An Electronic Lexical Database, Christiane Fellbaum (ed.), MIT Press, 1998 Dependency parsing Marie-Catherine de Marneffe, Bill MacCartney, and Christopher D. Manning. 2006. Generating Typed Dependency Parses from Phrase Structure Parses. 5th International Conference on Language Resources and Evaluation (LREC 2006), pp. 449-454.
2	Oct 2		Chris Lec 3 (ppt)	Word Sense Disambiguation Jurafsky and Martin 2nd edition Chapter 20, Computational Lexical Semantics, pages 1-16 McCarthy, D., Koeling, R., Weeds, J. and Carroll, J. (2004) Finding predominant word senses in untagged text. In Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics. Barclona, Spain. pp 280-287
2	Oct 4	Data HW 2	Dan Lec 4 (ppt) Lec 4 (6up pdf)	Temporal Relations M. Moens and Mark Steedman, 1988. Temporal Ontology and Temporal Reference. Computational Linguistics, vol. 14(2), 1988. Lapata, Mirella and Alex Lascarides. 2004. Inferring Sentence-internal Temporal Relations. In Proceedings of the North American Chapter of the Assocation of Computational Linguistics, 153-160. Boston
PART II: Discourse/Pragmatics
3	Oct 9	Data HW 3	Dan Lec 5 (ppt) Lec 5 (6up pdf)	Coherence Jurafsky and Martin 2nd edition Chapter 21, Discourse, pages 1-15 Marcu and Echihabi. 2002. An Unsupervised Approach to Recognizing Discourse Relations. ACL 2002 Regina Barzilay and Mirella Lapata. 2005. Modeling Local Coherence: An Entity-based Approach ACL 2005, 141-148.
3	Oct 11	Data HW 4	Andy Kehler guest Kehler slides from this class (pdf) Kehler slides from semantics worksop talk (pdf)	Anaphora and Coreference: "Adventures in Statistical Pronoun Interpretation" Today's speaker is one of our three invited speakers from the Semantics Workshop, Andy Kehler. Jurafsky and Martin 2nd edition Chapter 21, Discourse, pages 15-39 Andrew Kehler, Douglas Appelt, Lara Taylor, and Aleksandr Simma. 2004a. The (Non)Utility of Predicate-Argument Frequencies for Pronoun Interpretation Proceedings of the Annual Meeting of the North American Chapter of the Association for Computational Linguistics (HLT/NAACL 2004), Boston, MA, May 2004. Andrew Kehler, Douglas Appelt, Lara Taylor, and Aleksandr Simma. 2004b. Competitive Self-Trained Pronoun Interpretation. Proceedings of the Annual Meeting of the North American Chapter of the Association for Computational Linguistics (HLT/NAACL 2004), Boston, MA, May 2004.
PART I: Back to Computational Lexical Semantics
4	Oct 16	Data HW 5 Submit list of 5 papers for lit review	Chris Lec 7 (ppt)	Semantic/Thematic Roles Daniel Gildea and Martha Palmer. 2002. The Necessity of Parsing for Predicate Argument Recognition. ACL 2002. Charles J. Fillmore and B. T. Atkins. Towards a frame-based lexicon: the case of RISK. In Frames and Fields, edited by A. Lehrer and E. Kittay, Erlbaum Publishers, 75-102, 1992. OPTIONAL ADVANCED READING: Kristina Toutanova, Aria Haghighi, and Christopher D. Manning. 2005. Joint Learning Improves Semantic Role Labeling. Proceedings of the 43nd Annual Meeting of the Association for Computational Linguistics (ACL 2005), pp. 589-596.
4	Oct 18	Prog HW 1 submission instructions	Dan Lec 8 (ppt) Lec 8 (6up pdf)	Summarization Jurafsky and Martin 2nd edition Chapter 23, Question Answering and Summarization, pages 21-41 S. Teufel, M. Moens, 2002: Summarizing Scientific Articles -- Experiments with Relevance and Rhetorical Status. In Computational Linguistics, 28 (4), Dec. 2002. Regina Barzilay and Lillian Lee. 2004. Catching the drift: Probabilistic content models, with applications to generation and summarization. Proceedings of HLT-NAACL, pp. 113-120, 2004. Best Paper Award.
5	Oct 23		Nate	Learning Relations from the Web O. Etzioni and M. Cafarella and D. Downey and A. Popescu and T. Shaked and S. Soderland and D. Weld and A. Yates. 2005. Unsupervised Named-Entity Extraction from the Web: An Experimental Study AI Journal. Rion Snow, Daniel Jurafsky, and Andrew Y. Ng. 2005. Learning syntactic patterns for automatic hypernym discovery. Proceedings of NIPS 2004.
5	Oct 25	Lit Review	Patrick Pantel	Today's class is moved to tomorrow to hear Patrick Pantel lecture. Hence note special time and place: Friday 3:30-5:00 in Margaret Jacks Hall 460-126
5	Oct 26		Patrick Pantel	Modeling and Explaining Similarity Note special time and place! Friday 3:30-5:00 in Margaret Jacks Hall 460-126 Today's speaker is one of our three invited speakers from the Semantics Workshop, Patrick Pantel". Here is a bio and talk abstract. Timothy Chklovski and Patrick Pantel. 2004. VerbOcean: Mining the Web for Fine-Grained Semantic Verb Relations". In Proceedings of Conference on Empirical Methods in Natural Language Processing (EMNLP-04). pp. 33-40. Barcelona, Spain. Patrick Pantel and Marco Pennacchiotti. 2006. Espresso: Leveraging Generic Patterns for Automatically Harvesting Semantic Relations. In Proceedings of Conference on Computational Linguistics / Association for Computational Linguistics (COLING/ACL-06). pp. 113-120. Sydney, Australia.
PART III: Computational Formal Semantics
6	Oct 30		Chris Lec 11 (ppt)	Building Semantic Representations: Lambda calculus Patrick Blackburn and Johan Bos. 2003. Computational Semantics. Theoria, 18, 27-45. Christopher Manning. 2005. An Introduction to Formal Computational Semantics. MS, Stanford U. More detailed coverage [Optional]: Patrick Blackburn and Johan Bos. 2005. Representation and Inference for Natural Language: A first course in computational semantics. Stanford, CA: CSLI Publications.
PART IV: Dialog
6	Nov 1		Dan	Dialog 1: Grounding, Confirmation, Dialogue Acts, Simple Dialogue Agents Jurafsky and Martin 2nd edition Chapter 24, Dialog and Conversational Agents, page 1-29
7	Nov 6		Dan	Dialog 2: Information State, Markov Decision Processes Jurafsky and Martin 2nd edition Chapter 24, Dialog and Conversational Agents, page 30-49
PART V -- Paraphrase, Summarization, and Textual Inference
7	Nov 8		Mirella Lapata Lapata slides (pdf)	Constraint-based Sentence Compression: An Integer Programming Approach Today's speaker is one of our three invited speakers from the Semantics Workshop, Mirella Lapata. Here is a talk abstract. James Clarke and Mirella Lapata, 2007. Modelling Compression with Discourse Constraints. In Proceedings of the Conference on Empirical Methods in Natural Language Processing and on Computational Natural Language Learning. Prague, Czech Republic. (Best Paper Award). James Clarke and Mirella Lapata, 2006. Models for Sentence Compression: A Comparison across Domains, Training Requirements and Evaluation Measures. In Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics, 377-384. Sydney, Australia.
Back to PART III: Computational Formal Semantics
8	Nov 13	Data HW 6	Chris	Scope Ambiguity and Underspecified Representations Christopher Manning. 2005. An Introduction to Formal Computational Semantics. MS, Stanford U. Derrick Higgins and Jerrold M. Sadock (2003). A machine learning approach to modeling scope preferences. Computational Linguistics 29(1), 73-96. OPTIONAL ADVANCED READING: Patrick Blackburn and Johan Bos. 2005. Representation and Inference for Natural Language: A First Course in Computational Semantics. Stanford, CA: CSLI Publications. Chapter 3. OPTIONAL ADVANCED READING: Bob Carpenter. 1997. Type-Logical Semantics. Stanford, CA: CSLI Publications. Chapters 3 and 7.
8	Nov 15		Chris	Learning to Map to Logical Forms for First order inference Johan Bos. 2006. Three Stories on Automated Reasoning for Natural Language Understanding. Workshop on Empirically Successful Computerized Reasoning. [paper] [slides] Luke S. Zettlemoyer and Michael Collins. 2005. Learning to Map Sentences to Logical Form: Structured Classification with Probabilistic Categorial Grammars. In Proceedings of the Twenty First Conference on Uncertainty in Artificial Intelligence (UAI-05). Luke S. Zettlemoyer and Michael Collins. 2007. Online Learning of Relaxed CCG Grammars for Parsing to Logical Form. In Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL-07), 2007.
Back to PART V: Summarization, Paraphrase, Textual Entailment
9	Nov 27		Chris	Paraphrase Dekang Lin and Patrick Pantel. 2001. Discovery of Inference Rules for Question Answering. Natural Language Engineering 7(4):343-360. Regina Barzilay and Lillian Lee. 2003. Learning to Paraphrase: An Unsupervised Approach Using Multiple-Sequence Alignment. Proceedings of HLT-NAACL 2003, pp. 16--23
9	Nov 29		Chris	Textual Inference Bobrow, Danny, Cleo Condoravdi, Richard Crouch, Ronald Kaplan, Lauri Karttunen, Tracy Holloway King, Valeria de Paiva, and Annie Zaenen. 2005. A Basic Logic for Textual Inference. In Proceedings of the AAAI Workshop on Inference for Textual Question Answering, Pittsburgh, PA. MacCartney, Bill, Trond Grenager, Marie-Catherine de Marneffe, Daniel Cer, and Christopher D. Manning. 2006. Learning to recognize features of valid textual entailments. To appear in Proceedings of NAACL-2006.
PART V: Projects
10	Dec 4		Eric Y. Kathryn Marie/Rob Aman/Jyotika/Jeremy	Project Presentations
10	Dec 6		Rebecca Thad/Nathan/Luis Mike/Eric Anish	Project Presentations
	Dec 10			Final Project due noon