DATA HOMEWORK 6

Look at the Recognizing Textual Entailment 2007 (RTE3) development data set. Here is the XML file, with a stylesheet to format it nicely just for you :-). It gives pairs of a passage and an hypothesis. An automatic system is meant to determine the given gold standard answer, which is whether the hypothesis follows from the text (YES) or not (NO). Note that this is in a pragmatic sense of plausible inference, not strict logical entailment. Manning has written about the basis of this definition.

For the first 5 cases below, just write one sentence explaining the situation. For 6-7, write a little more. For 8, write nothing or any amount! But please include the text of the pairs that you are discussing in your write-up. That'll make it tons easier on Nate! Thanks!!

  1. Find one example for which reliably getting the correct answer depends on recognizing a synonym or hypernym.

  2. Find one example for which reliably getting the correct answer depends on non-trivial temporal inference. (That is, not just recognizing that two overt absolute date expressions match or mismatch.)

  3. Find one example for which reliably getting the correct answer depends on successfully doing anaphora resolution.

  4. Find one example for which analysis in terms of semantic roles of predicates would be helpful. (That is, the respective predicates must be different parts of speech, or have different diathesis patterns, so that you can't accurately match or determine a mismatch just at the level of grammatical relations.)

  5. Find one example for which learning class instances or relations, of the sort discussed in the Oct 23 lecture (or the Etzioni et al. paper) would be useful.

  6. Find one example, for which a satisfactory treatment demands a non-trivial amount of understanding and inference. (That is, definitely more than just synonyms, hyponyms, and the other things listed above.)

  7. Find any other example which seems interesting to you, and write about the issues involved in handling it. Look carefully! So often sentence understanding is so "obvious" to humans that people fail to notice the difficulties that a computer will encounter.

  8. Optional: Find an example where you disagree with the answer given as the gold standard. Discuss why you think the given answer is wrong, and what ontological or epistemological factors the decision turns on.