Description

Textual Entailment Recognition has been proposed recently as a generic task that captures major semantic inference needs across many NLP applications, such as Question Answering, Information Retrieval, Information Extraction, and Text Summarization. This task requires to recognize, given two text fragments, whether the meaning of one text is entailed (can be inferred) from the other text.

Task Type¶

RTE (Recognizing Textual Entailment) Sentence Pair Classification - Entailment - Not Entailment

Example¶

{
  "premise": "Автор поста написал в комментарии, что прорвалась канализация.",
  "hypothesis": "Автор поста написал про канализацию.",
  "label": "entailment",
  "idx": "6062"
}
        

How did we collect data? ¶

All text examples were collected from open news sources and literary magazines, then manually reviewed and supplemented by a human assessment on Yandex.Toloka

RTE Challanges (ACL)

• RTE-4 TAC 2008
• RTE-5 TAC 2009
• RTE-6 TAC 2010
• RTE-7 TAC 2011
• The Joint Student Response Analysis and 8th Recognizing Textual Entailment Challenge SemEval 2013

State of the Art

English RTE - Accuracy 92.5%

Related papers

• Wang A. et al. Superglue: A stickier benchmark for general-purpose language understanding systems //Advances in Neural Information Processing Systems. – 2019. – С. 3261-3275.