| Dataset | Score | Metric |
|---|---|---|
| LiDiRus | -0.003 | Matthew`s Corr |
| RCB | 0.395 / 0.288 | F1/Acc |
| PARus | 0.528 | Accuracy |
| MuSeRC | 0.477 / 0.03 | F1a/Em |
| TERRa | 0.508 | Accuracy |
| RUSSE | 0.99 | Accuracy |
| RWSD | 0.649 | Accuracy |
| DaNetQA | 0.742 | Accuracy |
| RuCoS | 0.82 / 0.816 | F1/EM |
Submission mBART RuCoS + DaNetQA + RUSSE + PARuS + RWSD + TERRa + LiDiRuS
https://raw.githubusercontent.com/IlyaGusev/rsg_mbart/master/train.sh
| Category | Score |
|---|---|
| LOGIC | 0.0800588403626313 |
| KNOWLEDGE | -0.0373761461220533 |
| PREDICATE-ARGUMENT STRUCTURE | 0.0476402167545748 |
| LEXICAL SEMANTICS | 0.0175440411433879 |
| Lexical Semantics - Lexical Entailment | 0.0 |
|---|---|
| Lexical Semantics - Morphological Negation | 0.0 |
| Lexical Semantics - Factivity | 0.0 |
| Lexical Semantics - Symmetry/Collectivity | 0.438529009653515 |
| Lexical Semantics - Redundancy | -0.0869565217391304 |
| Lexical Semantics - Named Entities | 0.0 |
| Lexical Semantics - Quantifiers | 0.150187852296528 |
| Predicate-Argument Structure Core Args | 0.3 |
| Predicate-Argument Structure Prepositional Phrases | 0.0 |
| Predicate-Argument Structure Ellipsis/Implicits | 0.0 |
| Predicate-Argument Structure Anaphora/Coreference | 0.0554926743180243 |
| Predicate-Argument Structure Active/Passive | -0.195917937881753 |
| Predicate-Argument Structure Nominalization | 0.0 |
| Predicate-Argument Structure Genitives/Partitives | 0.0 |
| Predicate-Argument Structure Datives | 0.0 |
| Predicate-Argument Structure Relative Clauses | -0.193772786325393 |
| Predicate-Argument Structure Coordination Scopes | 0.0 |
| Predicate-Argument Structure Intersectivity | 0.0 |
| Predicate-Argument Structure Restrictivity | 0.0 |
| Logic Negation | 0.0519875244910036 |
| Logic Double Negation | 0.0 |
| Logic Interval/Numbers | 0.0 |
| Logic Conjuction | 0.0 |
| Logic Disjunction | 0.132764159222842 |
| Logic Conditionals | 0.0 |
| Logic Universal | 0.0 |
| Logic Existential | 0.0 |
| Logic Temporal | 0.0 |
| Logic Upward Monotone | 0.0 |
| Logic Downward Monotone | 0.233713176221401 |
| Logic Non-Monotonic | 0.0 |
| Knowledge Common Sense | -0.0866356615802812 |
| Knowledge World Knowledge | 0.00934930846038786 |