Feb. 3, 2021, 7:56 p.m.
Team: hse_ling
| Dataset | Score | Metric |
|---|---|---|
| LiDiRus | 0.0 | Matthew`s Corr |
| RCB | 0.319 / 0.374 | F1/Acc |
| PARus | 0.48 | Accuracy |
| MuSeRC | 0.45 / 0.071 | F1a/Em |
| TERRa | 0.483 | Accuracy |
| RUSSE | 0.528 | Accuracy |
| RWSD | 0.597 | Accuracy |
| DaNetQA | 0.52 | Accuracy |
| RuCoS | 0.25 / 0.247 | F1/EM |
Random balanced guess based on class distrubution in the combined train and validation data
| Category | Score |
|---|---|
| LOGIC | -0.04431944169485313 |
| KNOWLEDGE | 0.11890162514613671 |
| PREDICATE-ARGUMENT STRUCTURE | -0.013878552845016493 |
| LEXICAL SEMANTICS | 0.0111864848025246 |
| Lexical Semantics - Lexical Entailment | 0.028957191003411698 |
|---|---|
| Lexical Semantics - Morphological Negation | 0.0 |
| Lexical Semantics - Factivity | -0.06804138174397717 |
| Lexical Semantics - Symmetry/Collectivity | 0.023124864503144014 |
| Lexical Semantics - Redundancy | 0.30692490702242564 |
| Lexical Semantics - Named Entities | 0.16692446522239718 |
| Lexical Semantics - Quantifiers | 0.10042420607280685 |
| Predicate-Argument Structure Core Args | 0.0075427137033570076 |
| Predicate-Argument Structure Prepositional Phrases | 0.029424494316824985 |
| Predicate-Argument Structure Ellipsis/Implicits | 0.2916666666666667 |
| Predicate-Argument Structure Anaphora/Coreference | -0.23644475046777222 |
| Predicate-Argument Structure Active/Passive | -0.05923488777590923 |
| Predicate-Argument Structure Nominalization | -0.24552983583446694 |
| Predicate-Argument Structure Genitives/Partitives | -0.20100756305184242 |
| Predicate-Argument Structure Datives | 0.0 |
| Predicate-Argument Structure Relative Clauses | 0.01642880193633814 |
| Predicate-Argument Structure Coordination Scopes | -0.04279604925109129 |
| Predicate-Argument Structure Intersectivity | 0.13245323570650436 |
| Predicate-Argument Structure Restrictivity | 0.07669649888473705 |
| Logic Negation | -0.04864043305310773 |
| Logic Double Negation | 0.0 |
| Logic Interval/Numbers | 0.05665250531028743 |
| Logic Conjuction | 0.051639777949432225 |
| Logic Disjunction | -0.31212739461154115 |
| Logic Conditionals | -0.040082172520323485 |
| Logic Universal | 0.11396057645963795 |
| Logic Existential | 0.04279604925109129 |
| Logic Temporal | -0.24039295346263034 |
| Logic Upward Monotone | -0.03226552647887426 |
| Logic Downward Monotone | 0.06726727939963124 |
| Logic Non-Monotonic | 0.03217447260438935 |
| Knowledge Common Sense | 0.20327058201279066 |
| Knowledge World Knowledge | 0.00995432721361685 |
| Dataset | Speed | RAM |
|---|---|---|
| LiDiRus | - | - |
| RCB | - | - |
| PARus | - | - |
| MuSeRC | - | - |
| TERRa | - | - |
| RUSSE | - | - |
| RWSD | - | - |
| DaNetQA | - | - |
| RuCoS | - | - |