3 февраля 2021 г. 19:56
Команда hse_ling
| Датасет | Результат | Метрика |
|---|---|---|
| LiDiRus | 0,0 | Кор, коэффициент Мэтью |
| RCB | 0,319 / 0,374 | F1/Точность |
| PARus | 0,48 | Точность |
| MuSeRC | 0,45 / 0,071 | F1a/Em |
| TERRa | 0,483 | Точность |
| RUSSE | 0,528 | Точность |
| RWSD | 0,597 | Точность |
| DaNetQA | 0,52 | Точность |
| RuCoS | 0,25 / 0,247 | F1/EM |
Random balanced guess based on class distrubution in the combined train and validation data
| Категория | Результат |
|---|---|
| LOGIC | -0,04431944169485313 |
| KNOWLEDGE | 0,11890162514613671 |
| PREDICATE-ARGUMENT STRUCTURE | -0,013878552845016493 |
| LEXICAL SEMANTICS | 0,0111864848025246 |
| Lexical Semantics - Lexical Entailment | 0,028957191003411698 |
|---|---|
| Lexical Semantics - Morphological Negation | 0,0 |
| Lexical Semantics - Factivity | -0,06804138174397717 |
| Lexical Semantics - Symmetry/Collectivity | 0,023124864503144014 |
| Lexical Semantics - Redundancy | 0,30692490702242564 |
| Lexical Semantics - Named Entities | 0,16692446522239718 |
| Lexical Semantics - Quantifiers | 0,10042420607280685 |
| Predicate-Argument Structure Core Args | 0,0075427137033570076 |
| Predicate-Argument Structure Prepositional Phrases | 0,029424494316824985 |
| Predicate-Argument Structure Ellipsis/Implicits | 0,2916666666666667 |
| Predicate-Argument Structure Anaphora/Coreference | -0,23644475046777222 |
| Predicate-Argument Structure Active/Passive | -0,05923488777590923 |
| Predicate-Argument Structure Nominalization | -0,24552983583446694 |
| Predicate-Argument Structure Genitives/Partitives | -0,20100756305184242 |
| Predicate-Argument Structure Datives | 0,0 |
| Predicate-Argument Structure Relative Clauses | 0,01642880193633814 |
| Predicate-Argument Structure Coordination Scopes | -0,04279604925109129 |
| Predicate-Argument Structure Intersectivity | 0,13245323570650436 |
| Predicate-Argument Structure Restrictivity | 0,07669649888473705 |
| Logic Negation | -0,04864043305310773 |
| Logic Double Negation | 0,0 |
| Logic Interval/Numbers | 0,05665250531028743 |
| Logic Conjuction | 0,051639777949432225 |
| Logic Disjunction | -0,31212739461154115 |
| Logic Conditionals | -0,040082172520323485 |
| Logic Universal | 0,11396057645963795 |
| Logic Existential | 0,04279604925109129 |
| Logic Temporal | -0,24039295346263034 |
| Logic Upward Monotone | -0,03226552647887426 |
| Logic Downward Monotone | 0,06726727939963124 |
| Logic Non-Monotonic | 0,03217447260438935 |
| Knowledge Common Sense | 0,20327058201279066 |
| Knowledge World Knowledge | 0,00995432721361685 |
| Датасет | Speed | RAM |
|---|---|---|
| LiDiRus | - | - |
| RCB | - | - |
| PARus | - | - |
| MuSeRC | - | - |
| TERRa | - | - |
| RUSSE | - | - |
| RWSD | - | - |
| DaNetQA | - | - |
| RuCoS | - | - |