Submission mBART



Total score: 0.536

Dataset Score Metric
Diagnostic -0.003 Matthew's Corr
Russian CB 0.288/0.395 F1/Acc
PARus 0.528 Accuracy
MuSeRC 0.477/0.03 F1a/Em
TERRa 0.508 Accuracy
RUSSE 0.99 Accuracy
RWSD 0.649 Accuracy
DaNetQA 0.742 Accuracy
RuCoS 0.82/0.816 F1/EM
Model description:

Submission mBART RuCoS + DaNetQA + RUSSE + PARuS + RWSD + TERRa + LiDiRuS


Parameter description:

https://raw.githubusercontent.com/IlyaGusev/rsg_mbart/master/train.sh


Diagnostic (Matthew`s Correlation): -0.003

Category Score
LOGIC 0.08005884036263133
KNOWLEDGE -0.03737614612205334
PREDICATE-ARGUMENT STRUCTURE 0.04764021675457481
LEXICAL SEMANTICS 0.017544041143387888
Lexical Semantics - Lexical Entailment 0.0
Lexical Semantics - Morphological Negation 0.0
Lexical Semantics - Factivity 0.0
Lexical Semantics - Symmetry/Collectivity 0.43852900965351466
Lexical Semantics - Redundancy -0.08695652173913043
Lexical Semantics - Named Entities 0.0
Lexical Semantics - Quantifiers 0.15018785229652767
Predicate-Argument Structure Core Args 0.3
Predicate-Argument Structure Prepositional Phrases 0.0
Predicate-Argument Structure Ellipsis/Implicits 0.0
Predicate-Argument Structure Anaphora/Coreference 0.05549267431802431
Predicate-Argument Structure Active/Passive -0.1959179378817529
Predicate-Argument Structure Nominalization 0.0
Predicate-Argument Structure Genitives/Partitives 0.0
Predicate-Argument Structure Datives 0.0
Predicate-Argument Structure Relative Clauses -0.19377278632539327
Predicate-Argument Structure Coordination Scopes 0.0
Predicate-Argument Structure Intersectivity 0.0
Predicate-Argument Structure Restrictivity 0.0
Logic Negation 0.05198752449100364
Logic Double Negation 0.0
Logic Interval/Numbers 0.0
Logic Conjuction 0.0
Logic Disjunction 0.13276415922284168
Logic Conditionals 0.0
Logic Universal 0.0
Logic Existential 0.0
Logic Temporal 0.0
Logic Upward Monotone 0.0
Logic Downward Monotone 0.23371317622140134
Logic Non-Monotonic 0.0
Knowledge Common Sense -0.08663566158028119
Knowledge World Knowledge 0.009349308460387862