Submission GPT2-medium_bbpe_v50



Total score: 0.492

Dataset Score Metric
Diagnostic -0.047 Matthew's Corr
Russian CB 0.368/0.438 F1/Acc
PARus 0.584 Accuracy
MuSeRC 0.692/0.278 F1a/Em
TERRa 0.605 Accuracy
RUSSE 0.804 Accuracy
RWSD 0.597 Accuracy
DaNetQA 0.756 Accuracy
RuCoS 0.24/0.241 F1/EM
Model description:

gpt2_medium_bbpe


Parameter description:

gpt2_medium_bbpe


Diagnostic (Matthew`s Correlation): -0.047

Category Score
LOGIC -0.05357420371496753
KNOWLEDGE -0.022806509148052368
PREDICATE-ARGUMENT STRUCTURE -0.0375603101573776
LEXICAL SEMANTICS -0.03370354535751484
Lexical Semantics - Lexical Entailment -0.11295985477355723
Lexical Semantics - Morphological Negation 0.03289758474798845
Lexical Semantics - Factivity 0.03253000243161778
Lexical Semantics - Symmetry/Collectivity -0.12171612389003691
Lexical Semantics - Redundancy 0.0
Lexical Semantics - Named Entities -0.10050378152592121
Lexical Semantics - Quantifiers 0.08499377746907319
Predicate-Argument Structure Core Args -0.13474201390907387
Predicate-Argument Structure Prepositional Phrases -0.10207090877654817
Predicate-Argument Structure Ellipsis/Implicits -0.03268602252303067
Predicate-Argument Structure Anaphora/Coreference -0.033078507751560196
Predicate-Argument Structure Active/Passive -0.1959179378817529
Predicate-Argument Structure Nominalization 0.0
Predicate-Argument Structure Genitives/Partitives -0.11470786693528087
Predicate-Argument Structure Datives 0.35043832202523123
Predicate-Argument Structure Relative Clauses 0.2
Predicate-Argument Structure Coordination Scopes -0.23220893480841856
Predicate-Argument Structure Intersectivity -0.25415212262620723
Predicate-Argument Structure Restrictivity 0.2748737083745107
Logic Negation -0.16891891891891891
Logic Double Negation 0.035533452725935076
Logic Interval/Numbers 0.1910789265895446
Logic Conjuction -0.29617443887954614
Logic Disjunction -0.24929568306713198
Logic Conditionals 0.047619047619047616
Logic Universal 0.0
Logic Existential -0.10482848367219183
Logic Temporal -0.21867346044008387
Logic Upward Monotone -0.003948992518393949
Logic Downward Monotone 0.16238586255274184
Logic Non-Monotonic 0.2163856337073152
Knowledge Common Sense -0.02526696126225807
Knowledge World Knowledge -0.015070243278652105