Submission GPT2-large_bbpe_v50



Total score: 0.539

Dataset Score Metric
Diagnostic 0.099 Matthew's Corr
Russian CB 0.406/0.447 F1/Acc
PARus 0.58 Accuracy
MuSeRC 0.699/0.327 F1a/Em
TERRa 0.699 Accuracy
RUSSE 0.835 Accuracy
RWSD 0.688 Accuracy
DaNetQA 0.756 Accuracy
RuCoS 0.25/0.248 F1/EM
Model description:

GPT2-large bbpe v50


Parameter description:

GPT2-large_bbpe_v50


Diagnostic (Matthew`s Correlation): 0.099

Category Score
LOGIC 0.13552034690912834
KNOWLEDGE 0.15930171626202408
PREDICATE-ARGUMENT STRUCTURE 0.11182969566291902
LEXICAL SEMANTICS 0.009998666933274088
Lexical Semantics - Lexical Entailment 0.055629187868262146
Lexical Semantics - Morphological Negation -0.06023386019368342
Lexical Semantics - Factivity -0.21081851067789195
Lexical Semantics - Symmetry/Collectivity 0.0
Lexical Semantics - Redundancy 0.2211629342323457
Lexical Semantics - Named Entities 0.0
Lexical Semantics - Quantifiers 0.15018785229652767
Predicate-Argument Structure Core Args 0.37532594530273455
Predicate-Argument Structure Prepositional Phrases 0.20558320075809686
Predicate-Argument Structure Ellipsis/Implicits -0.014731391274719742
Predicate-Argument Structure Anaphora/Coreference 0.09634388527686184
Predicate-Argument Structure Active/Passive -0.05484084971070818
Predicate-Argument Structure Nominalization 0.08268982305947231
Predicate-Argument Structure Genitives/Partitives -0.16666666666666666
Predicate-Argument Structure Datives 0.0
Predicate-Argument Structure Relative Clauses -0.022222222222222223
Predicate-Argument Structure Coordination Scopes 0.20894746626614943
Predicate-Argument Structure Intersectivity 0.10955793822221753
Predicate-Argument Structure Restrictivity -0.21004201260420147
Logic Negation 0.12253577034896795
Logic Double Negation 0.2842676218074806
Logic Interval/Numbers 0.1630980846658687
Logic Conjuction 0.22056438662814232
Logic Disjunction 0.05439611381401831
Logic Conditionals 0.0
Logic Universal -0.30403449605253013
Logic Existential 0.16834512458535864
Logic Temporal 0.12998877418150556
Logic Upward Monotone 0.27640672769878033
Logic Downward Monotone -0.035955873264830976
Logic Non-Monotonic 0.22291128503014115
Knowledge Common Sense 0.164150727849255
Knowledge World Knowledge 0.1383353822832248