Submission GPT2-small_bbpe_v50



Total score: 0.491

Dataset Score Metric
Diagnostic 0.054 Matthew's Corr
Russian CB 0.421/0.445 F1/Acc
PARus 0.544 Accuracy
MuSeRC 0.693/0.256 F1a/Em
TERRa 0.514 Accuracy
RUSSE 0.743 Accuracy
RWSD 0.675 Accuracy
DaNetQA 0.766 Accuracy
RuCoS 0.22/0.217 F1/EM
Model description:

GPT2-small_bbpe_v50


Parameter description:

GPT2-small_bbpe_v50


Diagnostic (Matthew`s Correlation): 0.054

Category Score
LOGIC 0.03118194086598855
KNOWLEDGE 0.010342231291592117
PREDICATE-ARGUMENT STRUCTURE 0.04074417333398244
LEXICAL SEMANTICS 0.1300000635426435
Lexical Semantics - Lexical Entailment 0.21929127541532256
Lexical Semantics - Morphological Negation -0.03289758474798845
Lexical Semantics - Factivity 0.02793993781135563
Lexical Semantics - Symmetry/Collectivity -0.07254762501100116
Lexical Semantics - Redundancy 0.16571045299983225
Lexical Semantics - Named Entities -0.1111111111111111
Lexical Semantics - Quantifiers 0.18071847525388818
Predicate-Argument Structure Core Args -0.06698622112410248
Predicate-Argument Structure Prepositional Phrases 0.08435021704156495
Predicate-Argument Structure Ellipsis/Implicits -0.08554415042912275
Predicate-Argument Structure Anaphora/Coreference 0.02667200160053352
Predicate-Argument Structure Active/Passive -0.14523855400757096
Predicate-Argument Structure Nominalization 0.0
Predicate-Argument Structure Genitives/Partitives 0.16666666666666666
Predicate-Argument Structure Datives 0.0890870806374748
Predicate-Argument Structure Relative Clauses 0.19377278632539327
Predicate-Argument Structure Coordination Scopes 0.2795426231258449
Predicate-Argument Structure Intersectivity 0.09307864628704345
Predicate-Argument Structure Restrictivity 0.0196078431372549
Logic Negation 0.06926662321769944
Logic Double Negation 0.243492376778837
Logic Interval/Numbers -0.040522044923655395
Logic Conjuction -0.20869967789998037
Logic Disjunction 0.11884057971014493
Logic Conditionals 0.2277100170213244
Logic Universal -0.3567530340063379
Logic Existential 0.060522753266880246
Logic Temporal 0.29012942659282975
Logic Upward Monotone -0.01292611223882469
Logic Downward Monotone 0.08112739143148366
Logic Non-Monotonic 0.11145564251507058
Knowledge Common Sense 0.014523380179577388
Knowledge World Knowledge -0.0015702786511989936