Submission Baseline TF-IDF



Total score: 0.372

Dataset Score Metric
Diagnostic -0.004 Matthew's Corr
Russian CB 0.288/0.395 F1/Acc
PARus 0.522 Accuracy
MuSeRC 0.477/0.03 F1a/Em
TERRa 0.496 Accuracy
RUSSE 0.632 Accuracy
RWSD 0.338 Accuracy
DaNetQA 0.763 Accuracy
RuCoS 0.0/0.002 F1/EM
Model description:

Baseline for SuperGLUE. tf-idf vectorizer plus linear regression The vectorizer you can download here: https://russiansuperglue.com/tasks/tf_idf


Parameter description:

Baseline for SuperGLUE. All information and scripts are available in our repo.


Diagnostic (Matthew`s Correlation): -0.004

Category Score
LOGIC -0.04399480227396054
KNOWLEDGE -0.015269750696891874
PREDICATE-ARGUMENT STRUCTURE 0.0023293218918414875
LEXICAL SEMANTICS 0.03199952133618396
Lexical Semantics - Lexical Entailment -0.03092053308710669
Lexical Semantics - Morphological Negation 0.0
Lexical Semantics - Factivity 0.20672455764868075
Lexical Semantics - Symmetry/Collectivity 0.17541160386140586
Lexical Semantics - Redundancy 0.0
Lexical Semantics - Named Entities 0.0
Lexical Semantics - Quantifiers 0.0
Predicate-Argument Structure Core Args 0.19245008972987526
Predicate-Argument Structure Prepositional Phrases -0.11862754599800839
Predicate-Argument Structure Ellipsis/Implicits 0.0
Predicate-Argument Structure Anaphora/Coreference 0.033078507751560196
Predicate-Argument Structure Active/Passive -0.1546720562224365
Predicate-Argument Structure Nominalization 0.0
Predicate-Argument Structure Genitives/Partitives 0.0
Predicate-Argument Structure Datives 0.0
Predicate-Argument Structure Relative Clauses 0.0
Predicate-Argument Structure Coordination Scopes 0.0
Predicate-Argument Structure Intersectivity 0.03765216631499367
Predicate-Argument Structure Restrictivity 0.0
Logic Negation -0.03653310829007939
Logic Double Negation 0.0
Logic Interval/Numbers -0.1630980846658687
Logic Conjuction 0.0
Logic Disjunction 0.0
Logic Conditionals -0.20365326999063918
Logic Universal 0.0
Logic Existential 0.0
Logic Temporal -0.010535659020683567
Logic Upward Monotone 0.029617443887954616
Logic Downward Monotone 0.0
Logic Non-Monotonic 0.0
Knowledge Common Sense -0.06323200493946762
Knowledge World Knowledge -0.03360690078067043