Submission Multilingual Bert

Nov. 18, 2020, 7:43 p.m.

Team: DeepPavlov

Model url: https://huggingface.co/bert-base-multilingual-cased


Total score: 0.495

Dataset Score Metric
LiDiRus 0.189 Matthew`s Corr
RCB 0.367 / 0.445 F1/Acc
PARus 0.528 Accuracy
MuSeRC 0.639 / 0.239 F1a/Em
TERRa 0.617 Accuracy
RUSSE 0.69 Accuracy
RWSD 0.669 Accuracy
DaNetQA 0.624 Accuracy
RuCoS 0.29 / 0.29 F1/EM
Model description:

bert-base-multilingual-cased


Parameter description:

Standard

Diagnostic (Matthew`s Correlation): 0.189

Category Score
LOGIC 0.13735664681948273
KNOWLEDGE 0.1265469818958828
PREDICATE-ARGUMENT STRUCTURE 0.18920483573043412
LEXICAL SEMANTICS 0.22242373756652029
Lexical Semantics - Lexical Entailment 0.0584543445233507
Lexical Semantics - Morphological Negation 0.3003124875651863
Lexical Semantics - Factivity 0.3266793038882825
Lexical Semantics - Symmetry/Collectivity -0.12171612389003691
Lexical Semantics - Redundancy 0.45652173913043476
Lexical Semantics - Named Entities 0.055641488407465724
Lexical Semantics - Quantifiers 0.2597177743262246
Predicate-Argument Structure Core Args 0.2297707831614211
Predicate-Argument Structure Prepositional Phrases 0.3972493441698867
Predicate-Argument Structure Ellipsis/Implicits -0.014731391274719742
Predicate-Argument Structure Anaphora/Coreference 0.23337331720804128
Predicate-Argument Structure Active/Passive 0.2559239653166382
Predicate-Argument Structure Nominalization -0.025949964805384102
Predicate-Argument Structure Genitives/Partitives 0.15724272550828775
Predicate-Argument Structure Datives 0.2182178902359924
Predicate-Argument Structure Relative Clauses 0.3333333333333333
Predicate-Argument Structure Coordination Scopes 0.20894746626614943
Predicate-Argument Structure Intersectivity 0.23234812346321748
Predicate-Argument Structure Restrictivity 0.0
Logic Negation 0.040398847130309966
Logic Double Negation 0.0
Logic Interval/Numbers 0.03262334552859464
Logic Conjuction 0.4173650061841514
Logic Disjunction 0.17035261607214167
Logic Conditionals -0.24039295346263034
Logic Universal 0.5606119105813882
Logic Existential -0.04279604925109129
Logic Temporal 0.11396057645963795
Logic Upward Monotone 0.4083133966424866
Logic Downward Monotone 0.2788995520575487
Logic Non-Monotonic 0.10244389988519648
Knowledge Common Sense 0.07314923469853085
Knowledge World Knowledge 0.15662622780364913