Submission Golden Transformer

March 5, 2021, 7:42 a.m.

Team: Avengers Ensemble

Model url: https://github.com/neverix/avengers-ensemble


Total score: 0.679

Dataset Score Metric
LiDiRus 0.0 Matthew`s Corr
RCB 0.406 / 0.546 F1/Acc
PARus 0.908 Accuracy
MuSeRC 0.941 / 0.819 F1a/Em
TERRa 0.871 Accuracy
RUSSE 0.587 Accuracy
RWSD 0.545 Accuracy
DaNetQA 0.917 Accuracy
RuCoS 0.92 / 0.924 F1/EM
Model description:

Решение - это ансамбль трансформеров. Тренируется много моделей, из них получаются вероятности и подаются в CatBoost. Один и тот же подход используется для всех задач, но больше всего моделей в RuCoS, DaNetQA, TERRA, MuSeRC и PARus. Для LiDiRus нет решения, потому что используется ансамбль.


Parameter description:

Diagnostic (Matthew`s Correlation): 0.0

Category Score
LOGIC
KNOWLEDGE
PREDICATE-ARGUMENT STRUCTURE
LEXICAL SEMANTICS
Lexical Semantics - Lexical Entailment
Lexical Semantics - Morphological Negation
Lexical Semantics - Factivity
Lexical Semantics - Symmetry/Collectivity
Lexical Semantics - Redundancy
Lexical Semantics - Named Entities
Lexical Semantics - Quantifiers
Predicate-Argument Structure Core Args
Predicate-Argument Structure Prepositional Phrases
Predicate-Argument Structure Ellipsis/Implicits
Predicate-Argument Structure Anaphora/Coreference
Predicate-Argument Structure Active/Passive
Predicate-Argument Structure Nominalization
Predicate-Argument Structure Genitives/Partitives
Predicate-Argument Structure Datives
Predicate-Argument Structure Relative Clauses
Predicate-Argument Structure Coordination Scopes
Predicate-Argument Structure Intersectivity
Predicate-Argument Structure Restrictivity
Logic Negation
Logic Double Negation
Logic Interval/Numbers
Logic Conjuction
Logic Disjunction
Logic Conditionals
Logic Universal
Logic Existential
Logic Temporal
Logic Upward Monotone
Logic Downward Monotone
Logic Non-Monotonic
Knowledge Common Sense
Knowledge World Knowledge

Performance:

Dataset Speed RAM
LiDiRus - -
RCB - -
PARus - -
MuSeRC - -
TERRa - -
RUSSE - -
RWSD - -
DaNetQA - -
RuCoS - -