June 20, 2020, 6:35 a.m.
Team: DeepPavlov
Model url: http://files.deeppavlov.ai/deeppavlov_data/bert/bg_cs_pl_ru_cased_L-12_H-768_A-12_pt.tar.gz
| Dataset | Score | Metric |
|---|---|---|
| LiDiRus | -0.01 | Matthew`s Corr |
| RCB | 0.418 / 0.34 | F1/Acc |
| PARus | 0.526 | Accuracy |
| MuSeRC | 0.677 / 0.27 | F1a/Em |
| TERRa | 0.566 | Accuracy |
| RUSSE | 0.829 | Accuracy |
| RWSD | 0.578 | Accuracy |
| DaNetQA | 0.759 | Accuracy |
| RuCoS | 0.24 / 0.242 | F1/EM |
SlavicBERT was trained on Russian News and four Wikipedias: Bulgarian, Czech, Polish, and Russian. Subtoken vocabulary was built using this data. Multilingual BERT was used as an initialization for SlavicBERT. The model is described in our ACL paper 2
SlavicBERT was trained on Russian News and four Wikipedias: Bulgarian, Czech, Polish, and Russian. Subtoken vocabulary was built using this data. Multilingual BERT was used as an initialization for SlavicBERT. The model is described in our ACL paper 2
| Category | Score |
|---|---|
| LOGIC | -0.0632310551535294 |
| KNOWLEDGE | -0.0451732759757929 |
| PREDICATE-ARGUMENT STRUCTURE | 0.0196785023856931 |
| LEXICAL SEMANTICS | -0.0619849549063655 |
| Lexical Semantics - Lexical Entailment | 0.0378595993036933 |
|---|---|
| Lexical Semantics - Morphological Negation | -0.390094748802747 |
| Lexical Semantics - Factivity | -0.165395353925991 |
| Lexical Semantics - Symmetry/Collectivity | 0.294883912309794 |
| Lexical Semantics - Redundancy | -0.273483017137309 |
| Lexical Semantics - Named Entities | -0.169030850945703 |
| Lexical Semantics - Quantifiers | -0.012515654358044 |
| Predicate-Argument Structure Core Args | 0.288888888888889 |
| Predicate-Argument Structure Prepositional Phrases | -0.0907913964923987 |
| Predicate-Argument Structure Ellipsis/Implicits | 0.0 |
| Predicate-Argument Structure Anaphora/Coreference | -0.203804372701054 |
| Predicate-Argument Structure Active/Passive | 0.272480483769182 |
| Predicate-Argument Structure Nominalization | 0.206724557648681 |
| Predicate-Argument Structure Genitives/Partitives | 0.114707866935281 |
| Predicate-Argument Structure Datives | 0.150187852296528 |
| Predicate-Argument Structure Relative Clauses | -0.0975900072948533 |
| Predicate-Argument Structure Coordination Scopes | -0.117501940896304 |
| Predicate-Argument Structure Intersectivity | 0.0314530346223414 |
| Predicate-Argument Structure Restrictivity | 0.0 |
| Logic Negation | -0.0198530638477829 |
| Logic Double Negation | 0.100503781525921 |
| Logic Interval/Numbers | 0.145896036510803 |
| Logic Conjuction | 0.0 |
| Logic Disjunction | 0.0 |
| Logic Conditionals | 0.0 |
| Logic Universal | -0.193476497487974 |
| Logic Existential | -0.244599795235114 |
| Logic Temporal | -0.186870636860463 |
| Logic Upward Monotone | 0.0296174438879546 |
| Logic Downward Monotone | 0.0527687954284643 |
| Logic Non-Monotonic | 0.0 |
| Knowledge Common Sense | -0.040078628209099 |
| Knowledge World Knowledge | -0.0638191925339519 |