20 июня 2020 г. 6:35
Команда DeepPavlov
Ссылка на модель http://files.deeppavlov.ai/deeppavlov_data/bert/bg_cs_pl_ru_cased_L-12_H-768_A-12_pt.tar.gz
| Датасет | Результат | Метрика |
|---|---|---|
| LiDiRus | -0,01 | Кор, коэффициент Мэтью |
| RCB | 0,418 / 0,34 | F1/Точность |
| PARus | 0,526 | Точность |
| MuSeRC | 0,677 / 0,27 | F1a/Em |
| TERRa | 0,566 | Точность |
| RUSSE | 0,829 | Точность |
| RWSD | 0,578 | Точность |
| DaNetQA | 0,759 | Точность |
| RuCoS | 0,24 / 0,242 | F1/EM |
SlavicBERT was trained on Russian News and four Wikipedias: Bulgarian, Czech, Polish, and Russian. Subtoken vocabulary was built using this data. Multilingual BERT was used as an initialization for SlavicBERT. The model is described in our ACL paper 2
SlavicBERT was trained on Russian News and four Wikipedias: Bulgarian, Czech, Polish, and Russian. Subtoken vocabulary was built using this data. Multilingual BERT was used as an initialization for SlavicBERT. The model is described in our ACL paper 2
| Категория | Результат |
|---|---|
| LOGIC | -0,0632310551535294 |
| KNOWLEDGE | -0,0451732759757929 |
| PREDICATE-ARGUMENT STRUCTURE | 0,0196785023856931 |
| LEXICAL SEMANTICS | -0,0619849549063655 |
| Lexical Semantics - Lexical Entailment | 0,0378595993036933 |
|---|---|
| Lexical Semantics - Morphological Negation | -0,390094748802747 |
| Lexical Semantics - Factivity | -0,165395353925991 |
| Lexical Semantics - Symmetry/Collectivity | 0,294883912309794 |
| Lexical Semantics - Redundancy | -0,273483017137309 |
| Lexical Semantics - Named Entities | -0,169030850945703 |
| Lexical Semantics - Quantifiers | -0,012515654358044 |
| Predicate-Argument Structure Core Args | 0,288888888888889 |
| Predicate-Argument Structure Prepositional Phrases | -0,0907913964923987 |
| Predicate-Argument Structure Ellipsis/Implicits | 0,0 |
| Predicate-Argument Structure Anaphora/Coreference | -0,203804372701054 |
| Predicate-Argument Structure Active/Passive | 0,272480483769182 |
| Predicate-Argument Structure Nominalization | 0,206724557648681 |
| Predicate-Argument Structure Genitives/Partitives | 0,114707866935281 |
| Predicate-Argument Structure Datives | 0,150187852296528 |
| Predicate-Argument Structure Relative Clauses | -0,0975900072948533 |
| Predicate-Argument Structure Coordination Scopes | -0,117501940896304 |
| Predicate-Argument Structure Intersectivity | 0,0314530346223414 |
| Predicate-Argument Structure Restrictivity | 0,0 |
| Logic Negation | -0,0198530638477829 |
| Logic Double Negation | 0,100503781525921 |
| Logic Interval/Numbers | 0,145896036510803 |
| Logic Conjuction | 0,0 |
| Logic Disjunction | 0,0 |
| Logic Conditionals | 0,0 |
| Logic Universal | -0,193476497487974 |
| Logic Existential | -0,244599795235114 |
| Logic Temporal | -0,186870636860463 |
| Logic Upward Monotone | 0,0296174438879546 |
| Logic Downward Monotone | 0,0527687954284643 |
| Logic Non-Monotonic | 0,0 |
| Knowledge Common Sense | -0,040078628209099 |
| Knowledge World Knowledge | -0,0638191925339519 |