Leaderboard

Version 1.1

Rank Name Team Link Score LiDiRus RCB PARus MuSeRC TERRa RUSSE RWSD DaNetQA RuCoS
1 HUMAN BENCHMARK AGI NLP 0.8 0.626 0.68 / 0.702 0.982 0.806 / 0.42 0.92 0.747 0.84 0.879 0.93 / 0.89
2 RuBERT conversational AGI NLP 0.546 0.186 0.468 / 0.432 0.61 0.656 / 0.256 0.639 0.894 0.675 0.749 0.255 / 0.251
3 Multilingual BERT AGI NLP 0.542 0.157 0.425 / 0.365 0.588 0.626 / 0.253 0.62 0.84 0.675 0.79 0.371 / 0.367
4 GPT2-large_bbpe_v50 AGI NLP 0.539 0.099 0.447 / 0.406 0.58 0.699 / 0.327 0.699 0.835 0.688 0.756 0.25 / 0.248
5 mBART - 0.536 -0.003 0.395 / 0.288 0.528 0.477 / 0.03 0.508 0.99 0.649 0.742 0.82 / 0.816
6 Plain RuBERT DeepPavlov 0.524 -0.026 0.393 / 0.338 0.532 0.712 / 0.309 0.636 0.877 0.662 0.78 0.38 / 0.379
7 GPT2-medium_bbpe_v50 AGI NLP 0.492 -0.047 0.438 / 0.368 0.584 0.692 / 0.278 0.605 0.804 0.597 0.756 0.24 / 0.241
8 GPT2-small_bbpe_v50 AGI NLP 0.491 0.054 0.445 / 0.421 0.544 0.693 / 0.256 0.514 0.743 0.675 0.766 0.22 / 0.217
9 Slavic BERT DeepPavlov 0.483 -0.01 0.418 / 0.34 0.526 0.677 / 0.27 0.566 0.829 0.578 0.759 0.24 / 0.242
10 Baseline TF-IDF v1.0 AGI NLP 0.438 -0.004 0.395 / 0.288 0.522 0.477 / 0.03 0.496 0.632 0.66 0.763 0.286 / 0.28