• Download data for all tasks from the section Tasks
• Investigate data carefully. We provide information about datasets on our site and publications.
• If you find errors or have some comments and suggestions - write us on russiansuperglue@gmail.com

If you do not know how to start you can check our jupyther notebook how to use pretrained jiant model
You can also train your model in any way, but be careful with output data!

• Use the IDs and labels present in the unlabeled test JSONLs to generate one JSONL of predictions for each of the test files. Each line of the prediction files should be a JSONL entry, with a sorted idx field to identify the example and a `label` field with the prediction.
• Make sure that each prediction JSONL is named according to the following:
• DaNetQA: DaNetQA.jsonl
• RCB: RCB.jsonl
• PARus: PARus.jsonl
• MuSeRC: MuSeRC.jsonl
• RuCoS: RuCoS.jsonl
• TERRa: TERRa.jsonl
• Russian words in Context: RUSSE.jsonl
• Russian Winograd Schema: RWSD.jsonl
• Broad Coverage Diagnostics: LiDiRus.jsonl
• You can also submit incomplete zip. In this case you will get the score only for those tasks that you upload.

Click on the "Login" link in the right corner of the webpage.

• If you already have an account - please, login. If you have not registered yet - click Register button.
  We only ask for name and email. You can register anonymously.
• After registration your private submission page will appear.
  You could make a new submission here, get its scores, delete bad one, ask for publication of your submit to leaderboard.

• On your submissions page click on + right to "My submissions" title.
• In the form please check all the information about your submission.
• There are several obligatory fields (Submission name, teamname and your zip)
• Please, fill in all of them if you would like to see your model on the leaderboard. It is obligatory. Our team need to verify everything before making it public.
• If anything is wrong with your submission, you will see the error page with a list of log comments.
• You may upload at most ten submissions a day.
• A sample submission in the required format is available here.

Congratulations! If you are not the leader yet, good luck! ;)