WMT25: Slavic LLMs for MT & QA

The task focuses on developing and improve LLMs under limited data and compute resources for three Slavic languages: Ukrainian (uk), Upper Sorbian (hsb) and Lower Sorbian (dsb). We consider two tasks that are to be evaluated jointly: Machine Translation (MT) and Multiple-Choice Question Answering (QA).

Official website: https://www2.statmt.org/wmt25/limited-resources-slavic-llm.html

The anonymous submission with the earliest submission date in each leaderboard corresponds to the baseline model.

Download test sets Register your team Create submission Competition updates

Leaderboard of WMT25: Slavic LLMs for MT & QA

wmtslavicllm2025-mt.cs-uk test set (cs-uk)

# Name BLEU chrF Date
1 Anonymous submission #157 --- --- July 22, 2025, 9:26 p.m.
2 Anonymous submission #129 --- --- July 11, 2025, 7:21 p.m.
BLEU and ChrF are sacreBLEU scores. Systems in bold face are your submissions. We only display the top-10 submissions per language pair. Submission validation errors denoted by -1.0 score.

Click on the column header to sort the table. Hold down the Shift key and select a second column to sort by multiple criteria.

wmtslavicllm2025-mt.de-dsb test set (de-dsb)

# Name BLEU chrF Date
1 Anonymous submission #140 61.6 79.8 July 18, 2025, 12:29 a.m.
2 Anonymous submission #171 60.9 79.8 July 23, 2025, 8:46 a.m.
3 Anonymous submission #167 60.1 79.2 July 23, 2025, 8:39 a.m.
4 Anonymous submission #151 59.7 79.1 July 22, 2025, 7:39 a.m.
5 Anonymous submission #135 58.2 76.4 July 16, 2025, 9:06 p.m.
6 Anonymous submission #144 49.2 72.1 July 18, 2025, 12:33 a.m.
7 Anonymous submission #146 41.6 67.5 July 21, 2025, 11:41 a.m.
8 Anonymous submission #179 39.9 66.6 July 23, 2025, 11:24 a.m.
9 Anonymous submission #124 1.7 13.7 July 11, 2025, 12:39 p.m.
BLEU and ChrF are sacreBLEU scores. Systems in bold face are your submissions. We only display the top-10 submissions per language pair. Submission validation errors denoted by -1.0 score.

Click on the column header to sort the table. Hold down the Shift key and select a second column to sort by multiple criteria.

wmtslavicllm2025-mt.de-hsb test set (de-hsb)

# Name BLEU chrF Date
1 Anonymous submission #139 76.3 88.2 July 18, 2025, 12:29 a.m.
2 Anonymous submission #170 74.4 87.4 July 23, 2025, 8:45 a.m.
3 Anonymous submission #162 73.5 87.0 July 23, 2025, 8:29 a.m.
4 Anonymous submission #153 72.7 86.8 July 22, 2025, 7:44 a.m.
5 Anonymous submission #136 74.2 86.1 July 17, 2025, 5:53 p.m.
6 Anonymous submission #143 62.5 80.8 July 18, 2025, 12:31 a.m.
7 Anonymous submission #180 55.9 77.6 July 23, 2025, 11:25 a.m.
8 Anonymous submission #156 55.9 77.5 July 22, 2025, 12:12 p.m.
9 Anonymous submission #174 3.2 24.1 July 23, 2025, 9:32 a.m.
10 Anonymous submission #123 1.3 15.6 July 11, 2025, 12:38 p.m.
BLEU and ChrF are sacreBLEU scores. Systems in bold face are your submissions. We only display the top-10 submissions per language pair. Submission validation errors denoted by -1.0 score.

Click on the column header to sort the table. Hold down the Shift key and select a second column to sort by multiple criteria.

wmtslavicllm2025-mt.en-uk test set (en-uk)

# Name BLEU chrF Date
1 Anonymous submission #158 0.0 3.1 July 22, 2025, 9:27 p.m.
2 Anonymous submission #133 0.0 0.4 July 15, 2025, 11:08 p.m.
3 Anonymous submission #159 -2.0 0.0 July 22, 2025, 10:58 p.m.
BLEU and ChrF are sacreBLEU scores. Systems in bold face are your submissions. We only display the top-10 submissions per language pair. Submission validation errors denoted by -1.0 score.

Click on the column header to sort the table. Hold down the Shift key and select a second column to sort by multiple criteria.

wmtslavicllm2025-qa.dsb test set (dsb-dsb)

# Name Accuracy Date
1 Anonymous submission #172 57.6 July 23, 2025, 8:47 a.m.
2 Anonymous submission #121 45.9 July 11, 2025, 12:37 p.m.
3 Anonymous submission #166 57.1 July 23, 2025, 8:38 a.m.
4 Anonymous submission #134 33.2 July 16, 2025, 9:02 p.m.
5 Anonymous submission #138 32.2 July 18, 2025, 12:28 a.m.
6 Anonymous submission #142 31.2 July 18, 2025, 12:31 a.m.
7 Anonymous submission #168 57.1 July 23, 2025, 8:40 a.m.
8 Anonymous submission #177 51.7 July 23, 2025, 11:07 a.m.
9 Anonymous submission #150 55.1 July 22, 2025, 7:37 a.m.
10 Anonymous submission #154 54.6 July 22, 2025, 9:08 a.m.
BLEU and ChrF are sacreBLEU scores. Systems in bold face are your submissions. We only display the top-10 submissions per language pair. Submission validation errors denoted by -1.0 score.

Click on the column header to sort the table. Hold down the Shift key and select a second column to sort by multiple criteria.

wmtslavicllm2025-qa.hsb test set (hsb-hsb)

# Name Accuracy Date
1 Anonymous submission #155 50.5 July 22, 2025, 11:31 a.m.
2 Anonymous submission #178 50.0 July 23, 2025, 11:23 a.m.
3 Anonymous submission #125 42.9 July 11, 2025, 12:40 p.m.
4 Anonymous submission #165 54.3 July 23, 2025, 8:36 a.m.
5 Anonymous submission #149 36.7 July 21, 2025, 6:14 p.m.
6 Anonymous submission #176 55.2 July 23, 2025, 11:07 a.m.
7 Anonymous submission #145 31.9 July 18, 2025, 12:34 a.m.
8 Anonymous submission #164 54.3 July 23, 2025, 8:34 a.m.
9 Anonymous submission #169 55.7 July 23, 2025, 8:41 a.m.
10 Anonymous submission #137 30.5 July 17, 2025, 5:54 p.m.
BLEU and ChrF are sacreBLEU scores. Systems in bold face are your submissions. We only display the top-10 submissions per language pair. Submission validation errors denoted by -1.0 score.

Click on the column header to sort the table. Hold down the Shift key and select a second column to sort by multiple criteria.

wmtslavicllm2025-qa.uk test set (uk-uk)

# Name Accuracy Date
1 Anonymous submission #160 35.8 July 23, 2025, 12:45 a.m.
2 Anonymous submission #126 31.2 July 11, 2025, 12:41 p.m.
BLEU and ChrF are sacreBLEU scores. Systems in bold face are your submissions. We only display the top-10 submissions per language pair. Submission validation errors denoted by -1.0 score.

Click on the column header to sort the table. Hold down the Shift key and select a second column to sort by multiple criteria.