Model performance ================= **News**: Trankit v1.0.0 is out. The new version has pretrained pipelines using XLM-Roberta Large which is much better than our previous version with XLM-Roberta Base. Checkout the performance comparison below on this page. Trankit v1.0.0 also provides a brand new `command-line interface `_ that helps users who are not familiar with Python can use Trankit more easily. Finally, the new version has a brand new `Auto Mode `_ in which a language detector is used to activate the language-specific models for processing the input, avoiding switching back and forth between languages in a multilingual pipeline. Universal Dependencies v2.5 --------------------------- The following table shows the performance comparison between `Trankit large `_ (using XLM-Roberta large) and base (using XLM-Roberta base), spaCy v2.3, UDPipe v1.2, and Stanza v1.1.1 on the test treebanks of the 5 major languages on the `Universal Dependencies v2.5 `_ corpora. Performance for each model are F1 scores obtained by using the `official evaluation script `_ of the CoNLL 2018 Shared Task. The meanings of the columns in the following tables are provided by the evaluation script as follows: * **Tokens**: how well do the gold tokens match system tokens. * **Sents.**: how well do the gold sentences match system sentences. * **Words**: how well can the gold words be aligned to system words. * **UPOS**: using aligned words, how well does UPOS match. * **XPOS**: using aligned words, how well does XPOS match. * **UFeats**: using aligned words, how well does universal FEATS match. * **AllTags**: using aligned words, how well does UPOS+XPOS+FEATS match. * **Lemmas**: using aligned words, how well does LEMMA match. * **UAS**: using aligned words, how well does HEAD match. * **LAS**: using aligned words, how well does HEAD+DEPREL(ignoring subtypes) match. * **CLAS**: using aligned words with content DEPREL, how well does HEAD+DEPREL(ignoring subtypes) match. * **MLAS**: using aligned words with content DEPREL, how well does HEAD+DEPREL(ignoring subtypes)+UPOS+UFEATS+FunctionalChildren(DEPREL+UPOS+UFEATS) match. * **BLEX**: using aligned words with content DEPREL, how well does HEAD+DEPREL(ignoring subtypes)+LEMMAS match. +----------------+----------------+--------+--------+-------+-------+-------+--------+--------+-------+-------+ | Treebank | System | Tokens | Sents. | Words | UPOS | XPOS | UFeats | Lemmas | UAS | LAS | +----------------+----------------+--------+--------+-------+-------+-------+--------+--------+-------+-------+ | | Trankit[large] | 99.95 | 96.79 | 99.39 | 96.8 | 94.93 | 94.99 | 94.92 | 89.74 | 85.98 | | +----------------+--------+--------+-------+-------+-------+--------+--------+-------+-------+ | | Trankit[base] | 99.93 | 96.59 | 99.22 | 96.31 | 94.08 | 94.28 | 94.65 | 88.39 | 84.68 | | Arabic-PADT +----------------+--------+--------+-------+-------+-------+--------+--------+-------+-------+ | | Stanza | 99.98 | 80.43 | 97.88 | 94.89 | 91.75 | 91.86 | 93.27 | 83.27 | 79.33 | | +----------------+--------+--------+-------+-------+-------+--------+--------+-------+-------+ | | UDPipe | 99.98 | 82.09 | 94.58 | 90.36 | 84.00 | 84.16 | 88.46 | 72.67 | 68.14 | +----------------+----------------+--------+--------+-------+-------+-------+--------+--------+-------+-------+ | | Trankit[large] | 97.75 | 99.4 | 97.75 | 95.13 | 94.94 | 97.28 | 97.75 | 87.38 | 84.82 | | +----------------+--------+--------+-------+-------+-------+--------+--------+-------+-------+ | | Trankit[base] | 97.01 | 99.7 | 97.01 | 94.21 | 94.02 | 96.59 | 97.01 | 85.19 | 82.54 | | Chinese-GSD +----------------+--------+--------+-------+-------+-------+--------+--------+-------+-------+ | | Stanza | 92.83 | 98.80 | 92.83 | 89.12 | 88.93 | 92.11 | 92.83 | 72.88 | 69.82 | | +----------------+--------+--------+-------+-------+-------+--------+--------+-------+-------+ | | UDPipe | 90.27 | 99.10 | 90.27 | 84.13 | 84.04 | 89.05 | 90.26 | 61.60 | 57.81 | +----------------+----------------+--------+--------+-------+-------+-------+--------+--------+-------+-------+ | | Trankit[large] | 98.67 | 90.49 | 98.67 | 96.65 | 96.47 | 96.75 | 97.25 | 91.29 | 89.4 | | +----------------+--------+--------+-------+-------+-------+--------+--------+-------+-------+ | | Trankit[base] | 98.48 | 88.35 | 98.48 | 95.95 | 95.71 | 96.26 | 96.84 | 90.14 | 87.96 | | +----------------+--------+--------+-------+-------+-------+--------+--------+-------+-------+ | English-EWT | Stanza | 99.01 | 81.13 | 99.01 | 95.40 | 95.12 | 96.11 | 97.21 | 86.22 | 83.59 | | +----------------+--------+--------+-------+-------+-------+--------+--------+-------+-------+ | | UDPipe | 98.90 | 77.40 | 98.90 | 93.26 | 92.75 | 94.23 | 95.45 | 80.22 | 77.03 | | +----------------+--------+--------+-------+-------+-------+--------+--------+-------+-------+ | | spaCy | 97.30 | 61.19 | 97.30 | 86.72 | 90.83 | - | 87.05 | - | - | +----------------+----------------+--------+--------+-------+-------+-------+--------+--------+-------+-------+ | | Trankit[large] | 99.77 | 98.8 | 99.72 | 97.89 | - | 97.34 | 97.9 | 94.97 | 93.37 | | +----------------+--------+--------+-------+-------+-------+--------+--------+-------+-------+ | | Trankit[base] | 99.7 | 96.63 | 99.66 | 97.85 | - | 97.16 | 97.80 | 94.00 | 92.34 | | +----------------+--------+--------+-------+-------+-------+--------+--------+-------+-------+ | French-GSD | Stanza | 99.68 | 94.92 | 99.48 | 97.30 | - | 96.72 | 97.64 | 91.38 | 89.05 | | +----------------+--------+--------+-------+-------+-------+--------+--------+-------+-------+ | | UDPipe | 99.68 | 93.59 | 98.81 | 95.85 | - | 95.55 | 96.61 | 87.14 | 84.26 | | +----------------+--------+--------+-------+-------+-------+--------+--------+-------+-------+ | | spaCy | 98.34 | 77.30 | 94.15 | 86.82 | - | - | 87.29 | 67.46 | 60.60 | +----------------+----------------+--------+--------+-------+-------+-------+--------+--------+-------+-------+ | | Trankit[large] | 99.91 | 99.39 | 99.91 | 99.06 | 99 | 98.85 | 99.15 | 94.77 | 93.29 | | +----------------+--------+--------+-------+-------+-------+--------+--------+-------+-------+ | | Trankit[base] | 99.94 | 99.13 | 99.93 | 99.02 | 98.94 | 98.80 | 99.17 | 94.11 | 92.41 | | +----------------+--------+--------+-------+-------+-------+--------+--------+-------+-------+ | Spanish-Ancora | Stanza | 99.98 | 99.07 | 99.98 | 98.78 | 98.67 | 98.59 | 99.19 | 92.21 | 90.01 | | +----------------+--------+--------+-------+-------+-------+--------+--------+-------+-------+ | | UDPipe | 99.97 | 98.32 | 99.95 | 98.32 | 98.13 | 98.13 | 98.48 | 88.22 | 85.10 | | +----------------+--------+--------+-------+-------+-------+--------+--------+-------+-------+ | | spaCy | 99.47 | 97.59 | 98.95 | 94.04 | - | - | 79.63 | 86.63 | 84.13 | +----------------+----------------+--------+--------+-------+-------+-------+--------+--------+-------+-------+ As can be seen from the table, both the large and base version of Trankit outperform other toolkits over different tasks (e.g., POS and morphological tagging) in which the improvement boost is substantial and significant for sentence segmentation and dependency parsing. For example, **English** enjoys a **9.36%** improvement for **sentence segmentation**, a **5.07%** and **5.81%** improvement for UAS and LAS in **dependency parsing**. For **Arabic**, Transit has a remarkable improvement of **16.36%** for **sentence segmentation** while **Chinese** observes **14.50%** and **15.0%** improvement of UAS and LAS for **dependency parsing**. Next, we show the detailed performance comparison of Trankit large (using XLM-Roberta large), Trankit base (using XLM-Roberta base), and Stanza v1.1.1 on 90 Universal Dependencies v2.5 treebanks. Over all 90 treebanks, both versions of Trankit outperform the previous state-of-the-art framework Stanza in most of the tasks, particularly for sentence segmentation (+4.28%), POS tagging (+2.00% for UPOS and +2.14% for XPOS), morphological tagging (+2.18%), and dependency parsing (+5.61% for UAS and +6.81% for LAS) while maintaining the competitive performance on tokenization, multi-word token expansion, and lemmatization. +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | | Tokens | Sentences | Words | UPOS | XPOS | UFeats | AllTags | Lemmas | UAS | LAS | CLAS | MLAS | BLEX | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Overall performance | Trankit[large] | 99.32 | 92.86 | 99.15 | 96.22 | 94.64 | 93.93 | 91.19 | 94.47 | 88.67 | 85.49 | 82.97 | 76.17 | 78.45 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 99.23 | 91.82 | 99.02 | 95.65 | 94.05 | 93.21 | 90.29 | 94.27 | 87.06 | 83.69 | 80.88 | 73.57 | 76.53 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 99.26 | 88.58 | 98.90 | 94.21 | 92.50 | 91.75 | 88.62 | 94.15 | 83.06 | 78.68 | 74.65 | 67.83 | 71.28 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[large] - Trankit[base] | 0.09 | 1.04 | 0.13 | 0.56 | 0.59 | 0.72 | 0.91 | 0.20 | 1.61 | 1.80 | 2.10 | 2.60 | 1.92 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[large] - Stanza | 0.07 | 4.28 | 0.26 | 2.00 | 2.14 | 2.18 | 2.57 | 0.32 | 5.61 | 6.81 | 8.32 | 8.34 | 7.17 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Afrikaans-AfriBooms | Trankit[large] | 99.84 | 100.00 | 99.84 | 98.87 | 96.63 | 98.68 | 96.54 | 97.47 | 91.61 | 89.35 | 85.30 | 82.64 | 81.65 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 99.72 | 100.00 | 99.72 | 98.55 | 96.10 | 98.26 | 95.89 | 97.39 | 91.03 | 88.79 | 84.46 | 81.31 | 80.91 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 99.75 | 99.65 | 99.75 | 97.56 | 94.27 | 97.03 | 94.24 | 97.48 | 87.51 | 84.45 | 78.58 | 74.70 | 75.39 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Ancient_Greek-Perseus | Trankit[large] | 99.71 | 98.70 | 99.71 | 93.97 | 87.25 | 91.66 | 86.88 | 88.52 | 83.48 | 78.56 | 73.79 | 60.72 | 61.97 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 99.66 | 98.70 | 99.66 | 92.72 | 84.89 | 89.95 | 84.30 | 88.18 | 80.95 | 75.57 | 70.26 | 55.75 | 58.84 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 99.98 | 98.85 | 99.98 | 92.54 | 85.22 | 91.06 | 84.98 | 88.26 | 78.75 | 73.35 | 67.88 | 54.22 | 57.54 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Ancient_Greek-PROIEL | Trankit[large] | 99.91 | 67.60 | 99.91 | 97.86 | 97.93 | 93.03 | 92.02 | 97.50 | 85.63 | 82.31 | 78.16 | 68.27 | 75.76 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 99.83 | 58.62 | 99.83 | 97.30 | 97.37 | 91.59 | 90.33 | 97.31 | 83.21 | 79.68 | 74.96 | 64.13 | 72.80 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 100.00 | 51.65 | 100.00 | 97.38 | 97.75 | 92.09 | 90.96 | 97.42 | 80.34 | 76.33 | 71.37 | 61.23 | 69.23 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Arabic-PADT | Trankit[large] | 99.95 | 96.79 | 99.39 | 96.80 | 94.93 | 94.99 | 94.61 | 94.92 | 89.74 | 85.98 | 83.92 | 78.47 | 79.96 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 99.93 | 96.59 | 99.22 | 96.31 | 94.08 | 94.28 | 93.69 | 94.65 | 88.39 | 84.68 | 82.35 | 76.46 | 78.46 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 99.98 | 80.43 | 97.88 | 94.89 | 91.75 | 91.86 | 91.51 | 93.27 | 83.27 | 79.33 | 76.24 | 70.58 | 72.79 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Armenian-ArmTDP | Trankit[large] | 98.52 | 99.46 | 97.26 | 95.03 | 97.26 | 90.42 | 89.56 | 93.45 | 86.47 | 83.01 | 80.91 | 72.45 | 77.07 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 98.41 | 98.92 | 97.23 | 94.36 | 97.23 | 88.89 | 87.84 | 93.28 | 84.22 | 80.14 | 77.47 | 67.92 | 73.94 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 98.96 | 95.22 | 96.58 | 92.49 | 96.58 | 88.19 | 86.94 | 92.27 | 78.18 | 72.46 | 68.50 | 60.39 | 65.88 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Basque-BDT | Trankit[large] | 99.89 | 100.00 | 99.89 | 96.95 | 99.89 | 93.96 | 92.30 | 96.69 | 90.22 | 87.47 | 86.54 | 78.10 | 83.21 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 99.81 | 99.92 | 99.81 | 96.23 | 99.81 | 92.35 | 90.40 | 96.46 | 88.24 | 85.25 | 84.29 | 74.20 | 80.92 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 100.00 | 100.00 | 100.00 | 96.23 | 100.00 | 93.09 | 91.34 | 96.52 | 86.19 | 82.76 | 81.29 | 73.56 | 78.26 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Belarusian-HSE | Trankit[large] | 99.81 | 82.28 | 99.81 | 96.46 | 31.77 | 82.82 | 27.96 | 80.37 | 77.48 | 73.36 | 70.18 | 51.71 | 52.39 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 99.53 | 84.68 | 99.53 | 91.58 | 36.18 | 78.47 | 27.19 | 79.48 | 69.40 | 65.02 | 64.51 | 46.24 | 48.41 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 99.38 | 78.24 | 99.38 | 91.92 | 31.34 | 77.73 | 26.31 | 79.48 | 69.28 | 63.88 | 58.49 | 41.88 | 44.05 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Bulgarian-BTB | Trankit[large] | 99.78 | 98.79 | 99.78 | 99.29 | 97.75 | 98.30 | 97.29 | 97.37 | 96.30 | 94.21 | 92.19 | 89.64 | 88.79 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 99.84 | 98.25 | 99.84 | 99.18 | 97.42 | 98.10 | 96.91 | 97.36 | 95.81 | 93.47 | 91.23 | 88.28 | 87.80 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 99.93 | 97.27 | 99.93 | 98.68 | 96.35 | 97.59 | 95.75 | 97.29 | 93.37 | 90.21 | 86.84 | 83.71 | 83.62 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Catalan-AnCora | Trankit[large] | 99.94 | 99.76 | 99.94 | 99.11 | 99.07 | 98.66 | 98.24 | 98.49 | 95.15 | 93.83 | 91.41 | 89.31 | 90.27 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 99.94 | 100.00 | 99.93 | 99.02 | 98.97 | 98.57 | 98.09 | 98.46 | 94.61 | 93.01 | 90.09 | 87.89 | 88.99 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 99.99 | 99.84 | 99.98 | 98.75 | 98.66 | 98.29 | 97.74 | 98.47 | 92.84 | 90.56 | 86.25 | 84.07 | 85.31 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Chinese-GSD | Trankit[large] | 97.75 | 99.40 | 97.75 | 95.13 | 94.94 | 97.28 | 94.45 | 97.75 | 87.38 | 84.82 | 83.44 | 79.82 | 83.44 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 97.01 | 99.70 | 97.01 | 94.21 | 94.02 | 96.59 | 93.56 | 97.01 | 85.19 | 82.54 | 80.91 | 77.42 | 80.91 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 92.83 | 98.80 | 92.83 | 89.12 | 88.93 | 92.11 | 88.18 | 92.83 | 72.88 | 69.82 | 66.81 | 63.26 | 66.81 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Classical_Chinese-Kyoto | Trankit[large] | 99.70 | 70.58 | 99.70 | 92.89 | 91.90 | 94.43 | 89.98 | 99.67 | 82.03 | 76.82 | 75.60 | 73.16 | 75.60 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 99.63 | 61.82 | 99.63 | 92.07 | 91.03 | 93.88 | 88.92 | 99.60 | 78.79 | 73.23 | 72.18 | 69.27 | 72.18 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 99.47 | 46.95 | 99.47 | 90.25 | 89.64 | 92.68 | 87.34 | 99.45 | 71.81 | 66.08 | 64.54 | 62.61 | 64.54 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Croatian-SET | Trankit[large] | 99.93 | 99.08 | 99.93 | 98.58 | 96.55 | 96.85 | 95.99 | 96.71 | 93.86 | 89.74 | 87.77 | 82.19 | 83.56 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 99.92 | 99.16 | 99.92 | 98.38 | 96.08 | 96.52 | 95.44 | 96.60 | 93.34 | 89.36 | 87.16 | 81.12 | 82.91 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 99.96 | 98.15 | 99.96 | 97.88 | 94.86 | 95.32 | 94.22 | 96.67 | 90.27 | 85.56 | 82.43 | 76.37 | 78.78 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Czech-CAC | Trankit[large] | 99.99 | 100.00 | 99.98 | 99.64 | 98.17 | 98.01 | 97.65 | 98.30 | 95.48 | 93.97 | 93.06 | 90.13 | 90.92 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 99.96 | 100.00 | 99.95 | 99.42 | 97.40 | 97.12 | 96.68 | 98.13 | 94.97 | 93.27 | 92.08 | 88.13 | 89.85 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 99.99 | 100.00 | 99.97 | 98.76 | 94.79 | 93.52 | 92.65 | 98.00 | 91.70 | 89.19 | 86.84 | 80.14 | 84.89 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Czech-CLTT | Trankit[large] | 99.89 | 98.89 | 99.85 | 99.17 | 93.80 | 94.02 | 93.53 | 97.81 | 92.53 | 90.76 | 88.90 | 81.07 | 86.35 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 99.82 | 100.00 | 99.76 | 98.93 | 93.36 | 93.68 | 93.21 | 97.66 | 90.20 | 88.01 | 85.13 | 77.66 | 82.80 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 99.93 | 100.00 | 99.84 | 98.92 | 91.89 | 91.97 | 91.28 | 97.48 | 86.67 | 83.38 | 79.35 | 70.70 | 77.56 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Czech-FicTree | Trankit[large] | 99.98 | 99.50 | 99.98 | 99.11 | 97.26 | 97.99 | 97.06 | 98.64 | 96.50 | 94.85 | 93.44 | 89.79 | 91.46 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 99.97 | 99.38 | 99.97 | 98.94 | 96.47 | 97.09 | 96.12 | 98.61 | 95.85 | 93.86 | 92.10 | 87.13 | 90.16 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 99.97 | 98.60 | 99.96 | 98.31 | 95.23 | 96.01 | 94.58 | 98.43 | 92.69 | 89.81 | 87.30 | 81.94 | 85.42 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Czech-PDT | Trankit[large] | 99.95 | 97.87 | 99.95 | 99.32 | 98.19 | 98.19 | 97.77 | 98.54 | 95.24 | 93.65 | 92.79 | 90.18 | 91.01 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 99.94 | 97.85 | 99.94 | 99.23 | 97.81 | 97.77 | 97.34 | 98.49 | 94.81 | 93.18 | 92.09 | 89.11 | 90.33 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 99.97 | 94.14 | 99.97 | 98.50 | 95.38 | 94.61 | 93.67 | 98.55 | 91.00 | 88.64 | 86.91 | 81.12 | 85.45 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Danish-DDT | Trankit[large] | 99.81 | 95.46 | 99.81 | 98.71 | 99.81 | 98.33 | 97.87 | 97.35 | 91.75 | 90.33 | 88.81 | 85.66 | 85.42 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 99.79 | 95.19 | 99.79 | 98.35 | 99.79 | 97.79 | 97.19 | 97.18 | 90.41 | 88.78 | 86.94 | 82.58 | 83.38 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 99.96 | 93.57 | 99.96 | 97.75 | 99.96 | 97.38 | 96.45 | 97.32 | 86.83 | 84.19 | 81.20 | 77.13 | 78.46 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Dutch-Alpino | Trankit[large] | 99.43 | 90.65 | 99.43 | 96.67 | 95.01 | 96.55 | 94.75 | 96.39 | 94.41 | 92.49 | 89.56 | 84.22 | 85.29 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 99.22 | 89.88 | 99.22 | 96.55 | 94.92 | 96.22 | 94.56 | 96.23 | 93.28 | 91.28 | 87.88 | 82.58 | 83.86 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 99.96 | 89.98 | 99.96 | 96.33 | 94.76 | 96.28 | 94.13 | 96.97 | 89.56 | 86.44 | 81.22 | 75.76 | 77.80 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Dutch-LassySmall | Trankit[large] | 99.36 | 92.60 | 99.36 | 96.52 | 95.57 | 96.63 | 94.99 | 97.37 | 92.25 | 89.68 | 86.51 | 82.56 | 84.19 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 99.21 | 91.09 | 99.21 | 96.20 | 95.18 | 96.25 | 94.53 | 97.19 | 91.09 | 88.18 | 84.63 | 80.47 | 82.44 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 99.90 | 77.95 | 99.90 | 95.97 | 94.87 | 96.22 | 94.05 | 97.59 | 85.34 | 81.93 | 75.54 | 71.98 | 73.49 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | English-EWT | Trankit[large] | 98.67 | 90.49 | 98.67 | 96.65 | 96.47 | 96.75 | 95.49 | 97.25 | 91.29 | 89.40 | 87.41 | 83.45 | 85.73 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 98.48 | 88.35 | 98.48 | 95.95 | 95.71 | 96.26 | 94.62 | 96.84 | 90.14 | 87.96 | 85.75 | 81.02 | 83.83 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 99.01 | 81.13 | 99.01 | 95.40 | 95.12 | 96.11 | 93.90 | 97.21 | 86.22 | 83.59 | 80.21 | 76.02 | 78.50 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | English-GUM | Trankit[large] | 99.52 | 91.60 | 99.52 | 96.66 | 96.51 | 97.47 | 95.77 | 96.63 | 91.61 | 89.09 | 85.58 | 81.29 | 81.80 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 99.45 | 91.63 | 99.45 | 96.39 | 96.24 | 97.19 | 95.46 | 96.55 | 91.04 | 88.43 | 84.80 | 80.19 | 80.81 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 99.82 | 86.35 | 99.82 | 95.89 | 95.91 | 96.87 | 94.99 | 96.80 | 87.06 | 83.57 | 78.42 | 74.68 | 74.97 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | English-LinES | Trankit[large] | 99.46 | 91.87 | 99.46 | 97.31 | 95.91 | 96.96 | 93.91 | 97.98 | 89.99 | 87.00 | 85.67 | 80.34 | 83.77 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 99.53 | 93.01 | 99.53 | 97.14 | 95.54 | 96.67 | 93.17 | 98.01 | 89.39 | 86.32 | 84.75 | 78.96 | 82.78 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 99.95 | 88.49 | 99.95 | 96.88 | 95.18 | 96.76 | 93.11 | 98.32 | 85.82 | 81.97 | 79.04 | 74.47 | 77.31 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | English-ParTUT | Trankit[large] | 99.71 | 100.00 | 99.65 | 96.86 | 96.65 | 95.77 | 94.63 | 97.71 | 93.51 | 91.10 | 87.37 | 81.25 | 85.06 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 99.66 | 100.00 | 99.60 | 96.79 | 96.55 | 95.94 | 94.67 | 97.64 | 93.15 | 90.95 | 87.21 | 81.37 | 84.96 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 99.68 | 100.00 | 99.59 | 96.15 | 95.83 | 95.21 | 93.92 | 97.45 | 90.31 | 87.35 | 82.56 | 76.19 | 80.53 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Estonian-EDT | Trankit[large] | 99.75 | 96.58 | 99.75 | 97.87 | 98.35 | 97.10 | 96.04 | 96.09 | 91.71 | 89.52 | 88.57 | 84.68 | 84.10 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 99.72 | 96.55 | 99.72 | 97.53 | 98.13 | 96.56 | 95.37 | 95.98 | 90.65 | 88.31 | 87.15 | 82.82 | 82.81 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 99.96 | 93.32 | 99.96 | 97.19 | 98.04 | 95.77 | 94.43 | 96.05 | 86.68 | 83.82 | 82.41 | 77.63 | 78.32 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Estonian-EWT | Trankit[large] | 97.76 | 82.58 | 97.76 | 94.26 | 94.93 | 91.49 | 89.91 | 85.71 | 82.18 | 78.49 | 76.41 | 69.32 | 64.00 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 96.96 | 83.72 | 96.96 | 92.07 | 93.16 | 89.17 | 86.89 | 84.65 | 78.21 | 73.79 | 71.59 | 62.91 | 59.90 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 99.20 | 67.14 | 99.20 | 88.86 | 91.70 | 87.16 | 83.43 | 85.62 | 67.23 | 60.07 | 56.21 | 48.32 | 47.38 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Finnish-FTB | Trankit[large] | 99.84 | 97.36 | 99.83 | 98.32 | 97.29 | 98.09 | 96.87 | 96.94 | 95.84 | 94.53 | 93.54 | 90.68 | 90.39 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 99.75 | 95.83 | 99.74 | 97.46 | 96.23 | 97.22 | 95.61 | 96.58 | 94.17 | 92.43 | 90.84 | 87.09 | 87.79 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 100.00 | 89.59 | 99.97 | 95.50 | 95.12 | 96.51 | 93.92 | 96.16 | 89.09 | 86.39 | 83.80 | 79.90 | 81.02 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Finnish-TDT | Trankit[large] | 99.71 | 97.22 | 99.72 | 98.48 | 98.78 | 96.84 | 96.33 | 95.59 | 94.98 | 93.77 | 92.92 | 88.97 | 88.02 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 99.62 | 95.98 | 99.62 | 97.99 | 98.44 | 96.52 | 95.76 | 95.39 | 93.47 | 91.94 | 90.78 | 86.55 | 86.00 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 99.77 | 93.05 | 99.73 | 96.97 | 97.72 | 95.36 | 94.44 | 94.98 | 88.62 | 86.18 | 84.66 | 79.73 | 80.24 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | French-GSD | Trankit[large] | 99.77 | 98.80 | 99.72 | 97.89 | 99.72 | 97.34 | 96.66 | 97.90 | 94.97 | 93.37 | 90.17 | 86.35 | 87.73 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 99.70 | 96.63 | 99.66 | 97.85 | 99.66 | 97.16 | 96.60 | 97.80 | 94.00 | 92.34 | 88.66 | 84.76 | 86.08 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 99.68 | 94.92 | 99.48 | 97.30 | 99.47 | 96.72 | 96.05 | 97.64 | 91.38 | 89.05 | 84.38 | 80.30 | 82.40 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | French-ParTUT | Trankit[large] | 99.76 | 98.63 | 99.65 | 97.66 | 97.35 | 94.55 | 93.82 | 96.09 | 95.05 | 93.32 | 90.65 | 81.49 | 84.88 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 99.74 | 98.63 | 99.69 | 97.77 | 97.54 | 94.20 | 93.66 | 96.01 | 94.20 | 92.67 | 89.26 | 78.71 | 83.56 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 99.82 | 100.00 | 99.37 | 96.60 | 96.37 | 93.98 | 93.41 | 95.48 | 90.71 | 88.37 | 83.37 | 74.41 | 77.88 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | French-Sequoia | Trankit[large] | 99.81 | 94.07 | 99.78 | 99.22 | 99.78 | 98.43 | 98.13 | 98.64 | 95.70 | 94.85 | 92.95 | 90.47 | 91.15 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 99.73 | 94.36 | 99.73 | 98.90 | 99.73 | 97.98 | 97.57 | 98.47 | 94.68 | 93.59 | 91.26 | 88.27 | 89.44 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 99.90 | 88.79 | 99.58 | 98.19 | 99.58 | 97.58 | 96.94 | 98.25 | 90.47 | 88.34 | 84.71 | 81.77 | 83.31 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | French-Spoken | Trankit[large] | 99.36 | 53.06 | 99.19 | 96.80 | 96.91 | 99.19 | 94.66 | 96.34 | 85.70 | 81.84 | 76.29 | 73.65 | 73.78 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 99.38 | 39.39 | 99.18 | 96.73 | 96.73 | 99.18 | 94.43 | 96.38 | 82.40 | 78.35 | 71.68 | 69.01 | 69.49 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 100.00 | 22.09 | 99.45 | 95.49 | 97.06 | 99.45 | 93.23 | 96.53 | 75.82 | 70.71 | 62.13 | 59.57 | 60.44 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Galician-CTG | Trankit[large] | 99.76 | 98.44 | 99.31 | 97.30 | 97.05 | 99.17 | 96.77 | 98.07 | 85.70 | 83.14 | 78.24 | 72.41 | 76.90 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 99.76 | 98.09 | 99.38 | 97.17 | 96.83 | 99.23 | 96.54 | 98.06 | 85.51 | 82.81 | 77.50 | 71.49 | 76.20 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 99.89 | 99.13 | 99.32 | 97.21 | 96.99 | 99.14 | 96.71 | 97.94 | 85.22 | 82.66 | 77.24 | 71.13 | 75.96 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Galician-TreeGal | Trankit[large] | 99.47 | 95.52 | 99.22 | 97.62 | 95.68 | 96.50 | 94.97 | 91.08 | 87.17 | 83.90 | 80.31 | 74.67 | 68.39 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 99.47 | 94.60 | 99.06 | 97.06 | 94.90 | 95.89 | 94.08 | 90.91 | 85.38 | 81.96 | 77.96 | 71.57 | 66.32 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 99.59 | 89.17 | 98.41 | 94.29 | 91.81 | 93.36 | 90.88 | 94.39 | 78.04 | 72.94 | 65.61 | 59.06 | 61.49 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | German-GSD | Trankit[large] | 99.71 | 89.72 | 99.72 | 95.23 | 97.68 | 91.68 | 87.21 | 96.58 | 89.01 | 85.20 | 81.49 | 65.82 | 77.20 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 99.75 | 92.72 | 99.75 | 95.04 | 97.57 | 91.51 | 86.86 | 96.60 | 88.73 | 84.77 | 80.78 | 64.76 | 76.58 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 99.53 | 85.79 | 99.53 | 94.07 | 96.98 | 89.52 | 84.51 | 96.37 | 85.39 | 80.61 | 75.38 | 58.57 | 71.24 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | German-HDT | Trankit[large] | 99.92 | 99.67 | 99.92 | 98.44 | 98.41 | 94.05 | 93.70 | 97.36 | 97.63 | 96.86 | 95.14 | 85.73 | 91.67 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 99.90 | 99.50 | 99.90 | 98.42 | 98.37 | 93.95 | 93.52 | 97.35 | 97.38 | 96.51 | 94.63 | 85.02 | 91.18 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 100.00 | 97.41 | 100.00 | 98.04 | 97.94 | 91.77 | 91.34 | 97.48 | 94.91 | 92.59 | 88.73 | 77.26 | 85.63 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Greek-GDT | Trankit[large] | 99.85 | 93.50 | 99.85 | 98.41 | 98.41 | 96.34 | 95.84 | 96.73 | 95.25 | 93.87 | 91.74 | 85.40 | 86.54 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 99.75 | 93.57 | 99.75 | 98.04 | 98.04 | 95.41 | 94.73 | 96.55 | 94.16 | 92.80 | 89.84 | 82.39 | 84.83 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 99.88 | 93.18 | 99.89 | 97.84 | 97.84 | 94.94 | 94.33 | 96.49 | 91.12 | 88.78 | 84.12 | 78.00 | 79.48 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Hebrew-HTB | Trankit[large] | 99.81 | 99.69 | 96.31 | 94.32 | 94.32 | 93.03 | 92.39 | 93.48 | 88.41 | 86.04 | 82.23 | 74.92 | 78.35 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 99.79 | 100.00 | 96.03 | 93.75 | 93.75 | 91.96 | 91.24 | 93.21 | 87.02 | 84.55 | 80.34 | 72.38 | 76.52 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 99.98 | 99.69 | 93.19 | 90.46 | 90.46 | 89.24 | 88.45 | 90.27 | 79.18 | 76.60 | 71.05 | 64.51 | 67.79 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Hindi-HDTB | Trankit[large] | 99.88 | 99.91 | 99.88 | 98.01 | 97.70 | 93.91 | 92.38 | 96.54 | 95.95 | 92.96 | 89.79 | 79.69 | 88.47 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 99.89 | 99.64 | 99.89 | 97.77 | 97.38 | 94.03 | 92.33 | 96.54 | 95.68 | 92.70 | 89.59 | 79.60 | 88.28 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 100.00 | 99.44 | 100.00 | 97.59 | 97.08 | 94.03 | 92.11 | 96.66 | 94.80 | 91.74 | 88.20 | 78.73 | 87.01 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Hungarian-Szeged | Trankit[large] | 99.59 | 99.33 | 99.59 | 97.49 | 99.59 | 95.23 | 94.40 | 94.45 | 91.31 | 87.78 | 86.83 | 78.95 | 80.31 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 99.41 | 98.00 | 99.41 | 96.97 | 99.41 | 94.47 | 93.47 | 94.28 | 89.43 | 85.70 | 85.08 | 76.13 | 78.73 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 99.87 | 97.00 | 99.87 | 96.03 | 99.87 | 93.76 | 92.94 | 94.25 | 83.62 | 78.86 | 77.14 | 69.46 | 71.87 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Indonesian-GSD | Trankit[large] | 99.89 | 95.54 | 99.89 | 93.39 | 95.06 | 96.11 | 89.22 | 99.53 | 86.33 | 79.81 | 77.99 | 69.08 | 77.60 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 99.86 | 95.37 | 99.86 | 93.57 | 94.18 | 95.67 | 88.65 | 99.49 | 86.55 | 80.28 | 78.64 | 69.42 | 78.26 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 99.99 | 93.78 | 99.99 | 93.68 | 94.79 | 96.00 | 89.17 | 99.61 | 85.17 | 79.19 | 77.04 | 68.86 | 76.68 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Irish-IDT | Trankit[large] | 99.47 | 98.24 | 99.47 | 94.72 | 93.74 | 80.90 | 77.94 | 92.64 | 83.47 | 76.86 | 70.64 | 48.55 | 64.06 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 99.32 | 97.25 | 99.32 | 93.88 | 92.46 | 80.36 | 76.72 | 92.34 | 82.52 | 74.91 | 67.96 | 46.29 | 61.34 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 99.76 | 95.93 | 99.76 | 93.90 | 92.43 | 78.19 | 75.00 | 91.79 | 82.65 | 74.03 | 66.11 | 42.98 | 59.09 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Italian-ISDT | Trankit[large] | 99.88 | 99.07 | 99.86 | 98.72 | 98.63 | 98.32 | 97.79 | 98.33 | 95.73 | 94.45 | 91.97 | 89.08 | 89.45 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 99.88 | 98.76 | 99.87 | 98.58 | 98.46 | 98.20 | 97.60 | 98.23 | 95.31 | 93.87 | 90.93 | 87.81 | 88.45 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 99.91 | 98.76 | 99.76 | 98.01 | 97.91 | 97.72 | 97.11 | 98.10 | 92.79 | 90.84 | 86.43 | 83.60 | 84.23 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Italian-ParTUT | Trankit[large] | 99.81 | 100.00 | 99.79 | 98.58 | 98.42 | 98.15 | 97.54 | 97.84 | 96.19 | 94.11 | 90.65 | 87.75 | 87.86 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 99.82 | 100.00 | 99.81 | 98.63 | 98.41 | 98.16 | 97.47 | 97.94 | 95.38 | 93.32 | 89.28 | 86.22 | 86.55 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 99.81 | 100.00 | 99.77 | 97.82 | 97.76 | 97.79 | 96.94 | 97.57 | 92.24 | 90.01 | 84.39 | 81.77 | 82.05 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Italian-PoSTWITA | Trankit[large] | 99.34 | 73.95 | 99.18 | 96.60 | 96.43 | 96.52 | 95.31 | 96.41 | 86.33 | 82.54 | 78.49 | 74.27 | 75.83 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 99.29 | 69.95 | 99.07 | 96.10 | 95.91 | 95.87 | 94.53 | 96.30 | 84.19 | 80.32 | 75.33 | 71.09 | 72.98 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 99.71 | 63.70 | 99.46 | 96.19 | 96.04 | 96.28 | 95.01 | 96.70 | 82.67 | 78.27 | 72.20 | 68.55 | 70.35 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Italian-TWITTIRO | Trankit[large] | 99.15 | 65.72 | 98.89 | 95.47 | 94.90 | 94.09 | 91.98 | 93.12 | 84.73 | 79.86 | 73.94 | 66.46 | 66.58 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 99.22 | 56.00 | 99.01 | 95.31 | 94.74 | 93.83 | 91.68 | 92.96 | 83.44 | 78.30 | 70.79 | 63.25 | 63.81 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 99.34 | 52.40 | 98.76 | 94.41 | 94.01 | 93.34 | 91.45 | 93.17 | 78.87 | 72.85 | 64.64 | 58.67 | 59.35 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Italian-VIT | Trankit[large] | 99.97 | 98.18 | 99.84 | 98.07 | 97.29 | 97.76 | 96.16 | 98.42 | 93.02 | 90.44 | 86.85 | 82.53 | 84.91 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 99.99 | 96.52 | 99.81 | 97.82 | 97.02 | 97.39 | 95.74 | 98.31 | 92.39 | 89.60 | 85.59 | 80.70 | 83.64 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 99.98 | 94.92 | 99.49 | 97.21 | 96.23 | 96.79 | 94.99 | 98.01 | 89.32 | 85.87 | 80.26 | 76.16 | 78.61 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Japanese-GSD | Trankit[large] | 95.25 | 95.88 | 95.25 | 93.66 | 93.47 | 95.23 | 93.44 | 94.68 | 86.67 | 85.56 | 78.00 | 76.02 | 77.62 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 94.57 | 95.49 | 94.57 | 92.86 | 92.44 | 94.56 | 92.42 | 93.99 | 84.58 | 83.38 | 75.60 | 73.67 | 75.14 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 92.67 | 94.57 | 92.67 | 91.16 | 90.84 | 92.66 | 90.84 | 92.02 | 81.20 | 80.16 | 71.39 | 69.85 | 71.01 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Kazakh-KTB | Trankit[large] | 95.98 | 81.71 | 95.37 | 77.94 | 77.47 | 63.01 | 55.46 | 50.60 | 47.46 | 37.98 | 36.01 | 19.00 | 12.05 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 94.48 | 90.00 | 93.62 | 75.94 | 75.67 | 62.28 | 54.51 | 49.76 | 46.42 | 36.84 | 34.72 | 18.65 | 11.81 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 93.46 | 88.56 | 94.16 | 56.23 | 56.10 | 42.73 | 36.96 | 52.12 | 44.33 | 25.21 | 20.28 | 7.63 | 10.01 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Korean-GSD | Trankit[large] | 98.57 | 98.08 | 98.57 | 95.71 | 90.88 | 98.35 | 88.90 | 91.93 | 89.87 | 87.22 | 85.97 | 83.63 | 79.65 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 98.63 | 97.67 | 98.63 | 95.63 | 90.32 | 98.43 | 88.26 | 91.96 | 88.48 | 85.77 | 84.26 | 81.98 | 78.08 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 99.88 | 96.65 | 99.88 | 96.18 | 90.14 | 99.66 | 88.00 | 92.69 | 87.29 | 83.53 | 81.34 | 79.29 | 75.31 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Korean-Kaist | Trankit[large] | 98.70 | 99.87 | 98.70 | 95.13 | 88.07 | 98.70 | 88.07 | 92.36 | 90.00 | 88.22 | 86.37 | 83.56 | 80.16 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 98.79 | 99.14 | 98.79 | 94.99 | 87.62 | 98.79 | 87.62 | 92.44 | 88.72 | 86.96 | 84.99 | 81.84 | 78.90 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 100.00 | 99.93 | 100.00 | 95.45 | 86.31 | 100.00 | 86.31 | 93.02 | 88.41 | 86.38 | 83.95 | 80.63 | 77.57 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Kurmanji-MG | Trankit[large] | 94.95 | 91.50 | 94.63 | 75.07 | 74.16 | 57.15 | 52.27 | 57.63 | 37.12 | 29.89 | 25.57 | 9.04 | 10.16 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 94.52 | 80.56 | 94.20 | 74.33 | 73.44 | 56.54 | 51.38 | 57.61 | 35.65 | 28.58 | 25.35 | 8.88 | 10.76 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 94.81 | 87.43 | 94.49 | 57.17 | 55.91 | 43.02 | 38.41 | 56.13 | 32.01 | 21.91 | 16.35 | 3.84 | 5.84 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Latin-ITTB | Trankit[large] | 100.00 | 94.54 | 100.00 | 98.97 | 97.29 | 97.98 | 96.41 | 99.13 | 93.25 | 91.87 | 90.75 | 87.86 | 90.00 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 100.00 | 94.57 | 100.00 | 98.76 | 96.74 | 97.54 | 95.68 | 99.07 | 92.42 | 90.91 | 89.45 | 86.12 | 88.71 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 99.99 | 80.66 | 99.99 | 98.09 | 95.38 | 96.43 | 93.80 | 98.90 | 87.61 | 85.36 | 84.23 | 80.28 | 83.60 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Latin-Perseus | Trankit[large] | 99.60 | 97.93 | 99.60 | 92.84 | 83.33 | 86.79 | 82.33 | 70.34 | 83.50 | 76.76 | 73.58 | 60.70 | 44.41 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 99.45 | 97.87 | 99.45 | 90.15 | 77.12 | 81.12 | 75.64 | 69.95 | 78.01 | 69.58 | 65.24 | 49.58 | 40.23 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 100.00 | 98.24 | 100.00 | 90.63 | 78.42 | 82.42 | 77.74 | 83.08 | 71.94 | 61.99 | 57.89 | 45.28 | 47.28 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Latin-PROIEL | Trankit[large] | 99.85 | 66.10 | 99.85 | 97.79 | 97.75 | 93.22 | 92.53 | 97.21 | 86.43 | 83.33 | 81.62 | 73.62 | 79.55 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 99.82 | 58.16 | 99.82 | 96.80 | 96.83 | 91.28 | 90.27 | 96.88 | 82.23 | 78.58 | 76.36 | 67.10 | 74.43 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 100.00 | 43.04 | 100.00 | 96.92 | 97.10 | 91.24 | 90.32 | 96.78 | 76.55 | 72.37 | 70.06 | 61.28 | 68.19 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Latvian-LVTB | Trankit[large] | 99.73 | 98.69 | 99.73 | 97.61 | 91.22 | 95.18 | 90.72 | 95.83 | 93.63 | 91.25 | 89.78 | 82.69 | 85.58 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 99.71 | 99.10 | 99.71 | 97.16 | 90.24 | 94.47 | 89.62 | 95.61 | 92.05 | 89.44 | 87.73 | 79.78 | 83.52 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 99.82 | 99.01 | 99.82 | 96.03 | 88.25 | 93.46 | 87.73 | 95.55 | 87.84 | 84.44 | 82.16 | 73.91 | 78.25 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Lithuanian-ALKSNIS | Trankit[large] | 99.84 | 95.72 | 99.84 | 97.45 | 93.98 | 94.46 | 93.30 | 94.30 | 90.48 | 87.67 | 86.66 | 79.86 | 80.20 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 99.82 | 95.10 | 99.82 | 97.03 | 92.35 | 93.00 | 91.54 | 94.05 | 88.30 | 84.96 | 83.59 | 75.11 | 77.35 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 99.87 | 88.79 | 99.87 | 93.37 | 85.67 | 87.84 | 84.84 | 92.51 | 78.54 | 73.11 | 70.66 | 60.81 | 65.53 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Lithuanian-HSE | Trankit[large] | 97.71 | 100.00 | 97.71 | 90.59 | 89.85 | 79.64 | 75.90 | 80.02 | 71.41 | 62.05 | 59.15 | 41.13 | 44.82 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 98.22 | 94.55 | 98.22 | 90.46 | 89.71 | 77.92 | 74.18 | 80.07 | 66.70 | 58.47 | 55.18 | 36.60 | 40.03 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 97.53 | 51.11 | 97.53 | 81.08 | 80.04 | 70.72 | 66.44 | 76.90 | 48.10 | 37.45 | 32.37 | 21.10 | 24.86 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Marathi-UFAL | Trankit[large] | 99.20 | 69.31 | 97.22 | 87.79 | 97.22 | 70.62 | 67.47 | 81.50 | 72.79 | 63.36 | 59.67 | 36.63 | 46.50 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 99.20 | 60.87 | 95.25 | 82.83 | 95.25 | 69.43 | 66.02 | 79.17 | 60.90 | 54.08 | 52.19 | 28.81 | 40.50 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 98.00 | 76.40 | 92.25 | 77.24 | 92.25 | 60.27 | 58.55 | 75.77 | 66.42 | 52.64 | 42.80 | 24.15 | 33.90 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Norwegian_Nynorsk-Nynorsk | Trankit[large] | 99.84 | 98.97 | 99.84 | 98.52 | 99.84 | 97.79 | 97.13 | 98.01 | 95.23 | 93.82 | 92.35 | 89.01 | 89.79 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 99.81 | 98.71 | 99.81 | 98.20 | 99.81 | 97.20 | 96.48 | 97.89 | 94.15 | 92.58 | 90.70 | 86.61 | 88.15 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 99.97 | 94.85 | 99.97 | 97.92 | 99.97 | 96.88 | 96.03 | 97.90 | 91.87 | 89.73 | 87.28 | 82.86 | 84.78 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Norwegian_Nynorsk-NynorskLIA | Trankit[large] | 99.76 | 99.53 | 99.76 | 96.48 | 99.76 | 95.59 | 93.57 | 97.49 | 81.96 | 77.85 | 73.57 | 67.19 | 71.00 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 99.74 | 99.53 | 99.74 | 96.31 | 99.74 | 95.41 | 93.29 | 97.50 | 80.86 | 76.44 | 71.96 | 65.82 | 69.71 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 100.00 | 99.69 | 100.00 | 95.92 | 100.00 | 94.82 | 92.70 | 97.72 | 77.82 | 72.94 | 67.56 | 61.32 | 65.54 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Norwegian-Bokmaal | Trankit[large] | 99.88 | 98.89 | 99.88 | 98.85 | 99.88 | 98.07 | 97.61 | 98.40 | 95.54 | 94.33 | 92.82 | 90.15 | 90.84 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 99.88 | 99.20 | 99.88 | 98.66 | 99.88 | 97.60 | 97.02 | 98.34 | 94.78 | 93.47 | 91.77 | 88.29 | 89.72 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 99.99 | 97.17 | 99.99 | 98.29 | 99.99 | 97.17 | 96.41 | 98.36 | 92.57 | 90.69 | 88.32 | 84.41 | 86.33 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Old_French-SRCMF | Trankit[large] | 99.91 | 100.00 | 99.91 | 96.96 | 96.83 | 98.32 | 96.45 | 99.91 | 94.30 | 90.75 | 88.69 | 85.43 | 88.69 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 99.84 | 100.00 | 99.84 | 96.36 | 96.21 | 97.75 | 95.72 | 99.84 | 92.82 | 88.76 | 86.12 | 82.63 | 86.12 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 100.00 | 100.00 | 100.00 | 96.05 | 96.09 | 97.74 | 95.56 | 100.00 | 91.38 | 86.35 | 83.39 | 80.05 | 83.39 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Old_Russian-TOROT | Trankit[large] | 98.87 | 51.91 | 98.87 | 94.70 | 94.63 | 89.61 | 88.02 | 90.87 | 78.64 | 74.60 | 71.82 | 63.30 | 66.49 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 98.44 | 42.22 | 98.44 | 92.63 | 92.66 | 86.75 | 84.52 | 90.00 | 74.14 | 68.92 | 65.57 | 55.81 | 60.56 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 100.00 | 35.69 | 100.00 | 93.63 | 93.83 | 86.76 | 84.80 | 91.35 | 72.94 | 67.00 | 63.60 | 54.13 | 59.18 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Persian-Seraji | Trankit[large] | 99.26 | 99.25 | 99.20 | 97.78 | 97.67 | 97.70 | 97.35 | 97.35 | 92.24 | 89.58 | 86.86 | 84.97 | 84.90 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 99.22 | 99.25 | 99.11 | 97.35 | 97.24 | 97.36 | 96.90 | 97.29 | 91.38 | 88.68 | 85.92 | 83.86 | 84.08 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 100.00 | 99.25 | 99.65 | 97.29 | 97.30 | 97.37 | 96.86 | 97.73 | 89.45 | 86.06 | 82.78 | 81.00 | 81.08 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Polish-LFG | Trankit[large] | 98.34 | 99.57 | 98.34 | 97.84 | 95.52 | 96.00 | 95.05 | 95.48 | 93.90 | 93.04 | 92.58 | 89.18 | 88.61 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 98.32 | 99.91 | 98.32 | 97.66 | 94.59 | 95.05 | 94.00 | 95.37 | 93.31 | 92.17 | 91.43 | 86.88 | 87.55 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 99.95 | 99.83 | 99.95 | 98.55 | 94.66 | 95.84 | 94.07 | 96.86 | 95.80 | 93.94 | 92.35 | 87.62 | 88.64 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Polish-PDB | Trankit[large] | 99.93 | 98.71 | 99.92 | 99.16 | 96.92 | 97.11 | 96.54 | 97.56 | 96.43 | 94.88 | 93.77 | 89.78 | 90.53 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 99.91 | 98.53 | 99.89 | 99.06 | 96.29 | 96.44 | 95.77 | 97.52 | 95.52 | 93.86 | 92.50 | 87.67 | 89.34 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 99.87 | 98.39 | 99.83 | 98.31 | 94.04 | 94.27 | 93.13 | 97.29 | 92.68 | 90.40 | 88.35 | 81.69 | 85.42 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Portuguese-Bosque | Trankit[large] | 99.75 | 97.18 | 99.67 | 97.52 | 99.67 | 96.50 | 95.17 | 97.97 | 93.31 | 90.91 | 87.75 | 81.09 | 85.36 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 99.70 | 97.48 | 99.65 | 97.27 | 99.65 | 96.50 | 94.95 | 97.89 | 92.76 | 90.25 | 86.96 | 80.03 | 84.52 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 99.77 | 94.30 | 99.67 | 97.04 | 99.67 | 96.36 | 94.91 | 97.80 | 90.67 | 87.57 | 82.59 | 76.78 | 80.30 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Portuguese-GSD | Trankit[large] | 99.81 | 97.10 | 99.72 | 98.43 | 98.43 | 99.61 | 98.41 | 99.20 | 95.35 | 94.37 | 92.23 | 90.46 | 91.47 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 99.82 | 96.76 | 99.71 | 98.30 | 98.30 | 99.61 | 98.28 | 99.19 | 94.92 | 93.95 | 91.65 | 89.58 | 90.89 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 99.96 | 98.00 | 99.87 | 98.18 | 98.18 | 99.79 | 98.17 | 95.83 | 92.83 | 91.36 | 87.44 | 85.87 | 86.75 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Romanian-Nonstandard | Trankit[large] | 98.74 | 98.00 | 98.74 | 96.23 | 91.64 | 90.51 | 89.07 | 94.66 | 90.68 | 86.89 | 83.13 | 70.10 | 78.50 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 98.68 | 98.57 | 98.68 | 96.04 | 91.48 | 90.33 | 88.89 | 94.57 | 90.14 | 86.40 | 82.40 | 69.46 | 77.93 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 98.96 | 97.53 | 98.96 | 95.40 | 90.73 | 89.79 | 88.19 | 94.63 | 87.24 | 82.71 | 77.60 | 65.24 | 73.52 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Romanian-RRT | Trankit[large] | 99.60 | 98.49 | 99.60 | 97.90 | 97.35 | 97.43 | 97.11 | 97.98 | 93.60 | 89.50 | 86.61 | 82.78 | 84.60 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 99.72 | 97.67 | 99.72 | 97.87 | 97.25 | 97.44 | 97.01 | 98.05 | 93.14 | 89.04 | 85.93 | 82.02 | 84.01 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 99.77 | 96.64 | 99.77 | 97.54 | 96.97 | 97.13 | 96.75 | 97.95 | 90.66 | 85.85 | 81.49 | 77.94 | 79.84 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Russian-GSD | Trankit[large] | 99.79 | 99.25 | 99.79 | 98.25 | 97.89 | 95.78 | 94.73 | 95.75 | 92.70 | 89.75 | 88.71 | 82.92 | 83.66 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 99.63 | 98.25 | 99.63 | 97.96 | 97.65 | 94.86 | 93.83 | 95.50 | 91.86 | 88.62 | 87.41 | 80.83 | 82.36 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 99.65 | 97.16 | 99.65 | 97.38 | 97.18 | 93.11 | 92.22 | 95.34 | 88.97 | 84.83 | 82.37 | 75.16 | 77.75 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Russian-SynTagRus | Trankit[large] | 99.71 | 99.45 | 99.71 | 99.06 | 99.71 | 98.20 | 97.99 | 97.99 | 95.66 | 94.65 | 93.78 | 91.68 | 91.51 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 99.71 | 99.14 | 99.71 | 98.94 | 99.71 | 97.85 | 97.59 | 97.89 | 95.19 | 94.08 | 93.13 | 90.59 | 90.77 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 99.57 | 98.86 | 99.57 | 98.20 | 99.57 | 95.91 | 95.59 | 97.51 | 92.38 | 90.60 | 89.01 | 85.04 | 86.78 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Russian-Taiga | Trankit[large] | 98.90 | 92.32 | 98.90 | 96.16 | 96.62 | 91.02 | 87.69 | 91.90 | 85.25 | 81.73 | 80.06 | 68.48 | 71.59 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 98.77 | 92.60 | 98.77 | 95.50 | 97.27 | 89.42 | 86.58 | 91.46 | 83.08 | 79.15 | 76.91 | 64.25 | 68.53 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 97.11 | 85.79 | 97.11 | 92.25 | 94.70 | 85.76 | 82.61 | 89.28 | 72.09 | 66.00 | 61.80 | 51.94 | 55.64 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Scottish_Gaelic-ARCOSG | Trankit[large] | 99.43 | 57.46 | 99.42 | 94.70 | 88.65 | 90.71 | 87.13 | 95.48 | 81.60 | 76.46 | 70.62 | 61.71 | 66.95 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 99.26 | 54.10 | 99.25 | 92.98 | 85.47 | 88.25 | 83.78 | 95.06 | 79.48 | 73.09 | 66.41 | 56.27 | 62.83 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 99.48 | 55.35 | 99.47 | 92.50 | 84.89 | 87.99 | 83.93 | 95.51 | 77.90 | 70.81 | 62.63 | 54.00 | 59.74 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Serbian-SET | Trankit[large] | 99.91 | 100.00 | 99.91 | 99.06 | 96.22 | 96.40 | 95.84 | 96.90 | 95.57 | 93.13 | 91.92 | 85.76 | 87.71 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 99.91 | 99.71 | 99.91 | 98.97 | 95.82 | 95.96 | 95.32 | 96.90 | 95.24 | 92.94 | 91.53 | 84.84 | 87.46 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 100.00 | 99.33 | 100.00 | 98.44 | 94.26 | 94.55 | 93.86 | 96.34 | 91.79 | 88.78 | 86.50 | 79.48 | 82.38 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Simplified_Chinese-GSDSimp | Trankit[large] | 97.66 | 98.20 | 97.66 | 94.99 | 94.78 | 97.24 | 94.32 | 97.66 | 87.01 | 84.63 | 83.23 | 79.84 | 83.23 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 96.94 | 99.70 | 96.94 | 94.17 | 93.98 | 96.51 | 93.52 | 96.94 | 84.64 | 81.96 | 80.14 | 76.30 | 80.14 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 92.92 | 99.10 | 92.92 | 89.05 | 88.84 | 92.12 | 88.03 | 92.92 | 73.44 | 70.44 | 67.69 | 64.07 | 67.69 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Slovak-SNK | Trankit[large] | 99.94 | 98.49 | 99.94 | 97.90 | 90.82 | 95.22 | 90.20 | 94.79 | 96.97 | 95.60 | 95.04 | 87.64 | 88.39 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 99.93 | 98.07 | 99.93 | 97.80 | 89.02 | 94.00 | 88.38 | 94.66 | 95.72 | 93.97 | 93.19 | 84.33 | 86.63 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 99.97 | 90.93 | 99.97 | 96.34 | 87.15 | 91.59 | 86.34 | 94.73 | 89.96 | 86.82 | 84.74 | 75.39 | 79.35 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Slovenian-SSJ | Trankit[large] | 99.97 | 100.00 | 99.97 | 99.24 | 97.83 | 97.99 | 97.60 | 97.55 | 96.91 | 96.06 | 94.88 | 91.78 | 91.37 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 99.93 | 99.81 | 99.93 | 99.03 | 96.70 | 96.97 | 96.38 | 97.49 | 95.94 | 94.99 | 93.53 | 89.09 | 90.12 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 99.91 | 91.60 | 99.91 | 98.29 | 95.08 | 95.37 | 94.56 | 97.34 | 91.63 | 89.60 | 87.18 | 82.35 | 84.37 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Slovenian-SST | Trankit[large] | 99.84 | 33.03 | 99.84 | 95.81 | 91.74 | 91.86 | 89.70 | 88.72 | 74.86 | 71.02 | 66.85 | 59.58 | 56.40 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 99.79 | 31.96 | 99.79 | 94.90 | 90.27 | 90.37 | 87.92 | 88.66 | 71.15 | 66.65 | 61.94 | 54.19 | 52.26 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 100.00 | 26.59 | 100.00 | 93.66 | 88.09 | 88.06 | 85.27 | 94.78 | 63.13 | 56.50 | 51.34 | 44.81 | 48.96 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Spanish-AnCora | Trankit[large] | 99.91 | 99.39 | 99.91 | 99.06 | 99.00 | 98.85 | 98.36 | 99.15 | 94.77 | 93.29 | 91.10 | 89.23 | 90.14 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 99.94 | 99.13 | 99.93 | 99.02 | 98.94 | 98.80 | 98.27 | 99.17 | 94.11 | 92.41 | 89.66 | 87.60 | 88.71 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 99.98 | 99.07 | 99.98 | 98.78 | 98.67 | 98.59 | 97.97 | 99.19 | 92.21 | 90.01 | 86.05 | 84.22 | 85.20 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Spanish-GSD | Trankit[large] | 99.93 | 99.30 | 99.88 | 97.38 | 99.88 | 96.96 | 95.23 | 98.59 | 93.47 | 91.56 | 88.58 | 80.91 | 86.50 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 99.91 | 98.94 | 99.88 | 97.41 | 99.88 | 96.88 | 95.23 | 98.62 | 92.66 | 90.50 | 87.01 | 79.83 | 85.04 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 99.96 | 95.97 | 99.87 | 96.69 | 99.87 | 96.40 | 94.44 | 98.44 | 89.61 | 86.73 | 81.22 | 73.96 | 79.19 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Swedish-LinES | Trankit[large] | 99.89 | 90.64 | 99.89 | 98.05 | 95.77 | 90.98 | 88.52 | 96.94 | 91.74 | 88.73 | 87.89 | 74.52 | 83.97 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 99.73 | 90.57 | 99.73 | 97.60 | 95.23 | 90.50 | 87.93 | 96.72 | 90.45 | 87.36 | 86.11 | 72.78 | 82.24 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 99.94 | 86.99 | 99.94 | 96.97 | 94.58 | 90.11 | 87.33 | 96.79 | 87.10 | 83.06 | 80.76 | 67.97 | 77.44 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Swedish-Talbanken | Trankit[large] | 99.91 | 99.26 | 99.91 | 99.06 | 98.09 | 98.04 | 97.43 | 97.87 | 94.41 | 92.97 | 92.08 | 88.67 | 89.28 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 99.87 | 99.38 | 99.87 | 98.76 | 97.77 | 97.73 | 97.03 | 97.82 | 93.61 | 91.87 | 90.72 | 86.97 | 88.03 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 99.97 | 98.85 | 99.97 | 97.65 | 96.57 | 96.70 | 95.63 | 97.51 | 88.96 | 85.91 | 83.59 | 79.17 | 80.78 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Tamil-TTB | Trankit[large] | 98.33 | 100.00 | 94.44 | 87.64 | 83.92 | 87.19 | 82.56 | 88.85 | 72.34 | 67.66 | 66.19 | 58.16 | 61.43 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 98.02 | 100.00 | 93.64 | 86.18 | 82.09 | 86.43 | 80.27 | 88.09 | 68.37 | 63.67 | 61.78 | 52.73 | 57.32 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 99.58 | 95.08 | 91.42 | 82.60 | 78.80 | 81.89 | 78.10 | 85.14 | 61.23 | 55.76 | 53.43 | 46.40 | 49.61 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Telugu-MTG | Trankit[large] | 98.89 | 98.62 | 98.89 | 94.31 | 94.31 | 98.33 | 94.31 | 98.89 | 93.19 | 87.08 | 84.46 | 80.65 | 84.46 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 98.89 | 98.62 | 98.89 | 94.32 | 94.32 | 97.92 | 94.32 | 98.89 | 91.97 | 84.35 | 81.10 | 78.44 | 81.10 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 100.00 | 97.95 | 100.00 | 92.93 | 92.93 | 99.17 | 92.93 | 100.00 | 89.32 | 79.89 | 74.88 | 71.25 | 74.88 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Turkish-IMST | Trankit[large] | 99.84 | 98.32 | 98.91 | 96.20 | 95.33 | 92.78 | 91.19 | 96.21 | 78.94 | 73.14 | 71.15 | 63.81 | 69.11 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 99.86 | 98.18 | 98.68 | 95.15 | 94.35 | 92.02 | 89.94 | 95.80 | 76.59 | 70.75 | 68.28 | 60.61 | 66.24 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 99.89 | 97.62 | 98.07 | 94.21 | 93.43 | 92.08 | 90.27 | 94.92 | 70.78 | 64.50 | 61.62 | 56.04 | 59.60 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Ukrainian-IU | Trankit[large] | 99.77 | 97.55 | 99.76 | 98.50 | 96.32 | 96.09 | 95.35 | 96.99 | 94.63 | 93.16 | 91.78 | 86.22 | 88.03 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 99.78 | 97.72 | 99.76 | 98.33 | 94.96 | 94.94 | 93.86 | 96.98 | 93.44 | 91.69 | 89.89 | 83.20 | 86.33 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 99.81 | 96.65 | 99.79 | 96.77 | 92.49 | 92.53 | 91.31 | 96.49 | 87.11 | 83.86 | 80.51 | 73.38 | 77.28 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Urdu-UDTB | Trankit[large] | 99.75 | 97.67 | 99.75 | 94.43 | 92.77 | 81.91 | 78.35 | 95.44 | 88.52 | 83.26 | 78.30 | 57.41 | 75.87 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 99.66 | 98.32 | 99.66 | 94.15 | 92.66 | 83.04 | 79.29 | 95.33 | 87.81 | 82.51 | 77.31 | 57.57 | 74.83 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 100.00 | 98.88 | 100.00 | 94.42 | 92.62 | 84.21 | 80.36 | 95.62 | 88.30 | 82.78 | 77.06 | 59.48 | 74.75 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Uyghur-UDT | Trankit[large] | 97.95 | 88.95 | 97.95 | 88.50 | 91.34 | 86.56 | 79.14 | 94.45 | 80.19 | 70.14 | 66.06 | 51.63 | 62.99 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 97.63 | 88.51 | 97.63 | 87.47 | 90.37 | 85.31 | 77.28 | 94.26 | 78.36 | 68.24 | 63.89 | 48.42 | 60.70 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 99.79 | 86.90 | 99.79 | 89.45 | 91.92 | 87.92 | 80.54 | 96.16 | 75.55 | 63.61 | 57.00 | 46.06 | 54.39 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | Vietnamese-VTB | Trankit[large] | 94.88 | 96.63 | 94.88 | 89.70 | 88.14 | 94.64 | 88.07 | 94.88 | 71.07 | 65.37 | 63.67 | 60.22 | 63.67 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Trankit[base] | 95.22 | 96.25 | 95.22 | 89.40 | 87.85 | 95.03 | 87.82 | 95.22 | 70.96 | 64.76 | 62.72 | 58.51 | 62.72 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ | | Stanza | 87.25 | 93.15 | 87.25 | 79.50 | 77.90 | 87.02 | 77.87 | 87.20 | 53.63 | 48.16 | 44.88 | 42.17 | 44.85 | +------------------------------+--------------------------------+--------+-----------+--------+-------+--------+--------+---------+--------+-------+-------+-------+-------+-------+ Performance for Stanza, UDPipe, and spaCy is obtained using their public pretrained models. The overall performance for Trankit and Stanza is computed as the macro-averaged F1 over 90 treebanks using the `official evaluation script `_ of the CoNLL 2018 Shared Task. Named Entity Recognition --------------------------- Performance comparison between Trankit large, Trankit base, and Stanza v1.1.1 on the test sets of 11 public NER datasets. Performance is based on entity micro-averaged F1. +----------+------------+----------------+----------------+---------------+ | Language | Corpus | Trankit[large] | Trankit[base] | Stanza v1.1.1 | +----------+------------+----------------+----------------+---------------+ | Arabic | AQMAR | 76.1 | 74.8 | 74.3 | +----------+------------+----------------+----------------+---------------+ | Chinese | OntoNotes | 80.5 | 80 | 79.2 | +----------+------------+----------------+----------------+---------------+ | Dutch | CoNLL02 | 93.8 | 91.8 | 89.2 | | +------------+----------------+----------------+---------------+ | | WikiNER | 95.0 | 94.8 | 94.8 | +----------+------------+----------------+----------------+---------------+ | English | CoNLL03 | 92.5 | 92.1 | 92.1 | | +------------+----------------+----------------+---------------+ | | OntoNotes | 90.2 | 89.6 | 88.8 | +----------+------------+----------------+----------------+---------------+ | French | WikiNER | 92.9 | 92.3 | 92.9 | +----------+------------+----------------+----------------+---------------+ | German | CoNLL03 | 85.3 | 84.6 | 81.9 | | +------------+----------------+----------------+---------------+ | | GermEval14 | 89.4 | 86.9 | 85.2 | +----------+------------+----------------+----------------+---------------+ | Russian | WikiNER | 93.2 | 92.8 | 92.9 | +----------+------------+----------------+----------------+---------------+ | Spanish | CoNLL02 | 89.2 | 88.9 | 88.1 | +----------+------------+----------------+----------------+---------------+