Nastavení

The SYN2020 corpus is a synchronous representative and reference corpus of contemporary written Czech, containing 100 million text words, including punctuation (tokens). It is a sequel of the representative corpora of the SYN series (SYN2000, SYN2005, SYN2010), issued at five-year intervals, and covers the time period since 1989. Each of the SYN series corpora primarily covers the language of the last five years preceding its publication; thus, SYN2020 focuses on the 2015–2019 period. None of the texts in SYN2020 were included in another corpus of this series (the corpora are mutually disjoint). The SYN2020 corpus is lemmatized and morphologically tagged, just as the SYN2015 corpus it also contains syntactic annotation, but in comparison with the other corpora there are a number of changes:

Changes with respect to other corpora of the SYN series

Tokenization

In the existing corpora of the SYN series, almost all combinations of alphabetic, numeric characters and punctuation marks that were written in the original texts without a space have so far been considered one token. Only punctuation marks at word boundaries (řekl , že) and some other combinations, such as the hyphen before the enclitic form li (mohu - li), have been tokenized in a separate way.

In SYN2020, the approach is opposite: numeric characters and punctuation marks are systematically identified as separate tokens, but some combinations of characters remain unseparated according to predefined rules and word lists (eg words such as česko-německý, wi-fi, r’n’b, Jang-c’-ťiang, CO2, 12letý). These principles are/will be presented on the tokenization of numeric and punctuation marks page.

Lemmatization

A fundamental change in the annotation of the SYN2020 corpus is the introduction of the so-called two-level lemmatization: now each word form is assigned a sublemma attribute in addition to the lemma one. While the lemma associates several variants of one word in accordance with the earlier corpora of the SYN series (eg the filozofie lemma represents all the forms with the filozof and also filosof root), the sublemmas define subgroups of word forms with respect to this variability (sublemma filozofie represents only word forms with the filozof root, the filosofie sublemma only the forms associated with the filosof root). In case the word has no variants, the sublemma is identical to the lemma (eg the kniha lemma represents the same set of forms as the kniha sublemma).

Different types of variants are accounted for as sublemmas (eg mýdlo/mejdlo, okno/vokno, citron/citrón, email/e-mail, myslet/myslit, mýt/mejt, péci/péct/píct, kuchyně/kuchyň, antivirus/antivir, sedm/sedum, tenhle/tendle/tenle, ačkoli/ačkoliv, proper names Robert/Róbert/Roberto, Atény/Athény) and by means of these sublemmas some specific groups of forms are distinguished that are traditionally covered under one lemma (eg negated forms of adjectives and adverbs černý/nečerný, hezky/nehezky, nominal forms of adjectives mladý/mlád and suppletive forms dobře/lépe/líp, člověk/lidé).

In connection with these changes, the lemmatization was significantly refined compared to the previous corpora of the SYN series, many lemmas were corrected and other tens of thousands of lemmas are now recognizable in the SYN2020 corpus. A detailed description of the changes is presented on the lemmatization page.

Morphological tagging

From the SYN2020 corpus onwards, each morphological tag has 15 positions (instead of the previous 16 ones). The annotation of verbal aspect is transferred from the canceled 16th position to the originally unused 13th one, otherwise the tag structure is identical to the structure present in existing corpora of the SYN series.

The annotation changes themselves concern the following three positions in the tag. In the 1st position (part of speech), the values ​​F (foreign word), B (abbreviation) and S (segment) are now distinguished. At the same time, the part-of-speech classification of some words and forms was re-evaluated (especially in the category of ​​numerals, predicatives and nominal forms of adjectives). In the 2nd position (detailed part-of-speech specification), new values were introduced in connection with the new parts of speech and some other were removed. The subdivision of numerals has been substantially modified (eg. the z value is now used for the numerals sto, tisíc, milion originally tagged as nouns) and the value 0 was added in order to identify non sentence-final punctuation. One change concerns the 15th position (variant): number 8 (so far reserved for abbreviations) is now used as a value coding another variant of colloquial Czech.

The reliability of automatic lemmatization and morphological tagging of the SYN2020 corpus is significantly higher than was the case with previous corpora of the SYN series.

A detailed overview of the changes is presented on the morphological marking page.

Verb tagging (verbtag)

A newly introduced verbal tag (verbtag) contains morphological information about the whole verb form, regardless of whether it is a compound form (viděl jsem) or a simple one (vidím). In the verbtag, on the one hand, the auxiliary verb differs from the autosemantic one, and on the other hand, for each autosemantic verb form, the following categories of manner, voice, person, number and tense are specified (valid for the whole verb form). The verb tag is assigned to each token in the corpus, but it takes appropriate values ​​only for verbs (and with one exception for deverbal adjectives). For the full presentation of the verbtag, see the verbtag page.

Multiple lemmatization and tagging (aggregate)

In the SYN2020 corpus, multiple lemmas and tags for a special group of words, so-called aggregates, are newly introduced. Aggregates are words that are written as one orthographic word in Czech, but from the point of view of syntax or specification of grammatical categories they behave as two orthographic words (exceptionally three). The aggregates concern conditional conjunctions (aby, kdyby), the connection of words with the the enclitical form s (dělalas, viděls, komus, vždyťs), the connection of prepositions with some pronouns (nač, očpak, zaň), or a combination of words of the last two types (načs). For each of these words, two (or three) lemmas, sublemmas, tags and verbtags are specified at the same time according to their respective parts. For detailed information on aggregates, see the aggregate page.