AplikaceAplikace
Nastavení

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
en:cnk:intercorp:historie [2022/11/23 14:31] – [Release 14] alexandrrosenen:cnk:intercorp:historie [2026/09/30 21:14] (current) – [Release 17ud] rosen
Line 1: Line 1:
 +======= InterCorp: Version history =======
 +
 +===== Release 17ud =====
 +
 +  * Released on 9 September 2026 (pilot version)
 +  * [[https://wiki.korpus.cz/doku.php/en:cnk:intercorp:verze17ud|Information about the corpus]]
 +
 +== Data: ==
 +
 +  * **New books** in the Core: 40 new Czech titles (1852 in total), 214 new titles across all languages (6669 in total)
 +  * The new [[https://jakobson.korpus.cz/~rosen/INTERCORP/ICv17ud/intercorp-tools-static/index.html|InterCorp Tools]] serve as a **guide to InterCorp**, providing a wide range of information about the corpus and replacing the tables and charts included in the documentation of previous versions; they also include a Hall of Fame listing everyone who contributed to the preparation of new texts for the InterCorp Core release 17ud
 +  * New language: **Kashubian** (code ''cb'', currently 2 books in the Core)
 +  * New text collection: **[[https://www.ted.com|TED Talks]]** -- 1383 talk transcripts in Czech, i.e. 2.4 million positions; 39,374 transcripts across all languages, i.e. 28.9 million positions
 +  * Additions to the **Project Syndicate** collection from 2021--2025
 +  * The **Bible** is now also available in Spanish and Upper Sorbian
 +
 +== Annotation: ==
 +
 +  * This is the first release available exclusively with linguistic annotation based on **Universal Dependencies**
 +  * In Slavic and Romance languages, multiword verb forms are now also annotated using the positional attribute ''cw_feats'' – **content-word features** ([[pojmy:content-word features|cw_feats]])
 +  * In the Core, **authors** are now also identified by a name that is consistent across languages (attribute ''text.rf_author'')
 +  * The positional attributes useful for navigating syntactic structure have been extended with references to the **word form of the head**: ''p_word'' (parent word form) and ''e_word'' (effective parent word form)
 +  * **Literature for children and young adults**, previously identified as ''text.txtype=children's lit.'', is now identified as ''text.audience=jun''; for other texts, this attribute has the value ''gen''; the ''text.txtype'' attribute can thus be used to distinguish text types (''fiction'', ''poetry'', ''drama'', ''non-fiction'')
 +  * **Newer language models** from [[https://lindat.mff.cuni.cz/services/udpipe/|UDPipe]] (ud-2.17) were used for linguistic annotation; Chinese alone is annotated using [[https://trankit.readthedocs.io/|Trankit]] for better results (thanks to [[https://cz.linkedin.com/in/benjamin-reeves-|Benjamin David Reeves]] and [[https://ufal.mff.cuni.cz/pavel-stranak|Pavel Straňák]])
 +  * New **segmentation and alignment tools** were used to prepare the TED Talks collection ([[https://github.com/thompsonb/vecalign|Vecalign]])
 +
 +== Search interface: ==
 +
 +  * The value of the positional attribute ''deprel'' (the syntactic function of a word) can be specified in Advanced Query in the same way as the values of ''upos'' and ''feats'', by selecting it from the **Insert tag** menu
 +  * Clicking any word in the concordance in one language highlights its most likely **translation equivalent** in the other language or languages (this feature also works with earlier versions of InterCorp)
 +
 +== Bug fixes: ==
 +
 +  * **Author names** were not given consistently for some texts within the same language (attribute ''text.author'')
 +  * In languages written in **Cyrillic**, characters visually identical to Latin characters were sometimes incorrectly encoded as Latin; we have managed to eliminate most of these errors
 +  * A number of texts were not identified as **literature for children and young adults**; this has been corrected in connection with moving this information to the new ''text.audience'' attribute; the number of texts identified as such has increased from 5 to 122
 +  * Due to minor corrections in the way **syntactic complexity and lexical diversity** are aggregated for individual texts, text types and languages, the aggregate values of these measures may in some cases differ slightly from the previous version even when the corpus content has not changed
 +
 +== Known issues: ==
 +
 +  * In **French**, ''Tense=Past'' and ''Tense=Pres'' are no longer specified in the ''feats'' attribute for participles (''VerbForm=Part''), so past and present participles cannot be distinguished in the InterCorp annotation. This is due to a change in the [[https://universaldependencies.org/treebanks/fr_gsd/index.html|French data]] used to train the [[https://lindat.mff.cuni.cz/services/udpipe/|UDPipe]] language model. The maintainers of the GSD treebank moved this information from the ''FEATS'' field to the ''MISC'' field, which is not included in the model. (For the data format, see [[https://universaldependencies.org/format.html|CoNLL-U]]; for the reasons behind the change, see the discussion [[https://github.com/UniversalDependencies/UD_French-GSD/issues/30|UD_French-GSD: Tense removed from FEATS]].)
 +===== Release 16ud =====
 +
 +Published 17 September 2024
 +
 +  * Contains the same texts as release 16
 +  * Mainly differs in the unified linguistic annotation of all languages according to the Universal Dependencies standard (cf. Release 13ud)
 +  * Metadata for each sentence and text now include measures of syntactic complexity, for each text also measures of lexical diversity
 +  * [[en:cnk:intercorp:verze16ud|Information about the corpus]]
 +
 +
 +
 +
 +===== Release 16 =====
 +
 +Published 12 October 2023
 +
 +
 +  * The core now contains all texts planned and approved for 2022 and submitted by the deadline for this version
 +  * The number of words in all languages and text types has tripled from 1 798 million to 5 290 million
 +  * This is mainly due to the update of the Subtitles package, which now contains 4 001 million words
 +  * 20 new languages were added to Subtitles and thus to the corpus as a whole - the corpus now contains 62 languages (including Czech)
 +  * The number of words in all languages except Czech is 4 893 million, of which 387 million represents the core and 4 506 million the collections
 +  * The total number of words in Czech texts is 398 million, including 125 million core and 273 million collection
 +  * [[en:cnk:intercorp:verze16|Information about the corpus]]
  
-====== InterCorp: Version history ====== 
  
 ===== Release 15 ===== ===== Release 15 =====

Trace: • akvizicni_korpus • verze8