BY DETECTING NON-COMPOSITIONALITY OF MULTIWORD EXPRESSIONS EXTRAÇÃO DE COMBINAÇÕES LEXICAIS RESTRITAS PELA DETEÇÃO DA NÃO COMPOSIONALIDADE DE EXPRESSÕES PLURIVERBAIS * Joana Veloso UNIVERSIDADE DO MINHO, PORTUGAL [email protected] Alberto Simões UNIVERSIDADE DO MINHO, PORTUGAL [email protected] Álvaro Iriarte S. UNIVERSIDADE DO MINHO, PORTUGAL [email protected] ,Q WKLV DUWLFOH DQ HYDOXDWLRQ RI D PHWKRG IRU H[WUDFWLQJ UHVWULFWHG OH[LFDO FRPELnations from parallel corpora by detecting non-compositionality of multiword H[SUHVVLRQV LQ WUDQVODWLRQ ZLOO EH SUHVHQWHG 7KLV PHWKRG SUHVXSSRVHV WKDW E\ ¿QGLQJVHTXHQFHVRIZRUGVZRUGELJUDPVDUHXVHGZKRVHWUDQVODWLRQGRHVQRW follow a simple word-to-word conversion of the component words, a collocation LVSUREDEO\SUHVHQW Keywords: &ROORFDWLRQV7UDQVODWLRQ3DUDOOHO&RUSRUD Neste artigo apresentamos uma avaliação sobre um método para extrair combinações lexicais restritas a partir de corpora paralelos, pela deteção da não comSRVLFLRQDOLGDGHGHH[SUHVV}HVSOXULYHUEDLVQDWUDGXomR(VWHPpWRGREDVHLDVHQD presunção de que, encontrando sequências de palavras (são usados bigramas de palavras) cuja tradução não siga a tradução palavra por palavra dos seus compoQHQWHVpSURYiYHOHVWDUVHSHUDQWHXPDFRORFDomR Palavras-chave: &RORFDo}HV7UDGXomR&RUSRUD3DUDOHORV :HDUHJUDWHIXOWRWKHDQRQ\PRXVUHYLHZHUVIRUWKHLUYDOXDEOHFRPPHQWV 5 Joana Veloso / Alberto Simões / Álvaro Iriarte S. 0. Introduction &ROORFDWLRQVFDQEHGH¿QHGLQYHU\GLIIHUHQWZD\VDFFRUGLQJWRGLIIHUHQW authors, and the same sequence of words can be considered or not a colloFDWLRQE\GLIIHUHQWUHVHDUFKHUVHYHQZKHQXVLQJDVLPLODUGH¿QLWLRQ $JUHDWQXPEHURIOLQJXLVWVGH¿QH³FROORFDWLRQ´LQWHUPVRIIUHTXHQF\ That is the criterion used by many authors within the computational linJXLVWLFV FRPPXQLW\ IRU FRUSXVEDVHG DXWRPDWLF FROORFDWLRQ H[WUDFWLRQ +RZHYHU RWKHU OLQJXLVWV HVSHFLDOO\ OH[LFRORJLVWV DQG OH[LFRJUDSKHUV believe that the fact that two lexemes frequently co-occur in context is not DJRRGHQRXJKUHDVRQWREHFRQVLGHUHGDFROORFDWLRQ0HO¶þuk et al. VWDWH ©.MHOOPHULOOXVWUHXQHVVDLG¶H[WUDWLRQDXWRPDWLTXHGHFROORFDWLRQVG¶XQ FRUSXVLQIRUPDWLVpVDQVLQWHUYHQWLRQGpFLVLRQQHOOHG¶XQOH[LFRORJXHFHGLFWLRQQDLUHHVWUHPSOLG¶H[SUHVVLRQVFRPPH0U6PLWKZDVDPHPEHUWKHDELOLWLHVDEDGWKLQJHWFTXLQ¶RQWULHQjYRLUDYHFOHVFROORFDWLRQVª &RVHULXGHQLHVWKDWDOH[HPHFRPELQDWLRQLVD³OH[LFDOVROLGDULW\´RQO\ EHFDXVH ZRUGV DUH RIWHQ FRPELQHG )RU &RVHULX D OH[LFDO VROLGDULW\ LV explained by lexical restrictions based on the linguistic content of a lexeme WKDWGULYHLWWRFRPELQHZLWKRWKHUOH[HPHV ©«ODSUREDELOLGDGHVWDGtVWLFDGHODVFRPELQDFLRQHVQRWLHQHSUiFWLFDPHQWH nada que ver con las solidaridades y no es prueba de su existencia: cavallo bianco es, probablemente, más frecuente que cavallo sauroSHURHQHOSULPHU caso, la probabilidad de la combinación depende de la realidad extralinJXtVWLFD HQ HO VHJXQGR HQ FDPELR HVWi GDGD OLQJtVWLFDPHQWH SRU HO FRQtenido de sauro.ª&RVHULX There are software tools that allow the automatic extraction of frequent lexical combinations, however, these tools do not distinguish between collocations, idioms, free phrases, compound names, or multiword technical WHUPV,WVKRXOGEHQRWHGWKDW³UHVWULFWHGOH[LFDOFRPELQDWLRQV´HVSHFLDOO\ FROORFDWLRQVZLWK³IUHTXHQWFRPELQDWLRQV´RIWZRRUPRUHOH[HPHVPXVW QRWEHFRQIXVHG ,WLVWUXHWKDWDFROORFDWLRQLVDOVRDIUHTXHQWFRPELQDWLRQRIWZRRUPRUH OH[HPHVEXWIUHTXHQWOH[LFDOFRPELQDWLRQVDUHQRWDOZD\VFROORFDWLRQV$ lexical combination such as ler um livro is a frequent combination, but it is a completely free phrase, a combination of words formed according to the Extraction of restricted lexical combinations by detecting non-compositionality of multiword expressions 5 syntactic rules (the verb ler can be combined with everything capable of EHLQJUHDG+RZHYHUWKHDGMHFWLYHVLQOH[LFDOFRPELQDWLRQVVXFKDVativLGDGHIHEULOPXGDQoDUDGLFDOYRQWDGHORXFDódioPRUWDODPRUFHJR are ¿[HGDOWKRXJKWKH\KDYHWKHVDPHPHDQLQJLQWHQVHDORW7KHVHOH[LFDO combinations are frequent because the choice of adjective is not free, that is, the fact that these two lexemes frequently appear combined is not the cause EXWWKHFRQVHTXHQFHRIWKHPEHLQJDFROORFDWLRQ$ORQVR5DPRV +RZHYHUIURPDOH[LFRJUDSKLFDOSRLQWRIYLHZWKHVHIUHTXHQWOH[LFDO combinations, although they are not collocations in the narrow sense that ZHXVHVKRXOGKDYHVSHFLDOWUHDWPHQWLQGLFWLRQDULHVDQGOH[LFDOGDWDEDVHV ,Q WKLV UHJDUG &RZLHV GLVWLQFWLRQ EHWZHHQ restricted collocations and open collocations LVXVHIXO&RZLHet al. ,WZLOOWKHUHIRUHEHDSSURSULDWHLQOH[LFRJUDSKLFDOSUDFWLFHWRUHFRJnize the existence of more or less free/restricted lexical combinations and to register usual or frequent free combinations in dictionaries (Ettinger, &RUSDV As we know, stability and semantic specialization are the main features RIWKLVW\SHRIQRQIUHHSKUDVHV&OHDUO\V\QWDFWLFFRQVWUDLQWVZLOOEHKLJKHU IRULGLRPVWKDQIRUFROORFDWLRQV)RUH[DPSOHWKHLGLRPperder a cabeça has more syntactic constraints than the collocation prestar atenção, which DOORZVVRPHFKDQJHVVHH$JXLODU$PDW+RZHYHUIURPD lexicographical point of view, it is impossible to establish general syntactic UXOHVIRUDOOLGLRPVDQGFROORFDWLRQV The criteria that allow us to determine if a lexical combination was OH[LFDOL]HGFDQQRWEHPRUSKRORJLFDORUV\QWDFWLFLQQDWXUH7KLVKDVPRUH to do with the consensus and the memory of the linguistic community that uses it: ©/H FULWqUH XOWLPH GH Gp¿QLWLRQ G¶XQH XQLWp OH[LFDOH HVW ELHQ LFL SDU H[FHOOHQFHOHFRQVHQVXVGHODFRPPXQDXWpOLQJXLVWLTXH>«@QRQSDVFRPPHHQ V\QWD[HRXHQPRUSKRORJLHSDUODUHFRQQDLVVDQFHG¶XQHERQQHIRUPDWLRQPDLV VXUODEDVHGHODPpPRULVDWLRQª3DLOODUG Ultimately, when it comes to distinguishing between free phrases and collocations, in addition to the more or less intuitive perceptions of speakers, the use of a foreign language serves to illustrate that choosing a collocDWLYHIRUDEDVHRIFROORFDWLRQLVQRWIUHH&DOGHUyQ7RPDV]F]\N 7KHUHIRUHZHDUHRIWKHRSLQLRQWKDWWKHXVHRIDIRUHLJQODQJXDJHFDQEHXVHIXOWRH[WUDFWFROORFDWLRQVIURPSDUDOOHOFRUSRUD 5 Joana Veloso / Alberto Simões / Álvaro Iriarte S. )RUWKHSXUSRVHRIWKLVZRUNRXUGH¿QLWLRQRIFROORFDWLRQLVLIDQ\RI WKHPHPEHUZRUGVRUWKHFRPSOHWHVHTXHQFHLQKHULWDGLIIHUHQWPHDQLQJRI LWVWKHLUXVXDOVHQVHZKHQXVHGLQFRQMXQFWLRQZLWKDQRWKHUZRUGWKHQWKLV VHTXHQFHRIZRUGVLVFRQVLGHUHGWREHDFROORFDWLRQ 2XUDVVXPSWLRQLVWKDWJLYHQDPXOWLZRUGH[SUHVVLRQtA and its translation tB, tA is a collocation if tB includes words which are not direct translations of any of tAZRUGV To test this assumption, we chose a mid-sized corpus, the European &HQWUDO%DQNFRUSXVIURPWKH3HU)LGHSURMHFW$UD~MR$OPHLGD6LP}HV 'LDV)RUWKHODQJXDJHVZHFKRVHWKH6SDQLVK3RUWXJXHVHODQJXDJH SDLU7KH PDLQ UHDVRQ IRU WKLV FKRLFH LV WKH SUR[LPLW\ RI WKH ODQJXDJHV DQG WKH ELOLQJXDO WUDQVODWLRQ GLFWLRQDULHV ZH KDG DYDLODEOH :H expect to make further tests on this hypothesis with other language pairs in WKHIXWXUHQDPHO\LQFOXGLQJ*HUPDQLFODQJXDJHV As for the text to be analyzed, we selected Spanish word pairs in which one of the words is an adjective and the other is a noun (in any RUGHU)RUWKLVSXUSRVHWKH)UHH/LQJ6LP}HV&DUYDOKR3DGUy 6WDQLORYVN\PRUSKRORJLFDODQDO\]HUZDVXVHG)RUHDFKZRUGSDLU HDFKSRVVLEOHZRUGWUDQVODWLRQZDVORRNHGXSLQWKHWUDQVODWLRQVHJPHQW This was possible with the help of a Spanish/Portuguese translation dicWLRQDU\$SHUWLXP&RUEt%HOORWHWDO ,I DQ\ RI WKH SRVVLEOH WUDQVODWLRQV IRU ERWK ZRUGV RFFXU WRJHWKHU LQ DQ\RUGHULQWKH3RUWXJXHVHSDUWWKDWZRUGSDLULVGLVFDUGHG2QWKHRWKHU hand, if only one of the words has a translation in the Portuguese part, the Spanish segment is stored, together with a snippet of the Portuguese transODWLRQ7KHQH[WVHFWLRQH[SODLQVWKLVDOJRULWKPLQPRUHGHWDLO The word pairs obtained, together with the respective translation, were PDQXDOO\HYDOXDWHGIRUZKHWKHUWKH\DUHRUQRWUHVWULFWHGFRPELQDWLRQV The evaluation section will discuss the details on the manual assessment of the obtained results, explaining the main problems found as well as the IXWXUHHQKDQFHPHQWVWRWKHSURSRVHGDOJRULWKP)LQDOO\DVHWRIFRQFOXVLRQVLVGUDZQIURPWKHREWDLQHGUHVXOWV 1. Related Work The task of identifying restricted lexical combinations, as we will state LQWKLVDUWLFOHLVQRWQHZ,WLVDUHOHYDQWSURFHGXUHIRUGLIIHUHQWWDVNVRQ 1DWXUDO/DQJXDJH3URFHVVLQJ1/3OLNHIRUH[DPSOH0DFKLQH7UDQVODWLRQ07ZKHUHLGLRPDWLFH[SUHVVLRQVFDQQRWEHWUDQVODWHGOLWHUDOO\(YHQ Extraction of restricted lexical combinations by detecting non-compositionality of multiword expressions 5 FROORFDWLRQV QHHG WR EH WUDQVODWHG ZLWK FDXWLRQ )RU H[DPSOH computer graphics cannot be translated into Portuguese as JUi¿FRVGHFRPSXWDGRU, not because it is a wrong translation, but because the term that was coined in the Portuguese community was FRPSXWDomRJUi¿FDJUDSKLFV¶FRPSXWDWLRQ The easiest way to detect sequences of words likely to be considered as DFROORFDWLRQRUDWOHDVWDVDFRPSRXQGWHUPLVWRXVHWKH0XWXDO,QIRUPDWLRQ0,RU3RLQWZLVH0XWXDO,QIRUPDWLRQ30,PHWULFV$VWKLVLVMXVWDQ association measure, authors use these indicators together with other techQLTXHVOLNHWKHXVDJHRISDWWHUQV*XLQRYDUW6LP}HV+RZHYHU by themselves, these two measures are not enough for the extraction of colORFDWLRQV$SULRUVWXG\3DYHOSUHVHQWVDYDVWDPRXQWRIPHDVXUHV WKDWFDQEHXVHGWRGHWHFWFROORFDWLRQV1HYHUWKHOHVVPRVWRIWKHPSHUIRUP badly by themselves, and as presented below, new approaches have been XVHG Probably the bigger challenge is to detect idiomatic expressions, mostly when they can also have a literal meaning (like break the ice that FDQ EH FRQVLGHUHG OLWHUDOO\ RU QRW /L 6SRUOHGHU SUHVHQW D VHW of different properties that can be extracted from texts in order to detect LI WKHVH H[SUHVVLRQV DUH EHLQJ XVHG OLWHUDOO\ 3URSHUWLHV DUH YHU\ GLYHUVH from the usage of prepositions before or after the expression, to graphs of FRKHVLRQEHWZHHQWKHGLIIHUHQWVHQWHQFHFRPSRQHQWV7KHVHSURSHUWLHVDUH WKHQXVHGLQD6XSSRUW9HFWRU0DFKLQHDOJRULWKP7KHVHVDPHDXWKRUV/L 6SRUOHGHUDOVRZRUNHGRQWKHXVHRI*DXVVLDQ0L[WXUH0RGHOV IRUWKLVVDPHWDVN7KHLUHYDOXDWLRQSRLQWVWRSUHFLVLRQIRUWKHGHWHFWLRQRIOLWHUDOH[SUHVVLRQVEXWRQO\WRGHWHFWQRQOLWHUDOLGLRPDWLF H[SUHVVLRQV 0X]Q\=HWWOHPR\HUDOVRXVHFODVVL¿FDWLRQWHFKQLTXHVWRGLVWLQJXLVKEHWZHHQLGLRPDWLFDQGQRQLGLRPDWLFH[SUHVVLRQV)RUWKDWWKH\ trained a binary perceptron based on two types of features: lexical features, like the usage of capital letters, and graph features, using relations LQIRUPDWLRQREWDLQHGIURP:RUG1HWDQG:LNWLRQDU\7KHSHUFHSWURQZDV WUDLQLQJRQ:LNWLRQDU\ODEHOHGGDWDDQGXVHGWKHQRQODEHOHGGDWDIRUWHVW SXUSRVHV7KHUHVXOWVJRXSWRRISUHFLVLRQDQGUHFDOORYHU The most relevant study found using multilingual information for the GHWHFWLRQRIFROORFDWLRQH[WUDFWLRQ6HUHWDQ:HKUOLGRHVQRWXVH translation information, but only a parser able to process text in different ODQJXDJHV7KHH[WUDFWLRQPHWKRGLWVHOIGRHVQRWWDNHDQ\UHDODGYDQWDJH RISDUDOOHOFRUSRUD 5 Joana Veloso / Alberto Simões / Álvaro Iriarte S. 2XUDSSURDFKXQGHUVWDQGVWKHWUDQVODWLRQDVDIXQFWLRQWKDWFDQVRPHKRZ DVVRFLDWH ³semantic” WR HDFK ZRUG 7KHUHIRUH LI WKH WUDQVODWLRQ ³VHPDQWLF´LVQRWFRPSRVLWLRQDOZHKDYHDFDQGLGDWHFROORFDWLRQ 2. The Hypothesis The hypothesis we are testing is: if a sequence with two words, an adjective and a noun, is translated by two other words, and only one of them is a translation of the original words found in a translation dictionary, then we KDYHDFDQGLGDWHFROORFDWLRQ 7KLVFDQEHEHWWHUH[SODLQHGXVLQJPDWKHPDWLFDOV\QWD[/HWXVGH¿QH the 7 function, that translates Spanish words into Portuguese, and the conFDWHQDWLRQRSHUDWRUDGRWZKLFKMRLQVWZRZRUGV The translation of two words wa and wb is considered to be compositional if (1) T(wa wb) = T(wa7wb) or even, T(wa wb) = T(wbT(wa) Therefore, we are looking for a pair of words (wa Zb) in which one of them is an adjective and the other a noun, and whose translation does not IROORZWKHHTXDWLRQSUHVHQWHGDERYH7KDWLVZHZDQWWR¿QGZa and wb where 7wa . wb) = T(wawcޔ7wa wb) = wc T(wb) with T(wa wc ޔT(wbwc The extraction algorithm used is very simple, and its main purpose is to test the hypothesis that the collocation extraction based on non-translaWLRQ FRPSRVLWLRQ LV SRVVLEOH7KH DOJRULWKP VWDUWV E\ LWHUDWLQJ RYHU HDFK WUDQVODWLRQ XQLW LQ WKH SDUDOOHO FRUSXV$ WUDQVODWLRQ XQLW LV FRPSRVHG E\ a segment SSP for the Spanish language, and a segment SPT for the PortuJXHVHODQJXDJH7KHQHDFKSRVVLEOHELJUDPIURPWKHVHJPHQW6SP is anaO\]HGXVLQJWKH)UHH/LQJPRUSKRORJLFDODQDO\]HUORRNLQJIRUDVHTXHQFH LQZKLFKRQHRIWKHZRUGVLVDQRXQDQGWKHRWKHUDQDGMHFWLYH1RWHWKDW DOWKRXJK)UHH/LQJKDVPRGXOHVWRGRSDUWRIVSHHFKWDJJLQJWKH\ZHUHQRW XVHG1HYHUWKHOHVVZHDUHDZDUHRIWKHSUREOHPVWKLVDSSURDFKDULVHVDQG ZHZLOOGLVFXVVWKHPODWHU :KHQVXFKDSDLURIZRUGVLVIRXQGWKHLUSRVVLEOHWUDQVODWLRQVHWVDUH FRPSXWHG1RWHWKDWHDFKZRUGFDQKDYHPRUHWKDQRQHWUDQVODWLRQDQG WKHUHIRUHZHQHHGWRFRQVWUXFWDVHWRIWUDQVODWLRQV7KLVWUDQVODWLRQZDV Extraction of restricted lexical combinations by detecting non-compositionality of multiword expressions 5 GRQHXVLQJWKH$SHUWLXPWUDQVODWLRQGLFWLRQDU\7KHQWKHVHWUDQVODWLRQVHWV are searched in the target language segment SPT,IDQ\RIWKHZRUGVIURP ERWKWUDQVODWLRQVHWVRFFXUWRJHWKHUWKHZRUGSDLULVGLVFDUGHG 2QWKHRWKHUKDQGLIRQHRIWKHZRUGVKDVDWUDQVODWLRQLQWKHWDUJHW segment, but the other does not, the Spanish word pair is saved for manual DVVHVVPHQW7RJHWKHUZLWKWKHZRUGSDLUDVHJPHQWRI3RUWXJXHVHZRUGVLQ WKHYLFLQLW\RIWKHIRXQGWUDQVODWLRQLVVHL]HGDQGDOVRVWRUHG7KLVOLVWZDV WKHQDVVHVVHGPDQXDOO\ 3. Assessments and Evaluation The assessment was performed manually using online resources as refHUHQFHVXFKDV,$7(,QWHU$FWLYH7HUPLQRORJ\IRU(XURSH-RKQVRQ 0DFSKDLODQGERWKSDSHUDQGRQOLQH6SDQLVK3RUWXJXHVHDQG3RUWXJXHVH6SDQLVK GLFWLRQDULHV$ /LQJXLVWLFV 06F VWXGHQW FODVVL¿HG HDFK word pair manually into one of the following classes: • Error: used for all entries whose Spanish and Portuguese segments DUHQRWUHODWHGZLWKHDFKRWKHU7KLVKDSSHQVPDLQO\EHFDXVHWKH DSSOLFDWLRQZDVQRWDEOHWR¿QGWKHVHTXHQFHRIZRUGVWKDWLQFOXGH the translation of the selected pair of words, or because the original FRUSXVKDGDOLJQPHQWHUURUV • Free combination: the pair of words is correctly translated, but it is not a restricted lexical combination (accordingly with the criWHULRQ ZH GH¿QHG HDUOLHU7KLV KDSSHQV PRVWO\ ZKHQ D SRVVLEOH ZRUGWUDQVODWLRQLVQRWLQFOXGHGLQWKHXVHGWUDQVODWLRQGLFWLRQDU\ • Restricted combination: the pair of words is correctly translated, DQGLWFRUUHVSRQGVWRDFROORFDWLRQRUDTXDVLSKUDVHPH ! " #$ $#$ $ %$& ZHWRRNWKHGHFLVLRQWRFRQVLGHULWDVDIUHHFRPELQDWLRQ7KLVPHDQVWKDW RXUHYDOXDWLRQLVOHVVIDYRUDEOHWRRXUK\SRWKHVLV )URPKHUHRQZHZLOOGLVFXVVHDFKFODVVSURYLGLQJUHDOH[DPSOHVIRU HDFKRIWKHP 5' Joana Veloso / Alberto Simões / Álvaro Iriarte S. 3.1. Errors The errors found are from very different kinds, such us from alignment problems, some minor bugs in the algorithm implementation, or the lack of translations from the translation dictionary: • The use of a morphological analyzer instead of a part-of-speech tagger OHDGVWRVRPHH[DPSOHVZLWKYHUEVPLVFODVVL¿HGDVQRXQVDQGRUDGMHFWLYHV1HYHUWKHOHVVFRQVLGHULQJWKDWRXUK\SRWKHVLVLVWKHH[LVWHQFHRI D VHTXHQFH ZLWK D QRXQ DQG DQ DGMHFWLYH WKH H[DPSOHV FODVVL¿HG DV DUHVXOWRIWKLVSUREOHPDUHLUUHOHYDQWIRUSURYLQJLW7DEOH()* shows VRPHRIWKHVHVLWXDWLRQV +,-./ 01 23,46./7 89 /3:;,<:=8>7 ?@/;/ , A/;-,. 98;4 ?,7 4=7:,B/>.C interpreted as a noun • DEFGH IJKLD presente DQH[R¿JXUDum modelo conjunto presente Se o conjunto apresentar LQIRUPH¿JXUD este relatório consta de o FHUWL¿FDGRIDOOD YHUL¿FDomRGHRFHUWL¿FDGRIDOKDU The algorithm, when searching the set of words in the context of the found translation broke the segment, losing the interesting part of the WUDQVODWLRQ7KLVWXUQHGWKHDVVHVVPHQWLPSRVVLEOH7KLVZDVDSUREOHP inherited from the bad segmentations performed by other tools like WKH VHJPHQWHU WRNHQL]HU DQG WKH VHQWHQFHDOLJQHU )RU H[DPSOH WKH DOLJQPHQWIRU³DFWLYLGDGHVSHVTXHUDV´ computed by the algorithm was ³GH¿QLWLYDGHDVDFWLYLGDGHVGH´*LYHQWKHPLVVLQJZRUGPDUNHGE\ WKHDVWHULVNWKLVVHJPHQWFRXOGQRWEHFODVVL¿HGFRUUHFWO\DQGWKHUHIRUHIHOOLQWKHHUURUFODVV-XVWOLNHZLWKWKHFDVHDERYHZHGRQRW KDYHDQ\GHWDLORQWKHYDOLGLW\RUQRWRIWKHK\SRWKHVLV7DEOHVKRZV IXUWKHUH[DPSOHVRIWKLVVHJPHQWDWLRQSUREOHP$OOWKHVHFDVHVFDQEH VDIHO\LJQRUHGIRUWKHK\SRWKHVLVWHVW%HWZHHQSDUHQWKHVHVZHVKRZ MNO ,QWKHVHWDEOHVWKHOHIWFROXPQLVWKH6SDQLVKH[WUDFWHGSDLUDQGWRWKHULJKWLVWKHVHJPHQW H[WUDFWHGIURPWKH3RUWXJXHVHVLGH,QLWDOLFVZHJLYHHPSKDVLVWRWKHWUDQVODWLRQRIWKHWHUPV IURPWKH¿UVWFROXPQ Extraction of restricted lexical combinations by detecting non-compositionality of multiword expressions 55 WKHPLVVLQJZRUGV7KLVVHHPHGWREHDSUREOHPRQWKHFRUSXVVHJPHQWDWLRQDQGDOLJQPHQWSURFHVV +,-./ P1 23,46./7 89 :;Q><,:/R 7/S4/>:7 tTUTVW VXYZW Sector de o tabaco em (cru) derechos humanos promoção de os direitos de (o homem) productos pesqueros mercado de os produtos de (pesca) Seguridad Alimentaria Europeia para a Segurança de (a alimentação) Seguridad Alimentaria Europeia para a Segurança de (a alimentar) • 7KHDOJRULWKPLPSOHPHQWDWLRQLVQRWSUHSDUHGWR¿QGDOORFFXUUHQFHVRI WKHZRUGWUDQVODWLRQV7KLVPHDQVWKDWLIWZRVLPLODUSDLUVRFFXUOLNH ³cantidad superior”DQG³calidad superior”) the algorithm will use the ¿UVWWUDQVODWLRQSDLUWZLFHDOLJQLQJ³TXDOLGDGHVXSHULRU´ZLWK³cantidad superior”DQGQRWWKHFRUUHFW³calidad superior”7KLVLVLQGHHG a bug introduced by our implementation, but when it was detected it was too late to perform a complete new extraction and restart the manXDOHYDOXDWLRQ7KHUHIRUHWKH\ZHUHLJQRUHGIRURXUK\SRWKHVLVWHVW 7DEOH VKRZV VRPH RI WKHVH H[DPSOHV ,Q LWDOLFV RQ WKH ULJKW WKH DOLJQHGVHJPHQW +,-./ [1 23,46./7 89 4=7,.=S>/R 7/S4/>:7 • \H]XQLGDG(FRQyPLFD que institui a Comunidade Europeia cantidad superior em uma TXDQWLGDGHLQIHULRU tiempo completo de trabalho a tempo parcial tercera columna referidas em a coluna 2 QDYHJDFLyQPDUtWLPD afectos a a navegação aérea The translation, sometimes, uses a pronoun to refer to a noun used on DSUHYLRXVVHQWHQFHZKLOHWKHRULJLQDOVHQWHQFHUHSHDWVWKHQRXQ6HH 7DEOHIRUVRPHH[DPSOHV 5^ Joana Veloso / Alberto Simões / Álvaro Iriarte S. Table 4: Examples of misalignments resulted from the use of pronouns Estado membro legislação de esse Estado ciertos productos regime de esses produtos YDORUHVOtPLWH ou de esses valores , último caso (Pesse caso , segundo Estado legislação de HVVH(VWDGR • 6RPH WUDQVODWLRQ XQLWV ZHUH QRW UHDOO\ WUDQVODWHG ,Q VRPH FDVHV WKH Portuguese version included the text in Spanish, and in some other, LQ (QJOLVK DV VKRZQ LQ7DEOH 6RPH RWKHUV DV VKRZQ LQ7DEOH include typos that, not being in the dictionary, activated our hypothesis E\PLVWDNH +,-./ _1 23,46./7 89 :;,>7.,:=8> Q>=:7 ?@/;/ ,: ./,7: 8>/ 89 :@/ 7=R/7 =7 untranslated medio ambiente contaminación del medio ambiente Autoridades nacionales Lista de las autoridades nacionales medio ambiente en el medio ambiente acuático Vivo Test ,Q9LYR7HVWIRU&KURPRVRPDO Table 6: Examples of translations with minor typos, mistakenly extracted as collocations SUHVHQWHDUWtFXOR de o presnete artigo OHJtWLPRWLWXODU seu legítimo titual a ocupar ]RQDJHRJUi¿FD HVSHFt¿FDVHPXPD]RQDJHRJUi¿UFD presente Reglamento 2presidente regulamento é proyectos transnacionales acompanhamento de os projectos trannacionais Extraction of restricted lexical combinations by detecting non-compositionality of multiword expressions 5` 3.2 Free Combinations The main interference with the algorithm, which could make it extract free combinations, is the lack of translations from the used translation dictioQDU\:KHQDZRUGLVQRWIRXQGLQWKHGLFWLRQDU\HLWKHUDVWKHZRUGQRW existing in the source language – 7DEOH RU WKH WDUJHW ODQJXDJH GR QRW include the used translation – 7DEOHWKHDOJRULWKPFRQVLGHUVWKHWUDQVODWLRQWREHLQFRUUHFWDQGWKHUHIRUHLWFDQEHXVHGIRURXUK\SRWKHVLV$VLPilar problem occurred with words not correctly lemmatized, and therefore, QRWIRXQGLQWKHWUDQVODWLRQGLFWLRQDU\ Table 7: Basic examples where the dictionary lacked a entry for one of the words Segundo resultado resultado : segundo resultado : primer trimestre em o primeiro trimestre de presente Directiva requisitos de a presente directiva primeros párrafos os dois primeiros parágrafos podem presente apêndice o presente apêndice , os Table 8: Basic examples where the dictionary lacked one of the synonyms Texto pertinente EEE ) ( 7H[WRUHOHYDQWH siguiente texto a seguinte redacção : 0HGLGDVYLJHQWHV 0HGLGDVH[LVWHQWHV última fabricación a última data de fabrico &RQRFLPLHQWRVVXFLQWRV Conhecimentos sumários de as )LQDOO\ WKHUH LV \HW DQRWKHU SUREOHP UHODWHG ZLWK WKH WH[WXDO GHL[LV, where there is a reference to a different position in the text that, different ODQJXDJHVUHIHUWRLQGLIIHUHQWZD\VExamples are cuadro anterior/TXDGUR acima, fórmula anterior/fórmula acima and criterios anteriores/critérios acima. As anterior and acima are not direct translations, the algorithm extracted them as restricted combinations (although they are free combiQDWLRQV 5 Joana Veloso / Alberto Simões / Álvaro Iriarte S. 3.3 Restricted Combinations and Collocations 2WKHUWKDQWKHFRUUHFWUHVWULFWHGFRPELQDWLRQVWKHUHDUHWZRVSHFLDONLQGV that should be mentioned: • There are situations where the pair of words in Spanish has a single word translation in Portuguese, either because in Portuguese one of the words is usually omitted, or because there is a word with the complete PHDQLQJRIWKHWZR6SDQLVKZRUGV7KLVVLWXDWLRQZDVQDPHGreduction DQGKDSSHQVDIHZWLPHV([DPSOHVDUHVKRZQLQ7DEOH7KHVHZHUH FRQVLGHUHG UHVWULFWHG FRPELQDWLRQV7KH EHVW H[DPSOHV IURP7DEOH DUHWKH¿UVWDQGWKHODVW,Q3RUWXJXHVHDQGDOWKRXJKWKHUHLVWKHFRQcept of meio ambiente, it is usually used only as ambiente. And in the case of FLJDUURVSHTXHxRV, Portuguese as a word for that: cigarrilhas 7KHVHVLWXDWLRQVZHUHYDOLGDWHGPDQXDOO\LQ,$7(,QWHU$FWLYH7HUPLQRORJ\IRU(XURSH +,-./ a1 23,46./7 89 ;/RQ<:=8>71 7=:Q,:=8>7 ?@/;/ :?8 ?8;R7 ,;/ <8;;/<:.C translated by only one word • medio ambiente protecção de o ambiente , DXGLWRUtDPHGLRDPELHQWDOHV de ecogestão e auditoria ( titular opositor parte de o titular trabajo anual uma unidade de trabalho , FLJDUURVSHTXHxRV cigarrilhas e cigarros , There is another situation with nouns (mostly geographic) that were PLVWDNHQO\ H[WUDFWHG DV VKRZQ LQ 7DEOH 7KHVH ZHUH H[WUDFWHG EHFDXVH RI WKH ZD\ WKH QRXQV DUH WUDQVODWHG7KLV WDEOH VKRZV WKUHH FROXPQV 7KH ¿UVW WZR DUH 6SDQLVK DQG 3RUWXJXHVH DQG WKH WKLUG D GLUHFW3RUWXJXHVHWUDQVODWLRQRIWKH6SDQLVKWHUP Extraction of restricted lexical combinations by detecting non-compositionality of multiword expressions 5 Table 10: Examples of nouns whose translation was a problem for the algorithm Sudeste Asiático Ásia de o Sudeste Sudoeste Asiático 5HS~EOLFD)HGHUDO RULJLQiULRVGHD5HS~EOLFD)HGHUDWLYD República Federal FRQWLQHQWDOHVSDxRO Espanha continental continental espanhol • *LYHQWKDWZHGHFLGHGWRDQDO\]HELJUDPVWKHUHDUHVLWXDWLRQVZKHUH WKHELJUDPLVSDUWRIDELJJHUUHVWULFWHGFRPELQDWLRQ7KLVLVXVXDOO\ HDV\WRGHWHFWJLYHQWKHVSHFL¿FDUHDRIWKHXVHGFRUSRUDDQGJLYHQWKDW the Portuguese segment includes more words than the two existing in 6SDQLVK7DEOHVKRZVH[DPSOHV +,-./ 001 b,7/7 89 ;/7:;=<:/R <84-=>,:=8>7 ?=:@ 48;/ :@,> :?8 ?8;R7 gestión medioambiental o sistema de gestão ambiental producción homogénea A unidade de produção homogénea SROtWLFDDJUtFROD de a política agrícola comum ejecución forzosa de PHGLGDVGHH[HFXomRIRUoDGD residuos radiactivos Gestão de os resíduos radioactivos :KHQ WKH DOJRULWKP UHWXUQHG LQWHUHVWLQJ UHVXOWV UHWXUQLQJ UHVWULFWHG lexical combinations, we were unable to distinguish between collocations and other types of restricted combinations, like quasi-phrasemes and idioms (this last type was not found, probably given the type of the used FRUSXV 5 Joana Veloso / Alberto Simões / Álvaro Iriarte S. Table 12: Examples where the algorithm worked Disposiciones legales disposições legislativas mercado interior mercado interno cadena alimentaria &DGHLD$OLPHQWDU Derecho interno direito nacional derechos humanos direitos fundamentais contingentes arancelarios contingentes pautais fronteras interiors fronteiras internas entidad contratante entidade adjudicante GtDVKiELOHV dias úteis DxRVQDWXUDOHV anos civis SHUVRQDItVLFD pessoa singular peso neto SHVROtTXLGR DxRVQDWXUDOHV anos civis precio neto SUHoROtTXLGR SHUVRQDVMXUtGLFDV pessoas colectivas amarillo oscuro amarelo torrado VHQWHQFLD¿UPH sentença transitada petróleo crudo RXUHVtGXRVGHSHWUyOHREUXWR atún rojo atum rabilho correo normal correio ordinário historiales médicos processos médicos 3.4 Analysis :KHQDSSOLHGWRWKH(XURSHDQ&HQWUDO%DQNFRUSRUDRXUDSSURDFKH[WUDFWHG PRUHWKDQSDLUVRIFDQGLGDWHFROORFDWLRQV7KH\ZHUHHYDOXDWHGE\ exhaustion (instead of evaluating a sample of random entries, the evaluator WDJJHGHDFKRQHRIWKHH[WUDFWHGFDQGLGDWHV7KLVPHDQVWKHHYDOXDWLRQLV QRWDIIHFWHGE\VDPSOHELDV7KLVWRJHWKHUZLWKWKHIDFWWKDWWKHHYDOXDWRU gave preference to free combinations over restricted combinations, means WKDWWKLVHYDOXDWLRQLVWKHEDVHOLQHRIWKHDOJRULWKP 7DEOH SUHVHQWV WKH QXPEHU RI FDVHV IRXQG DQG FODVVL¿HG DFFRUGLQJWRHDFKRIWKHSUHYLRXVO\PHQWLRQHGFODVVHV,IZHLJQRUHWKHFDVHVRI errors, nouns and reductions, we can note that restricted combinations are RQHTXDUWHURIWKHWRWDOQXPEHURIIRXQGFRPELQDWLRQV Extraction of restricted lexical combinations by detecting non-compositionality of multiword expressions 5 Table 13: Summary of cases found for each category. &DWHJRU\ )UHH&RPELQDWLRQV 5HVWULFWHG&RPELQDWLRQ Errors Reductions Nouns 1XPEHURI2FFXUUHQFHV Percentage 19 4. Conclusions 7KH¿UVWUHDFWLRQWRWKHUHVXOWVZDVRIGLVFRQWHQWDVDORWRIIUHHFRPELQDWLRQVZHUHIRXQG$VVRRQDVWKHH[DPSOHVZHUHDQDO\]HGZDVUHDOL]HG ¿UVWO\WKHWUDQVODWLRQUHVRXUFHODFNVFRYHUDJHDQGVHFRQGO\WKHDOJRULWKP XVHGPLVVHVWKHFRUUHFWOHPPDWL]DWLRQIRUVRPHZRUGV7KHVHWZRUHDVRQV FDQEH¿[HGRUPDGHEHWWHUXVLQJRWKHUDSSURDFKHVRUWRROVIRUWKHOHPmatization, and using other dictionaries or even probabilistic translation GLFWLRQDULHV 6LP}HV $OPHLGD 6LP}HV$OPHLGD 5DPRV &DUYDOKRWRHQULFKWKHWUDQVODWLRQFRYHUDJH Nevertheless, most of the situations found are easy to correct, and, therefore, further experiments should be performed before considering the PHWKRG LQDGHTXDWH ,Q IDFW ZLOO EH LQWHUHVWLQJ WR VHH KRZ WKLV DSSURDFK performs in a less noisy corpus, with better dictionaries, and with other ODQJXDJHV 7KLV GDWD EHLQJ PDQXDOO\ FODVVL¿HG FDQ EH XVHG WR WUDLQ PDFKLQH OHDUQLQJ DOJRULWKPV )RU VWDWLVWLFDO PDFKLQH WUDQVODWLRQ LW LV SRVVLEOH WR denote/specify which segments should be reused directly without any change (when they are idiomatic), and which segments can be generalized, allowing some of the words to be replaced, and reusing the translation VWUXFWXUH)RUWKHH[WUDFWLRQRIIXUWKHUFROORFDWLRQVIURPRWKHUFRUSRUDWKLV data can be used to train a supervised machine learning algorithm, or just EHXVHGDVDJROGHQVWDQGDUGIRUWKLVNLQGRIV\VWHP Analyzing the results obtained, the initial starting hypothesis should EHUHIRUPXODWHG8VLQJWKLVDSSURDFKUHVWULFWHGFRPELQDWLRQVDQGQRWMXVW DVSHFL¿FW\SHRIUHVWULFWHGFRPELQDWLRQVVXFKDVWKHFDVHRIFROORFDWLRQV DUHGHWHFWDEOH2IFRXUVHDXWKRUVOLNH0HO¶þXNGH¿QHIRUPDOW\SHV IRUHDFKRQHRIWKHVHUHVWULFWHGOH[LFDOFRPELQDWLRQV7KHSUREOHPLVWKH 5 Joana Veloso / Alberto Simões / Álvaro Iriarte S. QRQH[LVWHQFHRIDFOHDUGLVWLQFWLRQEHWZHHQWKHP6RPHOH[LFDOFRPELQDWLRQVZLOOEHFODVVL¿HGGLIIHUHQWO\DFFRUGLQJWRWKHZD\WKHOLQJXLVWGHFRPSRVHVVHPDQWLFDOO\WKHH[SUHVVLRQ There is another problem with our hypothesis, when a restricted combination coincides in the two languages being analyzed, because they can EHPLVWDNHQO\FRQVLGHUHGIUHHOH[LFDOFRPELQDWLRQVExamples of this problem are GHVHJXQGDPDQRHPVHJXQGDPmRyGLRPRUWDOyGLRPRUWDO, amor FLHJRDPRUFHJR:HH[SHFWWKDWWKLVPD\QRWEHDSUREOHPZKHQXVLQJ parallel corpora including more distinct languages (Portuguese/English, 6SDQLVK(QJOLVKHWF References AGUILAR-AMAT,$/DVFRORFDFLRQHVGHQRPEUH\DGMHWLYR8QSDVRKDFLDXQD WHRULDOp[LFRVHPiQWLFDGHODWUDGXFFLyQ7HVHGHGRXWRUDPHQWR>PLFUR¿FKDV@ %DUFHORQD8QLYHUVLWDW$XWzQRPDGH%DUFHORQD6HUYHLGH3XEOLFDFLRQV ALONSO RAMOS,0/DV)XQFLRQHV/p[LFDVHQHOPRGHOROH[LFRJUi¿FRGH, 0HO¶þuk. 7HVHGHGRXWRUDPHQWR0DGULG81(' ARAÚJO,6$OPHLGD--6LP}HV$'LDV,$SUHVHQWDomRGRSURMHFWR3HU )LGH3DUDOHOL]DQGRRSRUWXJXrVFRPVHLVRXWUDVOtQJXDV/LQJXDPiWLFD CALDERÓN CAMPOS,0 Sobre la elaboración de diccionarios monolingües de SURGXFFLyQ/DVGH¿QLFLRQHVORVHMHPSORV\ODVFRORFDFLRQHVOp[LFDV*UDQDGD 8QLYHUVLGDGGH*UDQDGD CORBÍ-BELLOT,$0)RUFDGD0/2UWL]5RMDV63pUH]2UWL]-$6iQFKH]5DPtUH] *6iQFKH]0DUWtQH])et al. $QRSHQVRXUFHVKDOORZWUDQVIHUPDFKLQH WUDQVODWLRQHQJLQHIRUWKHURPDQFHODQJXDJHVRI6SDLQ3URFHHGLQJVRIWKH(XURSHDQ $VVRFLDWLRQIRU0DFKLQH7UDQVODWLRQWK$QQXDO&RQIHUHQFHSSBudapest: (XURSHDQ$VVRFLDWLRQIRU0DFKLQH7UDQVODWLRQ CORPAS PASTOR,*8QHVWXGLRSDUDOHORGHORVVLVWHPDVIUDVHROyJLFRVGHOLQJOpV \GHOHVSDxRO7HVHGHGRXWRUDPHQWR8QLYHUVLGDG&RPSOXWHQVHGH0DGULG >PLFUR¿FKDV@0iODJD8QLYHUVLGDGGH0iODJD6HUYLFLRGHSXEOLFDFLRQHVHLQWHUFDPELRFLHQWt¿FR COSERIU,(3ULQFLSLRVGHVHPiQWLFDHVWUXFWXUDO0DGULG*UHGRV COWIE,$30DFNLQ50F&DLJ,52[IRUG'LFWLRQDU\RI&XUUHQW,GLRPDWLF (QJOLVKYRO,,,*HQHUDO,QWURGXFWLRQ2[IRUG283 ETTINGER,6)RUPDFLyQGHSDODEUDV\IUDVHRORJtDHQODOH[LFRJUDItD,Q+DHQVFK *:ROI/(WWLQJHU6:HUQHU5(GV/DOH[LFRJUDItDDe la lingüística WHyULFDDODOH[LFRJUDItDSUiFWLFDSS0DGULG*UHGRV Extraction of restricted lexical combinations by detecting non-compositionality of multiword expressions 5 GUINOVART,;*6LP}HV$Parallel corpus-based bilingual terminology H[WUDFWLRQ,Q/¶+RPPH0&6]XOPDQ6(GVWK,QWHUQDWLRQDO&RQIHUHQFH RQ7HUPLQRORJ\DQG$UWL¿FLDO,QWHOOLJHQFH7RXORXVH)UDQFH1RYHPEHU JOHNSON,,0DFSKDLO$,$7(±,QWHU$JHQF\7HUPLQRORJ\([FKDQJH 'HYHORSPHQWRID6LQJOH&HQWUDO7HUPLQRORJ\'DWDEDVHIRUWKH,QVWLWXWLRQVDQG Agencies of the European Union. ,Q&KRL.6(G3URFHHGLQJVRIWKH:RUNVKRS RQ7HUPLQRORJ\UHVRXUFHVDQGFRPSXWDWLRQ/5(&&RQIHUHQFH$WKqQHV*UqFH LI,/6SRUOHGHU&&ODVVL¿HU&RPELQDWLRQIRU&RQWH[WXDO,GLRP'HWHFWLRQ :LWKRXW/DEHOOHG'DWD,Q.RHKQ30LKDOFHD5(GV3URFHHGLQJVRIWKH &RQIHUHQFHRQ(PSLULFDO0HWKRGVLQ1DWXUDO/DQJXDJH3URFHVVLQJSS 6LQJDSRUH$VVRFLDWLRQIRU&RPSXWDWLRQDO/LQJXLVWLFV LI,/6SRUOHGHU&8VLQJ*DXVVLDQ0L[WXUHPRGHOVWR'HWHFW)LJXUDWLYH /DQJXDJH LQ &RQWH[W ,Q +XPDQ /DQJXDJH7HFKQRORJLHV7KH $QQXDO &RQIHUHQFHRIWKH1RUWK$PHULFDQ&KDSWHURIWKH$VVRFLDWLRQIRU&RPSXWDWLRQDO /LQJXLVWLFVSS/RV$QJHOHV&DOLIRUQLD$VVRFLDWLRQIRU&RPSXWDWLRQDO /LQJXLVWLFV MEL’ýUK,,3KUDVHPHVLQ/DQJXDJHDQG3KUDVHRORJ\LQ/LQJXLVWLFV,Q(YHUDHUW 0YDQ'HU/LQGHQ(-6FKHDN$6FK]HQGHU(GV,GLRPV6WUXFWXUDODQG 3V\FKRORJLFDO3HUVSHFWLYHVSS+LOOVGDOH1HZ-HUVH\+RYH8. /DZUHQFH(UOEDXP$VVRFLDWHV MEL’ýUK,,&ODV$3ROJXqUH$,QWURGXFWLRQjOD/H[LFRORJLH([SOLFDWLYH et Combinatoire. /RXYDLQOD1HXYH'XFXORW MUZNY,*=HWWOHPR\HU/6$XWRPDWLF,GLRP,GHQWL¿FDWLRQLQ:LNWLRQDU\,Q WKH3URFHHGLQJVRIWKH&RQIHUHQFHRQ(PSLULFDO0HWKRGVLQ1DWXUDO/DQJXDJH 3URFHVVLQJ(01/3SS6HDWWOH:DVKLQJWRQ86$ PADRÓ,/6WDQLORYVN\()UHH/LQJ7RZDUGV:LGHU0XOWLOLQJXDOLW\. ,Q 3URFHHGLQJV RIWKH/DQJXDJH5HVRXUFHVDQG(YDOXDWLRQ&RQIHUHQFH/5(&(/5$,VWDQEXO 7XUNH\ PAILLARD,0&RWH[WHFROORFDWLRQVOH[LTXH,Q*XLPLHU(G&RWH[WHHWFDOFXO GXVHQV$FWHVGHODWDEOHURQGHWHQXHj&DHQOHVHWIpYULHUSS &DHQ3UHVVHV8QLYHUVLWDLUHVGH&DHQ PAVEL,3$Q([WHQVLYH(PSLULFDO6WXG\RI&ROORFDWLRQ([WUDFWLRQ0HWKRGV,Q 3URFHHGLQJVRIWKH$&/6WXGHQW5HVHDUFK:RUNVKRSSS6WURXGVEXUJ3$ 86$$VVRFLDWLRQIRU&RPSXWDWLRQDO/LQJXLVWLFV SERETAN,9:HKUOL($FFXUDWH&ROORFDWLRQ([WUDFWLRQ8VLQJD0XOWLOLQJXDO 3DUVHU,Q3URFHHGLQJVRIWKHVW,QWHUQDWLRQDO&RQIHUHQFHRQ&RPSXWDWLRQDO /LQJXLVWLFVDQGWKHWK$QQXDO0HHWLQJRIWKH$VVRFLDWLRQIRU&RPSXWDWLRQDO /LQJXLVWLFVSS6WURXGVEXUJ3$86$$VVRFLDWLRQIRU&RPSXWDWLRQDO /LQJXLVWLFV 5' Joana Veloso / Alberto Simões / Álvaro Iriarte S. SIMÕES,$$OPHLGD--1$7RROVDVWDWLVWLFDOZRUGDOLJQHUZRUNEHQFK3URFHVDPLHQWR GHO/HQJXDMH1DWXUDO SIMÕES,$$OPHLGD--5DPRV&DUYDOKR1'H¿QLQJDSUREDELOLVWLFWUDQVODWLRQ GLFWLRQDULHVDOJHEUD,Q&RUUHLD/5HLV/3&DVFDOKR-*RPHV/*XHUUD+ &DUGRVR3(GV;9,3RUWXJXHVH&RQIHUHQFHRQ$UWL¿FLDO,QWHOOLJHQFH±(3,$ SS$QJUDGR+HURLVPR3RUWXJDO SIMÕES,$&DUYDOKR1'HVHQYROYLPHQWRGHDSOLFDo}HVHP3HUOFRP)UHH/LQJ /LQJXDPiWLFD TOMASZCZYK,-2Q%LOLQJXDO'LFWLRQDULHV7KH&DVHIRU%LOLQJXDO'LFWLRQDULHV IRU)RUHLJQ/DQJXDJH/HDUQHUV,Q+DUWPDQQ(G/H[LFRJUDSK\3ULQFLSOHVDQG 3UDFWLFHSS/RQGRQ$FDGHPLF3UHVV >5HFHELGRHPGHIHYHUHLURGHHDFHLWHSDUDSXEOLFDomRHPGHDEULO GH@
© Copyright 2026 Paperzz