wi4dZddlZddlmZmZGddZy)zi Polish Text Normalizer Utilities for normalizing Polish text, handling diacritics, and common patterns. N)DictOptionalc eZdZdZidddddddd d d d d dddddddddddddddddddd d!d"d#d"iZd$d%d&d'd(d)d*d+d,Zd-d-d-d.d.d/d/d0Zed1ed2efd3Z ed1ed2efd4Z ed1ed2efd5Z ed1ed2e e eeffd6Ze d@d1ed7ed8ed9ed2ef d:Zed;ed2efd<Zed1ed2e efd=Zed1ed2e efd>Zy?)APolishNormalizerzZ Normalizes Polish text including diacritics, currency, and common abbreviations. uąaućcuęeułluńnóouśsuźzużuĄAuĆCuĘEuŁLuŃNÓOuŚSuŹZuŻulicaalejaplacosiedle mieszkaniepokoje powierzchniau województwo)zul.zal.zpl.zos.zm.zpok.zpow.zwoj.PLNEURUSD)uzłzlr u€r!$r"textreturnc|syg}|D]-}|j|jj||/dj|S)z Remove Polish diacritics from text. Args: text: Text with diacritics Returns: Text without diacritics )appendDIACRITICS_MAPgetjoin)clsr%resultchars S/var/www/extractly/manual_agregator/description_extrator/utils/polish_normalizer.pyremove_diacriticsz"PolishNormalizer.remove_diacritics.sHD MM#,,00t< =wwvc|sy|}|jjD]E\}}tjdtj|zdz||tj }G|S)z Expand common Polish abbreviations. Args: text: Text with abbreviations Returns: Text with expanded abbreviations r(z\b)flags) ABBREVIATIONSitemsresubescape IGNORECASE)r-r%r.abbrfulls r0normalize_abbreviationsz(PolishNormalizer.normalize_abbreviationsBs`++113JD$VVEBIIdO3e;T6QSQ^Q^_F4 r2cx|sy|}|jjD]\}}|j||}|S)z Normalize currency symbols to standard codes. Args: text: Text with currency symbols Returns: Text with normalized currency codes r()CURRENCY_PATTERNSr6replace)r-r%r.symbolcodes r0normalize_currencyz#PolishNormalizer.normalize_currencyVsC11779LFD^^FD1F: r2cX|syd}tj||tj}|ro|jdj ddj dd}|jd}|j j |d } t|}||d Sy#t$rYywxYw) z Extract price and currency from text. Args: text: Text containing price information Returns: Dict with 'amount' and 'currency' or None Nu.(\d+(?:[\s.,]\d+)*)\s*(zł|PLN|EUR|€|USD|\$) r(,.r )amountcurrency) r7searchr:groupr@r?r+float ValueError)r-r%patternmatch amount_str currency_rawrKrJs r0 extract_pricezPolishNormalizer.extract_pricejsD '47 Q//R8@@cJJ ;;q>L,,00uEH z*$ (  s B B)(B)r1expand_abbreviationsrCc|sy|}|r|j|}|r|j|}|r|j|}tjdd|j }|S)aj Apply all normalization steps to text. Args: text: Input text remove_diacritics: Whether to remove diacritics expand_abbreviations: Whether to expand abbreviations normalize_currency: Whether to normalize currency symbols Returns: Normalized text r(\s+rF)r=rCr1r7r8strip)r-r%r1rUrCr.s r0normalize_textzPolishNormalizer.normalize_textsj" 008F ++F3F **62FV,224 r2addressc|sy|j|}tjdd|}tjdd|j}|S)z Normalize Polish address format. Args: address: Raw address string Returns: Normalized address r(z\s*,\s*z, rWrF)r=r7r8rX)r-rZr.s r0normalize_addressz"PolishNormalizer.normalize_addresssR,,W5 D&1V,224 r2c|syd}tj||tj}|r-|jdj dd} t |Sy#t $rYywxYw)z Extract area (square meters) from text. Args: text: Text containing area information Returns: Area in square meters or None Nu-(\d+(?:[.,]\d+)?)\s*(?:m²|m2|mkw|metr[óo]w)rErGrH)r7rLr:rMr@rNrO)r-r%rPrQarea_strs r0 extract_areazPolishNormalizer.extract_areasmC '47 {{1~--c37H X&  s A A'&A'c|syd}tj||tj}|r t|j dSy#t $rYywxYw)z Extract number of rooms from text. Args: text: Text containing room information Returns: Number of rooms or None Nz%(\d+)\s*(?:pok[o.]|pokoi|pokoje|room)rE)r7rLr:intrMrO)r-r%rPrQs r0 extract_roomszPolishNormalizer.extract_roomss[; '47  5;;q>**  sA AAN)FTT)__name__ __module__ __qualname____doc__r*r5r? classmethodstrr1r=rCrranyrTboolrYr\rNr_rarbr2r0rr sa  c"C)-s48# c"C)-s c#C*.s59# c  #C *.s N M SS&33&cc&!!$sCx.)A!!F053715!#!)-!,0!+/!;>!!F04#r2r)rfr7typingrrrrkr2r0rms  !ttr2