i\FddlZddlmZddlZddlmZddlmZddlmZddl m Z ddl m Z dd l mZmZdd lmZdd lmZdd lmZmZmZdd lmZddlmZddlmZmZmZm Z ddl!m"Z"m#Z#ddl$m%Z%m&Z&m'Z'm(Z(ddl)m*Z*ddl+m,Z,Gdde,eZ-Gdde*Z.Gdde#Z/Gdde"Z0GddejbZ2Gdd eZ3Gd!d"eZ4Gd#d$e&Z5Gd%d&e%Z6Gd'd(e(Z7ed)*Gd+d,e7Z8Gd-d.e'Z9gd/Z:y)0N)Callable) Tokenizer)Unigram)nn)create_bidirectional_mask)BaseModelOutput)ALL_ATTENTION_FUNCTIONSPreTrainedModel)Unpack)TokenizersBackend)TransformersKwargsauto_docstringcan_return_tuple)merge_with_config_defaults)capture_outputs)LlamaAttentionLlamaRotaryEmbeddingapply_rotary_pos_embeager_attention_forward)ParakeetCTCConfigParakeetEncoderConfig)ParakeetEncoderBlock ParakeetEncoderConvolutionModuleParakeetForCTCParakeetPreTrainedModel)ParakeetProcessor) T5Tokenizerc deZdZ d fd Z d deeezdededzdedef dZxZ S) LasrTokenizerNc t |d|||||||d|tt|jdd|_y)N) eos_token unk_token pad_token extra_idsadditional_special_tokensvocab vocab_filerF)unk_id byte_fallback)super__init__rr _vocab_scores _tokenizer) selfr#r$r%r&r'r(r)kwargs __class__s a/mnt/e/genesis-system/.venv/lib/python3.12/site-packages/transformers/models/lasr/modular_lasr.pyr.zLasrTokenizer.__init__,s[  &?!  $ ""#   token_idsskip_special_tokensclean_up_tokenization_spaces group_tokensreturnc t|tr|g}|r%tj|Dcgc]}|d }}|Dcgc]}||jk7s|}}t j |f|||d|Scc}wcc}w)Nr)r6r7r8) isinstanceint itertoolsgroupby pad_token_idr _decode)r1r6r7r8r9r2 token_grouptokens r4rAzLasrTokenizer._decodeIs i %" I ;D;L;LY;WXKQXIX)2PuUd>O>O5OUP P ((   3)E     YQs A4A9A9)zzzdNNN)FNT) __name__ __module__ __qualname__r.r=listboolstrrA __classcell__r3s@r4r!r!+sl"& @%*48!  c? " '+Tk      r5r!c eZdZy) LasrProcessorNrErFrGr,r5r4rNrNbsr5rNc\eZdZdZddddddddd d d d d dd d d dddddgddgddffd ZxZS)LasrEncoderConfiga This is the configuration class to store the configuration of a [`LasrEncoder`]. It is used to instantiate a `LasrEncoder` model according to the specified arguments, defining the model architecture. Configuration objects inherit from [`PreTrainedConfig`] and can be used to control the model outputs. Read the documentation from [`PreTrainedConfig`] for more information. Args: hidden_size (`int`, *optional*, defaults to 512): Dimension of the layers and the hidden states. num_hidden_layers (`int`, *optional*, defaults to 17): Number of hidden layers in the Transformer encoder. num_attention_heads (`int`, *optional*, defaults to 8): Number of attention heads for each attention layer in the Transformer encoder. intermediate_size (`int`, *optional*, defaults to 2048): Dimension of the "intermediate" (often named feed-forward) layer in the Transformer encoder. hidden_act (`str` or `function`, *optional*, defaults to `"silu"`): The non-linear activation function (function or string) in the encoder and pooler. attention_bias (`bool`, *optional*, defaults to `False`): Whether to use bias in the attention layers. convolution_bias (`bool`, *optional*, defaults to `False`): Whether to use bias in convolutions of the conformer's convolution module. conv_kernel_size (`int`, *optional*, defaults to 32): The kernel size of the convolution layers in the Conformer block. subsampling_conv_channels (`int`, *optional*, defaults to 256): The number of channels in the subsampling convolution layers. subsampling_conv_kernel_size (`int`, *optional*, defaults to 5): The kernel size of the subsampling convolution layers. subsampling_conv_stride (`int`, *optional*, defaults to 2): The stride of the subsampling convolution layers. num_mel_bins (`int`, *optional*, defaults to 128): Number of mel features. dropout (`float`, *optional*, defaults to 0.1): The dropout ratio for all fully connected layers in the embeddings, encoder, and pooler. dropout_positions (`float`, *optional*, defaults to 0.0): The dropout ratio for the positions in the input sequence. layerdrop (`float`, *optional*, defaults to 0.1): The dropout ratio for the layers in the encoder. activation_dropout (`float`, *optional*, defaults to 0.1): The dropout ratio for activations inside the fully connected layer. attention_dropout (`float`, *optional*, defaults to 0.1): The dropout ratio for the attention layers. max_position_embeddings (`int`, *optional*, defaults to 10000): The maximum sequence length that this model might ever be used with. initializer_range (`float`, *optional*, defaults to 0.02): The standard deviation of the truncated_normal_initializer for initializing all weight matrices. layer_norm_eps (`float`, *optional*, defaults to 1e-06): The epsilon used by the layer normalization layers. feed_forward_residual_weights (`tuple[float, float]`, *optional*, defaults to `[1.5, 0.5]`): The residual weights for the feed forward layers. conv_residual_weights (`tuple[float, float]`, *optional*, defaults to `[2.0, 1.0]`): The residual weights for the convolution layers. batch_norm_momentum (`float`, *optional*, defaults to 0.01): The momentum for the batch normalization layers. rope_parameters (`RopeParameters`, *optional*): Dictionary containing the configuration parameters for the RoPE embeddings. The dictionary should contain a value for `rope_theta` and optionally parameters used for scaling in case you want to use RoPE with longer `max_position_embeddings`. Example: ```python >>> from transformers import LasrEncoderModel, LasrEncoderConfig >>> # Initializing a `LasrEncoder` configuration >>> configuration = LasrEncoderConfig() >>> # Initializing a model from the configuration >>> model = LasrEncoderModel(configuration) >>> # Accessing the model configuration >>> configuration = model.config ``` This configuration class is based on the LasrEncoder architecture from Google Health AI. You can find more details and pre-trained models at [TODO/TODO](https://huggingface.co/TODO/TODO). isiluF rg?i'g{Gz?gư>g?g?g@g?g{Gz?Nc ||_||_||_||_||_t |did|d|d|d|d|d|d|d|d | d | d | d | d | d|d|d|d|d|d|||`|`y)N hidden_sizenum_hidden_layersnum_attention_headsintermediate_size hidden_actattention_biasconvolution_biasconv_kernel_sizesubsampling_conv_channels num_mel_binssubsampling_conv_kernel_sizesubsampling_conv_stridedropoutdropout_positions layerdropactivation_dropoutattention_dropoutmax_position_embeddingsinitializer_ranger,) rope_parameterslayer_norm_epsfeed_forward_residual_weightsconv_residual_weightsbatch_norm_momentumr-r.subsampling_factor scale_input)r1r\r]r^r_r`rarbrcrdrfrgrerhrirjrkrlrmrnrprqrrrsror2r3s r4r.zLasrEncoderConfig.__init__s8 /,-J*%:"#6   # / !4 0  "  *  . . '@ & *F %<  0   2! "0# $%<% &0) .  #  r5)rErFrG__doc__r.rKrLs@r4rQrQfsfK^"%%& ! %'*Cj"Cj 3::r5rQcJeZdZdZ ddeezffd ZedZxZ S) LasrCTCConfiga This is the configuration class to store the configuration of a [`LasrForCTC`]. It is used to instantiate a Lasr CTC model according to the specified arguments, defining the model architecture. Configuration objects inherit from [`PreTrainedConfig`] and can be used to control the model outputs. Read the documentation from [`PreTrainedConfig`] for more information. Args: vocab_size (`int`, *optional*, defaults to 512): Vocabulary size of the model. ctc_loss_reduction (`str`, *optional*, defaults to `"mean"`): Specifies the reduction to apply to the output of `torch.nn.CTCLoss`. Only relevant when training an instance of [`LasrForCTC`]. ctc_zero_infinity (`bool`, *optional*, defaults to `True`): Whether to zero infinite losses and the associated gradients of `torch.nn.CTCLoss`. Infinite losses mainly occur when the inputs are too short to be aligned to the targets. Only relevant when training an instance of [`LasrForCTC`]. encoder_config (`Union[dict, LasrEncoderConfig]`, *optional*): The config object or dictionary of the encoder. pad_token_id (`int`, *optional*, defaults to 0): Padding token id. Also used as blank token id. Example: ```python >>> from transformers import LasrForCTC, LasrCTCConfig >>> # Initializing a Lasr configuration >>> configuration = LasrCTCConfig() >>> # Initializing a model from the configuration >>> model = LasrForCTC(configuration) >>> # Accessing the model configuration >>> configuration = model.config ``` This configuration class is based on the Lasr CTC architecture from Google Health AI. You can find more details and pre-trained models at [TODO/TODO](https://huggingface.co/TODO/TODO). encoder_configc 0t|d|||||d|y)N) vocab_sizectc_loss_reductionctc_zero_infinityryr@r,)r-r.)r1r{r|r}ryr@r2r3s r4r.zLasrCTCConfig.__init__s0  !1/)%    r5c4|jjdzS)Nr)ryrg)r1s r4inputs_to_logits_ratioz$LasrCTCConfig.inputs_to_logits_ratio%s""::A==r5)rRmeanTNr) rErFrGrvdictrQr.propertyrrKrLs@r4rxrxsCF!37  00  $>>r5rxc\eZdZdeffd Zdej dej fdZxZS)LasrEncoderSubsamplingconfigc(t|tj|j|j |_tj|j |j |j|j|_ tj|j |j|j|j|_ tj|j|j |_ tj|_y)N) kernel_sizestride)r-r.rLinearrer\dense_0Conv1drfrgconv_0rdconv_1dense_1ReLUact_fn)r1rr3s r4r.zLasrEncoderSubsampling.__init__+s yy!4!4f6H6HI ii      ;;11   ii     , ,;;11   yy!A!A6CUCUV ggi r5input_featuresr:c,|j|j|}|jdd}|j|j|}|j|j |}|jdd}|j |S)Nr)rr transposerrr)r1r hidden_statess r4forwardzLasrEncoderSubsampling.forward=sz DLL$@A %//15  DKK $>?  DKK $>? %//15 ||M**r5) rErFrGrQr.torchTensorrrKrLs@r4rr*s+ 0 $+ell+u||+r5rc eZdZy)LasrEncoderRotaryEmbeddingNrOr,r5r4rrFsr5rc eZdZdedeffd Z d dejdeejejfdzdejdzde e d eejejff d Z xZ S) LasrEncoderAttentionr layer_idxc4t|||d|_y)NF)r-r. is_causalr1rrr3s r4r.zLasrEncoderAttention.__init__Js +r5Nrposition_embeddingsattention_maskr2r:c |jdd}g|d|j}|j|j|j dd}|j |j|j dd}|j |j|j dd} |\} } t||| | \}}tj|jjt} | |||| |f|jsdn |j|jd|\} }| j g|dj#} |j%| } | |fS)NrrrZ)rhscaling)shapehead_dimq_projviewrk_projv_projrr get_interfacer_attn_implementationrtrainingrlrreshape contiguouso_proj)r1rrrr2 input_shape hidden_shape query_states key_states value_statescossinattention_interface attn_output attn_weightss r4rzLasrEncoderAttention.forwardNsk$))#2. 88b8$--8 {{=166|DNNqRST [[/44\BLLQPQR {{=166|DNNqRST &S#7 jRUWZ#[ j(?(M(M KK , ,.E) %8      % $}}C$2H2HLL %  % ! \*k));;;;FFH kk+. L((r5NN) rErFrGrQr=r.rrtupler rrrKrLs@r4rrIs0SIM.2 ")||")#5<<#=>E") t+ ") +, ") u||U\\) * ")r5rc&eZdZddeffd ZxZS)LasrEncoderConvolutionModulerct|||d|_tj|j |j |_y)Nsame)momentum)r-r.paddingr BatchNorm1dr\rsnorm)r1r module_configr3s r4r.z%LasrEncoderConvolutionModule.__init__ts7 / NN6#5#5@Z@Z[ r5N)rErFrGrQr.rKrLs@r4rrss\0\\r5rc eZdZdedeffd Z d dejdejdzdejdzdee d ejf d Z xZ S) LasrEncoderBlockrrcTt||||j|_|j|_t j |j |jd|_t j |j |jd|_ t j |j |jd|_ t j |j |jd|_ t j |j |jd|_ y)NF)bias) r-r.rqrrr LayerNormr\rpnorm_feed_forward1 norm_self_att norm_convnorm_feed_forward2norm_outrs r4r.zLasrEncoderBlock.__init__{s +-3-Q-Q*%+%A%A""$,,v/A/A6CXCX_d"e\\&*<*S>SZ_`f&8&8&:O:OV[\"$,,v/A/A6CXCX_d"e V%7%79N9NUZ[ r5Nrrrr2r:c 0|}|j|j|}|jd|z|jd|zz}|j|}|jd|||d|\}}||z}|j |j ||} |jd|z|jd| zz}|}|j|j|}|jd|z|jd|zz}|j|}|S)Nrr)rrr)rr,) feed_forward1rrqr self_attnconvrrr feed_forward2rr) r1rrrr2residualnormalized_hidden_statesr_ conv_outputs r4rzLasrEncoderBlock.forwardsJ!**4+B+B=+QR  . .q 1H >> from transformers import AutoProcessor, LasrEncoder >>> from datasets import load_dataset, Audio >>> model_id = TODO >>> processor = AutoProcessor.from_pretrained(model_id) >>> encoder = ParakeetEncoder.from_pretrained(model_id) >>> ds = load_dataset("hf-internal-testing/librispeech_asr_dummy", "clean", split="validation") >>> ds = ds.cast_column("audio", Audio(sampling_rate=processor.feature_extractor.sampling_rate)) >>> inputs = processor(ds[0]["audio"]["array"]) >>> encoder_outputs = encoder(**inputs) >>> print(encoder_outputs.last_hidden_state.shape) ``` r)devicer)pr) target_length)r inputs_embedsrFT)rr)last_hidden_state)rrrarangerr unsqueezer functionalrhrri_get_output_attention_maskrrrrandrjrr ) r1rrr2rrr encoder_layerto_dropdropout_probabilitys r4rzLasrEncoder.forwards>7 ?? 5<< (;(;A(>}G[G[\ffghi S --mt||VZVcVc-d mm##C4+A+ADMM#Zmm##C4+A+ADMM#Z  %!<<^[h[n[nop[q<rN2;;')  "[[ MG}}&+jjn#&7"G -!!#1),c ! !    m4 ??r5r)rErFrGrQ__annotations__base_model_prefixr.rrrrrrr rr rrKrLs@r4rrs !0"/3?@ ?@ t+?@+, ?@  ?@ ?@r5rceZdZfdZxZS) LasrForCTCc 8tjdi|S)a Example: ```python >>> from transformers import AutoProcessor, LasrForCTC >>> from datasets import load_dataset, Audio >>> model_id = TODO >>> processor = AutoProcessor.from_pretrained(model_id) >>> model = LasrForCTC.from_pretrained(model_id) >>> ds = load_dataset("hf-internal-testing/librispeech_asr_dummy", "clean", split="validation") >>> ds = ds.cast_column("audio", Audio(sampling_rate=processor.feature_extractor.sampling_rate)) >>> inputs = processor(ds[0]["audio"]["array"], text=ds[0]["text"]) >>> predicted_ids = model.generate(**inputs) >>> transcription = processor.batch_decode(predicted_ids, skip_special_tokens=True) >>> print(transcription) ``` r,)r-generate) super_kwargsr3s r4rzLasrForCTC.generates, uw/,//r5)rErFrGrrKrLs@r4rrs 00r5r)rrrrNrQrxr!);r>collections.abcrr tokenizersrtokenizers.modelsrr masking_utilsrmodeling_outputsr modeling_utilsr r processing_utilsr tokenization_utils_tokenizersr utilsrrr utils.genericrutils.output_capturingrllama.modeling_llamarrrrparakeet.configuration_parakeetrrparakeet.modeling_parakeetrrrrparakeet.processing_parakeetrt5.tokenization_t5rr!rNrQrxModulerrrrrrrr__all__r,r5r4rs$ %6/F&>II75vvV =,4 K!24 n % H-HV6>%6>r+RYY+8')T\#C\.+.b1& X@%X@  X@v004 r5