i3 d dl Zd dlZddlmZ ddlmZ ddlmZm Z m
Z
ddl m Z e rd dl
Z
dZdZ e
j e Z e d
G d d e Zd gZy)
N )SequenceFeatureExtractor) BatchFeature)
TensorTypeis_librosa_availablelogging)requiresgh㈵>g p>)torchlibrosa)backendsc e Zd ZdZddgZ d fd ZddZ ddej e e
z e ej z e e e
z de d e dz d
e
ez dz d e dz d e
dz d
e dz de dz de dz de
dz de dz defdZ xZS )ParakeetFeatureExtractora
Constructs a Parakeet feature extractor.
This feature extractor inherits from [`~feature_extraction_sequence_utils.SequenceFeatureExtractor`] which contains
most of the main methods. Users should refer to this superclass for more information regarding those methods.
This class extracts mel-filter bank features from raw speech using a custom numpy implementation of the `Short Time
Fourier Transform` which should match pytorch's `torch.stft` equivalent.
Args:
feature_size (`int`, *optional*, defaults to 80):
The feature dimension of the extracted features.
sampling_rate (`int`, *optional*, defaults to 16000):
The sampling rate at which the audio files should be digitalized expressed in hertz (Hz).
hop_length (`int`, *optional*, defaults to 160):
Length of the overlapping windows for the STFT used to obtain the Mel Frequency coefficients.
n_fft (`int`, *optional*, defaults to 512):
Size of the Fourier transform.
win_length (`int`, *optional*, defaults to 400):
The window length for the STFT computation.
preemphasis (`float`, *optional*, defaults to 0.97):
A preemphasis filter coefficient. 0.0 means no preemphasis filter.
padding_value (`float`, *optional*, defaults to 0.0):
Padding value used to pad the audio. Should correspond to silences.
input_featuresattention_maskc " t
| d|||d| || _ || _ || _ || _ t j j |||d|dz d } t j | j t j | _
y )N) feature_size
sampling_rate
padding_value slaney)srn_fftn_melsfminfmaxnorm )super__init__
hop_lengthr
win_length preemphasisr filtersmelr
from_numpytofloat32 mel_filters) selfr r r! r r" r# r kwargsr) __class__s t/mnt/e/genesis-system/.venv/lib/python3.12/site-packages/transformers/models/parakeet/feature_extraction_parakeet.pyr z!ParakeetFeatureExtractor.__init__A s wl-_lwpvw$
$& oo)) E,S}_`O`go *
!++K8;;EMMJ c t j | j d| }t j || j | j
| j |dd }t j | }t j |j d j d }|j d }| j j | }||z }t j |t z }|j ddd }|S )
NF)periodicdeviceTconstant)r! r" windowreturn_complexpad_moder r )r
hann_windowr" stftr r! view_as_realsqrtpowsumr) r' logLOG_ZERO_GUARD_VALUEpermute)r* waveformr1 r3 r9
magnitudesr) mel_specs r- _torch_extract_fbank_featuresz6ParakeetFeatureExtractor._torch_extract_fbank_featuresc s ""4??U6Rzz JJ
''-
ZZ
q 1 5 5b 9:
^^A&
&&))&1 +99X(< <= ##Aq!,r. N
raw_speech
truncationpad_to_multiple_ofreturn_tensorsreturn_attention_maskpadding
max_lengthr do_normalizer1 return_token_timestampsreturnc |O|| j k7 rmt d| j j d| j d| j d| d t j d| j j d t
|t j rt j | }nUt
|t t f r?t
|d t j r"|D
cg c] }
t j |
}}
t
|t j xr t |j d
kD }|rVt |j d kD r>t j d | j j d
|j! d }t
|t t f }|r^|D ]Y }
t |
j d
kD s t j d | j j d
|
j! d }
[ |s|r4|D
cg c]( }
|
dddf j# t j$ * }}
n'|dddf j# t j$ g}|D
cg c]
}
t |
}}
t' ||d }| j) |||||d }|j* j- d }| j. t j0 |j d
|j2 j5 d |j6 j5 d
k }t j8 |dddd
f |ddd
df | j. |ddddf z z
gd
}|j; | d }| j= ||
}t j> |j6 | j@ d z d z z | j@ z
| jB }t j0 |j d
|
dddf |dddf k }|j5 d }||z }|jE d
|j5 d z }|j5 d
}||z
d z |z jE d
|d
z
j5 d z }t jF | j5 d
}||z
|tH z z }||z }t' ||d| S c c}
w c c}
w c c}
w )a
Main method to featurize and prepare for the model one or several sequence(s). Implementation uses PyTorch for
the STFT computation if available, otherwise a slower NumPy based one.
Args:
raw_speech (`np.ndarray`, `list[float]`, `list[np.ndarray]`, `list[list[float]]`):
The sequence or batch of sequences to be padded. Each sequence can be a numpy array, a list of float
values, a list of numpy arrays or a list of list of float values. Must be mono channel audio, not
stereo, i.e. single float per timestep.
truncation (`bool`, *optional*, default to `True`):
Activates truncation to cut input sequences longer than *max_length* to *max_length*.
pad_to_multiple_of (`int`, *optional*, defaults to None):
If set will pad the sequence to a multiple of the provided value.
This is especially useful to enable the use of Tensor Cores on NVIDIA hardware with compute capability
`>= 7.5` (Volta), or on TPUs which benefit from having sequence lengths be a multiple of 128.
return_attention_mask (`bool`, *optional*):
Whether to return the attention mask. If left to the default, will return the attention mask according
to the specific feature_extractor's default.
[What are attention masks?](../glossary#attention-mask)
For Parakeet models, `attention_mask` should always be passed for batched inference, to avoid subtle
bugs.
return_tensors (`str` or [`~utils.TensorType`], *optional*):
If set, will return tensors instead of list of python integers. Acceptable values are:
- `'tf'`: Return TensorFlow `tf.constant` objects.
- `'pt'`: Return PyTorch `torch.Tensor` objects.
- `'np'`: Return Numpy `np.ndarray` objects.
sampling_rate (`int`, *optional*):
The sampling rate at which the `raw_speech` input was sampled. It is strongly recommended to pass
`sampling_rate` at the forward call to prevent silent errors and allow automatic speech recognition
pipeline.
padding_value (`float`, *optional*, defaults to 0.0):
The value that is used to fill the padding values / vectors.
do_normalize (`bool`, *optional*, defaults to `False`):
Whether or not to zero-mean unit-variance normalize the input. Normalizing can help to significantly
improve the performance of the model.
device (`str`, *optional*, defaults to `'cpu'`):
Specifies the device for computation of the log-mel spectrogram of audio signals in the
`_torch_extract_fbank_features` method. (e.g., "cpu", "cuda")
return_token_timestamps (`bool`, *optional*, defaults to `None`):
Deprecated. Use `return_attention_mask` instead from which the number of frames can be inferred.
Whether or not to return the number of frames of the input raw_speech.
These num_frames can be used by the model to compute word level timestamps.
Nz3The model corresponding to this feature extractor: z& was trained using a sampling rate of zI. Please make sure that the provided `raw_speech` input was sampled with z and not .zDIt is strongly recommended to pass the `sampling_rate` argument to `zN()`. Failing to do so can result in silent errors that might be hard to debug.r r7 r z2Only mono-channel audio is supported for input to z;. We will take the mean of the channels to convert to mono.r6 )r
audio_lengthspt)rJ rK rF rG rH )r1 )dimr )r r )data tensor_type)%r
ValueErrorr, __name__loggerwarning
isinstancenpndarrayr
tensorlisttupleTensorlenshapemeanr' r( r padr squeezer# aranger1 unsqueezerQ cat masked_fillrD floor_divider r! r= r; EPSILON) r* rE rF rG rH rI rJ rK r rL r1 rM r+ speechis_batched_torchis_batched_sequencerQ batched_speech
padded_inputsr timemaskfeatures_lengthsr maskinput_features_maskedrc variancestds r- __call__z!ParakeetFeatureExtractor.__call__ s H $ 2 22 I$..JaJaIb c))-););(< =))-););(V>
>%Vc&de !! 1
!
'55==bA ' ||N$8$8$;NDYDYZdd ++55a89H #YY 2A2&q!"u(=@P@PSabcehfhehbhSi@i(ijpq N , 7 7 3 GN ;;NFS -- ' '$**/A*= =
JDOO
n&:&:1&=fMdTUgVYijkmqjqYrr ''+ . 5 $((Q(/2B2L2LR2PP~~a *T1a7$>CCCJN^abNbMmMmnpMqqjj",,Q/(4/C'MB$ "0"0 '
A I* V ?s & S
-S)S)P i> i i g
ףp=
?r )cpu)
FNNNlongestNNNrz N)rW
__module__ __qualname____doc__model_input_namesr rD r[ r\ r^ floatboolintstrr r rw
__classcell__)r, s @r- r r # s4 4 *+;< KD> !)-26-1 '!%$($( "/3[
JJe,tBJJ/??$tE{BSS[
[
$J [
j(4/ [
$d{
[
t[
$J[
Tz[
Tk[
d
[
"&[
[
r. r )numpyr[ r
!feature_extraction_sequence_utilsr feature_extraction_utilsr utilsr r r utils.import_utilsr r rk r?
get_loggerrW rX r __all__r r. r- r sx I 4 > > *
H %
'(v
7 v
)v
r &
&r.