鬜ieB ddlmZddlmZddlmZddlmZddlm Z e rddl Z eje Z d d e jd e jd e jdzd ed e jf dZde j"j$de jde jde jd e jf dZ d!d e jd e jde jdzd e jfdZ d"d e jd e jd e jdzde jdzd ed e jf dZde j"j$de jde jde jd e jf dZGddeZeZde jd e jfdZ d!ddddee j"j$dzd eded ee j"j$fdZy)#)Callable)wraps)logging)GeneralInterface)is_torch_availableNFinputweightbias is_transposedreturnc|r5tj|jd|jd}n4tj||jdjd}|||z}|S)aBatched linear layer supporting optional bias and transposed weights. Args: input (`torch.Tensor`): Input tensor of shape (batch_size, input_dim). weight (`torch.Tensor`): Weight tensor of shape (batch_size, output_dim, input_dim) if transposed is `False`, else of shape (batch_size, input_dim, output_dim). bias (`torch.Tensor`, *optional*): Bias tensor of shape (batch_size, output_dim). Default is `None`. is_transposed (`bool`, *optional*, defaults to `False`): Whether the weight tensor is transposed. Returns: `torch.Tensor`: Output tensor of shape (batch_size, output_dim). )torchbmm unsqueezesqueeze)r r r r outs Y/mnt/e/genesis-system/.venv/lib/python3.12/site-packages/transformers/integrations/moe.py_batched_linearrDse*ii*F3;;A>ii 34<|j}|jd}|jd}|jd}tj||j dj d|j d}|j d} |j d} | |jk} | jd|jdz } ||} |j| }|j| }|jr|j| nd}|jr|j| nd}t| |||j}|j!|}t||||j}| j"| j"k7r| j%d| } || j dz}|| j dj'|j(z}|j+|||j-d}|j'|j(S)Nrrdevicerr dim)rsizerarangerexpandreshape num_expertsclamp gate_up_proj down_projhas_biasgate_up_proj_biasdown_proj_biasrr _apply_gateshapegathertodtypeviewsum)rrrrr num_top_k num_tokens hidden_dim token_idxsample_weights expert_ids valid_maskexpert_ids_clampedselected_hidden_statesselected_gate_up selected_downselected_gate_up_biasselected_down_bias gate_up_out gated_outout_per_samplefinal_hidden_statess rbatched_mm_experts_forwardrFfs  ! !F  $I##A&J##B'J Z7AA!DKKBPYZbbcefI"**2.N$$R(Jd...J#))!T-=-=-AB+95(();<NN#56MJN--D223EF]aDHMM,,-?@W[" 02GW[WiWiK   -I%="4DDVDVN 1777'..q2DE#n&>&>r&BBN#j&:&:2&>&A&A.BVBV&WWN)--j)ZPTTYZT[  ! !-"5"5 66roffsclttjjdrEtjjj |j |j ||Sttdr1tj|j |j ||Std)aGrouped matrix multiplication dispatcher that uses torch.nn.functional.grouped_mm if available, else falls back to torch._grouped_mm. Args: input (`torch.Tensor`): Input tensor of shape (S, input_dim). weight (`torch.Tensor`): Weight tensor of shape (num_experts, output_dim, input_dim). offs (`torch.Tensor`, *optional*): Offsets tensor indicating the boundaries of each group in the input tensor. Returns: `torch.Tensor`: Output tensor of shape (S, output_dim). Raises: ImportError: If neither `torch.nn.functional.grouped_mm` nor `torch._grouped_mm` is available, indicating that the PyTorch version is incompatible. grouped_mmrG _grouped_mmzNeither torch.nn.functional.grouped_mm nor torch._grouped_mm is available. Please make sure you are using a PyTorch version that includes grouped_mm (2.9+).) hasattrrnn functionalrIr1r2rK ImportError)r r rGs rrKrKs,uxx""L1xx""--ehhv||.DfSW-XX  &  &,,!7dKK `  rcr|rt|||}nt||jdd|}|||z}|S)a*Grouped linear layer supporting optional bias and transposed weights. Args: input (`torch.Tensor`): Input tensor of shape (S, input_dim). weight (`torch.Tensor`): Weight tensor of shape (num_experts, output_dim, input_dim) if transposed is `False`, else of shape (num_experts, input_dim, output_dim). bias (`torch.Tensor`, *optional*): Bias tensor of shape (num_experts, output_dim). Default is `None`. offs (`torch.Tensor`, *optional*): Offsets tensor indicating the boundaries of each group in the input tensor. is_transposed (`bool`, *optional*, defaults to `False`): Whether the weight tensor is transposed. Returns: `torch.Tensor`: Output tensor of shape (S, output_dim). rJr)rK transpose)r r r rGr rs r_grouped_linearrSsF0%d3%!1!1"b!9E Dj Jrc|j}|jd}|jd}|jd}tj||j dj d|j d}|j d} |j d} ||} tj| } tj| } | | }| | }| | }|j}|j}|jr|j|nd}|jr|j|nd}|jdk(r|jn|j}tj ||j"d|j"dz }tj$|dtj&}t)|||||j*}|j-|}t)|||||j*}||j dz}|| }|j/|||j1d }|j3|j4S) Nrrrrcpu)binsminmax)r"r2r r!)rr#rr$rr%r&argsortr)r*r+r,r-typefloatinthistcr'cumsumint32rSr r.r3r4r1r2)rrrrrr5r6r7r8r9r:r=perminv_perm expert_ids_gsample_weights_gselected_hidden_states_gr>r?r@rA histc_inputnum_tokens_per_expertoffsetsrBrCout_per_sample_grDrEs rgrouped_mm_experts_forwardrisE  ! !F  $I##A&J##B'J Z7AA!DKKBPYZbbcefI"**2.N$$R(J+95 == $D}}T"Hd#L%d+5d;((NNMDHMMD22<@W[>Bmm,,\:QU +1++*>,$$&LDTDTDVK!KK $:J:JPQW[WgWgjkWklll0au{{KG" "24I7bfbtbtK   -I'="4gTM_M_ (*:*D*DR*HH&h/N)--j)ZPTTYZT[  ! !-"5"5 66rc:eZdZdZeedZdededeffd Z xZ S)ExpertsInterfacez9Interface for registering custom experts implementations.) batched_mmrIexperts_implementationdefaultr c|tjdn|dk7r||vrtd|dt|||S)zfReturn the requested `experts_implementation`. Also strictly check its validity, and raise if invalid.a You tried to access the `ExpertsInterface` with a `config._experts_implementation` set to `None`. This is expected if you use an Expert Module as a standalone Module. If this is not the case, something went wrong with the dispatch of `config._experts_implementation`eager`zL` is not a valid experts implementation registered in the `ExpertsInterface`)logger warning_onceKeyErrorsuperget)rrmrn __class__s r get_interfacezExpertsInterface.get_interface4s` ! )   N  $w .3IQU3U*++wx w{17;;r) __name__ __module__ __qualname____doc__rFri_global_mappingstrrrx __classcell__)rws@rrkrk,s4C10O .wrapper..__init__is- $ 8 8 8 DK$DM!.D rcptj|jj}||g|i|Sr)ALL_EXPERTS_FUNCTIONSrxr_experts_implementation)rrrexperts_forwardoriginal_forwards rforwardz.wrapper..forwardps:3AA 335EO#49$9&9 9rr.)rrrrLrr.)rrrrrr+r s @@rwrapperz+use_experts_implementation..wrapperesu%.. (00 }  /  /   : ! : }m4(;M %!)  ' r)rZrrMModule)rr r+rs `` ruse_experts_implementationrTsH"tEHHOO4ehhoo9N0 }%% Nr)NFr)NNF)collections.abcr functoolsrutilsr utils.genericrutils.import_utilsrr get_loggerryrrTensorboolrrMrrFrKrSrirkrrrZrrrrsU%,3   H %Z!%  << LL ,,    \\ D67 ((//67<<6767<< 67  \\ 67x!% << LL  ,,   \\  H!% $ # <<# LL# ,, # ,,  #  #  \\ #LC7 ((//C7<<C7C7<< C7  \\ C7L<'<.)* "5<< "ELL "37,QVin,(4/,JN,bf, %((//,r