謜iM ddlZddlZddlZddlZddlZddlmZmZmZddl m Z m Z ddl m Z ddl mZddlmZddlmZddlmZmZmZmZmZddlZed eZed eZd Zd ed e egeffdZddej@ej@dddeeezdzdedzdede gefde!dzd ejDf dZ#GddeZ$y)N)ArgumentDefaultsHelpFormatterArgumentParserArgumentTypeError)CallableIterable)copy)Enum)isclass)Path)AnyLiteralNewTypeUnionget_type_hints DataClass DataClassTypect|tr|S|jdvry|jdvrytd|d)N)yestruety1T)nofalsefn0FzTruthy value expected: got zJ but expected one of yes/no, true/false, t/f, y/n, 1/0 (case insensitive).) isinstanceboollowerr)vs U/mnt/e/genesis-system/.venv/lib/python3.12/site-packages/transformers/hf_argparser.pystring_to_boolr#$sO!Twwy22 4 4)!,v w  choicesreturncL|Dcic]}t||c}fdScc}w)aN Creates a mapping function from each choices string representation to the actual value. Used to support multiple value types for a single argument. Args: choices (list): List of choices. Returns: Callable[[str], Any]: Mapping function from string representation to actual value for each choice. c(j||SN)get)arg str_to_choices r"z+make_choice_type_function..=s}((c2r$)str)r%choicer,s @r"make_choice_type_functionr01s*8??VS[&(?M 22@s!)aliaseshelpdefaultdefault_factorymetadatar1r2r3r4r5c X|i}|||d<|||d<tjd|||d|S)aArgument helper enabling a concise syntax to create dataclass fields for parsing with `HfArgumentParser`. Example comparing the use of `HfArg` and `dataclasses.field`: ``` @dataclass class Args: regular_arg: str = dataclasses.field(default="Huggingface", metadata={"aliases": ["--example", "-e"], "help": "This syntax could be better!"}) hf_arg: str = HfArg(default="Huggingface", aliases=["--example", "-e"], help="What a nice syntax!") ``` Args: aliases (Union[str, list[str]], optional): Single string or list of strings of aliases to pass on to argparse, e.g. `aliases=["--example", "-e"]`. Defaults to None. help (str, optional): Help string to pass on to argparse that can be displayed with --help. Defaults to None. default (Any, optional): Default value for the argument. If not default or default_factory is specified, the argument is required. Defaults to dataclasses.MISSING. default_factory (Callable[[], Any], optional): The default_factory is a 0-argument function called to initialize a field's value. It is useful to provide default values for mutable types, e.g. lists: `default_factory=list`. Mutually exclusive with `default=`. Defaults to dataclasses.MISSING. metadata (dict, optional): Further metadata to pass on to `dataclasses.field`. Defaults to None. Returns: Field: A `dataclasses.Field` with the desired properties. r1r2)r5r3r4) dataclassesfield)r1r2r3r4r5kwargss r"HfArgr;@sOH%     khQ` kdj kkr$c HeZdZUdZeeed<ddeeezdzffd Zede de jfdZ defd Z dd eed ffd Zdd eeefded eed ffdZddeej.zded eed ffdZddeej.zded eed ffdZxZS)HfArgumentParsera This subclass of `argparse.ArgumentParser` uses type hints on dataclasses to generate arguments. The class is designed to play well with the native argparse. In particular, you can add more (non-dataclass backed) arguments to the parser after initialization and you'll get the output back after parsing as an additional namespace. Optional: To create sub argument groups use the `_argument_group_name` attribute in the dataclass. Args: dataclass_types (`DataClassType` or `Iterable[DataClassType]`, *optional*): Dataclass type, or list of dataclass types for which we will "fill" instances with the parsed args. kwargs (`dict[str, Any]`, *optional*): Passed to `argparse.ArgumentParser()` in the regular way. dataclass_typesNc |g}nt|ts|g}d|vr t|d<t|di|t j |r|g}t||_|jD]}|j|y)Nformatter_classr7) rrrsuper__init__r8 is_dataclasslistr>_add_dataclass_arguments)selfr>r:dtype __class__s r"rBzHfArgumentParser.__init__s  " OOX6./O F *(EF$ % "6"  # #O 4./O#O4)) 1E  ) )% 0 1r$parserr9c d|jg}d|jvr.|jd|jjdd|jj }t |j tr td|jdg}t |tr|g}t|j d|j }|tus,ttdrt |tjrt|j jvr\t!|j jdk7s!t d|j jvrt#d |jd t d|j jvrx|j jd tur|j jd n|j jd |_t|j d|j }nt$|j jvr{t d|j jd r|j jd n|j jd |_t|j d|j }i}|t&us4t |j t rt)|j t*r|t&ur|j j|d <n'|j Dcgc]}|j,c}|d <t/|d |d<|j0t2j4ur|j0|d<nd|d<n|j t$us|j t$dzk(rt |}t6|d<|j t$us*|j0U|j0t2j4ur8|j0t2j4urdn |j0}||d<d|d<d|d<nt9|rt)|t:rs|j jd |d<d|d<|j<t2j4ur|j=|d<n|j0t2j4urvd|d<np|j |d<|j0t2j4ur|j0|d<n5|j<t2j4ur|j=|d<nd|d<|j>g||i||j0durz|j t$us|j t$dzk(rQd|d<|j>d|jd|jjddfd|jd|yyycc}w)Nz--_-zpUnresolved type detected, which should have been done with the help of `typing.get_type_hints` method by defaultr1 __origin__ UnionTypezOnly `Union[X, NoneType]` (i.e., `Optional[X]`) is allowed for `Union` because the argument parser only supports one type per argument. Problem encountered in field 'z'.rr%typer3TrequiredF?nargsconst+z--no_z--no- store_false)actiondest) nameappendreplacer5rrrQr. RuntimeErrorpopgetattrrhasattrtypesrN__args__len ValueErrorrr issubclassr valuer0r3r8MISSINGr#r rDr4 add_argument) rIr9 long_optionsr:r1 origin_type bool_kwargsxr3s r"_parse_dataclass_fieldz'HfArgumentParser._parse_dataclass_fields UZZL)* %**    "UZZ%7%7S%A$B C D$$& ejj# &<  **Y+ gs #iGejj, C % GE;$?J{\a\k\kDl%**---EJJ''(A-d5::CVCV1V 66;jj\E Dz!4!447\>G>v> ==D ejjD&8EJJ$QU+r8rrrsrZitemsdelattr__dict__rd)rFrvreturn_remaining_stringslook_for_args_file args_filenameargs_file_flag args_filesargs_file_parsercfgcmd_args_file_pathsp file_args args_file namespaceremaining_argsoutputsrGrkeyskr!inputsobjs r"parse_args_into_dataclassesz,HfArgumentParser.parse_args_into_dataclassess`J ]/Ac#((mJ!!$}"56#CHH !!$sxx{"3"?"?"HI#1#3  --n3x-X-==4=H T&*3immN4I4I#4NPT&U#&%%8K&L1tAw&LMI' ? ##%!4!4!6!>I ? (,'79t#YRSRT=UD$($9$9t$9$D! >)) E$/$6$6u$=HqAFFHDH'+I'<'<'>Ltq!!t)adLFL & 1% &/&/C NN3    y!! "Q & NN9 % #-W-n- - #bcqbr!sttW; 9'MILs J8J J6 J J rvallow_extra_keysct|j}g}|jD]}tj|Dchc]}|j s|j }}|jD cic] \}} ||vs || } }} |j| j|di| } |j| |s|rtdt|t|Scc}wcc} }w)a< Alternative helper method that does not use `argparse` at all, instead uses a dict and populating the dataclass types. Args: args (`dict`): dict containing config values allow_extra_keys (`bool`, *optional*, defaults to `False`): Defaults to False. If False, will raise an exception if the dict contains keys that are not parsed. Returns: Tuple consisting of: - the dataclass instances in the same order as they were passed to the initializer. z0Some keys are not used by the HfArgumentParser: r7) setrr>r8rrrsrZrdifference_updater[rdsortedtuple) rFrvr unused_keysrrGrrrr!rrs r" parse_dictzHfArgumentParser.parse_dictfs $))+& )) E$/$6$6u$=HqAFFHDH'+zz|Atq!qDyadAFA  ) )&++- 8/&/C NN3    KOPVWbPcOdef fW~IAsC,C,7 C1C1 json_filectt|d5}tj|j }ddd|j |}t |S#1swY'xYw)at Alternative helper method that does not use `argparse` at all, instead loading a json file and populating the dataclass types. Args: json_file (`str` or `os.PathLike`): File name of the json file to parse allow_extra_keys (`bool`, *optional*, defaults to `False`): Defaults to False. If False, will raise an exception if the json file contains keys that are not parsed. Returns: Tuple consisting of: - the dataclass instances in the same order as they were passed to the initializer. zutf-8)encodingNr)openr jsonloadsreadrr)rFrropen_json_filedatars r"parse_json_filez HfArgumentParser.parse_json_files]"$y/G 4 5::n1134D 5//$9I/JW~ 5 5s $A!!A* yaml_filec|jtjt|j |}t |S)at Alternative helper method that does not use `argparse` at all, instead loading a yaml file and populating the dataclass types. Args: yaml_file (`str` or `os.PathLike`): File name of the yaml file to parse allow_extra_keys (`bool`, *optional*, defaults to `False`): Defaults to False. If False, will raise an exception if the json file contains keys that are not parsed. Returns: Tuple consisting of: - the dataclass instances in the same order as they were passed to the initializer. r)ryaml safe_loadr rr)rFrrrs r"parse_yaml_filez HfArgumentParser.parse_yaml_files7"//$..i1J1J1L"M`p/qW~r$r))NFTNN)F)__name__ __module__ __qualname____doc__rr__annotations__rB staticmethodrr8FieldrmrErrrdictr.r rrosPathLikerr __classcell__)rHs@r"r=r=os% m,,1 8O(ORV(V1"g~gk>O>OggR7m7.!& T y#~ TltCH~RWXacfXfRg8r{{):d_denpses_t,r{{):d_denpses_tr$r=)%r8rrrwraargparserrrcollections.abcrrrenumr inspectr pathlibr typingr r rrrrrrr#rDr.r0rgrrr;r=r7r$r"rs UU.??  K % -    3t 3#0D 3"'+&&)4)<)< ,l 49_t #,l *,l ,l b#g& ,l Tk ,l,l^{~{r$