cidZddlmZddlmZddlmZddlmZGddejZ Gdd ejZ Gd d ejZ y ) zModel-based Pointwise Metric.)Union) constants)_base)metric_prompt_templatecJeZdZdZdedeejefffd ZxZ S)PointwiseMetricaA Model-based Pointwise Metric. A model-based evaluation metric that evaluate a single generative model's response. For more details on when to use model-based pointwise metrics, see [Evaluation methods and metrics](https://cloud.google.com/vertex-ai/generative-ai/docs/models/determine-eval). Usage Examples: ``` candidate_model = GenerativeModel("gemini-1.5-pro") eval_dataset = pd.DataFrame({ "prompt" : [...], }) fluency_metric = PointwiseMetric( metric="fluency", metric_prompt_template=MetricPromptTemplateExamples.get_prompt_template('fluency'), ) pointwise_eval_task = EvalTask( dataset=eval_dataset, metrics=[ fluency_metric, MetricPromptTemplateExamples.Pointwise.GROUNDEDNESS, ], ) pointwise_result = pointwise_eval_task.evaluate( model=candidate_model, ) ``` metricrc(t|||y)aInitializes a pointwise evaluation metric. Args: metric: The pointwise evaluation metric name. metric_prompt_template: Pointwise metric prompt template for performing the model-based evaluation. A freeform string is also accepted. )rr N)super__init__)selfr r __class__s S/tmp/pip-target-z3e9_cxr/lib/python/vertexai/evaluation/metrics/pointwise_metric.pyr zPointwiseMetric.__init__=s #9  ) __name__ __module__ __qualname____doc__strrmetric_prompt_template_basePointwiseMetricPromptTemplater __classcell__rs@rrrs:@  !& ' E Es J!   rrcfeZdZdZej j ZdddddededeffdZ xZ S) CometzzA COMET metric. Evaluates a score for the given instance using https://huggingface.co/Unbabel/wmt22-comet-da COMET_22_SRC_REFNversionsource_languagetarget_languagerrr cHt|tj|||y)a.Initializes the COMET metric. Args: version: The COMET version to use for evaluation eg. "COMET_22_SRC_REF". source_language: Optional. The source language of the translation. target_language: Optional. The target language of the translation. namerrr N)r r r _metric_namer rrr rs rr zComet.__init__[s) ##++  r) rrrrrMetricCOMETr$rr rrs@rrrRsN ##))L *##        rrcfeZdZdZej j ZdddddededeffdZ xZ S) MetricXzyA MetricX metric. Evaluates a score for the given instance using https://github.com/google-research/metricx METRICX_24_SRC_REFNrrrr cHt|tj|||y)afInitializes the MetricX metric. Args: version: The MetricX version to use for evaluation. Can be one of "METRICX_24_SRC_REF", "METRICX_24_SRC", or "METRICX_24_REF". source_language: Optional. The source language of the translation. target_language: Optional. The target language of the translation. r"N)r r r)r$r%s rr zMetricX.__init__|s) %%++  r) rrrrrr&METRICXr$rr rrs@rr)r)ssN ##++L ,##        rr)N)rtypingrvertexai.evaluationrvertexai.evaluation.metricsrrr_ModelBasedMetricr_TranslationMetricrr)rrr3sP"$)- 3 e--3 l E $ $ B e&& r