Package com.google.genkit.ai.evaluation

Class EvaluationManager

java.lang.Object

com.google.genkit.ai.evaluation.EvaluationManager

public class EvaluationManager
extends Object

Manages the execution of evaluations.

The EvaluationManager coordinates running evaluations by:

• Loading datasets from the dataset store
• Running inference on the target action
• Executing evaluators on the results
• Storing evaluation results

Constructor Summary

Constructors

Constructor	Description
EvaluationManager(Registry registry)	Creates a new EvaluationManager.
EvaluationManager(Registry registry, DatasetStore datasetStore, EvalStore evalStore)	Creates a new EvaluationManager with custom stores.

Method Summary

Modifier and Type	Method	Description
DatasetStore	getDatasetStore()	Gets the dataset store.
EvalStore	getEvalStore()	Gets the eval store.
EvalRunKey	runEvaluation(RunEvaluationRequest request)	Runs a new evaluation.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

EvaluationManager

public EvaluationManager(Registry registry)

Creates a new EvaluationManager.

Parameters:

registry - the Genkit registry

EvaluationManager

public EvaluationManager(Registry registry,
 DatasetStore datasetStore,
 EvalStore evalStore)

Creates a new EvaluationManager with custom stores.

Parameters:

registry - the Genkit registry

datasetStore - the dataset store

evalStore - the eval store

Method Details

runEvaluation

public EvalRunKey runEvaluation(RunEvaluationRequest request)
                         throws Exception

Runs a new evaluation.

Parameters:

request - the evaluation request

Returns:

the evaluation run key

Throws:

Exception - if evaluation fails

getDatasetStore

public DatasetStore getDatasetStore()

Gets the dataset store.

getEvalStore

public EvalStore getEvalStore()

Gets the eval store.