Package com.google.genkit.plugins.evaluators

Class EvaluatorsPlugin

java.lang.Object
com.google.genkit.plugins.evaluators.EvaluatorsPlugin
All Implemented Interfaces:
Plugin
public class EvaluatorsPlugin extends Object implements Plugin
Evaluators plugin for Genkit.

This plugin provides a set of pre-defined evaluators for assessing the quality of your LLM outputs. Evaluators include:

  • FAITHFULNESS - Measures factual accuracy against provided context
  • ANSWER_RELEVANCY - Assesses how well the answer pertains to the question
  • ANSWER_ACCURACY - Compares output against a reference answer
  • MALICIOUSNESS - Detects harmful, misleading, or deceptive content
  • REGEX - Validates output against a regex pattern
  • DEEP_EQUAL - Checks deep equality between output and reference
  • JSONATA - Evaluates output using JSONata expressions

Example usage: 


 Genkit genkit = Genkit.builder()
 		.addPlugin(EvaluatorsPlugin.create(EvaluatorsPluginOptions.builder().judge("googleai/gemini-2.0-flash")
 				.metricTypes(List.of(GenkitMetric.FAITHFULNESS, GenkitMetric.ANSWER_RELEVANCY)).build()))
 		.build();

  • Constructor Details

    • EvaluatorsPlugin

      public EvaluatorsPlugin()
      Creates an EvaluatorsPlugin with default options.

    • EvaluatorsPlugin

      public EvaluatorsPlugin(EvaluatorsPluginOptions options)
      Creates an EvaluatorsPlugin with the specified options.
      Parameters:
      options - the plugin options

  • Method Details

    • create

      public static EvaluatorsPlugin create()
      Creates an EvaluatorsPlugin with default options.
      Returns:
      a new EvaluatorsPlugin

    • create

      public static EvaluatorsPlugin create(EvaluatorsPluginOptions options)
      Creates an EvaluatorsPlugin with the specified options.
      Parameters:
      options - the plugin options
      Returns:
      a new EvaluatorsPlugin

    • getName

      public String getName()
      Description copied from interface: Plugin
      Returns the unique identifier for the plugin. This name is used for registration and lookup.
      Specified by:
      getName in interface Plugin
      Returns:
      the plugin name

    • init

      public List<Action<?,?,?>> init()
      Description copied from interface: Plugin
      Initializes the plugin. This method is called once during Genkit initialization. The plugin should return a list of actions that it provides.
      Specified by:
      init in interface Plugin
      Returns:
      list of actions provided by this plugin

    • init

      public List<Action<?,?,?>> init(Registry registry)
      Description copied from interface: Plugin
      Initializes the plugin with access to the registry. This method is called once during Genkit initialization. The plugin should return a list of actions that it provides.

      Override this method instead of Plugin.init() when your plugin needs to resolve dependencies from the registry (e.g., embedders, models).

      Specified by:
      init in interface Plugin
      Parameters:
      registry - the Genkit registry for resolving dependencies
      Returns:
      list of actions provided by this plugin

    • getOptions

      public EvaluatorsPluginOptions getOptions()
      Gets the plugin options.
      Returns:
      the options