Package com.google.genkit.ai.middleware

Interface GenerationMiddleware

All Known Implementing Classes:
BaseGenerationMiddleware
public interface GenerationMiddleware
GenerationMiddleware provides hooks for different stages of the generation pipeline.

This is the V2 middleware interface that replaces the generic Middleware<I, O>. It provides three distinct hooks:

Each generate() call creates a fresh instance via newInstance(), enabling per-invocation state (e.g., counters, timers) without shared mutable state across requests.

Example: 


 public class LoggingMiddleware extends BaseGenerationMiddleware {
   private int modelCalls = 0;

   @Override
   public String name() { return "logging"; }

   @Override
   public GenerationMiddleware newInstance() { return new LoggingMiddleware(); }

   @Override
   public ModelResponse wrapModel(ActionContext ctx, ModelParams params, ModelNext next)
       throws GenkitException {
     modelCalls++;
     System.out.println("Model call #" + modelCalls);
     ModelResponse resp = next.apply(ctx, params);
     System.out.println("Model responded with " + resp.getText());
     return resp;
   }
 }

  • Method Details

    • name

      String name()
      Returns the middleware's unique identifier.

    • newInstance

      GenerationMiddleware newInstance()
      Returns a fresh instance for each generate() call, enabling per-invocation state.

      Stable state (e.g., API keys, configuration) should be preserved. Per-request state (e.g., counters) should be reset.

    • wrapGenerate

      ModelResponse wrapGenerate(ActionContext ctx, GenerateParams params, GenerateNext next) throws GenkitException
      Wraps each iteration of the generate tool loop.
      Parameters:
      ctx - the action context
      params - the generate parameters including the current request and iteration
      next - the next function in the chain
      Returns:
      the model response
      Throws:
      GenkitException - if processing fails

    • wrapModel

      ModelResponse wrapModel(ActionContext ctx, ModelParams params, ModelNext next) throws GenkitException
      Wraps each model API call.
      Parameters:
      ctx - the action context
      params - the model parameters including the request
      next - the next function in the chain
      Returns:
      the model response
      Throws:
      GenkitException - if processing fails

    • wrapTool

      Part wrapTool(ActionContext ctx, ToolParams params, ToolNext next) throws GenkitException
      Wraps each tool execution. May be called concurrently when multiple tools execute in parallel. Implementations must be safe for concurrent use.
      Parameters:
      ctx - the action context
      params - the tool parameters including the request part and resolved tool
      next - the next function in the chain
      Returns:
      the tool response part (includes part-level metadata)
      Throws:
      GenkitException - if processing fails

    • tools

      default List<Tool<?,?>> tools()
      Returns additional tools to make available during generation. These tools are dynamically added when the middleware is used.
      Returns:
      the list of additional tools, or empty list if none