Package com.google.genkit.ai.middleware

Class ModelParams

java.lang.Object
com.google.genkit.ai.middleware.ModelParams
public class ModelParams extends Object
Holds parameters for the GenerationMiddleware.wrapModel(com.google.genkit.core.ActionContext, com.google.genkit.ai.middleware.ModelParams, com.google.genkit.ai.middleware.ModelNext) hook.

  • Constructor Details

    • ModelParams

      public ModelParams(ModelRequest request, Consumer<ModelResponseChunk> streamCallback)
      Creates ModelParams.
      Parameters:
      request - the model request about to be sent
      streamCallback - the streaming callback, or null if not streaming

  • Method Details

    • getRequest

      public ModelRequest getRequest()
      Returns the model request about to be sent.

    • getStreamCallback

      public Consumer<ModelResponseChunk> getStreamCallback()
      Returns the streaming callback, or null if not streaming.

    • withRequest

      public ModelParams withRequest(ModelRequest request)
      Returns a new ModelParams with the given request, preserving the stream callback.

    • withStreamCallback

      public ModelParams withStreamCallback(Consumer<ModelResponseChunk> streamCallback)
      Returns a new ModelParams with the given stream callback, preserving the request.

      This is the primary mechanism for middleware to transform streaming output. A wrapModel middleware can wrap the original callback with one that intercepts, buffers, splits, combines, or filters chunks before forwarding them to the original callback.

      Example — "smooth stream" middleware that splits large chunks: 

      
 @Override
 public ModelResponse wrapModel(ActionContext ctx, ModelParams params, ModelNext next) {
   Consumer<ModelResponseChunk> original = params.getStreamCallback();
   if (original == null) return next.apply(ctx, params);

   Consumer<ModelResponseChunk> smoothed = chunk -> {
     String text = chunk.getText();
     // Split into smaller pieces and forward each
     for (String piece : splitText(text, 20)) {
       original.accept(ModelResponseChunk.text(piece));
     }
   };
   return next.apply(ctx, params.withStreamCallback(smoothed));
 }
      Parameters:
      streamCallback - the new streaming callback, or null to disable streaming