Package com.google.genkit.plugins.middleware

Class FallbackMiddleware

java.lang.Object

com.google.genkit.ai.middleware.BaseGenerationMiddleware

com.google.genkit.plugins.middleware.FallbackMiddleware

All Implemented Interfaces:

GenerationMiddleware

public class FallbackMiddleware
extends BaseGenerationMiddleware

Falls back to alternate models when the primary model fails.

Wraps the wrapGenerate hook: if generation with the primary model (the one on the incoming request) throws, each model in FallbackMiddleware.Options.models is tried in order by re-running the generate iteration with the request's model swapped. The first success is returned; if all fallbacks fail, the original error is rethrown.

Implemented at the generate level (rather than wrapModel) because switching model requires re-resolving the model action, which the core generate loop already does from options.model. As a consequence a failing iteration re-runs its tool calls under the fallback model; put side-effect-free tools before this middleware if that matters.

Mirrors the JS fallback middleware in @genkit-ai/middleware and the Go Fallback middleware.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	FallbackMiddleware.Options	Configuration parameters for FallbackMiddleware.

Constructor Summary

Constructors

Constructor	Description
FallbackMiddleware(FallbackMiddleware.Options options)

Method Summary

Modifier and Type	Method	Description
String	name()	Returns the middleware's unique identifier.
GenerationMiddleware	newInstance()	Returns a fresh instance for each generate() call, enabling per-invocation state.
ModelResponse	wrapGenerate(ActionContext ctx, GenerateParams params, GenerateNext next)	Wraps each iteration of the generate tool loop.

Methods inherited from class com.google.genkit.ai.middleware.BaseGenerationMiddleware

tools, wrapModel, wrapTool

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

FallbackMiddleware

public FallbackMiddleware(FallbackMiddleware.Options options)

Method Details

name

public String name()

Description copied from interface: GenerationMiddleware

Returns the middleware's unique identifier.

newInstance

public GenerationMiddleware newInstance()

Description copied from interface: GenerationMiddleware

Returns a fresh instance for each generate() call, enabling per-invocation state.

Stable state (e.g., API keys, configuration) should be preserved. Per-request state (e.g., counters) should be reset.

wrapGenerate

public ModelResponse wrapGenerate(ActionContext ctx,
 GenerateParams params,
 GenerateNext next)
                           throws GenkitException

Description copied from interface: GenerationMiddleware

Wraps each iteration of the generate tool loop.

Specified by:

wrapGenerate in interface GenerationMiddleware

Overrides:

wrapGenerate in class BaseGenerationMiddleware

Parameters:

ctx - the action context

params - the generate parameters including the current request and iteration

next - the next function in the chain

Returns:

the model response

Throws:

GenkitException - if processing fails