Package com.google.genkit

Class Genkit

java.lang.Object
com.google.genkit.Genkit
public class Genkit extends Object
Genkit is the main entry point for the Genkit framework. It provides methods to define and run flows, configure AI models, and interact with the Genkit ecosystem.

  • Constructor Details

    • Genkit

      public Genkit()
      Creates a new Genkit instance with default options.

    • Genkit

      public Genkit(GenkitOptions options)
      Creates a new Genkit instance with the given options.
      Parameters:
      options - the Genkit options

  • Method Details

    • builder

      public static Genkit.Builder builder()
      Creates a new Genkit builder.
      Returns:
      a new builder

    • create

      public static Genkit create(Plugin... plugins)
      Creates a Genkit instance with the given plugins.
      Parameters:
      plugins - the plugins to use
      Returns:
      a configured Genkit instance

    • init

      public void init()
      Initializes plugins.

    • defineFlow

      public <I, O> Flow<I,O,Void> defineFlow(String name, Class<I> inputClass, Class<O> outputClass, BiFunction<ActionContext,I,O> handler)
      Defines a flow.
      Type Parameters:
      I - the input type
      O - the output type
      Parameters:
      name - the flow name
      inputClass - the input class
      outputClass - the output class
      handler - the flow handler
      Returns:
      the flow

    • defineFlow

      public <I, O> Flow<I,O,Void> defineFlow(String name, Class<I> inputClass, Class<O> outputClass, BiFunction<ActionContext,I,O> handler, List<Middleware<I,O>> middleware)
      Defines a flow with middleware.
      Type Parameters:
      I - the input type
      O - the output type
      Parameters:
      name - the flow name
      inputClass - the input class
      outputClass - the output class
      handler - the flow handler
      middleware - the middleware to apply
      Returns:
      the flow

    • defineFlow

      public <I, O> Flow<I,O,Void> defineFlow(String name, Class<I> inputClass, Class<O> outputClass, Function<I,O> handler)
      Defines a flow with a simple handler.
      Type Parameters:
      I - the input type
      O - the output type
      Parameters:
      name - the flow name
      inputClass - the input class
      outputClass - the output class
      handler - the flow handler
      Returns:
      the flow

    • defineFlow

      public <I, O> Flow<I,O,Void> defineFlow(String name, Class<I> inputClass, Class<O> outputClass, Function<I,O> handler, List<Middleware<I,O>> middleware)
      Defines a flow with a simple handler and middleware.
      Type Parameters:
      I - the input type
      O - the output type
      Parameters:
      name - the flow name
      inputClass - the input class
      outputClass - the output class
      handler - the flow handler
      middleware - the middleware to apply
      Returns:
      the flow

    • defineTool

      public <I, O> Tool<I,O> defineTool(String name, String description, Map<String,Object> inputSchema, Class<I> inputClass, BiFunction<ActionContext,I,O> handler)
      Defines a tool.
      Type Parameters:
      I - the input type
      O - the output type
      Parameters:
      name - the tool name
      description - the tool description
      inputSchema - the input JSON schema
      inputClass - the input class
      handler - the tool handler
      Returns:
      the tool

    • defineTool

      public <I, O> Tool<I,O> defineTool(String name, String description, BiFunction<ActionContext,I,O> handler, Class<I> inputClass, Class<O> outputClass)
      Defines a tool with typed input and output classes.

      This is the preferred way to create tools with structured input/output. The schemas are automatically generated from the classes using their @JsonPropertyDescription and @JsonProperty annotations.

      Example usage: 

      
 Tool<RecipeRequest, MenuItem> tool = genkit.defineTool(
     "generateRecipe",
     "Generates a recipe based on cuisine and dietary preferences",
     (ctx, request) -> {
       // Tool implementation
       return new MenuItem(...);
     },
     RecipeRequest.class,
     MenuItem.class
 );
      Type Parameters:
      I - the input type
      O - the output type
      Parameters:
      name - the tool name
      description - the tool description
      handler - the tool handler
      inputClass - the input class (schema auto-generated)
      outputClass - the output class (schema auto-generated)
      Returns:
      the tool

    • prompt

      public <I> ExecutablePrompt<I> prompt(String name, Class<I> inputClass) throws GenkitException
      Loads a prompt by name from the prompts directory.

      This is similar to the JavaScript API: `ai.prompt('hello')`. The prompt is loaded from the configured promptDir (default: /prompts). The prompt is automatically registered as an action and cached for reuse.

      Example usage: 

      
 ExecutablePrompt<HelloInput> helloPrompt = genkit.prompt("hello", HelloInput.class);
 ModelResponse response = helloPrompt.generate(new HelloInput("John"));
      Type Parameters:
      I - the input type
      Parameters:
      name - the prompt name (without .prompt extension)
      inputClass - the input class
      Returns:
      the executable prompt
      Throws:
      GenkitException - if the prompt cannot be loaded

    • prompt

      public <I> ExecutablePrompt<I> prompt(String name, Class<I> inputClass, String variant) throws GenkitException
      Loads a prompt by name with an optional variant.

      Variants allow different versions of the same prompt to be tested. For example: "recipe" with variant "gemini25pro" loads "recipe.gemini25pro.prompt".

      Type Parameters:
      I - the input type
      Parameters:
      name - the prompt name (without .prompt extension)
      inputClass - the input class
      variant - optional variant name (e.g., "gemini25pro")
      Returns:
      the executable prompt
      Throws:
      GenkitException - if the prompt cannot be loaded

    • prompt

      public ExecutablePrompt<Map<String,Object>> prompt(String name) throws GenkitException
      Loads a prompt by name using a Map as input type.

      This is a convenience method when you don't want to define a specific input class.

      Parameters:
      name - the prompt name (without .prompt extension)
      Returns:
      the executable prompt with Map input
      Throws:
      GenkitException - if the prompt cannot be loaded

    • definePrompt

      public <I> Prompt<I> definePrompt(String name, String template, Class<I> inputClass, BiFunction<ActionContext,I,ModelRequest> renderer)
      Defines a prompt.
      Type Parameters:
      I - the input type
      Parameters:
      name - the prompt name
      template - the prompt template
      inputClass - the input class
      renderer - the prompt renderer
      Returns:
      the prompt

    • registerModel

      public void registerModel(Model model)
      Registers a model.
      Parameters:
      model - the model to register

    • registerEmbedder

      public void registerEmbedder(Embedder embedder)
      Registers an embedder.
      Parameters:
      embedder - the embedder to register

    • registerRetriever

      public void registerRetriever(Retriever retriever)
      Registers a retriever.
      Parameters:
      retriever - the retriever to register

    • registerIndexer

      public void registerIndexer(Indexer indexer)
      Registers an indexer.
      Parameters:
      indexer - the indexer to register

    • defineRetriever

      public Retriever defineRetriever(String name, BiFunction<ActionContext,RetrieverRequest,RetrieverResponse> handler)
      Defines and registers a retriever.

      This is the preferred way to create retrievers as it automatically registers them with the Genkit registry.

      Example usage: 

      
 Retriever myRetriever = genkit.defineRetriever("myStore/docs", (ctx, request) -> {
 	// Find similar documents
 	List<Document> docs = findSimilarDocs(request.getQuery());
 	return new RetrieverResponse(docs);
 });
      Parameters:
      name - the retriever name
      handler - the retrieval function
      Returns:
      the registered retriever

    • defineIndexer

      public Indexer defineIndexer(String name, BiFunction<ActionContext,IndexerRequest,IndexerResponse> handler)
      Defines and registers an indexer.

      This is the preferred way to create indexers as it automatically registers them with the Genkit registry.

      Example usage: 

      
 Indexer myIndexer = genkit.defineIndexer("myStore/docs", (ctx, request) -> {
 	// Index the documents
 	indexDocuments(request.getDocuments());
 	return new IndexerResponse();
 });
      Parameters:
      name - the indexer name
      handler - the indexing function
      Returns:
      the registered indexer

    • getModel

      public Model getModel(String name)
      Gets a model by name.
      Parameters:
      name - the model name
      Returns:
      the model

    • getEmbedder

      public Embedder getEmbedder(String name)
      Gets an embedder by name.
      Parameters:
      name - the embedder name
      Returns:
      the embedder

    • getRetriever

      public Retriever getRetriever(String name)
      Gets a retriever by name.
      Parameters:
      name - the retriever name
      Returns:
      the retriever

    • generate

      public <T> T generate(GenerateOptions options) throws GenkitException
      Generates a model response using the specified options.

      This method handles tool execution automatically. If the model requests tool calls, this method will execute the tools, add the results to the conversation, and continue generation until the model produces a final response.

      When a tool throws a ToolInterruptException, the generation is halted and the response is returned with FinishReason.INTERRUPTED. The caller can then use ResumeOptions to continue generation after handling the interrupt.

      Example with interrupts: 

      
 // First generation - may be interrupted
 ModelResponse response = genkit.generate(GenerateOptions.builder().model("googleai/gemini-pro")
 		.prompt("Transfer $100 to account 12345").tools(List.of(confirmTransfer)).build());

 // Check if interrupted
 if (response.isInterrupted()) {
 	Part interrupt = response.getInterrupts().get(0);
 
 	// Get user confirmation
 	boolean confirmed = askUserForConfirmation();
 
 	// Resume with user response
 	Part responseData = confirmTransfer.respond(interrupt, new ConfirmOutput(confirmed));
 	ModelResponse resumed = genkit.generate(GenerateOptions.builder().model("googleai/gemini-pro")
 			.messages(response.getMessages()).tools(List.of(confirmTransfer))
 			.resume(ResumeOptions.builder().respond(responseData).build()).build());
 }
      Parameters:
      options - the generate options
      Returns:
      the model response
      Throws:
      GenkitException - if generation fails

    • generateStream

      public ModelResponse generateStream(GenerateOptions<?> options, Consumer<ModelResponseChunk> streamCallback) throws GenkitException
      Generates a streaming model response using the specified options.

      This method invokes the model with streaming enabled, calling the provided callback for each chunk of the response as it arrives. This is useful for displaying responses incrementally to users.

      Example usage: 

      
 StringBuilder result = new StringBuilder();
 ModelResponse response = genkit.generateStream(
 		GenerateOptions.builder().model("openai/gpt-4o").prompt("Tell me a story").build(), chunk -> {
 			System.out.print(chunk.getText());
 			result.append(chunk.getText());
 		});
      Parameters:
      options - the generate options
      streamCallback - callback invoked for each response chunk
      Returns:
      the final aggregated model response
      Throws:
      GenkitException - if generation fails or model doesn't support streaming

    • generate

      public ModelResponse generate(String modelName, String prompt) throws GenkitException
      Generates a model response with a simple prompt.
      Parameters:
      modelName - the model name
      prompt - the prompt text
      Returns:
      the model response
      Throws:
      GenkitException - if generation fails

    • generateObject

      public <T> T generateObject(GenerateOptions options) throws GenkitException
      Generates a structured output from the model, returning a typed object.

      This method automatically generates a JSON schema from the provided class and deserializes the model's response into an instance of that class. You can add descriptions to fields using @JsonPropertyDescription and mark fields as required using @JsonProperty(required = true): 

       {
        @code
        public class MenuItem {
                @JsonProperty(required = true)
                @JsonPropertyDescription("The name of the menu item")
                private String name;
 
                @JsonPropertyDescription("A description of the menu item")
                private String description;
      Type Parameters:
      T - the output type
      Parameters:
      options - the generate options with outputClass set
      Returns:
      the generated object
      Throws:
      GenkitException - if generation or deserialization fails

    • generateObject

      public <T> T generateObject(String modelName, String prompt, Class<T> outputClass) throws GenkitException
      Generates a structured output from the model with a simple prompt.

      Convenience method that combines model name, prompt, and output class. 

      
 MenuItem item = genkit.generateObject("openai/gpt-4o-mini",
 		"Suggest a menu item for a pirate-themed restaurant.", MenuItem.class);
      Type Parameters:
      T - the output type
      Parameters:
      modelName - the model name
      prompt - the prompt text
      outputClass - the class to deserialize the response into
      Returns:
      the generated object
      Throws:
      GenkitException - if generation or deserialization fails

    • generateObject

      public <T> T generateObject(GenerateOptions<?> options, Class<T> outputClass) throws GenkitException
      Generates a structured output from the model, returning a typed object.

      Alternative method that takes outputClass as a separate parameter.

      Type Parameters:
      T - the output type
      Parameters:
      options - the generate options
      outputClass - the class to deserialize the response into
      Returns:
      the generated object
      Throws:
      GenkitException - if generation or deserialization fails

    • embed

      public EmbedResponse embed(String embedderName, List<Document> documents) throws GenkitException
      Generates a structured output from the model with a simple prompt.

      Convenience method that combines model name, prompt, and output class. Embeds documents using the specified embedder.

      Parameters:
      embedderName - the embedder name
      documents - the documents to embed
      Returns:
      the embed response
      Throws:
      GenkitException - if embedding fails

    • retrieve

      public List<Document> retrieve(String retrieverName, String query) throws GenkitException
      Retrieves documents using the specified retriever.

      This is the primary method for retrieval in RAG workflows. The returned documents can be passed directly to generate() via the .docs() option.

      Example usage: 

      
 // Retrieve relevant documents
 List<Document> docs = genkit.retrieve("myStore/docs", "What is the capital of France?");
 
 // Use documents in generation
 ModelResponse response = genkit.generate(GenerateOptions.builder().model("openai/gpt-4o-mini")
 		.prompt("Answer the question based on context").docs(docs).build());
      Parameters:
      retrieverName - the retriever name
      query - the query text
      Returns:
      the list of retrieved documents
      Throws:
      GenkitException - if retrieval fails

    • retrieve

      public List<Document> retrieve(String retrieverName, String query, RetrieverRequest.RetrieverOptions options) throws GenkitException
      Retrieves documents using the specified retriever with options.

      Example usage: 

      
 List<Document> docs = genkit.retrieve("myStore/docs", "query", RetrieverParams.builder().k(5).build());
      Parameters:
      retrieverName - the retriever name
      query - the query text
      options - retrieval options (e.g., k for number of results)
      Returns:
      the list of retrieved documents
      Throws:
      GenkitException - if retrieval fails

    • retrieve

      public List<Document> retrieve(String retrieverName, Document query) throws GenkitException
      Retrieves documents using a Document as the query.
      Parameters:
      retrieverName - the retriever name
      query - the query document
      Returns:
      the list of retrieved documents
      Throws:
      GenkitException - if retrieval fails

    • index

      public void index(String indexerName, List<Document> documents) throws GenkitException
      Indexes documents using the specified indexer.

      Example usage: 

      
 List<Document> docs = List.of(Document.fromText("Paris is the capital of France."),
 		Document.fromText("Berlin is the capital of Germany."));
 genkit.index("myStore/docs", docs);
      Parameters:
      indexerName - the indexer name
      documents - the documents to index
      Throws:
      GenkitException - if indexing fails

    • getIndexer

      public Indexer getIndexer(String name)
      Gets an indexer by name.
      Parameters:
      name - the indexer name
      Returns:
      the indexer

    • runFlow

      public <I, O> O runFlow(String flowName, I input) throws GenkitException
      Runs a flow by name.
      Type Parameters:
      I - the input type
      O - the output type
      Parameters:
      flowName - the flow name
      input - the flow input
      Returns:
      the flow output
      Throws:
      GenkitException - if execution fails

    • getRegistry

      public Registry getRegistry()
      Gets the registry.
      Returns:
      the registry

    • getOptions

      public GenkitOptions getOptions()
      Gets the options.
      Returns:
      the options

    • getPlugins

      public List<Plugin> getPlugins()
      Gets the registered plugins.
      Returns:
      the plugins

    • stop

      public void stop()
      Stops the Genkit instance and cleans up resources.

    • createSession

      public <S> Session<S> createSession()
      Creates a new session with default options.

      Sessions provide stateful multi-turn conversations with automatic history persistence. Each session can have multiple named conversation threads.

      Example usage: 

      
 Session<Void> session = genkit.createSession();
 Chat<Void> chat = session
 		.chat(ChatOptions.<Void>builder().model("openai/gpt-4o").system("You are a helpful assistant.").build());
 chat.send("Hello!");
      Type Parameters:
      S - the session state type
      Returns:
      a new session

    • createSession

      public <S> Session<S> createSession(SessionOptions<S> options)
      Creates a new session with the given options.

      Example usage: 

      
 // With custom state
 Session<MyState> session = genkit
 		.createSession(SessionOptions.<MyState>builder().initialState(new MyState("John")).build());

 // With custom store and session ID
 Session<MyState> session = genkit.createSession(SessionOptions.<MyState>builder()
 		.store(new RedisSessionStore<>()).sessionId("my-session-123").initialState(new MyState()).build());
      Type Parameters:
      S - the session state type
      Parameters:
      options - the session options
      Returns:
      a new session

    • loadSession

      public <S> CompletableFuture<Session<S>> loadSession(String sessionId, SessionOptions<S> options)
      Loads an existing session from a store.

      Example usage: 

      
 CompletableFuture<Session<MyState>> sessionFuture = genkit.loadSession("session-123",
 		SessionOptions.<MyState>builder().store(mySessionStore).build());
 Session<MyState> session = sessionFuture.get();
 if (session != null) {
 	Chat<MyState> chat = session.chat();
 	// Continue conversation...
 }
      Type Parameters:
      S - the session state type
      Parameters:
      sessionId - the session ID to load
      options - the session options (must include store)
      Returns:
      a CompletableFuture containing the session, or null if not found

    • chat

      public <S> Chat<S> chat(ChatOptions<S> options)
      Creates a simple chat without session persistence.

      This is a convenience method for quick interactions without full session management. Use createSession() for persistent multi-turn conversations.

      Example usage: 

      
 Chat<Void> chat = genkit
 		.chat(ChatOptions.<Void>builder().model("openai/gpt-4o").system("You are a helpful assistant.").build());
 ModelResponse response = chat.send("Hello!");
      Type Parameters:
      S - the state type (usually Void for simple chats)
      Parameters:
      options - the chat options
      Returns:
      a new chat instance

    • defineAgent

      public Agent defineAgent(AgentConfig config)
      Defines an agent that can be used as a tool in multi-agent systems.

      Agents are specialized conversational components that can be delegated to by other agents. When an agent is called as a tool, it takes over the conversation with its own system prompt, model, and tools.

      Example usage: 

      
 // Define a specialized agent
 Agent reservationAgent = genkit.defineAgent(AgentConfig.builder().name("reservationAgent")
 		.description("Handles restaurant reservations").system("You are a reservation specialist...")
 		.model("openai/gpt-4o").tools(List.of(reservationTool, lookupTool)).build());

 // Use in a parent agent
 Agent triageAgent = genkit.defineAgent(
 		AgentConfig.builder().name("triageAgent").description("Routes customer requests to specialists")
 				.system("You route customer requests to the appropriate specialist")
 				.agents(List.of(reservationAgent.getConfig())).build());

 // Start chat with triage agent
 Chat chat = genkit.chat(ChatOptions.builder().model("openai/gpt-4o").system(triageAgent.getSystem())
 		.tools(triageAgent.getAllTools(agentRegistry)).build());
      Parameters:
      config - the agent configuration
      Returns:
      the created agent

    • getAgent

      public Agent getAgent(String name)
      Gets an agent by name.
      Parameters:
      name - the agent name
      Returns:
      the agent, or null if not found

    • getAgentRegistry

      public Map<String,Agent> getAgentRegistry()
      Gets the agent registry.

      This returns an unmodifiable view of all registered agents.

      Returns:
      the agent registry

    • getAllToolsForAgent

      public List<Tool<?,?>> getAllToolsForAgent(Agent agent)
      Gets all tools for an agent, including sub-agent tools.

      This is a convenience method that collects all tools from an agent, including tools from any sub-agents defined in its configuration.

      Parameters:
      agent - the agent
      Returns:
      the list of all tools

    • defineInterrupt

      public <I, O> Tool<I,O> defineInterrupt(InterruptConfig<I,O> config)
      Defines an interrupt tool for human-in-the-loop interactions.

      Interrupts allow tools to pause generation and request user input. When a tool throws a ToolInterruptException, the chat returns early with the interrupt information, allowing the application to collect user input and resume.

      Example usage: 

      
 // Define an interrupt for confirming actions
 Tool<ConfirmInput, ConfirmOutput> confirmInterrupt = genkit.defineInterrupt(InterruptConfig
 		.<ConfirmInput, ConfirmOutput>builder().name("confirm").description("Asks user to confirm an action")
 		.inputType(ConfirmInput.class).outputType(ConfirmOutput.class).build());

 // Use in a chat with tools
 Chat chat = genkit.chat(
 		ChatOptions.builder().model("openai/gpt-4o").tools(List.of(someActionTool, confirmInterrupt)).build());

 ModelResponse response = chat.send("Book a table for 4");

 // Check for interrupts
 if (chat.hasPendingInterrupts()) {
 	List<InterruptRequest> interrupts = chat.getPendingInterrupts();
 	// Show UI to user, collect response
 	ConfirmOutput userResponse = getUserConfirmation(interrupts.get(0));
 
 	// Resume with user response
 	response = chat.send("",
 			SendOptions.builder().resumeOptions(
 					ResumeOptions.builder().respond(List.of(interrupts.get(0).respond(userResponse))).build())
 					.build());
 }
      Type Parameters:
      I - the interrupt input type
      O - the interrupt output type (user response)
      Parameters:
      config - the interrupt configuration
      Returns:
      the interrupt as a tool

    • currentSession

      public <S> Session<S> currentSession()
      Gets the current session from the context.

      This method can be called from within tool execution to access the current session state. It uses a thread-local context that is set during chat execution.

      Example usage: 

      
 Tool<Input, Output> myTool = genkit.defineTool("myTool", Input.class, Output.class, (ctx, input) -> {
 	Session<?> session = genkit.currentSession();
 	if (session != null) {
 		Object state = session.getState();
 		// Use session state...
 	}
 	return new Output();
 });
      Type Parameters:
      S - the session state type
      Returns:
      the current session, or null if not in a session context

    • defineEvaluator

      public <O> Evaluator<O> defineEvaluator(String name, String displayName, String definition, EvaluatorFn<O> evaluatorFn)
      Defines a new evaluator and registers it with the registry.

      Evaluators assess the quality of AI outputs. They can be used to:

      • Score outputs based on various criteria (accuracy, relevance, etc.)
      • Compare outputs against reference data
      • Run automated quality checks in CI/CD pipelines

      Example usage: 

      
 genkit.defineEvaluator("myEvaluator", "My Evaluator", "Checks output quality", (dataPoint, options) -> {
 	// Evaluate the output
 	double score = calculateScore(dataPoint.getOutput());
 	return EvalResponse.builder().testCaseId(dataPoint.getTestCaseId())
 			.evaluation(Score.builder().score(score).build()).build();
 });
      Type Parameters:
      O - the options type
      Parameters:
      name - the evaluator name
      displayName - the display name shown in the UI
      definition - description of what the evaluator measures
      evaluatorFn - the evaluation function
      Returns:
      the created evaluator

    • defineEvaluator

      public <O> Evaluator<O> defineEvaluator(String name, String displayName, String definition, boolean isBilled, Class<O> optionsClass, EvaluatorFn<O> evaluatorFn)
      Defines a new evaluator with full options.
      Type Parameters:
      O - the options type
      Parameters:
      name - the evaluator name
      displayName - the display name shown in the UI
      definition - description of what the evaluator measures
      isBilled - whether using this evaluator incurs costs
      optionsClass - the class for evaluator-specific options
      evaluatorFn - the evaluation function
      Returns:
      the created evaluator

    • getEvaluator

      public Evaluator<?> getEvaluator(String name)
      Gets an evaluator by name.
      Parameters:
      name - the evaluator name
      Returns:
      the evaluator
      Throws:
      GenkitException - if evaluator not found

    • evaluate

      public EvalRunKey evaluate(RunEvaluationRequest request) throws Exception
      Runs an evaluation using the specified request.

      This method:

      1. Loads the dataset
      2. Runs inference on the target action
      3. Executes all specified evaluators
      4. Stores and returns the results
      Parameters:
      request - the evaluation request
      Returns:
      the evaluation run key
      Throws:
      Exception - if evaluation fails

    • getEvaluationManager

      public EvaluationManager getEvaluationManager()
      Gets the evaluation manager.
      Returns:
      the evaluation manager

    • getDatasetStore

      public DatasetStore getDatasetStore()
      Gets the dataset store.
      Returns:
      the dataset store

    • getEvalStore

      public EvalStore getEvalStore()
      Gets the eval store.
      Returns:
      the eval store