Genkit JS API reference
    Preparing search index...

    Class GenkitBetaBeta

    Genkit BETA APIs.

    Hierarchy (View Summary)

    Index

    Constructors

    Properties

    flows: Action<any, any, any>[] = []

    List of flows that have been registered in this instance.

    options: GenkitOptions

    Developer-configured options.

    registry: Registry

    Accessors

    • get apiStability(): "beta" | "stable"
      Beta

      Returns "beta" | "stable"

    Methods

    • Cancels a given operation. Returns a new operation which will contain the updated status.

      Type Parameters

      • T

      Parameters

      Returns Promise<Operation<T>>

    • Beta

      Create a chat session with the provided options.

      const chat = ai.chat({
      system: 'talk like a pirate',
      })
      let response = await chat.send('tell me a joke')
      response = await chat.send('another one')

      Type Parameters

      • I

      Parameters

      • Optionaloptions: ChatOptions<I>

      Returns Chat

    • Beta

      Create a chat session with the provided preamble.

      const triageAgent = ai.definePrompt({
      system: 'help the user triage a problem',
      })
      const chat = ai.chat(triageAgent)
      const { text } = await chat.send('my phone feels hot');

      Type Parameters

      • I

      Parameters

      Returns Chat

    • Checks the status of of a given operation. Returns a new operation which will contain the updated status.

      let operation = await ai.generateOperation({
      model: googleAI.model('veo-2.0-generate-001'),
      prompt: 'A banana riding a bicycle.',
      });

      while (!operation.done) {
      operation = await ai.checkOperation(operation!);
      await new Promise((resolve) => setTimeout(resolve, 5000));
      }

      Type Parameters

      • T

      Parameters

      Returns Promise<Operation<T>>

    • Returns current action (or flow) invocation context. Can be used to access things like auth data set by HTTP server frameworks. If invoked outside of an action (e.g. flow or tool) will return undefined.

      Returns ActionContext | undefined

    • Creates embedder model for the provided EmbedderFn model implementation.

      Type Parameters

      Parameters

      • options: {
            configSchema?: ConfigSchema;
            info?: {
                dimensions?: number;
                label?: string;
                supports?: {
                    input?: ("video" | "image" | "text")[];
                    multilingual?: boolean;
                };
            };
            name: string;
        }
      • runner: EmbedderFn<ConfigSchema>

      Returns EmbedderAction<ConfigSchema>

    • Creates evaluator action for the provided EvaluatorFn implementation.

      Type Parameters

      • DataPoint extends z.ZodObject<
            {
                context: z.ZodOptional<z.ZodArray<z.ZodUnknown, "many">>;
                input: z.ZodUnknown;
                output: z.ZodOptional<z.ZodUnknown>;
                reference: z.ZodOptional<z.ZodUnknown>;
                testCaseId: z.ZodOptional<z.ZodString>;
                traceIds: z.ZodOptional<z.ZodArray<z.ZodString, "many">>;
            },
            "strip",
            z.ZodTypeAny,
            {
                context?: unknown[];
                input?: unknown;
                output?: unknown;
                reference?: unknown;
                testCaseId?: string;
                traceIds?: string[];
            },
            {
                context?: unknown[];
                input?: unknown;
                output?: unknown;
                reference?: unknown;
                testCaseId?: string;
                traceIds?: string[];
            },
        > = z.ZodObject<
            {
                context: z.ZodOptional<z.ZodArray<z.ZodUnknown, "many">>;
                input: z.ZodUnknown;
                output: z.ZodOptional<z.ZodUnknown>;
                reference: z.ZodOptional<z.ZodUnknown>;
                testCaseId: z.ZodOptional<z.ZodString>;
                traceIds: z.ZodOptional<z.ZodArray<z.ZodString, "many">>;
            },
            "strip",
            z.ZodTypeAny,
            {
                context?: unknown[];
                input?: unknown;
                output?: unknown;
                reference?: unknown;
                testCaseId?: string;
                traceIds?: string[];
            },
            {
                context?: unknown[];
                input?: unknown;
                output?: unknown;
                reference?: unknown;
                testCaseId?: string;
                traceIds?: string[];
            },
        >
      • EvalDataPoint extends z.ZodObject<
            {
                context: z.ZodOptional<z.ZodArray<z.ZodUnknown, "many">>;
                input: z.ZodUnknown;
                output: z.ZodOptional<z.ZodUnknown>;
                reference: z.ZodOptional<z.ZodUnknown>;
                traceIds: z.ZodOptional<z.ZodArray<z.ZodString, "many">>;
            } & { testCaseId: z.ZodString },
            "strip",
            z.ZodTypeAny,
            {
                context?: unknown[];
                input?: unknown;
                output?: unknown;
                reference?: unknown;
                testCaseId: string;
                traceIds?: string[];
            },
            {
                context?: unknown[];
                input?: unknown;
                output?: unknown;
                reference?: unknown;
                testCaseId: string;
                traceIds?: string[];
            },
        > = z.ZodObject<
            {
                context: z.ZodOptional<z.ZodArray<z.ZodUnknown, "many">>;
                input: z.ZodUnknown;
                output: z.ZodOptional<z.ZodUnknown>;
                reference: z.ZodOptional<z.ZodUnknown>;
                traceIds: z.ZodOptional<z.ZodArray<z.ZodString, "many">>;
            } & { testCaseId: z.ZodString },
            "strip",
            z.ZodTypeAny,
            {
                context?: unknown[];
                input?: unknown;
                output?: unknown;
                reference?: unknown;
                testCaseId: string;
                traceIds?: string[];
            },
            {
                context?: unknown[];
                input?: unknown;
                output?: unknown;
                reference?: unknown;
                testCaseId: string;
                traceIds?: string[];
            },
        >
      • EvaluatorOptions extends z.ZodTypeAny = z.ZodTypeAny

      Parameters

      Returns EvaluatorAction

    • Beta

      Defines and registers a custom model output formatter.

      Here's an example of a custom JSON output formatter:

      import { extractJson } from 'genkit/extract';

      ai.defineFormat(
      { name: 'customJson' },
      (schema) => {
      let instructions: string | undefined;
      if (schema) {
      instructions = `Output should be in JSON format and conform to the following schema:
      \`\`\`
      ${JSON.stringify(schema)}
      \`\`\`
      `;
      }
      return {
      parseChunk: (chunk) => extractJson(chunk.accumulatedText),
      parseMessage: (message) => extractJson(message.text),
      instructions,
      };
      }
      );

      const { output } = await ai.generate({
      prompt: 'Invent a menu item for a pirate themed restaurant.',
      output: { format: 'customJson', schema: MenuItemSchema },
      });

      Parameters

      • options: { name: string } & {
            constrained?: boolean;
            contentType?: string;
            format?: string;
            schema?: Record<string, any>;
        } & { defaultInstructions?: false }
      • handler: (
            schema?: any,
        ) => {
            instructions?: string;
            parseChunk?: (chunk: GenerateResponseChunk) => unknown;
            parseMessage(message: Message): unknown;
        }

      Returns {
          config: {
              constrained?: boolean;
              contentType?: string;
              format?: string;
              schema?: Record<string, any>;
          } & { defaultInstructions?: false };
          handler: (
              schema?: any,
          ) => {
              instructions?: string;
              parseChunk?: (chunk: GenerateResponseChunk) => unknown;
              parseMessage(message: Message): unknown;
          };
      }

    • Creates an indexer action for the provided IndexerFn implementation.

      Type Parameters

      Parameters

      • options: {
            configSchema?: IndexerOptions;
            embedderInfo?: {
                dimensions?: number;
                label?: string;
                supports?: {
                    input?: ("video" | "image" | "text")[];
                    multilingual?: boolean;
                };
            };
            name: string;
        }
      • runner: IndexerFn<IndexerOptions>

      Returns IndexerAction<IndexerOptions>

    • Defines and registers a schema from a JSON schema.

      Defined schemas can be referenced by name in prompts in place of inline schemas.

      Parameters

      • name: string
      • jsonSchema: any

      Returns any

    • Defines a new model and adds it to the registry.

      Type Parameters

      Parameters

      • options: { apiVersion: "v2" } & DefineModelOptions<CustomOptionsSchema>
      • runner: (
            request: GenerateRequest<CustomOptionsSchema>,
            options: ActionFnArg<
                {
                    aggregated?: boolean;
                    content: (
                        | {
                            custom?: Record<string, unknown>;
                            data?: unknown;
                            media?: undefined;
                            metadata?: Record<string, unknown>;
                            reasoning?: undefined;
                            resource?: undefined;
                            text: string;
                            toolRequest?: undefined;
                            toolResponse?: undefined;
                        }
                        | {
                            custom?: Record<string, unknown>;
                            data?: unknown;
                            media: { contentType?: string; url: string };
                            metadata?: Record<string, unknown>;
                            reasoning?: undefined;
                            resource?: undefined;
                            text?: undefined;
                            toolRequest?: undefined;
                            toolResponse?: undefined;
                        }
                        | {
                            custom?: Record<string, unknown>;
                            data?: unknown;
                            media?: undefined;
                            metadata?: Record<string, unknown>;
                            reasoning?: undefined;
                            resource?: undefined;
                            text?: undefined;
                            toolRequest: {
                                input?: unknown;
                                name: string;
                                partial?: boolean;
                                ref?: string;
                            };
                            toolResponse?: undefined;
                        }
                        | {
                            custom?: Record<string, unknown>;
                            data?: unknown;
                            media?: undefined;
                            metadata?: Record<string, unknown>;
                            reasoning?: undefined;
                            resource?: undefined;
                            text?: undefined;
                            toolRequest?: undefined;
                            toolResponse: { name: string; output?: unknown; ref?: string } & {
                                content?: (
                                    (...)
                                    | (...)
                                    | (...)
                                    | (...)
                                    | (...)
                                    | (...)
                                    | (...)
                                    | (...)
                                )[];
                            };
                        }
                        | {
                            custom?: Record<string, unknown>;
                            data?: unknown;
                            media?: undefined;
                            metadata?: Record<string, unknown>;
                            reasoning?: undefined;
                            resource?: undefined;
                            text?: undefined;
                            toolRequest?: undefined;
                            toolResponse?: undefined;
                        }
                        | {
                            custom: Record<string, any>;
                            data?: unknown;
                            media?: undefined;
                            metadata?: Record<string, unknown>;
                            reasoning?: undefined;
                            resource?: undefined;
                            text?: undefined;
                            toolRequest?: undefined;
                            toolResponse?: undefined;
                        }
                        | {
                            custom?: Record<string, unknown>;
                            data?: unknown;
                            media?: undefined;
                            metadata?: Record<string, unknown>;
                            reasoning: string;
                            resource?: undefined;
                            text?: undefined;
                            toolRequest?: undefined;
                            toolResponse?: undefined;
                        }
                        | {
                            custom?: Record<string, unknown>;
                            data?: unknown;
                            media?: undefined;
                            metadata?: Record<string, unknown>;
                            reasoning?: undefined;
                            resource: { uri: string };
                            text?: undefined;
                            toolRequest?: undefined;
                            toolResponse?: undefined;
                        }
                    )[];
                    custom?: unknown;
                    index?: number;
                    role?: "user"
                    | "model"
                    | "tool"
                    | "system";
                },
            >,
        ) => Promise<
            {
                candidates?: {
                    custom?: unknown;
                    finishMessage?: string;
                    finishReason: | "length"
                    | "stop"
                    | "other"
                    | "unknown"
                    | "interrupted"
                    | "blocked"
                    | "aborted";
                    index: number;
                    message: {
                        content: (
                            | {
                                custom?: (...)
                                | (...);
                                data?: unknown;
                                media?: undefined;
                                metadata?: (...) | (...);
                                reasoning?: undefined;
                                resource?: undefined;
                                text: string;
                                toolRequest?: undefined;
                                toolResponse?: undefined;
                            }
                            | {
                                custom?: (...)
                                | (...);
                                data?: unknown;
                                media: { contentType?: ...; url: ... };
                                metadata?: (...) | (...);
                                reasoning?: undefined;
                                resource?: undefined;
                                text?: undefined;
                                toolRequest?: undefined;
                                toolResponse?: undefined;
                            }
                            | {
                                custom?: (...)
                                | (...);
                                data?: unknown;
                                media?: undefined;
                                metadata?: (...) | (...);
                                reasoning?: undefined;
                                resource?: undefined;
                                text?: undefined;
                                toolRequest: { input?: ...; name: ...; partial?: ...; ref?: ... };
                                toolResponse?: undefined;
                            }
                            | {
                                custom?: (...)
                                | (...);
                                data?: unknown;
                                media?: undefined;
                                metadata?: (...) | (...);
                                reasoning?: undefined;
                                resource?: undefined;
                                text?: undefined;
                                toolRequest?: undefined;
                                toolResponse: (...) & (...);
                            }
                            | {
                                custom?: (...)
                                | (...);
                                data?: unknown;
                                media?: undefined;
                                metadata?: (...) | (...);
                                reasoning?: undefined;
                                resource?: undefined;
                                text?: undefined;
                                toolRequest?: undefined;
                                toolResponse?: undefined;
                            }
                            | {
                                custom: Record<(...), (...)>;
                                data?: unknown;
                                media?: undefined;
                                metadata?: (...) | (...);
                                reasoning?: undefined;
                                resource?: undefined;
                                text?: undefined;
                                toolRequest?: undefined;
                                toolResponse?: undefined;
                            }
                            | {
                                custom?: (...)
                                | (...);
                                data?: unknown;
                                media?: undefined;
                                metadata?: (...) | (...);
                                reasoning: string;
                                resource?: undefined;
                                text?: undefined;
                                toolRequest?: undefined;
                                toolResponse?: undefined;
                            }
                            | {
                                custom?: (...)
                                | (...);
                                data?: unknown;
                                media?: undefined;
                                metadata?: (...) | (...);
                                reasoning?: undefined;
                                resource: { uri: ... };
                                text?: undefined;
                                toolRequest?: undefined;
                                toolResponse?: undefined;
                            }
                        )[];
                        metadata?: Record<string, unknown>;
                        role: "user" | "model" | "tool" | "system";
                    };
                    usage?: {
                        cachedContentTokens?: number;
                        custom?: Record<string, number>;
                        inputAudioFiles?: number;
                        inputCharacters?: number;
                        inputImages?: number;
                        inputTokens?: number;
                        inputVideos?: number;
                        outputAudioFiles?: number;
                        outputCharacters?: number;
                        outputImages?: number;
                        outputTokens?: number;
                        outputVideos?: number;
                        thoughtsTokens?: number;
                        totalTokens?: number;
                    };
                }[];
                custom?: unknown;
                finishMessage?: string;
                finishReason?: | "length"
                | "stop"
                | "other"
                | "unknown"
                | "interrupted"
                | "blocked"
                | "aborted";
                latencyMs?: number;
                message?: {
                    content: (
                        | {
                            custom?: Record<string, unknown>;
                            data?: unknown;
                            media?: undefined;
                            metadata?: Record<string, unknown>;
                            reasoning?: undefined;
                            resource?: undefined;
                            text: string;
                            toolRequest?: undefined;
                            toolResponse?: undefined;
                        }
                        | {
                            custom?: Record<string, unknown>;
                            data?: unknown;
                            media: { contentType?: string; url: string };
                            metadata?: Record<string, unknown>;
                            reasoning?: undefined;
                            resource?: undefined;
                            text?: undefined;
                            toolRequest?: undefined;
                            toolResponse?: undefined;
                        }
                        | {
                            custom?: Record<string, unknown>;
                            data?: unknown;
                            media?: undefined;
                            metadata?: Record<string, unknown>;
                            reasoning?: undefined;
                            resource?: undefined;
                            text?: undefined;
                            toolRequest: {
                                input?: unknown;
                                name: string;
                                partial?: boolean;
                                ref?: string;
                            };
                            toolResponse?: undefined;
                        }
                        | {
                            custom?: Record<string, unknown>;
                            data?: unknown;
                            media?: undefined;
                            metadata?: Record<string, unknown>;
                            reasoning?: undefined;
                            resource?: undefined;
                            text?: undefined;
                            toolRequest?: undefined;
                            toolResponse: { name: string; output?: unknown; ref?: (...) | (...) } & {
                                content?: (...) | (...);
                            };
                        }
                        | {
                            custom?: Record<string, unknown>;
                            data?: unknown;
                            media?: undefined;
                            metadata?: Record<string, unknown>;
                            reasoning?: undefined;
                            resource?: undefined;
                            text?: undefined;
                            toolRequest?: undefined;
                            toolResponse?: undefined;
                        }
                        | {
                            custom: Record<string, any>;
                            data?: unknown;
                            media?: undefined;
                            metadata?: Record<string, unknown>;
                            reasoning?: undefined;
                            resource?: undefined;
                            text?: undefined;
                            toolRequest?: undefined;
                            toolResponse?: undefined;
                        }
                        | {
                            custom?: Record<string, unknown>;
                            data?: unknown;
                            media?: undefined;
                            metadata?: Record<string, unknown>;
                            reasoning: string;
                            resource?: undefined;
                            text?: undefined;
                            toolRequest?: undefined;
                            toolResponse?: undefined;
                        }
                        | {
                            custom?: Record<string, unknown>;
                            data?: unknown;
                            media?: undefined;
                            metadata?: Record<string, unknown>;
                            reasoning?: undefined;
                            resource: { uri: string };
                            text?: undefined;
                            toolRequest?: undefined;
                            toolResponse?: undefined;
                        }
                    )[];
                    metadata?: Record<string, unknown>;
                    role: "user" | "model" | "tool" | "system";
                };
                operation?: {
                    action?: string;
                    done?: boolean;
                    error?: z.objectOutputType<
                        { message: z.ZodString },
                        z.ZodTypeAny,
                        "passthrough",
                    >;
                    id: string;
                    metadata?: Record<string, any>;
                    output?: any;
                };
                raw?: unknown;
                request?: {
                    candidates?: number;
                    config?: any;
                    docs?: {
                        content: (
                            | {
                                custom?: ...;
                                data?: ...;
                                media?: ...;
                                metadata?: ...;
                                reasoning?: ...;
                                resource?: ...;
                                text: ...;
                                toolRequest?: ...;
                                toolResponse?: ...;
                            }
                            | {
                                custom?: ...;
                                data?: ...;
                                media: ...;
                                metadata?: ...;
                                reasoning?: ...;
                                resource?: ...;
                                text?: ...;
                                toolRequest?: ...;
                                toolResponse?: ...;
                            }
                            | {
                                custom?: ...;
                                data?: ...;
                                media?: ...;
                                metadata?: ...;
                                reasoning?: ...;
                                resource?: ...;
                                text?: ...;
                                toolRequest: ...;
                                toolResponse?: ...;
                            }
                            | {
                                custom?: ...;
                                data?: ...;
                                media?: ...;
                                metadata?: ...;
                                reasoning?: ...;
                                resource?: ...;
                                text?: ...;
                                toolRequest?: ...;
                                toolResponse: ...;
                            }
                            | {
                                custom?: ...;
                                data?: ...;
                                media?: ...;
                                metadata?: ...;
                                reasoning?: ...;
                                resource?: ...;
                                text?: ...;
                                toolRequest?: ...;
                                toolResponse?: ...;
                            }
                            | {
                                custom: ...;
                                data?: ...;
                                media?: ...;
                                metadata?: ...;
                                reasoning?: ...;
                                resource?: ...;
                                text?: ...;
                                toolRequest?: ...;
                                toolResponse?: ...;
                            }
                            | {
                                custom?: ...;
                                data?: ...;
                                media?: ...;
                                metadata?: ...;
                                reasoning: ...;
                                resource?: ...;
                                text?: ...;
                                toolRequest?: ...;
                                toolResponse?: ...;
                            }
                            | {
                                custom?: ...;
                                data?: ...;
                                media?: ...;
                                metadata?: ...;
                                reasoning?: ...;
                                resource: ...;
                                text?: ...;
                                toolRequest?: ...;
                                toolResponse?: ...;
                            }
                        )[];
                        metadata?: Record<string, any>;
                    }[];
                    messages: {
                        content: (
                            | {
                                custom?: (...)
                                | (...);
                                data?: unknown;
                                media?: undefined;
                                metadata?: (...) | (...);
                                reasoning?: undefined;
                                resource?: undefined;
                                text: string;
                                toolRequest?: undefined;
                                toolResponse?: undefined;
                            }
                            | {
                                custom?: (...)
                                | (...);
                                data?: unknown;
                                media: { contentType?: ...; url: ... };
                                metadata?: (...) | (...);
                                reasoning?: undefined;
                                resource?: undefined;
                                text?: undefined;
                                toolRequest?: undefined;
                                toolResponse?: undefined;
                            }
                            | {
                                custom?: (...)
                                | (...);
                                data?: unknown;
                                media?: undefined;
                                metadata?: (...) | (...);
                                reasoning?: undefined;
                                resource?: undefined;
                                text?: undefined;
                                toolRequest: { input?: ...; name: ...; partial?: ...; ref?: ... };
                                toolResponse?: undefined;
                            }
                            | {
                                custom?: (...)
                                | (...);
                                data?: unknown;
                                media?: undefined;
                                metadata?: (...) | (...);
                                reasoning?: undefined;
                                resource?: undefined;
                                text?: undefined;
                                toolRequest?: undefined;
                                toolResponse: (...) & (...);
                            }
                            | {
                                custom?: (...)
                                | (...);
                                data?: unknown;
                                media?: undefined;
                                metadata?: (...) | (...);
                                reasoning?: undefined;
                                resource?: undefined;
                                text?: undefined;
                                toolRequest?: undefined;
                                toolResponse?: undefined;
                            }
                            | {
                                custom: Record<(...), (...)>;
                                data?: unknown;
                                media?: undefined;
                                metadata?: (...) | (...);
                                reasoning?: undefined;
                                resource?: undefined;
                                text?: undefined;
                                toolRequest?: undefined;
                                toolResponse?: undefined;
                            }
                            | {
                                custom?: (...)
                                | (...);
                                data?: unknown;
                                media?: undefined;
                                metadata?: (...) | (...);
                                reasoning: string;
                                resource?: undefined;
                                text?: undefined;
                                toolRequest?: undefined;
                                toolResponse?: undefined;
                            }
                            | {
                                custom?: (...)
                                | (...);
                                data?: unknown;
                                media?: undefined;
                                metadata?: (...) | (...);
                                reasoning?: undefined;
                                resource: { uri: ... };
                                text?: undefined;
                                toolRequest?: undefined;
                                toolResponse?: undefined;
                            }
                        )[];
                        metadata?: Record<string, unknown>;
                        role: "user" | "model" | "tool" | "system";
                    }[];
                    output?: {
                        constrained?: boolean;
                        contentType?: string;
                        format?: string;
                        schema?: Record<string, any>;
                    };
                    toolChoice?: "required"
                    | "auto"
                    | "none";
                    tools?: {
                        description: string;
                        inputSchema?: Record<string, any> | null;
                        metadata?: Record<string, any>;
                        name: string;
                        outputSchema?: Record<string, any> | null;
                    }[];
                };
                usage?: {
                    cachedContentTokens?: number;
                    custom?: Record<string, number>;
                    inputAudioFiles?: number;
                    inputCharacters?: number;
                    inputImages?: number;
                    inputTokens?: number;
                    inputVideos?: number;
                    outputAudioFiles?: number;
                    outputCharacters?: number;
                    outputImages?: number;
                    outputTokens?: number;
                    outputVideos?: number;
                    thoughtsTokens?: number;
                    totalTokens?: number;
                };
            },
        >

      Returns ModelAction<CustomOptionsSchema>

    • Defines a new model and adds it to the registry.

      Type Parameters

      Parameters

      • options: DefineModelOptions<CustomOptionsSchema>
      • runner: (
            request: GenerateRequest<CustomOptionsSchema>,
            streamingCallback?: StreamingCallback<
                {
                    aggregated?: boolean;
                    content: (
                        | {
                            custom?: Record<string, unknown>;
                            data?: unknown;
                            media?: undefined;
                            metadata?: Record<string, unknown>;
                            reasoning?: undefined;
                            resource?: undefined;
                            text: string;
                            toolRequest?: undefined;
                            toolResponse?: undefined;
                        }
                        | {
                            custom?: Record<string, unknown>;
                            data?: unknown;
                            media: { contentType?: string; url: string };
                            metadata?: Record<string, unknown>;
                            reasoning?: undefined;
                            resource?: undefined;
                            text?: undefined;
                            toolRequest?: undefined;
                            toolResponse?: undefined;
                        }
                        | {
                            custom?: Record<string, unknown>;
                            data?: unknown;
                            media?: undefined;
                            metadata?: Record<string, unknown>;
                            reasoning?: undefined;
                            resource?: undefined;
                            text?: undefined;
                            toolRequest: {
                                input?: unknown;
                                name: string;
                                partial?: boolean;
                                ref?: string;
                            };
                            toolResponse?: undefined;
                        }
                        | {
                            custom?: Record<string, unknown>;
                            data?: unknown;
                            media?: undefined;
                            metadata?: Record<string, unknown>;
                            reasoning?: undefined;
                            resource?: undefined;
                            text?: undefined;
                            toolRequest?: undefined;
                            toolResponse: { name: string; output?: unknown; ref?: string } & {
                                content?: (...)[];
                            };
                        }
                        | {
                            custom?: Record<string, unknown>;
                            data?: unknown;
                            media?: undefined;
                            metadata?: Record<string, unknown>;
                            reasoning?: undefined;
                            resource?: undefined;
                            text?: undefined;
                            toolRequest?: undefined;
                            toolResponse?: undefined;
                        }
                        | {
                            custom: Record<string, any>;
                            data?: unknown;
                            media?: undefined;
                            metadata?: Record<string, unknown>;
                            reasoning?: undefined;
                            resource?: undefined;
                            text?: undefined;
                            toolRequest?: undefined;
                            toolResponse?: undefined;
                        }
                        | {
                            custom?: Record<string, unknown>;
                            data?: unknown;
                            media?: undefined;
                            metadata?: Record<string, unknown>;
                            reasoning: string;
                            resource?: undefined;
                            text?: undefined;
                            toolRequest?: undefined;
                            toolResponse?: undefined;
                        }
                        | {
                            custom?: Record<string, unknown>;
                            data?: unknown;
                            media?: undefined;
                            metadata?: Record<string, unknown>;
                            reasoning?: undefined;
                            resource: { uri: string };
                            text?: undefined;
                            toolRequest?: undefined;
                            toolResponse?: undefined;
                        }
                    )[];
                    custom?: unknown;
                    index?: number;
                    role?: "user"
                    | "model"
                    | "tool"
                    | "system";
                },
            >,
        ) => Promise<
            {
                candidates?: {
                    custom?: unknown;
                    finishMessage?: string;
                    finishReason: | "length"
                    | "stop"
                    | "other"
                    | "unknown"
                    | "interrupted"
                    | "blocked"
                    | "aborted";
                    index: number;
                    message: {
                        content: (
                            | {
                                custom?: (...)
                                | (...);
                                data?: unknown;
                                media?: undefined;
                                metadata?: (...) | (...);
                                reasoning?: undefined;
                                resource?: undefined;
                                text: string;
                                toolRequest?: undefined;
                                toolResponse?: undefined;
                            }
                            | {
                                custom?: (...)
                                | (...);
                                data?: unknown;
                                media: { contentType?: ...; url: ... };
                                metadata?: (...) | (...);
                                reasoning?: undefined;
                                resource?: undefined;
                                text?: undefined;
                                toolRequest?: undefined;
                                toolResponse?: undefined;
                            }
                            | {
                                custom?: (...)
                                | (...);
                                data?: unknown;
                                media?: undefined;
                                metadata?: (...) | (...);
                                reasoning?: undefined;
                                resource?: undefined;
                                text?: undefined;
                                toolRequest: { input?: ...; name: ...; partial?: ...; ref?: ... };
                                toolResponse?: undefined;
                            }
                            | {
                                custom?: (...)
                                | (...);
                                data?: unknown;
                                media?: undefined;
                                metadata?: (...) | (...);
                                reasoning?: undefined;
                                resource?: undefined;
                                text?: undefined;
                                toolRequest?: undefined;
                                toolResponse: (...) & (...);
                            }
                            | {
                                custom?: (...)
                                | (...);
                                data?: unknown;
                                media?: undefined;
                                metadata?: (...) | (...);
                                reasoning?: undefined;
                                resource?: undefined;
                                text?: undefined;
                                toolRequest?: undefined;
                                toolResponse?: undefined;
                            }
                            | {
                                custom: Record<(...), (...)>;
                                data?: unknown;
                                media?: undefined;
                                metadata?: (...) | (...);
                                reasoning?: undefined;
                                resource?: undefined;
                                text?: undefined;
                                toolRequest?: undefined;
                                toolResponse?: undefined;
                            }
                            | {
                                custom?: (...)
                                | (...);
                                data?: unknown;
                                media?: undefined;
                                metadata?: (...) | (...);
                                reasoning: string;
                                resource?: undefined;
                                text?: undefined;
                                toolRequest?: undefined;
                                toolResponse?: undefined;
                            }
                            | {
                                custom?: (...)
                                | (...);
                                data?: unknown;
                                media?: undefined;
                                metadata?: (...) | (...);
                                reasoning?: undefined;
                                resource: { uri: ... };
                                text?: undefined;
                                toolRequest?: undefined;
                                toolResponse?: undefined;
                            }
                        )[];
                        metadata?: Record<string, unknown>;
                        role: "user" | "model" | "tool" | "system";
                    };
                    usage?: {
                        cachedContentTokens?: number;
                        custom?: Record<string, number>;
                        inputAudioFiles?: number;
                        inputCharacters?: number;
                        inputImages?: number;
                        inputTokens?: number;
                        inputVideos?: number;
                        outputAudioFiles?: number;
                        outputCharacters?: number;
                        outputImages?: number;
                        outputTokens?: number;
                        outputVideos?: number;
                        thoughtsTokens?: number;
                        totalTokens?: number;
                    };
                }[];
                custom?: unknown;
                finishMessage?: string;
                finishReason?: | "length"
                | "stop"
                | "other"
                | "unknown"
                | "interrupted"
                | "blocked"
                | "aborted";
                latencyMs?: number;
                message?: {
                    content: (
                        | {
                            custom?: Record<string, unknown>;
                            data?: unknown;
                            media?: undefined;
                            metadata?: Record<string, unknown>;
                            reasoning?: undefined;
                            resource?: undefined;
                            text: string;
                            toolRequest?: undefined;
                            toolResponse?: undefined;
                        }
                        | {
                            custom?: Record<string, unknown>;
                            data?: unknown;
                            media: { contentType?: string; url: string };
                            metadata?: Record<string, unknown>;
                            reasoning?: undefined;
                            resource?: undefined;
                            text?: undefined;
                            toolRequest?: undefined;
                            toolResponse?: undefined;
                        }
                        | {
                            custom?: Record<string, unknown>;
                            data?: unknown;
                            media?: undefined;
                            metadata?: Record<string, unknown>;
                            reasoning?: undefined;
                            resource?: undefined;
                            text?: undefined;
                            toolRequest: {
                                input?: unknown;
                                name: string;
                                partial?: boolean;
                                ref?: string;
                            };
                            toolResponse?: undefined;
                        }
                        | {
                            custom?: Record<string, unknown>;
                            data?: unknown;
                            media?: undefined;
                            metadata?: Record<string, unknown>;
                            reasoning?: undefined;
                            resource?: undefined;
                            text?: undefined;
                            toolRequest?: undefined;
                            toolResponse: { name: string; output?: unknown; ref?: (...) | (...) } & {
                                content?: (...) | (...);
                            };
                        }
                        | {
                            custom?: Record<string, unknown>;
                            data?: unknown;
                            media?: undefined;
                            metadata?: Record<string, unknown>;
                            reasoning?: undefined;
                            resource?: undefined;
                            text?: undefined;
                            toolRequest?: undefined;
                            toolResponse?: undefined;
                        }
                        | {
                            custom: Record<string, any>;
                            data?: unknown;
                            media?: undefined;
                            metadata?: Record<string, unknown>;
                            reasoning?: undefined;
                            resource?: undefined;
                            text?: undefined;
                            toolRequest?: undefined;
                            toolResponse?: undefined;
                        }
                        | {
                            custom?: Record<string, unknown>;
                            data?: unknown;
                            media?: undefined;
                            metadata?: Record<string, unknown>;
                            reasoning: string;
                            resource?: undefined;
                            text?: undefined;
                            toolRequest?: undefined;
                            toolResponse?: undefined;
                        }
                        | {
                            custom?: Record<string, unknown>;
                            data?: unknown;
                            media?: undefined;
                            metadata?: Record<string, unknown>;
                            reasoning?: undefined;
                            resource: { uri: string };
                            text?: undefined;
                            toolRequest?: undefined;
                            toolResponse?: undefined;
                        }
                    )[];
                    metadata?: Record<string, unknown>;
                    role: "user" | "model" | "tool" | "system";
                };
                operation?: {
                    action?: string;
                    done?: boolean;
                    error?: z.objectOutputType<
                        { message: z.ZodString },
                        z.ZodTypeAny,
                        "passthrough",
                    >;
                    id: string;
                    metadata?: Record<string, any>;
                    output?: any;
                };
                raw?: unknown;
                request?: {
                    candidates?: number;
                    config?: any;
                    docs?: {
                        content: (
                            | {
                                custom?: ...;
                                data?: ...;
                                media?: ...;
                                metadata?: ...;
                                reasoning?: ...;
                                resource?: ...;
                                text: ...;
                                toolRequest?: ...;
                                toolResponse?: ...;
                            }
                            | {
                                custom?: ...;
                                data?: ...;
                                media: ...;
                                metadata?: ...;
                                reasoning?: ...;
                                resource?: ...;
                                text?: ...;
                                toolRequest?: ...;
                                toolResponse?: ...;
                            }
                            | {
                                custom?: ...;
                                data?: ...;
                                media?: ...;
                                metadata?: ...;
                                reasoning?: ...;
                                resource?: ...;
                                text?: ...;
                                toolRequest: ...;
                                toolResponse?: ...;
                            }
                            | {
                                custom?: ...;
                                data?: ...;
                                media?: ...;
                                metadata?: ...;
                                reasoning?: ...;
                                resource?: ...;
                                text?: ...;
                                toolRequest?: ...;
                                toolResponse: ...;
                            }
                            | {
                                custom?: ...;
                                data?: ...;
                                media?: ...;
                                metadata?: ...;
                                reasoning?: ...;
                                resource?: ...;
                                text?: ...;
                                toolRequest?: ...;
                                toolResponse?: ...;
                            }
                            | {
                                custom: ...;
                                data?: ...;
                                media?: ...;
                                metadata?: ...;
                                reasoning?: ...;
                                resource?: ...;
                                text?: ...;
                                toolRequest?: ...;
                                toolResponse?: ...;
                            }
                            | {
                                custom?: ...;
                                data?: ...;
                                media?: ...;
                                metadata?: ...;
                                reasoning: ...;
                                resource?: ...;
                                text?: ...;
                                toolRequest?: ...;
                                toolResponse?: ...;
                            }
                            | {
                                custom?: ...;
                                data?: ...;
                                media?: ...;
                                metadata?: ...;
                                reasoning?: ...;
                                resource: ...;
                                text?: ...;
                                toolRequest?: ...;
                                toolResponse?: ...;
                            }
                        )[];
                        metadata?: Record<string, any>;
                    }[];
                    messages: {
                        content: (
                            | {
                                custom?: (...)
                                | (...);
                                data?: unknown;
                                media?: undefined;
                                metadata?: (...) | (...);
                                reasoning?: undefined;
                                resource?: undefined;
                                text: string;
                                toolRequest?: undefined;
                                toolResponse?: undefined;
                            }
                            | {
                                custom?: (...)
                                | (...);
                                data?: unknown;
                                media: { contentType?: ...; url: ... };
                                metadata?: (...) | (...);
                                reasoning?: undefined;
                                resource?: undefined;
                                text?: undefined;
                                toolRequest?: undefined;
                                toolResponse?: undefined;
                            }
                            | {
                                custom?: (...)
                                | (...);
                                data?: unknown;
                                media?: undefined;
                                metadata?: (...) | (...);
                                reasoning?: undefined;
                                resource?: undefined;
                                text?: undefined;
                                toolRequest: { input?: ...; name: ...; partial?: ...; ref?: ... };
                                toolResponse?: undefined;
                            }
                            | {
                                custom?: (...)
                                | (...);
                                data?: unknown;
                                media?: undefined;
                                metadata?: (...) | (...);
                                reasoning?: undefined;
                                resource?: undefined;
                                text?: undefined;
                                toolRequest?: undefined;
                                toolResponse: (...) & (...);
                            }
                            | {
                                custom?: (...)
                                | (...);
                                data?: unknown;
                                media?: undefined;
                                metadata?: (...) | (...);
                                reasoning?: undefined;
                                resource?: undefined;
                                text?: undefined;
                                toolRequest?: undefined;
                                toolResponse?: undefined;
                            }
                            | {
                                custom: Record<(...), (...)>;
                                data?: unknown;
                                media?: undefined;
                                metadata?: (...) | (...);
                                reasoning?: undefined;
                                resource?: undefined;
                                text?: undefined;
                                toolRequest?: undefined;
                                toolResponse?: undefined;
                            }
                            | {
                                custom?: (...)
                                | (...);
                                data?: unknown;
                                media?: undefined;
                                metadata?: (...) | (...);
                                reasoning: string;
                                resource?: undefined;
                                text?: undefined;
                                toolRequest?: undefined;
                                toolResponse?: undefined;
                            }
                            | {
                                custom?: (...)
                                | (...);
                                data?: unknown;
                                media?: undefined;
                                metadata?: (...) | (...);
                                reasoning?: undefined;
                                resource: { uri: ... };
                                text?: undefined;
                                toolRequest?: undefined;
                                toolResponse?: undefined;
                            }
                        )[];
                        metadata?: Record<string, unknown>;
                        role: "user" | "model" | "tool" | "system";
                    }[];
                    output?: {
                        constrained?: boolean;
                        contentType?: string;
                        format?: string;
                        schema?: Record<string, any>;
                    };
                    toolChoice?: "required"
                    | "auto"
                    | "none";
                    tools?: {
                        description: string;
                        inputSchema?: Record<string, any> | null;
                        metadata?: Record<string, any>;
                        name: string;
                        outputSchema?: Record<string, any> | null;
                    }[];
                };
                usage?: {
                    cachedContentTokens?: number;
                    custom?: Record<string, number>;
                    inputAudioFiles?: number;
                    inputCharacters?: number;
                    inputImages?: number;
                    inputTokens?: number;
                    inputVideos?: number;
                    outputAudioFiles?: number;
                    outputCharacters?: number;
                    outputImages?: number;
                    outputTokens?: number;
                    outputVideos?: number;
                    thoughtsTokens?: number;
                    totalTokens?: number;
                };
            },
        >

      Returns ModelAction<CustomOptionsSchema>

    • Beta

      Defines a resource. Resources can then be accessed from a generate call.

      ai.defineResource({
      uri: 'my://resource/{param}',
      description: 'provides my resource',
      }, async ({param}) => {
      return [{ text: `resource ${param}` }]
      });

      await ai.generate({
      prompt: [{ resource: 'my://resource/value' }]

      Parameters

      • opts: ResourceOptions
      • fn: ResourceFn

      Returns ResourceAction

    • A veneer for interacting with embedder models in bulk.

      Type Parameters

      Parameters

      • params: {
            content:
                | string[]
                | {
                    content: (
                        | {
                            custom?: Record<string, unknown>;
                            data?: unknown;
                            media?: undefined;
                            metadata?: Record<string, unknown>;
                            reasoning?: undefined;
                            resource?: undefined;
                            text: string;
                            toolRequest?: undefined;
                            toolResponse?: undefined;
                        }
                        | {
                            custom?: Record<string, unknown>;
                            data?: unknown;
                            media: { contentType?: string; url: string };
                            metadata?: Record<string, unknown>;
                            reasoning?: undefined;
                            resource?: undefined;
                            text?: undefined;
                            toolRequest?: undefined;
                            toolResponse?: undefined;
                        }
                        | {
                            custom?: Record<string, unknown>;
                            data?: unknown;
                            media?: undefined;
                            metadata?: Record<string, unknown>;
                            reasoning?: undefined;
                            resource?: undefined;
                            text?: undefined;
                            toolRequest: {
                                input?: unknown;
                                name: string;
                                partial?: boolean;
                                ref?: string;
                            };
                            toolResponse?: undefined;
                        }
                        | {
                            custom?: Record<string, unknown>;
                            data?: unknown;
                            media?: undefined;
                            metadata?: Record<string, unknown>;
                            reasoning?: undefined;
                            resource?: undefined;
                            text?: undefined;
                            toolRequest?: undefined;
                            toolResponse: { name: string; output?: unknown; ref?: string } & {
                                content?: (...)[];
                            };
                        }
                        | {
                            custom?: Record<string, unknown>;
                            data?: unknown;
                            media?: undefined;
                            metadata?: Record<string, unknown>;
                            reasoning?: undefined;
                            resource?: undefined;
                            text?: undefined;
                            toolRequest?: undefined;
                            toolResponse?: undefined;
                        }
                        | {
                            custom: Record<string, any>;
                            data?: unknown;
                            media?: undefined;
                            metadata?: Record<string, unknown>;
                            reasoning?: undefined;
                            resource?: undefined;
                            text?: undefined;
                            toolRequest?: undefined;
                            toolResponse?: undefined;
                        }
                        | {
                            custom?: Record<string, unknown>;
                            data?: unknown;
                            media?: undefined;
                            metadata?: Record<string, unknown>;
                            reasoning: string;
                            resource?: undefined;
                            text?: undefined;
                            toolRequest?: undefined;
                            toolResponse?: undefined;
                        }
                        | {
                            custom?: Record<string, unknown>;
                            data?: unknown;
                            media?: undefined;
                            metadata?: Record<string, unknown>;
                            reasoning?: undefined;
                            resource: { uri: string };
                            text?: undefined;
                            toolRequest?: undefined;
                            toolResponse?: undefined;
                        }
                    )[];
                    metadata?: Record<string, any>;
                }[];
            embedder: EmbedderArgument<ConfigSchema>;
            metadata?: Record<string, unknown>;
            options?: z.TypeOf<ConfigSchema>;
        }

      Returns Promise<EmbeddingBatch>

    • Evaluates the given dataset using the specified evaluator.

      Type Parameters

      • DataPoint extends z.ZodObject<
            {
                context: z.ZodOptional<z.ZodArray<z.ZodUnknown, "many">>;
                input: z.ZodUnknown;
                output: z.ZodOptional<z.ZodUnknown>;
                reference: z.ZodOptional<z.ZodUnknown>;
                testCaseId: z.ZodOptional<z.ZodString>;
                traceIds: z.ZodOptional<z.ZodArray<z.ZodString, "many">>;
            },
            "strip",
            z.ZodTypeAny,
            {
                context?: unknown[];
                input?: unknown;
                output?: unknown;
                reference?: unknown;
                testCaseId?: string;
                traceIds?: string[];
            },
            {
                context?: unknown[];
                input?: unknown;
                output?: unknown;
                reference?: unknown;
                testCaseId?: string;
                traceIds?: string[];
            },
        > = z.ZodObject<
            {
                context: z.ZodOptional<z.ZodArray<z.ZodUnknown, "many">>;
                input: z.ZodUnknown;
                output: z.ZodOptional<z.ZodUnknown>;
                reference: z.ZodOptional<z.ZodUnknown>;
                testCaseId: z.ZodOptional<z.ZodString>;
                traceIds: z.ZodOptional<z.ZodArray<z.ZodString, "many">>;
            },
            "strip",
            z.ZodTypeAny,
            {
                context?: unknown[];
                input?: unknown;
                output?: unknown;
                reference?: unknown;
                testCaseId?: string;
                traceIds?: string[];
            },
            {
                context?: unknown[];
                input?: unknown;
                output?: unknown;
                reference?: unknown;
                testCaseId?: string;
                traceIds?: string[];
            },
        >
      • CustomOptions extends z.ZodTypeAny = z.ZodTypeAny

      Returns Promise<
          {
              evaluation: | {
                  details?: z.objectOutputType<
                      { reasoning: z.ZodOptional<z.ZodString> },
                      z.ZodTypeAny,
                      "passthrough",
                  >;
                  error?: string;
                  id?: string;
                  score?: string | number | boolean;
                  status?: "UNKNOWN" | "PASS" | "FAIL";
              }
              | {
                  details?: z.objectOutputType<
                      { reasoning: z.ZodOptional<z.ZodString> },
                      z.ZodTypeAny,
                      "passthrough",
                  >;
                  error?: string;
                  id?: string;
                  score?: string | number | boolean;
                  status?: "UNKNOWN" | "PASS" | "FAIL";
              }[];
              sampleIndex?: number;
              spanId?: string;
              testCaseId: string;
              traceId?: string;
          }[],
      >

    • Make a generate call to the default model with a simple text prompt.

      const ai = genkit({
      plugins: [googleAI()],
      model: googleAI.model('gemini-flash-latest'), // default model
      })

      const { text } = await ai.generate('hi');

      Type Parameters

      Parameters

      • strPrompt: string

      Returns Promise<GenerateResponse<z.TypeOf<O>>>

    • Make a generate call to the default model with a multipart request.

      const ai = genkit({
      plugins: [googleAI()],
      model: googleAI.model('gemini-flash-latest'), // default model
      })

      const { text } = await ai.generate([
      { media: {url: 'http://....'} },
      { text: 'describe this image' }
      ]);

      Type Parameters

      Parameters

      • parts: (
            | {
                custom?: Record<string, unknown>;
                data?: unknown;
                media?: undefined;
                metadata?: Record<string, unknown>;
                reasoning?: undefined;
                resource?: undefined;
                text: string;
                toolRequest?: undefined;
                toolResponse?: undefined;
            }
            | {
                custom?: Record<string, unknown>;
                data?: unknown;
                media: { contentType?: string; url: string };
                metadata?: Record<string, unknown>;
                reasoning?: undefined;
                resource?: undefined;
                text?: undefined;
                toolRequest?: undefined;
                toolResponse?: undefined;
            }
            | {
                custom?: Record<string, unknown>;
                data?: unknown;
                media?: undefined;
                metadata?: Record<string, unknown>;
                reasoning?: undefined;
                resource?: undefined;
                text?: undefined;
                toolRequest: {
                    input?: unknown;
                    name: string;
                    partial?: boolean;
                    ref?: string;
                };
                toolResponse?: undefined;
            }
            | {
                custom?: Record<string, unknown>;
                data?: unknown;
                media?: undefined;
                metadata?: Record<string, unknown>;
                reasoning?: undefined;
                resource?: undefined;
                text?: undefined;
                toolRequest?: undefined;
                toolResponse: { name: string; output?: unknown; ref?: string } & {
                    content?: ({ text: string; custom?: Record<string, unknown> | undefined; metadata?: Record<string, unknown> | undefined; media?: undefined; toolRequest?: undefined; toolResponse?: undefined; data?: unknown; reasoning?: undefined; resource?: undefined; } | ... 6 more ... | { ...; })[];
                };
            }
            | {
                custom?: Record<string, unknown>;
                data?: unknown;
                media?: undefined;
                metadata?: Record<string, unknown>;
                reasoning?: undefined;
                resource?: undefined;
                text?: undefined;
                toolRequest?: undefined;
                toolResponse?: undefined;
            }
            | {
                custom: Record<string, any>;
                data?: unknown;
                media?: undefined;
                metadata?: Record<string, unknown>;
                reasoning?: undefined;
                resource?: undefined;
                text?: undefined;
                toolRequest?: undefined;
                toolResponse?: undefined;
            }
            | {
                custom?: Record<string, unknown>;
                data?: unknown;
                media?: undefined;
                metadata?: Record<string, unknown>;
                reasoning: string;
                resource?: undefined;
                text?: undefined;
                toolRequest?: undefined;
                toolResponse?: undefined;
            }
            | {
                custom?: Record<string, unknown>;
                data?: unknown;
                media?: undefined;
                metadata?: Record<string, unknown>;
                reasoning?: undefined;
                resource: { uri: string };
                text?: undefined;
                toolRequest?: undefined;
                toolResponse?: undefined;
            }
        )[]

      Returns Promise<GenerateResponse<z.TypeOf<O>>>

    • Generate calls a generative model based on the provided prompt and configuration. If messages is provided, the generation will include a conversation history in its request. If tools are provided, the generate method will automatically resolve tool calls returned from the model unless returnToolRequests is set to true.

      See GenerateOptions for detailed information about available options.

      const ai = genkit({
      plugins: [googleAI()],
      })

      const { text } = await ai.generate({
      system: 'talk like a pirate',
      prompt: [
      { media: { url: 'http://....' } },
      { text: 'describe this image' }
      ],
      messages: conversationHistory,
      tools: [ userInfoLookup ],
      model: googleAI.model('gemini-flash-latest'),
      });

      Type Parameters

      Parameters

      Returns Promise<GenerateResponse<z.TypeOf<O>>>

    • Beta

      Starts a generate operation for long running generation models, typically for video and complex audio generation.

      See GenerateOptions for detailed information about available options.

      const operation = await ai.generateOperation({
      model: googleAI.model('veo-2.0-generate-001'),
      prompt: 'A banana riding a bicycle.',
      });

      The status of the operation and final result can be obtained using Genkit.checkOperation.

      Type Parameters

      Parameters

      Returns Promise<
          Operation<
              {
                  candidates?: {
                      custom?: unknown;
                      finishMessage?: string;
                      finishReason: | "length"
                      | "stop"
                      | "other"
                      | "unknown"
                      | "interrupted"
                      | "blocked"
                      | "aborted";
                      index: number;
                      message: {
                          content: (
                              | {
                                  custom?: (...)
                                  | (...);
                                  data?: unknown;
                                  media?: undefined;
                                  metadata?: (...) | (...);
                                  reasoning?: undefined;
                                  resource?: undefined;
                                  text: string;
                                  toolRequest?: undefined;
                                  toolResponse?: undefined;
                              }
                              | {
                                  custom?: (...)
                                  | (...);
                                  data?: unknown;
                                  media: { contentType?: ...; url: ... };
                                  metadata?: (...) | (...);
                                  reasoning?: undefined;
                                  resource?: undefined;
                                  text?: undefined;
                                  toolRequest?: undefined;
                                  toolResponse?: undefined;
                              }
                              | {
                                  custom?: (...)
                                  | (...);
                                  data?: unknown;
                                  media?: undefined;
                                  metadata?: (...) | (...);
                                  reasoning?: undefined;
                                  resource?: undefined;
                                  text?: undefined;
                                  toolRequest: { input?: ...; name: ...; partial?: ...; ref?: ... };
                                  toolResponse?: undefined;
                              }
                              | {
                                  custom?: (...)
                                  | (...);
                                  data?: unknown;
                                  media?: undefined;
                                  metadata?: (...) | (...);
                                  reasoning?: undefined;
                                  resource?: undefined;
                                  text?: undefined;
                                  toolRequest?: undefined;
                                  toolResponse: (...) & (...);
                              }
                              | {
                                  custom?: (...)
                                  | (...);
                                  data?: unknown;
                                  media?: undefined;
                                  metadata?: (...) | (...);
                                  reasoning?: undefined;
                                  resource?: undefined;
                                  text?: undefined;
                                  toolRequest?: undefined;
                                  toolResponse?: undefined;
                              }
                              | {
                                  custom: Record<(...), (...)>;
                                  data?: unknown;
                                  media?: undefined;
                                  metadata?: (...) | (...);
                                  reasoning?: undefined;
                                  resource?: undefined;
                                  text?: undefined;
                                  toolRequest?: undefined;
                                  toolResponse?: undefined;
                              }
                              | {
                                  custom?: (...)
                                  | (...);
                                  data?: unknown;
                                  media?: undefined;
                                  metadata?: (...) | (...);
                                  reasoning: string;
                                  resource?: undefined;
                                  text?: undefined;
                                  toolRequest?: undefined;
                                  toolResponse?: undefined;
                              }
                              | {
                                  custom?: (...)
                                  | (...);
                                  data?: unknown;
                                  media?: undefined;
                                  metadata?: (...) | (...);
                                  reasoning?: undefined;
                                  resource: { uri: ... };
                                  text?: undefined;
                                  toolRequest?: undefined;
                                  toolResponse?: undefined;
                              }
                          )[];
                          metadata?: Record<string, unknown>;
                          role: "user" | "model" | "tool" | "system";
                      };
                      usage?: {
                          cachedContentTokens?: number;
                          custom?: Record<string, number>;
                          inputAudioFiles?: number;
                          inputCharacters?: number;
                          inputImages?: number;
                          inputTokens?: number;
                          inputVideos?: number;
                          outputAudioFiles?: number;
                          outputCharacters?: number;
                          outputImages?: number;
                          outputTokens?: number;
                          outputVideos?: number;
                          thoughtsTokens?: number;
                          totalTokens?: number;
                      };
                  }[];
                  custom?: unknown;
                  finishMessage?: string;
                  finishReason?: | "length"
                  | "stop"
                  | "other"
                  | "unknown"
                  | "interrupted"
                  | "blocked"
                  | "aborted";
                  latencyMs?: number;
                  message?: {
                      content: (
                          | {
                              custom?: Record<string, unknown>;
                              data?: unknown;
                              media?: undefined;
                              metadata?: Record<string, unknown>;
                              reasoning?: undefined;
                              resource?: undefined;
                              text: string;
                              toolRequest?: undefined;
                              toolResponse?: undefined;
                          }
                          | {
                              custom?: Record<string, unknown>;
                              data?: unknown;
                              media: { contentType?: string; url: string };
                              metadata?: Record<string, unknown>;
                              reasoning?: undefined;
                              resource?: undefined;
                              text?: undefined;
                              toolRequest?: undefined;
                              toolResponse?: undefined;
                          }
                          | {
                              custom?: Record<string, unknown>;
                              data?: unknown;
                              media?: undefined;
                              metadata?: Record<string, unknown>;
                              reasoning?: undefined;
                              resource?: undefined;
                              text?: undefined;
                              toolRequest: {
                                  input?: unknown;
                                  name: string;
                                  partial?: boolean;
                                  ref?: string;
                              };
                              toolResponse?: undefined;
                          }
                          | {
                              custom?: Record<string, unknown>;
                              data?: unknown;
                              media?: undefined;
                              metadata?: Record<string, unknown>;
                              reasoning?: undefined;
                              resource?: undefined;
                              text?: undefined;
                              toolRequest?: undefined;
                              toolResponse: { name: string; output?: unknown; ref?: (...) | (...) } & {
                                  content?: (...) | (...);
                              };
                          }
                          | {
                              custom?: Record<string, unknown>;
                              data?: unknown;
                              media?: undefined;
                              metadata?: Record<string, unknown>;
                              reasoning?: undefined;
                              resource?: undefined;
                              text?: undefined;
                              toolRequest?: undefined;
                              toolResponse?: undefined;
                          }
                          | {
                              custom: Record<string, any>;
                              data?: unknown;
                              media?: undefined;
                              metadata?: Record<string, unknown>;
                              reasoning?: undefined;
                              resource?: undefined;
                              text?: undefined;
                              toolRequest?: undefined;
                              toolResponse?: undefined;
                          }
                          | {
                              custom?: Record<string, unknown>;
                              data?: unknown;
                              media?: undefined;
                              metadata?: Record<string, unknown>;
                              reasoning: string;
                              resource?: undefined;
                              text?: undefined;
                              toolRequest?: undefined;
                              toolResponse?: undefined;
                          }
                          | {
                              custom?: Record<string, unknown>;
                              data?: unknown;
                              media?: undefined;
                              metadata?: Record<string, unknown>;
                              reasoning?: undefined;
                              resource: { uri: string };
                              text?: undefined;
                              toolRequest?: undefined;
                              toolResponse?: undefined;
                          }
                      )[];
                      metadata?: Record<string, unknown>;
                      role: "user" | "model" | "tool" | "system";
                  };
                  operation?: {
                      action?: string;
                      done?: boolean;
                      error?: z.objectOutputType<
                          { message: z.ZodString },
                          z.ZodTypeAny,
                          "passthrough",
                      >;
                      id: string;
                      metadata?: Record<string, any>;
                      output?: any;
                  };
                  raw?: unknown;
                  request?: {
                      candidates?: number;
                      config?: any;
                      docs?: {
                          content: (
                              | {
                                  custom?: ...;
                                  data?: ...;
                                  media?: ...;
                                  metadata?: ...;
                                  reasoning?: ...;
                                  resource?: ...;
                                  text: ...;
                                  toolRequest?: ...;
                                  toolResponse?: ...;
                              }
                              | {
                                  custom?: ...;
                                  data?: ...;
                                  media: ...;
                                  metadata?: ...;
                                  reasoning?: ...;
                                  resource?: ...;
                                  text?: ...;
                                  toolRequest?: ...;
                                  toolResponse?: ...;
                              }
                              | {
                                  custom?: ...;
                                  data?: ...;
                                  media?: ...;
                                  metadata?: ...;
                                  reasoning?: ...;
                                  resource?: ...;
                                  text?: ...;
                                  toolRequest: ...;
                                  toolResponse?: ...;
                              }
                              | {
                                  custom?: ...;
                                  data?: ...;
                                  media?: ...;
                                  metadata?: ...;
                                  reasoning?: ...;
                                  resource?: ...;
                                  text?: ...;
                                  toolRequest?: ...;
                                  toolResponse: ...;
                              }
                              | {
                                  custom?: ...;
                                  data?: ...;
                                  media?: ...;
                                  metadata?: ...;
                                  reasoning?: ...;
                                  resource?: ...;
                                  text?: ...;
                                  toolRequest?: ...;
                                  toolResponse?: ...;
                              }
                              | {
                                  custom: ...;
                                  data?: ...;
                                  media?: ...;
                                  metadata?: ...;
                                  reasoning?: ...;
                                  resource?: ...;
                                  text?: ...;
                                  toolRequest?: ...;
                                  toolResponse?: ...;
                              }
                              | {
                                  custom?: ...;
                                  data?: ...;
                                  media?: ...;
                                  metadata?: ...;
                                  reasoning: ...;
                                  resource?: ...;
                                  text?: ...;
                                  toolRequest?: ...;
                                  toolResponse?: ...;
                              }
                              | {
                                  custom?: ...;
                                  data?: ...;
                                  media?: ...;
                                  metadata?: ...;
                                  reasoning?: ...;
                                  resource: ...;
                                  text?: ...;
                                  toolRequest?: ...;
                                  toolResponse?: ...;
                              }
                          )[];
                          metadata?: Record<string, any>;
                      }[];
                      messages: {
                          content: (
                              | {
                                  custom?: (...)
                                  | (...);
                                  data?: unknown;
                                  media?: undefined;
                                  metadata?: (...) | (...);
                                  reasoning?: undefined;
                                  resource?: undefined;
                                  text: string;
                                  toolRequest?: undefined;
                                  toolResponse?: undefined;
                              }
                              | {
                                  custom?: (...)
                                  | (...);
                                  data?: unknown;
                                  media: { contentType?: ...; url: ... };
                                  metadata?: (...) | (...);
                                  reasoning?: undefined;
                                  resource?: undefined;
                                  text?: undefined;
                                  toolRequest?: undefined;
                                  toolResponse?: undefined;
                              }
                              | {
                                  custom?: (...)
                                  | (...);
                                  data?: unknown;
                                  media?: undefined;
                                  metadata?: (...) | (...);
                                  reasoning?: undefined;
                                  resource?: undefined;
                                  text?: undefined;
                                  toolRequest: { input?: ...; name: ...; partial?: ...; ref?: ... };
                                  toolResponse?: undefined;
                              }
                              | {
                                  custom?: (...)
                                  | (...);
                                  data?: unknown;
                                  media?: undefined;
                                  metadata?: (...) | (...);
                                  reasoning?: undefined;
                                  resource?: undefined;
                                  text?: undefined;
                                  toolRequest?: undefined;
                                  toolResponse: (...) & (...);
                              }
                              | {
                                  custom?: (...)
                                  | (...);
                                  data?: unknown;
                                  media?: undefined;
                                  metadata?: (...) | (...);
                                  reasoning?: undefined;
                                  resource?: undefined;
                                  text?: undefined;
                                  toolRequest?: undefined;
                                  toolResponse?: undefined;
                              }
                              | {
                                  custom: Record<(...), (...)>;
                                  data?: unknown;
                                  media?: undefined;
                                  metadata?: (...) | (...);
                                  reasoning?: undefined;
                                  resource?: undefined;
                                  text?: undefined;
                                  toolRequest?: undefined;
                                  toolResponse?: undefined;
                              }
                              | {
                                  custom?: (...)
                                  | (...);
                                  data?: unknown;
                                  media?: undefined;
                                  metadata?: (...) | (...);
                                  reasoning: string;
                                  resource?: undefined;
                                  text?: undefined;
                                  toolRequest?: undefined;
                                  toolResponse?: undefined;
                              }
                              | {
                                  custom?: (...)
                                  | (...);
                                  data?: unknown;
                                  media?: undefined;
                                  metadata?: (...) | (...);
                                  reasoning?: undefined;
                                  resource: { uri: ... };
                                  text?: undefined;
                                  toolRequest?: undefined;
                                  toolResponse?: undefined;
                              }
                          )[];
                          metadata?: Record<string, unknown>;
                          role: "user" | "model" | "tool" | "system";
                      }[];
                      output?: {
                          constrained?: boolean;
                          contentType?: string;
                          format?: string;
                          schema?: Record<string, any>;
                      };
                      toolChoice?: "required"
                      | "auto"
                      | "none";
                      tools?: {
                          description: string;
                          inputSchema?: Record<string, any> | null;
                          metadata?: Record<string, any>;
                          name: string;
                          outputSchema?: Record<string, any> | null;
                      }[];
                  };
                  usage?: {
                      cachedContentTokens?: number;
                      custom?: Record<string, number>;
                      inputAudioFiles?: number;
                      inputCharacters?: number;
                      inputImages?: number;
                      inputTokens?: number;
                      inputVideos?: number;
                      outputAudioFiles?: number;
                      outputCharacters?: number;
                      outputImages?: number;
                      outputTokens?: number;
                      outputVideos?: number;
                      thoughtsTokens?: number;
                      totalTokens?: number;
                  };
              },
          >,
      >

    • Make a streaming generate call to the default model with a simple text prompt.

      const ai = genkit({
      plugins: [googleAI()],
      model: googleAI.model('gemini-flash-latest'), // default model
      })

      const { response, stream } = ai.generateStream('hi');
      for await (const chunk of stream) {
      console.log(chunk.text);
      }
      console.log((await response).text);

      Type Parameters

      Parameters

      • strPrompt: string

      Returns GenerateStreamResponse<z.TypeOf<O>>

    • Make a streaming generate call to the default model with a multipart request.

      const ai = genkit({
      plugins: [googleAI()],
      model: googleAI.model('gemini-flash-latest'), // default model
      })

      const { response, stream } = ai.generateStream([
      { media: {url: 'http://....'} },
      { text: 'describe this image' }
      ]);
      for await (const chunk of stream) {
      console.log(chunk.text);
      }
      console.log((await response).text);

      Type Parameters

      Parameters

      • parts: (
            | {
                custom?: Record<string, unknown>;
                data?: unknown;
                media?: undefined;
                metadata?: Record<string, unknown>;
                reasoning?: undefined;
                resource?: undefined;
                text: string;
                toolRequest?: undefined;
                toolResponse?: undefined;
            }
            | {
                custom?: Record<string, unknown>;
                data?: unknown;
                media: { contentType?: string; url: string };
                metadata?: Record<string, unknown>;
                reasoning?: undefined;
                resource?: undefined;
                text?: undefined;
                toolRequest?: undefined;
                toolResponse?: undefined;
            }
            | {
                custom?: Record<string, unknown>;
                data?: unknown;
                media?: undefined;
                metadata?: Record<string, unknown>;
                reasoning?: undefined;
                resource?: undefined;
                text?: undefined;
                toolRequest: {
                    input?: unknown;
                    name: string;
                    partial?: boolean;
                    ref?: string;
                };
                toolResponse?: undefined;
            }
            | {
                custom?: Record<string, unknown>;
                data?: unknown;
                media?: undefined;
                metadata?: Record<string, unknown>;
                reasoning?: undefined;
                resource?: undefined;
                text?: undefined;
                toolRequest?: undefined;
                toolResponse: { name: string; output?: unknown; ref?: string } & {
                    content?: ({ text: string; custom?: Record<string, unknown> | undefined; metadata?: Record<string, unknown> | undefined; media?: undefined; toolRequest?: undefined; toolResponse?: undefined; data?: unknown; reasoning?: undefined; resource?: undefined; } | ... 6 more ... | { ...; })[];
                };
            }
            | {
                custom?: Record<string, unknown>;
                data?: unknown;
                media?: undefined;
                metadata?: Record<string, unknown>;
                reasoning?: undefined;
                resource?: undefined;
                text?: undefined;
                toolRequest?: undefined;
                toolResponse?: undefined;
            }
            | {
                custom: Record<string, any>;
                data?: unknown;
                media?: undefined;
                metadata?: Record<string, unknown>;
                reasoning?: undefined;
                resource?: undefined;
                text?: undefined;
                toolRequest?: undefined;
                toolResponse?: undefined;
            }
            | {
                custom?: Record<string, unknown>;
                data?: unknown;
                media?: undefined;
                metadata?: Record<string, unknown>;
                reasoning: string;
                resource?: undefined;
                text?: undefined;
                toolRequest?: undefined;
                toolResponse?: undefined;
            }
            | {
                custom?: Record<string, unknown>;
                data?: unknown;
                media?: undefined;
                metadata?: Record<string, unknown>;
                reasoning?: undefined;
                resource: { uri: string };
                text?: undefined;
                toolRequest?: undefined;
                toolResponse?: undefined;
            }
        )[]

      Returns GenerateStreamResponse<z.TypeOf<O>>

    • Streaming generate calls a generative model based on the provided prompt and configuration. If messages is provided, the generation will include a conversation history in its request. If tools are provided, the generate method will automatically resolve tool calls returned from the model unless returnToolRequests is set to true.

      See GenerateOptions for detailed information about available options.

      const ai = genkit({
      plugins: [googleAI()],
      })

      const { response, stream } = ai.generateStream({
      system: 'talk like a pirate',
      prompt: [
      { media: { url: 'http://....' } },
      { text: 'describe this image' }
      ],
      messages: conversationHistory,
      tools: [ userInfoLookup ],
      model: googleAI.model('gemini-flash-latest'),
      });
      for await (const chunk of stream) {
      console.log(chunk.text);
      }
      console.log((await response).text);

      Type Parameters

      Parameters

      Returns GenerateStreamResponse<z.TypeOf<O>>

    • Beta

      Loads a session from the store.

      Parameters

      • sessionId: string
      • options: SessionOptions

      Returns Promise<Session<any>>

    • A flow step that executes the provided function. Each run step is recorded separately in the trace.

      ai.defineFlow('hello', async() => {
      await ai.run('step1', async () => {
      // ... step 1
      });
      await ai.run('step2', async () => {
      // ... step 2
      });
      return result;
      })

      Type Parameters

      • T

      Parameters

      • name: string
      • func: () => Promise<T>

      Returns Promise<T>

    • A flow step that executes the provided function. Each run step is recorded separately in the trace.

      ai.defineFlow('hello', async(name) => {
      const greeting = await ai.run('step1', name, async (input) => {
      return `Hello, ${input}!`;
      });
      const result = await ai.run('step2', greeting, async (input) => {
      // ... step 2
      });
      return result;

      Type Parameters

      • T

      Parameters

      • name: string
      • input: any
      • func: (input?: any) => Promise<T>

      Returns Promise<T>