BetaReadonlyflowsList of flows that have been registered in this instance.
ReadonlyoptionsDeveloper-configured options.
Readonly BetaregistryBetaCreate a chat session with the provided preamble.
const triageAgent = ai.definePrompt({
system: 'help the user triage a problem',
})
const chat = ai.chat(triageAgent)
const { text } = await chat.send('my phone feels hot');
Optionaloptions: ChatOptions<I>Checks the status of of a given operation. Returns a new operation which will contain the updated status.
let operation = await ai.generateOperation({
model: googleAI.model('veo-2.0-generate-001'),
prompt: 'A banana riding a bicycle.',
});
while (!operation.done) {
operation = await ai.checkOperation(operation!);
await new Promise((resolve) => setTimeout(resolve, 5000));
}
Returns current action (or flow) invocation context. Can be used to access things like auth
data set by HTTP server frameworks. If invoked outside of an action (e.g. flow or tool) will
return undefined.
Defines a new background model and adds it to the registry.
Defines and registers a dynamic action provider (e.g. mcp host)
Creates embedder model for the provided EmbedderFn model implementation.
Creates evaluator action for the provided EvaluatorFn implementation.
Defines and registers a flow function.
BetaDefines and registers a custom model output formatter.
Here's an example of a custom JSON output formatter:
import { extractJson } from 'genkit/extract';
ai.defineFormat(
{ name: 'customJson' },
(schema) => {
let instructions: string | undefined;
if (schema) {
instructions = `Output should be in JSON format and conform to the following schema:
\`\`\`
${JSON.stringify(schema)}
\`\`\`
`;
}
return {
parseChunk: (chunk) => extractJson(chunk.accumulatedText),
parseMessage: (message) => extractJson(message.text),
instructions,
};
}
);
const { output } = await ai.generate({
prompt: 'Invent a menu item for a pirate themed restaurant.',
output: { format: 'customJson', schema: MenuItemSchema },
});
create a handlebars helper (https://handlebarsjs.com/guide/block-helpers.html) to be used in dotprompt templates.
Creates an indexer action for the provided IndexerFn implementation.
BetaDefines and registers an interrupt.
Interrupts are special tools that halt model processing and return control back to the caller. Interrupts make it simpler to implement "human-in-the-loop" and out-of-band processing patterns that require waiting on external actions to complete.
Defines and registers a schema from a JSON schema.
Defined schemas can be referenced by name in prompts in place of inline schemas.
Defines a new model and adds it to the registry.
Defines a new model and adds it to the registry.
Creates a handlebars partial (https://handlebarsjs.com/guide/partials.html) to be used in dotprompt templates.
Defines and registers a prompt based on a function.
This is an alternative to defining and importing a .prompt file, providing the most advanced control over how the final request to the model is made.
Prompt metadata including model, model params, input/output schemas, etc
OptionaltemplateOrFn: string | PromptFn<I, z.ZodTypeAny>Creates a reranker action for the provided RerankerFn implementation.
BetaDefines a resource. Resources can then be accessed from a generate call.
ai.defineResource({
uri: 'my://resource/{param}',
description: 'provides my resource',
}, async ({param}) => {
return [{ text: `resource ${param}` }]
});
await ai.generate({
prompt: [{ resource: 'my://resource/value' }]
Creates a retriever action for the provided RetrieverFn implementation.
Defines and registers a schema from a Zod schema.
Defined schemas can be referenced by name in prompts in place of inline schemas.
defineSimpleRetriever makes it easy to map existing data into documents that can be used for prompt augmentation.
A Genkit retriever.
Defines and registers a tool that can return multiple parts of content.
Tools can be passed to models by name or value during generate calls to be called automatically based on the prompt and situation.
Defines and registers a tool.
Tools can be passed to models by name or value during generate calls to be called automatically based on the prompt and situation.
Defines a dynamic tool. Dynamic tools are just like regular tools (Genkit.defineTool) but will not be registered in the Genkit registry and can be defined dynamically at runtime.
Optionalfn: ToolFn<I, O>Embeds the given content using the specified embedder.
A veneer for interacting with embedder models in bulk.
Evaluates the given dataset using the specified evaluator.
Make a generate call to the default model with a simple text prompt.
const ai = genkit({
plugins: [googleAI()],
model: googleAI.model('gemini-flash-latest'), // default model
})
const { text } = await ai.generate('hi');
Make a generate call to the default model with a multipart request.
const ai = genkit({
plugins: [googleAI()],
model: googleAI.model('gemini-flash-latest'), // default model
})
const { text } = await ai.generate([
{ media: {url: 'http://....'} },
{ text: 'describe this image' }
]);
Generate calls a generative model based on the provided prompt and configuration. If
messages is provided, the generation will include a conversation history in its
request. If tools are provided, the generate method will automatically resolve
tool calls returned from the model unless returnToolRequests is set to true.
See GenerateOptions for detailed information about available options.
const ai = genkit({
plugins: [googleAI()],
})
const { text } = await ai.generate({
system: 'talk like a pirate',
prompt: [
{ media: { url: 'http://....' } },
{ text: 'describe this image' }
],
messages: conversationHistory,
tools: [ userInfoLookup ],
model: googleAI.model('gemini-flash-latest'),
});
BetaStarts a generate operation for long running generation models, typically for video and complex audio generation.
See GenerateOptions for detailed information about available options.
const operation = await ai.generateOperation({
model: googleAI.model('veo-2.0-generate-001'),
prompt: 'A banana riding a bicycle.',
});
The status of the operation and final result can be obtained using Genkit.checkOperation.
Make a streaming generate call to the default model with a simple text prompt.
const ai = genkit({
plugins: [googleAI()],
model: googleAI.model('gemini-flash-latest'), // default model
})
const { response, stream } = ai.generateStream('hi');
for await (const chunk of stream) {
console.log(chunk.text);
}
console.log((await response).text);
Make a streaming generate call to the default model with a multipart request.
const ai = genkit({
plugins: [googleAI()],
model: googleAI.model('gemini-flash-latest'), // default model
})
const { response, stream } = ai.generateStream([
{ media: {url: 'http://....'} },
{ text: 'describe this image' }
]);
for await (const chunk of stream) {
console.log(chunk.text);
}
console.log((await response).text);
Streaming generate calls a generative model based on the provided prompt and configuration. If
messages is provided, the generation will include a conversation history in its
request. If tools are provided, the generate method will automatically resolve
tool calls returned from the model unless returnToolRequests is set to true.
See GenerateOptions for detailed information about available options.
const ai = genkit({
plugins: [googleAI()],
})
const { response, stream } = ai.generateStream({
system: 'talk like a pirate',
prompt: [
{ media: { url: 'http://....' } },
{ text: 'describe this image' }
],
messages: conversationHistory,
tools: [ userInfoLookup ],
model: googleAI.model('gemini-flash-latest'),
});
for await (const chunk of stream) {
console.log(chunk.text);
}
console.log((await response).text);
Indexes documents using the provided indexer.
BetaLoads a session from the store.
Looks up a prompt by name (and optionally variant). Can be used to lookup
.prompt files or prompts previously defined with Genkit.definePrompt
Optionaloptions: { variant?: string }Reranks documents from a RerankerArgument based on the provided query.
Retrieves documents from the retriever based on the provided query.
A flow step that executes the provided function. Each run step is recorded separately in the trace.
ai.defineFlow('hello', async() => {
await ai.run('step1', async () => {
// ... step 1
});
await ai.run('step2', async () => {
// ... step 2
});
return result;
})
A flow step that executes the provided function. Each run step is recorded separately in the trace.
ai.defineFlow('hello', async(name) => {
const greeting = await ai.run('step1', name, async (input) => {
return `Hello, ${input}!`;
});
const result = await ai.run('step2', greeting, async (input) => {
// ... step 2
});
return result;
Stops all servers.
Genkit BETA APIs.