Optionaloptions: GenkitOptionsReadonlyconfiguredEnvironments that have been configured (at minimum dev).
ReadonlyflowsList of flows that have been registered in this instance.
ReadonlyoptionsDeveloper-configured options.
ReadonlyregistryRegistry instance that is exclusively modified by this Genkit instance.
Returns current action (or flow) invocation context. Can be used to access things like auth
data set by HTTP server frameworks. If invoked outside of an action (e.g. flow or tool) will
return undefined.
Creates embedder model for the provided EmbedderFn model implementation.
Creates evaluator action for the provided EvaluatorFn implementation.
Defines and registers a non-streaming flow.
Defines and registers a custom model output formatter.
Here's an example of a custom JSON output formatter:
import { extractJson } from 'genkit/extract';
ai.defineFormat(
{ name: 'customJson' },
(schema) => {
let instructions: string | undefined;
if (schema) {
instructions = `Output should be in JSON format and conform to the following schema:
\`\`\`
${JSON.stringify(schema)}
\`\`\`
`;
}
return {
parseChunk: (chunk) => extractJson(chunk.accumulatedText),
parseMessage: (message) => extractJson(message.text),
instructions,
};
}
);
const { output } = await ai.generate({
prompt: 'Invent a menu item for a pirate themed restaurant.',
output: { format: 'customJson', schema: MenuItemSchema },
});
create a handlebards helper (https://handlebarsjs.com/guide/block-helpers.html) to be used in dotpormpt templates.
Creates an indexer action for the provided IndexerFn implementation.
Defines and registers an interrupt.
Interrupts are special tools that halt model processing and return control back to the caller. Interrupts make it simpler to implement "human-in-the-loop" and out-of-band processing patterns that require waiting on external actions to complete.
Defines a new model and adds it to the registry.
Creates a handlebars partial (https://handlebarsjs.com/guide/partials.html) to be used in dotpormpt templates.
Defines and registers a prompt based on a function.
This is an alternative to defining and importing a .prompt file, providing the most advanced control over how the final request to the model is made.
Prompt metadata including model, model params, input/output schemas, etc
Creates a reranker action for the provided RerankerFn implementation.
Creates a retriever action for the provided RetrieverFn implementation.
defineSimpleRetriever makes it easy to map existing data into documents that can be used for prompt augmentation.
A Genkit retriever.
Defines and registers a tool.
Tools can be passed to models by name or value during generate calls to be called automatically based on the prompt and situation.
Embeds the given content using the specified embedder.
A veneer for interacting with embedder models in bulk.
Evaluates the given dataset using the specified evaluator.
Make a generate call to the default model with a simple text prompt.
const ai = genkit({
plugins: [googleAI()],
model: gemini15Flash, // default model
})
const { text } = await ai.generate('hi');
Make a generate call to the default model with a multipart request.
const ai = genkit({
plugins: [googleAI()],
model: gemini15Flash, // default model
})
const { text } = await ai.generate([
{ media: {url: 'http://....'} },
{ text: 'describe this image' }
]);
Generate calls a generative model based on the provided prompt and configuration. If
messages is provided, the generation will include a conversation history in its
request. If tools are provided, the generate method will automatically resolve
tool calls returned from the model unless returnToolRequests is set to true.
See GenerateOptions for detailed information about available options.
const ai = genkit({
plugins: [googleAI()],
})
const { text } = await ai.generate({
system: 'talk like a pirate',
prompt: [
{ media: { url: 'http://....' } },
{ text: 'describe this image' }
],
messages: conversationHistory,
tools: [ userInfoLookup ],
model: gemini15Flash,
});
Make a streaming generate call to the default model with a simple text prompt.
const ai = genkit({
plugins: [googleAI()],
model: gemini15Flash, // default model
})
const { response, stream } = ai.generateStream('hi');
for await (const chunk of stream) {
console.log(chunk.text);
}
console.log((await response).text);
Make a streaming generate call to the default model with a multipart request.
const ai = genkit({
plugins: [googleAI()],
model: gemini15Flash, // default model
})
const { response, stream } = ai.generateStream([
{ media: {url: 'http://....'} },
{ text: 'describe this image' }
]);
for await (const chunk of stream) {
console.log(chunk.text);
}
console.log((await response).text);
Streaming generate calls a generative model based on the provided prompt and configuration. If
messages is provided, the generation will include a conversation history in its
request. If tools are provided, the generate method will automatically resolve
tool calls returned from the model unless returnToolRequests is set to true.
See GenerateOptions for detailed information about available options.
const ai = genkit({
plugins: [googleAI()],
})
const { response, stream } = ai.generateStream({
system: 'talk like a pirate',
prompt: [
{ media: { url: 'http://....' } },
{ text: 'describe this image' }
],
messages: conversationHistory,
tools: [ userInfoLookup ],
model: gemini15Flash,
});
for await (const chunk of stream) {
console.log(chunk.text);
}
console.log((await response).text);
Indexes documents using the provided indexer.
Looks up a prompt by name and optional variant.
Optionaloptions: { variant?: string }Reranks documents from a RerankerArgument based on the provided query.
Retrieves documents from the retriever based on the provided query.
A flow step that executes the provided function. Each run step is recorded separately in the trace.
ai.defineFlow('hello', async() => {
await ai.run('step1', async () => {
// ... step 1
});
await ai.run('step2', async () => {
// ... step 2
});
return result;
})
A flow step that executes the provided function. Each run step is recorded separately in the trace.
ai.defineFlow('hello', async() => {
await ai.run('step1', async () => {
// ... step 1
});
await ai.run('step2', async () => {
// ... step 2
});
return result;
Genkitencapsulates a single Genkit instance including the Registry, ReflectionServer, FlowServer, and configuration.Do not instantiate this class directly. Use genkit.
Registry keeps track of actions, flows, tools, and many other components. Reflection server exposes an API to inspect the registry and trigger executions of actions in the registry. Flow server exposes flows as HTTP endpoints for production use.
There may be multiple Genkit instances in a single codebase.