Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .changeset/metal-ties-cry.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
'firebase': minor
'@firebase/ai': minor
---

Add support for Server Prompt Templates.
37 changes: 37 additions & 0 deletions common/api-review/ai.api.md
Original file line number Diff line number Diff line change
Expand Up @@ -100,6 +100,10 @@ export interface AudioTranscriptionConfig {
export abstract class Backend {
protected constructor(type: BackendType);
readonly backendType: BackendType;
// @internal (undocumented)
abstract _getModelPath(project: string, model: string): string;
// @internal (undocumented)
abstract _getTemplatePath(project: string, templateId: string): string;
}

// @public
Expand Down Expand Up @@ -567,9 +571,19 @@ export function getImagenModel(ai: AI, modelParams: ImagenModelParams, requestOp
// @beta
export function getLiveGenerativeModel(ai: AI, modelParams: LiveModelParams): LiveGenerativeModel;

// @beta
export function getTemplateGenerativeModel(ai: AI, requestOptions?: RequestOptions): TemplateGenerativeModel;

// @beta
export function getTemplateImagenModel(ai: AI, requestOptions?: RequestOptions): TemplateImagenModel;

// @public
export class GoogleAIBackend extends Backend {
constructor();
// @internal (undocumented)
_getModelPath(project: string, model: string): string;
// @internal (undocumented)
_getTemplatePath(project: string, templateId: string): string;
}

// Warning: (ae-internal-missing-underscore) The name "GoogleAICitationMetadata" should be prefixed with an underscore because the declaration is marked as @internal
Expand Down Expand Up @@ -1314,6 +1328,25 @@ export class StringSchema extends Schema {
toJSON(): SchemaRequest;
}

// @beta
export class TemplateGenerativeModel {
constructor(ai: AI, requestOptions?: RequestOptions);
// @internal (undocumented)
_apiSettings: ApiSettings;
generateContent(templateId: string, templateVariables: object): Promise<GenerateContentResult>;
generateContentStream(templateId: string, templateVariables: object): Promise<GenerateContentStreamResult>;
requestOptions?: RequestOptions;
}

// @beta
export class TemplateImagenModel {
constructor(ai: AI, requestOptions?: RequestOptions);
// @internal (undocumented)
_apiSettings: ApiSettings;
generateImages(templateId: string, templateVariables: object): Promise<ImagenGenerationResponse<ImagenInlineImage>>;
requestOptions?: RequestOptions;
}

// @public
export interface TextPart {
// (undocumented)
Expand Down Expand Up @@ -1412,6 +1445,10 @@ export interface UsageMetadata {
// @public
export class VertexAIBackend extends Backend {
constructor(location?: string);
// @internal (undocumented)
_getModelPath(project: string, model: string): string;
// @internal (undocumented)
_getTemplatePath(project: string, templateId: string): string;
readonly location: string;
}

Expand Down
4 changes: 4 additions & 0 deletions docs-devsite/_toc.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -198,6 +198,10 @@ toc:
path: /docs/reference/js/ai.startchatparams.md
- title: StringSchema
path: /docs/reference/js/ai.stringschema.md
- title: TemplateGenerativeModel
path: /docs/reference/js/ai.templategenerativemodel.md
- title: TemplateImagenModel
path: /docs/reference/js/ai.templateimagenmodel.md
- title: TextPart
path: /docs/reference/js/ai.textpart.md
- title: ThinkingConfig
Expand Down
52 changes: 52 additions & 0 deletions docs-devsite/ai.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,8 @@ The Firebase AI Web SDK.
| [getGenerativeModel(ai, modelParams, requestOptions)](./ai.md#getgenerativemodel_c63f46a) | Returns a [GenerativeModel](./ai.generativemodel.md#generativemodel_class) class with methods for inference and other functionality. |
| [getImagenModel(ai, modelParams, requestOptions)](./ai.md#getimagenmodel_e1f6645) | Returns an [ImagenModel](./ai.imagenmodel.md#imagenmodel_class) class with methods for using Imagen.<!-- -->Only Imagen 3 models (named <code>imagen-3.0-*</code>) are supported. |
| [getLiveGenerativeModel(ai, modelParams)](./ai.md#getlivegenerativemodel_f2099ac) | <b><i>(Public Preview)</i></b> Returns a [LiveGenerativeModel](./ai.livegenerativemodel.md#livegenerativemodel_class) class for real-time, bidirectional communication.<!-- -->The Live API is only supported in modern browser windows and Node &gt;<!-- -->= 22. |
| [getTemplateGenerativeModel(ai, requestOptions)](./ai.md#gettemplategenerativemodel_9476bbc) | <b><i>(Public Preview)</i></b> Returns a [TemplateGenerativeModel](./ai.templategenerativemodel.md#templategenerativemodel_class) class for executing server-side templates. |
| [getTemplateImagenModel(ai, requestOptions)](./ai.md#gettemplateimagenmodel_9476bbc) | <b><i>(Public Preview)</i></b> Returns a [TemplateImagenModel](./ai.templateimagenmodel.md#templateimagenmodel_class) class for executing server-side Imagen templates. |
| <b>function(liveSession, ...)</b> |
| [startAudioConversation(liveSession, options)](./ai.md#startaudioconversation_01c8e7f) | <b><i>(Public Preview)</i></b> Starts a real-time, bidirectional audio conversation with the model. This helper function manages the complexities of microphone access, audio recording, playback, and interruptions. |

Expand All @@ -47,6 +49,8 @@ The Firebase AI Web SDK.
| [ObjectSchema](./ai.objectschema.md#objectschema_class) | Schema class for "object" types. The <code>properties</code> param must be a map of <code>Schema</code> objects. |
| [Schema](./ai.schema.md#schema_class) | Parent class encompassing all Schema types, with static methods that allow building specific Schema types. This class can be converted with <code>JSON.stringify()</code> into a JSON string accepted by Vertex AI REST endpoints. (This string conversion is automatically done when calling SDK methods.) |
| [StringSchema](./ai.stringschema.md#stringschema_class) | Schema class for "string" types. Can be used with or without enum values. |
| [TemplateGenerativeModel](./ai.templategenerativemodel.md#templategenerativemodel_class) | <b><i>(Public Preview)</i></b> [GenerativeModel](./ai.generativemodel.md#generativemodel_class) APIs that execute on a server-side template.<!-- -->This class should only be instantiated with [getTemplateGenerativeModel()](./ai.md#gettemplategenerativemodel_9476bbc)<!-- -->. |
| [TemplateImagenModel](./ai.templateimagenmodel.md#templateimagenmodel_class) | <b><i>(Public Preview)</i></b> Class for Imagen model APIs that execute on a server-side template.<!-- -->This class should only be instantiated with [getTemplateImagenModel()](./ai.md#gettemplateimagenmodel_9476bbc)<!-- -->. |
| [VertexAIBackend](./ai.vertexaibackend.md#vertexaibackend_class) | Configuration class for the Vertex AI Gemini API.<!-- -->Use this with [AIOptions](./ai.aioptions.md#aioptions_interface) when initializing the AI service via [getAI()](./ai.md#getai_a94a413) to specify the Vertex AI Gemini API as the backend. |

## Interfaces
Expand Down Expand Up @@ -341,6 +345,54 @@ export declare function getLiveGenerativeModel(ai: AI, modelParams: LiveModelPar

If the `apiKey` or `projectId` fields are missing in your Firebase config.

### getTemplateGenerativeModel(ai, requestOptions) {:#gettemplategenerativemodel_9476bbc}

> This API is provided as a preview for developers and may change based on feedback that we receive. Do not use this API in a production environment.
>

Returns a [TemplateGenerativeModel](./ai.templategenerativemodel.md#templategenerativemodel_class) class for executing server-side templates.

<b>Signature:</b>

```typescript
export declare function getTemplateGenerativeModel(ai: AI, requestOptions?: RequestOptions): TemplateGenerativeModel;
```

#### Parameters

| Parameter | Type | Description |
| --- | --- | --- |
| ai | [AI](./ai.ai.md#ai_interface) | An [AI](./ai.ai.md#ai_interface) instance. |
| requestOptions | [RequestOptions](./ai.requestoptions.md#requestoptions_interface) | Additional options to use when making requests. |

<b>Returns:</b>

[TemplateGenerativeModel](./ai.templategenerativemodel.md#templategenerativemodel_class)

### getTemplateImagenModel(ai, requestOptions) {:#gettemplateimagenmodel_9476bbc}

> This API is provided as a preview for developers and may change based on feedback that we receive. Do not use this API in a production environment.
>

Returns a [TemplateImagenModel](./ai.templateimagenmodel.md#templateimagenmodel_class) class for executing server-side Imagen templates.

<b>Signature:</b>

```typescript
export declare function getTemplateImagenModel(ai: AI, requestOptions?: RequestOptions): TemplateImagenModel;
```

#### Parameters

| Parameter | Type | Description |
| --- | --- | --- |
| ai | [AI](./ai.ai.md#ai_interface) | An [AI](./ai.ai.md#ai_interface) instance. |
| requestOptions | [RequestOptions](./ai.requestoptions.md#requestoptions_interface) | Additional options to use when making requests. |

<b>Returns:</b>

[TemplateImagenModel](./ai.templateimagenmodel.md#templateimagenmodel_class)

## function(liveSession, ...)

### startAudioConversation(liveSession, options) {:#startaudioconversation_01c8e7f}
Expand Down
125 changes: 125 additions & 0 deletions docs-devsite/ai.templategenerativemodel.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,125 @@
Project: /docs/reference/js/_project.yaml
Book: /docs/reference/_book.yaml
page_type: reference

{% comment %}
DO NOT EDIT THIS FILE!
This is generated by the JS SDK team, and any local changes will be
overwritten. Changes should be made in the source code at
https://github.com/firebase/firebase-js-sdk
{% endcomment %}

# TemplateGenerativeModel class
> This API is provided as a preview for developers and may change based on feedback that we receive. Do not use this API in a production environment.
>

[GenerativeModel](./ai.generativemodel.md#generativemodel_class) APIs that execute on a server-side template.

This class should only be instantiated with [getTemplateGenerativeModel()](./ai.md#gettemplategenerativemodel_9476bbc)<!-- -->.

<b>Signature:</b>

```typescript
export declare class TemplateGenerativeModel
```

## Constructors

| Constructor | Modifiers | Description |
| --- | --- | --- |
| [(constructor)(ai, requestOptions)](./ai.templategenerativemodel.md#templategenerativemodelconstructor) | | <b><i>(Public Preview)</i></b> Constructs a new instance of the <code>TemplateGenerativeModel</code> class |

## Properties

| Property | Modifiers | Type | Description |
| --- | --- | --- | --- |
| [requestOptions](./ai.templategenerativemodel.md#templategenerativemodelrequestoptions) | | [RequestOptions](./ai.requestoptions.md#requestoptions_interface) | <b><i>(Public Preview)</i></b> Additional options to use when making requests. |

## Methods

| Method | Modifiers | Description |
| --- | --- | --- |
| [generateContent(templateId, templateVariables)](./ai.templategenerativemodel.md#templategenerativemodelgeneratecontent) | | <b><i>(Public Preview)</i></b> Makes a single non-streaming call to the model and returns an object containing a single [GenerateContentResponse](./ai.generatecontentresponse.md#generatecontentresponse_interface)<!-- -->. |
| [generateContentStream(templateId, templateVariables)](./ai.templategenerativemodel.md#templategenerativemodelgeneratecontentstream) | | <b><i>(Public Preview)</i></b> Makes a single streaming call to the model and returns an object containing an iterable stream that iterates over all chunks in the streaming response as well as a promise that returns the final aggregated response. |

## TemplateGenerativeModel.(constructor)

> This API is provided as a preview for developers and may change based on feedback that we receive. Do not use this API in a production environment.
>

Constructs a new instance of the `TemplateGenerativeModel` class

<b>Signature:</b>

```typescript
constructor(ai: AI, requestOptions?: RequestOptions);
```

#### Parameters

| Parameter | Type | Description |
| --- | --- | --- |
| ai | [AI](./ai.ai.md#ai_interface) | |
| requestOptions | [RequestOptions](./ai.requestoptions.md#requestoptions_interface) | |

## TemplateGenerativeModel.requestOptions

> This API is provided as a preview for developers and may change based on feedback that we receive. Do not use this API in a production environment.
>

Additional options to use when making requests.

<b>Signature:</b>

```typescript
requestOptions?: RequestOptions;
```

## TemplateGenerativeModel.generateContent()

> This API is provided as a preview for developers and may change based on feedback that we receive. Do not use this API in a production environment.
>

Makes a single non-streaming call to the model and returns an object containing a single [GenerateContentResponse](./ai.generatecontentresponse.md#generatecontentresponse_interface)<!-- -->.

<b>Signature:</b>

```typescript
generateContent(templateId: string, templateVariables: object): Promise<GenerateContentResult>;
```

#### Parameters

| Parameter | Type | Description |
| --- | --- | --- |
| templateId | string | The ID of the server-side template to execute. |
| templateVariables | object | A key-value map of variables to populate the template with. |

<b>Returns:</b>

Promise&lt;[GenerateContentResult](./ai.generatecontentresult.md#generatecontentresult_interface)<!-- -->&gt;

## TemplateGenerativeModel.generateContentStream()

> This API is provided as a preview for developers and may change based on feedback that we receive. Do not use this API in a production environment.
>

Makes a single streaming call to the model and returns an object containing an iterable stream that iterates over all chunks in the streaming response as well as a promise that returns the final aggregated response.

<b>Signature:</b>

```typescript
generateContentStream(templateId: string, templateVariables: object): Promise<GenerateContentStreamResult>;
```

#### Parameters

| Parameter | Type | Description |
| --- | --- | --- |
| templateId | string | The ID of the server-side template to execute. |
| templateVariables | object | A key-value map of variables to populate the template with. |

<b>Returns:</b>

Promise&lt;[GenerateContentStreamResult](./ai.generatecontentstreamresult.md#generatecontentstreamresult_interface)<!-- -->&gt;

100 changes: 100 additions & 0 deletions docs-devsite/ai.templateimagenmodel.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
Project: /docs/reference/js/_project.yaml
Book: /docs/reference/_book.yaml
page_type: reference

{% comment %}
DO NOT EDIT THIS FILE!
This is generated by the JS SDK team, and any local changes will be
overwritten. Changes should be made in the source code at
https://github.com/firebase/firebase-js-sdk
{% endcomment %}

# TemplateImagenModel class
> This API is provided as a preview for developers and may change based on feedback that we receive. Do not use this API in a production environment.
>

Class for Imagen model APIs that execute on a server-side template.

This class should only be instantiated with [getTemplateImagenModel()](./ai.md#gettemplateimagenmodel_9476bbc)<!-- -->.

<b>Signature:</b>

```typescript
export declare class TemplateImagenModel
```

## Constructors

| Constructor | Modifiers | Description |
| --- | --- | --- |
| [(constructor)(ai, requestOptions)](./ai.templateimagenmodel.md#templateimagenmodelconstructor) | | <b><i>(Public Preview)</i></b> Constructs a new instance of the <code>TemplateImagenModel</code> class |

## Properties

| Property | Modifiers | Type | Description |
| --- | --- | --- | --- |
| [requestOptions](./ai.templateimagenmodel.md#templateimagenmodelrequestoptions) | | [RequestOptions](./ai.requestoptions.md#requestoptions_interface) | <b><i>(Public Preview)</i></b> Additional options to use when making requests. |

## Methods

| Method | Modifiers | Description |
| --- | --- | --- |
| [generateImages(templateId, templateVariables)](./ai.templateimagenmodel.md#templateimagenmodelgenerateimages) | | <b><i>(Public Preview)</i></b> Makes a single call to the model and returns an object containing a single [ImagenGenerationResponse](./ai.imagengenerationresponse.md#imagengenerationresponse_interface)<!-- -->. |

## TemplateImagenModel.(constructor)

> This API is provided as a preview for developers and may change based on feedback that we receive. Do not use this API in a production environment.
>

Constructs a new instance of the `TemplateImagenModel` class

<b>Signature:</b>

```typescript
constructor(ai: AI, requestOptions?: RequestOptions);
```

#### Parameters

| Parameter | Type | Description |
| --- | --- | --- |
| ai | [AI](./ai.ai.md#ai_interface) | |
| requestOptions | [RequestOptions](./ai.requestoptions.md#requestoptions_interface) | |

## TemplateImagenModel.requestOptions

> This API is provided as a preview for developers and may change based on feedback that we receive. Do not use this API in a production environment.
>

Additional options to use when making requests.

<b>Signature:</b>

```typescript
requestOptions?: RequestOptions;
```

## TemplateImagenModel.generateImages()

> This API is provided as a preview for developers and may change based on feedback that we receive. Do not use this API in a production environment.
>

Makes a single call to the model and returns an object containing a single [ImagenGenerationResponse](./ai.imagengenerationresponse.md#imagengenerationresponse_interface)<!-- -->.

<b>Signature:</b>

```typescript
generateImages(templateId: string, templateVariables: object): Promise<ImagenGenerationResponse<ImagenInlineImage>>;
```

#### Parameters

| Parameter | Type | Description |
| --- | --- | --- |
| templateId | string | The ID of the server-side template to execute. |
| templateVariables | object | A key-value map of variables to populate the template with. |

<b>Returns:</b>

Promise&lt;[ImagenGenerationResponse](./ai.imagengenerationresponse.md#imagengenerationresponse_interface)<!-- -->&lt;[ImagenInlineImage](./ai.imageninlineimage.md#imageninlineimage_interface)<!-- -->&gt;&gt;

2 changes: 1 addition & 1 deletion packages/ai/integration/constants.ts
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ function formatConfigAsString(config: { ai: AI; model: string }): string {

const backends: readonly Backend[] = [
new GoogleAIBackend(),
new VertexAIBackend()
new VertexAIBackend('global')
];

const backendNames: Map<BackendType, string> = new Map([
Expand Down
Loading
Loading