> ## Documentation Index
> Fetch the complete documentation index at: https://api-tools.memories.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# Gemini Video
> Generate chat completions using Gemini VLM model with text, video, and image inputs.
**Product**: Visual Intelligence — Video Model APIs (atomic)
**Use case**: Call a Video Language Model (Gemini / Nova / Qwen) directly with your own prompt. Full control over model selection, prompt, and parameters. For ready-made tasks (no prompt needed), see Video Task APIs.
**Host**: `https://mavi-backend.memories.ai/serve/api/v2`
**Auth**: `Authorization: sk-mavi-...` (no `Bearer` prefix)
This endpoint allows you to generate chat completions with text, video, and image inputs using Gemini VLM model.
**Endpoint:** `POST https://mavi-backend.memories.ai/serve/api/v2/vu/chat/completions`
Video Understanding (VLM) endpoints use the `/vu` path prefix. Image Understanding (ILM) endpoints use `/iu` instead.
## Supported Models
All models require the `gemini:` prefix when used in the `model` parameter (e.g., `gemini:gemini-2.5-flash`).
### Premium Models
| Model | Input Price | Output Price |
| ------------------------ | ----------------------------------- | -------------------------------- |
| **gemini-3-pro-preview** | \$2/1M (≤200K), \$4/1M (>200K) | \$12/1M (≤200K), \$18/1M (>200K) |
| **gemini-2.5-pro** | \$1.25/1M (≤200K), \$2.5/1M (>200K) | \$10/1M (≤200K), \$15/1M (>200K) |
### Flash Models (High Performance)
| Model | Input Price | Output Price |
| ------------------------------------ | ---------------- | --------------- |
| **gemini-3-flash-preview** | \$0.5/1M tokens | \$3/1M tokens |
| **gemini-2.5-flash** | \$0.30/1M tokens | \$2.5/1M tokens |
| **gemini-2.5-flash-preview-09-2025** | \$0.30/1M tokens | \$2.5/1M tokens |
| **gemini-2.0-flash** | \$0.1/1M tokens | \$0.4/1M tokens |
### Lite Models (Cost-Effective)
| Model | Input Price | Output Price |
| ----------------------------------------- | ----------------- | --------------- |
| **gemini-2.5-flash-lite** | \$0.1/1M tokens | \$0.4/1M tokens |
| **gemini-2.5-flash-lite-preview-09-2025** | \$0.1/1M tokens | \$0.4/1M tokens |
| **gemini-2.0-flash-lite** | \$0.075/1M tokens | \$0.3/1M tokens |
When using these models, remember to include the `gemini:` prefix in your API calls:
* ✅ Correct: `"model": "gemini:gemini-2.5-flash"`
* ❌ Incorrect: `"model": "gemini-2.5-flash"`
### Request Body
| Parameter | Type | Required | Default | Description |
| ------------------ | ----------------------- | -------- | ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| model | string | Yes | - | The model to use (e.g., `gemini:gemini-2.5-flash`) |
| messages | array | Yes | - | Array of message objects. Each message contains:
- `role`: Role type, values: `system`, `user`, `assistant`
- `content`: Message content. A plain string is accepted for **text-only** turns, but **must be an array** as soon as you include a video or image — passing a bare string with an `input_file` intent silently drops the media and the upstream model rejects the call with `status: "errored"`. Array items can contain:
- `type`: Content type, `text` or `input_file`
- `text`: Text content (when type is text)
- `file_uri`: File URL or base64 encoded file (when type is input\_file). Note: video does not support base64
- `mime_type`: MIME type of the file (e.g., image/jpeg, video/mp4) |
| temperature | number | No | 0.7 | Controls randomness: 0.0-2.0, higher = more random |
| max\_tokens | integer | No | 1000 | Maximum number of tokens to generate |
| top\_p | number | No | 1.0 | Nucleus sampling: 0.0-1.0, consider tokens with top\_p probability mass |
| frequency\_penalty | number | No | 0.0 | Reduces repetition of frequent tokens: -2.0 to 2.0 |
| presence\_penalty | number | No | 0.0 | Increases likelihood of new topics: -2.0 to 2.0 |
| n | integer | No | 1 | Number of completions to generate |
| stream | boolean | No | false | Whether to stream the response |
| stop | string \| array \| null | No | null | Stop sequences. Can be a string, array of strings, or null |
| extra\_body | object | No | - | Additional body parameters. Contains:
- `metadata`: Metadata object
- `thinking_config`: Thinking configuration
- `thinking_budget`: Integer value for thinking budget
- `response_mime_type`: Response MIME type (`application/json` or `json_schema`)
- `responseSchema`: JSON schema object for structured output |
### Code Example
```python Python theme={null}
from openai import OpenAI
client = OpenAI(
api_key="sk-mavi-...",
base_url="https://mavi-backend.memories.ai/serve/api/v2/vu"
)
def call_my_vlm():
resp = client.chat.completions.create(
model="gemini:gemini-2.5-flash", # e.g. gemini:gemini-3-flash-preview or gemini:gemini-2.5-flash
messages=[
{"role": "system", "content": "You are a multimodal assistant. Keep your answers concise."},
{
"role": "user",
"content": [
{
"type": "text",
"text": "Please summarize the content of this video and image"
},
{
"type": "input_file",
"file_uri": "https://storage.googleapis.com/memories-test-data/gun5.png", # base64 or url, video does not support base64
"mime_type": "image/jpeg"
},
{
"type": "input_file",
"file_uri": "https://storage.googleapis.com/memories-test-data/test_1min.mp4", # base64 or url, video does not support base64
"mime_type": "video/mp4"
}
]
}
],
temperature=0.7, # Controls randomness: 0.0-2.0, higher = more random
max_tokens=1000, # Maximum number of tokens to generate
top_p=1.0, # Nucleus sampling: 0.0-1.0, consider tokens with top_p probability mass
frequency_penalty=0.0, # -2.0 to 2.0, reduces repetition of frequent tokens
presence_penalty=0.0, # -2.0 to 2.0, increases likelihood of new topics
n=1, # Number of completions to generate
stream=False, # Whether to stream the response
stop=None, # Stop sequences (list of strings)
extra_body={
"metadata": {
"thinking_config": {
"thinking_budget": 1024
},
"response_mime_type": "application/json", # application/json, json_schema
"responseSchema": {
"type": "OBJECT",
"properties": {
"video_summary": {
"type": "STRING",
"description": "Summary of the video content from 1 second to 8 seconds."
},
"image_summary": {
"type": "STRING",
"description": "Summary of the image content."
}
},
"required": [
"video_summary",
"image_summary"
]
}
}
}
)
return resp
# Usage example
result = call_my_vlm()
print(result)
```
### Response
Returns the chat completion response with structured output.
```json theme={null}
{
"id": "resp_5810813e-99b9-427a-8736-23cf34573627",
"object": "completion",
"model": "gemini:gemini-2.5-flash",
"created_at": 1767093284,
"status": "completed",
"choices": [
{
"text": "This video demonstrates the effects of vacuum on three different states of oranges.\n\n1. **Setup**: Three orange-related items are placed in a vacuum chamber: a whole orange",
"index": 0
}
],
"usage": {
"input_tokens": 16983,
"output_tokens": 37,
"total_tokens": 17020
},
"meta": {
"provider": "gemini",
"provider_model": "gemini-2.5-flash"
}
}
```
### Response Parameters
| Parameter | Type | Description |
| -------------------- | ------- | ------------------------------------------------- |
| id | string | Unique identifier for the completion |
| object | string | Object type, always "completion" |
| model | string | The model used for the completion |
| created\_at | integer | Unix timestamp of when the completion was created |
| status | string | Status of the completion (e.g., "completed") |
| choices | array | Array of completion choices |
| choices\[].text | string | Text content of the completion |
| choices\[].index | integer | Index of the choice in the choices array |
| usage | object | Token usage information |
| usage.input\_tokens | integer | Number of input tokens used |
| usage.output\_tokens | integer | Number of output tokens generated |
| usage.total\_tokens | integer | Total number of tokens used |
| meta | object | Metadata about the completion |
| meta.provider | string | Provider name (e.g., "gemini") |
| meta.provider\_model | string | Provider-specific model name |
### Error Responses
**Upstream errors are returned with `HTTP 200`** — not a 4xx/5xx. Check `status` and look for a top-level `error` object before parsing `choices`.
```json theme={null}
{
"id": "resp_...",
"object": "completion",
"model": "gemini:gemini-2.0-flash-lite",
"status": "errored",
"choices": [],
"usage": {"input_tokens": 0, "output_tokens": 0, "total_tokens": 0},
"error": {
"code": "INVALID_ARGUMENT",
"message": "Unable to submit request because Model input cannot be empty. ...",
"http_status": 400
},
"meta": {
"provider": "gemini",
"provider_model": "gemini-2.0-flash-lite",
"provider_http_status": 400,
"provider_error": { "error": { "code": 400, "message": "...", "status": "INVALID_ARGUMENT" } }
}
}
```
| Field | Type | Description |
| --------------------------- | ------- | ------------------------------------------------------------------------------------------- |
| status | string | `"completed"` on success, `"errored"` when the upstream model rejected the request. |
| error | object | Present only when `status == "errored"`. Contains the upstream error normalized. |
| error.code | string | Provider-defined error code (e.g. `INVALID_ARGUMENT`). |
| error.message | string | Human-readable failure reason from the provider. |
| error.http\_status | integer | The HTTP status the upstream provider would have returned (the mavi response is still 200). |
| meta.provider\_http\_status | integer | Mirror of `error.http_status`. |
| meta.provider\_error | object | Raw provider error envelope, in case you need its native shape. |
Callers should always verify `status == "completed"` (or that `choices` is non-empty) before reading `choices[0].text`.
### Gemini structured-output quirks
Two real behaviours to handle when you use `extra_body.metadata.response_mime_type = "application/json"`:
1. Gemini frequently wraps its JSON output in ` ```json ... ``` ` markdown code fences. Strip them before `json.loads`.
2. On long videos with a strict schema, Gemini occasionally **loops** — emitting the same set of entries repeatedly until it hits `max_tokens` and the JSON is left unclosed. Tolerate this by closing the array at the last complete `}` and deduplicating client-side.
See [`examples/agent/02_restaurant_service_quality.py`](https://github.com/Memories-ai-labs/api-docs/blob/main/examples/agent/02_restaurant_service_quality.py) (`extract_json` and `_dedup_events`) for a worked example.
## OpenAPI
````yaml visual-intelligence/gemini/openapi.json POST /vu/chat/completions
openapi: 3.1.0
info:
title: Gemini API Reference
description: REST APIs for Gemini ILM and VLM models
version: v1.0.0
servers:
- url: https://mavi-backend.memories.ai/serve/api/v2
security:
- ApiKeyAuth: []
paths:
/vu/chat/completions:
post:
summary: Chat Completions Gemini
description: >-
Generate chat completions using Gemini ILM or VLM models with image and
video inputs.
operationId: chat_completions_gemini
requestBody:
required: true
content:
application/json:
schema:
type: object
properties:
model:
type: string
description: The model to use (e.g., gemini:gemini-2.5-flash)
example: gemini:gemini-2.5-flash
messages:
type: array
description: Array of message objects
items:
type: object
properties:
role:
type: string
enum:
- system
- user
- assistant
content:
oneOf:
- type: string
- type: array
items:
type: object
properties:
type:
type: string
enum:
- text
- input_file
text:
type: string
file_uri:
type: string
description: URL or base64 encoded file
mime_type:
type: string
description: >-
MIME type of the file (e.g., image/jpeg,
video/mp4)
required:
- role
- content
temperature:
type: number
description: 'Controls randomness: 0.0-2.0, higher = more random'
minimum: 0
maximum: 2
default: 0.7
max_tokens:
type: integer
description: Maximum number of tokens to generate
default: 1000
top_p:
type: number
description: 'Nucleus sampling: 0.0-1.0'
minimum: 0
maximum: 1
default: 1
frequency_penalty:
type: number
description: 'Reduces repetition of frequent tokens: -2.0 to 2.0'
minimum: -2
maximum: 2
default: 0
presence_penalty:
type: number
description: 'Increases likelihood of new topics: -2.0 to 2.0'
minimum: -2
maximum: 2
default: 0
'n':
type: integer
description: Number of completions to generate
default: 1
stream:
type: boolean
description: Whether to stream the response
default: false
stop:
oneOf:
- type: string
- type: array
items:
type: string
- type: 'null'
description: Stop sequences
extra_body:
type: object
properties:
metadata:
type: object
properties:
thinking_config:
type: object
properties:
thinking_budget:
type: integer
response_mime_type:
type: string
enum:
- application/json
- json_schema
responseSchema:
type: object
properties:
type:
type: string
enum:
- OBJECT
properties:
type: object
required:
type: array
items:
type: string
required:
- model
- messages
responses:
'200':
description: Chat completion response
content:
application/json:
schema:
type: object
properties:
id:
type: string
example: resp_5810813e-99b9-427a-8736-23cf34573627
description: Unique identifier for the completion
object:
type: string
example: completion
description: Object type, always 'completion'
model:
type: string
example: gemini:gemini-2.5-flash
description: The model used for the completion
created_at:
type: integer
example: 1767093284
description: Unix timestamp of when the completion was created
status:
type: string
example: completed
description: Status of the completion
choices:
type: array
items:
type: object
properties:
text:
type: string
description: Text content of the completion
example: >-
This video demonstrates the effects of vacuum on
three different states of oranges.
index:
type: integer
description: Index of the choice in the choices array
example: 0
required:
- text
- index
usage:
type: object
properties:
input_tokens:
type: integer
description: Number of input tokens used
example: 16983
output_tokens:
type: integer
description: Number of output tokens generated
example: 37
total_tokens:
type: integer
description: Total number of tokens used
example: 17020
required:
- input_tokens
- output_tokens
- total_tokens
meta:
type: object
properties:
provider:
type: string
description: Provider name
example: gemini
provider_model:
type: string
description: Provider-specific model name
example: gemini-2.5-flash
required:
- provider
- provider_model
required:
- id
- object
- model
- created_at
- status
- choices
- usage
- meta
components:
securitySchemes:
ApiKeyAuth:
type: apiKey
in: header
name: Authorization
````