> ## Documentation Index
> Fetch the complete documentation index at: https://api-tools.memories.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Start Stream Video Understanding

> Start a real-time AI analysis session on a live RTMP video stream with a custom prompt.

<Info>
  **Product**: Visual Intelligence — Live Video Understanding
  **Use case**: Apply a custom AI prompt continuously to a live RTMP stream (e.g. captioning, activity recognition, custom monitoring), results to your callback every 5s
  **Mode**: Server-Pull (you give us an RTMP URL → we pull → results to your callback)
  **Host**: `https://stream.memories.ai`
  **Auth**: `Authorization: sk-mavi-...` (no `Bearer` prefix)
</Info>

Start a real-time analysis session on a live RTMP video stream. The server pulls the stream, runs an AI model against each frame window using your custom `user_prompt`, and pushes results to your callback URL every few seconds.

<Warning>
  Each API key supports a maximum of **4 concurrent streaming tasks**. Billing occurs every **5 seconds** per active stream. If a credit deduction fails, the stream is automatically terminated.
</Warning>

## Request Example

```python theme={null}
import requests

url = "https://stream.memories.ai/v1/understand/streamConnect"
headers = {
    "Authorization": "sk-mavi-...",
    "Content-Type": "application/json"
}

payload = {
    "url": "rtmp://your.server/live/stream",
    "user_prompt": "What stage is the food preparation at? Is it ready to eat?",
    "system_prompt": "Track the current cooking progress.",
    "callback": "https://your.app/callback",
    "thinking": False
}

response = requests.post(url, headers=headers, json=payload)
print(response.json())  # {"status": 0, "task_id": "..."}
```

## Parameters

<ParamField body="url" type="string" required>
  RTMP stream URL for live video input (e.g. `rtmp://your.server:1935/live/stream`).
</ParamField>

<ParamField body="user_prompt" type="string" required>
  The analysis instruction applied to each frame window — e.g. `"What is happening right now?"`.
</ParamField>

<ParamField body="system_prompt" type="string" required>
  Role or context for the AI — e.g. `"You are a real-time video monitoring system."`.
</ParamField>

<ParamField body="callback" type="string" required>
  Public URL to receive continuous analysis results. Called repeatedly throughout the stream lifetime.
</ParamField>

<ParamField body="thinking" type="boolean" default="false">
  Enable reasoning mode for deeper analysis.
</ParamField>

## Start Stream Response

```json theme={null}
{
  "status": 0,
  "task_id": "5347085a-1747-4731-bdde-3856d09502c4",
  "text": ""
}
```

Use the returned `task_id` to stop the stream via [Stop Stream Video Understanding](/visual-intelligence/stream/video-stream-understanding-stop).

## Callback Payload

Results are pushed to your `callback` URL continuously while the stream is active:

```json theme={null}
{
  "status": 0,
  "task_id": "5347085a-1747-4731-bdde-3856d09502c4",
  "data": {
    "text": "The chef is frying vegetables. The dish appears to be halfway done.",
    "timestamp": "2025-10-22T12:00:45Z"
  }
}
```

<ResponseField name="status" type="integer">
  `0` = success, `-1` = failure.
</ResponseField>

<ResponseField name="task_id" type="string">
  Identifies which stream this callback belongs to. Use this to stop the stream.
</ResponseField>

<ResponseField name="data.text" type="string">
  AI-generated analysis for the current frame window.
</ResponseField>

<ResponseField name="data.timestamp" type="string">
  UTC timestamp of the analysis frame.
</ResponseField>

## Error Codes

| Code   | Meaning                                          |
| ------ | ------------------------------------------------ |
| `-1`   | Invalid request or parameters                    |
| `1001` | Maximum concurrent streams reached (4 per key)   |
| `2001` | Credit deduction failed — stream auto-terminated |
| `5000` | Internal server error                            |

## Billing

* Charged every **5 seconds** per active stream
* If credit deduction fails, the stream stops automatically — recharge and call this endpoint again to resume
* This API does not support video file uploads or historical playback — live RTMP only


## OpenAPI

````yaml visual-intelligence/stream/understanding-openapi.json POST /v1/understand/streamConnect
openapi: 3.0.0
info:
  title: Stream Video Understanding API
  version: 1.0.0
  description: Real-time AI analysis on live RTMP video streams using custom prompts.
servers:
  - url: https://stream.memories.ai
    description: Stream Understanding service
security: []
paths:
  /v1/understand/streamConnect:
    post:
      tags:
        - Stream Understanding
      summary: Start Stream Video Understanding
      description: Start a real-time AI analysis session on a live RTMP video stream.
      operationId: startStreamUnderstanding
      requestBody:
        required: true
        content:
          application/json:
            schema:
              type: object
              required:
                - url
                - user_prompt
                - system_prompt
                - callback
              properties:
                url:
                  type: string
                  description: RTMP stream URL for live video input
                  example: rtmp://your.server:1935/live/stream
                user_prompt:
                  type: string
                  description: Analysis instruction applied to each frame window
                  example: What is happening right now?
                system_prompt:
                  type: string
                  description: Role or context for the AI
                  example: You are a real-time video monitoring system.
                callback:
                  type: string
                  description: Public URL to receive continuous analysis results
                  example: https://your.app/callback
                thinking:
                  type: boolean
                  description: Enable reasoning mode for deeper analysis
                  default: false
      responses:
        '200':
          description: Stream started successfully
          content:
            application/json:
              schema:
                type: object
                properties:
                  status:
                    type: integer
                    example: 0
                  task_id:
                    type: string
                    example: 5347085a-1747-4731-bdde-3856d09502c4
                  text:
                    type: string
                    example: ''
      security:
        - ApiKeyAuth: []
components:
  securitySchemes:
    ApiKeyAuth:
      type: apiKey
      in: header
      name: Authorization

````