> ## Documentation Index
> Fetch the complete documentation index at: https://docs.shipfastai.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# AI chat and completions API endpoints

> POST /api/ai/chat and POST /api/ai/completions — multi-turn chat, streaming SSE responses, and one-shot completions via OpenAI, Anthropic, or Gemini.

The AI Chat API provides two endpoints for interacting with large language models: a multi-turn chat endpoint that supports streaming, and a single-prompt completions endpoint. Both are available on the **Pro** and **Enterprise** tiers and are subject to rate limiting. All endpoints are mounted under `/api/ai/`.

<Note>
  These endpoints are available on Pro and Enterprise plans only. Requests from free-tier users will be rejected with a `403` response.
</Note>

***

## POST /api/ai/chat

Send a conversation to the configured LLM provider and receive a response. You can choose the provider (`openai`, `anthropic`, or `gemini`) and optionally stream the response as server-sent events.

**Headers:**

<ParamField header="Authorization" type="string" required>
  `Bearer <access_token>`
</ParamField>

**Request body:**

<ParamField body="messages" type="object[]" required>
  An ordered list of messages representing the conversation history. Each message must have a `role` and `content`.

  <Expandable title="message properties">
    <ResponseField name="role" type="string" required>
      The speaker role. One of `"system"`, `"user"`, or `"assistant"`.
    </ResponseField>

    <ResponseField name="content" type="string" required>
      The text content of the message.
    </ResponseField>
  </Expandable>
</ParamField>

<ParamField body="provider" type="string" default="openai">
  The LLM provider to use. One of `"openai"`, `"anthropic"`, or `"gemini"`. The provider must be configured with a valid API key in your backend environment.
</ParamField>

<ParamField body="model" type="string">
  The specific model to use (e.g., `"gpt-4o"`, `"claude-3-5-sonnet-20241022"`, `"gemini-1.5-pro"`). If omitted, the provider's default model is used.
</ParamField>

<ParamField body="temperature" type="number" default="0.7">
  Sampling temperature between `0.0` and `2.0`. Lower values produce more deterministic output; higher values increase creativity.
</ParamField>

<ParamField body="max_tokens" type="number" default="1000">
  Maximum number of tokens to generate. Must be between `1` and `16384`.
</ParamField>

<ParamField body="stream" type="boolean" default="false">
  When `true`, the response is streamed as server-sent events (SSE). Each event contains a `token` field with the next piece of text. The stream ends with `data: [DONE]`.
</ParamField>

### Non-streaming example

<CodeGroup>
  ```bash curl theme={null}
  curl --request POST \
    --url http://localhost:8000/api/ai/chat \
    --header "Authorization: Bearer <access_token>" \
    --header "Content-Type: application/json" \
    --data '{
      "messages": [
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Explain the difference between async and sync Python in one sentence."}
      ],
      "provider": "openai",
      "model": "gpt-4o",
      "temperature": 0.5,
      "max_tokens": 200
    }'
  ```

  ```python Python theme={null}
  import requests

  response = requests.post(
      "http://localhost:8000/api/ai/chat",
      headers={"Authorization": f"Bearer {access_token}"},
      json={
          "messages": [
              {"role": "system", "content": "You are a helpful assistant."},
              {"role": "user", "content": "Explain the difference between async and sync Python in one sentence."},
          ],
          "provider": "openai",
          "model": "gpt-4o",
          "temperature": 0.5,
          "max_tokens": 200,
      },
  )
  print(response.json()["content"])
  ```
</CodeGroup>

**Response** — `ChatResponse`:

<ResponseField name="content" type="string" required>
  The full generated text response from the model.
</ResponseField>

<ResponseField name="model" type="string" required>
  The model identifier that was used to generate the response.
</ResponseField>

<ResponseField name="usage" type="object" required>
  Token consumption breakdown for the request.

  <Expandable title="usage properties">
    <ResponseField name="prompt_tokens" type="number">Number of tokens in the input messages.</ResponseField>
    <ResponseField name="completion_tokens" type="number">Number of tokens in the generated response.</ResponseField>
    <ResponseField name="total_tokens" type="number">Total tokens consumed by the request.</ResponseField>
  </Expandable>
</ResponseField>

```json theme={null}
{
  "content": "Synchronous Python executes code line by line and blocks until each operation completes, while asynchronous Python uses `async`/`await` to pause and resume coroutines, allowing other tasks to run during waiting periods.",
  "model": "gpt-4o-2024-08-06",
  "usage": {
    "prompt_tokens": 38,
    "completion_tokens": 42,
    "total_tokens": 80
  }
}
```

### Streaming example

Set `"stream": true` to receive the response token by token as server-sent events. Each event is a JSON object with a `token` field. The final event is the literal string `[DONE]`.

<CodeGroup>
  ```bash curl theme={null}
  curl --request POST \
    --url http://localhost:8000/api/ai/chat \
    --header "Authorization: Bearer <access_token>" \
    --header "Content-Type: application/json" \
    --no-buffer \
    --data '{
      "messages": [{"role": "user", "content": "Count to five."}],
      "stream": true
    }'
  ```

  ```python Python theme={null}
  import requests
  import json

  with requests.post(
      "http://localhost:8000/api/ai/chat",
      headers={"Authorization": f"Bearer {access_token}"},
      json={"messages": [{"role": "user", "content": "Count to five."}], "stream": True},
      stream=True,
  ) as response:
      for line in response.iter_lines():
          if line:
              raw = line.decode("utf-8")
              if raw.startswith("data: "):
                  payload = raw[6:]
                  if payload == "[DONE]":
                      break
                  data = json.loads(payload)
                  print(data["token"], end="", flush=True)
  ```
</CodeGroup>

**SSE stream format:**

```
data: {"token": "One"}

data: {"token": ","}

data: {"token": " two"}

data: {"token": ", three, four, five."}

data: [DONE]
```

***

## POST /api/ai/completions

Generate a single completion from a plain text prompt, without a conversation history. Useful for summarization, classification, code generation, and other single-turn tasks.

**Headers:**

<ParamField header="Authorization" type="string" required>
  `Bearer <access_token>`
</ParamField>

**Request body:**

<ParamField body="prompt" type="string" required>
  The user's input prompt.
</ParamField>

<ParamField body="system_prompt" type="string">
  An optional system message that sets the model's behavior for this request (e.g., `"You are a JSON formatter."`).
</ParamField>

<ParamField body="provider" type="string" default="openai">
  The LLM provider to use. One of `"openai"`, `"anthropic"`, or `"gemini"`.
</ParamField>

<ParamField body="model" type="string">
  The specific model to use. If omitted, the provider's default model is used.
</ParamField>

<ParamField body="temperature" type="number" default="0.7">
  Sampling temperature between `0.0` and `2.0`.
</ParamField>

<ParamField body="max_tokens" type="number" default="1000">
  Maximum number of tokens to generate. Must be between `1` and `16384`.
</ParamField>

<CodeGroup>
  ```bash curl theme={null}
  curl --request POST \
    --url http://localhost:8000/api/ai/completions \
    --header "Authorization: Bearer <access_token>" \
    --header "Content-Type: application/json" \
    --data '{
      "prompt": "Summarize the following in one sentence: FastAPI is a modern, fast web framework for building APIs with Python 3.7+ based on standard Python type hints.",
      "system_prompt": "You are a concise technical writer.",
      "provider": "openai",
      "max_tokens": 100
    }'
  ```

  ```python Python theme={null}
  import requests

  response = requests.post(
      "http://localhost:8000/api/ai/completions",
      headers={"Authorization": f"Bearer {access_token}"},
      json={
          "prompt": "Summarize the following in one sentence: FastAPI is a modern, fast web framework...",
          "system_prompt": "You are a concise technical writer.",
          "provider": "openai",
          "max_tokens": 100,
      },
  )
  print(response.json()["completion"])
  ```
</CodeGroup>

**Response:**

<ResponseField name="completion" type="string" required>
  The generated text response.
</ResponseField>

<ResponseField name="model" type="string" required>
  The model identifier that produced the response.
</ResponseField>

<ResponseField name="usage" type="object" required>
  Token consumption breakdown (same shape as `ChatResponse.usage`).
</ResponseField>

```json theme={null}
{
  "completion": "FastAPI is a high-performance Python web framework for building APIs using standard type hints.",
  "model": "gpt-4o-2024-08-06",
  "usage": {
    "prompt_tokens": 52,
    "completion_tokens": 17,
    "total_tokens": 69
  }
}
```