Skip to content

Fast and Ultrafast mode

Available since gateway 0.4.1.

Choose a service tier per request with service_tier. Omitting the field keeps provider defaults; LunarGate does not enable premium processing automatically. This feature covers HTTP JSON and SSE on the existing Chat Completions and Responses endpoints. It does not add a new WebSocket proxy transport.

Provider mapping

Client value OpenAI Anthropic
omitted or auto native provider settings omit speed
default Standard speed: "standard"
fast or priority Fast speed: "fast", requires fast_mode capability
ultrafast Ultrafast through upstream Responses unsupported
flex or scale preserve native OpenAI semantics unsupported

OpenAI availability depends on the selected model/account. Ultrafast is supported through HTTP Responses, including a Chat client routed to a Responses upstream. An unsupported native combination returns the provider's error. Unsupported translated controls return 400 invalid_request_error with error.param set to service_tier. Retry and fallback preserve the requested mode and skip incompatible fallback targets; LunarGate never removes the tier to retry cheaply. A provider can report that it served a different tier, which remains visible.

OpenAI HTTP example

Configure a native Responses target. This example does not change route defaults for other requests:

providers:
  openai:
    api_key: "${OPENAI_API_KEY}"
routing:
  routes:
    - name: openai-responses
      match:
        path: /v1/responses
      targets:
        - provider: openai
          model: gpt-6.1-sol
          upstream_request_type: responses
          weight: 100

curl http://localhost:8080/v1/responses \
  -H "Content-Type: application/json" \
  -d '{"model":"openai/gpt-6.1-sol","input":"Say hello","service_tier":"ultrafast"}'

Include the configured gateway authentication header if local authentication is enabled. Add "stream": true for SSE. For Chat Completions, use messages and /v1/chat/completions; Fast works on native Chat targets, and Ultrafast requires a route whose target selects upstream_request_type: responses.

Anthropic configuration

providers:
  anthropic:
    api_key: "${ANTHROPIC_API_KEY}"
    default_model: claude-opus-5-5
    capabilities:
      fast_mode: true

The capability defaults to false. Enable it only for an account/model that supports Fast mode; the flag cannot grant upstream access. For preview models Claude Opus 5 and 4.8 (and the legacy Opus 4.6 compatibility case), LunarGate merges fast-mode-2026-02-01 into Anthropic-Beta when sending an explicit speed. It preserves other beta values and does not duplicate the Fast beta. Opus 5.5 does not need this beta. Explicit Standard does not require enabling the Fast capability.

SaaS and replay

Collector metrics and optional request logs carry requested_service_tier and effective_service_tier. The effective value comes from the provider response, never from the request: OpenAI's service_tier is retained; Anthropic usage.speed values fast and standard become fast and default. Metadata is collected with JSON/SSE, including include_usage: false and content sharing disabled. Missing provider confirmation or metadata from an older gateway stays unknown.

Sandbox offers Default, Standard, Fast and Ultrafast. Its selector updates the editable JSON; Default removes the field and replay preserves the original value. The selector is hidden for embeddings. Requests show actual tier and detail shows both requested and actual tier, including provider-side changes. Filters group fast/priority together; unknown matches missing metadata. CSV and public request APIs include both fields. Cache keys distinguish requested tiers, and a cache hit retains the original response tier plus the separate cache-hit marker.

Pricing behavior is unchanged by this feature. Future pricing ownership is SaaS; this change supplies the tier metadata needed for that later roadmap work.

Provider documentation