Skip to content

Model Providers

Choose the model provider on the Runtime Basics onboarding page. Test the connection before continuing.

Onboarding choice Saved provider Authentication
OpenAI openai its key, saved with /auth model
OpenAI Codex OAuth openai-codex ChatGPT/Codex OAuth store or an ephemeral Codex access-token file
xAI Grok OAuth xai-oauth xAI device OAuth store or an ephemeral xAI OAuth access-token file
Anthropic anthropic its key, saved with /auth model
Google AI gemini its key, saved with /auth model
xAI (API key) xai its key, saved with /auth model (also used for Grok voice)
OpenRouter litellm the OpenRouter API endpoint plus its key, saved with /auth model
OpenAI Compatible litellm its API base URL plus its key, saved with /auth model

Fresh onboarding selects OpenRouter, uses https://openrouter.ai/api/v1, and pre-fills DeepSeek V4.1 Flash as deepseek/deepseek-v4.1-flash. Imported configs and explicit operator choices still take precedence. Always confirm the provider, endpoint, and model shown on the Review page.

Model Profile Controls

The selected provider and model are saved with their model-specific controls:

  • reasoning_effort is an editable provider-specific hint. Common values are low, medium, high, and max, but the selected provider decides which values it accepts.
  • context_window_tokens controls prompt budgeting and compaction and must be at least 1024 when set.
  • supports_image_input, supports_audio_input, and supports_file_input are independent flags. File support refers to native document input (including supported text and Markdown files), not access to ordinary file tools.

Live changes use the same complete profile in control/LLM_OVERRIDES.yaml. A provider/model override change clears omitted model-specific fields to unknown or the provider default, avoiding accidental reuse of another model's settings. The LiteLLM API mode (litellm_api_mode) describes the route to the endpoint rather than one model, so a model-only switch keeps it; only a provider change resets it. Set it explicitly when the new model needs a different mode. Audio support is recorded as capability metadata; the general runtime does not automatically create a native-audio bridge for every provider.

When you ask TeamLead to switch an agent's model with access control on, the change applies at once if you are an administrator who gave TeamLead model choice (the Trusted profile does); otherwise an administrator approves it in the Inbox. The API mode is not part of model choice, because it changes the endpoint route. See Permissions.

Before a switch, TeamLead looks the model up: first on your own LiteLLM server (GET <litellm_api_base>/model/info, with the same key the agents already use for model requests), then in OpenRouter's model directory and LiteLLM's built-in model data. It takes the context window and the image, audio and file input from that lookup and asks you only when the sources disagree or say nothing. An agent or workspace-default override switch resets these settings. When the runtime cannot identify a model's window and none is set, it assumes 128,000 tokens and the runtime logs a warning (event=context_window_fallback); a window that is too small makes the runtime hide tool results early. The server's key stays in the request header and never appears in what the agent reads. To give your server's answer, set max_input_tokens, supports_vision, supports_pdf_input, supports_audio_input and supports_reasoning in each model's model_info in the LiteLLM configuration.

TeamLead then tries the new model once with the settings it would use (a short test call) before it asks you anything. If the model answers, TeamLead tells you it works and asks you to confirm. Your confirmation applies it at once with access control off, or with access control on when you gave TeamLead model choice; otherwise an administrator approves it in the Inbox. If the test fails, nothing is staged or changed, and TeamLead tells you the error. The model is tested again right before the settings are saved.

The same tests run for instance-baseline changes (llm.*), using the baseline's own endpoint and request settings. Baseline fields you omit keep their configured values, so include the new model's window and input capabilities explicitly. Baseline changes take effect after the reported restart.

After a model change applies, TeamLead shows a table of every model setting that agent type now runs with: provider, model, reasoning effort and summary, request API, context window, and image, audio and file input. For an OpenAI reasoning model (gpt-5 and later, o-series) reached through LiteLLM's Chat Completions API, it also suggests the Responses API (litellm_api_mode: responses) with reasoning_summary: auto, which lets you read the model's reasoning summaries. That suggestion changes nothing, and changing the request API needs an administrator's approval.

Model Call Time Limits

Three llm settings stop a model call that hangs, so a task does not stay "in progress" forever:

  • request_timeout_seconds (default 120) is the HTTP timeout: the longest wait to connect, or for the next bytes of a response.
  • stream_idle_timeout_seconds (default 300) is the longest wait for the next piece of model output: text, a reasoning summary, a tool call, or the end of the answer. Keep-alive pings that some proxies (such as a LiteLLM proxy) send on an idle stream do not count, so they no longer keep a stalled call open.
  • model_call_timeout_seconds (default 1800) is the longest one model call may take in total.

When a limit is reached, the runtime closes the call and logs event=model_stream_timeout with the limit that was reached. If model failover is set up, the task continues on the next fallback model; otherwise the task ends and the person sees "The model service timed out before completing this request, and no fallback model was available. Please try again." Reasoning models can think for minutes without sending anything. If a model with high reasoning effort is stopped while it is still working, raise stream_idle_timeout_seconds, for all agents or for one agent type under agent_types.<type>.llm.

xAI Grok OAuth

xAI is supported directly. Select xAI Grok OAuth, click Connect xAI, approve the device flow, and run Test Connection. If no model is supplied, the provider uses grok-4.5.

For an existing instance:

./.venv/bin/python assistant_cli.py xai-oauth-login \
  --config <instance_root>/instance_config.yaml
./.venv/bin/python assistant_cli.py xai-oauth-status \
  --config <instance_root>/instance_config.yaml

The provider stores rotating credentials in the instance-owned auth store. XAI_OAUTH_ACCESS_TOKEN_FILE is available for ephemeral automation. The xai-oauth provider does not fall back to XAI_API_KEY; that variable is used by API-key workflows such as Grok voice. See xAI Grok OAuth.

OpenAI Codex OAuth

Use openai-codex for ChatGPT/Codex subscription OAuth rather than an OpenAI API key. Authenticate a file-backed Codex CLI home and mount it read-only, or import it into the instance-owned Universal Agents auth store. See OpenAI Codex OAuth.

Model Keys

Model keys never go in instance_config.yaml or chat messages. Each key belongs to one endpoint address (for example https://bifrost.example.com), and agents send it only to that address.

Save a key with /auth model (administrators and owners). The command opens a secure broker page that lists the endpoints your configuration uses, which key each one has, and anything that overrides it. The page has one form: pick an endpoint from the Endpoint list (the team's endpoints, OpenAI, Anthropic, Google Gemini, xAI, or New OpenAI-compatible address) and it fills in the address, model and request API; then enter the key. Saving first sends one short test request with the key; if the test fails, nothing is saved and the result, in red just above the form, says what most likely went wrong (a refused key, an address where nothing answers, an unknown model) and shows the provider's answer. You can type an address with or without /v1: the test tries both and the page says which one answered. Agents use a saved key on their next request, without a restart. No key needed is for local servers that accept requests without one. The page stays open for 24 hours; its Close buttons (top and bottom) end it.

Environment variables still work and win. LITELLM_API_KEY (the OpenAI-compatible endpoint) and EMBEDDING_API_KEY (the embedding service), or their _FILE forms, override the key saved for that endpoint. LITELLM_API_KEY also covers the embedding service when it has the same address as the model endpoint. /auth model and /auth status say when an override is in effect, and the Status page names where it comes from: the instance's team.env, a secret file, or the container's own environment. The launcher passes team.env values only to the processes it starts, so a shell opened with docker exec does not show them; look in the file. To use the saved key instead, remove the variable from team.env (or its secret file) and restart the container.

When a provider rejects a key (expired, revoked or out of credit), the request stops and you are told whose key it was and who fixes it. Administrators get an Inbox notice. An administrator runs /auth model and saves a working key; this works even while no agent can reach its model. Then send the request again. A key set by an environment variable is fixed in team.env instead, followed by a container restart.

If the broker cannot read its credential store, the request stops with a credential-store error. This does not trigger provider overload retries or switch to a fallback model. An administrator checks that the broker and Sealbox are running. For an existing Sealbox tenant with missing, damaged or mismatched token/key files, restore the original files from a backup. Do not delete the tenant mapping or generate replacement keys, because existing encrypted credentials need the original material.

An owner without a Sealbox tenant has no saved credentials yet. Looking up their key creates no tenant; the first authorized save creates it. Someone without a personal key can use the team's key when their settings allow it.

Your own model and keys: /auth model me. Anyone on the team can open this page (members also get it from plain /auth model). It uses the same one-form layout:

  • With Use this as my model ticked, the endpoint you test becomes your model: any provider and any address, tested before it is saved. Every agent's work for you then uses it with your key, including agent types an administrator set to a particular model: your chats, your scheduled runs, and work agents hand to each other for you. You can say whether it accepts images or files and give its context window (otherwise agents assume the window of the team's model). Stop Using My Model removes it; your keys stay.
  • Without it, the key is yours for an endpoint the team already uses: agents keep the team's models but use your key for your work.

Work for nobody, such as internal maintenance, keeps the team's models and keys. Embeddings (memory search) and Grok voice always use the team's keys. If a provider rejects your key, your request stops and you are told; it never switches to the team's key, so nobody's spending changes without a decision. When a person is removed from the team, their keys and their model are deleted with their other sign-ins.

Who may use the team's keys. Administrators decide per person whether their work may use the team's model keys: --team-keys or --own-keys when adding them, and /user team-keys <usr_...> on|off later (see Multi-User Onboarding). People added without a choice get the team's model-key sharing: shared until someone chooses, so they may use the team's keys. Owners and administrators change it with /accounts share model or /accounts private model (see Shared or private); it decides only what people added from now on start with, and people already in the team keep their setting. The setting gateway.users.new_people_use_team_model_keys is removed, and startup refuses a configuration that still has it. Someone without the team's keys and without a model of their own is told "You haven't set up your own model yet. Use /auth model me." and their requests do not run. This also applies to keys from environment variables, which are the team's too. The Status page lists such people as a warning, and people with their own model as a note.

Switching the OpenAI-compatible endpoint. Ask TeamLead, for example "switch to Bifrost". TeamLead sees every endpoint whose key is already saved (addresses only, never keys) and picks the one you named; give the address only if no saved one matches. A new endpoint needs its key saved first with /auth model → New OpenAI-compatible address, or TeamLead tells you that is missing. When the change is prepared, one test request goes to the endpoint (a bare host is tried with /v1 first, then without), and the form that answers is the one saved. The test waits as long as agents' own requests do (llm.request_timeout_seconds, 120 seconds by default), because reasoning models can think for a minute before their first output. An administrator approves the change in the Inbox; its text names the new host and whose key the requests use. Before anything is saved, one test request goes to the new endpoint with the team's model, and a failure changes nothing. No restart is needed: each agent switches at the start of its next request (one that is running finishes on the old endpoint), so for a minute some agents may still use the old one. Because keys belong to an address, the old endpoint's key is never sent to the new one. If LITELLM_API_BASE is set in team.env or the container environment, it overrides the saved endpoint, so the switch is refused until the operator removes it and restarts.

OpenAI, Anthropic, Google AI and xAI each have one fixed address (api.openai.com, api.anthropic.com, Google's Gemini API, api.x.ai), so choosing the provider chooses the endpoint; save their keys on /auth model the same way. OPENAI_API_KEY, ANTHROPIC_API_KEY, GEMINI_API_KEY and XAI_API_KEY (or their _FILE forms) override the saved key while they are set. Only the OpenAI-compatible endpoint (llm.litellm_api_base) can be switched to another service.

For OpenRouter or another OpenAI-compatible endpoint, provide both the API base URL and a model ID accepted by that endpoint. In Universal Agents, llm.provider: litellm is equivalent to choosing the OpenAI API-compatible runtime path; it does not require an OpenAI-operated endpoint or a separately deployed LiteLLM proxy.

Built-in LiteLLM capability and pricing data comes with the installed release. Updating LiteLLM updates that data. Startup does not download a model catalog. For newer models, you can set capabilities explicitly or use the model lookup to read your endpoint's model information or OpenRouter's current directory.

For Fireworks and Novita, UA selects its reduced provider-compatible tool and terminal-output schemas from either the provider marker in the model ID or the configured API-base hostname.

Model Failover Is Opt-In

Fresh configurations keep llm.failover.enabled: false and have no fallback candidates. To enable cross-provider switching, configure an ordered llm.failover.fallbacks list and then enable the policy explicitly. This setting is under Advanced Setup during onboarding.

An active fallback lease never takes precedence over a later explicit model profile change in control/LLM_OVERRIDES.yaml. Removing the leased target or changing the primary model invalidates the lease immediately; a failure from a request that was already in flight applies the new primary without creating a new lease for the superseded model. Disabling failover also clears an active lease while retaining provider-health history.

LiteLLM Connection Failures

UA keeps explicit overload and rate-limit responses separate from endpoint connectivity failures. An overload may offer retry choices. A refused connection instead reports a connectivity/configuration failure after any enabled and configured model fallback is unavailable or exhausted.

Check llm.litellm_api_base and any LITELLM_API_BASE environment override from inside the UA runtime. In Docker, localhost and 127.0.0.1 identify the UA container itself; use a container-network service name, a host-gateway address, or the externally reachable OpenAI-compatible endpoint URL as appropriate.

Model Keys and /auth

Model API keys (OpenAI-compatible endpoints such as OpenRouter or Bifrost, OpenAI, Anthropic, Gemini, xAI and embedding services) are saved with /auth model and /auth model me; see Model Keys. The OpenAI Codex and xAI Grok OAuth sign-ins are not part of /auth: they use their own stores, set up as described in OpenAI Codex OAuth and xAI Grok OAuth.

Response-copy mismatch diagnostics

openai_reasoning_output_sources_mismatch compares completed streamed items with the final response's copy of those items. The runtime keeps the completed stream for tool-loop replay when it contains tool calls; a reasoning-only partial stream uses the final response instead. This warning does not mean a tool failed.

Lifecycle status and equivalent JSON argument spacing/key order are logged at debug level. So is encrypted_content that differs only because OpenAI encrypted the same reasoning separately for each copy: both copies are present, have the same length, and neither carries LiteLLM's litellm_enc: wrapper. Changes to arguments, call identity or reasoning text still warn, as does encrypted content that is missing from one copy, changes length, or is wrapped by the proxy on only one side. Logs identify the response, differing item indexes and field names without including arguments, reasoning text or encrypted content. Use those fields to investigate a provider/proxy discrepancy; an index-only warning cannot establish its cause.

When encrypted_content differs, encrypted_content_diagnostics describes each differing item without its content: both lengths, whether each copy carries LiteLLM's litellm_enc: routing wrapper, whether the copies match once that wrapper is removed, and how many leading characters they share. A wrapper on only one side with equal_after_proxy_unwrap: True means the LiteLLM proxy rewrote one copy. Same-length unwrapped copies that share only their first 12 characters are OpenAI's normal separate encryption of each copy: its encrypted content is a Fernet token whose first 12 characters encode the format version and timestamp, followed by a fresh random value. A long shared prefix with different lengths means one copy was cut short.