Chat Completions: context, tools and errors#
To integrate with a model through Sherpa AI Server, use POST /api/v1/threads/chat/completions. It performs one request without creating a chat or run. The client supplies the message history.
Authentication and model selection#
A valid API token with Chat create permission is required. POST /api/v1/threads/chat/preflight requires read permission. Authentication determines the account. There is no need to send account_guid.
Set model to the name of a model card in the current account. The server selects an active instance of that card. Legacy selection by active instance name in the same account is also supported. An unknown or foreign name returns 404 model_not_found. A card without active instances returns 503 model_unavailable.
An explicit parameter overrides the model card and server environment settings. Missing parameters use the card, then the environment. The entire chat_template_kwargs object is replaced. null, {} and an omitted field differ. Supply only one output budget field: max_tokens or max_completion_tokens.
Checking context before generation#
Send the complete intended request to /api/v1/threads/chat/preflight with Authorization: Bearer <API_TOKEN> and Content-Type: application/json. Replace Support model with an available card name in this example:
{
"model": "Support model",
"messages": [{"role": "user", "content": "Read the test file"}],
"tools": [{
"type": "function",
"function": {
"name": "read_file",
"parameters": {
"type": "object",
"properties": {"path": {"type": "string"}},
"required": ["path"],
"additionalProperties": false
}
}
}],
"max_tokens": 512,
"chat_template_kwargs": {"enable_thinking": false}
}
Preflight does not generate output, execute tools or create history. The result is in data:
| Field | Meaning |
|---|---|
prompt_tokens |
Input size including messages, tool schemas, and reserves for images and response format. |
output_tokens |
Output budget. null when unknown. |
context_limit |
Known context window. null when no limit is configured. |
fits |
Whether the request and output budget fit. null when information is insufficient. |
exact, count_method |
Exact tokenizer count or estimate. Images and response format may add estimated reserves. |
would_reject |
Whether an authoritative bound rejects the request. An estimate alone does not block generation. |
fallback_reason |
Reason for estimation, such as an unavailable tokenizer or a cloud provider without one. |
allowed_max_output |
Context remaining after the calculated input. An estimate is advisory. |
If would_reject=true, reduce history, schemas or output budget and check again. Then send the same request to /api/v1/threads/chat/completions. The server checks the budget for the actual instance, including after failover. Instances may have different limits. Preflight does not pin an instance for the next request.
Tool calls#
An external API client executes returned calls itself. tools declares functions and JSON Schema for their arguments. Include additional input fields in the schema and function.arguments.
In the next request, retain the assistant message with tool_calls and append role=tool results. function.arguments and tool.content are strings. tool_call_id links each result to its call. Application statuses, errors, artifacts and metadata are JSON inside the content string:
[
{"role":"assistant","content":null,"tool_calls":[{"id":"call_1","type":"function","function":{"name":"read_file","arguments":"{\"path\":\"sample.txt\"}"}}]},
{"role":"tool","tool_call_id":"call_1","content":"{\"status\":\"ok\",\"data\":\"Example text\",\"artifacts\":[]}"}
]
Each new call ID must be unique in the history. Tool results must follow their assistant message before any message of another role. Retain invalid arguments together with the error result so the model can repair the call. Do not repeat successful actions with side effects.
In internal Agent mode, repairable arguments are returned to the model for correction. The number of such errors alone does not terminate the task. Repeating an identical invalid call puts the run in waiting with continuation.required=true. After correcting the cause, explicitly resume through the normal execution method. Context and completed results are retained. Cancellation and actual access restrictions remain effective.
/api/v1/threads/responses uses the separate Responses format. Do not mix its items with Chat Completions messages.
Errors and retries#
An error includes error.code, repairable, retryable, request_id and safe details. When repairable=true, correct the request using the details. When retryable=true, a delayed retry is allowed, respecting Retry-After. These flags do not authorize repeating completed tool actions.
| Code | Action |
|---|---|
invalid_arguments, invalid_tool_history |
Correct parameters, schema or call/result pairs. |
context_limit |
Reduce input or budget. Check numeric information in details. |
model_not_found, model_unavailable |
Check the name, account and active instances. |
access_denied |
Check the token, permissions and server access to the provider. |
upstream_rate_limited |
Respect Retry-After. |
upstream_timeout, upstream_unavailable, invalid_upstream_response |
Check retryable and retry later when appropriate. |
For SSE, set stream=true. Read through completion and handle event: error. HTTP 200 indicates the start of the stream and does not guarantee successful completion. Read failures or interrupted streams produce a machine error with the same request_id.
Request diagnostics#
Clients should send X-Request-ID: 1 to 128 characters from A-Z, a-z, 0-9, ., _, :, -. If missing or invalid, the server creates a new ID.
JSON and SSE responses include X-Request-ID and X-Sherpa-Attempt-Count. When a card is selected, X-Sherpa-Model-Guid, X-Sherpa-Model-Instance-Guid and X-Sherpa-Configuration-Revision are available. The revision distinguishes effective configuration changes. JSON also includes request_meta.request_id. Errors before model selection have no instance information.
When contacting an administrator, retain the ID, time, error code and diagnostic headers. The existing server log correlates attempts, failure reasons, latency, usage and stream completion by ID. Background runs are linked by run GUID and an ID of the form run:<GUID>. The scheduling event links the external ID to the GUID. Only infrastructure administrators can access the log. Message bodies, tool arguments and keys are excluded from the new diagnostic record.
Timeouts for individual model calls depend on connection settings. Context checking and diagnostics do not impose a duration limit on an external robot.