6ioAPI
RUEN

Retries and failures

Networks and providers sometimes fail. This page says what to do so as not to lose money or pay twice.

We do not retry a paid request ourselves: if the provider did not answer, the generation may still have happened. So the retrying is up to you, and it is safe with the Idempotency-Key header.

Idempotency-Key

Add the Idempotency-Key header to a paid request: any unique string up to 255 characters, for example a UUID. If the request breaks off and you send it again with the same key, a second generation does not start and the money is not charged a second time.

curl https://api.6io.io/v1/chat/completions \
  -H "Authorization: Bearer sixio_sk_..." \
  -H "Idempotency-Key: 6f1d3a52-8b0e-4c1c-9a4e-2f1f0f8b7c11" \
  -H "Content-Type: application/json" \
  -d '{"model": "op/openai/gpt-4o-mini", "messages": [{"role": "user", "content": "Hello!"}]}'
What you sent The answer
a new key ordinary processing
the same key and the same request 409 idempotent_request_exists, the X-Sixio-Request-Id header holds the ID of the first request; there is no new generation
the same key, a different request 422 idempotency_key_reused
the same key after a refusal with no charge (for example, 503 upstream_busy or a provider 4xx) ordinary processing: the key is freed together with the reserve

The key works within your account. One key is one job, not one attempt: on a retry take the same one.

A finished answer is not returned on a retry: we do not store the texts of requests and answers. After a 409 you can look at the state of the first request with GET /v1/requests/{id}, but the text of the answer can no longer be fetched. If you need the answer reliably, use background jobs: their result is kept for a day.

What to do on an error

What you see What happened The money What to do
429 and Retry-After an account or IP limit was hit not charged wait Retry-After seconds and retry
503 upstream_busy the provider did not accept the request because of the rate, or could not be reached not charged retry after Retry-After
503 service_unavailable the service is temporarily unavailable not charged retry in a few seconds
402 insufficient_funds the available funds are not enough not charged top up the balance
4xx from the model something is wrong in the request the reserve is lifted fix the request
502 or 503 upstream_unavailable the provider did not answer, the generation may have gone through the reserve is held do not retry blindly: look at GET /v1/requests/{id} or retry with the same Idempotency-Key
the connection dropped after sending you do not know whether the request arrived possibly charged retry with the same Idempotency-Key: a 409 answer shows the ID of the first request

A safe retry in code

import time
import uuid

import requests

API = "https://api.6io.io/v1"
H = {"Authorization": "Bearer sixio_sk_..."}
body = {
    "model": "op/openai/gpt-4o-mini",
    "messages": [{"role": "user", "content": "Hello!"}],
    "max_tokens": 100,
}
key = str(uuid.uuid4())  # one key per job, not per attempt

for attempt in range(4):
    try:
        r = requests.post(f"{API}/chat/completions", headers={**H, "Idempotency-Key": key}, json=body, timeout=120)
    except requests.RequestException:
        time.sleep(2)  # a drop: the outcome is unknown, a retry with the same key is safe
        continue
    if r.status_code == 409:  # such a request has already been accepted
        print("already accepted, ID:", r.headers["X-Sixio-Request-Id"])
        break
    if r.status_code in (429, 503):  # nothing is charged, or the outcome is being checked (then the next answer will be 409)
        time.sleep(int(r.headers.get("Retry-After", "5")))
        continue
    print(r.status_code, r.json())
    break

If the outcome stays unknown

A request with an unknown outcome is not lost. We check at the provider what it cost and close the money: we charge the actual cost or lift the reserve. You can see it in billing_status (billing_pending, then billing_review). If the amount could not be confirmed automatically within a day, the request goes to manual review. The reserve stays meanwhile, and an administrator decides.