LLM API fallback means that when a model provider returns errors or goes down, your application retries sensibly and then sends the request to a second model or provider, instead of failing in front of the user. Done well, it needs four parts: a clear rule for which errors to retry and which to fail over, a fallback model you have actually tested with your prompts and tools, a circuit breaker so you stop hammering a provider that is down, and a fallback that meets the same data and cost rules as the primary.

Short version
Retry transient errors (429 with retry-after, 500, 504, Anthropic's 529 overloaded_error, connection drops) a couple of times with backoff, which the official SDKs already do. If the provider still fails, switch to a tested fallback model, either in your own code or through a gateway such as LiteLLM or OpenRouter. Never retry 400, 401, 403, 413 or a spend-limit 429: those will not fix themselves.

Why this is in the news

On November 13, 2026 Anthropic's status page posted an incident titled "Elevated errors on claude.ai, Claude Code, and platform.claude.com". The link reached the top of Hacker News (486 points and 383 comments when we checked), alongside "Claude Is Down" (197 points, 173 comments). We have not read the incident's timeline and say nothing here about its cause or length. The point for builders is simpler: every hosted model has bad hours, and an application that calls exactly one model has no plan for them. Search demand is clear: Google autocomplete suggests "llm api fallback", "llm gateway fallback", "litellm fallback chain", "anthropic 529 overloaded" and "claude overloaded error api". Autocomplete shows that people type these phrases, not how often.

Which errors to retry, and which to fail over

Anthropic's error reference lists these HTTP codes for the Claude API. The same split applies to other providers, though the codes and names differ.

CodeTypeWhat to do
400invalid_request_errorDo not retry or fail over. Fix the request.
401 / 403authentication_error / permission_errorDo not retry. Fix the key or permissions and alert someone.
402billing_errorDo not retry. A billing problem; alert someone.
413request_too_largeDo not retry. Shrink the request.
429rate_limit_errorWait for retry-after and retry; fail over if it persists. A spend-limit 429 is different (below).
500api_errorRetry with exponential backoff, then fail over.
504timeout_errorRetry, or use streaming for long requests, then fail over.
529overloaded_errorRetry with backoff, then fail over.

What a 529 overloaded error means

Anthropic's reference describes 529 as "The API is temporarily overloaded" and warns that it "can occur when the API experiences high traffic across all users". In other words, it is not about your account or your request: retrying the same request a little later usually works, and if it keeps happening, a second provider is the only thing that helps. That makes 529 the clearest case for failover.

Not every 429 is worth retrying

Anthropic's rate-limit page says a 429 usually carries a retry-after header, "the number of seconds to wait until you can retry the request", plus anthropic-ratelimit-* headers showing what is left. Two kinds of 429 are different. One has error_code enforced_spend_limit_reached: your own spend cap is hit, there is no retry-after, and retrying will not help. The other is ramp_limit_exceeded; the page notes that "a sharp increase in usage can also cause 429 errors", so a sudden traffic jump can be throttled even below your normal limit. Read the error body, not just the status code.

Errors in the middle of a stream

With streaming, the API can return 200 and then send an error event partway through. Anthropic's reference says normal error handling does not apply there. Decide in advance whether a half-finished answer is retried from the start, shown with a notice, or discarded, and make sure your fallback logic sees these errors at all.

Retries you already have

Before writing your own retry loop, check what the SDK does. Anthropic's Python SDK retries connection errors, 408, 409, 429 and 500-and-above errors "twice, with a short exponential backoff" by default; the error reference adds that the official SDKs honor retry-after when present. The default timeout is 10 minutes, and the SDK refuses non-streaming requests expected to take longer. OpenAI's Python SDK also retries "2 times by default" on the same classes of error, and notes that requests "are retried only when their body can be safely resent". Both take a max_retries setting, where 0 turns retries off.

Two consequences. First, if you wrap the SDK in your own retry loop without turning its retries off, three attempts become nine. Second, a 10-minute timeout is far too long for a chat reply: set a shorter timeout per call so that failover can happen while the user is still waiting.

Three ways to add failover

1. In your own code

For one or two models this is often enough. A minimal pattern in Python, with the SDK's own retries left on and a short timeout:

import anthropic, openai

claude = anthropic.Anthropic(timeout=30, max_retries=2)
backup = openai.OpenAI(timeout=30, max_retries=2)

FAILOVER = {429, 500, 504, 529}

def answer(prompt):
    try:
        r = claude.messages.create(
            model="claude-sonnet-5-5", max_tokens=1024,
            messages=[{"role": "user", "content": prompt}])
        return r.content[0].text, "claude"
    except (anthropic.APIConnectionError, anthropic.APITimeoutError):
        pass
    except anthropic.APIStatusError as e:
        if e.status_code not in FAILOVER:
            raise  # 400, 401, 403, 413: fix it, do not hide it
    r = backup.chat.completions.create(
        model="gpt-6-luna",
        messages=[{"role": "user", "content": prompt}])
    return r.choices[0].message.content, "backup"

This is a sketch, not production code: it does not check for a spend-limit 429, has no circuit breaker, and sends the same prompt to both models. Return which model answered so you can log it.

2. A self-hosted gateway such as LiteLLM

LiteLLM's proxy puts many providers behind one OpenAI-style endpoint and handles retries and fallbacks in config. Its documentation describes three fallback lists: fallbacks "for all remaining errors", context_window_fallbacks for prompts that are too long for a model, and content_policy_fallbacks for content-policy refusals. Fallbacks run in order, and only after num_retries is used up. allowed_fails and cooldown_time (in seconds) take a failing deployment out of rotation for a while, which is a circuit breaker. A minimal setup looks like this:

litellm_settings:
  num_retries: 2
  request_timeout: 30
  fallbacks: [{"primary-model": ["backup-model"]}]

router_settings:
  allowed_fails: 3
  cooldown_time: 30

Here primary-model and backup-model are names you define in model_list. A request can opt out with "disable_fallbacks": true, and LiteLLM's spend logs record which fallbacks were attempted.

3. A hosted router such as OpenRouter

OpenRouter accepts a models array in priority order. Its fallback page says "any error can trigger the use of a fallback model", including context-length errors, moderation flags, rate limiting and downtime, and that requests "are priced using the model that was ultimately used", which is returned in the response's model field. Separately, its provider object controls which hosts may serve one model: order, allow_fallbacks (true by default), only, ignore, data_collection and zdr among others. Setting allow_fallbacks to false means only the providers you list are tried.

A hosted router is the fastest to set up, but it adds a company to your data path and a dependency of its own. A self-hosted gateway keeps data paths in your hands but is one more service to run.

What breaks when you switch models

Failover is easy to configure and easy to get wrong. The problems show up in the answers, not the logs:

Data and compliance rules still apply

A fallback provider receives the same user data as the primary. If your contract, privacy notice or customer promises name a provider, a data region or a retention setting, the fallback has to meet them too, or the fallback has to be switched off for that traffic. If you rely on zero data retention, confirm the fallback route has it; our zero data retention guide covers what each provider offers. OpenRouter's zdr and data_collection settings and LiteLLM's per-request disable_fallbacks are the controls to look at.

Testing and monitoring failover

How we approach this

For client applications we treat the model as a dependency that will sometimes fail, like a payment gateway. We pick a primary and a fallback that both pass the client's evaluation set and data rules, keep the SDKs' short retries, set timeouts that fit the user's patience, add a circuit breaker, and log which model answered every request. The wider integration work is in our guide to integrating AI APIs into your application.

Sources: Anthropic, API errors, Anthropic, rate limits, Anthropic, Python SDK, OpenAI Python library README, LiteLLM, proxy fallbacks and retries, LiteLLM, routing, OpenRouter, model fallbacks, OpenRouter, provider routing, Claude status, November 13, 2026 incident (title only). All read on November 14, 2026.