Fallback And Redundancy Management

via “fallback-and-retry-logic-with-cooldown-management”

Unified API for 100+ LLM providers — OpenAI format, load balancing, spend tracking, proxy server.

Unique: Implements a cooldown management system (cooldown_manager.py) that tracks per-deployment failure rates and temporarily deprioritizes failed providers. Uses exponential backoff (1s, 2s, 4s, 8s, ...) for retries and configurable cooldown periods (default 30s) before re-enabling a provider. Fallback chains are defined in router configuration and evaluated sequentially until success.

vs others: More sophisticated than simple retry (includes cooldown and failure tracking); supports custom fallback chains vs fixed fallback logic; automatic provider deprioritization vs manual intervention