Free Models Router

Automatically selects and routes inference requests to available free models on OpenRouter's network using probabilistic load balancing. The router maintains a real-time registry of free models across multiple providers (Meta, Mistral, etc.), filters them based on task compatibility and availability, and randomly distributes requests to balance load and prevent any single model from being rate-limited. This eliminates the need for developers to manually track which free models are currently available or manage fallback logic.

Exposes a standardized OpenAI Chat Completions API interface that accepts requests in OpenAI's message format and returns responses in OpenAI's completion schema, enabling drop-in compatibility with existing OpenAI client libraries (Python, Node.js, Go, etc.). The router translates incoming OpenAI-formatted requests into provider-specific formats for the selected backend model, then normalizes responses back to OpenAI schema, hiding provider heterogeneity from the caller.

Maintains a dynamic registry of free models from multiple inference providers (Meta Llama, Mistral, Nous Research, etc.) and distributes requests across them using probabilistic selection. The router queries provider availability in real-time, filters models by task type (text generation, image generation) and capability (context window, parameter count), and selects a model from the available pool. This prevents single-provider dependency and maximizes uptime by automatically falling back to alternative models when one provider's free tier is exhausted.

Executes text-to-text inference requests (chat completions, code generation, summarization, translation) by routing prompts to the selected free model and returning generated text. The router handles message formatting, context window management, and response parsing, supporting both single-turn and multi-turn conversations through OpenAI-compatible message arrays. Supports streaming responses for real-time output delivery.

Routes image generation requests (text-to-image) to available free image generation models on OpenRouter, handling prompt formatting, parameter translation, and image encoding/decoding. The router selects from the free image model pool based on availability and distributes requests to prevent rate-limiting on any single model. Returns generated images in standard formats (PNG, JPEG) with metadata about the model used and generation parameters.

Implements a transformation layer that converts incoming requests from OpenAI format into provider-specific request formats, and normalizes responses back to OpenAI schema. The middleware handles parameter mapping (temperature, max_tokens, top_p), message formatting, and response parsing, abstracting provider-specific API differences. This enables the router to support multiple backend providers without exposing their heterogeneous APIs to clients.

Monitors the availability and rate-limit status of free models in the pool by querying provider health endpoints and tracking request success/failure rates. The router maintains a real-time registry of which models are currently available, their current load, and estimated wait times, using this data to filter the selection pool and avoid routing requests to exhausted or unavailable models. This prevents requests from failing due to rate limits or provider downtime.