One endpoint, 130+ models from 90+ providers: KeiRouter is a self-hosted AI gateway, so apps, coding tools, and agents connect once and stop juggling provider keys. Your code speaks OpenAI Chat, OpenAI Responses, Anthropic Messages, or native Gemini, and KeiRouter translates to whichever upstream you want, including AWS Bedrock, Azure AI Foundry, Vertex, Snowflake Cortex, and self-hosted runtimes like Ollama and vLLM. Fallback chains keep requests moving when a provider goes down or hits a rate limit, using priority, round-robin, latency, or cost ordering, so coding sessions do not stop mid-task. Five token savers compress prompts on the way in and trim responses on the way out, with a live dashboard showing savings recovered. A response cache answers repeated prompts instantly for zero marginal cost, with optional semantic matching. Per-key budgets and rate plans set USD, token, RPM, and concurrency limits with auto-cutoff, stopping surprise bills. Guardrails scan for PII, prompt injection, toxicity, and banned topics at global, provider, model, chain, and key scopes, with a monitor showing what got blocked and what it cost. The dashboard tracks usage by provider and model, latency, a live console tracing each request through its fallbacks, and provider health. You can import from 9router or OmniRoute, white-label the portal for your team, and run replicas on Postgres and Redis for high availability. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. MIT licensed.

SmarterRouter stops you from playing model roulette with your local AI setup, choosing the right model for every prompt so you never have to. Point your applications at one familiar endpoint and send the request; the router studies the prompt, weighs category and complexity against benchmark results gathered on your actual hardware, and hands the job to the model most likely to answer well. It discovers the models your backend already serves, profiles each one with standard tests the first time it meets them, and keeps learning from the feedback your usage generates. When the first choice fails, a backup model takes over without your application noticing. Repeated questions come straight back from cache instead of burning another generation. Your graphics memory is watched continuously across one card or several, and models are unloaded before an ambitious request runs the machine dry. Cloud providers can join the same routing pool when a question deserves them, while everything local stays local and free. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. MIT licensed.

Point every coding agent and chat client at one self-hosted gateway, then choose which model answers from a single browser console. Add providers by preset or custom OpenAI-compatible URL, or sign in with the subscriptions you already pay for. Routing groups bundle several models into one name and move to the next member when a quota runs out, so the calling agent never sees the error. Create a gateway key per client, cap each with daily token or cost budgets, restrict its models, and watch usage counted separately. The Usage pages chart tokens, cost, request counts, cache hit rate, and a yearly heatmap, split by model, provider, or agent, with CSV export. A Routing view replays how each request was routed, with timing for masking, connecting, and the vendor. Masking swaps passwords, API keys, and tokens for placeholders before a prompt leaves your server and restores them in the answer. The browser page is protected by a key you set at launch. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. MIT licensed.

One endpoint for every model account you own: 9Router Pro sits between your applications and a fleet of LLM providers, decides which account answers each request, and keeps the whole operation visible on a live dashboard. Its signature trick is prompt injection scoped three ways, globally, per API key, or per model, delivered either as a system prompt or as a scripted chat history that primes the model far harder than instructions alone. Keys are products here: each carries a rolling token budget over 24 hours, 7 days, or 30 days plus rate limits per minute, hour, and day, so a resold plan enforces itself with polite 429 responses. Routing stays healthy on its own, with accounts tiered as clean, suspect, or cooldown, least-recently-used spreading across the fleet, flaky upstream responses retried against the next account, and proxy pools health-checked with one-click cleanup of the dead. The monitoring dashboard leads with VPS specs, fastest and most-used providers, top API key, and usage spectrum charts, while backup and restore cover providers, combos, keys, and settings section by section. Everything runs as a single application with one SQLite file and a dashboard localized into 34 languages. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. MIT licensed.

Give every team, project, and application one governed door to AI models instead of scattering raw provider keys across codebases. TokenHub is an OpenAI-compatible gateway that serves chat completions, responses, embeddings, reranking, and image generation, plus Anthropic Messages endpoints, in front of OpenAI, Azure OpenAI, Anthropic, Gemini, DeepSeek, Qwen, Codex subscriptions, local models, and more than 150 provider templates. Administrators publish a model catalog and write routing policies with priority, weight, failover order, and cost or quality preferences, so a provider outage never reaches client code. Users create project-scoped API keys with quotas and concurrency limits, team leaders manage project spaces and members, and each role gets a workspace that matches its job. Usage analytics and request logs attribute every call to a user, project, team, model, and cost center, and provider reconciliation compares internal usage with upstream bills. Sign in through OAuth or OIDC, control access with RBAC, and keep an audit trail of administrative changes. The console switches between English, Chinese, Japanese, and Russian, and includes an API documentation workspace for developers. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. Apache-2.0 licensed.

Backed by 56,000+ GitHub stars and over 240 million Docker pulls, LiteLLM delivers the open-source AI gateway trusted by Netflix, Lemonade, Rocket Money, and thousands of engineering teams to route every LLM request through one unified API. The Rust-core gateway adds sub-millisecond overhead per request with 8ms P95 latency at 1,000 RPS, 15x throughput improvement and 11x lower memory footprint compared to Python-only proxies. A single OpenAI-compatible endpoint connects to 100+ providers and 1,800+ models spanning OpenAI, Anthropic, Google Gemini, AWS Bedrock, Azure OpenAI, Vertex AI, Hugging Face, vLLM, Nvidia NIM, Ollama, and Mistral with day-zero support for new model releases. The Auto Router V2 classifies request complexity across four tiers using rule-based scoring, semantic keyword matching, and adaptive Thompson sampling to route each request to the most cost-effective model without API calls or training data. Virtual API keys enable multi-tenant governance with per-team, per-user, and per-project cost tracking, budget caps with automatic fallback rerouting, and role-based access control. Built-in guardrails provide PII masking, prompt injection detection, and model-graded evaluation before requests reach providers. The Agent Gateway extends routing from model calls to agent workflows with MCP server integration. Observability integrates with Langfuse, Arize Phoenix, OpenTelemetry, and MLflow for complete request tracing. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. MIT licensed.

Instead of scattering provider credentials and vendor-specific SDK logic across dozens of microservices, engineering teams route every LLM call through GoModel, an open-source AI gateway that unifies 34 model providers behind OpenAI-compatible and Anthropic-compatible endpoints. Pointing existing OpenAI or Anthropic SDKs at the gateway unlocks chat completions, the Responses API, Anthropic Messages, embeddings, batch jobs, and realtime audio across OpenAI, Anthropic, Gemini, Bedrock, Azure, Groq, DeepSeek, Ollama, and vLLM. Platform operators publish virtual model aliases that distribute traffic via weighted round-robin or lowest-cost routing while rotating multiple API keys per provider and failing over through circuit breakers during upstream outages. Sticky session routing keeps multi-turn agent conversations on the same target to preserve warm prompt caches, while exact-match and vector semantic caching via Qdrant, pgvector, Pinecone, or Weaviate cut duplicate token costs. Versioned workflows and pre-dispatch guardrails enforce prompt injection rules, rate limits, and hard USD spend budgets across hierarchical tenant user paths before requests are dispatched. The built-in admin dashboard offers live audit logs, per-tenant cost analytics, LiteLLM key migration, an interactive request playground, Model Context Protocol tool aggregation, and OpenTelemetry and Prometheus metrics. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. MIT licensed.

Point Claude Code, Codex CLI, Gemini CLI, or any OpenAI, Anthropic, or Gemini SDK at one address and Nyro routes every request to whichever model provider you choose. The gateway translates protocols on the fly, including streaming, reasoning blocks, and tool calls, so a tool built for one API can run on a model from another. Define virtual model names, bind each to one or more providers, and pick weighted load balancing or priority failover; unhealthy targets are skipped and recover automatically. Issue API keys with limits on requests and tokens per minute and per day, and bind each key only to the models it may call. Exact and semantic caching return repeated answers instantly. A request log records provider, model, token counts, latency, and streaming speed, and usage charts break traffic down by model and provider. The Connect page generates ready-to-paste samples in Python, TypeScript, and cURL, plus one-click configuration for popular coding CLIs. Choose an embedded database for a single host or an external PostgreSQL or MySQL server for production, and run the gateway, the admin UI, or both as separate processes. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. Apache-2.0 licensed.

One OpenAI-compatible endpoint, 1,600+ models behind it: Portkey Gateway sits between your app and every major LLM provider, deciding in real time where each request should go. When a provider times out or returns errors, automatic retries with exponential backoff kick in, and fallback rules quietly move traffic to your second-choice model before users notice anything. Load balancing spreads calls across multiple API keys and providers by weight, conditional routing sends each request down a different path based on its content or metadata, and per-request timeouts cut off providers that hang. Guardrails inspect inputs and outputs as traffic passes through: block banned words, screen for sensitive content, and enforce the checks your compliance team asked for, with dozens of pre-built rules ready to attach. Text is only the start: the same API shape reaches vision models, speech-to-text, text-to-speech, image generation, and realtime websocket APIs, so multimodal features stop multiplying your integration work. Routing policies live in JSON configs you attach per request or per client, which means experiments, cost controls, and provider migrations ship as configuration changes, not redeploys. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. MIT licensed.

One OpenAI-compatible API fronts more than 250 models across 40 providers in LLM Gateway, with routing, cost analytics, and key management in a single dashboard. Existing OpenAI SDK code keeps working: change the base URL and model name to reach OpenAI, Anthropic, Google, xAI, DeepSeek, Groq, and dozens more. Every request lands in an activity log with its model, cache status, token counts, duration, and cost, and opening a request shows time to first token, throughput, and cached token usage. Spend splits by model, provider, project, and API key over 7 or 30 days, so the expensive endpoint stops hiding inside one monthly invoice. Scoped API keys carry spend caps and recurring limits, and requests stop automatically at the cap. Provider errors, timeouts, and rate limits trigger a retry on another provider for the same model before the caller hears about it. Coding agents and automation tools get their own usage views with requests, tokens, and spend per tool. Self-hosting runs the unified Docker image or a Kubernetes Helm chart on your own network. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. AGPL-3.0 licensed.

Inference Gateway puts a single OpenAI-compatible API endpoint in front of OpenAI, Anthropic, Groq, Cohere, Ollama, DeepSeek, Google, Mistral, MiniMax, Moonshot, Nvidia, and llama.cpp, so your application code never changes when you switch models or providers. The Go binary starts on port 8080 and normalizes authentication, streaming protocols, and response formats across all backends transparently. Native Model Context Protocol support auto-discovers tools from connected MCP servers and injects them into LLM requests without client-side management, enabling server-side tool execution across any provider that supports function calling. Agent-to-Agent protocol integration allows distributed agent communication through a declarative Agent Definition Language that generates production-ready Go or Rust servers from a single YAML manifest. The dedicated Kubernetes Operator manages Gateway, Agent, MCP, and Orchestrator custom resources with automatic HPA scaling, OIDC authentication, and service discovery that rebuilds MCP configurations when the discovered server set changes. Prometheus metrics and OpenTelemetry tracing provide full request-level observability across the entire inference pipeline. Middleware controls enable per-request provider selection, model routing, and fallback strategies. Official SDKs in Go, Python, TypeScript, and Rust provide typed client interfaces with streaming support. Docker Compose deployment requires only environment variables for API keys. A CNCF Sandbox applicant. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. Apache 2.0 licensed.

Every prompt your application sends is a small buying decision, and OrcaRouter Lite makes it a smart one instead of a habit. Point your app at one OpenAI-compatible endpoint on your server, and each request is shopped across the providers whose keys you hold: OpenAI, Anthropic, Google, xAI, DeepSeek, and dozens more. Send model set to auto and the router picks the cheapest model that can actually do the job, checking whether the request needs tools, vision, or JSON mode before it commits your money. Prefer a different tradeoff and you can switch the whole gateway between balanced, cheapest, fastest, and quality strategies, with quality picks driven by real benchmark scores rather than vibes. Clients that speak Anthropic's or Gemini's native protocols connect directly too, so Claude Code, the OpenAI SDK, and google-genai all land on the same routing pipeline. Streaming responses flow through untouched, and every answer reports which model really served it. The built-in dashboard shows recent requests, spend, latency percentiles, unreachable providers, and a savings tile that tells you what your traffic would have cost if you had blindly used a flagship model for everything. Your provider keys stay encrypted on your own box, your analytics never leave it, and if every local provider has a bad day, the hosted OrcaRouter can step in as a failover safety net. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. MIT licensed.

Routerly is the one front door for every AI model your apps use: point your OpenAI or Anthropic client at it, and it chooses the right model for each request, enforces your budget, and keeps the receipts. Nothing in your code changes except the base URL, and from that moment streaming chat completions and messages flow to OpenAI, Anthropic, Gemini, Mistral, Cohere, xAI or your own Ollama models behind a single token. Nine routing policies score every candidate in parallel: cheapest, fastest, healthiest, most capable, biggest context, least rate-limited, fairest, most budget left, and one that asks a small model to judge which model fits the task. When the winner errors or times out, the next candidate takes over automatically, and the recorded routing trace shows exactly why each model won or lost. Guardrails can block or log content that breaks your rules, and phone numbers, card numbers, SSNs, IBANs and emails can be scrubbed out before anything reaches a provider. Every call is logged with tokens, USD cost and latency, and hard limits per model, project and token reject overspending requests before a provider can charge for them. Projects keep separate tokens, model lists and budgets for each team or tool, role-based permissions decide who may change what, and alerts land in Slack, email, webhooks or PagerDuty when budgets or providers misbehave. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. AGPL-3.0 licensed.

BitRouter is a context-aware LLM router that learns which model delivers the cheapest successful outcome per workflow step, cutting agent costs by up to 80% while maintaining 96% quality versus all-frontier baselines. Point any agent runtime at http://localhost:4356 with a one-line OPENAI_BASE_URL change and BitRouter routes to OpenAI, Anthropic, Google, Groq, DeepSeek, Mistral, Moonshot, MiniMax, Nvidia, and any OpenAI-compatible endpoint simultaneously, normalizing authentication, streaming, and cross-protocol translation between wire formats. The act-observe-evaluate-learn loop traces every hop with cost, tokens, and latency attribution, scores each decision against a versioned policy-lock.yaml, then tightens routes automatically with no LLM judge in the path. Native MCP gateway auto-discovers tools from connected servers and makes them routable and governed alongside model calls. Agent Client Protocol integration enables the TUI to manage Claude Code, Codex, OpenCode, OpenClaw, Gemini, and Copilot sessions in real time with inline tool-call approval and live streaming. Built-in guardrails inspect, redact, or block risky content before requests leave your network. Virtual keys scope API access per agent or user without exposing upstream credentials. Per-agent spend caps and loop guards contain runaway cost automatically. Multi-account failover reroutes mid-run so rate limits never re-pay completed work. Ships as a single Rust binary via npm or Cargo. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. Apache 2.0 licensed.

Send every AI request through one gateway that holds your model providers, API keys, and access rules. The web console starts with a first-run admin setup, then an LLM Provider page where OpenAI-compatible endpoints, Anthropic Claude, Azure OpenAI, Tongyi Qianwen, Moonshot, Zhipu, and other services are added with their tokens. AI Route Config maps path prefixes and model match rules to weighted target services, with token failover and first-packet timeouts to keep traffic moving when a key fails. Consumer Management issues key, OAuth2, and JWT credentials, and each route can limit access to named consumers. Plugin Management lists ready-made AI plugins for caching, context limits, data masking, prompt templates, prompt decorators, load balancing, and intent checks, plus traffic and security plugins that can be configured per route. MCP Management hosts MCP servers behind the same authentication and rate limits so agents call tools through one governed address. Service Sources import backends from Nacos, Zookeeper, Consul, Eureka, static addresses, or domains, and domain and certificate pages handle HTTPS, including free automatic certificates from Let's Encrypt. An LLM proxy for a product team, a shared model endpoint for several agents, or a replacement for an ingress controller all fit this console. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. Apache-2.0 licensed.

Open-weight language models get a private chat service on your own server, and llama.cpp serves its browser UI the moment the server starts. Conversations appear in a sidebar with search, and each answer shows its token count, generation time, and tokens per second, so you can see how a model really performs. Code replies render in highlighted blocks with a copy button, and messages can be edited, regenerated, forked, or deleted. A Models page lists what the server can run with context sizes and load status, and Settings cover themes, system messages, display options, tools, agentic behavior, import and export, and sampling controls for temperature, top K, top P, and penalties. Remote MCP servers connect by URL, and uploads let multimodal models read files and images. Behind the UI sit OpenAI and Anthropic compatible chat endpoints, embeddings, reranking, function calling, JSON schema constrained output, parallel decoding, and continuous batching for several users at once. Quantized models run on plain CPUs, and official images cover CUDA, ROCm, Vulkan, and other accelerators. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. MIT licensed.

Xinference lets a team swap a hosted model API for models running on its own machines, with the same client code and a web console for managing them. The Launch Model page lists 300+ models across language, vision, embeddings, reranking, images, audio, video, and experimental world generation, and a Recommend configuration button fills in engine, size, format, and quantization for your hardware. Custom models, including traditional machine learning models, can be registered beside the built-in ones. Chat models can run on vLLM, SGLang, TRT-LLM, Transformers, llama.cpp, or MLX across CPUs, GPUs, and Apple Silicon. Workers and replicas can be added to serve one large model across several machines. Running Models shows each replica and a built-in chat window, and Performance Tests measure throughput and latency at several concurrency levels, with saved runs to compare. Endpoints speak the OpenAI and Anthropic protocols with streaming and tool calling where a model supports it. API keys, user management, and an audit center cover access control. The interface is available in five languages. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. Apache 2.0 licensed.

With over 48,000 GitHub stars and monthly releases since March 2023, LocalAI is the self-hosted AI engine that replaces every OpenAI endpoint with a single Docker container running on your own infrastructure — serving chat completions, image generation, text-to-speech, speech-to-text, embeddings, vision, video generation, and function calling through identical API schemas that require zero application code changes. The composable backend architecture isolates each inference engine as a separate gRPC service running in its own OCI container, so llama.cpp, vLLM, SGLang, transformers, whisper.cpp, diffusers, MLX, Stable Diffusion, and Flux install on demand without touching the core, can run on separate machines, and a fault in one never affects others. Hardware acceleration spans NVIDIA CUDA 12 and 13, AMD ROCm, Intel oneAPI/SYCL, Apple Silicon Metal, Vulkan, and NVIDIA Jetson L4T — or runs entirely on CPU without any GPU. Built-in AI agents support autonomous tool use, retrieval-augmented generation, Model Context Protocol integration, and skill-based workflows directly in the web interface. The model gallery provides curated YAML configuration files for hundreds of models that install with a single command, while P2P federated inference distributes model shards across multiple machines for running models larger than any single node's memory. Multi-user API key authentication with quotas and role-based access enables team deployments. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. MIT licensed.

GPUStack is a control plane that turns a pile of GPU servers into a model-serving platform, where you pick a model from a catalog and the system decides which machine, engine, and settings run it. Deploy language, embedding, reranker, speech, and image models from the Catalog page, then watch each replica move from downloading to Running on the Deployments screen. A built-in Playground offers chat, side-by-side model comparison, embedding and rerank testing, audio, and image generation and editing, so a model gets judged before it is wired into an app. Add GPU workers with a generated Docker command, register a Kubernetes cluster, or provision cloud machines, and manage several clusters from one server. Create API keys for teammates, apply access control, and read token, GPU-hour, and storage usage by user, key, or model. Auto Tune benchmarking searches for the highest throughput that still meets your latency targets. GPU Instances hand out SSH-accessible machines for fine-tuning and interactive work. Single sign-on works through OIDC or SAML. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. Apache 2.0 licensed.

One binary. A fleet of machines. One calm endpoint for inference. Paddler balances large language model traffic across whatever hardware you already own, instead of renting tokens by the million. The balancer exposes an inference service, a management service, and a web admin panel from a single self-contained program. Agents on separate machines generate tokens and embeddings through slots, each carrying its own context and KV cache. Inference runs on the built-in llama.cpp engine, CPU or GPU. Need a different model? Swap it dynamically, no rebuild, no redeploy. Capacity follows demand: agents join the fleet when traffic grows and leave when it fades, and request buffering holds early work in a queue so scaling up from zero hosts drops nothing. Operators watch fleet health through observability metrics, configure models, chat templates, and inference parameters in the admin panel, and test prompts graphically before users ever see a change. A desktop edition drafts office workstations into service beside the rack servers. For medical, financial, or otherwise sensitive prompts, everything stays inside your own network, and predictable hardware replaces per-token pricing exposure. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. Apache License 2.0 licensed.