Instead of scattering provider credentials and vendor-specific SDK logic across dozens of microservices, engineering teams route every LLM call through GoModel, an open-source AI gateway that unifies 34 model providers behind OpenAI-compatible and Anthropic-compatible endpoints. Pointing existing OpenAI or Anthropic SDKs at the gateway unlocks chat completions, the Responses API, Anthropic Messages, embeddings, batch jobs, and realtime audio across OpenAI, Anthropic, Gemini, Bedrock, Azure, Groq, DeepSeek, Ollama, and vLLM. Platform operators publish virtual model aliases that distribute traffic via weighted round-robin or lowest-cost routing while rotating multiple API keys per provider and failing over through circuit breakers during upstream outages. Sticky session routing keeps multi-turn agent conversations on the same target to preserve warm prompt caches, while exact-match and vector semantic caching via Qdrant, pgvector, Pinecone, or Weaviate cut duplicate token costs. Versioned workflows and pre-dispatch guardrails enforce prompt injection rules, rate limits, and hard USD spend budgets across hierarchical tenant user paths before requests are dispatched. The built-in admin dashboard offers live audit logs, per-tenant cost analytics, LiteLLM key migration, an interactive request playground, Model Context Protocol tool aggregation, and OpenTelemetry and Prometheus metrics. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. MIT licensed.

Manifest LLM Gateway is a self-hosted gateway that puts every model you pay for behind one OpenAI- and Anthropic-compatible endpoint. Connect provider API keys, reuse subscriptions like ChatGPT Plus or Claude Max, or point it at Ollama and LM Studio on your own hardware. Send a request with the model set to auto and your routing rules choose the target by model, header, complexity, or task type, with fallbacks that take over the instant a model fails. Each agent or app gets its own harness, a named connection with a private key, spending limits, and notifications when usage climbs. The Requests view logs every call and every retry attempt, with status, cost, tokens, and the full request and response body readable as a conversation. Autofix repairs a malformed request, for example one using a renamed parameter, and resends it before your app ever sees an error. A Playground tests models from the dashboard, a command line tool and MCP server handle setup from a terminal or an AI assistant, and an n8n node plugs it into automations. Over 35 provider connections cover more than 300 models. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. MIT licensed.

One endpoint, 130+ models from 90+ providers: KeiRouter is a self-hosted AI gateway, so apps, coding tools, and agents connect once and stop juggling provider keys. Your code speaks OpenAI Chat, OpenAI Responses, Anthropic Messages, or native Gemini, and KeiRouter translates to whichever upstream you want, including AWS Bedrock, Azure AI Foundry, Vertex, Snowflake Cortex, and self-hosted runtimes like Ollama and vLLM. Fallback chains keep requests moving when a provider goes down or hits a rate limit, using priority, round-robin, latency, or cost ordering, so coding sessions do not stop mid-task. Five token savers compress prompts on the way in and trim responses on the way out, with a live dashboard showing savings recovered. A response cache answers repeated prompts instantly for zero marginal cost, with optional semantic matching. Per-key budgets and rate plans set USD, token, RPM, and concurrency limits with auto-cutoff, stopping surprise bills. Guardrails scan for PII, prompt injection, toxicity, and banned topics at global, provider, model, chain, and key scopes, with a monitor showing what got blocked and what it cost. The dashboard tracks usage by provider and model, latency, a live console tracing each request through its fallbacks, and provider health. You can import from 9router or OmniRoute, white-label the portal for your team, and run replicas on Postgres and Redis for high availability. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. MIT licensed.

One gateway, every AI model: APIPark gives your applications a single, consistent API for OpenAI, Anthropic, Google Gemini, DeepSeek, and 100+ other models, and gives your team a portal to publish and share whatever gets built on top. Standardized request and response formats mean swapping a provider or rewriting a prompt never touches client code. Pair a model with a prompt template and it becomes a ready-made API for sentiment analysis or translation, listed in the developer portal with its own readme and generated API docs. Model Fallback swaps in the highest-priority available model when one misbehaves. API key balancing spreads traffic across several keys for the same provider. Already have REST services? Import an OpenAPI document, file the service under a category and tags, and let consumers subscribe, with optional manual review before anyone gets access. Any service can be switched on as an MCP tool that AI assistants can discover and call, and local open-source models can be deployed and exposed the same way. Usage charts follow requests, traffic, and tokens per service and consumer. Request logs flow out to files, HTTP endpoints, Kafka, NSQ, or Syslog. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. Apache-2.0 licensed.

One endpoint for every model account you own: 9Router Pro sits between your applications and a fleet of LLM providers, decides which account answers each request, and keeps the whole operation visible on a live dashboard. Its signature trick is prompt injection scoped three ways, globally, per API key, or per model, delivered either as a system prompt or as a scripted chat history that primes the model far harder than instructions alone. Keys are products here: each carries a rolling token budget over 24 hours, 7 days, or 30 days plus rate limits per minute, hour, and day, so a resold plan enforces itself with polite 429 responses. Routing stays healthy on its own, with accounts tiered as clean, suspect, or cooldown, least-recently-used spreading across the fleet, flaky upstream responses retried against the next account, and proxy pools health-checked with one-click cleanup of the dead. The monitoring dashboard leads with VPS specs, fastest and most-used providers, top API key, and usage spectrum charts, while backup and restore cover providers, combos, keys, and settings section by section. Everything runs as a single application with one SQLite file and a dashboard localized into 34 languages. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. MIT licensed.

One endpoint, every model you run, and a government-grade paper trail for each request. OpenGateLLM sits in front of your AI infrastructure as the single door: applications call one OpenAI-compatible API, and the gateway decides which backend answers, a self-hosted vLLM cluster, an Ollama box in the corner, a Hugging Face TEI embeddings service, or a hosted provider when you allow one. You publish friendly model names as routers and attach real provider APIs behind them, then load-balance across those backends by shuffle or least-busy so a second GPU node starts earning its keep the moment you register it. Chat completions, embeddings, reranking, OCR, and audio transcription each get their standard endpoint, so existing OpenAI SDK code, LangChain, and LlamaIndex projects point at a new base URL and keep working. Governance is built in rather than bolted on: organizations, users, and roles control who may call which model, API keys authenticate every request, and rate limiting keeps one enthusiastic team from eating the whole cluster. Usage monitoring follows every inference, and carbon footprint estimates attach to models through their hosting zone and parameter counts. Retrieval-augmented generation against Elasticsearch or Qdrant plugs into the same surface. No chat history is stored, which is rather the point when the data is yours. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. MIT licensed.

One OpenAI-compatible endpoint, 1,600+ models behind it: Portkey Gateway sits between your app and every major LLM provider, deciding in real time where each request should go. When a provider times out or returns errors, automatic retries with exponential backoff kick in, and fallback rules quietly move traffic to your second-choice model before users notice anything. Load balancing spreads calls across multiple API keys and providers by weight, conditional routing sends each request down a different path based on its content or metadata, and per-request timeouts cut off providers that hang. Guardrails inspect inputs and outputs as traffic passes through: block banned words, screen for sensitive content, and enforce the checks your compliance team asked for, with dozens of pre-built rules ready to attach. Text is only the start: the same API shape reaches vision models, speech-to-text, text-to-speech, image generation, and realtime websocket APIs, so multimodal features stop multiplying your integration work. Routing policies live in JSON configs you attach per request or per client, which means experiments, cost controls, and provider migrations ship as configuration changes, not redeploys. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. MIT licensed.

Your apps keep using the OpenAI SDK while Antirion, a self-hosted AI gateway, decides which provider answers, what each team may spend, and what gets logged. Clients speak the OpenAI, Anthropic, Gemini, or Bedrock APIs, and Antirion translates between them, so a Claude deployment can answer an OpenAI-style call, streams included. Add providers such as Amazon Bedrock, Vertex AI, Mistral, Groq, DeepSeek, or a vLLM server, then publish them as named Models. Routing rules send traffic by team or project with fallback chains, weighted splits, least-cost or least-latency targets, retries, hedging, and circuit breakers. Budgets and rate limits in dollars or tokens apply per organization, team, project, and key, with projections and alerts before money runs out. Guardrails check for personal data, secrets, keywords, and regex matches, then block, redact, flag, or divert. Exact and semantic caching answers repeated prompts without calling a provider. The request log records every attempt, route, token count, cost, and guardrail verdict, with Prometheus metrics and OTLP traces. An MCP gateway and A2A proxy put tools and agents behind the same keys. Sign in with OIDC, SAML, or GitHub, provision users with SCIM, and review every change in the audit log. Changes you choose can require a change request or a second person's approval, and the Playground sends requests from the console. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. Apache-2.0 licensed.

Bifrost is an open-source AI gateway that unifies 23+ LLM providers into a single OpenAI-compatible endpoint with automatic failover, semantic caching, and built-in cost governance, so one provider going down never takes your production AI application with it. Point your existing OpenAI or Anthropic SDK at Bifrost's local endpoint and gain access to OpenAI, Anthropic, AWS Bedrock, Google Vertex, Azure, Groq, Mistral, and Ollama without changing application code. Define fallback chains that automatically switch providers when one returns errors or exceeds latency thresholds, keeping response times stable during outages. The built-in web dashboard at port 8080 lets you configure providers, create virtual API keys, monitor live request traffic, and review analytics without editing configuration files. Semantic caching combines exact hash matching with vector similarity search via Weaviate, serving cached responses for identical or paraphrased prompts in sub-millisecond time to cut costs on repetitive workloads. The MCP gateway connects AI agents to external tools like filesystems, databases, and web APIs, exposing them to clients such as Claude Desktop and Cursor with per-key allow-lists. Four-tier budget hierarchy at customer, team, virtual key, and provider levels enforces spend caps, rate limits, and model restrictions across your organization. Extend functionality through custom Go plugins for analytics, monitoring, or security middleware. Native Prometheus metrics and OpenTelemetry distributed tracing give operations teams full production observability. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. Apache 2.0 licensed.

Routerly is the one front door for every AI model your apps use: point your OpenAI or Anthropic client at it, and it chooses the right model for each request, enforces your budget, and keeps the receipts. Nothing in your code changes except the base URL, and from that moment streaming chat completions and messages flow to OpenAI, Anthropic, Gemini, Mistral, Cohere, xAI or your own Ollama models behind a single token. Nine routing policies score every candidate in parallel: cheapest, fastest, healthiest, most capable, biggest context, least rate-limited, fairest, most budget left, and one that asks a small model to judge which model fits the task. When the winner errors or times out, the next candidate takes over automatically, and the recorded routing trace shows exactly why each model won or lost. Guardrails can block or log content that breaks your rules, and phone numbers, card numbers, SSNs, IBANs and emails can be scrubbed out before anything reaches a provider. Every call is logged with tokens, USD cost and latency, and hard limits per model, project and token reject overspending requests before a provider can charge for them. Projects keep separate tokens, model lists and budgets for each team or tool, role-based permissions decide who may change what, and alerts land in Slack, email, webhooks or PagerDuty when budgets or providers misbehave. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. AGPL-3.0 licensed.

Juggling separate API keys, incompatible payload schemas, and unpredictable rate limits across OpenAI, Anthropic, and Google Gemini slows down every AI workflow, and Octopus solves it by consolidating all your upstream model providers into one self-hosted API gateway. Point Claude Code, OpenAI Codex CLI, or any SDK client at a single gateway URL and translate requests transparently between OpenAI Chat Completions, OpenAI Responses, Anthropic Messages, and Google Gemini endpoints without modifying client code. Group multiple provider channels under a unified model alias so your coding agents automatically fail over to backup providers and shield active tasks from upstream HTTP errors or rate limits. Watch live end-to-end request routing on the interactive topology canvas as prompts flow from client applications through model groups to specific upstream channels. Synchronize available model catalogs directly from connected providers and pull up-to-date token pricing from models.dev, or define custom per-million input, output, and cache-read rates for private deployments. Inspect detailed request logs, monitor token consumption trends, and audit daily API spend across every channel from the web dashboard. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. AGPL-3.0 licensed.

Backed by 56,000+ GitHub stars and over 240 million Docker pulls, LiteLLM delivers the open-source AI gateway trusted by Netflix, Lemonade, Rocket Money, and thousands of engineering teams to route every LLM request through one unified API. The Rust-core gateway adds sub-millisecond overhead per request with 8ms P95 latency at 1,000 RPS, 15x throughput improvement and 11x lower memory footprint compared to Python-only proxies. A single OpenAI-compatible endpoint connects to 100+ providers and 1,800+ models spanning OpenAI, Anthropic, Google Gemini, AWS Bedrock, Azure OpenAI, Vertex AI, Hugging Face, vLLM, Nvidia NIM, Ollama, and Mistral with day-zero support for new model releases. The Auto Router V2 classifies request complexity across four tiers using rule-based scoring, semantic keyword matching, and adaptive Thompson sampling to route each request to the most cost-effective model without API calls or training data. Virtual API keys enable multi-tenant governance with per-team, per-user, and per-project cost tracking, budget caps with automatic fallback rerouting, and role-based access control. Built-in guardrails provide PII masking, prompt injection detection, and model-graded evaluation before requests reach providers. The Agent Gateway extends routing from model calls to agent workflows with MCP server integration. Observability integrates with Langfuse, Arize Phoenix, OpenTelemetry, and MLflow for complete request tracing. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. MIT licensed.

Respan Gateway is a single-binary LLM gateway that lets the SDK you already use call any AI model from any provider, translating between OpenAI, Anthropic, and Gemini API formats on the way through. Point an OpenAI, Anthropic, or Gemini client at it and you reach chat models on sixteen providers, plus any OpenAI-compatible endpoint of your own, with streaming, tools, images, and reasoning translated. Traffic control works the way operators think: name a route, send it to the cheapest or fastest provider, retry on failure, fall back to a second vendor, or hedge a slow call by racing another provider. Models that never had tools get them anyway, because MCP servers and built-in search, browsing, code, document, and memory tools run inside ordinary chat requests behind the gateway. Realtime voice crosses vendors too, with an OpenAI Realtime client able to talk to Gemini Live, alongside images, speech, transcription, embeddings, and video through one endpoint. A live request log in the web UI shows the client key, model, provider, status, tokens, cost, and latency of every call, so runaway agents and surprise bills are visible immediately. Everything is a plugin on one public Rust API, from routing and pricing to tracing, with examples for guardrails, PII masking, and billing. One TOML file configures keys, providers, routes, and tools, validates before it starts, and reloads on signal. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. Apache-2.0 licensed.

GPT Load has become the go-to self-hosted AI gateway for teams managing multiple LLM provider credentials behind a single URL. The Go-built proxy transparently preserves native API formats for OpenAI Chat Completions, OpenAI Responses, Anthropic Messages, and Google Gemini — applications swap one base URL and keep their existing SDK integration untouched. Twenty built-in channels cover official APIs (OpenAI, Anthropic, Gemini, xAI), cloud platforms (Azure OpenAI, AWS Bedrock, Google Vertex AI), model services (DeepSeek, Moonshot AI, SiliconFlow, Zhipu AI, Alibaba Cloud, Volcengine, OpenRouter, Groq), and subscription accounts (Codex, Claude, Antigravity, Grok) using the same credential management, scheduling, and health system. The intelligent key pool rotates among valid credentials using atomic counters for fair high-concurrency distribution, automatically blacklists failing keys after configurable thresholds, and recovers them via scheduled health checks — all transparent to clients. Weighted load balancing distributes traffic across multiple upstream endpoints while per-credential proxies route individual keys through different egress paths. The Vue 3 management dashboard provides real-time statistics, group configuration, key testing, request logs with full debugging context, per-model cost estimates, and route inspection. Docker deployment exposes the management interface and proxy on port 3001 with SQLite by default, MySQL or PostgreSQL optional, and Redis for distributed leader-follower cluster scaling. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. MIT licensed.

Every employee gets one AI key, and every dollar of model spending gets a name next to it. AIHelms is an admin console and user portal that sits in front of your model providers, so a team can use Claude, DeepSeek, GPT, and local models through one entry point, with IT keeping control. Administrators onboard providers, publish models, and balance traffic across several deployments of the same model, with both OpenAI and Anthropic request formats accepted. Each AI identity carries a model allow-list, quota, rate limits, and an expiry date. Budgets can be set per person, department, project, or model, with soft alerts and hard cut-offs. Internal settlement prices sit beside real provider costs, so departments can be billed with or without a markup. Dashboards show adoption, daily calls, token use, and top users, and weekly or monthly efficiency reports drill down by department. A built-in marketplace registers, approves, and distributes MCP servers and agent Skills, and an AI policy scan flags risky Skills before anyone installs them. Audit logs record every administrator action. Employees see their key, endpoint, models, and monthly spend in the portal. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. GPL-3.0 licensed.

New API is a Go-powered LLM gateway that funnels over 40 AI providers, from OpenAI and Anthropic to Google Gemini, Azure, AWS Bedrock, DeepSeek, and Mistral, through a single OpenAI-compatible endpoint with intelligent routing and enterprise cost controls. Bidirectional format conversion translates between OpenAI Chat Completions, Claude Messages, and Gemini GenerateContent APIs transparently, so clients call any provider using their preferred format while the gateway handles the rest. Routing logic supports weighted random channel selection, priority-based failover, and automatic retry on provider errors to maximize uptime. The React admin dashboard shows usage charts, token consumption tracking, per-model cost breakdowns, and real-time request monitoring. Multi-tenant architecture includes three role levels, token-based authentication, per-user quota management, and a three-phase billing system with tiered pricing via a custom expression language. Multimodal support covers text completion, vision, embeddings, text-to-speech, speech-to-text, image generation, and creative task providers like Midjourney-Proxy and Suno-API. The interface ships in five languages including English, Chinese, Japanese, and French. Docker deployment runs with SQLite or MySQL for persistence. 45,000+ GitHub stars. AGPL-3.0 licensed.

Inference Gateway puts a single OpenAI-compatible API endpoint in front of OpenAI, Anthropic, Groq, Cohere, Ollama, DeepSeek, Google, Mistral, MiniMax, Moonshot, Nvidia, and llama.cpp, so your application code never changes when you switch models or providers. The Go binary starts on port 8080 and normalizes authentication, streaming protocols, and response formats across all backends transparently. Native Model Context Protocol support auto-discovers tools from connected MCP servers and injects them into LLM requests without client-side management, enabling server-side tool execution across any provider that supports function calling. Agent-to-Agent protocol integration allows distributed agent communication through a declarative Agent Definition Language that generates production-ready Go or Rust servers from a single YAML manifest. The dedicated Kubernetes Operator manages Gateway, Agent, MCP, and Orchestrator custom resources with automatic HPA scaling, OIDC authentication, and service discovery that rebuilds MCP configurations when the discovered server set changes. Prometheus metrics and OpenTelemetry tracing provide full request-level observability across the entire inference pipeline. Middleware controls enable per-request provider selection, model routing, and fallback strategies. Official SDKs in Go, Python, TypeScript, and Rust provide typed client interfaces with streaming support. Docker Compose deployment requires only environment variables for API keys. A CNCF Sandbox applicant. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. Apache 2.0 licensed.

Anyone who pays for Claude Code, Codex, and Gemini CLI separately ends up juggling three endpoints, three sets of keys, and three kinds of rate-limit surprise, and CliRelay collapses all of that into one self-hosted gateway with a control panel. Clients speak the OpenAI Chat or Responses, Anthropic Messages, or Gemini protocol, and CliRelay translates each request for whichever upstream account is healthy. Channel groups limit where each key may send traffic, round-robin, fill-first, and session-sticky scheduling spread it, and cooldowns with automatic failover move requests off accounts that have run out of quota. Add upstreams from the /manage console through browser sign-in, device codes, or bulk credential import, covering Claude, Codex, Gemini, Antigravity, Vertex AI, AWS Bedrock, Grok, Qwen, Kimi, and any OpenAI-compatible service. Portal accounts own client keys with daily, period, and lifetime quotas, RPM and TPM limits, and per-model pricing. A monitor center scores health and charts live traffic, P50 to P99 latency, and time to first token, while every request lands in a searchable log with tokens, latency, and cost. Tenants, roles with resource.action permissions, content moderation profiles, an outbound proxy pool, and an audit trail make shared deployments workable, and the console speaks English, Simplified Chinese, and Russian. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. MIT licensed.

One OpenAI-compatible API fronts more than 250 models across 40 providers in LLM Gateway, with routing, cost analytics, and key management in a single dashboard. Existing OpenAI SDK code keeps working: change the base URL and model name to reach OpenAI, Anthropic, Google, xAI, DeepSeek, Groq, and dozens more. Every request lands in an activity log with its model, cache status, token counts, duration, and cost, and opening a request shows time to first token, throughput, and cached token usage. Spend splits by model, provider, project, and API key over 7 or 30 days, so the expensive endpoint stops hiding inside one monthly invoice. Scoped API keys carry spend caps and recurring limits, and requests stop automatically at the cap. Provider errors, timeouts, and rate limits trigger a retry on another provider for the same model before the caller hears about it. Coding agents and automation tools get their own usage views with requests, tokens, and spend per tool. Self-hosting runs the unified Docker image or a Kubernetes Helm chart on your own network. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. AGPL-3.0 licensed.

An upstream dies mid-stream. The session does not have to. ccLoad is a self-hosted AI gateway that stands between your coding agents and every key, relay, and subscription you own, retrying the next source before the first byte reaches your client. Claude Code, Codex, Gemini, and OpenAI-compatible tools all point at one address and present one ccLoad token. Behind it, each source becomes a channel with a model list, a priority, and health-based ordering. Requests are matched by the model name the client asked for. Protocol differences disappear in transit, with Anthropic, OpenAI, Codex Responses, and Gemini converted in either direction, streamed responses included. Failure gets a narrow blast radius. A revoked key cools down alone, a failing model cools down alone, and a dead URL cools down alone, each with exponential backoff that respects upstream reset times. Errors disguised as successful responses still trigger failover. Subscription accounts work as channels beside API keys, tokens refresh on their own, and quota use is tracked against each provider's reset window. Every request lands in the logs with its cost, priced with cache reads, cache writes, and long-context tiers included. Tokens carry model restrictions, budgets, and concurrency caps. Channels carry daily spend limits. Sensitive text can be masked on the way out and restored on the way back. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. MIT licensed.