Large language models get a polished front end that can run fully offline: Open WebUI is the self-hosted front end of choice. It talks to local model runners, primarily Ollama, and to any OpenAI-compatible API, so LM Studio, vLLM, Groq, Mistral, OpenRouter, and cloud providers all plug into the same chat interface and can be mixed per conversation. RAG is built in: upload files to knowledge bases or reference them in chat with the # command, backed by a choice of nine vector databases (ChromaDB and PGVector officially maintained) and multiple extraction engines including Tika and Docling, with hybrid BM25-plus-vector search and cross-encoder reranking. Web search results from providers like SearXNG, Brave, and Tavily inject directly into conversations. Extensibility comes from Python tools and functions that run inside the chat, a Pipelines plugin framework, and native MCP support. Multi-user features include RBAC, SSO, and group permissions, and the instance itself exposes an OpenAI-compatible API your own apps can call.

With over 48,000 GitHub stars and monthly releases since March 2023, LocalAI is the self-hosted AI engine that replaces every OpenAI endpoint with a single Docker container running on your own infrastructure — serving chat completions, image generation, text-to-speech, speech-to-text, embeddings, vision, video generation, and function calling through identical API schemas that require zero application code changes. The composable backend architecture isolates each inference engine as a separate gRPC service running in its own OCI container, so llama.cpp, vLLM, SGLang, transformers, whisper.cpp, diffusers, MLX, Stable Diffusion, and Flux install on demand without touching the core, can run on separate machines, and a fault in one never affects others. Hardware acceleration spans NVIDIA CUDA 12 and 13, AMD ROCm, Intel oneAPI/SYCL, Apple Silicon Metal, Vulkan, and NVIDIA Jetson L4T — or runs entirely on CPU without any GPU. Built-in AI agents support autonomous tool use, retrieval-augmented generation, Model Context Protocol integration, and skill-based workflows directly in the web interface. The model gallery provides curated YAML configuration files for hundreds of models that install with a single command, while P2P federated inference distributes model shards across multiple machines for running models larger than any single node's memory. Multi-user API key authentication with quotas and role-based access enables team deployments. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. MIT licensed.

OpenLLM serves any large language model as an OpenAI-compatible API endpoint from a single CLI command, handling model download, backend selection, quantization, and port binding automatically. It supports the full spectrum of popular models including Llama 3.3, Qwen2.5, DeepSeek, Mistral, and Phi3, choosing between vLLM and PyTorch inference backends based on hardware capabilities. When vLLM is available, continuous batching with PagedAttention achieves up to 23x throughput improvement over naive serving, while GPTQ and bitsandbytes quantization reduces memory requirements for GPU-constrained deployments. The server exposes a RESTful API on port 3000 with full OpenAI client library compatibility, enabling drop-in replacement for commercial providers in any application using the standard chat completions format. A built-in web chat UI at the /chat endpoint provides immediate interactive testing without external clients. Custom model repositories allow teams to maintain private catalogs of fine-tuned models alongside the default repository that tracks the latest releases. Deployment workflows generate production-ready Docker images automatically, with Kubernetes manifest support for orchestrated scaling. Native integration with LangChain and LlamaIndex supports RAG pipelines, Transformers Agents enables tool-calling workflows, and HuggingFace Hub handles model discovery. Server-Sent Events enable real-time token streaming across all API endpoints. Backed by BentoML's production ML infrastructure. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. Apache 2.0 licensed.

Imp runs large AI chat models privately on one NVIDIA RTX 50 or RTX PRO graphics card, exposing them through OpenAI and Anthropic compatible APIs plus a built-in browser chat. Start it with a single Docker command, point it at a model, and the chat opens in your browser with saved conversations and a live readout of generation speed. Coding agents such as Claude Code and Codex connect by changing one base URL, so your editor talks to the model on your own hardware instead of a rented cloud endpoint. Tool calls, structured JSON output, streaming responses, and reasoning models all work over the wire, and requests can be protected with an API key, per-IP rate limits, and concurrency caps. Load GGUF files you already have or NVFP4 and FP8 SafeTensors checkpoints straight from Hugging Face with no conversion step before the first answer. When a model is bigger than the card, mixture-of-experts weights stay in system RAM while the busiest experts are cached on the GPU, so a 123.6 GiB model still answers from a 32 GB card. Supported models can also read images alongside text, and tokenize, health, and model listing endpoints make the server easy to wire into scripts and monitors. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. MIT licensed.

Self-hosting a language model is only half the job; people still need somewhere pleasant to talk to it, and Chat UI is that window, the same interface that powers HuggingChat. Point it at any OpenAI-compatible endpoint, your own Ollama or llama.cpp server, the Hugging Face router, or a hosted gateway like OpenRouter, and the models it serves appear in the picker automatically, with no per-model configuration. Conversations are saved to your database and stay browsable in the sidebar, so a thread from last week can be reopened, continued, and compared against a fresh answer from another model. Multimodal models accept image uploads right in the composer, and models that support it can call MCP tools mid-conversation to fetch data or run actions. For a shared team deployment, OpenID Connect sign-in keeps individual histories and settings separate, and per-user defaults travel with each account. Everything a consumer chatbot would keep on someone else's servers, prompts, uploaded files, and usage history, stays in the MongoDB you control on the same machine. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. Apache-2.0 licensed.

A private ChatGPT built with Next.js: Lobe Chat is the open-source AI chat interface teams self-host instead. Its main advantage is provider breadth: one interface connects to 40+ model providers, including OpenAI, Anthropic Claude, Google Gemini, Mistral, Groq, AWS Bedrock, Azure, and local models served through Ollama, so you can switch models per conversation and compare outputs. It handles multi-modal work: image recognition, image generation, text-to-speech, and speech-to-text. A plugin system based on function calling and the Model Context Protocol (MCP) adds external tools like web search and code execution. Run it in standalone mode as a single container with settings in browser storage, or in database mode with PostgreSQL and S3-compatible storage for persistent history, multi-user auth, and RAG knowledge bases built from uploaded documents with pgvector retrieval. Because tools arrive through function calling and MCP rather than a proprietary plugin format, custom internal tools can be exposed to the assistant with a standard server over STDIO or HTTP. Hundreds of pre-configured assistant roles import from the community marketplace. For teams the cost model matters: provider API keys billed per token typically undercut a ChatGPT Plus seat per person, and self-hosting keeps API keys, uploaded files, embeddings, and conversation history entirely on your own server.

Every major model provider behind one ChatGPT-style interface: LibreChat spans OpenAI, Anthropic, Google, Azure, AWS Bedrock, Vertex AI, Groq, Mistral, OpenRouter, DeepSeek, and any OpenAI-compatible endpoint including local Ollama. You can switch models mid-conversation and compare providers without changing tools. Its Agents framework builds no-code custom assistants with tool access via Model Context Protocol servers, file search over uploaded documents through an optional pgvector-backed RAG service, and a sandboxed Code Interpreter that executes Python, JavaScript, Go, C++, Java, PHP, and Rust. Artifacts render React components, HTML, and Mermaid diagrams directly in chat, and image generation works through DALL-E and other configured providers. Multi-user support is enterprise-grade, with OAuth, SAML, LDAP, and two-factor authentication, per-user conversation history in MongoDB, and Meilisearch-powered search across all messages and files, plus reusable presets, forkable threads, and persistent memory across conversations. The economics favor teams: instead of a ChatGPT Plus seat per person, everyone shares one instance billed per API token, with access to every provider rather than one - and providers see individual API calls, not your accumulated organizational knowledge. Deployment is Docker Compose; API keys and endpoints are configured through .env and librechat.yaml.

big-AGI is an open-source generative AI workspace that provides a unified, local-first interface for orchestrating multi-model reasoning, automated code execution, and custom persona workflows across private infrastructure. Users query multiple large language models simultaneously through the Beam scatter-gather engine, which prompts independent AI systems in parallel, compares candidate completions side by side, and merges optimal passages into a single refined response. Knowledge workers assemble tailored AI personas equipped with specialized system instructions, custom temperature settings, and predefined document context to handle domain-specific tasks ranging from architectural design reviews to legal contract analysis. The application renders rich multimedia outputs including interactive Mermaid sequence diagrams, LaTeX mathematical formulas, syntax-highlighted code blocks with live execution previews, and AI-generated image generation canvases. Teams integrate local inference servers like Ollama and LocalAI alongside commercial API endpoints to route confidential datasets strictly through internal networks while monitoring per-prompt token usage and operational latency. Users attach complex PDF documents, spreadsheets, and source code repositories for automatic parsing and semantic retrieval, while local-first storage engines ensure private chat transcripts and custom presets remain encrypted on host drives. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. MIT licensed.

Thirteen-plus LLM providers, one unified client: NextChat (formerly ChatGPT-Next-Web) is an open-source AI chat interface built on Next.js that spans OpenAI GPT-4, Anthropic Claude, Google Gemini, DeepSeek, Groq, Azure endpoints, and self-hosted backends like Ollama, LocalAI, and RWKV-Runner. Its defining trait is minimalism - the first screen loads in about 100 KB, the desktop client is roughly 5 MB, and there is no database or user system to operate; chat history lives locally in the browser with optional WebDAV or UpStash Redis sync. The Mask system saves reusable prompt-template personas you can share and debug, long conversations auto-compress to fit context windows, and Markdown rendering covers LaTeX, Mermaid diagrams, and code highlighting with streaming responses. Plugins add web search and calculators, MCP support enables external tool calling, and Artifacts previews generated content in a separate pane. Ships as a web app, Docker image, and Tauri desktop builds for Windows, macOS, and Linux, translated into 20+ languages. MIT-licensed.

Bring your own API keys and work with OpenAI GPT models, Anthropic Claude, Google Gemini, Mistral, DeepSeek, Grok, Azure endpoints, and local models in one organized workspace: TypingMind is a unified chat frontend for large language models, replacing a browser tab per provider. Parallel chat sends the same prompt to multiple models and compares answers side by side, and models can be switched mid-conversation. A prompt library stores reusable, tagged prompts with variables, and the AI Agents system builds specialized assistants that bundle a base model, custom instructions, assigned plugins, and uploaded knowledge files for RAG. Plugins extend every connected model with web search, image generation (DALL-E, Stable Diffusion), Deep Research, URL reading via Firecrawl, and Zapier automation - plus MCP server integrations for Notion, Atlassian, and other external tools, and a JavaScript extension API for custom behavior. Chats store locally by default with optional sync. Self-hosting puts the interface on your own domain and, for teams, adds branding, member access limits, and shared prompt and agent libraries.

One clean interface in front of Anthropic, OpenAI, Google Gemini, Cohere, and more: Chat Chat is a Next.js front door to the major AI providers, ending the juggling of separate subscriptions, tabs, and UIs per model. Bring your own API keys, pick a provider and model per conversation, and switch between them as the task demands: Claude for long-form reasoning, GPT for code, Gemini for multimodal work - the interface stays identical. Beyond configured presets, custom providers plug in with their own API endpoints and keys, which covers OpenAI-compatible gateways and local inference servers. The design splits into two dedicated modes: a chat interface for conversational work with customizable system prompts, and a search interface that pairs AI processing with query handling for research-style questions. The stack is modern and hackable - Next.js 14, Tailwind CSS, shadcn/ui on Radix primitives, Jotai for state - with full internationalization including English, Chinese, and Japanese. Self-hosting means your conversation history and API keys live on your instance rather than a third-party wrapper service, and pay-per-token API pricing typically beats stacking multiple monthly chat subscriptions. AGPL-licensed and deliberately simple to deploy: one container, environment variables for keys, done.

Why should your chat history live on someone else's servers? Chatpad AI - a React/TypeScript front end for the OpenAI API, built on the Mantine component library - is designed around that question. Enter your own OpenAI API key and start chatting with GPT models; every conversation, prompt, and setting is stored locally in your browser via DexieJS over IndexedDB, with no tracking, no cookies, and no backend database at all. That architecture is the point - the Docker image is just Nginx serving static files, making it one of the lightest AI deployments in the catalog, and pay-per-token API pricing typically undercuts a ChatGPT Plus subscription for moderate use. The interface earns its "premium quality" tagline with the details: a persona selector that switches communication styles per conversation, a saved-prompts library for messages you reuse constantly, organized chat history, and full data export/import so conversations move between browsers or into backups as files you control. A JSON config file customizes defaults - models, API endpoints, UI options - without rebuilding the image. AGPL-licensed, with desktop builds available upstream. For teams that want ChatGPT's utility with a self-hosted, zero-telemetry footprint, Chatpad is the minimal, sane answer.

Ollaya serves decision models that answer typed questions about a message, ticket, or document with calibrated probabilities, through a self-hosted API shaped like the hosted service it can replace. Send a state, which can be an email, a support ticket, or any JSON, plus typed choice, score, and yes or no questions, and the model answers in one forward pass in milliseconds without generating text. Pull models by name the way Ollama users already know, with run, list, ps, show, copy, and remove commands managing the local store. Downloads verify every layer against its sha256 digest, resume when interrupted, and share blobs between models so storage stays lean. Existing TypeSafe code keeps working by changing one environment variable, because the /v1/systemone and /v1/decisions endpoints are wire-identical and the official SDK sends its key the same way. The native /api adds routing information and timings, model loading and unloading, and streamed pull progress for your own tooling. Put it to work triaging inboxes, routing requests to the right model or team, screening prompts for injection and jailbreak attempts, or replacing per-token LLM classification calls with a local answer. Agents reach the same engine through the bundled MCP server, and LiteLLM, LangChain, DSPy, LlamaIndex, n8n, and Dify connect through their existing integrations. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. Apache-2.0 licensed.

Backed by 56,000+ GitHub stars and over 240 million Docker pulls, LiteLLM delivers the open-source AI gateway trusted by Netflix, Lemonade, Rocket Money, and thousands of engineering teams to route every LLM request through one unified API. The Rust-core gateway adds sub-millisecond overhead per request with 8ms P95 latency at 1,000 RPS, 15x throughput improvement and 11x lower memory footprint compared to Python-only proxies. A single OpenAI-compatible endpoint connects to 100+ providers and 1,800+ models spanning OpenAI, Anthropic, Google Gemini, AWS Bedrock, Azure OpenAI, Vertex AI, Hugging Face, vLLM, Nvidia NIM, Ollama, and Mistral with day-zero support for new model releases. The Auto Router V2 classifies request complexity across four tiers using rule-based scoring, semantic keyword matching, and adaptive Thompson sampling to route each request to the most cost-effective model without API calls or training data. Virtual API keys enable multi-tenant governance with per-team, per-user, and per-project cost tracking, budget caps with automatic fallback rerouting, and role-based access control. Built-in guardrails provide PII masking, prompt injection detection, and model-graded evaluation before requests reach providers. The Agent Gateway extends routing from model calls to agent workflows with MCP server integration. Observability integrates with Langfuse, Arize Phoenix, OpenTelemetry, and MLflow for complete request tracing. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. MIT licensed.

Drag nodes onto a canvas and ship an LLM app: Flowise is an open-source visual builder for AI agents and LLM applications, written in Node.js on LangChain.js and licensed Apache-2.0. You assemble flows by dragging nodes onto a canvas: models, prompts, memory, vector stores, retrievers, and tools, then wire them together and test in the built-in chat panel. Three builder types cover increasing complexity: Assistant for simple RAG chat over uploaded files, Chatflow for single-agent systems with techniques like rerankers and Graph RAG, and Agentflow for multi-agent orchestration with branching, looping, shared flow state, and human-in-the-loop checkpoints. Over 100 integrations connect data sources, vector databases, and both proprietary and open-source models, plus MCP client and server nodes for standard tool interop. Finished flows are exposed as REST APIs, embedded chat widgets, or via JS and Python SDKs - each flow gets an endpoint the moment it is saved, removing the deployment gap between a working prototype and something your application can call. Execution logs, visual step debugging, and external log streaming trace behavior, while input moderation and rate limiting act as guardrails; RBAC, SSO, and workspaces cover team deployments. Self-hosting keeps prompts, encrypted credentials, and conversation data on your own instance, which matters when flows handle internal documents or customer data - and wiring a model, prompt, memory, and vector store on the canvas replaces the boilerplate a hand-coded LangChain project would need.

Langflow turns LLM application development into a visual canvas where every node maps to a real LangChain primitive (chains, agents, retrievers, memory, vector stores, and tools) that teams wire together without writing integration boilerplate. The platform supports 15+ LLM provider components including OpenAI, Anthropic, Google Gemini, Mistral, Groq, Cohere, Azure OpenAI, HuggingFace, and Ollama for fully local inference, with LiteLLM proxying to over 100 additional providers through a single OpenAI-compatible endpoint. Vector database integrations cover Pinecone, Weaviate, Chroma, Qdrant, Astra DB, OpenSearch, FAISS, and Milvus, while built-in Knowledge Bases introduced in version 1.8 allow RAG pipelines without any external vector service. Multi-agent orchestration enables agent-to-agent communication with conversation management, persistent Memory Bases for cross-session context retrieval, and step-by-step reasoning visibility in the interactive Playground. Every flow automatically becomes a callable REST API endpoint via the /run route and an MCP server exposable to Claude Desktop, Cursor, or any MCP-compatible client. Every component is a real Python class that developers can customize, extend, or replace, while Extension Bundles package third-party integrations as independent pip packages for modular installation. Tool integrations include web search, Slack, Gmail, Google Drive, GitHub, and custom REST API calls. Docker deployment with PostgreSQL persistence runs on port 7860. Backed by 153,000+ stars and DataStax. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. MIT licensed.