Deploy Now

Stars

771

Forks

75

Watchers

1

Developer links

Overmind

When an agent fails in production, Overmind scores the trace against your metrics and trains a specialized model on the evidence. Point it at the agent's repository and it maps capabilities, prompts, tools, and tasks into a graph, then matches every incoming OpenTelemetry trace to the capability that produced it and scores it on arrival. Auto-instrumentation picks up the LLM SDKs you already use, any OTel exporter can post to the API, and existing traces from Langfuse, LangSmith, Braintrust, or Galileo sync through connectors. Production traces and uploaded files become versioned datasets, with local English OCR for scanned PDFs and images, and optimisers run prompt, tool, and control-flow experiments in your repository and hand back the winner as a git diff. You fine-tune models whose weights you own, backtest them against the agent's own traces, and serve them, alongside optional OpenRouter frontier models, through one OpenAI-compatible API. Cursor, Claude Code, OpenCode, and Codex drive the whole loop through an MCP server and /overmind commands that cover tracing, datasets, fine-tuning, optimising, and backtesting. Self-hosting keeps traces and training data in your network: one Docker Compose file starts it, a seed command loads a demo workspace, and your own OpenRouter, Modal, and AWS keys power judging, training, and checkpoints. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. AGPL-3.0 licensed.

Overmind
Overmind
Overmind
Overmind
Overmind
Overmind

Benefits

  • Failures Become Training Data
  • Production traces are scored as they arrive and matched to the capability that produced them, so real failures and successes can become versioned eval and training datasets.
  • Models You Actually Own
  • Fine-tuned weights are yours to download, retrain, or roll back, and backtests compare each candidate against your agent's own traces before you switch the active model.
  • Improvements Arrive as Git Diffs
  • Optimisers run prompt, tool, and control-flow experiments inside your repository, and the winning variant comes back as a git diff you can review like any other code change.
  • Driven From Your Coding Agent
  • An MCP server and ready-made commands let Cursor, Claude Code, OpenCode, and Codex instrument tracing, build datasets, fine-tune, optimise, and backtest without leaving the chat.

Features

  • Capability Graph
  • Scans an agent's repository into a graph of capabilities, prompts, tools, and tasks; traces, datasets, evals, and fine-tuned models are all organized around those capabilities.
  • OpenTelemetry Trace Scoring
  • Instruments the LLM SDKs you already use, accepts any OTel exporter at /api/v1/traces, syncs traces from Langfuse, LangSmith, Braintrust, or Galileo, and scores each trace as it arrives.
  • Versioned Datasets and Workshop
  • Production traces or uploaded files become eval and training datasets, with local OCR for scanned PDFs and images and immutable cell versions plus execution receipts in the Workshop.
  • Evals and Optimisers
  • Define what good means per capability, measure it on live traces and in batch, then let optimisers test prompt, tool, and control-flow changes in your repository.
  • Fine-Tuning and Inference
  • Fine-tune models on Modal workers from your own data, then serve trained models, plus optional OpenRouter frontier models, behind a single OpenAI-compatible API.
  • MCP Server and CLI
  • The /api/mcp/ server exposes tools for traces, datasets, evals, training, and optimisation, while overmind init and sync prepare your IDE configuration and install the project API key.

Apps Similar to Overmind