Categories
DevOps CI/CD Kubernetes Workflow Orchestration Workflow Engine Data Pipelines Machine Learning Pipelines Batch ProcessingStars
Forks
Watchers
Developer links
Argo Workflows
Container-native and declarative, Argo Workflows is a workflow engine that runs multi-step jobs in parallel on Kubernetes, each step in its own container. Workflows are written in YAML as ordered steps or as directed acyclic graphs, so dependencies stay explicit and independent branches start at the same moment. The web console lists every run with its phase, progress, and duration, then opens any workflow as a live graph that marks each node as pending, running, failed, or finished. Selecting a node reveals its logs, inputs, and outputs, and retry or resubmit sends a failed run back through the pipeline. Loops fan a pipeline out over a list of items, conditionals skip steps that do not apply, and retry strategies, timeouts, and exit handlers decide what happens when a step misbehaves. Reusable WorkflowTemplates live in the cluster and accept parameters from the console, the argo CLI, or the REST API; CronWorkflows launch the same pipelines on a schedule. Artifacts travel between steps and out to S3, Google Cloud Storage, Azure Blob Storage, or Git, and the archive keeps finished runs in a SQL database for later review. Suspend steps hold a pipeline until someone approves it, and OAuth2 or OIDC single sign-on signs operators in to the console. Machine learning pipelines, batch data processing, infrastructure automation, and CI/CD jobs all fit the same model. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. Apache-2.0 licensed.
Benefits
- Parallel branches shorten long jobs
- Dependencies are declared once as a DAG, and the controller starts every independent branch as its own pod at the same time, cutting wall-clock time for batch and machine learning runs.
- Failures are visible and recoverable
- Retry strategies, timeouts, and exit handlers are set per step or per workflow, and the console shows which node failed so operators can retry or resubmit from the same page.
- Reusable templates standardize pipelines
- WorkflowTemplates and CronWorkflows stored in the cluster let teams run the same parameterized pipeline on demand or on a schedule, from the console, the CLI, or the REST API.
- Results outlive the pods that made them
- Artifacts move between steps and into S3, Google Cloud Storage, Azure Blob Storage, or Git, and the archive keeps finished runs in a SQL database after their pods are garbage collected.
Features
- DAG and Steps Workflows
- Define each workflow as a Kubernetes custom resource in YAML using steps or a directed acyclic graph, with loops, conditionals, parameters, and reusable templates.
- Live Workflow Graph Console
- The React web console lists runs by phase and renders each workflow as a graph with per-node status, duration, inputs, outputs, logs, and resource usage.
- Templates and Cron Scheduling
- WorkflowTemplates, ClusterWorkflowTemplates, and CronWorkflows run reusable pipelines on demand or on schedules, with concurrency policies, multiple schedules, and backfill.
- Artifact Repositories
- Pass files between steps and store outputs in S3, Google Cloud Storage, Azure Blob Storage, Artifactory, Alibaba Cloud OSS, HTTP, Git, or plugin-provided drivers.
- REST, gRPC, and CLI Access
- Argo Server exposes HTTP and gRPC APIs, the argo CLI manages runs from a terminal, and SDKs for Java, Go, Python (Hera), and TypeScript generate workflows from code.
- SSO, Archive, and Metrics
- OAuth2 and OIDC single sign-on, a PostgreSQL, MySQL, or MariaDB workflow archive, and Prometheus metrics support day-to-day operation of production clusters.