Categories
Self-Hosted Observability Telemetry Data Lake SQL Database Time-Series Database Apache Parquet IoT DataStars
Forks
Watchers
Developer links
Arc
Arc keeps the telemetry you cannot afford to lose in one open time-series database, so last minute's sensor readings and last year's can be queried together with plain SQL. Metrics usually end up split between a fast recent store and a forgotten archive, and Arc removes that split. Data lands as open Apache Parquet on local disk, S3, Azure Blob, or any S3-compatible store such as MinIO, R2, or SeaweedFS, while background compaction merges small files and hot and cold tiers move older data to cheaper storage. Writers can send MessagePack columnar batches, InfluxDB Line Protocol, Prometheus Remote Write, MQTT topic subscriptions, or TLE satellite data, and existing Telegraf agents point at it without changing the collection layer. Queries support window functions, CTEs, joins, and aggregations, and results return as JSON, columnar MessagePack, or Apache Arrow IPC, with an optional Arrow Flight SQL listener for ADBC clients. Retention policies, continuous queries for rollups, a write-ahead log, backup and restore, token authentication with RBAC, per-token quotas, and GDPR-style deletes cover the data lifecycle. Grafana, Superset, a VS Code extension, a Python SDK, and the arcli command line connect to it, and the companion Launchpad web console adds a SQL console, log viewer, monitoring charts, alerts, and MQTT management. It runs as a single binary and serves HTTP on port 8000. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. AGPL-3.0 licensed.
Benefits
- Keep Full-Resolution History
- Automatic compaction merged 43 small files into one and shrank a 372 MB sample to 36 MB, so keeping every reading at full resolution stays affordable on local disk or object storage.
- Migrate From InfluxDB Gradually
- Arc accepts InfluxDB Line Protocol directly, so existing Telegraf inputs can dual-write to it, letting you compare results over an overlapping window before moving dashboards one at a time.
- Own Your Data Files
- Everything is stored as open Apache Parquet on infrastructure you control, so other Parquet-capable tools can read the same files and leaving Arc later never needs an export project.
- Query Recent And Historical Together
- Standard SQL with window functions, CTEs, joins, and time bucketing runs across hot and cold data in one query, with no proprietary query language to learn first.
Features
- Multi-Protocol Ingestion
- Write with MessagePack columnar batches, InfluxDB Line Protocol, Prometheus Remote Write 1.0, MQTT subscriptions, or TLE satellite data; the README reports up to 38.1M records per second.
- Parquet Storage And Compaction
- Records flush to Parquet on local disk, S3, Azure Blob, or MinIO, then tiered hourly and daily compaction merges small files in the background.
- Retention And Continuous Queries
- Retention policies remove data after a set period, and continuous queries roll raw readings into aggregates, such as one-minute CPU averages, on a schedule.
- Flexible Query Output
- Results return as JSON, columnar MessagePack, or Arrow IPC, and an optional Arrow Flight SQL listener serves ADBC and other Flight SQL clients over gRPC.
- Launchpad Web Console
- The companion Launchpad console provides a SQL console, log viewer, monitoring charts, and management pages for tokens, retention, alerts, continuous queries, and MQTT.
- Secure And Durable Operation
- Token authentication with RBAC, per-token quotas, an optional write-ahead log, backup and restore, and a FIPS 140-3 build suit regulated and air-gapped deployments.