8 apps Big Data
Apache DolphinScheduler screenshot thumbnail

Apache DolphinScheduler

When nightly SQL, Spark, and Python jobs outgrow cron, Apache DolphinScheduler replaces the tangle with a visual workflow orchestrator where every dependency is a line on a drag-and-drop canvas. More than 30 built-in task types cover Shell, SQL, Python, HTTP, Spark, Flink, Kubernetes, SeaTunnel, and Jupyter, while logic nodes add sub-workflows, conditions, switches, and dependencies across projects. Workflows can also be written as code through a Python SDK, YAML, or the Open API and run on timezone-aware cron schedules. Backfill reruns a workflow across a date range or a hand-picked list of dates, in serial or parallel, and any instance can be modified, rolled back, and rerun without touching the saved template. Kill, pause, and resume running tasks, and choose whether a failure ends the run or lets parallel branches continue. Alert groups route results to Email, Slack, Telegram, PagerDuty, Feishu, DingTalk, webhooks, or custom scripts. Tenants, worker groups, and environments isolate teams on shared infrastructure, with project and data source permissions. A Resource Center stores scripts and task files locally or on HDFS, MinIO, and cloud object storage, and the monitor page shows master, worker, and database health at a glance. Standalone, Cluster, Docker, and Kubernetes deployments scale from one node to many masters and workers that can join or leave a running cluster. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. Apache 2.0 licensed.

Deploy
Apache Zeppelin screenshot thumbnail

Apache Zeppelin

Analysts explore data where it lives with Apache Zeppelin, a web notebook where each paragraph can speak a different language: run Spark in Scala, query through SQL, prototype in Python, and plot the results without leaving the page. Pluggable interpreters connect notebooks to Spark, Flink, JDBC databases, Neo4j, Elasticsearch, and dozens of other engines, all sharing variables through the ZeppelinContext so a DataFrame built in one paragraph feeds the next in another language. Built-in visualizations turn query output into bar, pie, line, and scatter charts with pivot controls, while dynamic forms add dropdowns and inputs that make notebooks reusable reports for colleagues. Multiple Spark and Scala versions coexist in one instance, with interpreter binding modes from globally shared to isolated per note, and execution on local, standalone, YARN, or Kubernetes clusters. Teams collaborate through multi-user support with authentication and note permissions, schedule notebooks as jobs, and drive everything through a REST API. The 0.12.1 maintenance release hardens JWT validation and path traversal defenses, upgrades the Angular frontend, and stabilizes the Spark, Flink, Python, Neo4j, and File interpreters. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. Apache-2.0 licensed.

Deploy
Apache Druid screenshot thumbnail

Apache Druid

When dashboards must answer in under a second over billions of streaming events, Apache Druid is the real-time analytics database built for the job. Point a supervisor at an Apache Kafka topic or Amazon Kinesis stream and events are queryable on arrival, late data included, with exactly-once guarantees. Batch loads run as SQL: INSERT and REPLACE statements ingest files, and EXTERN reads external sources. Schema auto-discovery adds new columns and types as data lands. Joins run at ingestion or at query time. A web console gathers the daily work in one place: a tabbed Query view with EXPLAIN PLAN and query history, a Connect external data wizard, and pages for datasources, segments, retention rules, supervisors, tasks, services, and lookups. Configurable tiers place hot and cold data on different hardware. Quality-of-service settings shield priority workloads from heavy ones. Clickstream analytics, ad campaign reporting, network telemetry, application performance tracking, and IoT metrics all fit. Applications connect over the HTTP SQL API or the Avatica JDBC driver, and native JSON queries remain available for advanced needs. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. Apache-2.0 licensed.

Deploy
QuestDB screenshot thumbnail

QuestDB

Powering trading floors at firms processing millions of market data events per second and trusted by organizations including Airbus, Yahoo, and Copenhagen Atomics, QuestDB has earned over 17,000 GitHub stars as the time-series database that refuses to make you choose between ingestion speed and query latency. The column-oriented, time-partitioned storage engine processes millions of rows per second on ingest through write-ahead logging with instant durability, while SIMD-accelerated parallel execution returns analytical queries over billions of rows in milliseconds. Time-series SQL extensions add ASOF JOIN for point-in-time lookups across tables with misaligned timestamps, SAMPLE BY for downsampling at arbitrary time intervals, LATEST ON for last-value queries, and WINDOW JOIN for sliding window aggregations — all through standard SQL syntax accessible via the PostgreSQL wire protocol, REST API, or the QuestDB Wire Protocol (QWP) that streams Apache Arrow at 220 million rows per second. Multi-tier storage automatically moves data from the WAL through native columnar format to Parquet files on S3-compatible object storage, keeping hot data fast and cold data portable without manual tiering. The built-in web console provides an interactive SQL editor with syntax highlighting, autocomplete, and charting for quick data exploration. Views, materialized views, and n-dimensional arrays support real-time analytics, while native Parquet export enables direct integration with Python, DuckDB, and AI/ML frameworks. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. Apache 2.0 licensed.

Deploy
Apache Doris screenshot thumbnail

Apache Doris

Dashboards should not wait minutes for yesterday's data, so Apache Doris answers analytical SQL over fresh data with a massively parallel, fully vectorized engine that processes column batches instead of row after row. Frontends receive queries over the MySQL protocol, plan them with a cost-based optimizer, and coordinate execution, while Backends store columnar tablets and run scans, joins, and aggregations in parallel pipelines across cores. Data lands continuously through stream and batch loads, broker imports from object storage, and change capture from operational databases, and it is queryable immediately for reporting that reflects the last minutes rather than the last nightly job. Tables outside Doris join the same SQL through catalogs for Hive, Iceberg, Paimon, Hudi, and JDBC sources, with local caching and materialized views accelerating the lakehouse queries your team runs every day. Search workloads live here too: an inverted index and a SEARCH function provide full-text retrieval, vector indexes support similarity search, and AI functions call language models from SQL for extraction and summarization, so hybrid queries mix keywords, semantics, and structured filters in one statement. Workload groups, resource controls, and compute-storage separation in cloud mode keep interactive dashboards fast while heavy ETL spills safely to disk. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. Apache-2.0 licensed.

Deploy
Quickwit screenshot thumbnail

Quickwit

With over 11,000 GitHub stars and now backed by Datadog while remaining fully Apache 2.0 licensed, Quickwit delivers the search performance Elasticsearch users expect at a fraction of the infrastructure cost by moving the index to object storage instead of expensive local SSDs. The Rust-based engine, built on the Tantivy search library with SIMD-accelerated vectorized processing and zero garbage collection overhead, achieves sub-second search latency directly against Amazon S3, Azure Blob Storage, Google Cloud Storage, or any S3-compatible backend like MinIO and Ceph. The Elasticsearch-compatible REST API covers ingest, search, query DSL, and aggregations, enabling existing log shippers including Vector, Fluent Bit, and Syslog to migrate without rewriting configurations. Native OpenTelemetry Protocol endpoints accept logs and traces via gRPC, while Jaeger integration provides a drop-in distributed tracing backend. Ingestion from Apache Kafka, Amazon Kinesis, and Apache Pulsar supports streaming pipelines with multi-index partitioning, and the schemaless JSON indexing mode eliminates the need for upfront schema definitions. Stateless searchers and indexers scale horizontally on Kubernetes or bare metal, with a control plane that distributes indexing tasks and a janitor that manages retention policies and GDPR-compliant deletions. The built-in web UI displays search results and cluster state, while the official Grafana data source enables log exploration dashboards. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. Apache 2.0 licensed.

Deploy
Apache NiFi screenshot thumbnail

Apache NiFi

Deployed at thousands of enterprises across financial services, healthcare, government, and telecommunications, Apache NiFi is the industry-standard platform for building automated data pipelines through a visual drag-and-drop browser interface that requires zero coding for common integration patterns. The flow-based programming model connects over 300 built-in processors covering relational databases via ExecuteSQL and PutDatabaseRecord, Apache Kafka with PublishKafka and ConsumeKafka, HTTP endpoints through InvokeHTTP and ListenHTTP, cloud storage for AWS S3, Azure Blob, and Google Cloud Storage, SFTP/FTP file transfers, and JSON, XML, CSV, and Avro transformations. Data provenance tracking logs every routing decision, transformation, and delivery for every FlowFile, creating a searchable lineage graph from source to destination with full content replay capability for auditing and debugging. Guaranteed delivery uses configurable backpressure thresholds, prioritized queuing with latency or throughput optimization, and automatic retry with exponential backoff, ensuring no data loss even during downstream outages. The zero-leader clustering architecture distributes processing across nodes with automatic load balancing, while site-to-site protocol enables secure data transfer between NiFi instances across network boundaries. Security includes OpenID Connect and SAML 2.0 single sign-on, role-based access control with fine-grained policies per component, and TLS encryption for all communication. Custom processors can be written in Java and packaged as NAR bundles, or implemented directly in Python through the native scripting framework. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. Apache 2.0 licensed.

Deploy
Apache Kafka screenshot thumbnail

Apache Kafka

Used by over 80% of Fortune 100 companies including LinkedIn, Netflix, Uber, and Goldman Sachs, Apache Kafka processes trillions of messages per day as the world's most widely deployed distributed event streaming platform. Since version 4.0 released in March 2025, Kafka operates exclusively with KRaft consensus, replacing Apache ZooKeeper entirely with an internal Raft-based metadata quorum managed by controller nodes, reducing operational complexity and eliminating external coordination dependencies. Topics are organized as append-only partitioned commit logs with configurable replication factors across brokers, delivering network-limited throughput with end-to-end latencies as low as 2 milliseconds. Kafka Streams provides a client library for building stateful stream processing applications with exactly-once semantics, windowed aggregations, joins across streams and tables, and interactive queries against local state stores. Kafka Connect integrates with hundreds of systems including PostgreSQL, MySQL, Elasticsearch, Amazon S3, MongoDB, HDFS, and JMS through a standardized connector framework with distributed worker mode and automatic offset management. Share Groups introduced in version 4.2 deliver queue-style consumption semantics alongside traditional consumer groups, enabling Kafka to serve both pub-sub and point-to-point messaging patterns natively. The Schema Registry enforces Avro, Protobuf, and JSON Schema compatibility rules across producers and consumers, preventing schema evolution from breaking downstream applications. Tiered Storage offloads older log segments to object storage like S3 while maintaining transparent consumer access, dramatically reducing local broker storage costs for long-retention topics. Running on a dedicated VPS on RepoCloud with guaranteed CPU, RAM, and SSD, full root SSH access, and a browser serial console. Apache 2.0 licensed.

Deploy