Open source · Dagster
Dagster: data asset-centric orchestrator
Software-defined assets, typing/validation, observability, and hybrid execution
Dagster is an open-source orchestrator for data teams built around software-defined assets (SDA). It provides typing/schema checks, resource/IO management, materializations, and observability with local or cloud (Dagster Cloud) execution for maintainable pipelines.
Software-defined assets
Declare assets with upstream dependencies and materialization logic for lineage.
Typing & validation
Built-in type/schema checks and metadata propagation reduce runtime issues.
Observability & retries
Logging, events, alerts, retries, and backfills with UI for asset/topology views.
Flexible execution
Run locally or on K8s/hybrid with resources; integrates well with dbt/warehouses.
适用场景 / Use cases
- Asset-centric ELT/ML pipelines
- Data lineage and quality guardrails
- Typed, observable orchestration
- Hybrid deployment and team collaboration