Open source · Apache Airflow
Airflow: Python DAG workflow orchestrator
Scheduled/event triggers, dependencies, monitoring, and rich providers
Apache Airflow is an open-source workflow orchestrator using Python-defined DAGs for data pipelines and batch jobs. It offers scheduling, retries, backfills, dependency management, and a broad provider ecosystem across executors for self-hosted teams.
Python DAGs
Define tasks and dependencies in code with dynamic DAGs and templating.
Scheduling & retries
Cron/scheduled runs, event triggers, retries, SLAs, and backfills.
Visual monitoring
Web UI for DAG graphs, Gantt, logs, manual reruns, and task clearing.
Ecosystem & extensibility
Large provider set with Kubernetes/Local/Celery executors for flexibility.
适用场景 / Use cases
- Batch ETL/ELT orchestration
- Data quality and dependency-aware scheduling
- ML training/deployment pipelines
- Self-hosted auditable scheduling platform