Architecture

This section features an overview of detailed Apache Airflow Celery Executor architecture.

The CeleryExecutor Apache Airflow installation type consist of several components:

• Workers: execute the assigned tasks.
• Scheduler: responsible for adding the necessary tasks to the queue.
• Web server: HTTP Server provides access to DAG/task status information
• Database: contains information about the status of tasks, DAGs, Variables, connections, etc. Flowz uses PostgreSQL.
• Celery: distributed task queue mechanism. Flowz uses Redis as the broker.

Apache Airflow in CeleryExecutor mode is fit for production-grade deployments.