Airflow instance runtime

Launch the Airflow webserver UI in SequentialExector mode to visualise and interact with dashboard (Ctrl-C to stop):

make local-airflow-start

You can access your local Airflow webserver console via http://localhost:8888.

The new sample DAG is rendered under the Apache Airflow dashboard's graph view as follows: Apache Airflow dashboard graph view

Flowz pre-defines a set of Apache Airflow configuration items for a customised view. These are defined in the resources/files/environment/local file and can be edited to suit your preferences:

Apache Airflow customised configuration settings.

# airflow.cfg [core]
AIRFLOW__CORE__DAGS_FOLDER=${AIRFLOW_HOME}/dags
AIRFLOW__CORE__PLUGINS_FOLDER=${AIRFLOW_HOME}/plugins
AIRFLOW__CORE__LOAD_EXAMPLES=False
# Enable DAG Serialization
AIRFLOW__CORE__STORE_SERIALIZED_DAGS=True
AIRFLOW__CORE__MIN_SERIALIZED_DAG_UPDATE_INTERVAL=30
AIRFLOW__CORE__STORE_DAG_CODE=True
# Enable DAGs at startup
AIRFLOW__CORE__DAGS_ARE_PAUSED_AT_CREATION=True
# Quieten the webserver UI warnings
AIRFLOW__CORE__UNIT_TEST_MODE=True
AIRFLOW__CORE__EXECUTOR=SequentialExecutor

# airflow.cfg [webserver]
AIRFLOW__WEBSERVER__DAG_DEFAULT_VIEW=graph
AIRFLOW__WEBSERVER__WEB_SERVER_PORT=8888
AIRFLOW__WEBSERVER__WORKERS=1

# airflow.cfg [logging]
AIRFLOW__LOGGING__LOGGING_CONFIG_CLASS=flowz.config.logging.DEFAULT_LOGGING_CONFIG
AIRFLOW__LOGGING__BASE_LOG_FOLDER=${AIRFLOW_HOME}/logs
AIRFLOW__LOGGING__LOGGING_LEVEL=INFO
AIRFLOW__LOGGING__LOG_FILENAME_TEMPLATE=dag_id={{ ti.dag_id }}/run_id={{ ti.run_id }}/task_id={{ ti.task_id }}/{% if ti.map_index >= 0 %}map_index={{ ti.map_index }}/{% endif %}attempt={{ try_number }}.log

# airflow.cfg [database]
AIRFLOW__DATABASE__SQL_ALCHEMY_CONN=sqlite:///${AIRFLOW_HOME}/airflow.db
# Don't load default connections
AIRFLOW__DATABASE__LOAD_DEFAULT_CONNECTIONS=False