Flowz workflow management primer

Flowz is a code repository templater that provides a development environment for integrating your DAGs into the Apache Airflow workflow management system with minimal fuss. Flowz also provides default tooling for the full software development lifecycle and deployment into a containerised environment.

Wait, what is a DAG?

As per the Apache Airflow DAG documentation:

A DAG (Directed Acyclic Graph) is the core concept of Airflow, collecting Tasks together, organized with dependencies and relationships to say how they should run.

Programmatically, a DAG is nothing more than a Python module. As such, consider your workflows as a Python logic whereby software development principles can be applied. Mainly code linting, formatting and tests. Flowz provides a default set of tooling for the SDLC. These are interchangeable to suit your team and projects requirements.

Note

Flowz does not intend to tell you how you should do things. It's here to help you work common tasks.

Where do I start?

Creating DAGs

As Flowz is an Apache Airflow DAG code repository templater, begin by forking a copy into your preferred code repository in the first instance. Next, follow the instructions under the Flowz getting started section.

Getting ready for production: building the container image

Flowz provides the tooling to support a cloud-native deployment model. Follow the: guide under container image management to containerise your workflows so that they are ready for deployment.