Example DAGs using hooks and operators from Airflow Plugins
This repository contains example DAGs that can be used "out-of-the-box" using operators found in the Airflow Plugins organization. These DAGs have a range of use cases and vary from moving data (see ETL) to background system automation that can give your Airflow "super-powers".
The example DAGs found here can be split into three main categories:
These DAGs focus on pulling data from various systems and putting them into Amazon Redshift, with S3 as a staging store. These represent the simplest implementation of an "ETL" workflow and can either be used "out-of-the-box" or extended to add additional custom logic.
These DAGs demonstrate simple implementations of custom operators and Airflow setups. They are typically not "copy-and-paste" DAGs but rather walk through how something would work.
These DAGs are used on the system administration level and can be thought of as "meta-DAGs" that maintain various states and configurations within Airflow itself. In some cases, these DAGs are used in concert with other custom operators, such as the
Contributions of your own DAGs are very welcome. Please see some of the example DAGs for a sense of general formatting guidelines.