DevOps for DataOps: Building a CI/CD Pipeline for Apache Airflow DAGs

Introduction

In this post, we will learn how to use GitHub Actions to build an effective CI/CD workflow for our Apache Airflow DAGs. We will use the DevOps concepts of Continuous Integration and Continuous Delivery to automate the testing and deployment of Airflow DAGs to Amazon Managed…

--

--

--

AWS Senior Solutions Architect | 8x AWS Certified Pro | Polyglot Developer | DataOps | DevOps | Technology consultant, writer, and speaker

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Coding Basics for Beginners

The Agile Waterfall Model

Serve it up with Servant!

The Magic Behind the Commons Simulator — Part 1: Game Flow

Command Line Interface vs Graphical User Interface

Radioactive man tied to a pole with goggles, about to scream ‘my eyes! The goggles do nothing!’ From The Simpsons

Unity tip of the day: on-click events

Why everyone should learn to Code ? The Reason!

Stumbling on Coding

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Gary A. Stafford

Gary A. Stafford

AWS Senior Solutions Architect | 8x AWS Certified Pro | Polyglot Developer | DataOps | DevOps | Technology consultant, writer, and speaker

More from Medium

My thoughts on AWS Managed Workflows for Apache Airflow

Bilbo Baggins going on an adventure

Efficiently Transforming, Compressing (in-memory) and Ingesting CSV files to AWS S3 using Python

Lakehouse Automation on AWS with Apache Airflow

Ingesting Clickstream Data with Python, Kinesis, and Terraform