Building a Data Lake on AWS with Apache Airflow

Introduction

In the following video demonstration, we will programmatically build a simple data lake on AWS using a combination of services, including Amazon Managed Workflows for Apache Airflow (Amazon MWAA), AWS Glue Data Catalog, AWS Glue Crawlers, AWS Glue Jobs, AWS Glue Studio, Amazon Athena, Amazon

--

--

--

AWS Senior Solutions Architect | 8x AWS Certified Pro | Polyglot Developer | DataOps | DevOps | Technology consultant, writer, and speaker

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Delete Actions In Microsoft D365 F&O

How to Run Your Streamlit Apps in VSCode

The Challenges of Live Linear Video Ingest — Part Two: System Design and Implementation

鬼灭之刃 剧场版 无限列车篇 完整版本 (2020-HD) Kimetsu no Yaiba: Mugen Ressha-Hen 完整版觀看電~看电影.

What is it like going from a coding bootcamp to your first internship — Part 2: Day 1, or “How I…

Raspberry Pi Pico: Hardware projects just got fun again

Take Command of your PagerDuty Incident Response

Unity Command Line Build Error “The only standalone target supported is Windows x64 with OpenXR.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Gary A. Stafford

Gary A. Stafford

AWS Senior Solutions Architect | 8x AWS Certified Pro | Polyglot Developer | DataOps | DevOps | Technology consultant, writer, and speaker

More from Medium

Ingesting Clickstream Data with Python, Kinesis, and Terraform

Abstracting Data Loading with Airflow DAG Factories

DevOps for DataOps: Building a CI/CD Pipeline for Apache Airflow DAGs

My thoughts on AWS Managed Workflows for Apache Airflow

Bilbo Baggins going on an adventure