This course was created with the
course builder. Create your online course today.
Start now
Create your course
with
Autoplay
Autocomplete
Previous Lesson
Complete and Continue
Apache Airflow: Complete Hands-On Beginner to Advanced Class
Introduction
Introduction (1:17)
Your Airflow Journey (2:36)
What is Apache Airflow? (1:40)
Comparing Airflow to Other Tools (0:45)
Course Prerequisites
Extra: Install Conda (Virtual Environment Manager) (2:43)
Quiz 1: Airflow Basics
Getting Started with Apache Airflow
Components of Airflow (2:12)
Install Airflow on MacOS (5:45)
Install Airflow on Linux
Install Airflow on Windows (19:30)
Install and Run Airflow with Docker
Run Airflow Locally (3:22)
Introduction to the Airflow UI (4:05)
Introduction to the Airflow CLI (2:37)
Quiz 2: Airflow Setup
Core Concepts in Apache Airflow
What are DAGs? (4:49)
What are Default Arguments? (2:00)
What are Tasks and Operators? (5:21)
How to Define Dependencies? (5:16)
Quiz 3: Core Concepts
Loading Data to a Data Warehouse
Use Case (2:06)
Set Up (4:55)
Connections (2:50)
Load Data from Storage to BigQuery (8:01)
Run SQL Query in BigQuery (9:47)
Use Hook to List Storage Objects (5:42)
Cross-Task Communication (XComs) (6:59)
Jinja Templating and Macros (5:33)
Variables (4:44)
Quiz 4: Advanced Concepts
Analysing Data using PySpark
Use Case (5:06)
Set Up (2:36)
Create Dataproc Hadoop Cluster (4:49)
Branching (4:33)
Submit a PySpark Job (4:13)
Subdags (9:19)
Trigger Rules (3:55)
DAG Documentation (6:39)
Quiz 5: Advanced Concepts
Extending Airflow with Custom Plugins
Create a Custom Operator (17:38)
Create a Custom Sensor (8:33)
Run Custom Plugins (7:20)
Quiz 6: Custom Plugins
Testing Airflow DAGs
Load Test DAGs (6:00)
Unit Test DAGs and Operators (12:08)
Unit Test Custom Operators (10:25)
Quiz 7: Testing
Airflow in Production
Executors (7:45)
Configure Local Executor (21:50)
Configure Celery Executor
Service Level Agreements (SLAs) (3:48)
Security: Authentication, Roles, Encryption (7:10)
Write Logs to a Remote Location (4:32)
Monitor Airflow with StatsD, Prometheus and Grafana (19:26)
Error Tracking with Sentry (3:29)
Managed Airflow Services
Quiz 8: Airflow in Production
Finale
Clean Up in Google Cloud
Additional Resources
What's Next?
Course Feedback
Clean Up in Google Cloud
Lesson content locked
If you're already enrolled,
you'll need to login
.
Enroll in Course to Unlock