I am recently building Airflow on Kubernetes for my company. Here is a record of my all the resource I found and holes I ran into.

Here gathers every resource online helps me setup.

The main purpose is to use Helm building Airflow on Kubernetes with the ability to auto-scale and DAGs are git-synced.

I think to understand the structure of Kubernetes is important. You can install Minikube to build a single node Kubernetes.

Documents

Airflow

Home

Kubernetes

Getting started

Helm

Quickstart Guide

Minikube

minikube start

Guide

Deploying Airflow with Helm

Helm Chart for Apache Airflow - helm-chart Documentation

Others

Running Airflow-2.0 on Kubernetes locally using Minikube

序言

Example

Configuration YAML

airflow 1.10.12 for Kubernetes

Official Helm Chart of Airflow

apache/airflow

Other Approaches

airflow-2.0

infinitelambda/minikube-airflow