Deploy Apache Airflow with Azure Kubernetes Services — 2

Liangjun Jiang
2 min readJan 30, 2020

--

It has been a while since I wrote the first part of this article. Surprisedly, I have received quite many visits. Now I started to feel bad about my polished and careful writing.

Use Helm to deploy an NGINX ingress controller

helm install stable/nginx-ingress \     --namespace ingress-basic \     --set controller.replicaCount=2 \     --set controller.nodeSelector."beta\.kubernetes\.io/os"=linux \     --set defaultBackend.nodeSelector."beta\.kubernetes\.io/os"=linux

If you search around, you might be run into [Microsoft’s ingress controller documentation](https://docs.microsoft.com/en-us/azure/aks/ingress-basic). It will work for most of part. Just need to pay attention that it sets a *namespace*, but when we deploy our *airflow*, we didn’t specify a *namespace*, the namespace is *default*.

Use this command to see how the nginx-ingress goes

kubectl get service -l app=nginx-ingressNAME                                                 TYPE           CLUSTER-IP     EXTERNAL-IP   PORT(S)                      AGEaspiring-labradoodle-nginx-ingress-controller        LoadBalancer   10.0.61.144    40.117.74.8   80:30386/TCP,443:32276/TCP   6m2saspiring-labradoodle-nginx-ingress-default-backend   ClusterIP      10.0.192.145   <none>        80/TCP                       6m2s

We can use the following *ingress controller* configuration file for our AKS airflow *web* service.

apiVersion: extensions/v1beta1kind: Ingressmetadata:name: YOUR-airflow-ingress-NAMEnamespace: defaultannotations:kubernetes.io/ingress.class: nginxnginx.ingress.kubernetes.io/rewrite-target: $1spec:rules:- http:paths:- backend:serviceName: webservicePort: 8080path: /admin(/|$)(.*)- backend:serviceName: flowerservicePort: 5555path: /(.*)

And we need to modify *values.xml* a little bit to use the same IP or domain for both *web* and *flower* dashboard.

```values.xml line of 252ingress:## enable ingressenabled: true## Configure the webserver endpointweb:path: ""host: ""annotations:tls:## Set to "true" to enable TLS termination at the ingressenabled: false#### Configure the flower endpointflower:path: "/flower(/|$)(.*)"livenessPath: /## hostname for flowerhost: ""annotations:kubernetes.io/ingress.class: nginxnginx.ingress.kubernetes.io/rewrite-target: /$2nginx.ingress.kubernetes.io/use-regex: "true"tls:enabled: false

Then move to *40.117.74.8* from your browser to see the *web* service. and *40.117.74.8/flower* to see the *flower* service

Common Error

## *Error: could not find tiller*

```

kubectl -n kube-system delete deployment tiller-deploy

kubectl -n kube-system delete service/tiller-deploy

helm init

```

## *Error: found in requirements.yaml, but missing in charts/ directory: postgresql, redis*

```

helm dependency update

```

## See whether Pod is ready

The service is *not* immediate available once we run a command. You can check the either service status or pod status by running

```

kubectl get pods

```

Or

```

kubectl get services

```

--

--