rancher-partner-charts/charts/haproxy/haproxy

HAProxy Kubernetes Ingress Controller

An ingress controller is a Kubernetes resource that routes traffic from outside your cluster to services within the cluster. HAProxy Kubernetes Ingress Controller uses ConfigMap to store the haproxy configuration.

Detailed documentation can be found within the Official Documentation.

Additional configuration details can be found in annotation reference and in image arguments reference.

Introduction

This chart bootstraps an HAProxy kubernetes-ingress deployment/daemonset on a Kubernetes cluster using the Helm package manager.

Prerequisites

• Kubernetes 1.17+ (recommended 1.20+)
• Helm 3.6+ (recommended 3.7+)

Before you begin

Setting up a Kubernetes Cluster

The quickest way to setup a Kubernetes cluster is with Azure Kubernetes Service, AWS Elastic Kubernetes Service or Google Kubernetes Engine using their respective quick-start guides.

For setting up Kubernetes on other cloud platforms or bare-metal servers refer to the Kubernetes getting started guide.

Install Helm

Get the latest Helm release.

Adding Helm chart repo

Once you have Helm installed, add the repo as follows:

helm repo add haproxytech https://haproxytech.github.io/helm-charts

helm repo update

Installing the chart

To install the chart with Helm v3 as my-release deployment:

helm install my-release haproxytech/kubernetes-ingress

NOTE: To install the chart with Helm v2 (legacy Helm) the syntax requires adding deployment name to --name parameter:

helm install haproxytech/kubernetes-ingress \
  --name my-release

By default Helm chart will install several custom resource definitions in the cluster if they are missing.

Installing with unique name

To auto-generate controller and its resources names when installing, use the following:

helm install haproxytech/kubernetes-ingress \
  --generate-name

Installing from a private registry

To install the chart using a private registry for controller into a separate namespace prod.

NOTE: Helm v3 requires namespace to be precreated (eg. with kubectl create namespace prod)

helm install my-ingress haproxytech/kubernetes-ingress  \
  --namespace prod \
  --set controller.image.tag=SOMETAG \
  --set controller.imageCredentials.registry=myregistry.domain.com \
  --set controller.imageCredentials.username=MYUSERNAME \
  --set controller.imageCredentials.password=MYPASSWORD

Alternatively, use a pre-configured (existing) imagePullSecret in the same namespace:

helm install my-ingress haproxytech/kubernetes-ingress  \
  --namespace prod \
  --set controller.image.tag=SOMETAG \
  --set controller.existingImagePullSecret name-of-existing-image-pull-secret

Using values from YAML file

As opposed to using many --set invocations, much simpler approach is to define value overrides in a separate YAML file and specify them when invoking Helm:

mylb.yaml:

controller:
  kind: DaemonSet
  ingressClass: haproxy
  service:
    type: LoadBalancer
    annotations:
      service.beta.kubernetes.io/aws-load-balancer-cross-zone-load-balancing-enabled: "true"
      service.beta.kubernetes.io/aws-load-balancer-internal: 0.0.0.0/0

And invoking Helm becomes (compare to the previous example):

helm install my-ingress -f mylb.yml haproxytech/kubernetes-ingress

A typical YAML file for TCP services looks like (provided that configmap "default/tcp" was created) :

controller:
  service:
    tcpPorts:
      - name: mysql
        port: 3306
        targetPort: 3306
  extraArgs:
    - --configmap-tcp-services=default/tcp

Installing as DaemonSet

Default controller mode is Deployment, but it is possible to use DaemonSet as well:

helm install my-ingress haproxytech/kubernetes-ingress \
  --set controller.kind=DaemonSet

Installing in multi-ingress environment

It is also possible to set controller ingress class to be used in multi-ingress environments:

helm install my-ingress haproxytech/kubernetes-ingress \
  --set controller.kind=DaemonSet \
  --set controller.ingressClass=haproxy

NOTE: make sure your Ingress routes have corresponding ingress.class: haproxy annotation.

Installing with service annotations

On some environments like EKS and GKE there might be a need to pass service annotations. Syntax can become a little tedious however:

helm install my-ingress haproxytech/kubernetes-ingress \
  --set controller.kind=DaemonSet \
  --set controller.ingressClass=haproxy \
  --set controller.service.type=LoadBalancer \
  --set controller.service.annotations."service\.beta\.kubernetes\.io/aws-load-balancer-internal"="0.0.0.0/0" \
  --set controller.service.annotations."service\.beta\.kubernetes\.io/aws-load-balancer-cross-zone-load-balancing-enabled"="true"

NOTE: With helm --set it is needed to put quotes and escape dots in the annotation key and commas in the value string.

Installing with Horizontal Pod Autoscaler (HPA)

HPA automatically scales number of replicas in Deployment or Replication Controller and adjusts replica count for the controller:

helm install my-ingress haproxytech/kubernetes-ingress \
  --set controller.autoscaling.enabled=true

Installing the ServiceMonitor

If you're using the Prometheus Operator, you can automatically install the ServiceMonitor definition in order to automate the scraping options according to your needs.

helm repo add prometheus-community https://prometheus-community.github.io/helm-charts

helm install prometheus prometheus-community/kube-prometheus-stack \
  --set prometheus.prometheusSpec.podMonitorSelectorNilUsesHelmValues=false \
  --set prometheus.prometheusSpec.serviceMonitorSelectorNilUsesHelmValues=false

helm install my-ingress haproxytech/kubernetes-ingress \
  --set controller.serviceMonitor.enabled=true

Installing with Kubernetes Event-driven Autoscaling (KEDA)

KEDA is an improved scaling solution built on top of HPA which allows autoscaling criteria based on information from any event source including Prometheus metrics collected from HAProxy native Prometheus Exporter.

To enable KEDA, you will also need to install Prometheus Operator and ServiceMonitor enabled (serverAddress has to match prometheus-kube-prometheus-prometheus service IP):

mykeda.yaml:

controller:
  kind: Deployment
  serviceMonitor:
    enabled: true
  keda:
    enabled: true
    minReplicas: 1
    maxReplicas: 5
    triggers:
      - type: prometheus
        metadata:
          serverAddress: http://10.96.206.247:9090
          metricName: haproxy_frontend_current_sessions
          threshold: "100"
          query: sum(rate(haproxy_frontend_current_sessions{proxy="http"}[2m]))

Note: Other options to trigger scaling can be found in Prometheus native exporter documentation, but some ideas are:

• haproxy_process_idle_time_percent
• haproxy_frontend_current_sessions
• haproxy_backend_current_queue

And to install:

helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo add kedacore https://kedacore.github.io/charts

helm repo update

helm install prometheus prometheus-community/kube-prometheus-stack \
  --set prometheus.prometheusSpec.podMonitorSelectorNilUsesHelmValues=false \
  --set prometheus.prometheusSpec.serviceMonitorSelectorNilUsesHelmValues=false

kubectl create namespace keda
helm install keda kedacore/keda --namespace keda

helm install mytest haproxytech/kubernetes-ingress -f mykeda.yaml

Upgrading the chart

To upgrade the my-release deployment:

helm upgrade my-release haproxytech/kubernetes-ingress

By default Helm does not upgrade CRDs during an upgrade, so before doing an upgrade it is mandatory to upgrade CRDs to the latest version by hand before doing a Helm chart upgrade.

kubectl apply -f https://raw.githubusercontent.com/haproxytech/helm-charts/main/kubernetes-ingress/crds/core.haproxy.org_defaults.yaml
kubectl apply -f https://raw.githubusercontent.com/haproxytech/helm-charts/main/kubernetes-ingress/crds/core.haproxy.org_globals.yaml
kubectl apply -f https://raw.githubusercontent.com/haproxytech/helm-charts/main/kubernetes-ingress/crds/core.haproxy.org_backends.yaml

Uninstalling the chart

To uninstall/delete the my-release deployment:

helm delete my-release

Debugging

It is possible to generate a set of YAML files for testing/debugging:

helm install my-release haproxytech/kubernetes-ingress \
  --debug \
  --dry-run

Contributing

We welcome all contributions. Please refer to guidelines on how to make a contribution.