feat!: replace traefik with nginx ingress (#388)

* feat: remove traefik crds Signed-off-by: Devin Buhl <devin@buhl.casa> * feat: change variable names in config sample env Signed-off-by: Devin Buhl <devin@buhl.casa> * feat: change variable names in configure Signed-off-by: Devin Buhl <devin@buhl.casa> * feat: change variable names in cluster-settings Signed-off-by: Devin Buhl <devin@buhl.casa> * feat: change variable names in configure Signed-off-by: Devin Buhl <devin@buhl.casa> * feat: change README.md to ingress-nginx Signed-off-by: Devin Buhl <devin@buhl.casa> * feat: remove traefik helm release Signed-off-by: Devin Buhl <devin@buhl.casa> * feat: remove traefik helm repo Signed-off-by: Devin Buhl <devin@buhl.casa> * feat: remove error-pages Signed-off-by: Devin Buhl <devin@buhl.casa> * feat: remove other traefik references Signed-off-by: Devin Buhl <devin@buhl.casa> * feat: add ingress-nginx helm repo Signed-off-by: Devin Buhl <devin@buhl.casa> * feat: add ingress-nginx helm release Signed-off-by: Devin Buhl <devin@buhl.casa>
2022-07-07 09:51:41 -04:00
1 parent bcb6f003d2
commit 6a758ee101
@@ -27,10 +27,10 @@ export BOOTSTRAP_METALLB_LB_RANGE=""
 # The load balancer IP for k8s_gateway, choose from one of the available IPs above
 # e.g. 192.168.1.220
 export BOOTSTRAP_METALLB_K8S_GATEWAY_ADDR=""
-# The load balancer IP for traefik, choose from one of the available IPs above
+# The load balancer IP for the ingress controller, choose from one of the available IPs above
 # that doesn't conflict with any other IP addresses here
 # e.g. 192.168.1.221
-export BOOTSTRAP_METALLB_TRAEFIK_ADDR=""
+export BOOTSTRAP_METALLB_INGRESS_ADDR=""

 # Age Public Key - string should start with age
 # e.g. age15uzrw396e67z9wdzsxzdk7ka0g2gr3l460e0slaea563zll3hdfqwqxdta
@@ -26,7 +26,7 @@ The following components will be installed in your [k3s](https://k3s.io/) cluste
 - [calico](https://www.tigera.io/project-calico/) - Container networking interface for inter pod and service networking
 - [external-dns](https://github.com/kubernetes-sigs/external-dns) - Operator to publish DNS records to Cloudflare (and other providers) based on Kubernetes ingresses
 - [k8s_gateway](https://github.com/ori-edge/k8s_gateway) - DNS resolver that provides local DNS to your Kubernetes ingresses
- [traefik](https://traefik.io) - Kubernetes ingress controller used for a HTTP reverse proxy of Kubernetes ingresses
+- [ingress-nginx](https://kubernetes.github.io/ingress-nginx/) - Kubernetes ingress controller used for a HTTP reverse proxy of Kubernetes ingresses
 - [local-path-provisioner](https://github.com/rancher/local-path-provisioner) - provision persistent local storage with Kubernetes

 _Additional applications include [hajimari](https://github.com/toboshii/hajimari), [error-pages](https://github.com/tarampampam/error-pages), [echo-server](https://github.com/Ealenn/Echo-Server), [system-upgrade-controller](https://github.com/rancher/system-upgrade-controller), [reflector](https://github.com/emberstack/kubernetes-reflector), [reloader](https://github.com/stakater/Reloader), and [kured](https://github.com/weaveworks/kured)_
@@ -384,11 +384,11 @@ task cluster:resources

 [k8s_gateway](https://github.com/ori-edge/k8s_gateway) is deployed on the IP choosen for `${BOOTSTRAP_METALLB_K8S_GATEWAY_ADDR}`. Inorder to test DNS you can point your clients DNS to the `${BOOTSTRAP_METALLB_K8S_GATEWAY_ADDR}` IP address and load `https://hajimari.${BOOTSTRAP_CLOUDFLARE_DOMAIN}` in your browser.

-You can also try debugging with the command `dig`, e.g. `dig @${BOOTSTRAP_METALLB_K8S_GATEWAY_ADDR} hajimari.${BOOTSTRAP_CLOUDFLARE_DOMAIN}` and you should get a valid answer containing your `${BOOTSTRAP_METALLB_TRAEFIK_ADDR}` IP address.
+You can also try debugging with the command `dig`, e.g. `dig @${BOOTSTRAP_METALLB_K8S_GATEWAY_ADDR} hajimari.${BOOTSTRAP_CLOUDFLARE_DOMAIN}` and you should get a valid answer containing your `${BOOTSTRAP_METALLB_INGRESS_ADDR}` IP address.

 If your router (or Pi-Hole, Adguard Home or whatever) supports conditional DNS forwarding (also know as split-horizon DNS) you may have DNS requests for `${SECRET_DOMAIN}` only point to the  `${BOOTSTRAP_METALLB_K8S_GATEWAY_ADDR}` IP address. This will ensure only DNS requests for `${SECRET_DOMAIN}` will only get routed to your [k8s_gateway](https://github.com/ori-edge/k8s_gateway) service thus providing DNS resolution to your cluster applications/ingresses.

-To access services from the outside world port forwarded `80` and `443` in your router to the `${BOOTSTRAP_METALLB_TRAEFIK_ADDR}` IP, in a few moments head over to your browser and you _should_ be able to access `https://echo-server.${BOOTSTRAP_CLOUDFLARE_DOMAIN}` from a device outside your LAN.
+To access services from the outside world port forwarded `80` and `443` in your router to the `${BOOTSTRAP_METALLB_INGRESS_ADDR}` IP, in a few moments head over to your browser and you _should_ be able to access `https://echo-server.${BOOTSTRAP_CLOUDFLARE_DOMAIN}` from a device outside your LAN.

 Now if nothing is working, that is expected. This is DNS after all!

@@ -71,15 +71,13 @@ spec:
    ingress:
      main:
        enabled: true
-        ingressClassName: "traefik"
+        ingressClassName: "nginx"
        annotations:
          cert-manager.io/cluster-issuer: "letsencrypt-staging"
          external-dns.alpha.kubernetes.io/target: "ipv4.${SECRET_DOMAIN}"
          external-dns/is-public: "true"
          hajimari.io/enable: "true"
          hajimari.io/icon: "video-input-antenna"
-          traefik.ingress.kubernetes.io/router.entrypoints: "websecure"
-          traefik.ingress.kubernetes.io/router.middlewares: "networking-cloudflare-only@kubernetescrd"
        hosts:
          - host: &host "{{ .Release.Name }}.${SECRET_DOMAIN}"
            paths:
@@ -105,13 +105,12 @@ spec:
    ingress:
      main:
        enabled: true
-        ingressClassName: "traefik"
+        ingressClassName: "nginx"
        annotations:
          cert-manager.io/cluster-issuer: "letsencrypt-staging"
          hajimari.io/enable: "true"
          hajimari.io/icon: "weather-sunset"
-          traefik.ingress.kubernetes.io/router.entrypoints: "websecure"
-          traefik.ingress.kubernetes.io/router.middlewares: "networking-internal-only@kubernetescrd"
+          nginx.ingress.kubernetes.io/whitelist-source-range: "10.0.0.0/8,172.16.0.0/12,192.168.0.0/16"
        hosts:
          - host: "hajimari.${SECRET_DOMAIN}"
            paths:
@@ -1,40 +0,0 @@
---
-apiVersion: helm.toolkit.fluxcd.io/v2beta1
-kind: HelmRelease
-metadata:
-  name: error-pages
-  namespace: networking
-spec:
-  interval: 15m
-  chart:
-    spec:
-      chart: error-pages
-      version: 1.1.2
-      sourceRef:
-        kind: HelmRepository
-        name: k8s-at-home
-        namespace: flux-system
-      interval: 15m
-  install:
-    createNamespace: true
-    remediation:
-      retries: 5
-  upgrade:
-    remediation:
-      retries: 5
-  dependsOn:
-    - name: traefik
-      namespace: networking
-  values:
-    image:
-      repository: ghcr.io/tarampampam/error-pages
-      tag: 2.16.0
-    env:
-      TEMPLATE_NAME: lost-in-space
-      SHOW_DETAILS: "false"
-    resources:
-      requests:
-        memory: 100Mi
-        cpu: 100m
-      limits:
-        memory: 250Mi
@@ -1,19 +0,0 @@
---
-apiVersion: traefik.containo.us/v1alpha1
-kind: IngressRoute
-metadata:
-  name: error-pages
-  namespace: networking
-spec:
-  entryPoints:
-    - websecure
-  routes:
-    - kind: Rule
-      match: HostRegexp(`{host:.+}`)
-      priority: 1
-      services:
-        - kind: Service
-          name: error-pages
-          port: 8080
-  tls:
-    secretName: error-pages-tls
@@ -1,6 +0,0 @@
---
-apiVersion: kustomize.config.k8s.io/v1beta1
-kind: Kustomization
-resources:
-  - helm-release.yaml
-  - ingress-route.yaml
@@ -0,0 +1 @@
+173.245.48.0/20\,103.21.244.0/22\,103.22.200.0/22\,103.31.4.0/22\,141.101.64.0/18\,108.162.192.0/18\,190.93.240.0/20\,188.114.96.0/20\,197.234.240.0/22\,198.41.128.0/17\,162.158.0.0/15\,104.16.0.0/13\,104.24.0.0/14\,172.64.0.0/13\,131.0.72.0/22\,2400:cb00::/32\,2606:4700::/32\,2803:f800::/32\,2405:b500::/32\,2405:8100::/32\,2a06:98c0::/29\,2c0f:f248::/32
@@ -0,0 +1,84 @@
+---
+apiVersion: helm.toolkit.fluxcd.io/v2beta1
+kind: HelmRelease
+metadata:
+  name: ingress-nginx
+  namespace: networking
+spec:
+  interval: 15m
+  chart:
+    spec:
+      chart: ingress-nginx
+      version: 4.1.4
+      sourceRef:
+        kind: HelmRepository
+        name: ingress-nginx
+        namespace: flux-system
+      interval: 15m
+  install:
+    createNamespace: true
+    remediation:
+      retries: 5
+  upgrade:
+    remediation:
+      retries: 5
+  dependsOn:
+    - name: cert-manager
+      namespace: kube-system
+  values:
+    controller:
+      replicaCount: 1
+      extraEnvs:
+        - name: TZ
+          value: "${TIMEZONE}"
+      service:
+        annotations:
+          metallb.universe.tf/loadBalancerIPs: |-
+            ${METALLB_INGRESS_ADDR}
+        externalTrafficPolicy: Local
+      publishService:
+        enabled: true
+      ingressClassResource:
+        default: true
+      config:
+        client-header-timeout: 120
+        client-body-buffer-size: "100M"
+        client-body-timeout: 120
+        custom-http-errors: |-
+          401,403,404,500,501,502,503
+        enable-brotli: "true"
+        forwarded-for-header: "CF-Connecting-IP"
+        hsts-max-age: "31449600"
+        keep-alive: 120
+        keep-alive-requests: 10000
+        proxy-body-size: "100M"
+        ssl-protocols: "TLSv1.3 TLSv1.2"
+        use-forwarded-headers: "true"
+      metrics:
+        enabled: false
+      podAnnotations:
+        configmap.reloader.stakater.com/reload: "cloudflare-proxied-networks"
+      resources:
+        requests:
+          cpu: 10m
+          memory: 250Mi
+        limits:
+          memory: 500Mi
+    defaultBackend:
+      enabled: true
+      image:
+        repository: ghcr.io/tarampampam/error-pages
+        tag: 2.16.0
+      replicaCount: 1
+      extraEnvs:
+        - name: TEMPLATE_NAME
+          value: lost-in-space
+        - name: SHOW_DETAILS
+          value: "false"
+  valuesFrom:
+    # Cloudflare Networks
+    # https://www.cloudflare.com/ips/
+    - targetPath: controller.config.proxy-real-ip-cidr
+      kind: ConfigMap
+      name: cloudflare-proxied-networks
+      valuesKey: cloudflare-proxied-networks.txt
@@ -0,0 +1,12 @@
+---
+apiVersion: kustomize.config.k8s.io/v1beta1
+kind: Kustomization
+namespace: networking
+resources:
+  - helm-release.yaml
+configMapGenerator:
+  - name: cloudflare-proxied-networks
+    files:
+      - cloudflare-proxied-networks.txt
+generatorOptions:
+  disableNameSuffixHash: true
@@ -3,9 +3,8 @@ apiVersion: kustomize.config.k8s.io/v1beta1
 kind: Kustomization
 resources:
  - namespace.yaml
-  - error-pages
  - cloudflare-ddns
  - external-dns
+  - ingress-nginx
  - metallb
  - k8s-gateway
-  - traefik
@@ -1,30 +0,0 @@
---
-apiVersion: networking.k8s.io/v1
-kind: Ingress
-metadata:
-  name: traefik-dashboard
-  namespace: networking
-  annotations:
-    cert-manager.io/cluster-issuer: "letsencrypt-staging"
-    hajimari.io/appName: "traefik"
-    hajimari.io/enable: "true"
-    hajimari.io/icon: "web"
-    traefik.ingress.kubernetes.io/router.entrypoints: "websecure"
-    traefik.ingress.kubernetes.io/router.middlewares: "networking-internal-only@kubernetescrd"
-spec:
-  ingressClassName: "traefik"
-  tls:
-    - hosts:
-        - "traefik.${SECRET_DOMAIN}"
-      secretName: "traefik-dashboard-tls"
-  rules:
-    - host: "traefik.${SECRET_DOMAIN}"
-      http:
-        paths:
-          - path: /
-            pathType: Prefix
-            backend:
-              service:
-                name: traefik
-                port:
-                  number: 9000
@@ -1,5 +0,0 @@
---
-apiVersion: kustomize.config.k8s.io/v1beta1
-kind: Kustomization
-resources:
-  - ingress.yaml
@@ -1,95 +0,0 @@
---
-apiVersion: helm.toolkit.fluxcd.io/v2beta1
-kind: HelmRelease
-metadata:
-  name: traefik
-  namespace: networking
-spec:
-  interval: 15m
-  chart:
-    spec:
-      chart: traefik
-      version: 10.22.0
-      sourceRef:
-        kind: HelmRepository
-        name: traefik
-        namespace: flux-system
-      interval: 15m
-  install:
-    createNamespace: true
-    remediation:
-      retries: 5
-  upgrade:
-    remediation:
-      retries: 5
-  dependsOn:
-    - name: cert-manager
-      namespace: kube-system
-    - name: metallb
-      namespace: networking
-  values:
-    deployment:
-      kind: Deployment
-      replicas: 1
-    service:
-      enabled: true
-      type: LoadBalancer
-      annotations:
-        metallb.universe.tf/loadBalancerIPs: |-
-          ${METALLB_TRAEFIK_ADDR}
-      spec:
-        externalTrafficPolicy: Local
-    logs:
-      general:
-        format: json
-        level: DEBUG
-      access:
-        enabled: true
-        format: json
-    ingressClass:
-      enabled: true
-      isDefaultClass: true
-      fallbackApiVersion: v1
-    ingressRoute:
-      dashboard:
-        enabled: false
-    globalArguments:
-      - "--api.insecure=true"
-      - "--serverstransport.insecureskipverify=true"
-      - "--providers.kubernetesingress.ingressclass=traefik"
-      - "--metrics.prometheus=true"
-      - "--metrics.prometheus.entryPoint=metrics"
-      - "--entryPoints.websecure.forwardedHeaders.trustedIPs=10.0.0.0/8,192.168.0.0/16,172.16.0.0/12,103.21.244.0/22,103.22.200.0/22,103.31.4.0/22,104.16.0.0/13,104.24.0.0/14,108.162.192.0/18,131.0.72.0/22,141.101.64.0/18,162.158.0.0/15,172.64.0.0/13,173.245.48.0/20,188.114.96.0/20,190.93.240.0/20,197.234.240.0/22,198.41.128.0/17,2400:cb00::/32,2606:4700::/32,2803:f800::/32,2405:b500::/32,2405:8100::/32,2a06:98c0::/29,2c0f:f248::/32"
-    additionalArguments:
-      - "--providers.kubernetesingress.ingressendpoint.ip=${METALLB_TRAEFIK_ADDR}"
-      - "--providers.kubernetesingress.allowexternalnameservices=true"
-      - "--providers.kubernetescrd.allowexternalnameservices=true"
-    ports:
-      traefik:
-        expose: true
-      web:
-        redirectTo: websecure
-      websecure:
-        tls:
-          enabled: true
-          options: "default"
-      metrics:
-        port: 8082
-        expose: true
-        exposedPort: 8082
-    tlsOptions:
-      default:
-        minVersion: VersionTLS12
-        maxVersion: VersionTLS13
-        sniStrict: true
-    pilot:
-      enabled: false
-    experimental:
-      plugins:
-        enabled: false
-    resources:
-      requests:
-        memory: 100Mi
-        cpu: 500m
-      limits:
-        memory: 500Mi
@@ -1,7 +0,0 @@
---
-apiVersion: kustomize.config.k8s.io/v1beta1
-kind: Kustomization
-resources:
-  - helm-release.yaml
-  - dashboard
-  - middlewares
@@ -1,47 +0,0 @@
---
-apiVersion: traefik.containo.us/v1alpha1
-kind: Middleware
-metadata:
-  name: cloudflare-ips
-  namespace: networking
-spec:
-  # https://www.cloudflare.com/ips/
-  ipWhiteList:
-    sourceRange:
-      - 173.245.48.0/20
-      - 103.21.244.0/22
-      - 103.22.200.0/22
-      - 103.31.4.0/22
-      - 141.101.64.0/18
-      - 108.162.192.0/18
-      - 190.93.240.0/20
-      - 188.114.96.0/20
-      - 197.234.240.0/22
-      - 198.41.128.0/17
-      - 162.158.0.0/15
-      - 104.16.0.0/13
-      - 104.24.0.0/14
-      - 172.64.0.0/13
-      - 131.0.72.0/22
-      - 2400:cb00::/32
-      - 2606:4700::/32
-      - 2803:f800::/32
-      - 2405:b500::/32
-      - 2405:8100::/32
-      - 2a06:98c0::/29
-      - 2c0f:f248::/32
-      # include rfc1918 ranges since traefik chains do not support OR operations
-      # https://github.com/traefik/traefik/issues/6007
-      - 10.0.0.0/8
-      - 172.16.0.0/12
-      - 192.168.0.0/16
---
-apiVersion: traefik.containo.us/v1alpha1
-kind: Middleware
-metadata:
-  name: cloudflare-only
-  namespace: networking
-spec:
-  chain:
-    middlewares:
-      - name: cloudflare-ips
@@ -1,14 +0,0 @@
---
-apiVersion: traefik.containo.us/v1alpha1
-kind: Middleware
-metadata:
-  name: error-pages
-  namespace: networking
-spec:
-  errors:
-    status:
-      - "400-599"
-    query: /{status}.html
-    service:
-      name: error-pages
-      port: 8080
@@ -1,22 +0,0 @@
---
-apiVersion: traefik.containo.us/v1alpha1
-kind: Middleware
-metadata:
-  name: internal-ips
-  namespace: networking
-spec:
-  ipWhiteList:
-    sourceRange:
-      - 10.0.0.0/8
-      - 172.16.0.0/12
-      - 192.168.0.0/16
---
-apiVersion: traefik.containo.us/v1alpha1
-kind: Middleware
-metadata:
-  name: internal-only
-  namespace: networking
-spec:
-  chain:
-    middlewares:
-      - name: internal-ips
@@ -1,7 +0,0 @@
---
-apiVersion: kustomize.config.k8s.io/v1beta1
-kind: Kustomization
-resources:
-  - error-pages.yaml
-  - cloudflare-only.yaml
-  - internal-only.yaml
@@ -2,8 +2,8 @@
 apiVersion: source.toolkit.fluxcd.io/v1beta2
 kind: HelmRepository
 metadata:
-  name: traefik
+  name: ingress-nginx
  namespace: flux-system
 spec:
  interval: 1h
-  url: https://helm.traefik.io/traefik
+  url: https://kubernetes.github.io/ingress-nginx
@@ -5,11 +5,11 @@ resources:
  - emberstack.yaml
  - external-dns.yaml
  - hajimari.yaml
+  - ingress-nginx.yaml
  - jetstack.yaml
  - k8s-at-home.yaml
  - k8s-gateway.yaml
  - metallb.yaml
  - metrics-server.yaml
  - stakater.yaml
-  - traefik.yaml
  - weaveworks-kured.yaml
@@ -4,4 +4,3 @@ kind: Kustomization
 resources:
  - cert-manager
  - system-upgrade-controller
-  - traefik
@@ -1,30 +0,0 @@
---
-apiVersion: source.toolkit.fluxcd.io/v1beta2
-kind: GitRepository
-metadata:
-  name: traefik-crd-source
-  namespace: flux-system
-spec:
-  interval: 30m
-  url: https://github.com/traefik/traefik-helm-chart.git
-  ref:
-    # renovate: registryUrl=https://helm.traefik.io/traefik chart=traefik
-    tag: v10.22.0
-  ignore: |
-    # exclude all
-    /*
-    # path to crds
-    !/traefik/crds/
---
-apiVersion: kustomize.toolkit.fluxcd.io/v1beta2
-kind: Kustomization
-metadata:
-  name: traefik-crds
-  namespace: flux-system
-spec:
-  interval: 30m
-  prune: false
-  wait: true
-  sourceRef:
-    kind: GitRepository
-    name: traefik-crd-source
@@ -1,5 +0,0 @@
---
-apiVersion: kustomize.config.k8s.io/v1beta1
-kind: Kustomization
-resources:
-  - crds.yaml
@@ -48,7 +48,7 @@ main() {
        if [[ "${BOOTSTRAP_KUBE_VIP_ENABLED}" == "true" ]]; then
            envsubst < "${PROJECT_DIR}/tmpl/cluster/kube-vip-daemonset.yaml" \
                > "${PROJECT_DIR}/cluster/apps/kube-system/kube-vip/daemon-set.yaml"
-	fi
+        fi
        # generate cluster secrets
        envsubst < "${PROJECT_DIR}/tmpl/cluster/cluster-secrets.sops.yaml" \
            > "${PROJECT_DIR}/cluster/config/cluster-secrets.sops.yaml"
@@ -83,7 +83,7 @@ main() {
                yq --inplace '.k3s_server_manifests_urls += [{"url": "https://kube-vip.io/manifests/rbac.yaml", "filename": "kube-vip-rbac.yaml"}]' "${PROJECT_DIR}/provision/ansible/inventory/group_vars/kubernetes/k3s.yml"
            fi
        fi
-	# generate ansible hosts file and secrets
+        # generate ansible hosts file and secrets
        generate_ansible_hosts
        generate_ansible_host_secrets
        setup_github_webhook
@@ -157,7 +157,7 @@ _has_valid_ip() {
 verify_addressing() {
    local found_kube_vip="false"
    local found_k8s_gateway="false"
-    local found_traefik="false"
+    local found_ingress="false"

    # Verify the metallb min and metallb ceiling are in the same network
    metallb_subnet_min=$(echo "${BOOTSTRAP_METALLB_LB_RANGE}" | cut -d- -f1 | cut -d. -f1,2,3)
@@ -194,8 +194,8 @@ verify_addressing() {
        if [[ "${addr}" == "${BOOTSTRAP_METALLB_K8S_GATEWAY_ADDR}" ]]; then
            found_k8s_gateway="true"
        fi
-        if [[ "${addr}" == "${BOOTSTRAP_METALLB_TRAEFIK_ADDR}" ]]; then
-            found_traefik="true"
+        if [[ "${addr}" == "${BOOTSTRAP_METALLB_INGRESS_ADDR}" ]]; then
+            found_ingress="true"
        fi

        for var in "${!BOOTSTRAP_ANSIBLE_HOST_ADDR_@}"; do
@@ -216,8 +216,8 @@ verify_addressing() {
        exit 1
    fi

-    if [[ "${found_traefik}" == "false" ]]; then
-        _log "ERROR" "The IP for traefik '${BOOTSTRAP_METALLB_TRAEFIK_ADDR}' should be in the provided metallb range '${BOOTSTRAP_METALLB_LB_RANGE}'"
+    if [[ "${found_ingress}" == "false" ]]; then
+        _log "ERROR" "The IP for ingress '${BOOTSTRAP_METALLB_INGRESS_ADDR}' should be in the provided metallb range '${BOOTSTRAP_METALLB_LB_RANGE}'"
        exit 1
    fi
 }
@@ -267,7 +267,7 @@ verify_metallb() {
    local ip_ceil=
    _has_envar "BOOTSTRAP_METALLB_LB_RANGE"
    _has_envar "BOOTSTRAP_METALLB_K8S_GATEWAY_ADDR"
-    _has_envar "BOOTSTRAP_METALLB_TRAEFIK_ADDR"
+    _has_envar "BOOTSTRAP_METALLB_INGRESS_ADDR"

    ip_floor=$(echo "${BOOTSTRAP_METALLB_LB_RANGE}" | cut -d- -f1)
    ip_ceil=$(echo "${BOOTSTRAP_METALLB_LB_RANGE}" | cut -d- -f2)
@@ -275,7 +275,7 @@ verify_metallb() {
    _has_valid_ip "${ip_floor}" "BOOTSTRAP_METALLB_LB_RANGE"
    _has_valid_ip "${ip_ceil}" "BOOTSTRAP_METALLB_LB_RANGE"
    _has_valid_ip "${BOOTSTRAP_METALLB_K8S_GATEWAY_ADDR}" "BOOTSTRAP_METALLB_K8S_GATEWAY_ADDR"
-    _has_valid_ip "${BOOTSTRAP_METALLB_TRAEFIK_ADDR}" "BOOTSTRAP_METALLB_TRAEFIK_ADDR"
+    _has_valid_ip "${BOOTSTRAP_METALLB_INGRESS_ADDR}" "BOOTSTRAP_METALLB_INGRESS_ADDR"
 }

 verify_git_repository() {
@@ -377,8 +377,8 @@ verify_ansible_hosts() {
            exit 1
        fi

-        if [[ "${!node_addr}" == "${BOOTSTRAP_METALLB_TRAEFIK_ADDR}" ]]; then
-            _log "ERROR" "The traefik load balancer IP '${BOOTSTRAP_METALLB_TRAEFIK_ADDR}' should not be the same as the IP for node '${!node_addr}'"
+        if [[ "${!node_addr}" == "${BOOTSTRAP_METALLB_INGRESS_ADDR}" ]]; then
+            _log "ERROR" "The ingress load balancer IP '${BOOTSTRAP_METALLB_INGRESS_ADDR}' should not be the same as the IP for node '${!node_addr}'"
            exit 1
        fi

@@ -17,7 +17,7 @@ k3s_server:
  disable:
    # Disable flannel - replaced with Calico
    - flannel
-    # Disable traefik - installed with Flux
+    # Disable traefik - replaced with ingress-nginx
    - traefik
    # Disable servicelb - replaced with metallb and install with Flux
    - servicelb
@@ -8,4 +8,4 @@ data:
  TIMEZONE: ${BOOTSTRAP_TIMEZONE}
  METALLB_LB_RANGE: ${BOOTSTRAP_METALLB_LB_RANGE}
  METALLB_K8S_GATEWAY_ADDR: ${BOOTSTRAP_METALLB_K8S_GATEWAY_ADDR}
-  METALLB_TRAEFIK_ADDR: ${BOOTSTRAP_METALLB_TRAEFIK_ADDR}
+  METALLB_INGRESS_ADDR: ${BOOTSTRAP_METALLB_INGRESS_ADDR}
@@ -8,9 +8,8 @@ metadata:
    cert-manager.io/cluster-issuer: "letsencrypt-staging"
    external-dns/is-public: "true"
    external-dns.alpha.kubernetes.io/target: "ipv4.${SECRET_DOMAIN}"
-    traefik.ingress.kubernetes.io/router.middlewares: "networking-cloudflare-only@kubernetescrd"
 spec:
-  ingressClassName: traefik
+  ingressClassName: nginx
  rules:
    - host: "flux-receiver.${SECRET_DOMAIN}"
      http:
@@ -36,8 +36,3 @@ spec:
      kind: Kustomization
      name: flux-system
      namespace: flux-system
-
-    - apiVersion: kustomize.toolkit.fluxcd.io/v1beta2
-      kind: Kustomization
-      name: traefik-crds
-      namespace: flux-system
				`@@ -0,0 +1 @@`
				`173.245.48.0/20\,103.21.244.0/22\,103.22.200.0/22\,103.31.4.0/22\,141.101.64.0/18\,108.162.192.0/18\,190.93.240.0/20\,188.114.96.0/20\,197.234.240.0/22\,198.41.128.0/17\,162.158.0.0/15\,104.16.0.0/13\,104.24.0.0/14\,172.64.0.0/13\,131.0.72.0/22\,2400:cb00::/32\,2606:4700::/32\,2803:f800::/32\,2405:b500::/32\,2405:8100::/32\,2a06:98c0::/29\,2c0f:f248::/32`