de1ux

Traefik: gotchas

6/22/2021

Background

Last year, I started running Traefik full-time in my homelab. Coming from nginx, I’m happy that its

But it’s not all roses. I’ve lost a few evenings (and a couple Saturdays) troubleshooting my Traefik setup.

Hopefully these gotchas will saves someone, somewhere a few hours of head-scratching.

I can’t reach the Traefik dashboard

No, it’s not that you disabled the dashboard in the configuration file.

$ kubectl port-forward \
    $(kubectl get pods --selector "app.kubernetes.io/name=traefik" --output=name) 9000:9000

After binding to the traefik pod, always remember to open localhost:9000/dashboard/.

localhost:9000, localhost:9000/dashboard – both of these will 404. It must be localhost:9000/dashboard/

How do I install Traefik v2?

First, k3s ships with an old version of Traefik. Don’t use it – most of the docs / support online are for v2.

$ curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="--no-deploy traefik --node-label role=server " sh -

Install k3s servers without the old version of Traefik. Then install Traefik v2 from the helm chart.

How do I customize Traefik?

The docs are loaded with YAML / TOML examples, but its not clear how they end up in the Traefik container.

One way is to create a config map

apiVersion: v1
kind: ConfigMap
metadata:
  name: traefik-config
data:
  traefik-config.yaml: |
    entryPoints:
      web:
        address: ":80"
      websecure:
        address: ":443"

Customize the values in the Traefik helm chart to volume mount the config map

additionalArguments:
  - --providers.file.filename=/config/traefik-config.yaml

logs:
  general:
    level: DEBUG  # verbose, but handy for configuration error messages


volumes:
  - mountPath: /config
    name: traefik-config
    type: configMap

And install

$ helm install traefik traefik/traefik -f traefik.yml

All the Traefik configuration that doesn’t pertain to individual ingress routes – the traefik-config.yaml is where it should go.

How do I route traffic to a deployment?

With Traefik v1, you’d use native k8s ingress objects.

Don’t do this. Install Traefik v2.

With Traefik v2, you need three parts: a Deployment, a Service, and a Traefik CRD IngressRoute.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: key
  labels:
    app: key
spec:
  selector:
    matchLabels:
      app: key
  replicas: 1
  template:
    metadata:
      labels:
        app: key
    spec:
      nodeSelector:
        role: worker
      containers:
      - name: key
        image: docker.home/key:latest
        imagePullPolicy: Always
        resources:
          requests:
            memory: "128Mi"
            cpu: "100m"
          limits:
            memory: "256Mi"
            cpu: "200m"

---

apiVersion: v1
kind: Service
metadata:
  name: key
spec:
  selector:
    app: key
  ports:
  - port: 80
    name: key

---

apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: key
spec:
  entryPoints:
    - web
  routes:
    - match: Host(`key.home`) && PathPrefix(`/`)
      kind: Rule
      services:
        - name: key
          port: 80

(for this example: *.home DNS requests are forwarded to my server running Traefik, the deployment is an HTTP server listening on port 80)

How do I allow self-signed certs through Traefik? (tls passthrough)

Popular homelab containers like the docker registry, unifi controller require HTTPS. If you don’t provide certs, they’ll default to the container’s self-signed certs.

apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: docker-registry
spec:
  entryPoints:
    - websecure
  routes:
    - match: Host(`docker.home`)
      services:
        - name: docker-registry
          port: 5000
  tls:
    passthrough: true

The Deployment, Service objects stay the same.

How do I specify custom certs through Traefik?

If you have your own certificate pair, the easiest approach is to slap it on the default Traefik cert store.

Change the traefik config map

apiVersion: v1
kind: ConfigMap
metadata:
  name: traefik-config
data:
  traefik-config.yaml: |
    tls:
      stores:
        default:
          defaultCertificate:
            certFile: /certs/ca.crt
            keyFile: /certs/ca.key

And update the helm chart values to volume mount certs into the Traefik deployment

...

volumes:
  - mountPath: /config
    name: traefik-config
    type: configMap
  - mountPath: /certs
    name: my-certs
    type: configMap

...

Where my-certs could be a config map, secret, etc

apiVersion: v1
kind: ConfigMap
metadata:
  name: my-certs
data:
  ca.key: |
    -----BEGIN RSA PRIVATE KEY-----
                ...
    -----END RSA PRIVATE KEY-----    

  ca.crt: |
    -----BEGIN CERTIFICATE-----
                ...
    -----END CERTIFICATE-----

Lastly, set the passthrough to false.

apiVersion: traefik.containo.us/v1alpha1
kind: IngressRouteTCP
metadata:
  name: key
spec:
  entryPoints:
    - websecure
  routes:
    - match: HostSNI(`key.de1ux.com`)
      services:
        - name: key
          port: 80
  tls:
    passthrough: false