Traefik 2.1.0 → 3.7.13

Headers middleware: accessControlAllowOrigin

accessControlAllowOrigin is deprecated. This field will be removed in future 2.x releases. Please configure your allowed origins in accessControlAllowOriginList instead.

Kubernetes CRD

In v2.2, new Kubernetes CRDs called TLSStore and IngressRouteUDP were added. While updating an installation to v2.2, one should apply that CRDs, and update the existing ClusterRole definition to allow Traefik to use that CRDs.

To add that CRDs and enhance the permissions, the following definitions need to be applied to the cluster.

TLSStore

apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: tlsstores.traefik.containo.us

spec:
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: TLSStore
    plural: tlsstores
    singular: tlsstore
  scope: Namespaced

IngressRouteUDP

apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: ingressrouteudps.traefik.containo.us

spec:
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: IngressRouteUDP
    plural: ingressrouteudps
    singular: ingressrouteudp
  scope: Namespaced

ClusterRole

kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: traefik-ingress-controller

rules:
  - apiGroups:
      - ""
    resources:
      - services
      - endpoints
      - secrets
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - extensions
      - networking.k8s.io
    resources:
      - ingresses
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - extensions
      - networking.k8s.io
    resources:
      - ingresses/status
    verbs:
      - update
  - apiGroups:
      - traefik.containo.us
    resources:
      - middlewares
      - middlewaretcps
      - ingressroutes
      - traefikservices
      - ingressroutetcps
      - ingressrouteudps
      - tlsoptions
      - tlsstores
      - serverstransports
    verbs:
      - get
      - list
      - watch

After having both resources applied, Traefik will work properly.

Kubernetes Ingress

To enable HTTPS, it is not sufficient anymore to only rely on a TLS section in the Ingress.

Expose an Ingress on 80 and 443

Define the default TLS configuration on the HTTPS entry point.

Ingress

kind: Ingress
apiVersion: networking.k8s.io/v1beta1
metadata:
  name: example

spec:
  tls:
  - secretName: my-tls-secret

  rules:
  - host: example.com
    http:
      paths:
      - path: "/foo"
        backend:
          serviceName: example-com
          servicePort: 80

Entry points definition and enable Ingress provider:

File (YAML)

# Static configuration

entryPoints:
  web:
    address: :80
  websecure:
    address: :443
    http:
      tls: {}

providers:
  kubernetesIngress: {}

File (TOML)

# Static configuration

[entryPoints.web]
  address = ":80"

[entryPoints.websecure]
  address = ":443"
  [entryPoints.websecure.http]
    [entryPoints.websecure.http.tls]

[providers.kubernetesIngress]

CLI

# Static configuration

--entryPoints.web.address=:80
--entryPoints.websecure.address=:443
--entryPoints.websecure.http.tls=true
--providers.kubernetesIngress=true

Use TLS only on one Ingress

Define the TLS restriction with annotations.

Ingress

kind: Ingress
apiVersion: networking.k8s.io/v1beta1
metadata:
  name: example-tls
  annotations:
    traefik.ingress.kubernetes.io/router.entrypoints: websecure
    traefik.ingress.kubernetes.io/router.tls: "true"

spec:
  tls:
  - secretName: my-tls-secret

  rules:
  - host: example.com
    http:
      paths:
      - path: ""
        backend:
          serviceName: example-com
          servicePort: 80

Entry points definition and enable Ingress provider:

File (YAML)

# Static configuration

entryPoints:
  web:
    address: :80
  websecure:
    address: :443

providers:
  kubernetesIngress: {}

File (TOML)

# Static configuration

[entryPoints.web]
  address = ":80"

[entryPoints.websecure]
  address = ":443"

[providers.kubernetesIngress]

CLI

# Static configuration

--entryPoints.web.address=:80
--entryPoints.websecure.address=:443
--providers.kubernetesIngress=true

InsecureSNI removal

In v2.2.2 we introduced a new flag (insecureSNI) which was available as a global option to disable domain fronting. Since v2.2.5 this global option has been removed, and you should not use it anymore.

HostSNI rule matcher removal

In v2.2.2 we introduced a new rule matcher (HostSNI) for HTTP routers which was allowing to match the Server Name Indication at the router level. Since v2.2.5 this rule has been removed for HTTP routers, and you should not use it anymore.

X.509 CommonName Deprecation

The deprecated, legacy behavior of treating the CommonName field on X.509 certificates as a host name when no Subject Alternative Names are present, is now disabled by default.

It means that if one is using https with your backend servers, and a certificate with only a CommonName, Traefik will not try to match the server name indication with the CommonName anymore.

It can be temporarily re-enabled by adding the value x509ignoreCN=0 to the GODEBUG environment variable.

More information: https://golang.org/doc/go1.15#commonname

File Provider

The file parser has been changed, since v2.3 the unknown options/fields in a dynamic configuration file are treated as errors.

IngressClass

In v2.3, the support of IngressClass, which is available since Kubernetes version 1.18, has been introduced. In order to be able to use this new resource the Kubernetes RBAC must be updated.

ServersTransport

In v2.4.0, the support of ServersTransport has been introduced. It is therefore necessary to update RBAC and CRD definitions.

Non-ASCII Domain Names

In v2.4.8, we introduced a new check on domain names used in HTTP router rule Host and HostRegexp expressions, and in TCP router rule HostSNI expression. This check ensures that provided domain names don't contain non-ASCII characters. If not, an error is raised, and the associated router will be shown as invalid in the dashboard.

This new behavior is intended to show what was failing silently previously and to help troubleshooting configuration issues. It doesn't change the support for non-ASCII domain names in routers rules, which is not part of the Traefik feature set so far.

In order to use non-ASCII domain names in a router's rule, one should use the Punycode form of the domain name. For more information, please read the HTTP routers rule part or TCP router rules part of the documentation.

Tracing Span

In v2.4.9, we changed span error to log only server errors (>= 500).

K8S CrossNamespace

In v2.4.10, the default value for allowCrossNamespace has been changed to false.

K8S ExternalName Service

In v2.4.10, by default, it is no longer authorized to reference Kubernetes ExternalName services. To allow it, the allowExternalNameServices option should be set to true.

Kubernetes CRD

In v2.5, the Traefik CRDs have been updated to support the new API version apiextensions.k8s.io/v1. As required by apiextensions.k8s.io/v1, we have included the OpenAPI validation schema.

After deploying the new Traefik CRDs, the resources will be validated only on creation or update.

Please note that the unknown fields will not be pruned when migrating from apiextensions.k8s.io/v1beta1 to apiextensions.k8s.io/v1 CRDs. For more details check out the official documentation.

Kubernetes Ingress

Traefik v2.5 moves forward for the Ingress provider to support Kubernetes v1.22.

Traefik now supports only v1.14+ Kubernetes clusters, which means the support of extensions/v1beta1 API Version ingresses has been dropped.

The extensions/v1beta1 API Version should now be replaced either by networking.k8s.io/v1beta1 or by networking.k8s.io/v1 (as of Kubernetes v1.19+).

The support of the networking.k8s.io/v1beta1 API Version will stop in Kubernetes v1.22.

Headers middleware: ssl redirect options

sslRedirect, sslTemporaryRedirect, sslHost and sslForceHost are deprecated in Traefik v2.5.

For simple HTTP to HTTPS redirection, you may use EntryPoints redirections.

For more advanced use cases, you can use either the RedirectScheme middleware or the RedirectRegex middleware.

Headers middleware: accessControlAllowOrigin

accessControlAllowOrigin is no longer supported in Traefik v2.5.

X.509 CommonName Deprecation Bis

Following up on the deprecation started previously, as the x509ignoreCN=0 value for the GODEBUG is deprecated in Go 1.17, the legacy behavior related to the CommonName field cannot be enabled at all anymore.

Errors middleware

In v2.5.4, when the errors service is configured with the PassHostHeader option to true (default), the forwarded Host header value is now set to the client request Host value and not 0.0.0.0. Check out the Errors middleware documentation for more details.

HTTP/3

Traefik v2.6 introduces the AdvertisedPort option, which allows advertising, in the Alt-Svc header, a UDP port different from the one on which Traefik is actually listening (the EntryPoint's port). By doing so, it introduces a new configuration structure http3, which replaces the enableHTTP3 option (which therefore doesn't exist anymore). To enable HTTP/3 on an EntryPoint, please check out the HTTP/3 configuration documentation.

Kubernetes Gateway API Provider

In v2.6, the Kubernetes Gateway API provider now only supports the version v1alpha2 of the specification and route namespaces selectors, which requires Traefik to fetch and watch the cluster namespaces. Therefore, the RBAC and CRD definitions must be updated.

Metrics

In v2.6.1, the metrics system does not support any more custom HTTP method verbs to prevent potential metrics cardinality overhead. In consequence, for metrics having the method label, if the HTTP method verb of a request is not one defined in the set of common methods for HTTP/1.1 or the PRI verb (for HTTP/2), the value for the method label becomes EXTENSION_METHOD, instead of the request's one.

Tracing

In v2.6.1, the Datadog tags added to a span changed from service.name to traefik.service.name and from router.name to traefik.router.name.

TLS client authentication

In v2.8, the caOptional option is deprecated as TLS client authentication is a server side option. This option available in the ForwardAuth middleware, as well as in the HTTP, Consul, Etcd, Redis, ZooKeeper, Marathon, Consul Catalog, and Docker providers has no effect and must not be used anymore.

Consul Enterprise Namespaces

In v2.8, the namespace option of Consul and Consul Catalog providers is deprecated, please use the namespaces options instead.

Traefik Pilot

In v2.8, the pilot.token and pilot.dashboard options are deprecated. Please check our Blog for migration instructions later this year.

Since v2.5.0, the PreferServerCipherSuites is deprecated and ignored by Go, in v2.8.2 the preferServerCipherSuites option is also deprecated and ignored in Traefik.

In v2.8.2, Traefik now reject certificates signed with the SHA-1 hash function. (details)

Traefik Pilot

In v2.9, Traefik Pilot support has been removed.

Nomad Namespace

In v2.10, the namespace option of the Nomad provider is deprecated, please use the namespaces options instead.

Kubernetes CRDs

In v2.10, the Kubernetes CRDs API Group traefik.containo.us is deprecated, and its support will end starting with Traefik v3. Please use the API Group traefik.io instead.

As the Kubernetes CRD provider still works with both API Versions (traefik.io/v1alpha1 and traefik.containo.us/v1alpha1), it means that for the same kind, namespace and name, the provider will only keep the traefik.io/v1alpha1 resource.

In addition, the Kubernetes CRDs API Version traefik.containo.us/v1alpha1 will not be supported in Traefik v3 itself.

Please note that it is a requirement to update the CRDs and the RBAC in the cluster before upgrading Traefik. To do so, please apply the required CRDs and RBAC manifests for v2.10:

kubectl apply -f https://raw.githubusercontent.com/traefik/traefik/v2.10/docs/content/reference/dynamic-configuration/kubernetes-crd-rbac.yml
kubectl apply -f https://raw.githubusercontent.com/traefik/traefik/v2.10/docs/content/reference/dynamic-configuration/kubernetes-crd-definition-v1.yml

Traefik Hub

In v2.10, Traefik Hub configuration has been removed because Traefik Hub v2 doesn't require this configuration.

IPWhiteList (HTTP)

In v2.11, the IPWhiteList middleware is deprecated, please use the IPAllowList middleware instead.

IPWhiteList (TCP)

In v2.11, the IPWhiteList middleware is deprecated, please use the IPAllowList middleware instead.

TLS CipherSuites

By default, cipher suites without ECDHE support are no longer offered by either clients or servers during pre-TLS 1.3 handshakes. This change can be reverted with the tlsrsakex=1 GODEBUG setting. (https://go.dev/doc/go1.22#crypto/tls)

The RSA key exchange cipher suites are way less secure than the modern ECDHE cipher suites and exposes to potential vulnerabilities like the Marvin Attack. Decision has been made to support ECDHE cipher suites only by default.

The following ciphers have been removed from the default list:

• TLS_RSA_WITH_AES_128_CBC_SHA
• TLS_RSA_WITH_AES_256_CBC_SHA
• TLS_RSA_WITH_AES_128_GCM_SHA256
• TLS_RSA_WITH_AES_256_GCM_SHA384

To enable these ciphers, please set the option CipherSuites in your TLS configuration or set the environment variable GODEBUG=tlsrsakex=1.

Minimum TLS Version

By default, the minimum version offered by crypto/tls servers is now TLS 1.2 if not specified with config.MinimumVersion, matching the behavior of crypto/tls clients. This change can be reverted with the tls10server=1 GODEBUG setting. (https://go.dev/doc/go1.22#crypto/tls)

To enable TLS 1.0, please set the option MinVersion to VersionTLS10 in your TLS configuration or set the environment variable GODEBUG=tls10server=1.

Maximum Router Priority Value

Before v2.11.1, the maximum user-defined router priority value is:

• MaxInt32 for 32-bit platforms,
• MaxInt64 for 64-bit platforms.

Please check out the go documentation for more information.

In v2.11.1, Traefik reserves a range of priorities for its internal routers and now, the maximum user-defined router priority value is:

• (MaxInt32 - 1000) for 32-bit platforms,
• (MaxInt64 - 1000) for 64-bit platforms.

EntryPoint.Transport.RespondingTimeouts.<Timeout>

Starting with v2.11.1 the following timeout options are deprecated:

• <entryPoint>.transport.respondingTimeouts.readTimeout
• <entryPoint>.transport.respondingTimeouts.writeTimeout
• <entryPoint>.transport.respondingTimeouts.idleTimeout

They have been replaced by:

• <entryPoint>.transport.respondingTimeouts.http.readTimeout
• <entryPoint>.transport.respondingTimeouts.http.writeTimeout
• <entryPoint>.transport.respondingTimeouts.http.idleTimeout

EntryPoint.Transport.RespondingTimeouts.TCP.LingeringTimeout

Starting with v2.11.1 a new lingeringTimeout entryPoints option has been introduced, with a default value of 2s.

The lingering timeout defines the maximum duration between each TCP read operation on the connection. As a layer 4 timeout, it applies during HTTP handling but respects the configured HTTP server readTimeout.

This change avoids Traefik instances with the default configuration hanging while waiting for bytes to be read on the connection.

We suggest to adapt this value accordingly to your situation. The new default value is purposely narrowed and can close the connection too early.

Increasing the lingeringTimeout value could be the solution notably if you are dealing with the following errors:

• TCP: Error while handling TCP connection: readfrom tcp X.X.X.X:X->X.X.X.X:X: read tcp X.X.X.X:X->X.X.X.X:X: i/o timeout
• HTTP: '499 Client Closed Request' caused by: context canceled
• HTTP: ReverseProxy read error during body copy: read tcp X.X.X.X:X->X.X.X.X:X: use of closed network connection

LingeringTimeout

Starting with v2.11.2 the <entrypoint>.transport.respondingTimeouts.tcp.lingeringTimeout introduced in v2.11.1 has been removed.

RespondingTimeouts.TCP and RespondingTimeouts.HTTP

Starting with v2.11.2 the respondingTimeouts.tcp and respondingTimeouts.http sections introduced in v2.11.1 have been removed. To configure the responding timeouts, please use the respondingTimeouts section.

EntryPoint.Transport.RespondingTimeouts.ReadTimeout

Starting with v2.11.2 the entryPoints readTimeout option default value changed to 60 seconds.

For HTTP, this option defines the maximum duration for reading the entire request, including the body. For TCP, this option defines the maximum duration for the first bytes to be read on the connection.

The default value was previously set to zero, which means no timeout.

This change has been done to avoid Traefik instances with the default configuration to be hanging forever while waiting for bytes to be read on the connection.

Increasing the readTimeout value could be the solution notably if you are dealing with the following errors:

• TCP: Error while handling TCP connection: readfrom tcp X.X.X.X:X->X.X.X.X:X: read tcp X.X.X.X:X->X.X.X.X:X: i/o timeout
• HTTP: '499 Client Closed Request' caused by: context canceled
• HTTP: ReverseProxy read error during body copy: read tcp X.X.X.X:X->X.X.X.X:X: use of closed network connection

SwarmMode

In v3, the provider Docker has been split into 2 providers:

• Docker provider (without Swarm support)
• Swarm provider (Swarm support only)

An example usage of v2 Docker provider with Swarm

File (YAML)

providers:
  docker:
    swarmMode: true

File (TOML)

[providers.docker]
    swarmMode=true

CLI

--providers.docker.swarmMode=true

This configuration is now unsupported and would prevent Traefik to start.

Remediation

In v3, the swarmMode should not be used with the Docker provider, and, to use Swarm, the Swarm provider should be used instead.

An example usage of the Swarm provider

File (YAML)

providers:
  swarm:
    endpoint: "tcp://127.0.0.1:2377"

File (TOML)

[providers.swarm]
    endpoint="tcp://127.0.0.1:2377"

CLI

--providers.swarm.endpoint=tcp://127.0.0.1:2377

TLS.CAOptional

Docker provider tls.CAOptional option has been removed in v3, as TLS client authentication is a server side option (see https://pkg.go.dev/crypto/tls#ClientAuthType).

An example usage of the TLS.CAOptional option

File (YAML)

providers:
  docker:
    tls: 
      caOptional: true

File (TOML)

[providers.docker.tls]
    caOptional=true

CLI

--providers.docker.tls.caOptional=true

Remediation

The tls.caOptional option should be removed from the Docker provider install configuration.

Kubernetes Gateway API

Experimental Channel Resources (TLSRoute and TCPRoute)

In v3, the Kubernetes Gateway API provider does not enable support for the experimental channel API resources by default.

Remediation

The experimentalChannel option should be used to enable the support for the experimental channel API resources.

An example usage of the Kubernetes Gateway API provider with experimental channel support enabled

File (YAML)

providers:
  kubernetesGateway:
    experimentalChannel: true

File (TOML)

[providers.kubernetesGateway]
    experimentalChannel = true
  # ...

CLI

--providers.kubernetesgateway.experimentalchannel=true

Experimental Configuration

HTTP3

In v3, HTTP/3 is no longer an experimental feature. It can be enabled on entry points without the associated experimental.http3 option, which is now removed. It is now unsupported and would prevent Traefik to start.

An example usage of v2 Experimental http3 option

File (YAML)

experimental:
  http3: true

File (TOML)

[experimental]
    http3=true

CLI

--experimental.http3=true

Remediation

The http3 option should be removed from the install configuration experimental section. To configure http3, please checkout the entrypoint configuration documentation.

Consul provider

namespace

The Consul provider namespace option was deprecated in v2 and is now removed in v3. It is now unsupported and would prevent Traefik to start.

An example usage of v2 Consul namespace option

File (YAML)

consul:
  namespace: foobar

File (TOML)

[consul]
    namespace=foobar

CLI

--consul.namespace=foobar

Remediation

In v3, the namespaces option should be used instead of the namespace option.

An example usage of Consul namespaces option

File (YAML)

consul:
  namespaces:
    - foobar

File (TOML)

[consul]
    namespaces=["foobar"]

CLI

--consul.namespaces=foobar

TLS.CAOptional

Consul provider tls.CAOptional option has been removed in v3, as TLS client authentication is a server side option (see https://pkg.go.dev/crypto/tls#ClientAuthType).

An example usage of the TLS.CAOptional option

File (YAML)

providers:
  consul:
    tls: 
      caOptional: true

File (TOML)

[providers.consul.tls]
    caOptional=true

CLI

--providers.consul.tls.caOptional=true

Remediation

The tls.caOptional option should be removed from the Consul provider install configuration.

ConsulCatalog provider

namespace

The ConsulCatalog provider namespace option was deprecated in v2 and is now removed in v3. It is now unsupported and would prevent Traefik to start.

An example usage of v2 ConsulCatalog namespace option

File (YAML)

consulCatalog:
  namespace: foobar

File (TOML)

[consulCatalog]
    namespace=foobar

CLI

--consulCatalog.namespace=foobar

Remediation

In v3, the namespaces option should be used instead of the namespace option.

An example usage of ConsulCatalog namespaces option

File (YAML)

consulCatalog:
  namespaces:
    - foobar

File (TOML)

[consulCatalog]
    namespaces=["foobar"]

CLI

--consulCatalog.namespaces=foobar

Endpoint.TLS.CAOptional

ConsulCatalog provider endpoint.tls.CAOptional option has been removed in v3, as TLS client authentication is a server side option (see https://pkg.go.dev/crypto/tls#ClientAuthType).

An example usage of the Endpoint.TLS.CAOptional option

File (YAML)

providers:
  consulCatalog:
    endpoint:
      tls: 
        caOptional: true

File (TOML)

[providers.consulCatalog.endpoint.tls]
    caOptional=true

CLI

--providers.consulCatalog.endpoint.tls.caOptional=true

Remediation

The endpoint.tls.caOptional option should be removed from the ConsulCatalog provider install configuration.

Nomad provider

namespace

The Nomad provider namespace option was deprecated in v2 and is now removed in v3. It is now unsupported and would prevent Traefik to start.

An example usage of v2 Nomad namespace option

File (YAML)

nomad:
  namespace: foobar

File (TOML)

[nomad]
    namespace=foobar

CLI

--nomad.namespace=foobar

Remediation

In v3, the namespaces option should be used instead of the namespace option.

An example usage of Nomad namespaces option

File (YAML)

nomad:
  namespaces:
    - foobar

File (TOML)

[nomad]
    namespaces=["foobar"]

CLI

--nomad.namespaces=foobar

Endpoint.TLS.CAOptional

Nomad provider endpoint.tls.CAOptional option has been removed in v3, as TLS client authentication is a server side option (see https://pkg.go.dev/crypto/tls#ClientAuthType).

An example usage of the Endpoint.TLS.CAOptional option

File (YAML)

providers:
  nomad:
    endpoint:
      tls: 
        caOptional: true

File (TOML)

[providers.nomad.endpoint.tls]
    caOptional=true

CLI

--providers.nomad.endpoint.tls.caOptional=true

Remediation

The endpoint.tls.caOptional option should be removed from the Nomad provider install configuration.

Rancher v1 Provider

In v3, the Rancher v1 provider has been removed because Rancher v1 is no longer actively maintained, and Rancher v2 is supported as a standard Kubernetes provider.

An example of Traefik v2 Rancher v1 configuration

File (YAML)

providers:
  rancher: {}

File (TOML)

[providers.rancher]

CLI

--providers.rancher=true

This configuration is now unsupported and would prevent Traefik to start.

Remediation

Rancher 2.x requires Kubernetes and does not have a metadata endpoint of its own for Traefik to query. As such, Rancher 2.x users should utilize the Kubernetes CRD provider directly.

Also, all Rancher provider related configuration should be removed from the install configuration.

Marathon provider

Marathon maintenance ended on October 31, 2021. In v3, the Marathon provider has been removed.

An example of v2 Marathon provider configuration

File (YAML)

providers:
  marathon: {}

File (TOML)

[providers.marathon]

CLI

--providers.marathon=true

This configuration is now unsupported and would prevent Traefik to start.

Remediation

All Marathon provider related configuration should be removed from the install configuration.

HTTP Provider

TLS.CAOptional

HTTP provider tls.CAOptional option has been removed in v3, as TLS client authentication is a server side option (see https://pkg.go.dev/crypto/tls#ClientAuthType).

An example usage of the TLS.CAOptional option

File (YAML)

providers:
  http:
    tls: 
      caOptional: true

File (TOML)

[providers.http.tls]
    caOptional=true

CLI

--providers.http.tls.caOptional=true

Remediation

The tls.caOptional option should be removed from the HTTP provider install configuration.

ETCD Provider

TLS.CAOptional

ETCD provider tls.CAOptional option has been removed in v3, as TLS client authentication is a server side option (see https://pkg.go.dev/crypto/tls#ClientAuthType).

An example usage of the TLS.CAOptional option

File (YAML)

providers:
  etcd:
    tls: 
      caOptional: true

File (TOML)

[providers.etcd.tls]
    caOptional=true

CLI

--providers.etcd.tls.caOptional=true

Remediation

The tls.caOptional option should be removed from the ETCD provider install configuration.

Redis Provider

TLS.CAOptional

Redis provider tls.CAOptional option has been removed in v3, as TLS client authentication is a server side option (see https://pkg.go.dev/crypto/tls#ClientAuthType).

An example usage of the TLS.CAOptional option

File (YAML)

providers:
  redis:
    tls: 
      caOptional: true

File (TOML)

[providers.redis.tls]
    caOptional=true

CLI

--providers.redis.tls.caOptional=true

Remediation

The tls.caOptional option should be removed from the Redis provider install configuration.

InfluxDB v1

InfluxDB v1.x maintenance ended in 2021. In v3, the InfluxDB v1 metrics provider has been removed.

An example of Traefik v2 InfluxDB v1 metrics configuration

File (YAML)

metrics:
  influxDB: {}

File (TOML)

[metrics.influxDB]

CLI

--metrics.influxDB=true

This configuration is now unsupported and would prevent Traefik to start.

Remediation

All InfluxDB v1 metrics provider related configuration should be removed from the install configuration.

Pilot

Traefik Pilot is no longer available since October 4th, 2022.

An example of v2 Pilot configuration

File (YAML)

pilot:
  token: foobar

File (TOML)

[pilot]
    token=foobar

CLI

--pilot.token=foobar

In v2, Pilot configuration was deprecated and ineffective, it is now unsupported and would prevent Traefik to start.

Remediation

All Pilot related configuration should be removed from the install configuration.

Kubernetes Ingress Path Matching

In v3, the Kubernetes Ingress default path matching does not support regexes anymore.

Remediation

Two levels of remediation are possible:

• Interpret the default path matcher PathPrefix with v2 syntax.

This can done globally for all routers with the install configuration or on a per-router basis by using the traefik.ingress.kubernetes.io/router.rulesyntax annotation.

• Adapt the path regex to be compatible with the Go regex syntax and change the default path matcher to use the PathRegexp matcher with the traefik.ingress.kubernetes.io/router.pathmatcher annotation.

Traefik RBAC Update

In v3, the support of TCPServersTransport has been introduced. When using the KubernetesCRD provider, it is therefore necessary to update RBAC and CRDs (See requirements).

Content-Type Auto-Detection

In v3, the Content-Type header is not auto-detected anymore when it is not set by the backend. One should use the ContentType middleware to enable the Content-Type header value auto-detection.

Observability

Open Connections Metric

In v3, the open connections metric has been replaced with a global one because it was erroneously at the HTTP level, and providing misleading information. While previously produced at the entryPoint, router, and service levels, it is now replaced with a global metric. The equivalent to traefik_entrypoint_open_connections, traefik_router_open_connections and traefik_service_open_connections is now traefik_open_connections.

Configuration Reload Failures Metrics

In v3, the traefik_config_reloads_failure_total and traefik_config_last_reload_failure metrics have been suppressed since they could not be implemented.

gRPC Metrics

In v3, the reported status code for gRPC requests is now the value of the Grpc-Status header.

Tracing

In v3, the tracing feature has been revamped and is now powered exclusively by OpenTelemetry (OTel).

Important

Traefik v3 no longer supports direct output formats for specific vendors such as Instana, Jaeger, Zipkin, Haystack, Datadog, and Elastic. Instead, it focuses on pure OpenTelemetry implementation, providing a unified and standardized approach for observability.

Here are two possible transition strategies:

1. OTLP Ingestion Endpoints:

Most vendors now offer OpenTelemetry Protocol (OTLP) ingestion endpoints. You can seamlessly integrate Traefik v3 with these endpoints to continue leveraging tracing capabilities.

2. Legacy Stack Compatibility:

For legacy stacks that cannot immediately upgrade to the latest vendor agents supporting OTLP ingestion, using OpenTelemetry (OTel) collectors with appropriate exporters configuration is a viable solution. This allows continued compatibility with the existing infrastructure.

Please check the OpenTelemetry Tracing provider documentation for more information.

Internal Resources Observability

In v3, observability for internal routers or services (e.g.: ping@internal) is disabled by default. To enable it one should use the new addInternals option for AccessLogs, Metrics or Tracing. Please take a look at the observability documentation for more information:

• AccessLogs
• Metrics
• Tracing

Access logs

In v3, the ServiceURL field is not an object anymore but a string representation. An update may be required if you index access logs.

Router Rule Matchers

In v3, a new rule matchers syntax has been introduced for HTTP and TCP routers. The default rule matchers syntax is now the v3 one, but for backward compatibility this can be configured. The v2 rule matchers syntax is deprecated and its support will be removed in the next major version. For this reason, we encourage migrating to the new syntax.

By default, the defaultRuleSyntax install option is automatically set to v3, meaning that the default rule is the new one.

New V3 Syntax Notable Changes

The Headers and HeadersRegexp matchers have been renamed to Header and HeaderRegexp respectively.

PathPrefix no longer uses regular expressions to match path prefixes.

Path and PathPrefix no longer support path parameter placeholders (e.g., {id}, {name}). Routes using placeholders like Path(`/route/{id}`) will not match in v3 syntax. Use PathRegexp instead for dynamic path segments.

QueryRegexp has been introduced to match query values using a regular expression.

HeaderRegexp, HostRegexp, PathRegexp, QueryRegexp, and HostSNIRegexp matchers now uses the Go regexp syntax.

All matchers now take a single value (except Header, HeaderRegexp, Query, and QueryRegexp which take two) and should be explicitly combined using logical operators to mimic previous behavior.

Query can take a single value to match is the query value that has no value (e.g. /search?mobile).

HostHeader has been removed, use Host instead.

Remediation

Configure the Default Syntax In Install Configuration

The default rule matchers syntax is the expected syntax for any router that is not self opt-out from this default value. It can be configured in the install configuration.

An example configuration for the default rule matchers syntax

File (YAML)

# install configuration
core:
  defaultRuleSyntax: v2

File (TOML)

# install configuration
[core]
    defaultRuleSyntax="v2"

CLI

# install configuration
--core.defaultRuleSyntax=v2

Configure the Syntax Per Router

The rule syntax can also be configured on a per-router basis. This allows you to have heterogeneous router configurations and ease migration.

An example router with syntax configuration

Docker & Swarm

labels:
  - "traefik.http.routers.test.ruleSyntax=v2"

Kubernetes

apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
  name: test.route
  namespace: default

spec:
  routes:
    - match: PathPrefix(`/foo`, `/bar`)
      syntax: v2
      kind: Rule

Consul Catalog

- "traefik.http.routers.test.ruleSyntax=v2"

File (YAML)

http:
  routers:
    test:
      ruleSyntax: v2

File (TOML)

[http.routers]
  [http.routers.test]
    ruleSyntax = "v2"

Migrate Path Placeholders to PathRegexp

In v2, Path and PathPrefix supported path parameter placeholders like {id} for matching dynamic path segments. In v3, this is no longer supported and PathRegexp should be used instead.

Migrating a route with path placeholders

v2 syntax (no longer works in v3):

match: Host(`example.com`) && Path(`/products/{id}`)

v3 syntax using PathRegexp:

match: Host(`example.com`) && PathRegexp(`^/products/[^/]+$`)

For more complex patterns with multiple placeholders:

v2 syntax:

match: Host(`example.com`) && Path(`/users/{userId}/orders/{orderId}`)

v3 syntax:

match: Host(`example.com`) && PathRegexp(`^/users/[^/]+/orders/[^/]+$`) ## matches any non-slash characters
match: Host(`example.com`) && PathRegexp(`^/users/[a-zA-Z0-9_-]+/orders/[a-zA-Z0-9_-]+$`) ## restricts to alphanumeric, hyphens, and underscores

IPWhiteList

In v3, we renamed the IPWhiteList middleware to IPAllowList without changing anything to the configuration.

Deprecated Options Removal

• The tracing.datadog.globaltag option has been removed.
• The tls.caOptional option has been removed from the ForwardAuth middleware, as well as from the HTTP, Consul, Etcd, Redis, ZooKeeper, Consul Catalog, and Docker providers.
• sslRedirect, sslTemporaryRedirect, sslHost, sslForceHost and featurePolicy options of the Headers middleware have been removed.
• The forceSlash option of the StripPrefix middleware has been removed.
• The preferServerCipherSuites option has been removed.

TCP LoadBalancer terminationDelay option

The TCP LoadBalancer terminationDelay option has been deprecated. This option can now be configured directly on the TCPServersTransport level, please take a look at this documentation

Kubernetes CRDs API Group traefik.containo.us

In v3, the Kubernetes CRDs API Group traefik.containo.us has been removed. Please use the API Group traefik.io instead.

Kubernetes Ingress API Group networking.k8s.io/v1beta1

In v3, the Kubernetes Ingress API Group networking.k8s.io/v1beta1 (removed since Kubernetes v1.22) support has been removed.

Please use the API Group networking.k8s.io/v1 instead.

Traefik CRD API Version apiextensions.k8s.io/v1beta1

In v3, the Traefik CRD API Version apiextensions.k8s.io/v1beta1 (removed since Kubernetes v1.22) support has been removed.

Please use the CRD definition with the API Version apiextensions.k8s.io/v1 instead.

Kubernetes Provider RBACs

Starting with v3.1, Traefik's Kubernetes Providers use the EndpointSlices API (requires Kubernetes >=v1.21) for service endpoint discovery. This change also introduces NodePort load-balancing capabilities.

The following RBAC updates are required for all Kubernetes providers:

• Remove endpoints permissions and add endpointslices:

# Remove this section from your RBAC
# - apiGroups: [""]
#   resources: ["endpoints"]
#   verbs: ["get", "list", "watch"]

# Add this section instead
- apiGroups:
    - discovery.k8s.io
  resources:
    - endpointslices
  verbs:
    - list
    - watch

• Add nodes permissions for NodePort support:

- apiGroups:
    - ""
  resources:
    - nodes
  verbs:
    - get
    - list
    - watch

Affected Providers

These changes apply to:

• KubernetesIngress provider
• KubernetesCRD provider
• KubernetesGateway provider

Gateway API: KubernetesGateway Provider

The KubernetesGateway Provider is no longer experimental in v3.1 and can be enabled without the experimental.kubernetesgateway option.

Deprecated Configuration:

Experimental kubernetesgateway option (deprecated)

File (YAML)

experimental:
  kubernetesgateway: true

File (TOML)

[experimental]
    kubernetesgateway=true

CLI

--experimental.kubernetesgateway=true

Migration Steps:

1. Remove the kubernetesgateway option from the experimental section
2. Configure the provider using the KubernetesGateway Provider documentation

IngressClass Lookup

The disableIngressClassLookup option has been deprecated and will be removed in the next major version.

Migration Required:

• Old: disableIngressClassLookup
• New: disableClusterScopeResources

The new option provides broader control over cluster scope resources discovery, including both IngressClass and Nodes resources.

Kubernetes CRD Provider

New optional fields have been added to several CRDs. These updates are backward compatible and only add new functionality.

Apply the latest CRDs:

kubectl apply -f https://raw.githubusercontent.com/traefik/traefik/v3.3/docs/content/reference/dynamic-configuration/kubernetes-crd-definition-v1.yml

Updated Resources:

• TraefikService (PR #11032)
• RateLimit & InFlightReq middlewares (PR #9747)
• Compress middleware (PR #10943)

Kubernetes Gateway Provider Standard Channel

Starting with v3.2, the Kubernetes Gateway Provider now supports GRPCRoute resources.

Therefore, in the corresponding RBACs (see KubernetesGateway provider RBACs), the grcroutes and grpcroutes/status rights have to be added.

Required RBAC Updates:

...
- apiGroups:
    - gateway.networking.k8s.io
  resources:
    - grpcroutes
  verbs:
    - get
    - list
    - watch
- apiGroups:
    - gateway.networking.k8s.io
  resources:
    - grpcroutes/status
  verbs:
    - update
...

Kubernetes Gateway Provider Experimental Channel

Due to breaking changes in Kubernetes Gateway v1.2.0-rc1, Traefik v3.3 only supports Kubernetes Gateway v1.2.x when experimental features are enabled.

New Feature: BackendTLSPolicy Support

The provider now supports BackendTLSPolicy resources.

Therefore, in the corresponding RBACs (see KubernetesGateway provider RBACs), the backendtlspolicies and backendtlspolicies/status rights have to be added.

Required RBAC Updates:

  ...
  - apiGroups:
      - ""
    resources:
      - configmaps
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - gateway.networking.k8s.io
    resources:
      - backendtlspolicies
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - gateway.networking.k8s.io
    resources:
      - backendtlspolicies/status
    verbs:
      - update
  ...

X-Forwarded-Prefix Header Changes

In v3.2.1, the X-Forwarded-Prefix header is now handled like other X-Forwarded-* headers - Traefik removes it when sent from untrusted sources.

This change improves security by preventing header spoofing from untrusted clients. Refer to the Forwarded headers documentation for configuration details.

Swarm Provider Label Updates

In v3.2.2, Swarm-specific labels have been deprecated and will be removed in a future version.

Migration Required:

ACME DNS Certificate Resolver

In v3.3, DNS challenge configuration options have been reorganized for better clarity.

Migration Required:

Tracing Global Attributes

In v3.3, the tracing configuration has been clarified to better reflect its purpose.

Migration Required:

• Old: tracing.globalAttributes
• New: tracing.resourceAttributes

The old option name was misleading as it specifically adds resource attributes for the collector, not global span attributes.

OpenTelemetry Request Duration Metric

In v3.3.4, the OpenTelemetry Request Duration metric unit has been standardized to match other providers and naming conventions.

Change Details:

• Metric: traefik_(entrypoint|router|service)_request_duration_seconds
• Old Unit: Milliseconds
• New Unit: Seconds

This change ensures consistency across all metrics providers and follows standard naming conventions.

Compress Middleware Default Encodings

In v3.3.5, the default compression algorithms have been reordered to favor gzip compression.

New Default: gzip, br, zstd

This change affects requests that either:

• Don't specify preferred algorithms in the Accept-Encoding header
• Have no order preference in their Accept-Encoding header

The reordering helps ensure better compatibility with older clients that may not support newer compression algorithms.

Request Path Sanitization

Starting with v3.3.6, incoming request paths are now automatically cleaned before processing for security and consistency.

What's Changed:

The following path segments are now interpreted and collapsed:

• /../ (parent directory references)
• /./ (current directory references)
• Duplicate slash segments (//)

Disabling Sanitization:

# EntryPoint HTTP configuration
entryPoints:
  web:
    address: ":80"
    http:
      sanitizePath: false  # Not recommended

Security Warning

Setting sanitizePath: false is not safe. This option should only be used with legacy clients that don't properly URL-encode data. Always ensure requests are properly URL-encoded instead of disabling this security feature.

Example Risk: Base64 data containing "/" characters can lead to unsafe routing when path sanitization is disabled and the data isn't URL-encoded.

Kubernetes CRD Provider

Load-Balancing Strategy Updates

Starting with v3.4, HTTP service definitions now support additional load-balancing strategies for better traffic distribution.

Apply Updated CRDs:

kubectl apply -f https://raw.githubusercontent.com/traefik/traefik/v3.4/docs/content/reference/dynamic-configuration/kubernetes-crd-definition-v1.yml

New Strategy Values:

• wrr (Weighted Round Robin)
• p2c (Power of Two Choices)

Deprecation

The RoundRobin strategy is deprecated but still supported (equivalent to wrr). It will be removed in the next major release.

Refer to the HTTP Services Load Balancing documentation for detailed information.

ServersTransport CA Certificate Configuration

A new rootCAs option has been added to the ServersTransport and ServersTransportTCP CRDs. It supports both ConfigMaps and Secrets for CA certificates and replaces the rootCAsSecrets option.

Apply Updates:

# Update CRDs
kubectl apply -f https://raw.githubusercontent.com/traefik/traefik/v3.4/docs/content/reference/dynamic-configuration/kubernetes-crd-definition-v1.yml

# Update RBACs
kubectl apply -f https://raw.githubusercontent.com/traefik/traefik/v3.4/docs/content/reference/dynamic-configuration/kubernetes-crd-rbac.yml

New Configuration Format:

---
apiVersion: traefik.io/v1alpha1
kind: ServersTransport
metadata:
  name: foo
  namespace: bar
spec:
  rootCAs:
    - configMap: ca-config-map
    - secret: ca-secret
      
---      
apiVersion: traefik.io/v1alpha1
kind: ServersTransportTCP
metadata:
  name: foo
  namespace: bar
spec:
  rootCAs:
    - configMap: ca-config-map
    - secret: ca-secret

Deprecation

The rootCAsSecrets option (Secrets only) is still supported but deprecated. It will be removed in the next major release.

Rule Syntax Configuration

In v3.4, rule syntax configuration options will be removed in the next major version.

Deprecated Options:

• core.defaultRuleSyntax (static configuration)
• ruleSyntax (router option)

These options were transitional helpers for migrating from v2 to v3 syntax. Please ensure all router rules use v3 syntax before the next major release.

Request Path Normalization

Starting with v3.4.1, request paths are now normalized according to RFC 3986 standards for better consistency and security.

Normalization Process:

1. Unreserved Character Decoding: Characters like %2E (.) are decoded to their literal form
2. Case Normalization: Percent-encoded characters are uppercased (%2e becomes %2E)

This follows RFC 3986 percent-encoding normalization and case normalization standards.

Processing Order:

1. Path normalization (cannot be disabled)
2. Path sanitization (if enabled)

Reserved Character Handling in Routing

Starting with v3.4.1, reserved characters (per RFC 3986) remain encoded during router rule matching to prevent routing ambiguity.

Why This Matters: Reserved characters change the meaning of request paths when decoded. Keeping them encoded during routing prevents security vulnerabilities and ensures predictable routing behavior.

Request Path Matching Examples

The following table illustrates how path matching behavior has changed:

MultiPath TCP

Since v3.4.5, the MultiPath TCP support introduced with v3.4.2 has been removed. It appears that enabling MPTCP on some platforms can cause Traefik to stop with the following error logs message:

• set tcp X.X.X.X:X->X.X.X.X:X: setsockopt: operation not supported

However, it can be re-enabled by setting the multipathtcp variable in the GODEBUG environment variable, see the related go documentation.

Observability

TraceVerbosity on Routers and Entrypoints

Starting with v3.5.0, a new traceVerbosity option is available for both entrypoints and routers. This option allows you to control the level of detail for tracing spans. Routers can override the value inherited from their entrypoint.

Impact:

• If you rely on tracing, review your configuration to explicitly set the desired verbosity level.
• Existing configurations will default to minimal unless overridden, which will result in fewer spans being generated than before.

Possible values are:

• minimal: produces a single server span and one client span for each request processed by a router.
• detailed: enables the creation of additional spans for each middleware executed for each request processed by a router.

See the updated documentation for entrypoints and dynamic routers.

K8s Resource Attributes

Since v3.5.0, the semconv attributes k8s.pod.name and k8s.pod.uid are injected automatically in OTel resource attributes when OTel tracing/logs/metrics are enabled.

For that purpose, the following right has to be added to the Traefik Kubernetes RBACs:

  ...
  - apiGroups:
      - ""
    resources:
      - pods
    verbs:
      - get
  ...

Deprecation of ProxyProtocol option

Starting with v3.5.2, the proxyProtocol option for TCP LoadBalancer is deprecated. This option can now be configured at the TCPServersTransport level, please check out the documentation for more details.

Kubernetes CRD Provider

To use the new proxyprotocol option in the Kubernetes CRD provider, you need to update your CRDs.

Apply Updated CRDs:

kubectl apply -f https://raw.githubusercontent.com/traefik/traefik/v3.5/docs/content/reference/dynamic-configuration/kubernetes-crd-definition-v1.yml

Certificate Metric Renamed with OpenTelemetry

Starting with v3.5.4, and when using OpenTelemetry, the traefik_tls_certs_not_after_milliseconds metric is renamed to traefik_tls_certs_not_after_seconds. This change aligns the metric name with its real unit precision, which is in seconds.

Kubernetes Gateway API Provider

Starting with v3.6.0, the Kubernetes Gateway API provider only supports version v1.4.0 of the specification, which requires the Gateway API CRDs to be updated.

Apply Updated CRDs:

kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.4.0/standard-install.yaml

For the experimental channel:

kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.4.0/experimental-install.yaml

Kubernetes CRD Provider

To use the new leasttime load-balancer algorithm with the Kubernetes CRD provider, you need to update your CRDs.

Apply Updated CRDs:

kubectl apply -f https://raw.githubusercontent.com/traefik/traefik/v3.6/docs/content/reference/dynamic-configuration/kubernetes-crd-definition-v1.yml

Ingress NGINX Provider

The KubernetesIngressNGINX Provider is no longer experimental in v3.6.2 and can be enabled without the experimental.kubernetesIngressNGINX option.

Deprecated Configuration:

Experimental kubernetesIngressNGINX option (deprecated)

File (YAML)

experimental:
  kubernetesIngressNGINX: true

File (TOML)

[experimental]
    kubernetesIngressNGINX=true

CLI

--experimental.kubernetesIngressNGINX=true

Migration Steps:

1. Remove the kubernetesIngressNGINX option from the experimental section
2. Configure the provider using the kubernetesIngressNGINX Provider documentation

Encoded Characters in Request Path

Starting with v3.6.4, for security reasons, Traefik now rejects requests with a path containing a specific set of encoded characters by default.

When such a request is received, Traefik responds with a 400 Bad Request status code.

Here is the list of the encoded characters that are rejected by default, along with the corresponding configuration option to allow them:

Please check out the entrypoint encodedCharacters option documentation for more details.

Encoded Characters Configuration Default Values

Since v3.6.7, the options for encoded characters now have a true default value. This means that Traefik will not reject requests with a path containing a specific set of encoded characters by default. It is now up to the users to configure the security hardening of encoded characters.

Here is the list of the encoded characters that can be configured to false to disallow them:

Note: This check is not done against query parameters, but only against the request path as defined in RFC3986 section-3.

Please check out the entrypoint encodedCharacters option documentation for more details.

Health Check Request Path

Since v3.6.8, the configured path for the health check request is now verified to be a relative URL, and the health check will fail if it is not.

maxResponseBodySize configuration on ForwardAuth middleware

In v3.6.9, a new maxResponseBodySize option has been added to the ForwardAuth middleware configuration. The default value for this option is -1, which means there is no limit to the response body size. However, it is strongly recommended to set this option to a suitable value to avoid performance and security issues, such as DoS attacks and memory exhaustion.

Please check out the ForwardAuth middleware documentation for more details.

Kubernetes CRD Provider

To use the new maxResponseBodySize option in the ForwardAuth middleware with the Kubernetes CRD provider, you need to update your CRDs.

Apply Updated CRDs:

kubectl apply -f https://raw.githubusercontent.com/traefik/traefik/v3.6/docs/content/reference/dynamic-configuration/kubernetes-crd-definition-v1.yml

Kubernetes CRD: Chain middleware and allowCrossNamespace

In v3.6.14, the Chain middleware now honors the Kubernetes CRD provider's allowCrossNamespace option. Previously, a Chain could reference middlewares in other namespaces regardless of the allowCrossNamespace configuration.

If allowCrossNamespace is set to false (the default) and a Chain middleware references a middleware in a different namespace from its own, the whole Chain is now rejected and an error is logged.

ForwardAuth middleware: trustForwardHeader

Starting with v3.6.14, the trustForwardHeader option has been deprecated and will be removed in the next major version. Configure the trusted IPs at the EntryPoint level by using the forwardedHeaders.trustedIPs option, and set trustForwardHeader to true on this middleware.

When trustForwardHeader is not explicitly set, Traefik logs a warning as its behavior is inconsistent: some X-Forwarded-* headers (e.g. X-Forwarded-For, X-Forwarded-Proto) are removed while others (e.g. X-Forwarded-Prefix) are forwarded untouched.

To silence the warning and avoid security concerns, explicitly set trustForwardHeader to true or false in your ForwardAuth middleware configuration.

Please check out the ForwardAuth middleware documentation for more details.

In v3.6.15, a new errorRequestHeaders option has been added to the Errors middleware.

By default, the behavior is unchanged: all original request headers are forwarded to the error page service. If the error page service is in a separate trust domain, consider using errorRequestHeaders to restrict which headers are forwarded.

Please check out the Error Pages middleware documentation for more details.

Docker provider: minimum Docker Engine version

Starting with v3.6.16, the Docker provider requires Docker API version v1.40 or above (Docker Engine v19.03). Users running older (end of life) versions of Docker Engine should update their Docker Engine or use the DOCKER_API_VERSION environment variable to override the API version used by Traefik.

Ingress NGINX Provider

Starting with v3.7.0, the Ingress NGINX provider now supports the nginx.ingress.kubernetes.io/custom-headers annotation to add custom headers to the response forwarded to the client.

Therefore, in the corresponding RBACs (see KubernetesIngressNGINX provider RBACs) the configmaps right has been added.

Required RBAC Updates:

  ...
  - apiGroups:
      - ""
    resources:
      - configmaps
    verbs:
      - list
      - watch  
  ...

Kubernetes Gateway API Provider

Starting with v3.7.0, the Kubernetes Gateway API provider supports version v1.5.1 of the specification, which requires the Gateway API CRDs to be updated.

TLSRoute has graduated to the Standard channel and no longer requires the experimentalChannel option. The experimentalChannel option is now only needed for TCPRoute.

Apply Updated CRDs:

kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.5.1/standard-install.yaml

For the experimental channel:

kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.5.1/experimental-install.yaml

Kubernetes CRD Provider

To use the new options of the retry middleware or the new ingressClassName field with the Kubernetes CRD provider, you need to update your CRDs.

Apply Updated CRDs:

kubectl apply -f https://raw.githubusercontent.com/traefik/traefik/v3.7/docs/content/reference/dynamic-configuration/kubernetes-crd-definition-v1.yml

Wildcard Host and HostSNI

Since v3.7.0, the Host and HostSNI matchers support wildcard subdomain matching (e.g., *.example.com). This allows matching any direct subdomain of a domain with a single-level wildcard prefix. For example, *.example.com matches foo.example.com but not foo.bar.example.com or example.com itself.

This feature is only available with the v3 rule syntax (the default).

TLSOptions with Wildcard Domains

Since v3.7.0, TLSOptions can now be associated with routers using wildcard Host and HostSNI matchers (e.g., Host(*.example.com)). This enables configuring different TLS options for wildcard domains.

Previously, TLSOptions selection was limited to exact Host matches, and using HostRegexp or wildcards would fall back to the default TLS options with a warning message like: No domain found in rule HostRegexp(...) the TLS option foo cannot be applied.

Note: TLSOptions for HostRegexp matchers remains unsupported. Use wildcard Host matchers as an alternative.

Kubernetes providers: crossProviderNamespaces

In v3.7.1, a new crossProviderNamespaces option is available on the Kubernetes CRD, Ingress, and Gateway providers.

Traefik offers the possibility to reference resources from one provider to another (cross-provider references).

However, in the context of Kubernetes providers, those references (e.g. myservice@kubernetescrd) allow a user to cross namespace boundaries, as well as exposing @internal services, that only the operator should be able to expose.

This new crossProviderNamespaces option restricts in which namespaces Kubernetes resources are allowed to use cross-provider references.

The behavior is as follows:

Please check out the Kubernetes CRD, Kubernetes Ingress, and Kubernetes Gateway provider documentation for more details.

Kubernetes Gateway API Provider

Starting with v3.7.3, the QPS and Burst values of the Kubernetes client used by the Kubernetes Gateway API provider have been increased to 50 and 100 respectively (10x the default values of the Kubernetes client).

The Kubernetes Gateway API provider writes status updates intensively to comply with the Kubernetes Gateway API specification. This change helps avoid performance issues related to Kubernetes API rate limiting, which can increase the setup time when a new routing configuration is built.

These values are configurable through the kubernetesGateway.qps and kubernetesGateway.burst provider options.

BasicAuth Middleware

From version v3.7.3 onwards, the BasicAuth middleware requires a non-empty users configuration in order to be built successfully. Previously, the middleware would be built successfully but always return a 401 status code for any request. Now, an error occurs and any routers using it will be unmounted. For the same request, a 404 status code is served instead of a 401 status code.

StripPrefix and StripPrefixRegex Middleware

From version v3.7.3 onwards, the StripPrefix middleware and the StripPrefixRegex middleware reject requests (400 Bad Request) when stripping the configured prefix produces a path that differs from its normalised form (i.e. a path containing . or .. segments that would be collapsed by normalisation).

This prevents the stripped path from being interpreted as a different resource by the upstream service.

Examples with a configured prefix of /api:

Kubernetes Gateway API Provider: generated service and middleware names

Starting with v3.7.6, the Kubernetes Gateway API provider derives the name of the services and middlewares it generates from the route rule they belong to, instead of from the backend reference alone. This is required to prevent distinct route rules referencing the same backend from colliding on a single generated configuration.

Generated service names were previously built from the backend reference only:

<backend namespace>-<backend name>-<port>

They are now prefixed with the route rule they are generated for:

<route kind>-<route namespace>-<route name>-gw-<gateway namespace>-<gateway name>-ep-<entry point>-<rule index>-<rule hash>-svc-<backend namespace>-<backend name>-<backend index>

For example, a whoami backend in the default namespace previously exposed as default-whoami-http-80@kubernetesgateway is now exposed as httproute-default-http-app-1-gw-default-my-gateway-ep-web-0-af329269dd38031b03e3-svc-default-whoami-0@kubernetesgateway.

Observability

These names are user-visible: they appear in the dashboard and API, in the access logs ServiceName field, and in the service label of the metrics. Dashboards, alerting rules, and log queries that match on Gateway API service names must be updated accordingly.

UnderscoreHeadersStrategy

Deprecated since v3.7.12

Please use the aliasHeadersStrategy option instead, which handles every aliasing character instead of the underscore only.

From version v3.6.20 onwards, a new underscoreHeadersStrategy entry point option defines how request headers with underscores in their names are handled before routing:

• keep (default): request headers with underscores are forwarded as is.
• delete: any request header whose name contains an underscore character is silently removed from the request.
• reject: any request carrying a header whose name contains an underscore character is rejected with a 400 Bad Request response.

The default value is keep, so the existing behavior is preserved.

This option exists because underscores are valid characters in HTTP header names, but Go canonicalizes header names only on dashes. As a result, a middleware managing a header in its dash form (e.g. X-Auth-User set by the ForwardAuth authResponseHeaders option) does not see, and therefore cannot overwrite or remove, an underscore variant of that header (e.g. X_Auth_User).

Many backends map both forms to the same variable (CGI, WSGI, PHP, NGINX, ...): for them, X-Auth-User and X_Auth_User are the same header. Against such a backend, a client can smuggle the underscore variant past a middleware that only manages the dash form, and have the backend read the spoofed value, bypassing the protection the middleware was meant to provide.

Security

When an entry point fronts a backend that interprets underscores and dashes in header names identically, keeping the default keep strategy is not recommended, as it leaves the backend open to the header spoofing described above. Set underscoreHeadersStrategy to delete or reject on such entry points.

Wildcard Host matcher

From version v3.7.7 onwards, the Host matcher treats a bare * as a catch-all, consistent with the TCP HostSNI() matcher. ``Host(``)`` now matches every request regardless of its host, including requests with no host at all.

Previously, the * was treated as a single wildcard label, so Host(`*`) only matched hosts made of a single segment (e.g. localhost) and not multi-segment hosts (e.g. example.com).

Please check out the HTTP routing rules documentation for more details.

Kubernetes CRD: Errors middleware errorRequestHeaders

The errorRequestHeaders option, introduced in v3.6.15 for the Errors middleware, was not exposed on the Kubernetes CRD provider. Starting with v3.7.8, it can now be configured on the Middleware CRD.

To use this new option, the Kubernetes CRDs must be updated in the cluster before upgrading Traefik. To do so, please apply the CRDs manifest for v3.7:

kubectl apply -f https://raw.githubusercontent.com/traefik/traefik/v3.7/docs/content/reference/dynamic-configuration/kubernetes-crd-definition-v1.yml

Please check out the Error Pages middleware documentation for more details.

HTTP/1 CONNECT requests

Starting with v3.7.9, HTTP/1 CONNECT requests are rejected with a 501 Not Implemented response.

HTTP/1 CONNECT requests were not functional before this change, so this rejection makes it explicit.

Kubernetes Gateway API Provider: generated router, middleware, and service names

Starting with v3.7.10, the Kubernetes Gateway API provider derives the hash suffix of the router names it generates from all the route, Gateway, and listener identifying fields, instead of from the route rule alone. This is required to prevent distinct routes whose namespace, name, Gateway, and listener happen to concatenate to the same string from colliding on a single generated router name.

Generated router names keep the same overall shape:

<route kind>-<route namespace>-<route name>-gw-<gateway namespace>-<gateway name>-ep-<entry point>-<rule index>-<hash>

but the <hash> suffix changes, since it is no longer computed from the route rule alone.

For example, an HTTPRoute named http-app-1 in the default namespace previously generated the router httproute-default-http-app-1-gw-default-my-gateway-ep-web-0-af329269dd38031b03e3 and is now generated as httproute-default-http-app-1-gw-default-my-gateway-ep-web-0-799bcbf0c2317c5d1c93.

Middleware names are derived from the router name, so they change accordingly, for example <router name>-requestheadermodifier-0.

Generated service names were previously built from the backend reference only:

<backend namespace>-<backend name>-<port>

They are now prefixed with the route rule they are generated for:

<router name>-svc-<backend namespace>-<backend name>-<backend index>

For example, a whoami backend in the default namespace previously exposed as default-whoami-http-80@kubernetesgateway is now exposed as httproute-default-http-app-1-gw-default-my-gateway-ep-web-0-799bcbf0c2317c5d1c93-svc-default-whoami-0@kubernetesgateway.

Observability

These names are user-visible: they appear in the dashboard and API, in the access logs RouterName and ServiceName fields, and in the router/service labels of the metrics. Dashboards, alerting rules, and log queries that match on Gateway API router, middleware, or service names must be updated accordingly.

Kubernetes Gateway API Provider

Starting with v3.7.10, the Kubernetes Gateway API provider supports version v1.6.1 of the specification.

TCPRoute graduated to the Standard channel in Gateway API v1.6.0, with a new v1 version. Traefik v3.7 still watches TCPRoute through its v1alpha2 version, which the Standard channel CRDs no longer serve.

If you use TCPRoute and upgrade to Gateway API v1.6.1 CRDs, you must install the experimental channel CRDs. The experimentalChannel option remains required to enable TCPRoute support in Traefik.

Standard channel CRDs with experimentalChannel enabled

Traefik cannot watch the v1alpha2 version of TCPRoute, and the Kubernetes Gateway provider never completes its startup: no Gateway API resource is served, not only TCPRoute. No error is logged, Traefik keeps running, and other providers are unaffected.

Traefik v3.7 remains backward compatible with the v1.5.x CRDs: upgrading the CRDs in the cluster is only required to rely on the v1.6.1 resources.

Upcoming in v3.8

Traefik v3.8 will require updating the Gateway API CRDs to v1.6.x, and in return the experimentalChannel option will no longer be needed for TCPRoute.

(Optional) Apply v1.6.1 CRDs:

kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.6.1/standard-install.yaml

For the experimental channel (needed for TCPRoute):

kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.6.1/experimental-install.yaml

Safe naming configuration option

Starting with v3.7.11, the new safeNaming provider option enables collision-safe naming for the routers, middlewares and services generated by the Kubernetes CRD provider, instead of the current naming scheme, under which names can collide across namespaces or resources.

Generated names are derived from the identity of the object they come from instead of being flattened and normalized, and the ones generated for a route are derived from the route index instead of its rule. For example, for a Kubernetes Service named whoami and an IngressRoute named test.route, both in the default namespace:

default-whoami-80                          ->    default_whoami_80
default-test-route-6b204d94623b3df4370c    ->    default_test.route_0

The option is disabled by default, which preserves the existing behavior.

Observability

These names are user-visible: they appear in the dashboard and API, in the access logs RouterName and ServiceName fields, and in the router and service labels of the metrics. Dashboards, alerting rules, and log queries that match on Kubernetes CRD router, middleware or service names must be updated accordingly when safeNaming is enabled.

Please check out the safeNaming provider documentation for more details.

Conflicting TLS options

The TLS options are configured on a router, but they are applied during the TLS handshake, before the routing occurs, and are therefore mapped to the host names found in the router rule rather than to the router itself. When several routers on the same entry point serve the same host name with different TLS options, Traefik cannot decide which options to apply, and falls back to the default TLS options for that host name.

The default TLS options being the fallback of the conflict resolution, they should not be less secure than the options they can replace: a router relying on a mutual TLS authentication (clientAuth), for example, no longer enforces it if a conflict on its host name falls back to default TLS options that do not require it. See GHSA-g55h-rg46-x9c5 for more details.

Starting with v3.7.11, the new core.strictTLSOptions install configuration option disables this fallback: the routers involved in the conflict are marked in error and are not built at all.

The option is disabled by default, to preserve the existing behavior, but enabling it is recommended:

File (YAML)

## Install configuration
core:
  strictTLSOptions: true

File (TOML)

## Install configuration
[core]
  strictTLSOptions = true

CLI

## Install configuration
--core.strictTLSOptions=true

Disabled routers

This option fails closed: a conflict disables all the routers serving the conflicting host name on the concerned entry point, until the conflict is resolved.

Please check out the Conflicting TLS Options documentation for more details.

Kubernetes CRD: cluster-wide default TLSOption and TLSStore

In the Kubernetes CRD provider, the TLSOption and the TLSStore named default are cluster-wide, whatever the namespace they are defined in. Anyone allowed to create one in a single namespace can therefore replace the TLS policy, mutual TLS authentication included, of the routers that do not reference TLS options explicitly.

Starting with v3.7.11, the new defaultTLSResourcesNamespace provider option reserves these resources to a namespace the cluster operator controls.

The option is empty by default, to preserve the existing behavior, but setting it is recommended:

File (YAML)

## Install configuration
providers:
  kubernetesCRD:
    defaultTLSResourcesNamespace: traefik

File (TOML)

## Install configuration
[providers.kubernetesCRD]
  defaultTLSResourcesNamespace = "traefik"

CLI

## Install configuration
--providers.kubernetescrd.defaultTLSResourcesNamespace=traefik

Ignored resources

A default resource defined outside of the configured namespace is ignored, and cannot be referenced under its namespaced name either. For a TLSStore, this also applies to the certificates it defines.

Please check out the Kubernetes CRD provider documentation for more details.

AliasHeadersStrategy

From version v3.7.12 onwards, a new aliasHeadersStrategy entry point option deprecates and replaces the underscoreHeadersStrategy option introduced in v3.6.20.

Go canonicalizes header names on dashes only, so it handles X-Auth-User, X_Auth_User and X.Auth.User as three distinct headers, while the backends deriving their variable names from the header names (CGI, WSGI, PHP, NGINX, ...) uppercase the name and replace every character that is neither a letter nor a digit with an underscore: for them, the three names above are the same HTTP_X_AUTH_USER variable.

The underscoreHeadersStrategy option only handles the header names containing an underscore character, which leaves the other aliasing forms untouched. Its behavior is unchanged. Underscores are only one of the characters building such an alias: every character HTTP allows in a header name except the letters, the digits and the dash does, that is !, #, $, %, &, ', *, +, ., ^, _, `, | and ~. The aliasHeadersStrategy option handles them all:

• keep (default): request headers with an aliasing name are forwarded as is.
• delete: any request header whose name contains a character which is neither a letter, a digit, nor a dash is silently removed from the request.
• reject: any request carrying a header whose name contains a character which is neither a letter, a digit, nor a dash is rejected with a 400 Bad Request response.

The default value is keep, so the existing behavior is preserved.

File (YAML)

entryPoints:
  websecure:
    address: ':443'
    http:
      aliasHeadersStrategy: delete

File (TOML)

[entryPoints.websecure]
  address = ":443"

  [entryPoints.websecure.http]
    aliasHeadersStrategy = "delete"

CLI

--entryPoints.websecure.address=:443
--entryPoints.websecure.http.aliasHeadersStrategy=delete

Setting underscoreHeadersStrategy logs a deprecation warning. Configuring both options with different values makes the install configuration invalid.

Traefik now logs a warning at startup for every entry point left without this option configured, as the middlewares managing request headers rely on it to not be spoofed with an aliasing name.

Please check out the entry point aliasHeadersStrategy option and the Headers with Aliasing Names documentation for more details.

TCP and UDP Weighted Services: Negative Weights

Negative weights in a TCP or UDP weighted service are now rejected when the service is built. Such a configuration previously made the load-balancer selection loop endlessly while holding its lock, which consumed a CPU core and blocked every subsequent connection to that service until Traefik was restarted.

A weighted service declaring a negative weight is now reported as disabled in the API, with the error attached, and the routers referencing it are not created. Replace any negative weight with a positive one, or with 0 to take a child service out of the rotation.

TCP and UDP Weighted Services: Child Service Errors

When a child of a TCP or UDP weighted service cannot be built, for instance because it does not exist, the parent weighted service is now reported as disabled in the API, with the error attached, as the HTTP weighted service already was. Such a parent service previously reported neither an error nor a status.

This only changes what is reported: the routers referencing the parent service were already not created.

h2c Upgrade Requests

Starting with v3.7.13, Traefik does not forward the Upgrade: h2c and HTTP2-Settings request headers to the backends anymore.

Traefik does not implement the deprecated h2c upgrade mechanism, it only serves unencrypted HTTP/2 with prior knowledge. Forwarding those headers let a backend accept an upgrade that Traefik itself had not negotiated with the client.

A backend that used to accept such an upgrade now serves those requests over HTTP/1.1. To reach a backend over unencrypted HTTP/2, declare its servers with the h2c scheme:

File (YAML)

## Dynamic configuration
http:
  services:
    my-service:
      loadBalancer:
        servers:
        - url: "h2c://private-ip-server-1:8080"

File (TOML)

## Dynamic configuration
[http.services]
  [http.services.my-service.loadBalancer]

    [[http.services.my-service.loadBalancer.servers]]
      url = "h2c://private-ip-server-1:8080"

The providers exposing a scheme instead of a server URL take it the same way, with the traefik.http.services.<service_name>.loadbalancer.server.scheme=h2c label, the scheme: h2c option of the Kubernetes CRD, or the traefik.ingress.kubernetes.io/service.serversscheme: h2c Kubernetes Ingress annotation.

Request Target Validation

Starting with v3.7.13, a request whose target is rootless, that is a target made of a scheme followed by something that does not start with a slash, is rejected with a 400 Bad Request response. Such a target is none of the four forms allowed by rfc9112#section-3.2.

A rootless target, for instance http:example.com/admin, carries no path, hence escaping the path handling and the routing rules while still designating a resource on the backend. This rejection is not configurable, as allowing such a target would reintroduce the bypass.

As the validation happens before the routing, the rejected requests are not reported in the access logs. They are only visible at the DEBUG log level.

Consul Catalog and Nomad: colliding service configurations

The Consul Catalog and the Nomad providers build the configuration of each service instance separately, and then merge all of them together. Until v3.7.13, the instances were identified by the normalized concatenation of their node, service name and service ID, which is ambiguous: two distinct instances could produce the same identifier, and the configuration of all but the last of them was silently discarded.

Starting with v3.7.13, this identifier is unambiguous, and every instance contributes to the merge.

Newly detected conflicts

The configurations that were previously discarded are now merged. A router or a middleware defined by several instances is only kept if all of them define it identically, and a service only if the instances agree on everything but their servers, which are then pooled together. Otherwise the resource is removed altogether, an error is logged, and a router served before the upgrade can disappear. The merge order changes as well, and with it the order of the servers of a load-balancer.