Traefik 2.11.0 → 3.0.0

Maximum Router Priority Value

Before v2.11.1, the maximum user-defined router priority value is:

• MaxInt32 for 32-bit platforms,
• MaxInt64 for 64-bit platforms.

Please check out the go documentation for more information.

In v2.11.1, Traefik reserves a range of priorities for its internal routers and now, the maximum user-defined router priority value is:

• (MaxInt32 - 1000) for 32-bit platforms,
• (MaxInt64 - 1000) for 64-bit platforms.

EntryPoint.Transport.RespondingTimeouts.<Timeout>

Starting with v2.11.1 the following timeout options are deprecated:

• <entryPoint>.transport.respondingTimeouts.readTimeout
• <entryPoint>.transport.respondingTimeouts.writeTimeout
• <entryPoint>.transport.respondingTimeouts.idleTimeout

They have been replaced by:

• <entryPoint>.transport.respondingTimeouts.http.readTimeout
• <entryPoint>.transport.respondingTimeouts.http.writeTimeout
• <entryPoint>.transport.respondingTimeouts.http.idleTimeout

EntryPoint.Transport.RespondingTimeouts.TCP.LingeringTimeout

Starting with v2.11.1 a new lingeringTimeout entryPoints option has been introduced, with a default value of 2s.

The lingering timeout defines the maximum duration between each TCP read operation on the connection. As a layer 4 timeout, it applies during HTTP handling but respects the configured HTTP server readTimeout.

This change avoids Traefik instances with the default configuration hanging while waiting for bytes to be read on the connection.

We suggest to adapt this value accordingly to your situation. The new default value is purposely narrowed and can close the connection too early.

Increasing the lingeringTimeout value could be the solution notably if you are dealing with the following errors:

• TCP: Error while handling TCP connection: readfrom tcp X.X.X.X:X->X.X.X.X:X: read tcp X.X.X.X:X->X.X.X.X:X: i/o timeout
• HTTP: '499 Client Closed Request' caused by: context canceled
• HTTP: ReverseProxy read error during body copy: read tcp X.X.X.X:X->X.X.X.X:X: use of closed network connection

LingeringTimeout

Starting with v2.11.2 the <entrypoint>.transport.respondingTimeouts.tcp.lingeringTimeout introduced in v2.11.1 has been removed.

RespondingTimeouts.TCP and RespondingTimeouts.HTTP

Starting with v2.11.2 the respondingTimeouts.tcp and respondingTimeouts.http sections introduced in v2.11.1 have been removed. To configure the responding timeouts, please use the respondingTimeouts section.

EntryPoint.Transport.RespondingTimeouts.ReadTimeout

Starting with v2.11.2 the entryPoints readTimeout option default value changed to 60 seconds.

For HTTP, this option defines the maximum duration for reading the entire request, including the body. For TCP, this option defines the maximum duration for the first bytes to be read on the connection.

The default value was previously set to zero, which means no timeout.

This change has been done to avoid Traefik instances with the default configuration to be hanging forever while waiting for bytes to be read on the connection.

Increasing the readTimeout value could be the solution notably if you are dealing with the following errors:

• TCP: Error while handling TCP connection: readfrom tcp X.X.X.X:X->X.X.X.X:X: read tcp X.X.X.X:X->X.X.X.X:X: i/o timeout
• HTTP: '499 Client Closed Request' caused by: context canceled
• HTTP: ReverseProxy read error during body copy: read tcp X.X.X.X:X->X.X.X.X:X: use of closed network connection

SwarmMode

In v3, the provider Docker has been split into 2 providers:

• Docker provider (without Swarm support)
• Swarm provider (Swarm support only)

An example usage of v2 Docker provider with Swarm

File (YAML)

providers:
  docker:
    swarmMode: true

File (TOML)

[providers.docker]
    swarmMode=true

CLI

--providers.docker.swarmMode=true

This configuration is now unsupported and would prevent Traefik to start.

Remediation

In v3, the swarmMode should not be used with the Docker provider, and, to use Swarm, the Swarm provider should be used instead.

An example usage of the Swarm provider

File (YAML)

providers:
  swarm:
    endpoint: "tcp://127.0.0.1:2377"

File (TOML)

[providers.swarm]
    endpoint="tcp://127.0.0.1:2377"

CLI

--providers.swarm.endpoint=tcp://127.0.0.1:2377

TLS.CAOptional

Docker provider tls.CAOptional option has been removed in v3, as TLS client authentication is a server side option (see https://pkg.go.dev/crypto/tls#ClientAuthType).

An example usage of the TLS.CAOptional option

File (YAML)

providers:
  docker:
    tls: 
      caOptional: true

File (TOML)

[providers.docker.tls]
    caOptional=true

CLI

--providers.docker.tls.caOptional=true

Remediation

The tls.caOptional option should be removed from the Docker provider install configuration.

Kubernetes Gateway API

Experimental Channel Resources (TLSRoute and TCPRoute)

In v3, the Kubernetes Gateway API provider does not enable support for the experimental channel API resources by default.

Remediation

The experimentalChannel option should be used to enable the support for the experimental channel API resources.

An example usage of the Kubernetes Gateway API provider with experimental channel support enabled

File (YAML)

providers:
  kubernetesGateway:
    experimentalChannel: true

File (TOML)

[providers.kubernetesGateway]
    experimentalChannel = true
  # ...

CLI

--providers.kubernetesgateway.experimentalchannel=true

Experimental Configuration

HTTP3

In v3, HTTP/3 is no longer an experimental feature. It can be enabled on entry points without the associated experimental.http3 option, which is now removed. It is now unsupported and would prevent Traefik to start.

An example usage of v2 Experimental http3 option

File (YAML)

experimental:
  http3: true

File (TOML)

[experimental]
    http3=true

CLI

--experimental.http3=true

Remediation

The http3 option should be removed from the install configuration experimental section. To configure http3, please checkout the entrypoint configuration documentation.

Consul provider

namespace

The Consul provider namespace option was deprecated in v2 and is now removed in v3. It is now unsupported and would prevent Traefik to start.

An example usage of v2 Consul namespace option

File (YAML)

consul:
  namespace: foobar

File (TOML)

[consul]
    namespace=foobar

CLI

--consul.namespace=foobar

Remediation

In v3, the namespaces option should be used instead of the namespace option.

An example usage of Consul namespaces option

File (YAML)

consul:
  namespaces:
    - foobar

File (TOML)

[consul]
    namespaces=["foobar"]

CLI

--consul.namespaces=foobar

TLS.CAOptional

Consul provider tls.CAOptional option has been removed in v3, as TLS client authentication is a server side option (see https://pkg.go.dev/crypto/tls#ClientAuthType).

An example usage of the TLS.CAOptional option

File (YAML)

providers:
  consul:
    tls: 
      caOptional: true

File (TOML)

[providers.consul.tls]
    caOptional=true

CLI

--providers.consul.tls.caOptional=true

Remediation

The tls.caOptional option should be removed from the Consul provider install configuration.

ConsulCatalog provider

namespace

The ConsulCatalog provider namespace option was deprecated in v2 and is now removed in v3. It is now unsupported and would prevent Traefik to start.

An example usage of v2 ConsulCatalog namespace option

File (YAML)

consulCatalog:
  namespace: foobar

File (TOML)

[consulCatalog]
    namespace=foobar

CLI

--consulCatalog.namespace=foobar

Remediation

In v3, the namespaces option should be used instead of the namespace option.

An example usage of ConsulCatalog namespaces option

File (YAML)

consulCatalog:
  namespaces:
    - foobar

File (TOML)

[consulCatalog]
    namespaces=["foobar"]

CLI

--consulCatalog.namespaces=foobar

Endpoint.TLS.CAOptional

ConsulCatalog provider endpoint.tls.CAOptional option has been removed in v3, as TLS client authentication is a server side option (see https://pkg.go.dev/crypto/tls#ClientAuthType).

An example usage of the Endpoint.TLS.CAOptional option

File (YAML)

providers:
  consulCatalog:
    endpoint:
      tls: 
        caOptional: true

File (TOML)

[providers.consulCatalog.endpoint.tls]
    caOptional=true

CLI

--providers.consulCatalog.endpoint.tls.caOptional=true

Remediation

The endpoint.tls.caOptional option should be removed from the ConsulCatalog provider install configuration.

Nomad provider

namespace

The Nomad provider namespace option was deprecated in v2 and is now removed in v3. It is now unsupported and would prevent Traefik to start.

An example usage of v2 Nomad namespace option

File (YAML)

nomad:
  namespace: foobar

File (TOML)

[nomad]
    namespace=foobar

CLI

--nomad.namespace=foobar

Remediation

In v3, the namespaces option should be used instead of the namespace option.

An example usage of Nomad namespaces option

File (YAML)

nomad:
  namespaces:
    - foobar

File (TOML)

[nomad]
    namespaces=["foobar"]

CLI

--nomad.namespaces=foobar

Endpoint.TLS.CAOptional

Nomad provider endpoint.tls.CAOptional option has been removed in v3, as TLS client authentication is a server side option (see https://pkg.go.dev/crypto/tls#ClientAuthType).

An example usage of the Endpoint.TLS.CAOptional option

File (YAML)

providers:
  nomad:
    endpoint:
      tls: 
        caOptional: true

File (TOML)

[providers.nomad.endpoint.tls]
    caOptional=true

CLI

--providers.nomad.endpoint.tls.caOptional=true

Remediation

The endpoint.tls.caOptional option should be removed from the Nomad provider install configuration.

Rancher v1 Provider

In v3, the Rancher v1 provider has been removed because Rancher v1 is no longer actively maintained, and Rancher v2 is supported as a standard Kubernetes provider.

An example of Traefik v2 Rancher v1 configuration

File (YAML)

providers:
  rancher: {}

File (TOML)

[providers.rancher]

CLI

--providers.rancher=true

This configuration is now unsupported and would prevent Traefik to start.

Remediation

Rancher 2.x requires Kubernetes and does not have a metadata endpoint of its own for Traefik to query. As such, Rancher 2.x users should utilize the Kubernetes CRD provider directly.

Also, all Rancher provider related configuration should be removed from the install configuration.

Marathon provider

Marathon maintenance ended on October 31, 2021. In v3, the Marathon provider has been removed.

An example of v2 Marathon provider configuration

File (YAML)

providers:
  marathon: {}

File (TOML)

[providers.marathon]

CLI

--providers.marathon=true

This configuration is now unsupported and would prevent Traefik to start.

Remediation

All Marathon provider related configuration should be removed from the install configuration.

HTTP Provider

TLS.CAOptional

HTTP provider tls.CAOptional option has been removed in v3, as TLS client authentication is a server side option (see https://pkg.go.dev/crypto/tls#ClientAuthType).

An example usage of the TLS.CAOptional option

File (YAML)

providers:
  http:
    tls: 
      caOptional: true

File (TOML)

[providers.http.tls]
    caOptional=true

CLI

--providers.http.tls.caOptional=true

Remediation

The tls.caOptional option should be removed from the HTTP provider install configuration.

ETCD Provider

TLS.CAOptional

ETCD provider tls.CAOptional option has been removed in v3, as TLS client authentication is a server side option (see https://pkg.go.dev/crypto/tls#ClientAuthType).

An example usage of the TLS.CAOptional option

File (YAML)

providers:
  etcd:
    tls: 
      caOptional: true

File (TOML)

[providers.etcd.tls]
    caOptional=true

CLI

--providers.etcd.tls.caOptional=true

Remediation

The tls.caOptional option should be removed from the ETCD provider install configuration.

Redis Provider

TLS.CAOptional

Redis provider tls.CAOptional option has been removed in v3, as TLS client authentication is a server side option (see https://pkg.go.dev/crypto/tls#ClientAuthType).

An example usage of the TLS.CAOptional option

File (YAML)

providers:
  redis:
    tls: 
      caOptional: true

File (TOML)

[providers.redis.tls]
    caOptional=true

CLI

--providers.redis.tls.caOptional=true

Remediation

The tls.caOptional option should be removed from the Redis provider install configuration.

InfluxDB v1

InfluxDB v1.x maintenance ended in 2021. In v3, the InfluxDB v1 metrics provider has been removed.

An example of Traefik v2 InfluxDB v1 metrics configuration

File (YAML)

metrics:
  influxDB: {}

File (TOML)

[metrics.influxDB]

CLI

--metrics.influxDB=true

This configuration is now unsupported and would prevent Traefik to start.

Remediation

All InfluxDB v1 metrics provider related configuration should be removed from the install configuration.

Pilot

Traefik Pilot is no longer available since October 4th, 2022.

An example of v2 Pilot configuration

File (YAML)

pilot:
  token: foobar

File (TOML)

[pilot]
    token=foobar

CLI

--pilot.token=foobar

In v2, Pilot configuration was deprecated and ineffective, it is now unsupported and would prevent Traefik to start.

Remediation

All Pilot related configuration should be removed from the install configuration.

Kubernetes Ingress Path Matching

In v3, the Kubernetes Ingress default path matching does not support regexes anymore.

Remediation

Two levels of remediation are possible:

• Interpret the default path matcher PathPrefix with v2 syntax.

This can done globally for all routers with the install configuration or on a per-router basis by using the traefik.ingress.kubernetes.io/router.rulesyntax annotation.

• Adapt the path regex to be compatible with the Go regex syntax and change the default path matcher to use the PathRegexp matcher with the traefik.ingress.kubernetes.io/router.pathmatcher annotation.

Traefik RBAC Update

In v3, the support of TCPServersTransport has been introduced. When using the KubernetesCRD provider, it is therefore necessary to update RBAC and CRDs (See requirements).

Content-Type Auto-Detection

In v3, the Content-Type header is not auto-detected anymore when it is not set by the backend. One should use the ContentType middleware to enable the Content-Type header value auto-detection.

Observability

Open Connections Metric

In v3, the open connections metric has been replaced with a global one because it was erroneously at the HTTP level, and providing misleading information. While previously produced at the entryPoint, router, and service levels, it is now replaced with a global metric. The equivalent to traefik_entrypoint_open_connections, traefik_router_open_connections and traefik_service_open_connections is now traefik_open_connections.

Configuration Reload Failures Metrics

In v3, the traefik_config_reloads_failure_total and traefik_config_last_reload_failure metrics have been suppressed since they could not be implemented.

gRPC Metrics

In v3, the reported status code for gRPC requests is now the value of the Grpc-Status header.

Tracing

In v3, the tracing feature has been revamped and is now powered exclusively by OpenTelemetry (OTel).

Important

Traefik v3 no longer supports direct output formats for specific vendors such as Instana, Jaeger, Zipkin, Haystack, Datadog, and Elastic. Instead, it focuses on pure OpenTelemetry implementation, providing a unified and standardized approach for observability.

Here are two possible transition strategies:

1. OTLP Ingestion Endpoints:

Most vendors now offer OpenTelemetry Protocol (OTLP) ingestion endpoints. You can seamlessly integrate Traefik v3 with these endpoints to continue leveraging tracing capabilities.

2. Legacy Stack Compatibility:

For legacy stacks that cannot immediately upgrade to the latest vendor agents supporting OTLP ingestion, using OpenTelemetry (OTel) collectors with appropriate exporters configuration is a viable solution. This allows continued compatibility with the existing infrastructure.

Please check the OpenTelemetry Tracing provider documentation for more information.

Internal Resources Observability

In v3, observability for internal routers or services (e.g.: ping@internal) is disabled by default. To enable it one should use the new addInternals option for AccessLogs, Metrics or Tracing. Please take a look at the observability documentation for more information:

• AccessLogs
• Metrics
• Tracing

Access logs

In v3, the ServiceURL field is not an object anymore but a string representation. An update may be required if you index access logs.

Router Rule Matchers

In v3, a new rule matchers syntax has been introduced for HTTP and TCP routers. The default rule matchers syntax is now the v3 one, but for backward compatibility this can be configured. The v2 rule matchers syntax is deprecated and its support will be removed in the next major version. For this reason, we encourage migrating to the new syntax.

By default, the defaultRuleSyntax install option is automatically set to v3, meaning that the default rule is the new one.

New V3 Syntax Notable Changes

The Headers and HeadersRegexp matchers have been renamed to Header and HeaderRegexp respectively.

PathPrefix no longer uses regular expressions to match path prefixes.

Path and PathPrefix no longer support path parameter placeholders (e.g., {id}, {name}). Routes using placeholders like Path(`/route/{id}`) will not match in v3 syntax. Use PathRegexp instead for dynamic path segments.

QueryRegexp has been introduced to match query values using a regular expression.

HeaderRegexp, HostRegexp, PathRegexp, QueryRegexp, and HostSNIRegexp matchers now uses the Go regexp syntax.

All matchers now take a single value (except Header, HeaderRegexp, Query, and QueryRegexp which take two) and should be explicitly combined using logical operators to mimic previous behavior.

Query can take a single value to match is the query value that has no value (e.g. /search?mobile).

HostHeader has been removed, use Host instead.

Remediation

Configure the Default Syntax In Install Configuration

The default rule matchers syntax is the expected syntax for any router that is not self opt-out from this default value. It can be configured in the install configuration.

An example configuration for the default rule matchers syntax

File (YAML)

# install configuration
core:
  defaultRuleSyntax: v2

File (TOML)

# install configuration
[core]
    defaultRuleSyntax="v2"

CLI

# install configuration
--core.defaultRuleSyntax=v2

Configure the Syntax Per Router

The rule syntax can also be configured on a per-router basis. This allows you to have heterogeneous router configurations and ease migration.

An example router with syntax configuration

Docker & Swarm

labels:
  - "traefik.http.routers.test.ruleSyntax=v2"

Kubernetes

apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
  name: test.route
  namespace: default

spec:
  routes:
    - match: PathPrefix(`/foo`, `/bar`)
      syntax: v2
      kind: Rule

Consul Catalog

- "traefik.http.routers.test.ruleSyntax=v2"

File (YAML)

http:
  routers:
    test:
      ruleSyntax: v2

File (TOML)

[http.routers]
  [http.routers.test]
    ruleSyntax = "v2"

Migrate Path Placeholders to PathRegexp

In v2, Path and PathPrefix supported path parameter placeholders like {id} for matching dynamic path segments. In v3, this is no longer supported and PathRegexp should be used instead.

Migrating a route with path placeholders

v2 syntax (no longer works in v3):

match: Host(`example.com`) && Path(`/products/{id}`)

v3 syntax using PathRegexp:

match: Host(`example.com`) && PathRegexp(`^/products/[^/]+$`)

For more complex patterns with multiple placeholders:

v2 syntax:

match: Host(`example.com`) && Path(`/users/{userId}/orders/{orderId}`)

v3 syntax:

match: Host(`example.com`) && PathRegexp(`^/users/[^/]+/orders/[^/]+$`) ## matches any non-slash characters
match: Host(`example.com`) && PathRegexp(`^/users/[a-zA-Z0-9_-]+/orders/[a-zA-Z0-9_-]+$`) ## restricts to alphanumeric, hyphens, and underscores

IPWhiteList

In v3, we renamed the IPWhiteList middleware to IPAllowList without changing anything to the configuration.

Deprecated Options Removal

• The tracing.datadog.globaltag option has been removed.
• The tls.caOptional option has been removed from the ForwardAuth middleware, as well as from the HTTP, Consul, Etcd, Redis, ZooKeeper, Consul Catalog, and Docker providers.
• sslRedirect, sslTemporaryRedirect, sslHost, sslForceHost and featurePolicy options of the Headers middleware have been removed.
• The forceSlash option of the StripPrefix middleware has been removed.
• The preferServerCipherSuites option has been removed.

TCP LoadBalancer terminationDelay option

The TCP LoadBalancer terminationDelay option has been deprecated. This option can now be configured directly on the TCPServersTransport level, please take a look at this documentation

Kubernetes CRDs API Group traefik.containo.us

In v3, the Kubernetes CRDs API Group traefik.containo.us has been removed. Please use the API Group traefik.io instead.

Kubernetes Ingress API Group networking.k8s.io/v1beta1

In v3, the Kubernetes Ingress API Group networking.k8s.io/v1beta1 (removed since Kubernetes v1.22) support has been removed.

Please use the API Group networking.k8s.io/v1 instead.

Traefik CRD API Version apiextensions.k8s.io/v1beta1

In v3, the Traefik CRD API Version apiextensions.k8s.io/v1beta1 (removed since Kubernetes v1.22) support has been removed.

Please use the CRD definition with the API Version apiextensions.k8s.io/v1 instead.