Configuring Cilium Networking for Your Shoot Cluster
This section explains how to enable the Cilium networking extension for a Gardener Shoot and describes every option you can set in the Cilium NetworkConfig. It also covers the routing modes (VXLAN, Geneve and native routing) and when to use each one.
Enabling Cilium
Cilium is currently the only CNI extension provided by OSC and is enabled by default for every Shoot created on OSC. To verify that Cilium is enabled, check the Shoot manifest for the label networking.extensions.gardener.cloud/cilium: 'true' and ensure that the networking section defines all necessary CIDRs.
providerConfig is optional. Without it, Cilium runs with the defaults described here
Concrete configuration recommendet by OSC is described here
Defaults applied automatically
OSC strongly recommends using the default setting for kubeProxy (kubeProxy.enabled: false).
Routing modes and tunnels
Cilium can move pod-to-pod traffic between nodes in two ways:
- Encapsulation (tunnel): Cilium wraps each packet from a pod in an extra UDP header and sends it to the other node's IP address. The underlying network only sees node IPs.
- Native routing (no tunnel): Packets leave the node with the real pod IP as the source and destination. The underlying network must know how to route the pod CIDRs.
Two fields control this: overlay.enabled and tunnel.
What each tunnel value does
tunnel: vxlan
- How it works: Cilium wraps each pod packet in a VXLAN header and sends it over UDP. Cilium uses UDP port 8472 by default.
- Overhead: About 50 bytes per packet, so the pod MTU is lower than the node MTU. Cilium adjusts this automatically.
- Network requirements: Nodes need IP connectivity to each other, and UDP 8472 must be open between them. The infrastructure doesn't need any routes for pod CIDRs.
- Use it when: You need something that works on any infrastructure. The underlying network can't or shouldn't know pod IPs, or the nodes are in different L2 segments or subnets.
- Trade-offs: Encapsulation adds a little CPU cost and lowers the MTU. Outside the cluster, pod IPs are hidden behind node IPs.
tunnel: geneve
- How it works: Cilium wraps each pod packet in a Geneve header and sends it over UDP port 6081.
- Overhead: Similar to VXLAN, about 50 bytes or more. Geneve also supports extensible option headers.
- Network requirements: Nodes need IP connectivity to each other, and UDP 6081 must be open between them.
- Use it when: You want an overlay but your environment or security policy prefers Geneve, or you plan to use Cilium features that rely on Geneve options.
- Trade-offs: Same as VXLAN. Some older network devices and firewalls handle VXLAN better.
tunnel: disabled (native routing)
- How it works: No encapsulation. Cilium sets
routing-mode: native, and packets carry the real pod IPs on the wire. - Overhead: None. You get the full MTU and the best throughput and latency.
- Network requirements: The underlying network must route the pod CIDR (
spec.networking.pods) to the correct node. - Use it when: Performance matters and your infrastructure can route pod IPs.
- Trade-offs: Pod traffic fails if the network can't route pod CIDRs. Egress Gateway is not available.
In OSC infrastructure, native routing is supported and highly recommended. Therefore, unless there is a technical reason to use a different mode, use native routing what means tunnel: disabled. For IPv4 native routing, the Shoot’s Pod CIDR is used as the native routing CIDR. A Variant with tunnel: disabled is the only variant that is fully tested on OSC infrastructure. If it is used a different mode, for example tunnel: geneve or vxlan than OSC cannot guarantee proper performance and reliability.
How overlay and tunnel combine
Cilium decides the effective routing from both fields:
In this distribution, overlay.enabled: true without an explicit tunnel value or with tunnel: disabled gives you native routing, not VXLAN. If you need encapsulation together with overlay.enabled: true, set tunnel: vxlan or tunnel: geneve explicitly.
With overlay.enabled: false, any tunnel value you set is overwritten with disabled.
Load balancing modes
These modes only apply when Cilium replaces kube-proxy.
OSC recomended setup
OSC recommends network settings that make the most of our infrastructure capabilities. Using native routing with direct server return (DSR) load balancing provides optimized traffic management, improved scalability, and superior network performance without the need for tunneling protocols.
NetworkConfig options in providerConfig section
Feature guides
Hubble (observability)
To open the UI locally:
Running without kube-proxy
New Shoots run without kube-proxy by default, and Cilium handles all Service, NodePort and LoadBalancer traffic in eBPF.
Service mesh (Istio )
Istio sidecars often expect traffic handling in a way that can conflict with Cilium’s socket-level load balancing inside pod namespaces.
L2 announcements
After that, create CiliumL2AnnouncementPolicy and CiliumLoadBalancerIPPool resources in the Shoot. This feature requires kube-proxy replacement.