Stage 5 · Platform
Networking & Service Mesh
CNI Dataplanes
Calico iptables, Cilium eBPF, overlay routing, and kube-proxy replacement tradeoffs.
CNI Dataplane Overview
The CNI dataplane handles packet forwarding, NAT, and policy enforcement on each node. Traditional CNI plugins use iptables rules managed by kube-proxy. Modern plugins use eBPF to bypass iptables entirely, providing better performance and observability.
iptables mode:
Pod -> veth -> bridge -> iptables rules -> veth -> pod
eBPF mode:
Pod -> veth -> eBPF program -> veth -> pod
(bypasses iptables, conntrack, and kube-proxy)eBPF programs run in the kernel before iptables. This eliminates the overhead of traversing hundreds or thousands of iptables rules. For clusters with many Services, eBPF provides significant performance improvements.
iptables Dataplane
iptables mode uses Linux netfilter rules for traffic routing. kube-proxy creates rules for each Service (ClusterIP, NodePort) and each endpoint. Calico manages network policies using iptables. This is the traditional approach, reliable but slow at scale.
apiVersion: projectcalico.org/v3
kind: FelixConfiguration
metadata:
name: default
spec:
# Use iptables for dataplane
dataplaneDriver: iptables
# Logging for debugging
logFilePath: /var/log/calico/felix.log
logSeverityScreen: Info
# iptables-specific settings
iptablesRefreshInterval: "30s"
iptablesPostWriteCheckInterval: "1s"
iptablesLockInterval: "10s"
# Wireguard for node-to-node encryption
wireguardEnabled: falseFelix is Calico's node agent. It programs iptables rules for network policies and manages the dataplane. iptablesRefreshInterval controls how often rules are refreshed. Lower values catch changes faster but use more CPU.
iptables uses linear rule matching. With thousands of Services and pods, rule evaluation becomes slow. This causes connection setup latency. Monitor iptables-restore duration — if it exceeds 5 seconds, consider switching to eBPF.
eBPF Dataplane
eBPF (extended Berkeley Packet Filter) allows custom programs to run in the kernel at various hook points. Cilium uses eBPF to implement networking, load balancing, and policy enforcement without iptables. This provides O(1) service lookups instead of O(n).
apiVersion: cilium.io/v2
kind: CiliumClusterNetworkPolicy
metadata:
name: default-policy
spec:
endpointSelector: {}
egress:
- toEndpoints:
- matchLabels:
k8s:io.kubernetes.pod.namespace: kube-system
k8s-app: kube-dns
toPorts:
- ports:
- port: "53"
protocol: UDP
- port: "53"
protocol: TCP
- toEntities:
- kube-apiserver
- toFQDNs:
- matchName: ".example.com"Cilium policies can filter by L3 (IP), L4 (port), and L7 (HTTP method, path). eFQDNs allow DNS-based policies — traffic to *.example.com is allowed. The eBPF program evaluates these rules at wire speed.
Overlay vs Native Routing
Overlay networking encapsulates pod traffic in outer packets (VXLAN, Geneve) to route across nodes. Native routing uses the underlying network infrastructure directly. Overlay is simpler to set up; native routing provides better performance.
| Feature | Overlay (VXLAN) | Native Routing |
|---|---|---|
| Setup | Simple — no network config | Requires BGP or static routes |
| Performance | Encapsulation overhead | No overhead |
| MTU | Reduced (1500 - encapsulation) | Full 1500 |
| Debugging | Harder — outer packet hides inner | Easier — direct visibility |
apiVersion: projectcalico.org/v3
kind: BGPConfiguration
metadata:
name: default
spec:
logSeverityScreen: Info
nodeToNodeMeshEnabled: true
asNumber: 64512
serviceClusterIPs:
- cidr: 10.96.0.0/16
serviceExternalIPs: []
---
apiVersion: projectcalico.org/v3
kind: BGPPeer
metadata:
name: router-peer
spec:
peerIP: 10.0.0.1
asNumber: 64500
nodeSelector: all()BGP peering announces pod CIDRs to the network fabric. Routers know how to reach pods on any node without encapsulation. nodeToNodeMeshEnabled creates full mesh BGP between all Calico nodes.
kube-proxy Replacement
Cilium can replace kube-proxy entirely using eBPF for Service load balancing. This eliminates the kube-proxy process and its iptables rules. eBPF uses hash maps for O(1) lookups and supports Maglev (consistent hashing) for better load distribution.
apiVersion: v1
kind: ConfigMap
metadata:
name: cilium-config
namespace: kube-system
data:
# Replace kube-proxy
kube-proxy-replacement: strict
# or "partial" to keep kube-proxy for some functions
# Enable direct routing (no encapsulation)
tunnel: disabled
# Enable host routing for pod-to-host traffic
enable-host-routing: "true"
# Maglev consistent hashing for Services
enable-service-topology: "true"
# Enable Bandwidth Manager for eBPF-based rate limiting
enable-bandwidth-manager: "true"kube-proxy-replacement: strict means Cilium handles all Service load balancing via eBPF. tunnel: disabled uses native routing. enable-service-topology routes traffic to the nearest pod endpoint.
Performance Benchmarks
eBPF provides measurable improvements over iptables: lower latency, higher throughput, and better CPU efficiency. These improvements scale with the number of Services and pods — the more Services, the bigger the eBPF advantage.
Connection setup latency (p99):
iptables: 2.3ms
eBPF: 0.4ms
Throughput (TCP connections/sec):
iptables: 45,000
eBPF: 85,000
CPU usage (kube-proxy/Cilium agent):
iptables: 12% (kube-proxy) + 3% (calico) = 15%
eBPF: 5% (cilium agent only)
Rule update time (1000 Services):
iptables: 4.2 seconds
eBPF: 0.1 secondsAt 1000 Services, eBPF is 5x faster for connection setup and 2x faster for throughput. CPU usage drops by 66%. Rule updates are 40x faster. These gains increase with cluster size.
Choose eBPF when: you have >500 Services, need L7 policy enforcement, require low-latency networking, or want to eliminate kube-proxy. Stick with iptables if: your cluster is small, you need maximum compatibility, or your kernel is older than 5.4.
Mark this lesson complete to store local progress and unlock a cleaner resume path the next time you visit.