Stage 5 · Platform
Kubernetes Architecture Deep Dive
API Extensions
CRDs, API aggregation, and webhook conversion.
Extension Mechanisms
Kubernetes is designed to be extended. When built-in resources are not enough, you can extend the API with Custom Resource Definitions (CRDs) or API Aggregation. CRDs are simpler and preferred for most use cases. API Aggregation is for complex cases requiring a separate API server.
| Feature | CRD | API Aggregation |
|---|---|---|
| Complexity | Low — declarative YAML | High — separate API server |
| Storage | etcd via kube-apiserver | Custom storage backend |
| Versioning | Conversion webhooks | Native API server versioning |
| Use case | Custom resources | Metrics server, custom APIs |
Custom Resource Definitions
CRDs define a new resource type in your cluster. Once installed, you can kubectl get, create, update, and delete instances of the CRD. The kube-apiserver stores them in etcd like native resources. Controllers watch these resources and reconcile the desired state.
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
name: webapps.platform.example.com
spec:
group: platform.example.com
versions:
- name: v1
served: true
storage: true
schema:
openAPIV3Schema:
type: object
properties:
spec:
type: object
required: ["replicas", "image"]
properties:
replicas:
type: integer
minimum: 1
maximum: 10
image:
type: string
pattern: "^[a-z0-9.-]+:[a-z0-9.-]+$"
env:
type: array
items:
type: object
properties:
name:
type: string
value:
type: string
required: ["name"]
additionalPrinterColumns:
- name: Replicas
type: integer
jsonPath: .spec.replicas
- name: Ready
type: integer
jsonPath: .status.readyReplicas
- name: Age
type: date
jsonPath: .metadata.creationTimestamp
scope: Namespaced
names:
plural: webapps
singular: webapp
kind: WebApp
shortNames: ["wa"]The openAPIV3Schema field defines validation rules. The API server rejects invalid objects before they reach etcd. additionalPrinterColumns add custom columns to kubectl get output.
A CRD defines the resource type. An operator defines the controller logic that reconciles instances of that CRD. You need both: the CRD tells Kubernetes what the resource is; the operator tells it what to do with it.
API Aggregation
API Aggregation lets you register a custom API server that extends the Kubernetes API. The kube-apiserver proxies requests to your API server for specific API paths. This is used by the metrics server, which provides custom.metrics.k8s.io and custom.metrics.k8s.io APIs.
apiVersion: apiregistration.k8s.io/v1
kind: APIService
metadata:
name: v1beta1.custom.metrics.k8s.io
spec:
service:
name: metrics-server
namespace: kube-system
group: custom.metrics.k8s.io
version: v1beta1
insecureSkipTLSVerify: true
groupPriorityMinimum: 100
versionPriority: 100APIService registers a backend service for a specific API group and version. kube-apiserver forwards matching requests to the service. groupPriorityMinimum and versionPriority determine which API server handles overlapping groups.
Webhook Conversion
CRDs support multiple versions. When a client requests an older version, the API server calls a conversion webhook to convert between versions. This lets you evolve your CRD schema without breaking existing clients. The webhook is an HTTPS endpoint you implement and deploy.
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
name: webapps.platform.example.com
spec:
group: platform.example.com
conversion:
strategy: Webhook
webhook:
clientConfig:
service:
name: conversion-webhook
namespace: system
path: /convert
caBundle: <base64-encoded-ca>
versions:
- name: v1
served: true
storage: true
- name: v2
served: true
storage: falseThe conversion webhook receives a ConversionReview request containing the object in the stored version and the requested version. It returns the converted object. caBundle verifies the webhook's TLS certificate.
Schema Validation
CRD schemas use OpenAPI v3 to validate objects. This catches errors at admission time, before the controller processes them. Kubernetes v1.25+ supports x-kubernetes-validations (CEL) for complex validation rules that cannot be expressed with JSON Schema alone.
schema:
openAPIV3Schema:
type: object
x-kubernetes-validations:
- rule: "self.spec.replicas <= self.spec.maxReplicas"
message: "replicas must not exceed maxReplicas"
- rule: "has(self.spec.env) ? all(self.env, e, has(e.name) && has(e.value)) : true"
message: "env entries must have both name and value"
properties:
spec:
type: object
properties:
replicas:
type: integer
maxReplicas:
type: integer
env:
type: arrayCEL (Common Expression Language) rules are evaluated server-side. They can reference other fields in the object (cross-field validation) and are more powerful than JSON Schema alone.
CRD Lifecycle
CRDs go through install, establish, and optionally upgrade phases. After installation, the CRD enters the Established condition when the API server has synced its schema. If you update the schema, existing instances may become invalid and need remediation.
# Check CRD status
kubectl get crd webapps.platform.example.com -o yaml | jq .status
# List all instances of a CRD
kubectl get webapps --all-namespaces
# Delete all instances before deleting CRD
kubectl delete webapps --all-namespaces
kubectl delete crd webapps.platform.example.comYou must delete all instances before deleting a CRD. Otherwise, the CRD deletion hangs because it waits for all instances to be removed. Use --cascade=orphan if you want to keep the underlying objects.
Use kubectl apply --dry-run=server to validate objects against the CRD schema without persisting. This catches schema errors before they reach production. Also run kubectl get --raw /apis/platform.example.com/v1 to verify the CRD is registered.
Mark this lesson complete to store local progress and unlock a cleaner resume path the next time you visit.