Eric Campbell 6 min read Infrastructure

Kubernetes Architecture: Control Plane and Nodes

Kubernetes implements a classic master-worker architecture pattern, separating cluster management from workload execution. This separation isn't just academic—it directly impacts how you scale,...

Key Insights

  • Kubernetes separates the control plane (cluster management) from nodes (workload execution), enabling independent scaling and high availability of each layer
  • The API server is the single source of truth for all cluster operations—every component communicates through it, making Kubernetes fundamentally API-driven
  • Understanding component responsibilities and communication patterns is essential for debugging production issues, as failures manifest differently depending on which layer is affected

Introduction to Kubernetes Architecture

Kubernetes implements a classic master-worker architecture pattern, separating cluster management from workload execution. This separation isn’t just academic—it directly impacts how you scale, secure, and troubleshoot your infrastructure.

The control plane manages the cluster state, makes scheduling decisions, and responds to events. Nodes execute the actual workloads. When your pods won’t start, understanding whether the issue is in the control plane (scheduling, API access) or on nodes (runtime, networking) cuts debugging time in half.

Here’s the high-level architecture:

┌─────────────────────────────────────────┐
│         Control Plane                   │
│  ┌──────────┐  ┌──────┐  ┌───────────┐ │
│  │API Server│  │ etcd │  │ Scheduler │ │
│  └──────────┘  └──────┘  └───────────┘ │
│  ┌────────────────────┐                 │
│  │Controller Manager  │                 │
│  └────────────────────┘                 │
└─────────────────────────────────────────┘
              │
              │ (API calls)
              │
    ┌─────────┴─────────┬─────────┐
    │                   │         │
┌───▼────┐         ┌────▼───┐  ┌─▼──────┐
│ Node 1 │         │ Node 2 │  │ Node 3 │
│kubelet │         │kubelet │  │kubelet │
│proxy   │         │proxy   │  │proxy   │
│runtime │         │runtime │  │runtime │
└────────┘         └────────┘  └────────┘

Control Plane Components

The control plane is the brain of your Kubernetes cluster. Every decision about what runs where, every response to failures, and every API query flows through these components.

API Server (kube-apiserver) is the front door to Kubernetes. It’s the only component that talks directly to etcd, and every other component—including kubectl—communicates through it. The API server validates and processes RESTful requests, implements authentication and authorization, and provides the watch mechanism that drives Kubernetes’ reactive architecture.

# Check control plane component status
kubectl get componentstatuses

NAME                 STATUS    MESSAGE             ERROR
scheduler            Healthy   ok                  
controller-manager   Healthy   ok                  
etcd-0               Healthy   {"health":"true"}

When you create a pod, the API server receives your request, validates it against admission controllers, and persists it to etcd:

# Example API server request (what kubectl does behind the scenes)
curl -X POST https://api-server:6443/api/v1/namespaces/default/pods \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d @pod-definition.json

etcd is the distributed key-value store that holds all cluster state. It’s the single source of truth. If etcd loses data, your cluster loses its memory. Here’s what a pod looks like in etcd:

/registry/pods/default/nginx-pod
{
  "kind": "Pod",
  "apiVersion": "v1",
  "metadata": {
    "name": "nginx-pod",
    "namespace": "default",
    "uid": "abc-123"
  },
  "spec": { ... },
  "status": { ... }
}

Scheduler (kube-scheduler) watches for newly created pods with no assigned node and selects a node for them based on resource requirements, constraints, affinity rules, and taints/tolerations. It doesn’t actually start the pod—it just updates the pod’s spec with a node assignment.

Controller Manager (kube-controller-manager) runs control loops that watch cluster state and make changes to achieve desired state. The ReplicaSet controller ensures you have the right number of pod replicas. The Node controller monitors node health. The Endpoint controller populates Service endpoints. Each controller is a separate process, but they’re compiled into a single binary for simplicity.

Cloud Controller Manager (optional) lets cloud providers integrate their infrastructure with Kubernetes. It handles cloud-specific operations like provisioning load balancers or attaching storage volumes.

Node Components

Nodes are the worker machines that run your containerized applications. Each node runs several components that manage pod lifecycle and networking.

kubelet is the primary node agent. It receives pod specifications from the API server and ensures those containers are running and healthy. It reports node and pod status back to the control plane and performs health checks.

# Inspect nodes with detailed information
kubectl get nodes -o wide

NAME       STATUS   ROLES    AGE   VERSION   INTERNAL-IP   OS-IMAGE
node-1     Ready    <none>   30d   v1.28.0   10.0.1.10     Ubuntu 22.04
node-2     Ready    <none>   30d   v1.28.0   10.0.1.11     Ubuntu 22.04
node-3     Ready    <none>   30d   v1.28.0   10.0.1.12     Ubuntu 22.04

A typical kubelet configuration file looks like this:

apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
authentication:
  anonymous:
    enabled: false
  webhook:
    enabled: true
authorization:
  mode: Webhook
clusterDomain: cluster.local
clusterDNS:
  - 10.96.0.10
containerRuntimeEndpoint: unix:///var/run/containerd/containerd.sock
maxPods: 110
podCIDR: 10.244.1.0/24

kube-proxy maintains network rules on nodes, implementing the Kubernetes Service abstraction. It programs iptables or IPVS rules to forward traffic to the correct pods.

# Sample iptables rules created by kube-proxy for a ClusterIP service
-A KUBE-SERVICES -d 10.96.0.1/32 -p tcp -m tcp --dport 443 \
  -j KUBE-SVC-NPX46M4PTMTKRN6Y

-A KUBE-SVC-NPX46M4PTMTKRN6Y -m statistic --mode random \
  --probability 0.33 -j KUBE-SEP-ENDPOINT1

-A KUBE-SVC-NPX46M4PTMTKRN6Y -m statistic --mode random \
  --probability 0.50 -j KUBE-SEP-ENDPOINT2

Container Runtime (containerd, CRI-O, Docker) actually runs the containers. Kubernetes uses the Container Runtime Interface (CRI) to communicate with the runtime, making it pluggable.

Communication Flow Between Control Plane and Nodes

Understanding the communication flow during pod creation reveals how Kubernetes’ declarative model works in practice.

  1. You run kubectl create -f pod.yaml
  2. kubectl sends an HTTP POST to the API server
  3. API server authenticates, authorizes, and runs admission controllers
  4. API server writes the pod to etcd with status “Pending”
  5. Scheduler watches for pods without a node assignment
  6. Scheduler selects a node and updates the pod spec via API server
  7. kubelet on the selected node watches for pods assigned to it
  8. kubelet tells the container runtime to pull images and start containers
  9. kubelet reports pod status back to API server
  10. API server updates etcd with the new status
# Watch this flow in real-time
kubectl get events --watch

LAST SEEN   TYPE     REASON      OBJECT           MESSAGE
0s          Normal   Scheduled   pod/nginx-pod    Successfully assigned default/nginx-pod to node-2
0s          Normal   Pulling     pod/nginx-pod    Pulling image "nginx:latest"
2s          Normal   Pulled      pod/nginx-pod    Successfully pulled image
2s          Normal   Created     pod/nginx-pod    Created container nginx
2s          Normal   Started     pod/nginx-pod    Started container nginx

The watch mechanism is critical. Components don’t poll—they establish long-lived watch connections to the API server. When state changes, the API server pushes updates immediately. This makes Kubernetes highly responsive while minimizing network overhead.

High Availability and Production Considerations

Production clusters require redundancy at both the control plane and node levels. A single-master cluster is a single point of failure.

Multi-master setup runs multiple API server instances behind a load balancer. All instances are active and can serve requests. The scheduler and controller manager use leader election—only one instance is active at a time, but others stand ready to take over.

# etcd cluster configuration for HA
apiVersion: v1
kind: Pod
metadata:
  name: etcd
spec:
  containers:
  - name: etcd
    image: k8s.gcr.io/etcd:3.5.9-0
    command:
    - etcd
    - --name=etcd-1
    - --initial-cluster=etcd-1=https://10.0.1.10:2380,etcd-2=https://10.0.1.11:2380,etcd-3=https://10.0.1.12:2380
    - --initial-cluster-state=new
    - --data-dir=/var/lib/etcd

Check etcd cluster health regularly:

# Inside an etcd pod
etcdctl --endpoints=https://127.0.0.1:2379 \
  --cacert=/etc/kubernetes/pki/etcd/ca.crt \
  --cert=/etc/kubernetes/pki/etcd/server.crt \
  --key=/etc/kubernetes/pki/etcd/server.key \
  endpoint health

https://10.0.1.10:2379 is healthy: successfully committed proposal
https://10.0.1.11:2379 is healthy: successfully committed proposal
https://10.0.1.12:2379 is healthy: successfully committed proposal

Node redundancy means distributing workloads across multiple nodes using pod anti-affinity:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: web-app
spec:
  replicas: 3
  template:
    spec:
      affinity:
        podAntiAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
          - labelSelector:
              matchExpressions:
              - key: app
                operator: In
                values:
                - web-app
            topologyKey: kubernetes.io/hostname

This ensures replicas land on different nodes. If one node fails, your application stays available.

Hands-On: Inspecting Your Cluster Architecture

Knowing how to inspect your cluster’s architecture is invaluable for troubleshooting.

# Get comprehensive cluster information
kubectl cluster-info dump --output-directory=/tmp/cluster-state

# Check control plane pods (in kubeadm clusters)
kubectl get pods -n kube-system

NAME                              READY   STATUS    RESTARTS   AGE
etcd-master                       1/1     Running   0          30d
kube-apiserver-master             1/1     Running   0          30d
kube-controller-manager-master    1/1     Running   0          30d
kube-scheduler-master             1/1     Running   0          30d

# Examine API server logs
kubectl logs -n kube-system kube-apiserver-master | tail -50

# Check node resource allocation
kubectl describe node node-1

Allocated resources:
  CPU Requests: 750m (37% of 2 cores)
  Memory Requests: 1.5Gi (40% of 4Gi)
  
# View kubelet logs (on the node itself)
journalctl -u kubelet -f

When troubleshooting, check control plane health first. If the API server is down, nothing works. If etcd is unhealthy, the cluster can’t maintain state. If a node’s kubelet is failing, only that node’s pods are affected.

Conclusion

Kubernetes’ architecture separates concerns cleanly: the control plane maintains desired state and makes decisions, while nodes execute workloads. This separation enables you to scale the control plane and worker nodes independently, implement high availability at each layer, and isolate failures.

The API-driven design means everything flows through the API server, creating a single point of control with multiple implementation paths. Understanding this architecture isn’t just theoretical—it’s the foundation for effective operations, troubleshooting, and capacity planning. When you know which component is responsible for what, you can quickly identify whether a problem is in scheduling, networking, storage, or runtime execution, and apply the right fix.

Liked this? There's more.

Every week: one practical technique, explained simply, with code you can use immediately.