Bring It Online

Bootstrap the worker nodes

Where you are

The control plane is running. The API server, scheduler, and controller manager are up, so the cluster has a brain. Now it needs hands. In this lab you bootstrap two worker nodes and join them to the cluster.

Joining a worker is a credential problem before it is anything else. A worker runs a kubelet, and the kubelet has to talk to the API server to register itself and receive work. But the API server does not accept anonymous callers. It wants proof of identity. So the worker needs credentials to reach the API server, which is exactly why you generated the certificates and kubeconfig files earlier. That identity material is what breaks the loop. You already made it; here you put it to use.

You will install five things on each node: runc, the container networking plugins, containerd, the kubelet, and kube-proxy. Definitions for each are at the end.

Prerequisites

The commands in this section must be run from the jumpbox.

Copy the Kubernetes binaries and systemd unit files to each worker instance:

for HOST in node-0 node-1; do
  SUBNET=$(grep ${HOST} machines.txt | cut -d " " -f 4)
  sed "s|SUBNET|$SUBNET|g" \
    configs/10-bridge.conf > 10-bridge.conf

  sed "s|SUBNET|$SUBNET|g" \
    configs/kubelet-config.yaml > kubelet-config.yaml

  scp 10-bridge.conf kubelet-config.yaml \
  root@${HOST}:~/
done

for HOST in node-0 node-1; do
  scp \
    downloads/worker/* \
    downloads/client/kubectl \
    configs/99-loopback.conf \
    configs/containerd-config.toml \
    configs/kube-proxy-config.yaml \
    units/containerd.service \
    units/kubelet.service \
    units/kube-proxy.service \
    root@${HOST}:~/
done

for HOST in node-0 node-1; do
  scp \
    downloads/cni-plugins/* \
    root@${HOST}:~/cni-plugins/
done

The commands in the next section must be run on each worker instance: node-0, node-1. Login to the worker instance using the ssh command. Example:

ssh root@node-0

Provisioning a Kubernetes worker node

Install the OS dependencies:

{
  apt-get update
  apt-get -y install socat conntrack ipset kmod
}

The socat binary enables support for the kubectl port-forward command.

Disable swap

Kubernetes has limited support for the use of swap memory, as it is difficult to provide guarantees and account for pod memory utilization when swap is involved.

Verify if swap is disabled:

swapon --show

If output is empty then swap is disabled. If swap is enabled run the following command to disable swap immediately:

swapoff -a

To ensure swap remains off after reboot consult your Linux distro documentation.

Create the installation directories:

mkdir -p \
  /etc/cni/net.d \
  /opt/cni/bin \
  /var/lib/kubelet \
  /var/lib/kube-proxy \
  /var/lib/kubernetes \
  /var/run/kubernetes

Install the worker binaries:

{
  mv crictl kube-proxy kubelet runc \
    /usr/local/bin/
  mv containerd containerd-shim-runc-v2 containerd-stress /bin/
  mv cni-plugins/* /opt/cni/bin/
}

Configure CNI networking

Create the bridge network configuration file:

mv 10-bridge.conf 99-loopback.conf /etc/cni/net.d/

To ensure network traffic crossing the CNI bridge network is processed by iptables, load and configure the br-netfilter kernel module:

{
  modprobe br-netfilter
  echo "br-netfilter" >> /etc/modules-load.d/modules.conf
}

{
  echo "net.bridge.bridge-nf-call-iptables = 1" \
    >> /etc/sysctl.d/kubernetes.conf
  echo "net.bridge.bridge-nf-call-ip6tables = 1" \
    >> /etc/sysctl.d/kubernetes.conf
  sysctl -p /etc/sysctl.d/kubernetes.conf
}

Configure containerd

Install the containerd configuration files:

{
  mkdir -p /etc/containerd/
  mv containerd-config.toml /etc/containerd/config.toml
  mv containerd.service /etc/systemd/system/
}

Configure the kubelet

Create the kubelet-config.yaml configuration file:

{
  mv kubelet-config.yaml /var/lib/kubelet/
  mv kubelet.service /etc/systemd/system/
}

Configure the Kubernetes proxy

{
  mv kube-proxy-config.yaml /var/lib/kube-proxy/
  mv kube-proxy.service /etc/systemd/system/
}

Start the worker services

{
  systemctl daemon-reload
  systemctl enable containerd kubelet kube-proxy
  systemctl start containerd kubelet kube-proxy
}

Check if the kubelet service is running:

systemctl is-active kubelet

active

Be sure to complete the steps in this section on each worker node, node-0 and node-1, before moving on to the next section.

Verification

Run the following commands from the jumpbox machine.

List the registered Kubernetes nodes:

ssh root@server \
  "kubectl get nodes \
  --kubeconfig admin.kubeconfig"

NAME     STATUS   ROLES    AGE    VERSION
node-0   Ready    <none>   1m     v1.32.3
node-1   Ready    <none>   10s    v1.32.3

Both nodes report Ready. The kubelet on each node used its credentials to register with the API server, and the API server accepted it. The credential loop is closed.

Key terms

kubelet: The agent that runs on every worker node. It registers the node with the API server, watches for pods assigned to its node, and tells the container runtime to start and stop the containers those pods describe. It also reports node and pod health back to the control plane.

kube-proxy: The network component that runs on every worker node. It programs the node’s iptables rules so that traffic sent to a Service address is forwarded to one of the pods backing that Service.

containerd: The container runtime. It pulls container images and manages the full lifecycle of containers on the node. The kubelet talks to containerd rather than starting containers itself.

runc: The low-level tool that actually creates and runs a container from an image and configuration. containerd calls runc to do the final work of starting a container process.

CNI: Container Network Interface. A standard for plugins that attach pods to a network and assign each pod an IP address. The bridge and loopback plugins you installed here are CNI plugins.

Reflect

You distributed certificates and kubeconfig files in an earlier lab, long before any worker existed. Why was it necessary to create that identity material before the kubelet could register, rather than letting the worker request credentials from the API server at join time?

Next: Configure kubectl for remote access