Add pod network routes

Where you are

During bootstrap you left cross-node pod networking unconfigured on purpose. Each node hands out pod IP addresses from its own subnet, but nothing yet tells one node how to reach the subnet of another. So a pod on node-0 cannot reach a pod on node-1. In this lab you add the routes that connect those subnets, so pods on node-0 and node-1 can reach each other. The smoke test in the next lab depends on this working.

How pods get their addresses

A pod scheduled to a node receives an IP address from that node’s pod subnet. Until you add routes, a pod cannot reach a pod on a different node, because each node has no path to the other node’s subnet. This is the gap you close now.

In this lab you create a route for each worker node. Each route maps a node’s pod subnet to that node’s internal IP address, so traffic for that subnet is sent to the right node.

There are other ways to implement the Kubernetes networking model. This lab uses static routes because they are simple and easy to inspect.

Gather the addresses

First collect the internal IP address and pod subnet for each worker node. The following block reads them from machines.txt into shell variables.

{
  SERVER_IP=$(grep server machines.txt | cut -d " " -f 1)
  NODE_0_IP=$(grep node-0 machines.txt | cut -d " " -f 1)
  NODE_0_SUBNET=$(grep node-0 machines.txt | cut -d " " -f 4)
  NODE_1_IP=$(grep node-1 machines.txt | cut -d " " -f 1)
  NODE_1_SUBNET=$(grep node-1 machines.txt | cut -d " " -f 4)
}

Add the routes

On the server, add a route to each worker node’s subnet. This lets the server reach pods on both nodes.

ssh root@server <<EOF
  ip route add ${NODE_0_SUBNET} via ${NODE_0_IP}
  ip route add ${NODE_1_SUBNET} via ${NODE_1_IP}
EOF

On node-0, add a route to node-1’s subnet.

ssh root@node-0 <<EOF
  ip route add ${NODE_1_SUBNET} via ${NODE_1_IP}
EOF

On node-1, add a route to node-0’s subnet.

ssh root@node-1 <<EOF
  ip route add ${NODE_0_SUBNET} via ${NODE_0_IP}
EOF

Verification

Print the routing table on the server. You should see a route for each worker subnet.

ssh root@server ip route

default via XXX.XXX.XXX.XXX dev ens160 
10.200.0.0/24 via XXX.XXX.XXX.XXX dev ens160 
10.200.1.0/24 via XXX.XXX.XXX.XXX dev ens160 
XXX.XXX.XXX.0/24 dev ens160 proto kernel scope link src XXX.XXX.XXX.XXX 

Print the routing table on node-0. You should see a route to node-1’s subnet.

ssh root@node-0 ip route

default via XXX.XXX.XXX.XXX dev ens160 
10.200.1.0/24 via XXX.XXX.XXX.XXX dev ens160 
XXX.XXX.XXX.0/24 dev ens160 proto kernel scope link src XXX.XXX.XXX.XXX 

Print the routing table on node-1. You should see a route to node-0’s subnet.

ssh root@node-1 ip route

default via XXX.XXX.XXX.XXX dev ens160 
10.200.0.0/24 via XXX.XXX.XXX.XXX dev ens160 
XXX.XXX.XXX.0/24 dev ens160 proto kernel scope link src XXX.XXX.XXX.XXX 

Key terms

Pod subnet is the range of IP addresses a node assigns to the pods running on it. Each node has its own pod subnet, and the ranges do not overlap.

Route is a rule that tells a node where to send traffic for a given destination range. Here, each route sends traffic for a node’s pod subnet to that node’s internal IP address.

Routing table is the full list of routes on a node. A node checks this table to decide where to send each outgoing packet.

Reflect

The routes you added are static, meaning you set them by hand. If you added a third worker node later, what would you have to do by hand on each existing node, and what does that tell you about how this approach holds up as a cluster grows?

Next: Run the smoke test